Every Shopify store is built on a fixed URL scheme that you can’t fully rewrite. Products live under /products/, collections under /collections/, blog posts under /blogs/, and pages under /pages/. Those prefixes are hard-coded into the platform. Understanding how they interact is the difference between a store that gets crawled efficiently and one that quietly burns through its crawl budget on near-identical URLs.
Where Shopify Routes Live
Shopify assigns a handle to every product, collection, and page, and that handle becomes the tail of the URL. A product called “Merino Wool Beanie” becomes /products/merino-wool-beanie. The handle is editable in the admin, but the prefix is not. You can’t move a product out of /products/ or nest it under a category folder the way some other platforms allow.
This matters because Shopify also lets the same product be reached through a collection. When you view a product inside a collection, the URL can appear as /collections/hats/products/merino-wool-beanie. That’s the same product, the same content, at a longer address. The canonical version Shopify generates points back to the clean /products/merino-wool-beanie, which helps, but the collection-scoped URLs still exist and can still be linked and crawled.
The Collections Duplication Trap
The bigger crawl problem sits inside collection pages themselves. Collections generate paginated URLs, and they also generate a long tail of filtered and sorted variations. A single collection can spawn addresses like ?sort_by=price-ascending, ?sort_by=best-selling, and combinations of tag filters such as /collections/hats/wool. Each of these serves largely the same set of products in a slightly different order, and each is a distinct URL a crawler may try to fetch.
On a store with dozens of collections and several filter options, this multiplies fast. You can end up with thousands of low-value URLs that all point at reshuffled versions of the same inventory. Search engines don’t need to index the “sorted by price, descending” view of your beanie collection, but they’ll happily spend requests discovering it. That wasted effort means the pages you actually care about get visited less often and updated more slowly in the index. Auditing which parameter and filter URLs are being crawled is one of the first things a specialist like best-shopify-seo-agency will check, because the fix is usually structural rather than a matter of adding more content.
The collection-scoped product URLs compound the issue. If your theme links to products through their collection path, you’re feeding crawlers the long-form duplicates instead of the canonical short form. Even with canonical tags in place, consistently linking to the clean URL reduces the noise a crawler has to sort through.
Fixing Crawl Waste
Start with your internal links. Configure the theme so product links resolve to /products/handle rather than the collection-nested version. Most modern themes already do this, but custom builds and older templates often don’t, so it’s worth inspecting the actual href values in your rendered HTML.
Next, handle the parameter sprawl. Your robots.txt on Shopify is now editable through the robots.txt.liquid file, which lets you disallow crawling of sort and filter parameters you don’t want indexed. Be deliberate here: block the reshuffled and filtered variants, but never block the canonical collection and product URLs themselves. Pair that with a clean XML sitemap, which Shopify generates automatically and which should list only the pages you want found.
Keep your collection structure shallow and intentional. A flat set of well-chosen collections that each map to real customer intent is easier to crawl and rank than a deep web of overlapping tag-based collections that duplicate each other’s products. Consolidate collections that serve the same purpose, and delete empty or seasonal ones you’re no longer using.
If you do one thing this week, pull your store’s list of crawled URLs from your search console property and sort by parameter. The pattern of wasted requests will show you exactly which filters to disallow first.