Does Shopify's Default robots.txt Block AI Crawlers? We Read the File

No. Shopify's default robots.txt does not name a single AI crawler, and it does not block any of them. GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, PerplexityBot and Google-Extended all fall under the general User-agent: * group, which allows the public store. If an AI crawler is blocked on your store, something other than Shopify's default did it.

Articles on this question disagree, and none we found shows a dated copy of the file. So we fetched it.

How we checked

On 13 November 2026 at about 02:46 UTC we downloaded /robots.txt from three Shopify stores:

• theme-dawn-demo.myshopify.com and theme-horizon-demo.myshopify.com, Shopify's demo stores for its Dawn and Horizon themes. Nothing on them is customised.
• www.williampainter.com, a sunglasses store and one of our lab stores.

We then compared the files line by line. All three were 122 lines long and identical except for one number: each disallows a path made of the store's own ID. That is what a file Shopify generates for every store looks like. It also tells us the lab store has not customised its robots.txt.

What the file contains

The default file has three parts.

A comment block addressed to AI agents. The first 21 lines are comments, which crawlers ignore when applying rules. They describe the storefront and point agents to three places: the store's /agents.md, its UCP discovery file at /.well-known/ucp, and its UCP and MCP endpoint at /api/ucp/mcp. They then list rules for agents, including that agents must identify themselves and "obtain buyer authorization" before shopping actions. The block ends with Shopify's terms of service and a contact address, bots@shopify.com. The same block also asks personal-shopper agents to suggest a Shop product to their users. That line is Shopify's, it was in all three files we fetched, and you cannot remove it without replacing the template.

A User-agent: * group. It opens with Allow: /, then disallows the parts of a store that are private or not meant for crawlers: the admin, cart, checkout, account and order pages, Shopify's internal endpoints, the cart and recommendations AJAX endpoints, and sort, filter and theme-preview URLs. Product, collection, page and blog URLs are not disallowed. Some comments in this group are aimed at agents too. One says that for the store's AJAX endpoints, "agents should use UCP/MCP instead".

A User-agent: adsbot-google group. A comment explains why: Google's ads crawler "ignores robots.txt unless specifically named", so Shopify repeats its rules for it.

Then the sitemap line. That is the whole file. There is no group for any AI company's crawler.

Which AI crawlers that lets in

Because no AI crawler is named, each one follows the * group and can read the public store. Here is what each company says its crawler does:

Two things follow. First, the search crawlers and the training crawlers are separate, so a store that wants to appear in AI search but not be used for training can block only the training crawlers. Second, blocking Google-Extended does nothing to AI Overviews or AI Mode, which are part of Google Search.

Where blocks actually come from

In our benchmark of 2,704 Shopify stores, scanned in June 2026, 1% had a robots.txt that blocked at least one AI search crawler (OAI-SearchBot, PerplexityBot or Googlebot). Since the default allows them, those stores changed something. The usual sources of a block are:

• A custom robots.txt.liquid template, often copied from a blog post or added by an agency, with Disallow: / under an AI crawler's name.
• A network-level block. A bot-protection setting at a CDN or firewall in front of the store can refuse AI crawlers before robots.txt is ever read. Your robots.txt then looks fine while the crawler gets an error.
• An app that edits the theme's robots template.

To check your own store, open yourstore.com/robots.txt and search for the crawler names above. If none appears and the * group has Allow: /, robots.txt is not the problem. A network block will not show in the file. If you use a CDN or bot-protection service in front of Shopify, check its settings for AI crawler rules as well.

Blocking AI crawlers does not take you out of Shopify Catalog

This is the part most guides miss. Shopify's help center says that if you sell through agentic storefronts such as ChatGPT or Microsoft Copilot, "your product data is syndicated to those channels through Shopify Catalog, independently of /robots.txt". Blocking AI crawlers "affects only open-web discoverability".

So there are two separate routes. Shopify Catalog sends product data to the AI channels you have turned on, whatever your robots.txt says. Crawlers read your public pages directly, and robots.txt is one of the things that decides whether they may. To stop a channel, use the agentic storefront settings, not robots.txt. Our Shopify Catalog guide covers that side.

If you do want to change it

Shopify lets you replace the generated file by adding a robots.txt.liquid template to your theme. The developer docs show how to keep the default groups with robots.default_groups and add your own group below them. A group that blocks only OpenAI's training crawler looks like this:

{% for group in robots.default_groups %}
  {{- group.user_agent }}
  {%- for rule in group.rules -%}
    {{ rule }}
  {%- endfor -%}
  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

User-agent: GPTBot
Disallow: /

Before you do, read Shopify's warnings. The help center calls it "an unsupported customization", says Support cannot help with it, and says "incorrect use of the feature can result in loss of all traffic". It also notes that robots.txt rules are "directional and advisory", so not every crawler obeys them. The developer docs recommend saving your current file first and comparing it with the rendered output.

Keeping the default loop matters. If you write the file by hand instead, you stop receiving Shopify's changes to the default, including the agent comment block and any new rules Shopify adds. To go back to the default, delete the template.

What we would do

For most stores: leave robots.txt alone. The default already lets AI search crawlers read your products and keeps them out of checkout and accounts. If you have a reason to keep your content out of model training, add groups for the training crawlers only, keep the default loop, and check the live file afterwards.

The more useful work is on what those crawlers find when they arrive. robots.txt only decides who may read the store. Your agents.md and llms.txt say how to shop it, and your structured data says what each product is. The comparison of all three files explains how they fit together, and the AI discovery files page and setup docs show what AgentReady adds to the two files Shopify lets you customise.

To see crawler by crawler whether your robots.txt lets the AI search crawlers in, alongside the rest of your store's machine-readable facts, run the free audit.