OAI-SearchBot and robots.txt: how to let ChatGPT see your store (without opting into training)

To let ChatGPT see your store, allow OAI-SearchBot in your robots.txt and in every layer in front of your site (firewall, CDN, bot protection, storefront password). If you run ChatGPT Ads, allow OAI-AdsBot too. You can still refuse model training by disallowing GPTBot: OpenAI treats each crawler setting independently.

This guide lists what each OpenAI crawler does, gives robots.txt rules you can copy, and walks through the places where a store usually blocks ChatGPT without knowing it.

The four OpenAI user agents, and what each one is for

OpenAI publishes its crawlers on a single page, the overview of OpenAI crawlers. Four matter to an online store.

Three facts from the same page shape every decision below:

• The settings are independent. OpenAI writes that “a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training.”
• Blocking search has a visible effect. “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”
• Changes are not instant. “It can take ~24 hours from a site’s robots.txt update for our systems to adjust.”

What each one means for a store

OAI-SearchBot is the one that matters for discovery. When a shopper asks ChatGPT for a product and the answer draws on the web, this is the crawler that read your pages. OpenAI’s help article on shopping with ChatGPT Search explains that product results draw on structured metadata from first-party and third-party providers, such as price and product description, and on other third-party content. If OAI-SearchBot cannot read your product pages, you leave that ground to other sources.

GPTBot is a separate decision. Allowing it or not changes whether your content may be used to train OpenAI’s models. It does not change whether you appear in ChatGPT search. Many stores block it and allow OAI-SearchBot; both choices are legitimate.

OAI-AdsBot only matters if you advertise. It visits the landing pages you submit with ChatGPT Ads. OpenAI states that “the data collected by OAI-AdsBot is not used to train generative AI foundation models.”

ChatGPT-User is a person, not a crawler. OpenAI notes that it “is not used for crawling the web in an automatic fashion.” It shows up when someone asks ChatGPT to open a specific page, for example your product link.

The robots.txt rules to write

A store that wants to be found in ChatGPT search, run ChatGPT Ads, and keep its content out of training can use:

User-agent: OAI-SearchBot
Allow: /

User-agent: OAI-AdsBot
Allow: /

User-agent: GPTBot
Disallow: /

A store that is happy for its content to be used for training as well simply drops the GPTBot block or sets it to Allow: /.

Three details avoid most mistakes:

1. A specific group beats the wildcard. A crawler follows the most specific User-agent group that names it. If your file has User-agent: * with Disallow: /, a separate User-agent: OAI-SearchBot group with Allow: / lets OAI-SearchBot in. Without that group, it is locked out with everyone else.
2. Keep your normal private paths private. Allowing / does not oblige you to expose the cart, checkout or account pages. You can keep the same Disallow lines you use for other crawlers, such as /cart or /checkout, inside the OAI-SearchBot group.
3. Do not forget the images. OpenAI’s advertiser guidance asks merchants with product feeds to also allow OAI-SearchBot to crawl the product image URLs, and warns that a 403 on the image host can block feed processing. If your images live on a separate domain or CDN, that host needs the same rules.

Where stores block ChatGPT without knowing it

robots.txt is only the first gate. In practice, most blocks happen in a layer the merchant never looked at. OpenAI’s advertiser guidance asks you to review human-verification and anti-bot logic and to exempt its crawlers, “ideally by allowlisting our crawler user agents.”

CDN and firewall rules

Many stores sit behind a CDN or a web application firewall that challenges anything that does not look like a browser. A crawler cannot solve a challenge page, so it receives the challenge instead of your product page, and from the outside everything looks normal: your site loads fine for you.

Cloudflare is the most common case. Its July 2026 changelog on AI traffic explains that AI traffic is now grouped into three behaviors you can allow or block: search, agent and training. For new domains from September 15, 2026, search stays allowed by default, while training and agent traffic are blocked on pages that display ads. Check these settings and Cloudflare’s AI crawler controls, and make sure OAI-SearchBot (and OAI-AdsBot if you advertise) is allowed.

If your firewall only accepts a fixed list of IP ranges, OpenAI publishes them: https://openai.com/searchbot.json for OAI-SearchBot, https://openai.com/adsbot.json for OAI-AdsBot, https://openai.com/gptbot.json for GPTBot and https://openai.com/chatgpt-user.json for ChatGPT-User. Lists change over time, so read them from the source rather than copying them once.

WordPress security plugins

On WooCommerce, the robots.txt file is often generated by an SEO plugin, and bot blocking is often done by a security plugin or by the host. Two plugins can disagree: the SEO plugin allows a crawler while the security plugin rate-limits or blocks it. Open https://your-store.com/robots.txt to see the file actually served, then check the security plugin’s blocked-traffic log for the user agents above.

A Shopify storefront password

A Shopify store with the online store password turned on shows the password page to every visitor, crawlers included. That is expected while you build, and easy to forget at launch. Check that the password is off on the store you want ChatGPT to see.

To change Shopify’s robots.txt, add a robots.txt.liquid template to your theme, as described in Shopify’s help article on editing robots.txt.liquid. Shopify recommends using Liquid to add or remove directives so the default rules keep updating, and notes that this is an unsupported customization: its support team cannot help with the edits.

Maintenance modes and geo-blocking

A “coming soon” plugin, a maintenance mode left on, or a rule that only serves your store to one country can all return something other than your product page to a crawler. If you block countries, test what a request from outside them receives.

How to check that OpenAI can actually read your pages

Do not stop at reading your robots.txt. Check what the crawlers really get.

1. Read the served file. Open /robots.txt on your live domain, not the file in your theme or plugin settings, and confirm the rules above are present.
2. Look at your access logs. Search your server or CDN logs for OAI-SearchBot, OAI-AdsBot and GPTBot. A healthy pattern is a mix of product pages answered with status 200. A wall of 403, 429 or 503 responses means a layer in front of your site is blocking them.
3. Check the requests are genuine. Anyone can copy a user agent string. If you need certainty, compare the request IP with OpenAI’s published ranges listed above.
4. Check your image host. Repeat the log check on the domain that serves your product images.
5. Wait a day before judging. After a robots.txt change, OpenAI says its systems take about 24 hours to adjust.

Being readable is not the same as being recommended

Allowing OAI-SearchBot makes your store eligible to be read. Whether ChatGPT recommends your products then depends on what it reads: complete and consistent product data, prices and availability that match your pages, reviews, and a valid product feed. The levers are covered in how to get your products recommended by ChatGPT, and the feed rules in the OpenAI product feed specification, explained.

Two Convrail features sit on either side of that work. The product feed checks every product against the OpenAI specification before delivery and shows the exact fix for each refusal, so the data you send is accepted. AI search visibility then tracks the buying questions your shoppers ask, every week, and shows how often your store is named and which shops are named instead.

Checklist

• robots.txt allows OAI-SearchBot on your product and category pages
• robots.txt allows OAI-AdsBot if you run ChatGPT Ads
• GPTBot is allowed or disallowed on purpose, according to your training policy
• CDN, firewall and bot protection let OpenAI’s crawlers through
• The image host serves product images to OAI-SearchBot
• No storefront password, maintenance mode or geo-block hides the live store
• Access logs show OpenAI requests answered with 200