What is robots.txt?
robots.txt is a text file at the root of a website that tells crawlers which pages they may fetch. If it forbids Google's crawler from downloading a landing page or an image, Merchant Center can't check the product.
What Google says
Google says that if you recently updated your robots.txt file, your products continue to show on Google for up to 30 days using previously approved images.
Which rules read robots.txt?
3 rules in our library mention robots.txt. G-032 "robots.txt blocks Googlebot, Googlebot-Image or Storebot-Google" and GA-004 "AdsBot explicitly blocked in robots.txt" look at what it blocks. GS-011 "Sitemap missing or not referenced" appears because the sitemap is usually listed in robots.txt.
Sources
- support.google.com/merchants/answer/12467444 · Last checked
- support.google.com/merchants/answer/12467444 · Last checked
Last updated