User-agent: * Disallow: /Admin/ Disallow: /admin/ Disallow: /Custom/* Disallow: /My-Account Disallow: /Change-Password Disallow: /My-Wishlist Disallow: /My-Orders Disallow: /My-Address Disallow: /Checkout Disallow: /Shopping-Cart Disallow: /Billing-Delivery Disallow: /Payment Disallow: /Confirmation Disallow: /Thank-You Disallow: /Shopping-Cart # Google adsbot ignores robots.txt unless specifically named! User-agent: adsbot-google Disallow: / User-agent: Yahoo-MMCrawler Disallow: / User-agent: MSNBot-Media Disallow: / User-agent: Nutch Disallow: / User-agent: MJ12bot User-agent: Slurp Crawl-Delay: 10 Sitemap: https://www.archant.co.nz/sitemap.xml