# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 User-agent: * Allow: / Disallow: /admin Disallow: /checkout Disallow: /interns/search?* Disallow: /interns/search/?* Disallow: /organizations?* Disallow: /organizations/?* User-agent: AhrefsBot User-agent: MJ12bot User-agent: Yandex User-agent: spbot User-agent: ezooms User-agent: sistrix User-agent: Mail.RU Disallow: /