# CRELoaded Generated Robots.txt # Robot Exclusion File -- robots.txt # Author: CRELoaded Team # Last Updated : September 11th, 2007 User-Agent: * Disallow: /jaysaonline/ Disallow: /production/ Disallow: /Scripts/ Disallow: /sloppyblog/ Disallow: /thanks.php Disallow: /newsletters/ #To prevent only a select bots from any scans #User-agent: crawlers.looksmart.com #Disallow: / #To prevent any images from being scanned #Disallow: *.gif #Disallow: *.jpg #Disallow: *.png