# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-agent: * Disallow: /tags-categories/ Disallow: /tag/ Disallow: /dev/ Disallow: /simple-backup/ Disallow: /sliders/ Disallow: /wp-snapshots/ Disallow: /sidebar/ Disallow: /pdf/ Disallow: /category/ Disallow: /author/ Disallow: /archives/ Disallow: /forum/