# spiderpic.com robots.txt file User-agent: * Disallow: # Sitemap: siteMap_index.xml # Crawl-delay: 5