User-agent: Scrapy Allow: / Sitemap: http://pegfitzpatrick.com/sitemap_index.xml User-agent: * Crawl-Delay: 30