## Enable robots.txt rules for all crawlers User-agent: * Sitemap: http://www.decorall.kg/sitemap.xml ## Do not crawl development files and folders Disallow: /*.sql$ Disallow: /*.tgz$ ## Do not crawl admin page Disallow: /admin/ ## Do not crawl Disallow: /cart ## Do not crawl common server technical folders and files Disallow: /cgi-bin/