Robots.txt generator is free online seo tool to create a Robots.txt file for your website is crawled by search engines you want. It helps you to generate effective 'robots.txt' file, that helps to ensure Google and other search engines are able to crawl and index your website properly.

What is a Robots.txt File?

Robots.txt is a file on your server which instructs search engine crawlers, spiders, and robots to index which part of site or not to index which section or pages of your website. It communicates with automated robots send by search engines. A robots.txt file can work differently for different search engines robots or bots.

The robots.txt file should be placed on the root web directory on your server.

Very simple only copy the above code and save to 'robots.txt' file on root web directory on your server or hosting.

Example of a robots.txt file:

User-Agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /out/
Sitemap: http://247seochecker.com/sitemap.xml

Select User Agent: *
Select Action: *
Directories and Files:
XML Sitemap:


Your Robots.txt File:



Seo Checker performs a multi-factor SEO analysis.

A web page analysis with more than 40 factors affect the ranking on search engines.