Robots.txt generator is free online seo tool to create a Robots.txt file for your website is crawled by search engines you want. It helps you to generate effective 'robots.txt' file, that helps to ensure Google and other search engines are able to crawl and index your website properly.
Robots.txt is a file on your server which instructs search engine crawlers, spiders, and robots to index which part of site or not to index which section or pages of your website. It communicates with automated robots send by search engines. A robots.txt file can work differently for different search engines robots or bots.
The robots.txt file should be placed on the root web directory on your server.
Very simple only copy the above code and save to 'robots.txt' file on root web directory on your server or hosting.Example of a robots.txt file:
User-Agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /out/ Sitemap: http://247seochecker.com/sitemap.xml