Add Sitemaps Autodiscovery in Robots.txt File

Sitemaps are essential to get all permalinks of your site well indexed by search engines. Recently Google, Microsoft and Yahoo! agreed to a standard Sitemaps protocol. I use the Google Sitemaps Generator WordPress Plugin to easily generate my sitemap xml file and the latest plugin version lets you ping Google and Yahoo! each time you post. Now adding sitemap autodiscovery links in Robots.txt file just made this a lot easier., Google, Microsoft Live Search and Yahoo! have announced support of autodiscovery of Sitemaps. The new open-format autodiscovery allows webmasters to specify the location of their Sitemaps within their robots.txt file, eliminating the need to submit sitemaps to each search engine separately.

This step is very useful as webmasters can easily submit their content to the search engines and benefit from reduced unnecessary traffic by the crawlers. The search engines get information with regards to pages to index as well as metadata with clues about which pages are newly updated and which pages are identified as the most important and search users get more fresh content.

Everyone webmasters ensures their site has a robots.txt file hosted on the root server, to guide search engine crawlers, spiders and bots about which parts of the website they want to get indexed and blocked. So just by adding a single line of code to the already existent robots.txt file, you can help search engines find your sitemaps file.

How to modify the robots.txt file?

Simply add the sitemap url following the word sitemap like this


Its simple. Update your robots.txt file today and help search engines index your site better.

Update: The correct way to add Sitemap link to robots.txt file


  1. Mr.Byte says:

    But, why have you put that under disallow section in your site?

  2. Decapper says:

    Thanks for that. Having 6 accounts on six different search engine sites trying to keep my sitemap up to date and also making sure the search engines have all my site maps was starting to get a headache. Thanks

  3. pondicherry times says:

    yes i agree with you , but some site the sitemap like index.php?action=sitemap;xml
    but this may getting some problem ..
    its there any solution to over this problem

Leave a Reply

Your email address will not be published. Required fields are marked *