{"id":2257,"date":"2007-04-18T22:27:04","date_gmt":"2007-04-18T16:57:04","guid":{"rendered":"https:\/\/www.quickonlinetips.com\/archives\/2007\/04\/the-correct-way-to-add-sitemap-link-to-robotstxt-file\/"},"modified":"2020-05-24T09:53:41","modified_gmt":"2020-05-24T04:23:41","slug":"the-correct-way-to-add-sitemap-link-to-robotstxt-file","status":"publish","type":"post","link":"https:\/\/www.quickonlinetips.com\/archives\/2007\/04\/the-correct-way-to-add-sitemap-link-to-robotstxt-file\/","title":{"rendered":"The Correct Way to Add Sitemap Link to Robots.txt File"},"content":{"rendered":"<p>How to add add Sitemap link to Robots.txt file?\u00a0As you are aware that Ask, Google, Microsoft and Yahoo! have announced support of <a href=\"https:\/\/www.quickonlinetips.com\/archives\/2007\/04\/add-sitemaps-autodiscovery-in-robotstxt-file\/\">autodiscovery of Sitemaps<\/a> via robots.txt file, eliminating the need to <strong>submit sitemaps<\/strong> to each search engine separately. Although the instruction seemed simple enough, I tried inserting the sitemap url and noticed errors in the Google Webmaster console.<\/p>\n<h3>Add Sitemap Link to Robots.txt File<\/h3>\n<p>The simple instructions are to insert the sitemap link in the robots.txt file like this<br \/>\n<code>Sitemap: <\/code><\/p>\n<p>I inserted the Sitemap as instructed<\/p>\n<p><code>Sitemap:<br \/>\nUser-agent: *<br \/>\nDisallow:<\/code><\/p>\n<p>It all seemed fine until I checked my <a href=\"https:\/\/www.google.com\/webmasters\/tools\/siteoverview?hl=en\" rel=\"noopener\">Google Webmaster tools<\/a> dashboard and spotted errors in the robots.txt analysis and got these <strong>parsing errors<\/strong>.<br \/>\n<img decoding=\"async\" src=\"https:\/\/www.quickonlinetips.com\/archives\/wp-content\/uploads\/sitemaperror.jpg\" alt=\"Sitemaps Errors\" \/><\/p>\n<p>Since I spotted this error, I checked the analysis report daily after <strong>modifying my robots.txt<\/strong> file in different ways (after all these top search engines had this all worked out before). I needed to wait for about a day to recheck reports since the webmaster tools check for a new robots.txt file approximately once per day. And this is what finally worked without error<\/p>\n<p><code>Sitemap:<br \/>\nUser-agent: *<br \/>\nDisallow:<\/code><\/p>\n<p>The basic change that is required is that you need a empty line after the sitemap link. Now there are no errors.The lesson learnt is that you need to <strong>edit your robots.txt file with care<\/strong>, since it directly influences the way search engines index your site and the Google Webmaster tools is a valuable tool to see if it is configured in the correct way.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>How to add add Sitemap link to Robots.txt file?\u00a0As you are aware that Ask, Google, Microsoft and Yahoo! have announced&#8230;<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[6,41],"tags":[],"class_list":["post-2257","post","type-post","status-publish","format-standard","hentry","category-blogging","category-search-engine-optimization"],"_links":{"self":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts\/2257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/comments?post=2257"}],"version-history":[{"count":0,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts\/2257\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/media?parent=2257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/categories?post=2257"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/tags?post=2257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}