DIGITAL SANKALP

Meta tag verification for Google, Bing and Alexa

Meta tag verification for Google, Bing and Alexa

Know your websites statistics and performance in web using Google webmaster tools , Bing webmaster tools and Alexa and in order to sh...

10:00 am

How to submit your URL to Google, Bing and Yahoo – Add your website

How to submit your URL to Google, Bing and Yahoo – Add your website

Every site needs to get ranked in search engines, but before that a website must first get indexed in search results. Search engines are s...

9:56 am

How to submit your URL to Google, Bing and Yahoo – Add your website

How to submit your URL to Google, Bing and Yahoo – Add your website

Every site needs to get ranked in search engines, but before that a website must first get indexed in search results. Search engines are s...

9:56 am

How to create sitemap for Blogger blog – Blogger sitemap XML

<div class="post-content" style="margin:0px;padding:0px;color:rgb(102,102,102);font-family:'Droid Sans';line-height:22px"><p style="margin:0px 0px 1.2em;padding:0px"><img alt="Creating sitemap in Blogger - Blogger sitemap XML" class="alignright" height="159" src="http://blogtimenow.com/wp-content/uploads/2013/10/blogger-sitemap.jpg" style="margin: 0px 0px 15px 30px; padding: 0px; border-width: 0px; max-width: 100%; height: auto; box-sizing: border-box; float: right;" title="Blogger sitemap" width="164">A Sitemap is nothing but a list of accessible pages in your website. Sitemaps helps search engines like <a href="http://blogtimenow.com/how-to/submit-url-google-bing-yahoo-add-website/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Submitting your site to Google Yahoo and Bing">Google, Yahoo and Bing</a> to easily crawl pages in your site which helps in better index. As a blogger you must create a sitemap so whenever you make a new post search engines can crawl and index them easily. In this post let's see how to create sitemap for Blogger blog and submitting it to <a href="http://blogtimenow.com/seo/how-to-use-google-webmaster-tools-effectively-for-seo/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Google webmaster tools">Google webmaster tools</a> and in robots.txt.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">Default Blogger sitemap</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">By default your blogger blog will have sitemap, but the issue with that sitemap is it only shows your recent blog posts (<a href="http://sureyea.blogspot.com/feeds/posts/default?orderby=UPDATED" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Default Blogger sitemap">example</a>). A perfect sitemap is that it should contain the list of all you pages so that search engines know the complete structure of your site. Let's see how to create sitemap for Blogger blog.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">How to create sitemap for Blogger</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">Creating sitemap is very simple and this sitemap works for both <a href="http://blogtimenow.com/blogging/self-hosted-blog-vs-third-party-blog-hosting-ultimate-comparison/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Self hosted VS third party blog hosting">self-hosted Blogger blogs and normal Blogger blogs</a>. Just use this (<span style="margin:0px;padding:0px;color:rgb(255,102,0)"><strong style="margin:0px;padding:0px">atom.xml?redirect=false&start-index=1&max-results=500</strong></span>)next to your blogs URL. See example below.</p>  <p style="margin:0px 0px 1.2em;padding:0px"><a href="http://yourblogname.blogspot.com/">http://yourblogname.blogspot.com/</a><span style="margin:0px;padding:0px;color:rgb(255,102,0)"><strong style="margin:0px;padding:0px">atom.xml?redirect=false&start-index=1&max-results=500</strong></span></p>  <p style="margin:0px 0px 1.2em;padding:0px">Now you have created sitemap for your Blogger blog, but you need to tell search engines about your sitemap so that bots can know your site structure. There are two ways to tell search engines about your sitemap.</p>  <p style="margin:0px 0px 1.2em;padding:0px">1. Showing your sitemap in robots.txt file and 2. Submitting your sitemaps in Google webmaster tools</p><h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em">  <span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">Adding your Blogger sitemap in robots.txt file</strong></span></h3><p style="margin:0px 0px 1.2em;padding:0px">Login to your <a href="http://blogtimenow.com/blogging/beginners-guide-blogging-blogger/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Blogger Guide">Blogger blog</a> and go to <strong style="margin:0px;padding:0px">dashboard >> settings >> search preferences</strong> and edit the custom robots.txt. Enable and paste the following text and click save changes. Make sure to change your blog name in Blogger sitemap below.</p>  <div id="highlighter_311991" class="syntaxhighlighter  " style="width:2284.90625px;margin:1em 0px!important;padding:1px!important;border:0px!important;outline:0px!important;background-image:none!important;background-color:rgb(15,25,42)!important;float:none!important;vertical-align:baseline!important;height:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important">  <div class="lines no-wrap" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;overflow-x:auto!important;overflow-y:hidden!important">  <div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">1</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">User-agent: *</code></td>  </tr></tbody></table></div><div class="line alt2" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">2</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Disallow: /search</code></td>  </tr></tbody></table></div><div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">3</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Allow: /</code></td>  </tr></tbody></table></div><div class="line alt2" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">4</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">   </td></tr></tbody></table></div><div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">5</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Sitemap: <a href="http://sureyea.blogspot.in/atom.xml?redirect=false&start-index=1&max-results=500">http://sureyea.blogspot.in/atom.xml?redirect=false&start-index=1&max-results=500</a></code></td>  </tr></tbody></table></div></div></div><p style="margin:0px 0px 1.2em;padding:0px;text-align:center"><img alt="How to create sitemap for Blogger - Blogger sitemap XML" class="aligncenter" height="219" src="http://blogtimenow.com/images/blogger-sitemap2.jpg" style="margin: 0px auto 15px; padding: 0px; border-width: 0px; max-width: 100%; height: auto; box-sizing: border-box; display: block;" width="690"></p>  <p style="margin:0px 0px 1.2em;padding:0px">So whenever search engines crawls your site they will access your Blogger sitemap from robots.txt file.</p><h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em">  <span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">Submitting your Blogger sitemap in Google webmaster tools</strong></span></h3><p style="margin:0px 0px 1.2em;padding:0px">Login to your <a href="http://blogtimenow.com/seo/how-to-use-google-webmaster-tools-effectively-for-seo/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Guide to Google webmaster tools">Google webmaster tools</a> and select you website. In your site dashboard click sitemaps below. Now click add/test sitemaps, add your sitemap (just the<strong style="margin:0px;padding:0px"> atom.xml?redirect=false&start-index=1&max-results=500</strong>) and submit your sitemap.</p>  <p style="margin:0px 0px 1.2em;padding:0px"><img alt="How to create sitemap in Blogger blog" class="aligncenter" height="170" src="http://blogtimenow.com/images/blogger-sitemap3.jpg" style="margin: 0px auto 15px; padding: 0px; border-width: 0px; max-width: 100%; height: auto; box-sizing: border-box; text-align: center; display: block;" width="356"></p>  <p style="margin:0px 0px 1.2em;padding:0px">That's it you have successfully submitted your sitemap in Google webmaster tools. Now it's ready for crawling and indexing. You can find the number of pages crawled and indexed in Google webmaster tools.</p>  <p style="margin:0px 0px 1.2em;padding:0px"><img alt="Blogger sitemap - Submitting sitemap in Google webmaster tools" class="aligncenter" height="266" src="http://blogtimenow.com/images/blogger-sitemap1.jpg" style="margin: 0px auto 15px; padding: 0px; border-width: 0px; max-width: 100%; height: auto; box-sizing: border-box; text-align: center; display: block;" width="334"></p>  <p style="margin:0px 0px 1.2em;padding:0px">If you like then also submit your Bloggers sitemap to <a href="http://blogtimenow.com/seo/use-bing-webmaster-tools-seo/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Guide to Bing webmaster tools">Bing webmaster tools</a>.</p>  <p style="margin:0px 0px 1.2em;padding:0px">Hope this helped you on how to create sitemap for Blogger and submitting it in robots.txt and Google webmaster tools. Please leave your comments below.</p></div><br>

How to create sitemap for Blogger blog – Blogger sitemap XML

A Sitemap is nothing but a list of accessible pages in your website. Sitemaps helps search engines like Google, Yahoo and Bing to easily c...

9:50 am

How to create sitemap for Blogger blog – Blogger sitemap XML

How to create sitemap for Blogger blog – Blogger sitemap XML

A Sitemap is nothing but a list of accessible pages in your website. Sitemaps helps search engines like Google, Yahoo and Bing to easily c...

9:50 am

Create & add custom robots.txt file in Blogger – Crawl & index

<h1 class="post-title" style="padding:4px 0px 0px;line-height:36px;font-family:Arapey;font-weight:normal;margin:0px 0px 2px!important"><br></h1><div class="post-content" style="margin:0px;padding:0px;color:rgb(102,102,102);font-family:'Droid Sans';line-height:22px">  <a id="dd_start" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219);float:left;clear:both"></a><p style="margin:0px 0px 1.2em;padding:0px"><img alt="How to create & add custom robots.txt file in Blogger" class="alignright" height="288" src="http://blogtimenow.com/wp-content/uploads/2013/11/custom-robots.txt-file-blogger.jpg" style="margin: 0px 0px 15px 30px; padding: 0px; border-width: 0px; max-width: 100%; height: auto; box-sizing: border-box; float: right;" width="353">Have you ever heard the name robots.txt or have you added your own custom robots.txt file in your Blogger blog. Here we will discuss about what <strong style="margin:0px;padding:0px">robots.txt</strong> is and how to <strong style="margin:0px;padding:0px">create and add custom robots.txt file in Blogger</strong>. All that deals with crawling and indexing of your blog which covers your blog <a href="http://blogtimenow.com/category/seo/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="SEO">SEO</a>, so take note on this article.</p>  <p style="margin:0px 0px 1.2em;padding:0px">In Blogger you have a section called search preferences that covers your blogs Meta tags, errors /redirection & crawling and indexing. There you can manage your blogs view with search engines also to maintain a healthy blog you need to customize your blogs search preferences effectively. A few posts back we made an article on how to use <a href="http://blogtimenow.com/blogging/blogger-custom-redirects-old-url-new-url/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Blogger custom redirects">Blogger custom redirects</a>. In case if you had a <a href="http://blogtimenow.com/seo/free-broken-link-checker-tool-websites/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Broken link checker Tools">broken links</a> in your blog or if you <a href="http://blogtimenow.com/blogging/changing-blogger-post-url-permalink-published/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="How to change blog post URL">changed your blog post URL</a> then you can make use of Blogger custom redirects. Here let's see about <strong style="margin:0px;padding:0px">robots.txt</strong>and the use of adding custom robots.txt file in Blogger.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">What is robots.txt?</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">Search engine like Google sends spiders or crawlers whatever it may be a kind of program that travels all around the web. When these crawlers or web spiders reach your site they first go through your robots.txt file to check any robots exclusion protocol that is before crawling and indexing your pages.</p>  <p style="margin:0px 0px 1.2em;padding:0px">Robots.txt is a normal text file which is available on all websites that is used by a webmaster to advice these crawlers about accessing several pages on a website. The pages that are restricted in your robots.txt file will won't be crawled and indexed in search results. However all those page are viewable publicly to normal humans.</p>  <p style="margin:0px 0px 1.2em;padding:0px">Each Blogger blog will have a robots.txt file that comes by default and it looks something like the one below. You can check your blogs robots.txt file by adding <strong style="margin:0px;padding:0px">/robots.txt</strong> next to your domain name. (<a href="http://yoursite.blogspot.com">http://yoursite.blogspot.com</a><strong style="margin:0px;padding:0px">/robots.txt</strong>)</p>  <div id="highlighter_471567" class="syntaxhighlighter  " style="width:2284.90625px;margin:1em 0px!important;padding:1px!important;border:0px!important;outline:0px!important;background-image:none!important;background-color:rgb(15,25,42)!important;float:none!important;vertical-align:baseline!important;height:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important">  <div class="lines no-wrap" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;overflow-x:auto!important;overflow-y:hidden!important">  <div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">1</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">User-agent: Mediapartners-Google</code></td>  </tr></tbody></table></div><div class="line alt2" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">2</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Disallow: </code></td>  </tr></tbody></table></div><div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">3</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">   </td></tr></tbody></table></div><div class="line alt2" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">4</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">User-agent: *</code></td>  </tr></tbody></table></div><div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">5</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Disallow: /search</code></td>  </tr></tbody></table></div><div class="line alt2" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">6</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Allow: /</code></td>  </tr></tbody></table></div><div class="line alt1" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">7</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">   </td></tr></tbody></table></div><div class="line alt2" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <table style="border-width:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;border-collapse:collapse!important">  <tbody style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important"><tr style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important">  <td class="number" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:3em!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(56,86,111)!important">  <code style="margin:0px!important;padding:0px 0.3em 0px 0px!important;border:0px!important;outline:0px!important;background-image:none!important;text-align:right!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:2.7em!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:block!important">8</code></td>  <td class="content" style="margin:0px!important;padding:0px 0px 0px 0.5em!important;border-width:0px 0px 0px 3px!important;border-left-style:solid!important;border-left-color:rgb(67,90,95)!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:top!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-size:1em!important;direction:ltr!important;color:rgb(185,189,182)!important;white-space:pre!important">  <code class="plain" style="margin:0px!important;padding:0px!important;border:0px!important;outline:0px!important;background-image:none!important;float:none!important;vertical-align:baseline!important;height:auto!important;width:auto!important;line-height:1.1em!important;font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important;font-size:1em!important;direction:ltr!important;display:inline!important;color:rgb(209,237,255)!important">Sitemap: <a href="http://yoursite.blogspot.com/feeds/posts/default?orderby=UPDATED">http://yoursite.blogspot.com/feeds/posts/default?orderby=UPDATED</a></code></td>  </tr></tbody></table></div></div></div><h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">So what are these?</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">As you can see above the default robots.txt file has few things like user-agent, media partners-Google, user-agent:*, disallow and site map. If you are not aware about these, then here is the explanation.</p>  <p style="margin:0px 0px 1.2em;padding:0px">First you need to know about User agent which is a software agent or client software that will act on behalf of you.</p><p style="margin:0px 0px 1.2em;padding:0px"><strong style="margin:0px;padding:0px">Mediapartners-Google</strong> – Media partner Google is the user agent for Google adsense that is used to server better relevant ads on your site based on your content. So if you disallow this they you will won't able to see any ads on your blocked pages.</p>  <p style="margin:0px 0px 1.2em;padding:0px"><strong style="margin:0px;padding:0px">User-agent:*</strong> – So you all know what user-agent is, so what is <strong style="margin:0px;padding:0px">user-agent:*</strong>. The user-agent that is marked with <strong style="margin:0px;padding:0px">(*) asterisk</strong> is applicable to all crawlers and robots that can be Bing robots, affiliate crawlers or any client software it can be.</p>  <p style="margin:0px 0px 1.2em;padding:0px"><strong style="margin:0px;padding:0px">Disallow:</strong> By adding disallow you are telling robots not to crawl and index the pages. So below the user-agent:* you can see <strong style="margin:0px;padding:0px">Disallow: /search</strong> which means you are disallowing your blogs search results by default. You are disallowing crawlers in to the directory /search that comes next after your domain name. That is a search page like <strong style="margin:0px;padding:0px"><a href="http://yoursite.blogspot.com/search/label/yourlabel">http://yoursite.blogspot.com/search/label/yourlabel</a></strong> will not be crawled and never be indexed.</p>  <p style="margin:0px 0px 1.2em;padding:0px"><strong style="margin:0px;padding:0px">Allow</strong> – Allow: / simply refers to or you are specifically allowing search engines to crawl those pages.</p><p style="margin:0px 0px 1.2em;padding:0px">  <strong style="margin:0px;padding:0px">Sitemap:</strong> Sitemap helps to crawl and index all your accessible pages and so in default robots.txt you can see that your blog specifically allowing crawlers in to sitemaps. You can learn more about <a href="http://blogtimenow.com/blogging/create-blogger-sitemap/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Blogger sitemap">Blogger sitemap</a> here. There is an issue with default Blogger sitemap, so learn <a href="http://blogtimenow.com/blogging/create-blogger-sitemap/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="How to create sitemap in Blogger">how to create sitemap in Blogger</a> and notify search engines.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">What pages should I disallow in Blogger?</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">This question is little tricky and we cannot predict what pages to allow and what to disallow in your Blog. You can disallow pages like privacy policy, Terms & conditions, cloaked affiliate links, <a href="http://blogtimenow.com/blogging/labels-use-labels-blogger-sort-content/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Labels">labels</a> as well as search results and it depends all upon you. Since you get some reasonable traffic from search results it is not recommended that you disallow the <a href="http://blogtimenow.com/blogging/labels-use-labels-blogger-sort-content/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Labels in Blogger">labels page</a>, privacy policy page and TOS page.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">How to disallow pages in Blogger using robots.txt</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">You can disallow search engines to crawl and index particular pages or posts in Blogger using your robots.txt file.</p><p style="margin:0px 0px 1.2em;padding:0px">We don't have the reason to block search engines on any particular posts and if you wish so then just add <strong style="margin:0px;padding:0px">Disallow: /year/month/your-post-url.html</strong> in your robots.txt file. That is copy your post URL next to your domain name and add it in your robots.txt file.</p>  <p style="margin:0px 0px 1.2em;padding:0px">Same what you will need to do for disallowing any particular pages. Copy the page URL next to your domain name and add it like this <strong style="margin:0px;padding:0px">Disallow: /p/your-page.html</strong> in your robots.txt file.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">The best and recommended robots.txt file for Blogger</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">Only use custom robots.txt file if you are <strong style="margin:0px;padding:0px">100%</strong> sure on what you are doing. Improper use of custom robots.txt can harm your site rankings. So for best results it is recommended that you use default robots.txt in Blogger which works good. But change the default sitemap in your robots.txt and add your <a href="http://blogtimenow.com/blogging/create-blogger-sitemap/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Sitemap for Blogger">custom sitemap for Blogger</a>.</p>  <h3 style="margin:20px 0px 0.4em;padding:0px;line-height:1.4em;font-family:ABeeZee;color:rgb(0,0,0);font-weight:normal;font-size:1.4em"><span style="margin:0px;padding:0px;color:rgb(50,91,163)"><strong style="margin:0px;padding:0px">How to create and add custom robots.txt file in Blogger</strong></span></h3>  <p style="margin:0px 0px 1.2em;padding:0px">In wordpress we have to create a robots.txt file in notepad and upload it to the web root directory. But here in Blogger you can add the robots.txt file easily from your blog dashboard. To add a custom robots.txt just login to your Blogger profile and select your blog. Now head to <strong style="margin:0px;padding:0px">dashboard >> settings >> search preferences</strong> and you can see custom robots.txt in crawling and indexing section. Click edit and enable custom robots.txt content and add your robots.txt file</p>  <p style="margin:0px 0px 1.2em;padding:0px"><img alt="How to create and add custom robots.txt file in Blogger" class="aligncenter" height="271" src="http://blogtimenow.com/images/custom-robots.txt-file-blogger2.jpg" style="margin: 0px auto 15px; padding: 0px; border-width: 0px; max-width: 100%; height: auto; box-sizing: border-box; text-align: center; display: block;" width="695"></p>  <p style="margin:0px 0px 1.2em;padding:0px">Once done click save changes. Now to check your robots.txt just add <strong style="margin:0px;padding:0px">/robots.txt</strong> at the end of your blog URL and you can see your custom robots.txt file. After adding your custom robots.txt file you can submit your blog to search engines. Learn how to <a href="http://blogtimenow.com/how-to/submit-url-google-bing-yahoo-add-website/" style="margin:0px;padding:0px;outline:none;color:rgb(31,69,219)" target="_blank" title="Submit your blog to Google, Yahoo and Bing">submit your blog to Google, Bing and Yahoo</a>.</p>  <p style="margin:0px 0px 1.2em;padding:0px">Hope this post clearly explained you about robots.txt and how to create and add custom robots.txt file in Blogger. Please share it and if you have any other doubts on Blogger robots.txt then feel free to comment below.</p>  </div><span style="color:rgb(102,102,102);font-family:'Droid Sans';line-height:22px">- </span>

Create & add custom robots.txt file in Blogger – Crawl & index

Have you ever heard the name robots.txt or have you added your own custom robots.txt file in your Blogger blog. Here we will discuss about...

9:37 am

DIGITAL SANKALP

Meta tag verification for Google, Bing and Alexa

How to submit your URL to Google, Bing and Yahoo – Add your website

How to submit your URL to Google, Bing and Yahoo – Add your website

How to create sitemap for Blogger blog – Blogger sitemap XML

How to create sitemap for Blogger blog – Blogger sitemap XML

Create & add custom robots.txt file in Blogger – Crawl & index

Popular Posts

Comments

Blog Archive

Blog Archive