Indexing as many pages in your web site as doable might be very tempting for entrepreneurs who’re attempting to boost their search engine authority.

However, whereas it’s true that publishing extra pages which can be related for a selected key phrase (assuming they’re additionally top quality) will enhance your rating for that key phrase, typically there’s truly extra worth in preserving sure pages in your web site out of a search engine’s index.

… Say what?!

Stick with us, of us. This put up will stroll you thru why you may need to take away sure webpages from the SERPs (search engine outcomes pages), and precisely learn how to go about doing it.

De-Indexing a Web page from Google

There are a number of events the place you could need to exclude a webpage — or a portion of a webpage — from search engine crawling and indexing like:

• To prevent duplicate content (when there may be multiple model of a web page listed by the major search engines, as in a printer-friendly model of your content material) from being listed
• To deal with admin and login pages for inside use until they’re meant for use by a group
• For a thank-you page (i.e., the web page a customer lands on after converting on one of your landing pages) the place the customer will get entry to no matter supply that touchdown web page promised, equivalent to a hyperlink to an e-book PDF

Here is what the thank-you web page for our SEO tips ebook appears to be like like, for instance:

You need anybody who lands in your thank-you pages to get there as a result of they’ve already crammed out a kind on a touchdown web page — not as a result of they discovered your thank-you web page in search.

Why not? As a result of anybody who finds your thank-you web page in search can entry your lead-generating affords immediately — with out having to offer you their data to move by way of your lead-capture kind. Any marketer who understands the value of landing pages understands how necessary it’s to seize these guests as leads first, earlier than they will entry your affords.

Backside line: In case your thank-you pages are simply discoverable by way of a easy Google search, you could be leaving invaluable leads on the desk.

What’s worse, you could even discover that a few of your highest-ranking pages for a few of your long-tail key phrases may be your thank-you pages — which implies you possibly can be inviting tons of of potential results in bypass your lead-capture kinds. That is a fairly compelling purpose why you’d need to take away a few of your net pages from SERPs.

So, how do you go about “de-indexing” sure pages from serps? Listed here are 3 ways to do it.

3 Methods to De-Index a Webpage From Search Engines

Robots.txt to De-Index

Use if: You need extra management over what you de-index, and you’ve got the mandatory technical sources.

One approach to take away a web page from search engine outcomes is by including a robots.txt file to your web site. The benefit of utilizing this technique is that you would be able to get extra management over what you might be permitting bots to index. The end result? You may proactively maintain undesirable content material out of search outcomes.

Inside a robots.txt file, you’ll be able to specify whether or not you’d like to dam bots from a single web page, a complete listing, and even only a single picture or file. There’s additionally an possibility to stop your web site from being crawled whereas nonetheless enabling Google AdSense advertisements to work in case you have them.

That being stated, of the 2 choices obtainable to you, this one requires probably the most technical kung fu. To discover ways to create a robots.txt file, you will need to read through this article that explains exactly how to do it..

HubSpot prospects: You may discover ways to set up a robots.txt file in your web site here, and discover ways to customise the contents of the Robots.txt file here.

If you happen to don’t want all of the management of a robots.txt file and are in search of a better, much less technical resolution, then this second possibility is for you.

Htaccess No Index No Comply with to De-Index

Use if: Your web site is operating on Apache, and mod_headers is enabled, this can be a fast resolution.

On this circumstance, you possibly can connect this single line to your .htaccess file:

Header set X-Robots-Tag “noindex, nofollow”

To suggest that your web site might be listed, however by no means proven in any Google search outcomes.

Meta No Index No Comply with to De-Index

Use if: You need a better resolution to de-indexing a complete webpage, and/or de-indexing the hyperlinks on a complete webpage.

Utilizing a metatag to stop a web page from showing in SERPs — and/or the hyperlinks on a web page — is each simple and efficient. It requires solely a tiny little bit of technical know-how — actually, it is actually only a copy/paste job in case you’re utilizing the best content management system.

The tags that allow you to do this stuff are known as “noindex” and “nofollow.” Earlier than I get into learn how to add these tags, let’s take a second to outline and distinguish between the 2. They’re, in any case, two utterly completely different directives — and so they can be utilized both on their very own, or alongside each other.

What’s a “noindex” tag?

Whenever you add a “noindex” metatag to a webpage, it tells a search engine that regardless that it will possibly crawl the web page, it can’t add the web page into its search index.

So any web page with the “noindex” directive on it is going to not go into the search engine’s search index, and might subsequently not be proven in search engine outcomes pages.

What’s a “nofollow” tag?

Whenever you add a “nofollow” metatag to a webpage, it disallows serps from crawling the hyperlinks on that web page. This additionally signifies that any rating authority the web page has on SERPs will not be handed on to pages it hyperlinks to.

So any web page with a “nofollow” directive on it is going to have all its hyperlinks ignored by Google and different serps.

Like I stated earlier than, you’ll be able to add a “noindex” directive both by itself, or along with a “nofollow” directive. You can even add a “nofollow” directive by itself, too.

When to Use “noindex” and “nofollow” Individually

Add solely a “noindex” tag if you do not desire a search engine to index your net web page in search, however you do need it to comply with the hyperlinks on that web page — thereby giving rating authority to the opposite pages your web page hyperlinks to.

Paid touchdown pages are an excellent instance of this. You don’t need serps to index touchdown pages that individuals are presupposed to pay to see, however you may want the pages it hyperlinks to learn from its authority.

Add solely a “nofollow” tag if you do desire a search engine to index your net web page in search, however you do not need it to comply with the hyperlinks on that web page.

There aren’t too many examples of if you’d add a “nofollow” tag to a complete web page with out additionally including a “noindex” tag. Whenever you’re determining what to do on a given web page, it is extra a query of whether or not so as to add your “noindex” tag with or and not using a “nofollow” tag.

When to Use “noindex, nofollow” Collectively

Add each a “noindex” and “nofollow” tag if you do not need serps to index a webpage in search, and you don’t need it to comply with the hyperlinks on that web page.

Thank-you pages are an excellent instance of this example. You don’t need serps to index your thank-you web page, nor would you like them to comply with the hyperlink to your supply and begin indexing the content material of that supply, both.

The best way to Add a “noindex” and/or a “nofollow” Metatag

Step 1: Copy one of many following tags.

For “noindex”:

<META NAME=”robots” CONTENT=”noindex”>

For “nofollow”:

<META NAME=”robots” CONTENT=”nofollow”>

For each “noindex” and “nofollow”:

<META NAME=”robots” CONTENT=”noindex,nofollow”>

Step 2: Add the tag to the <head> part of your web page’s HTML, a.ok.a. the web page’s header.

If you happen to’re a HubSpot buyer, that is tremendous simple — click on right here or scroll down for these directions particular to HubSpot customers.

If you happen to’re not a HubSpot buyer, then you definitely’ll have to stick this tag into the code in your webpage manually. Don’t fret — it is fairly easy. Here is the way you do it.

First, open the supply code of the net web page you are attempting to de-index. Then, paste the total tag into a brand new line inside the <head> part of your web page’s HTML, generally known as the web page’s header. The screenshots beneath will stroll you thru it.

The <head> tag signifies the start of your header:

Here is the metatag for each “noindex” and “nofollow” pasted inside the header:

And the </head> tag signifies the tip of the header:

Growth! That’s it. This tag tells a search engine to show round and go away, leaving the web page out of any search outcomes.

No Index No Comply with in HubSpot

Including the “noindex” and “nofollow” meta tags is even simpler. All it’s important to do is open the HubSpot software to the web page you need to add these tags to and select the “Settings” tab.

Subsequent, below Superior Choices and click on into “Head HTML.” Within the window beneath, paste the suitable code snippet. Within the instance beneath, I’ve added each a “noindex” and a “nofollow” tag since it is a thank-you web page.

Press “Save,” and also you’re golden.

Efficiently No Index No Following a Web page

You’ve simply magically erased your web page from search engine outcomes. Now, you can begin capturing extra of these misplaced leads once more.

Now, understand that you will not see outcomes instantaneously. Your modifications will not kick in till the subsequent time a search engine crawls your web page. Relying on how typically you usually publish new pages in your web site, it truly might take a number of weeks. The extra continuously you publish content material, the extra typically serps will crawl your web site. One of the best ways to maintain observe of how continuously Google visits your web site is to look at your crawl stats in Google Webmaster Tools.

Backside line: If you happen to discover that your web page continues to be showing in Google’s search outcomes even with the “noindex” tag, it is most likely as a result of Google hasn’t crawled your web site because you added the tag. You may request that Google recrawl your web page utilizing the Fetch as Google tool.

Additionally, be aware that some serps’ net crawlers may interpret these directives otherwise than Google does, so your web page may nonetheless seem in outcomes from different serps. However for Google, it will work simply positive — as soon as it does get round to crawling your web site.

Regardless, you can sleep a bit of simpler figuring out you’ve finally made your web site a greater place on your advertising and marketing.

Editor’s be aware: This text was initially revealed July 2016 and has been up to date for comprehensiveness.