How to De-index Individual Pages in WordPress

Title banner for the page - reads "How to De-index Individual Pages in WordPress"

For a variety of reasons, you may have some pages on your WordPress website which you need to prevent Google from indexing. There is a right way to do it and a wrong way. Maybe you’ve just received the nasty notice from Google regarding your robots.txt file and how you need to remove the “noindex” directives from within and have no idea how to proceed.

In any case, we’ve got you covered! Depending on your needs, you may be able to use a plugin or you can add some simple code that provides a nice and lightweight option.

Google Warning

Screenshot of the email from Google regarding use of "noindex" in robots.txt

Email sent out by Google regarding the “noindex” directive

Odds are that you’ve ended up here because Google sent you a warning message regarding your use of “noindex” in your robots.txt file. The contents of that email are as follows:

“Remove “noindex” statements from the robots.txt of https://example.com/

To owner of https://snowwolfplows.com/,

Google has identified that your site’s robots.txt file contains the unsupported rule “noindex”.

This rule was never officially supported by Google and on September 1, 2019 it will stop working. Please see our help center to learn how to block pages from the Google index.”

While it may sounds very concerning, it’s really not a huge deal and we can solve this issue pretty easily. And take note: you won’t have any issues until at least September 1st, 2019. So as of the time of this writing, you have plenty of time to fix it. If you haven’t done anything by then, what will happen is that the specified pages will start to get indexed in Google.

The reason your are getting this email is because, like many websites, yours has added a “noindex” line to the robots.txt file to direct Google not to index specific pages on your site. Apparently, this was never an officially supported command. Google has been abiding by it, but at this point they’ve decided they won’t anymore.

I’ve heard it said that they are campaigning to make some kind of standard, though I’m not sure why they wouldn’t just make this the standard, as it’s so easy!

Easiest Solution: Plugins

It’s not all that hard to de-index pages using plugins! Particularly if you are already using Yoast SEO. Yoast actually has a feature built right in that allows you to de-index pages and posts. Just scroll down to the Yoast SEO section of your post from the backend, click the “Advanced” tab, and under “Allow search engines to show this post in search results?”, click the dropdown and select “No”.

Screenshot of Yoast SEO plugin showing how to de-index posts in WordPress

Under “Allow search engines to show this post in search results”, select “No”. In this screenshot, we currently have the default selected, which is “Yes”.

If you use All-in-One SEO Pack, unfortunately you are out of luck. Frankly it boggles the mind, but that plugin has no option for de-indexing pages.

I’m sure there are other plugins that can do this, too, but I haven’t found any. Yoast is my recommendation! Or, you can add some lightweight code.

Lightweight Option: Custom Function

It’s not too hard to add some lightweight code to your site that will de-index specific pages on your site. You’ll have to manually add the page/post IDs, but actually using it is not very hard, and it shouldn’t really add any load time to your site which is nice.

It’s fairly easy to add! All you need to do is find and edit the functions.php file from your theme. Note: you should really only be doing this if you are using a child theme. Otherwise, if your theme gets updated, these changes will be overwritten and you’ll have to start over. Also make sure you back everything up ahead of time, and generally you need to have some idea of what you’re doing.

Add this code before the final ?> at the end of the file, or just after everything if that doesn’t exist.

Then, replace the 1,2,3 in the $no_index array with your post ideas. In this example, posts 1, 2, and 3 will be de-indexed. Replace this with your post IDs, placing a comma between each.

To find the post ID in a page, just go to that page, click “edit page/post” from the toolbar (or navigate to it from the backend), and get the ID from the edit page URL. The ID is always included in that URL. For instance, this post that I’m editing right now has a URL of:

The post ID is 3818, so that’s the number you would add to the code. If you have other pages, find their IDs and add each one with a comma between.

Make sure you upload the new version of the file back to your site, and it should be all set!

Conclusion

It’s not generally too hard to de-index pages on your WordPress site properly. Unfortunately, robots.txt won’t work for this purpose anymore but we can easily get around that.

Google tends to take it in to their own hands to control the development of the internet, and this is the new example of that. I’m not sure exactly what was wrong with the old method and why they decided it was inadequate without offering an alternative, but here we are!

Let me know if you have any trouble with this.

 

About Brian Johnson

Brian Johnson is a website developer and designer living in Minneapolis, Minnesota with a passion for code and WordPress. He spends his days building WordPress websites for small businesses, developing new code with the online community, and living life.

Leave a Reply

Your email address will not be published. Required fields are marked *