We recently encountered a situation where a key page of a client’s website simply disappeared from the Google search index. One day it was there in first position on page one of Google next day it was gone!
The first step you might well think would be to panic and that’s pretty much a natural reaction to this kind of occurrence, after all in the modern ultra competitive online market place losing your websites ranking even by a few places can have a financial effect on the business.
We are dealing with systems with rules here not magic so following some straightforward steps to deal with the problem and put in place a solution should be the order of the day.
As a Webmaster your first thought might be to just go straight to Google Webmaster Tools and re-submit the site to the search index. Good idea, but if the page has an issue that you are unaware this might not work. The best plan would be to spend a little time investigating and try and deduce what has happened to the page.
Hopefully these series of checks should help you identify the problem and get the website or web page back in Google search where it belongs.
It might seem obvious but it’s always good to cross off the obvious steps before moving on to the more complex ones. The first thing to do is to check to see if the page or website in question is still live; that any page or pages haven’t been accidently deleted or hidden through a setting in the CMS.
Just surf to the web page or website in question to check this. I did and the page was still there, first check crossed off the list.
In this scenario we are talking about one web page of a website not an entire website, but the steps taken should be the same.
The first step I took was to identify if the other pages of the site where ranking or had also disappeared from the search index. You can do this by typing the following into the Google search field:
Site:[your domain url].
From this I ascertained that the rest of the website was there it was just the one page at this stage that was missing.
I also discovered from my tests and searches through Google that the web page hadn’t just dropped down a few places or a page or two in the search rankings it had completely disappeared from the Google search index.
As a good webmaster you would be using Google Webmaster Tools, a great tool supplied by Google to help you manage your website in their search index.
There are a number of things to check in Webmaster Tools.
Manual Actions - Check here to see if there have been any manual spam submissions of your web page.
Remove URLs - You can also check here to make sure the page hasn’t been added as a removal request whether on purpose or by accident.
Crawl errors - Check here to see if any crawl errors have been reported.
Robots.txt tester - This is an important one to check. This feature of Webmaster Tools allows you to check the robots.txt for your website, making sure that page has not been set to be disallowed which means the Googlebot won’t index it. You can also use the Googlebot test feature located at the bottom of this page.
Fetch as Google - You can also use this tool to test the pages and also if necessary re-submit the page to the Google search index. In Webmaster Tools go to 'Crawl > Fetch as Google' and then enter the url of the page to see what Google makes of that page, see image below.
From all these checks I discovered no issues with the page!
This can be a tricky one to check and it would depend on how familiar you are with the SEO of the website, both on-page and off-page. In my case I had built the website and had been the only SEO person working on the site so I was pretty sure nothing had been done to cause Google to penalise the site.
Another check I made was to see if the page had been copied or ‘plagiarised’. If it had then it might have been penalised by Google, unlikely but worth a check.
This might have happened for a number of reasons; the page content might have been used elsewhere on the client’s website therefore duplicating the content something Google doesn’t like as we know. Someone might have taken the content and used it on their own website, or the site might have been cloned and an entire copy of the site might be running somewhere on the web! I have actually experienced this last one but that is a story for another blog post.
There are some free online tools you can use to check this including: http://www.quetext.com/ or https://searchenginereports.net/plagiarism-checker/
It’s worth checking to see if the site or web page appears in other search engines, run a search through Bing www.bing.com to see if the page appears. If it does then it’s a Google search specific issue.
Another check would be to see if the web server that hosts the website restarted recently. It could have been the case that the server restarted at the same time the Google bot was visiting the website. This would mean that the bot records the site or page as missing and its then removed from the Google search index.
Another possible cause was that the page was being found as 404 (Page not found) rather that a 200 (page ok). A long shot but worth checking out, you can use this tool to check the setting of a page: http://tools.seobook.com/server-header-checker/
In my case this wasn’t a problem so I moved on.
After making all these checks I couldn’t find anything wrong with the web page so I decided to re-submit the web page to the Google search index and wait to see if it reappeared. 24 hours later the page was back in position 1 on page 1 of Google as if it had never left!
What caused the disappearance? I can’t say 100% but it may well have been a server outage that caused the page to go missing. A couple of days before I noticed the page missing, the server that hosted the site had restarted after a routine update. It could have just been a horrible coincidence that the server restarted as the Google bot was visiting the page.
As with most things Google we’ll never know 100% what happened but if you follow these logical steps you should be able to find clues as to why and then rectify the problem.
Contact us
Call: +44 (0) 1702 482011
Email: info@phusemedia.com
Phuse Media Ltd, Polar House, 103 Rectory Grove, Leigh-on-sea, Essex, SS9 2HW
Read our reviews on Google
