Have you tried searching for your website on Google and couldn’t find it? There’s a good chance that Google can’t find and index your website, which is a nightmare for SEO.
So why can’t Google find your site?
Here’s a checklist to find the problem and possibly fix the issue:
1. WordPress Settings
Check to make sure WordPress isn’t blocking search engines from indexing your site
Login to your WordPress site, and make sure the Search Engine Visibility box is unticked:
2. Robots Meta Tag
The robots meta tag is a piece of code that is placed in the header of your pages.
If you right click on the page that can’t be found on Google, and click “View Page Source” – look for this particular code:
<meta name=”robots” content=”noindex,follow”/>
If you see the noindex, then you need to get your web developer to change it to index instead.
3. XML Sitemap
When you add a new page to your website, make sure that it gets added to your sitemap.
An XML sitemap is like a directory of all the pages of your website that gives search engines a listing of all your pages.
You can check your XML sitemap by going to www.yourwebsite.com.au/sitemap.xml
4. Robots.txt File
The robots.txt file tells web robots which pages to crawl (and not crawl) on your website.
The robots.txt file also allows you to only allow specific bots to crawl your site and disallow others. To check your own robots.txt file – go to www.yourwebsite.com.au/robots.txt
The default robots.txt file for WordPress sites should look like this:
User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php
Bonus tip: Test your robots.txt file
Google offers a robots.txt file tester which you can use here: Robots.txt tester tool
You can put in any page from your website to see if Googlebot will be able to crawl that page:
If you see this:
Talk to your web developer about your robots.txt file, and check with your hosting provider to make sure that they haven’t blocked Googlebot’s IP addresses.
