Technical SEO is one of the most debated topics and misunderstood pillars of SEO. Although it is true that technical SEO is much more important for large websites than smaller ones, technical SEO is still a highly important ranking factor for your website in 2024. Some technical SEO issues can prevent pages from indexing, removing your pages from Google and stripping them of their ability to rank entirely. With our experience, we’ve seen almost every type of issue and have had to troubleshoot and identify solutions. This article provides you with real evidence-based insights and solutions from what we have found has worked for frustrating but common technical SEO issues.
Other issues can cause user experience issues on your website, losing potential sales and negatively impacting your overall Google rankings. This article shows you how to identify and more importantly, fix the most common technical SEO issues with actionable solutions and provides examples of these common issues and where to find them.
We are a leading SEO company in Bristol and across the UK. If you are having difficulties with the technical SEO of your website, want a free review of your current SEO strategy and performance or are looking to harness the power of SEO for your business – get in touch with our specialist team to arrange for a free audit.
Tools You Need Either Installed on Your Computer Website to Get Started:
- Google Search Console (Free)
- Screaming Frog SEO Spider (Free License Under 500 Page Crawl)
The Most Common Technical SEO Issues:
- The website’s sitemap contains 404 pages or 301 pages.
- Pages are not indexing on Google search results or are intermittently indexing and being removed.
- The sitemap is missing completely.
- Incorrect robots.txt file.
- Duplicate content on pages.
- Mixing Alt text/attributes on images.
- Broken Links
Using Google Search Console
Google Search Console is the most powerful tool for comprehensively assessing how Google crawls your website. Mastering this tool is key to ensuring that you are regularly reviewing and fixing technical issues that can occur. Here is what you can check in Google Search Console:
Sitemaps
Your sitemap and whether it is submitted to Google and submitted correctly.
Pages
An assessment of pages that aren’t indexed. Some of these pages reported should be non-indexable but you need to check that there aren’t any pages that aren’t currently indexed that should be.
Penalty Monitoring
Regular checks that you have no manual actions or security issues that affect the website. Google will tell you if you have a manual action that needs to be addressed (due to malignant activity against terms of service) and security issues or breaches. We have extensive experience in reversing manual actions – if you have a manual action against your website property – get in touch for a free audit.
Your Websites 404 Pages or 301 Pages
Your sitemap is a list that you provide search engines to confirm pages that need to be crawled. Common SEO plugins such as Yoast and Rank Math for WordPress will automatically.
If you have broken pages or needlessly redirected pages within this sitemap – this can affect the indexing of your pages by the crawl refusing to accurately crawl pages which can cause inconsistent indexing. To fix this:
WordPress:
- Update your robots.txt file to disallow any broken or redirected pages. This will force your CMS to update and force out the broken page.
Non WordPress:
- Rewrite your sitemap ensuring to remove any 301 pages or 404 pages. You need to check this on websites with manual sitemaps as your sitemap will not automatically update when pages are deleted. We have provided Google’s basic example, simply update your URLs and the page URL as well as the date of the time you are updating the sitemap and update the priority of pages at 1.0 for the Homepage, 0.8 for major pages and between 0.6 and 0.4 for secondary and blog article pages. Your developer can assist in creating your XML sitemap and regularly updating it.
<?xml version=”1.0″ encoding=”UTF-8″?>
<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>
<url>
<loc>https://www.example.com/foo.html</loc>
<lastmod>2022-06-04</lastmod>
</url>
</urlset>
- Re-verify your sitemap with Google Search Console to confirm everything is working correctly.
Why Are My Pages Not Indexing on Google Search Results or Are Intermittently Indexing and Being Removed?
There are a number of reasons why this may occur. The sitemap issue troubleshooting above is one instance of this. We have also seen multiple instances of larger websites where a sitemap is being regularly updated with a large number of pages being removed which has then caused major pages to randomly become unindexed or non-crawlable to Google. This then majorly affected major traffic-driving keywords.
Working example – we worked on an estate agent website where properties were being constantly deleted and re-added to the website. The automatic sitemap was therefore generating 100s of 404 pages and then within a day, another 100 pages were gone and 100 new pages appeared. This forced Google to just refuse to index the homepage and other major pages – leading to a huge drop in organic performance.
Solution – we uploaded a custom sitemap which only featured the top 40 pages driving organic traffic as well as the article area and articles. We removed all properties from indexing at all to prevent the issue and this bolstered and restored all major organic keywords with 10 organic keywords returning to the Rank 1 position.
How to Add a Website to Google Search Console
First begin by heading to search.google.com and clicking “Start now”.
To add your website to Google Search Console and begin appearing on Google Search Results Pages, you need to first verify your website. This involves adding a small file to your website which links your Google Search Console to the site.
Website ownership verification can be done in a number of ways including the following:
- Adding a the Google generated TXT file to your server records.
- Adding a block of HTML code to your homepage.
- Install the link through Google Analytics.
- Install the link through Google Tag Manager.
How to Setup Sitemaps in Google Search Console
A common issue that we see when taking over websites for SEO campaigns, is that the sitemap hasn’t been submitted at all or Google Search Console hasn’t even been set up for the property. Google Search Console also needs to be set up again when a website migrates from a www. to a non www address. This is a very quick win that can see boosts in Google rankings as well as allowing you to ensure that the pages you want to appear on Google are being submitted for consideration.
To submit your sitemap once verified or to submit a sitemap:
- Create an XML sitemap and upload to your server or request this with your developer for non-WordPress or custom CMS systems.
- For WordPress – major SEO plugins automatically generate your sitemap for you.
- Enter your sitemap URL and submit it to confirm validation.
Incorrect Robots.txt File
The robots.txt file is an additional file that can be updated to disallow pages from being crawled at all by the search engine. You can also update it to block certain websites from crawling your website, such as AI websites that take your content to be used such as Chat GPT. It’s vital that you check this file doesn’t contain issues such as disallow: /. This is used to hide a website completely such as when a new website is being built. You would be shocked to find out how many websites we have dealt with where this has been forgotten to be removed from the file when the website goes live again and all pages are now blocked from being crawled and therefore indexed or discovered by Google.
Simply update any pages you want to be disallowed as a new line. Ensure the sitemap is displayed within your robots.txt file like so.
Duplicate Content on Pages
It’s a myth that Google has a specific penalty for duplicate content however Google will rank the original page that was first indexed and then simply ignore or refuse to index duplicate pages. This causes these pages to never rank on Google so you need to ensure that every new page has unique, high-quality content.
- For pages that shouldn’t be duplicates – ensure that all new content is unique and high-quality avoiding duplicate text where possible.
- For pages where you feature duplicate content such as multiple product pages for one product description where duplicate content is going to appear – you need to use correct canonical tagging. Create one main product URL for ranking purposes and canonicalise all other product variations back to this main product URL. Product filters can also be used to circumvent this issue.
Working example – Shopify setup is often prone to not setting this up automatically as standard. You need to make sure that a single product URL is being generated in build which you can canonicalise back to.
Mixing Alt Text/Attributes on Images
Alt Text attributes on images are still important in 2024. You need to regularly review image uploads for alt-text and ensure that these are updated across your current images on your website. These should not be stuffed with keywords, these attributes should be descriptions of the image displayed to help with visually impaired users. Accessibility is tipped to be one of the next major factors that Google looks at in terms of user experience which often holds ranking advantages on Google as well as helps make the internet more accessible to all.
Broken Links on Pages or Needless 301 Redirects
Broken page links on your website or needless 301 issues are another factor that Google will look at. This is not as important on smaller websites but these issues still need to be rectified.
Using Screaming Frog, you can quickly identify 301 errors or 404 errors and this tool will give you a list of pages where the 301 or 404 link is being displayed. You will quickly notice patterns if the link is present across a large number of pages – it’s likely the link is within the header, footer or a page template across the website.
Conculsion
As we can see from the guide above, Technical SEO has many facets and this article but only scratches the surface. If you would like to learn more about how we can help with SEO, web design or other aspects of digital marketing then please get in touch.
