One of the most common SEO mistakes I encounter every day is the mishandling of 404 errors. URLs that return a 404 HTTP response code are a normal part of the web; just about every site has them. 404 errors can be served for a couple of reasons. Maybe you are trying to access a page that has been moved or no longer exists on the sites server, or maybe you misspelled the URL for that particular page. Although 404 errors do not have a direct correlation to your sites rankings, it is believed that too many of these errors can act as a signal to search engines that maybe your sites content isn’t the best or most up to date for the query. The most important part is understanding how to approach them correctly in order to preserve user experience and link juice. This can be accomplished by creating a customized 404 page and installing 301 redirects when appropriate.

What Should I Do With Them?

The web is constantly changing and so are the sites that populate it. Many sites want to change content, and should often, in order to stay fresh. But what do you do in the case of retired pages or a change in URL structure? In many cases I see sites set up to redirect all 404 errors to the homepage. Unfortunately this is killing the site rather than helping it. Approaching 404s in this manner is essentially devaluing your homepage by sending the user and search engines to a page that is not relevant to the original URL.

Another problem I see often is when a site is set up to take the user or search engine to a custom error page but instead of returning  404 response it will return a 200 OK, this is known as a soft 404.

 Webmaster Tools Report

The goal here is to get the visitors to where they want to go, and to keep search engines from leaving the site and continue crawling once a 404 error is reached. So how do we accomplish this?

Well, there are a couple of ways to ensure that visitors and search engines stay on your site while also retaining some of that traffic and link juice from those retired pages.

  • Implement 301 Redirects: Redirecting all 404 errors is not recommended. 301 permanent redirects should be used in cases where your URL has changed. Pages that have been retired should return a 404 response header instead of returning 404-like content.
  • Custom 404 Page: If you retire product pages or pages with content that is no longer relevant to your site then it is perfectly fine to let those 404. Like I said before, 404 errors are perfectly normal. Your approach to them is what really matters. This is when a custom 404 page comes into play. A custom error page offers the user and search engines more options than just dead end.

It is also important to remember that if you do retire pages that will return a 404, you must remove any links to that page from within your site or they will still be indexed.

Importance of a Custom 404 Page

One of the most overlooked aspects of site design and SEO is the implementation of a custom 404 page. Too often I see sites using the default error page instead (shown below), which can result in lost visitors and keep search engines from crawling the entire site.

Generic 404 Page

What is wrong with the page above? Do you think the average user knows what a 404 HTTP Response Code is? This is why customized 404 pages are so very important. A customize page will allow visitors and search engines to continue navigating your website in the event of a broken link or deleted page. Remember, search engines do not have a back button. If you do not include links for the crawling process to continue then the spider will see it as a dead end and jump off site.

Custom error pages also offer the user a more personalized experience and a better explanation of what a 404 is. There are some important elements that every sites 404 page should contain that will keep visitors browsing and search engines crawling:

  • Maintain Consistent Design: The customized page should include the same basic design as the rest of the site. I usually look at this as a “gutted out” version of one of the sites pages. The header, navigation and footer should all be intact so the user has options when a 404 error is triggered.
  • Explain In Normal Words: Like I said before, the average user does not know what a 404 HTTP response code is. Try adding a simple message in the body of the page saying something like “Sorry, the page you are looking for no longer exists.”
  • Provide Links to Relevant Content: A common method I have seen quite often is the inclusion of links right below the error message that takes the user to the most popular product or content pages of the site. Including a search bar in this area is also a great addition that will enable the user to find exactly what they are looking for.

  • Colby Phillips

    Solid advice. Its amazing how many sites treat 404 pages as dead end streets and don’t offer users alternative paths to actual content.

  • Clever PhD

    Have you considered showing a 200 on the page with a noindex nofollow meta tag and then having a message that the page has moved for the user. This then tells Google to remove the page from the index and then they stop crawling it.

    I mention this as I have seen soft 404s for when you have a many to one relationship with a bunch of pages 301ing to a single page (Google likes to have more of a 1 to 1 relationship with 301s). I have seen when we have too many 301s in a many to one relationship that I get a lot of soft 404 errors. The 200 with the noindex tag looks to solve this.

    • Stephen Craig

      Thanks for the Reply!

      If there’s a good reason to keep the page around a while for visitors, like referring them to an updated version of a product, or a page with content that is very similar, that can be a smart approach.

      This is especially true for sites that feature products or opportunities in newsletters, or in email notifications, and that may urge people to visit a URL even after a product might be sold out, or a job might be filled.

      At some point though, it might be a good idea to then 404 the page, especially after traffic dies down to it, or it’s no longer such a fresh offer.

      (1) No more Gizmo 2.0′s are available.

      (2) But Gizmo 3.0′s are now out.

      (3) robots meta noindex the Gizmo 2.0 page

      (4) Tell people on the Gizmo 2.0 page that the Gizmo 3.0 is now out, and provide a link.

      (5) Check Google Analytics in a month to see if people are still coming to the Gizmo 2.0 page, and if not then delete the page so that if anyone tries to visit the page, the see the custom error page, and get a 404 server error message.

      (6) if there are still lots of visitors, keep it around for another month, and check it again in Google Analytics. Rinse and Repeat if necessary.

      Hope this helps!

  • vinca swanson

    Thanks for the article – I’m a big fan of customized 404 pages. They further explain a company’s attention to customer service. Question for you: do you think it’s good practice to have the 404 page and a search page be the same page?