Magento 301 Redirect Mapping: Ultimate Guide
Tags: magento
Published: 2019-07-11 10:26:08

Explaining Magento 301 Redirect Mapping

When there are planned changes in the website's URL structure, a 301 redirect map will help guide users and search engine crawlers to the new URLs. A well developed 301 redirect map is often the key to a successful website migration. Without implemented redirects, users will end up on 404 error pages, which would affect the amount of organic traffic and lead to a decrease in conversion rate. Without a properly implemented 301 map, all traffic sources will be affected.

Example of website migration without 301 mapping implemented (All traffic, GA)Example of website migration without 301 mapping implemented (All traffic, GA)

Overall, if changes in URL structure are implemented, all indexable URLs should be redirected using 301 redirects. The bare minimum is to make sure that at least the following crucial URLs won't return a 404 error page:

Landing pages which are receiving traffic, especially organic. Mapping such pages will support both SEO and overall user experience. Not only organic traffic, but also referral traffic, social media, bookmarks, e-mail campaigns and other sources of traffic might rely on old URLs. If the user lands on a 404 page instead of the correct one, it will cause friction and could be a reason for not completing the intended purchase.

URLs with valuable backlinks. Each backlink from a resource with high authority adds value to a website from an SEO perspective. More resources pointing to a specific page make it more trusted and authoritative in the eyes of Google. If a backlink leads to a 404 error page, the value is lost. Therefore, in cases when an URL has a lot of backlinks, even if the page doesn’t receive much traffic and won’t be present after the migration is done, there should be a 301 redirect in place.

How to handle Magento 301 Redirect Mapping
There are 6 main steps for coming up with a redirect map:

Understanding the scope. What, when and how is being migrated. Good understanding of changes in URL structure as well as principles how new URLs are generated will help be more efficient.

Gather. the full list of new URLs and the full list of old URLs.

Compare. Place the two lists side by side in a single sheet (Google Spreadsheet or Open Office).

Map. Now, match each old URL with its corresponding new URL.

Hand Off.Once you’ve developed a full 301 redirect map, give it to the developers to import in Back-end

Test before and after go-live.One missed page can directly lead to a decrease in website traffic and conversion rate.

Preparations for Magento 301 Redirect Mapping
Software and accesses.
Make sure you have the following:

Screaming Frog or any other SEO crawler

Open Office (or Excel)is quite good software to work with a big amount of cells (when working with large e-commerce stores with many products). Google spreadsheet and Excel may crash or/and be slow.

SEO tool with information about backlinks, e.g., Ahrefs, Semrush, Moz or other

Project-specific accesses. Google Analytics and Google Search Console accesses of the project. For a full analysis, it's possible to analyze also server log files.

Information on project
To avoid surprises, it’s very important to collect information about the scope of the work. Especially if SEO specialists are involved closer to the end of the migration, instead of working since the very beginning of the project.

Migration type. What type of migration is planned and does it involve major changes in URL structure (e.g. address change from example.eu to example.com).

The scope of work. Exactly which domains, subdomains, language versions, related websites (landing pages, blogs etc.) are planned to be migrated.

Changes in content. Is it planned to migrate all content? Maybe some set of pages will be deleted? What about product categories? Blog posts? Often not all products are being migrated as well, it's good to clarify approx. percentage of how many products will be actually migrated.

Changes in naming. Are category names staying the same? Are subcategory titles the same? CMS pages? etc.

URL structure. Are there major changes planned regarding URL structure? For example, www.example.com/category/product1-123.html to www.example.com/product1. Pay attention to the endings of URLs and beginnings of URLs. If the old website had .html at the end of each URL and now it no longer does, there should be redirects set for all pages even if, in general, page titles and URL structure stay the same. The same applies to situations when example.com/gb is changed to example.com/en/ and even when example.com/gb is changed to example.com/gb/. Regarding changes which affect the homepage and/or the whole website — consult with developers. They should be able to effortlessly create server-side redirect rules.

Existing maps. There already is often some pre-made map used in the planning phase.

Existing and Staging website walkthrough. Understand how they are different.

Note, even if it's said that product names will stay the same and as a result, all URLs will stay the same you still need to crosscheck every URL.

Overall Principles of Magento 301 redirect mapping
In the case of e-commerce stores, it’s easier to map URLs grouped by type. The typical structure of Magento stores includes — category pages, subcategory pages, product pages, CMS pages, home page, customer account, and cart/checkout related pages.

Collecting URLs for Magento 301 redirect mapping
You can use the following sources to get URLs:

For products and categories URLs — can ask developers to export URLs from the database

XML Sitemap

Whole website crawl

Google Analytics — top landing pages

SEO tools— URLs with backlinks

Mapping product pages
Example of Magento 2 Product page with the standard URL https://www.elekcig.de/joyetech-ego-aio-kit(https://www.elekcig.de/joyetech-ego-aio-kit)Example of Magento 2 Product page with the standard URL (https://www.elekcig.de/joyetech-ego-aio-kit)

If a store has thousands of products, they probably had some standard approach to product URL structure for both the old and new websites. In case of Magento 2, it’s typically www.example.com/product1 (product name right after domain name). If the old website had the same approach, then mapping won’t be needed (need to test though if it’s really, really the same using formulas in Excel)

If there are any changes then:

ask developers to export the product URL list from the old website and from the new website

adjust the URLs in excel e.g, If you had www.example.com/products/product1 remove products/ (can use Find and Replace for that) so that structure matches.

compare both lists using vlookup formula to check if all products from the new website can be found in the old one.

If yes, then it’s great! Prepare the document listing old PDP and correlating new PDP URLs.

This method applies if you are sure that the product titles stay more or less the same and the majority of products are being migrated. If the naming of the products doesn’t match, try to find some principles how they are created and try to make them similar. Or use some additional data. e.g., if there is the SKU or ID in the old URL, and this data is being migrated to the new website, ask developers to export SKU next to new URLs and you will get the opportunity to map them www.example.com/products/product1-2939 www.example.com/products/product12

Try to avoid manual mapping of product pages! If it’s only about changes in directories, then the redirects can be applied by developers as one specific rule which applies to all URLs. No need to upload a list with all product URLs in Magento Back-End.

Mapping category pages
If there are changes in category URLs then again try to look for some principles that have changed overall, e.g., directory is excluded or added. Similarly, as with products, it's important to crosscheck how the naming of categories has changed. There might be some manual mapping required as well.

CMS pages
Lafayette 148 NY CMS pageLafayette 148 NY CMS page https://www.lafayette148ny.com/art-of-the-season

Usually, e-commerce stores don’t have many custom CMS pages so you might map them manually. You can get such pages by crawling the old and new websites, excluding previously mapped categories as well as product pages. URLs which are left can be mapped.

Others
Make sure the homepage is mapped, if there are changes! Also, blog articles if there are any.

Top Landing Pages
GA is also a great source of URLs that need to be redirected — there might be URLs which are not internally linked and still get some traffic. The top landing pages, which are receiving traffic and are not included in the original redirects map, might need additional 301 redirects.

Select a period of 1–3 months (can be exceptions due to the seasonality of business etc.)

We won't need Paid traffic as a source, however, there might be URLs related to organic, referral, social and direct traffic worth mapping. So can use these channels as a source of information. The minimum is organic traffic. You can't be sure that GA data is 100% accurate (it can be broken due to the wrong setup), but you can quickly crosscheck if Google/organic as a source is approximately matching

Navigate to Landing page report and export Landing pages to .csv file

Export pages to the same Spreadsheet Open Office file where the Sitemap is.

Compare the list using VLOOKUP formula.

If some additional URLs are discovered, crawl them in Screaming Frog to check the status of pages (It’s suggested to create 301 redirects also if there is no 404 error)

To crawl specific URLs in Screaming Frog change Mode to List. Click on Upload => Enter Manually…

URLs with valuable backlinks
Identify pages which have backlinks from external sources (more than few and preferably with higher scores) using some SEO tool:

Export the list of URLs

Compare with the existing list of data from sitemap and GA

Crosscheck newly discovered URLs via Screaming Frog

Add to list

XML sitemap
Example of an XML SitemapExample of an XML Sitemap

Usually, the sitemap includes those URLs which are considered valuable from an SEO perspective, therefore they should be mapped in the 301 map. In case of Magento stores, if there weren’t used any extensions to generate XML, the sitemap probably will be missing CMS pages, homepage etc. Don’t consider XML sitemaps to be reliable by default!

However, it’s a good way for gathering all product and category URLs, especially if you can’t gather data directly from a database.

To make sure that URLs included in the sitemap are real and existing (no 404 errors, no 301 etc.) the sitemap should be crawled using Screaming Frog. Before downloading the sitemap, you can check the health of the sitemap in the Search Console (SC => Crawl => Sitemap). If SC is reporting some major problems with the sitemap, e.g. 404 errors, the sitemap might be outdated and can’t be trusted.

How to do sitemap crawling:

Find the location of the XML sitemap. Typically sitemaps should be listed in www.example.com/robots.txt. If the sitemap is not included in robots.txt file, check the Search Console (the sitemaps should be submitted there and you will get the specific URL). If you have access to the back-end of the store, you can easily find sitemap there.

Open Screaming Frog. Change the Mode (main navigation menu) to List

Choose Download Sitemap



Crawl sitemap

Export the results of the crawl as a .CSV file

Import to Open Office or Google Spreadsheet.

Delete unneeded columns. Keep — URLs, titles, meta descriptions, response codes, meta robots 1

Hide pages which have other response codes that 200

Full website crawl
To spot all internally linked URLs, it’s possible to do the whole website crawl in Screaming Frog. Be careful as canonicalized pages, pages with noindex tag, URLs with parameters etc. could be included — pages which don’t need 301 redirects, because they weren’t indexed. If there are no external URLs pointing to the URL with parameters and no incoming traffic, it’s possible to exclude it from 301 mapping.

Identifying redirect chains
When you have collected the list of URLs, run an additional crawl and create a redirect chain report in Screaming Frog.

Export the redirect chain report to Open Office/Google Spreadsheet to identify redirect chains with more than 2 URLs.

In such case, you will need to map both the 1st and 2nd URL in the chain.

Compare
After the full list of URLs from the old website is compiled and there is an accessible list of new URLs (can crawl the staging site after the content migration is done) they should be placed in one spreadsheet.

Map
Map means to match old URL with corresponding new URL. If there is no exact match can look for the closest alternative. If some product is missing, can map it to category page etc.

Handoff
Developers should receive one spreadsheet with 2 columns — Old URL and new URL.

Magento 301 Redirect Testing
Run 2 tests:

When URL redirects are implemented in the BE of the staging website, crawl the list to see if all redirects are working as expected.

Create a redirect chain report and crawl the second URLs in the list to make sure that they all report 200 status on the staging website.

After the go-live, run another crawl of your 301 redirects list to check if everything is still working as planned.

Bugs
You still might miss some valuable pages so monitor 404 reports in GA and add additional redirects if needed in the upcoming 2–3 weeks after go-live!
Let us help you:
Check out our Magento SEO and migration services here: https://scandiweb.com/services/magento-seo

Looking for more tips and tricks on Magento 301 redirect mapping? Perhaps you have some you'd like to share with us? Drop us a line at marketing@scandiweb.com!