Managing a Product Catalog in MongoDB - Some User Notes 
Tags: mongodb, pim
Published: 2019-11-04 18:39:17

When we began the complete overhaul of our frontend and backend at Eddie Bauer, we sought to create new tech stack that would create a better, faster customer experience as well as simplify our convoluted system architecture. Looking back on the successful project, I can confidently identify our transition to MongoDB from our Oracle SQL databases as one of the biggest wins. The transition to a document store model with MongoDB made a huge difference for the subsequently built infrastructure as well as the speed and development benefits it provided so I will briefly touch on why I believe transitioning to MongoDB was the right decision. 

First, the use of a document store solved a number of long-running issues with our product catalog. EB has a couple thousand products spread amongst multiple categories that are technical and non-technical; casual and outdoor; north-shore Nancy and mountain-guide Matt. The issue we frequently encountered with this set-up is the schema was very disparate amongst the categories with many category specific fields. Additionally, we expected fields to change frequently as we designed new technical product with a new class of features or identified new impactful data fields to add. Our developers found it very tiresome to continually modify our relational database to accommodate these changes that would inevitably impact the rest of our feeds to other systems and vendors. The use of a document library with MongoDB, meant we could manage these changes much better as the fields were document specific and the only work that was required was on the feed receiver's end. MongoDB is schema-less, so every feed item had the ability to contain different properties and there was no requirement to pre-define anything except the primary key.

Additional to the the management of our schema, the query speed and general performance of Mongo were way ahead of our old system. One of our goals with the project was to improve our site speed which at the time was an abysmal 5 seconds. In fact, some pages were even recorded at 11 seconds! In an e-commerce world where 30% of consumers leave the page if it takes more than 3 seconds to load, the increase in performance Mongo helped allow was welcome. Mongo excels when datasets become too large for quick querying and may have been a little overkill as we only had slightly below 1 million records, but we still saw a performance increase in our querying speed.

On top of all this, MongoDB was easily integrated with the rest of our tech stack that uses Javascript and NodeJS. Mongo offers first-class support for Javascript and allowed us to operate an entirely flexible JSON document store. Essentially, Mongo shifted the complexity to the application to have that use the data as it wished. When we eventually moved to a React framework, the flexibility in our development was critical. 

The transition to MongoDB was my first introduction to a non-relational DB and I was surprised by how much it benefited our system architecture. Ultimately, I would recommend its use to anyone dealing with a large and volatile product catalog with a Javascript-based application on top.