The Illusion of Freedom: The Hidden Cost of Schema-less Databases
Tags: database, sql, mongodb, programming
Published: 2023-03-14 19:40:22

As software developers, we love the idea of flexibility and freedom. That's why schema-less databases like MongoDB can be so appealing - they promise a world without migrations or schema changes. But have you ever stopped to think about how you interrogate the structure of your data? Do you have a centralized, browsable definition, or are you spelunking through pages of code to find your answers?

## The Illusion of Freedom

Schema-less databases, such as MongoDB, offer developers a sense of freedom and flexibility when it comes to managing their data. These databases allow for a more ad-hoc approach to data modeling, which can be useful in many situations. Long drawn out discussions about schema layout, and hand-wringing over writing and running migrations as data evolves are a thing of the past!

At first glance, this sounds like a dream come true - but as the project grows and evolves over time, the true cost of a schema-less database becomes apparent. The lack of enforced structure can lead to a poorly-managed, inaccurate, and distributed "lens" onto the shape of the data. This "lens" is essentially expressed in the codebase itself - in the way that data is read, written, and manipulated.

As a result, it can become difficult to maintain data integrity and ensure data consistency. Performing complex queries can also become a challenge, as the ad-hoc approach to data modeling can make it unclear how different pieces of data relate to each other. Basically, while a schema-less database can offer a sense of freedom and flexibility, it often leads to a more complex, error-prone, and difficult-to-maintain system in the long run.

## The Implicit Schema

Even in a schema-less database, there is always a schema that exists, whether it's explicit or implicit. An implicit schema is one that is not defined upfront, but rather emerges from the way that data is read, written, and manipulated in the codebase.

For example, imagine that you're working with a MongoDB database and you have a collection of users. In your codebase, you'll likely have various functions that read and write data to this collection, each of which makes assumptions about the shape and format of the data. Over time, these assumptions become ingrained in the codebase, forming an implicit schema.

The problem with an implicit schema is that it can be difficult to maintain over time. As the codebase evolves and new developers join the team, it can be hard to keep track of all the different assumptions that have been made about the data. This can lead to inconsistencies and errors that can be hard to debug.

While a schema-less database may provide a sense of flexibility, relying on an implicit schema can ultimately lead to a less robust and less maintainable system in the long run.

## The Explicit Schema

An explicit schema is a defined structure for data that offers several advantages over an implicit schema. With an explicit schema, it's easier to maintain data integrity and consistency, perform complex queries, and integrate with other systems. By defining the structure of the data upfront, developers can avoid creating an implicit schema that can be difficult to manage and maintain. This can make the codebase easier to understand and onboard new developers to the project.

> _Note:_ SQL databases and other schema-driven stores are not a magic bullet. Migrations and schema definitions can be a headache in and of themselves. I'll cover this in another post soon.

## Making Tradeoffs

While schema-less databases like MongoDB can seem like a developer's dream come true, they often result in poorly managed, inaccurate, distributed lenses onto the shape of our data. By relying on an implicit schema, we're just kicking the schema management can down the road, and making life harder for ourselves in the long run. So, next time you're tempted by the siren song of a schema-less database, ask yourself: is the illusion of freedom really worth the potential headaches down the line? Alternatively, I'd love to hear from anybody who maintains schema sanity at scale with a NoSQL DB as their core datastore.