5 Pitfalls of NoSQL Databases
By: Maciej Szymczyk
Tags: Data, Database, Sql, Data Engineering, NoSQL

I recorded a video in which I talk about the advantages of NoSQL databases. The response was interesting, but I had the impression that not everyone sees the two sides of the coin. The facts are that they can cause us a lot of problems 😉.

Schema Management

Each NoSQL database approaches the schema in its own way. In some there is no schema (MongoDB), in some, it is dynamic (Elasticsearch), and in some it resembles the one from relational databases (Cassandra). In the conceptual model, data ALWAYS have a pattern. Entities, fields, names, types, relations. Regardless of the type of base, the physical model is a representation of the conceptual model.

NoSQL databases give us more freedom in terms of schema. In MongoDB, we can add two different documents with the same field names but different types. Does this make sense? Rather not. Can this happen? Of course, it can. A simple human error can break our app.

Another issue is related to relationships between entities. Even if there are no relations in our db, we have to document the relationships between the data. From the relational database we can generate an ERD diagram. In case of NoSQL databases this may not work.

When using NoSQL databases we have to remember about schema management and data validation issues. Without it the data can “explode”. Interesting fact: Some companies replace MongoDB with PostgreSQL.

Lower margin of error

Performance of NoSQL databases is the result of proper data modeling, indexing and partitioning. In a relational database we can add columns, transform tables, flip data from one table to another, add an index if we have forgotten about it before. In case of NoSQL databases, this will not be possible in all cases. We may need to use some external tools like Apache Spark or even drop and recreate our data model.

In Elasticsearch, if we don’t get the schema/mapping of an index, we have to use e.g. Reindex API, which means that we have to re-index data to another index.

In Cassandra, we can only filter by partition and clustering keys. If we forgot to add one of the columns to the key, there is a possibility of adding an index, but this can kill performance if the cardinality of the set is large.

No ACID

The ACID properties simplify code writing. We do not need to handle errors related to the fact that the data in the X-table is already there, and not yet in the Y-table (if at all). From the CAP theorem we know that there are consistent and eventual consistent databases. The most popular database of this type is Apache Cassandra. Eventual consistency requires a different approach to data modeling and application logic. The code should be written in a more defensive way, as it is not certain that the record you just changed is already available from another part of the application. HBase is an example of a consistent database, but even Cloudera believes that it will not replace a relational database. Some databases advertise themselves as consistent, and consistency is only ensured to some extent. For example, MongoDB offers transactions, but multi-document transactions are available only since version 4.0.

No SQL

The disadvantage of NoSQL is the lack of… SQL 😁. We may like it or not, but SQL is the basis for data. Many analysts use SQL every day and learning another language may discourage them from using the database. There is a reason why Spark SQL or Beam SQL was created.

Limited analysis and/or no JOINs

This is a little of a discussion of differences between OLTP and OLAP systems. We are used to using GROUP BY and JOIN clauses, but not every database will offer such features. Aggregations and mergers may be very limited (if possible) due to the nature of the database. In the case of Apache Cassandra, analytical capabilities are often achieved by putting together the Apache Spark cluster. You will learn how to connect one to the other from this story of mine.

Summary

Is it worth using NoSQL then? Of course, it is. We must remember that each database was created to solve a class of problems. Bending “destiny” is possible, but there are consequences. Screws are easier to screw in with a screwdriver, to hammer nails with a hammer and to pain walls with a roller.

An interesting alternative is NewSQL databases. An example of such bases are CockroachDb and Spanner. They solve the problems of traditional relational databases (mainly the issue of scaling) while maintaining ACID properties.