SQL vs. NoSQL Database: When to Use, How to Choose
By: Satish Chandra Gupta
Tags: NoSQL, Artificial Intelligence, Big Data, Technology, Data Engineering

NoSQL for Semi-structured Data

NoSQL datastores cater to semi-structured data types: key-value, wide column, document (tree), and graph.

Key-Value Datastore

A key-value store is a dictionary or hash table database. It is designed for CRUD operations with a unique key for each record:

Create(key, value): Add a key-value pair to the datastore

Read(key): Lookup the value associated with the key

Update(key, value): Change the existing value for the key

Delete(key): Delete the (key, value) record from the datastore

The values do not have a fixed schema and can be anything from primitive values to compound structures. Key-value stores are highly partitionable (thus scale horizontally). Redis is a popular key-value store.

Wide-column Datastore

A wide-column store has tables, rows, and columns. But the names of the columns and their types may be different for each row in the same table. Logically, It is a versioned sparse matrix with multi-dimensional mapping (row-value, column-value, timestamp). It is like a two-dimensional key-value store, with each cell value versioned with a timestamp.

Wide-column datastores are highly partitionable. It has a notion of column families that are stored together. The logical coordinates of a cell are: (Row Key, Column Name, Version). The physical lookup is as following: Region Dictionary ⇒ Column Family Directory ⇒ Row Key ⇒ Column Family Name ⇒ Column Qualifier ⇒ Version. So, wide-column stores are actually row-oriented databases.

Apache HBase was the first open-source wide-column datastore. Check out HBase in Practice, for core concepts of wide-column datastores.

Document Datastore

Document stores are for storing and retrieving a document consisting of nested objects. a tree structure such as XML, JSON, and YAML.

In a key-value store, the value is opaque. But the document stores exploit the tree structure of the value to offer richer operations. MongoDB is a popular example of a document store.

Graph Datastore

Graph databases are like document stores but are designed for graphs instead of document trees. For example, a graph database will suit to store and query a social connection network.

Neo4J is a prominent graph database. It is also common to use JanusGraph kind of index over a wide-column store.

SQL vs. NoSQL Database Comparision

Non-relational NoSQL datastores gained popularity for two reasons:

RDBMS did not scale horizontally for Big Data

Not all data fits into strict RDBMS schema

NoSQL datastores offer horizontal scale at various CAP Theorem tradeoffs. As per CAP Theorem, a distributed datastore can give at most 2 of the following 3 guarantees:

Consistency: Every read receives the most recent write or an error.

Every read receives the most recent write or an error. Availability: Every request gets a (non-error) response, regardless of the individual states of the nodes.

Every request gets a (non-error) response, regardless of the individual states of the nodes. Partition tolerance: The cluster does not fail despite an arbitrary number of messages being dropped (or delayed) by the network between nodes.

Note that the consistency definitions in CAP Theorem and ACID Transactions are different. ACID consistency is about data integrity (data is consistent w.r.t. relations and constraints after every transaction). CAP is about the state of all nodes being consistent with each other at any given time.

Only a few NoSQL datastores are ACID-complaint. Most NoSQL datastore support BASE model:

Basically Available: Data is replicated on many storage systems and is available most of the time.

Data is replicated on many storage systems and is available most of the time. Soft-state: Replicas are not consistent all the time; so the state may only be partially correct as it may not yet have converged.

Replicas are not consistent all the time; so the state may only be partially correct as it may not yet have converged. Eventually consistent: Data will become consistent at some point in the future, but no guarantee when.

Difference between SQL and NoSQL

Differences between RDBMS and NoSQL databases stem from their choices for:

Data Model: RDBMS databases are used for normalized structured (tabular) data strictly adhering to a relational schema. NoSQL datastores are used for non-relational data, e.g. key-value, document tree, graph.

RDBMS databases are used for normalized structured (tabular) data strictly adhering to a relational schema. NoSQL datastores are used for non-relational data, e.g. key-value, document tree, graph. Transaction Guarantees: All RDBMS databases support ACID transactions, but most NoSQL datastores offer BASE transactions.

All RDBMS databases support ACID transactions, but most NoSQL datastores offer BASE transactions. CAP Tradeoffs: RDBMS databases prioritize strong consistency over everything else. But NoSQL datastores typically prioritize availability and partition tolerance (horizontal scale) and offer only eventual consistency.

SQL vs. NoSQL Performance

RDBMS are designed for fast transactions updating multiple rows across tables with complex integrity constraints. SQL queries are expressive and declarative. You can focus on what a transaction should accomplish. RDBMS will figure out how to do it. It will optimize your query using relational algebra and find the best execution plan.

NoSQL datastores are designed for efficiently handling a lot more data than RDBMS. There are no relational constraints on the data, and it does not need to be even tabular. NoSQL offers performance at a higher scale by typically giving up strong consistency. Data access is mostly through REST APIs. NoSQL query languages (such as GraphQL) are not yet as mature as SQL in design and optimizations. So you need to take care of both what and how to do it efficiently.

RDBMS scale vertically. You need to upgrade hardware (more powerful CPU, higher storage capacity) to handle the increasing load.

NoSQL datastores scale horizontally. NoSQL is better in handling partitioned data, so you can scale by adding more machines.