# MongoDB

## Summary


MongoDB is a NoSQL database. More specifically, it's a document-oriented database. It stores data as *documents*. Documents themselves are organized into an abstraction called *collections*. Documents consist of key-value pairs. Thus, MongoDB's a hybrid of relational databases and key-value stores. 

MongoDB came out in 2009 as a response to the limitations of relational SQL databases. These imposed strict schema that didn't suit some use cases. Schema migrations were a pain. Scaling was vertical and expensive. These are the problems that MongoDB set out to solve. 

MongoDB can be deployed on-premise or on the cloud.  There are a host of tools to manage custom deployments. MongoDB is managed by MongoDB, Inc. Although the code is open, it's licensing is not really considered open source. 

Despite some criticisms, MongoDB continues to grow in adoption. It's fully managed cloud service, MongoDB Atlas, is particularly popular.

## Discussion

### How does MongoDB organize data?

Whereas a SQL database organizes data into tables containing rows and columns, MongoDB organizes data into **collections** containing **documents** and **fields**. A MongoDB instance can manage multiple databases. Each database can have multiple collections. A collection encapsulates multiple documents that share some similarity.

A document is really a JSON document containing key-value pairs. Values can be numbers, strings, dates, arrays or even other JSON documents. Internally, documents are stored in a binary format called **Binary JSON (BSON)**. Objects in many programming languages can be mapped easily to MongoDB documents. This simplifies application code and dispenses with Object–Relational Mapping (ORM). 

MongoDB has two storage engines: WiredTiger and In-Memory. 

Consider a news article with title, image, URL, tags, and comments. In SQL, article attributes, image, tags and comments would be stored in separate tables. In MongoDB, the article and its associated image, tags and comments are stored in a single document. Retrieval of this data is simple without needing SQL's complex join queries. Thus, MongoDB employs a **document data model**. 


### What are the main features of MongoDB?

MongoDB's key features relate to its document-oriented model. Being **schema-less**, the model is flexible and avoids schema migrations. If required, schema validation can be enabled. 

Even without a schema, MongoDB supports **ad hoc queries**. This is powered by *MongoDB Query Language (MQL)*. Real-time aggregations are supported. 

**Indexes** are supported for faster search. A primary key is automatically created for each document and this is indexed. In addition, secondary indexes are supported. 

For reliability, **replica sets** are used, with each set containing duplicates of the data. These are also useful to load balance the reads. A typical replica set has one primary node and many secondary nodes. 

Administrators can choose the right **speed-durability trade-off** as suited for the application. By default, journaling is enabled by which writes are flushed to journal file every 100ms. Journaling can be disabled for faster write performance and instead replicated for better durability. 

MongoDB scales well due to **sharding**. Large collections are distributed across nodes. Queries are routed to relevant shards, thereby balancing the load. Sharding can be range-based, hash-based or tag-based. Individual shards are also part of replica sets. 


### What are the use cases where MongoDB fits?

One critic of MongoDB, identified only two use cases for MongoDB: for storing semi-structured data where no strict schema can be imposed; and for prototyping, where we might experiment with different design choices for the schema. 

In an alternative viewpoint, MongoDB is claimed to be suited for web applications, where data relations are common. This is attributed to MongoDB's abstractions (collections, documents), indexes, dynamic queries and aggregations. 

In Agile methodology, quick iterations are essential. MongoDB saves time because it doesn't deal with schemas and migrations. 

MongoDB has been applied to analytics and logging. Atomic updates (increment counters, append to arrays) and capped collections (keep only recent documents) are two features that are relevant here. Logs can be stored in the database rather than in files. 

Caching is another use case. MongoDB can be used as a caching layer. 


### What are the deployment options of MongoDB?

**MongoDB Community Server** or **MongoDB Enterprise Server** can be downloaded, installed and self-managed. This is available for various platforms (Windows, macOS, Linux). The enterprise version is also part of MongoDB Enterprise Advanced subscription. This gives comprehensive support even when you run MongoDB on your own infrastructure. 

An alternative is to use MongoDB as a fully managed service in the cloud. This is called **MongoDB Atlas**. This comes with the proven benefits of cloud computing: availability, scalability, security, performance, automation, and more. As of October 2021, this is available for AWS, Google Cloud, and Azure. 


### Which are the tools for managing or working with MongoDB?

Here are some useful tools: 

    + **MongoDB Shell**: Called `mongosh`, it's a CLI to connect, query and work with a database. Available since v5.0, it deprecates an earlier shell called `mongo`.
    + **MongoDB Compass**: A GUI for MongoDB. Variants include Readonly Edition and Isolated Edition (disables unnecessary network connections).
    + **MongoDB CLI for Cloud**: Called `mongocli`, it's a CLI for managing cloud services such as MongoDB Atlas, MongoDB Cloud Manager, and MongoDB Ops Manager.
    + **MongoDB Database Tools**: Include useful utilities for binary import/export, data import/export, and diagnostics.
    + **MongoDB Connectors**: A set of connectors to interface with Business Intelligence (BI) tools, Apache Spark, Apache Kafka and Kubernetes.
    + **MongoDB Charts**: Data visualization for data stored in MongoDB Atlas and Atlas Data Lake.
    + **MongoDB Ops Manager**: For monitoring, backup, automation and query optimization. Part of MongoDB Enterprise Advanced subscription.
    + **MongoDB Cloud Manager**: It's a hosted management platform. Similar features as in Ops Manager but SaaS and updated more frequently. You need to install only relevant agents (monitoring, automation, or backup) in your own self-managed deployments. Less relevant if you're using MongoDB Atlas.

### What do you mean by MongoDB drivers?

Applications need to connect to MongoDB instances and run queries. Developers often write these queries at the application layer but connecting to a MongoDB instance is more complex and requires networking knowledge. To simplify the developer's job, the latter is implemented by MongoDB and provided as libraries. These are called **MongoDB drivers**.

Currently, drivers are available in Java, Node.js, C/C++, C#/.NET, Go, Python, PHP, Scala, Ruby, Rust and Swift. Apart from these, there are also community-supported drivers. 

For some drivers, it's important to check for compatibility before installing them. For example, Rust driver version 1.0 doesn't support MongoDB 5.0 and all Rust driver versions requires at least MongoDB 3.6. In terms of language compatibility, Rust driver requires at least Rust 1.48. 

While drivers help interacting with MongoDB programmatically, it's possible to do the same via MongoDB Shell (CLI) or MongoDB Compass (GUI). MongoDB Charts for visualization and MongoDB Connectors are other ways to interact with MongoDB. 


### What are some criticisms of MongoDB?

MongoDB's document data model is ill-suited for social media data. For example, a user's activity stream will link to other users (friends, commenters, likers). This results in data duplication and an update nightmare to keep data consistent. Normalizing the activity stream and executing additional queries puts more burden on the application. 

In 2015, Slootweg pointed out many of MongoDB's problems: data loss, ignored errors, slow, locking issues, insecure defaults, non-ACID, etc. If a developer uses Mongoose (a library for relations and schema validation), he proposes that PostgreSQL is a better option. 

MongoDB's schema-less approach was blamed for data inconsistency and loss. Application code that attempted to ensure consistency become complex and buggy. Meanwhile, in the late 2000s and early 2010s, other NoSQL databases arrived that solved specific use cases in better ways: Redis for real-time data and caching, Cassandra for Big Data, Elasticsearch and Solr for search, InfluxDB for time-series (IoT) data, and more. 

Most applications have relational data with schema and require migrations. MongoDB's ObjectIds are hard to work with. So are aggregation pipelines, which can hit memory limits. There's vendor lock-in to MongoDB, Inc. 


### Where can I learn more about MongoDB?

Visit the official MongoDB website to learn more. You can download the MongoDB Community Server to try out the database. If necessary, you can later migrate to the MongoDB Enterprise Server. Alternatively, try the free tier of MongoDB Atlas. 

Read the Getting Started guides. Read the official documentation. This includes installation guide, FAQ, command reference, tools, data modelling, and much more. 

Check out the Developer Hub and the MongoDB Blog. Participate in the MongoDB Community Forum to get help or be informed of the latest developments.

As a developer, you can learn the basic database commands from Beugnet's MongoDB Cheat Sheet. 

Among the books worth reading are *The Little MongoDB Book*, *MongoDB in Action* and *Practical MongoDB Aggregations*.

Those who prefer to learn from videos can check out these YouTube playlists: MongoDB Tutorial for Beginners and MongoDB Dev & Admin Tutorial Videos

## Milestones

2007

Eliot Horowitz, Dwight Merriman, and Kevin Ryan start a company named *10gen*. Their goal is to build a web-centric PaaS based on open source components. In the process, they realize that relational databases are hard to scale. Thus was born the idea for MongoDB. Horowitz recounts later that, "MongoDB was born out of our frustration using tabular databases in large, complex production deployments."  

Feb  
2009

**MongoDB 1.0** is released. By August, this version becomes generally available for production environments.  

Dec  
2009

MongoDB 1.2 is released with support for **aggregations**. This is implemented in Node.js via a Map-Reduce API. This is slow and queries are non-intuitive for developers. With the release of MongoDB 2.2 (August 2012), this is remedied with the **Aggregation Framework**. This is far more intuitive, powerful, efficient, and scalable. It soon becomes the go-to tool for developers. However, MongoDB continues to support Map-Reduce for legacy reasons. 

Aug  
2010

MongoDB 1.6 is released. Sharding is now production ready, thus making MongoDB **horizontally scalable**. Replica sets provide automated failover. They deprecate replica pairs used earlier. 

2013

The LAMP stack has been popular web stack for many years. Valeri Karpov coins the term **MEAN**, a new web stack that stands for MongoDB, Express, Angular and Node. Variants of this stack such as MERN and MEVN also include MongoDB. In these stacks, MongoDB fulfils the database layer. Its JSON-like documents fits nicely with JavaScript frameworks. 

Mar  
2015

MongoDB 3.0 is released. **WiredTiger** storage engine is available. There's also a pluggable storage engine API. **MongoDB Ops Manager** is available. 

Jun  
2015

MongoDB Management Service (MMS) is renamed to **MongoDB Cloud Manager**. This is useful for monitoring, alerting, taking automated backups, upgrading without any downtime, and easily creating sharded clusters and replica sets. MMS was first released in 2011. 

Dec  
2015

MongoDB 3.2 is released. WiredTiger becomes the default storage engine. The previous default was MMAPv1. Config servers, which were previously mirrored, can now be deployed as replica sets. **Partial indexes** are supported, meaning that only some documents are indexed. This reduces storage requirements and performance costs for maintaining indexes. **Schema validation** is now possible on a per-collection basis during updates and insertions. 

Jun  
2016

**MongoDB Atlas** is launched as a fully managed database in the cloud, more commonly termed as Database as a Service (DBaaS). Due to automation, there's no need to manually install, configure, take backups, and monitor. For hybrid environments, MongoDB Cloud Manager can be used. For full on-premise deployments, MongoDB Ops Manager can be used. 

Nov  
2016

MongoDB 3.4 is released. Among the new features are **read-only views** and associating shards with one or more **zones**. To enable language-specific rules for string comparison, **collations** are introduced. Linearizable read concern level is introduced. Along with "multiple" write concern, this allows multiple threads to safely read/write on a single document. 

Jun  
2018

MongoDB 4.0 is released. **Multi-document transactions** are now possible. Transactions can be across multiple operations, collections, databases, documents, and shards. 

Jan  
2019

Amazon Web Services (AWS) launches **DocumentDB**, a database that emulates the MongoDB 3.6 API. Clients that use MongoDB can use DocumentDB as a drop-in replacement to MongoDB server. AWS could do this because MongoDB uses AGPL licensing. In anticipation of this, MongoDB, Inc. changed the licensing to **Server Side Public License (SSPL)** back in October 2018. SSPL required vendors to open source the entire stack. This move is not well received by the open source community. RedHat drops MongoDB from its RHEL distributions. 

Jul  
2021

MongoDB 5.0 is released. Native **time series collections**, clustered indexing, and window functions make it easier to run IoT and financial analytics. **Live resharding** can happen without any downtime. MongoDB Shell `mongosh` is introduced and it deprecates `mongo` shell.