Populating MongoDB schema
Tags: mongodb, webdev, javascript, node
Published: 2021-07-29 09:50:50

We'll be creating only the backend of the application for understanding how to post data in to a MongoDB schema that references another schema.

### TL;DR

How can we get the following JSON data with the user schema referencing the todo schema?

```json
{
  "todo": [
    {
      "_id": "61023642610b8d4ce4f56f81",
      "title": "test-title-1",
      "description": "test-description-1",
      "__v": 0
    },
    {
      "_id": "6102365b610b8d4ce4f56f84",
      "title": "test-title-2",
      "description": "test-description-2",
      "__v": 0
    }
  ],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}
```
#### User Model

{% gist https://gist.github.com/mritunjaysaha/49ca2895c51cfb49cdde0b312fddb049 %}

#### Todo Model

{% gist https://gist.github.com/mritunjaysaha/596f563e10115ab0648b269519607660 %}

Here, the `User` schema is referencing the `Todo` schema. To get the JSON data with the `todo` data we need to do the following

- While creating the todo data we need to add the `ObjectId` of the `new todo` to the `todo` array of the `User`. At this stage the data will look something like this.

```json
{
  "todo": ["61023642610b8d4ce4f56f81", "6102365b610b8d4ce4f56f84"],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}
```

- To get the data of the todo created by the user we will reference the `Todo` table using the `populate` method which will get the data of the `todo`.

It is like joining two tables in `SQL` where `User` table references the `Todo` table using the `primary key` of the `Todo table`. Here, the `primary key` of the `Todo table` is the `ObjectId`.

[GitHub repo](https://github.com/mritunjaysaha/mongodb-schema-populate-blog)

### Initialize project

- Initialize our backend using `npm` and install necessary packages.
- Set up a MongoDB database.
- Set up server using `Node` and `Express`.
- Create a database schema to define a `Todo`.
- Set up API routes to `create` user and todo and `read` user and todo.
- Testing our `API` routes using [Insomnia](https://insomnia.rest/).

### Install

- `VS Code` or any other editor
- Latest version of `Node.js`
- `Insomnia` or Postman
- `Prettier` VS code extension to format the code

### 1. Initializing our project

Create a new folder and name it anything that you like and then open the folder in VS code and run the following code from the command prompt.

```bash
npm init -y
```

After running this command you will find a `package.json` if the folder.

### 2. Setting up package.json

#### i. Install the following dependencies

Run the following commands in the terminal to install the dependencies

```bash
npm i cors dotenv express mongoose
```

`cors`: allows cross-origin api calls
`dotenv`: needed to access data from `.env` files
`express`: web application framework for node.js
`mongoose`: It is needed to define the database schema and connecting to `mongoDB`

#### ii. Install the following development dependencies

Now install the following development dependencies, `-D` is used to install the development dependencies.

```bash
npm i -D nodemon
```

After installing the dependencies the `package.json` folder should look as follows.

```json
// package.json
{
  "name": "mongodb-schema-populate-blog",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/mritunjaysaha/mongodb-schema-populate-blog.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog/issues"
  },
  "homepage": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog#readme",
  "dependencies": {
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.3"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}
```

#### iii. change the `main` entry point to `server.js`

Now, create a `server.js` file and a `.env`. The `server.js` file will be the entry point of the server and the `.env` file will contain the `MONGO_URI`. We also have to make the following changes in the `package.json`

```json
//package.json
{
  "name": "mongodb-schema-populate-blog",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/mritunjaysaha/mongodb-schema-populate-blog.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog/issues"
  },
  "homepage": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog#readme",
  "dependencies": {
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.3"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}
```

Now, create the following folders

- `config`: Inside the `config` folder, create a file named `db.js`. This file will contain the required code for connecting to the `MongoDB` database.

- `controllers`: The `controllers` folder will contain the files which will have the methods for the end points to communicate with the database.

- `models`: The `models` folder, will contain the files which will define the `MongoDB schema`

- `routers`: The `routers` folder will contain the files with the `endpoints`.

At this stage the file structure should look as follows

```
.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json
```

#### iv. Change the `scripts` to the following

```json
"scripts": {
  "start":"node server.js",
  "dev":"nodemon server.js"
}

```

The `package.json` file should look as follows

```json
{
  "name": "mern-todo",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js", //added
    "dev": "nodemon server.js" //added
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "config": "^3.3.6",
    "cors": "^2.8.5",
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.2"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}
```

#### v. Setting up server

We will do the following to setup the server

- Import `express`
- Initialize our app using `express()`
- Set up a `get` method for the endpoint `http://localhost:8000` using `app.get()`
- Set the `PORT` to `8000` for our server to run
- Have our app to listen to `PORT` using `app.listen()`

```
.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js <-- we are here
├── package-lock.json
└── package.json
```

The code will look as follows

{% gist https://gist.github.com/mritunjaysaha/c0d14f8bddff905e3fdddb95d8018343 %}

And start the server using `nodemon` using the following code. Make sure you are running the following command from the project directory.

```bash
npm run dev
```

If the server has started successfully then it should show the following message in the terminal

```
[nodemon] 2.0.11
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node server.js`
server is running on http://localhost:8000
```

You can also open `http://localhost:8000` on your browser.

#### vi. Getting the `MONGO URI` from [mongoDB](https://cloud.mongodb.com)

To connect to the database we will need the link for the `mongoDB` collection.

1. Log in to [mongoDB](https://cloud.mongodb.com)
2. Create a new project
3. Build a cluster
4. Select cloud provider
5. Create cluster
6. wait for the cluster to be created.
7. Click on connect
8. click on `allow access from anywhere`. Then `Add IP address`

   ![add connection](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w49xfzzwlmuu2r1tmwc2.png)

9. Create a database user. You'll need the `username` and `password` for the `MongoDB URI`.
10. Click on the `Choose a connection method`
11. Click on `Connect your application`
12. Select the following driver and version

    ![connect cluster](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dzdi5nff5kjvmkyhn4p6.png)

13. Copy the `mongodb+srv` and paste it in the `.env` file

#### vii. Setting up `.env` file

```
//.env
MONGO_URI = mongodb+srv://<username>:<password>@cluster0.owmij.mongodb.net
```

Replace the `<username>` and `<password>` with your database username and password which you will set in step 9.

#### viii. Connecting to database

```
.
├── config
│   └── db.js <-- we are here
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json
```

Now, open the `db.js` file which is in the `config` folder and add the following changes.

- Import `mongoose`
- Import `MONGO_URI` from `.env`
- Define the `connectDB` methof for connecting to the database
- Export the `connectDB` method to be called in `server.js`

{% gist https://gist.github.com/mritunjaysaha/133d0825109012eda55c014a566bd537 %}

Add the following changes in the `server.js` file.

- Import `dotenv`
- Import `connectDB` method from `config/db.js`
- Call the `connectDB` method.

Let us make the the following changes in `server.js`

{% gist https://gist.github.com/mritunjaysaha/0d8af33025f9a37193bdd3acd22db644 %}

Save the changes it will restart the server or use the command `npm run dev`. The terminal should show a message of `MongoDB is connected` which we have added in the `db.js` under the try block.

#### ix. Defining database schema

Create a `todo.js` file in the models folder. We will define the database schema in this file.

```
.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js <-- we are here
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json
```

- Import `mongoose`
- Create a `Schema` called `TodoSchema`
- We will add two fields for our todo; `title` and `description`
- Type of `title` will be `String` and it is a mandatory field
- Type of `description` will be `String` and it is not a mandatory field
- Export the model

The code will look as follows

{% gist https://gist.github.com/mritunjaysaha/596f563e10115ab0648b269519607660 %}

Create a schema for the user using the above steps.

After making the changes, the user model will look something like this

{% gist https://gist.github.com/mritunjaysaha/49ca2895c51cfb49cdde0b312fddb049 %}

#### x. Defining the controllers

```
.
├── config
│   └── db.js
├── controllers
│   └── user.js <-- we are here
├── models

│   └── todo.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json
```

- Import `Todo` and `User` schemas
- Define `createUser` method will create a new user
- Define `createTodo` method will do the following
  - create a new todo
  - save the todo
  - use the `userId` to find the user
  - update the `todo` array with the `ObjectId` of the new todo
- Define `getUser` to get the user details. The output of this method we can see that `todo` consists of some random value which is the `ObjectId` of the `todo` that the user has created. We cannot figure out what the todo contains.

```json
{
  "todo": ["61023642610b8d4ce4f56f81", "6102365b610b8d4ce4f56f84"],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}
```

- Define `getAllTodo` method we will use the `userId` to find the user and then use the `populate` method to reference the `todo` with the `ObjectId` from the `Todo` table. The `exec` method is used to check for errors and return the populated data.

```json
{
  "todo": [
    {
      "_id": "61023642610b8d4ce4f56f81",
      "title": "test-title-1",
      "description": "test-description-1",
      "__v": 0
    },
    {
      "_id": "6102365b610b8d4ce4f56f84",
      "title": "test-title-2",
      "description": "test-description-2",
      "__v": 0
    }
  ],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}
```

{% gist https://gist.github.com/mritunjaysaha/d98225ef8bd99825228c7e637b4c1088 %}

#### xi. Defining the end points

```
.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models

│   └── todo.js
├── node_modules
├── routes
│   └── user.js <-- we are here
├── .env
├── server.js
├── package-lock.json
└── package.json
```

We will define the end points to `create` users and todo and to `read` them.

- Import `express`
- Import all the methods from `controllers`
- Initialize `router`
- Define a `POST` method to `create` a user
- Define a `POST` method to `create` a todo and save it in the user
- Define a `GET` method to `read` user data
- Define a `GET` method to `read` user data and todo data

After making the above changes the code will look something like this

{% gist https://gist.github.com/mritunjaysaha/7c80abdcaf0379f475d97e7a5a89125a %}

#### xii. Adding the routes end points in the `server.js`

```
.
├── config
│   └── db.js
├── controllers
│   └── todo.js
├── models
│   └── todo.js
├── node_modules
├── routes
│   └── todo.js
├── .env
├── server.js <-- we are here
├── package-lock.json
└── package.json
```

The final part of completing the backend is to add the endpoints to the `server.js` file.

- Import `routes/todo.js`
- Add the routes endpoints to the middleware

{% gist https://gist.github.com/mritunjaysaha/d96ab1421fff8f8898d47c09c2f8c758 %}

### 3 Testing the end points using `Insomnia`

- Create a user

We will send a `POST` request to `http://localhost:8000/api/user`

![create user](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s4s8vgiz7uud5usrbpcg.png)

- Create some todo

We will send a `POST` request to `http://localhost:8000/api/user/todo/:userId`

copy the `_id` from the response of the `create a user request`

![create todo 1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0dg408nqt6zpagjc6y3y.png)

![create todo 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2or6buuctpn89g8v69fp.png)

- Read the user data

We will send a `GET` request to `http://localhost:8000/api/user/:userId`

![user data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rqzs56doi9jiul2tb7m5.png)

- Read the populated user data

We will send a `POST` request to `http://localhost:8000/api/user/todo/:userId`

![populated data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6sucnsy8ppr282whtrfc.png)

You can check the code in [GitHub](https://github.com/mritunjaysaha/mongodb-schema-populate-blog)