Hugo + Node.js Koa App Connected to MongoDB
Tags: hugo, node, koa, mongodb
Published: 2019-09-12 00:09:55

This project demonstrates how to create a development environment utilizing a Docker stack of Nginx to serve the static website, Nodejs for the api applications, MongoDB for the api data and Traefik for reverse proxy. This stack is suitable for deployment to staging and production environments.

#### Prerequisites

These products will need to be installed in order to complete this tutorial.

* [Docker](https://docs.docker.com/)
* [Docker Compose](https://docs.docker.com/compose/)
* [Hugo](https://gohugo.io)
* [Node.js](https://nodejs.org/en/)

### Project Setup

Create a directory for the entire project, e.g., `hugo-koa-mongo`. All of the project files  will go in this folder. This folder will be referred to as the project root.

### Hugo Static Website Generator

To get started, open a terminal in the project root and create a new Hugo site in a folder named `www` as follows.

    hugo new site www


#### Add a Theme

There are numerous themes available at [themes.gohugo.io](https://themes.gohugo.io/) to choose from. You can install one of them if you prefer or use this example to install my [hugo-starter](https://github.com/jimfrenette/hugo-starter) theme. Download and extract the theme into the `www/themes/starter` folder, or use [Git](https://git-scm.com/) and clone the theme from it's git repository. For example,

    git init
    cd www
    git submodule add https://github.com/jimfrenette/hugo-starter.git themes/starter


After the theme has been installed, update the `config.toml` site configuration file to use the theme. For example,

##### config.toml

    theme = "starter"


Preview the site on the hugo dev server

    cd www

    hugo server


If the site loads, we're ready to move onto the next step.


### MongoDB

We will spin up a MongoDB Docker container for the `api` database. To demonstrate, we need to populate it with some data. For this, I have exported tables from the [Chinook](https://github.com/jimfrenette/chinook-database) database into `csv` files which can then be imported using `mongoimport`.

You can download the `csv` files within the [source code](https://github.com/jimfrenette/hugo-koa-mongo/tree/part-1/docker/entrypoint-initdb.d) for this project or complete the process on your own as follows.

1.  Download the [Chinook_Sqlite.sqlite](https://github.com/jimfrenette/chinook-database/blob/master/ChinookDatabase/DataSources/Chinook_Sqlite.sqlite) database.
 
2.  Open it with [DB Browser for SQLite](https://sqlitebrowser.org/)

3.  Export these tables to csv files:
    -   `Album.csv`
    -   `Artist.csv`
    -   `Genre.csv`
    -   `MediaType.csv`
    -   `Track.csv`

We're going to copy an entrypoint folder with a shell script and all the `csv` files we exported into the MongoDB Docker image in order to populate the database. In the project root, create a new folder named `docker` with an `entrypoint-initdb.d` folder as follows.

    mkdir -p docker/entrypoint-initdb.d


Copy or move all of the exported `csv` files into the `docker/entrypoint-initdb.d` folder.

In the `docker` folder, create a `mongo.dockerfile` that will create an image from mongo and copy the files in `entrypoint-initdb.d` into the `docker-entrypoint-initdb.d` folder of the new image.

##### mongo.dockerfile

    FROM mongo

    COPY ./entrypoint-initdb.d/* /docker-entrypoint-initdb.d/


In the `docker/entrypoint-initdb.d` folder, create this `importChinook.sh` script. This script will run when the image is created to populate MongoDB using the `csv` files.

##### importChinook.sh

    mongoimport --db chinook --collection Album --type csv -f AlbumId,Title,ArtistId --file /docker-entrypoint-initdb.d/Album.csv
    mongoimport --db chinook --collection Artist --type csv -f ArtistId,Name --file /docker-entrypoint-initdb.d/Artist.csv
    mongoimport --db chinook --collection Genre --type csv -f GenreId,Name --file /docker-entrypoint-initdb.d/Genre.csv
    mongoimport --db chinook --collection MediaType --type csv -f MediaTypeId,Name --file /docker-entrypoint-initdb.d/MediaType.csv
    mongoimport --db chinook --collection Track --type csv -f TrackId,Name,AlbumId,MediaTypeId,GenreId,Composer,Milliseconds,Bytes,UnitPrice --file /docker-entrypoint-initdb.d/Track.csvnpm i nodemon -D


### Node.js Koa API

The API is built using [Koa.js](http://koajs.com) Next generation web framework for [Node.js](https://nodejs.com/ "Nodejs"). This app will accept requests to `/api` and return `json` data from the MongoDB Docker container.

In the project root, create a folder named `api` with `src/server/chinook` and `src/server/routes` folders within. For example,

    mkdir -p api/src/server/{chinook,routes}


In the `api/src/server/routes` folder, create a `chinook` folder for the respective routes.

Project structure

<figure style="max-width:290px">
  <img 
 src="https://thepracticaldev.s3.amazonaws.com/i/c3x2j533c6gm0vbtwo03.png"
    alt="Project structure image with folders and file tree"
  />
</figure>

Initialize the Node.js app with `npm init` to create the `package.json` manifest file that will include all of the application dependency definitions and npm script commands for starting and building the app. For example,

    cd api

    npm init -y


> The following `npm i` or `npm install` commands are run from the `api` directory. When the install commands are run, the `package.json` file is updated with the respective package version info.

Install the MongoDB Node.js driver, `mongodb`.

    npm i mongodb


Install [mongoose](https://mongoosejs.com/) for a schema-based solution to model the application data. It also includes built-in type casting, validation, query building, business logic hooks and more.

    npm i mongoose


#### Models

In the `src/server/chinook` folder, create the data models. For example,

##### album.js

```javascript
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const AlbumSchema = new Schema({
    AlbumId: Number,
    Name: String,
    ArtistId: Number
},{ 
    collection: 'Album'
});

const chinook = mongoose.connection.useDb('chinook');

module.exports = chinook.model('Album', AlbumSchema);
```

##### artist.js

```javascript
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

/*
 notice there is no ID. That's because Mongoose will assign
 an ID by default to all schemas

 by default, Mongoose produces a collection name by passing the model name to
 the utils.toCollectionName method.
 This method pluralizes the name Artist to Artists.
 Set this option if you need a different name for your collection.
*/

const ArtistSchema = new Schema({
    ArtistId: Number,
    Name: String
},{ 
    collection: 'Artist'
});

const chinook = mongoose.connection.useDb('chinook');

module.exports = chinook.model('Artist', ArtistSchema);
```

##### track.js

```javascript
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const TrackSchema = new Schema({
    TrackId: Number,
    Name: String,
    AlbumId: Number,
    MediaTypeId: Number,
    GenreId: Number,
    Composer: String,
    Milliseconds: Number,
    Bytes: Number,
    UnitPrice: String
},{ 
    collection: 'Track'
});

const chinook = mongoose.connection.useDb('chinook');

module.exports = chinook.model('Track', TrackSchema);
```

#### Koa

Install `koa` and `koa-router`.

    npm i koa koa-router


#### Routes

In the `src/server/routes` folder, create the default `api` route. For example,

##### index.js

```javascript
const Router = require('koa-router');
const router = new Router();

router.get('/api/', async (ctx) => {
  ctx.body = {
    status: 'success',
    message: 'hello, world!'
  };
})

module.exports = router;
```

In the `src/server/routes/chinook` folder, create the `api/chinook` routes. For example,

##### album.js

```javascript
const Router = require('koa-router');

const connect = require('../../chinook/connect');

connect();

const router = new Router();
const BASE_URL = `/api/chinook`;

const Album = require('../../chinook/album');

function getAlbums(artist) {
    return new Promise((resolve, reject) => {
        var query = Album.find({ 'ArtistId': artist });
        query.exec((err, results) => {
            if (err) return handleError(err);
            resolve(results);
        });
    });
}

router.get(BASE_URL + '/albums/:artist', async (ctx) => {
    try {
        ctx.body = await getAlbums(ctx.params.artist);
    } catch (err) {
        console.log(err)
    }
})

module.exports = router;
```

##### artist.js

```javascript
const Router = require('koa-router');

const connect = require('../../chinook/connect');

connect();

const router = new Router();
const BASE_URL = `/api/chinook`;

const Artist = require('../../chinook/artist');

function getArtists() {
    return new Promise((resolve, reject) => {
        var query = Artist.find();
        query.exec((err, results) => {
            if (err) return handleError(err);
            resolve(results);
        });
    });
}

router.get(BASE_URL + '/artists', async (ctx) => {
    try {
        ctx.body = await getArtists();
    } catch (err) {
        console.log(err)
    }
})

module.exports = router;
```

##### track.js

```javascript
const Router = require('koa-router');

const connect = require('../../chinook/connect');

connect();

const router = new Router();
const BASE_URL = `/api/chinook`;

const Track = require('../../chinook/track');

function getTracks(album) {
    return new Promise((resolve, reject) => {
        var query = Track.find({ 'AlbumId': album });
        query.exec((err, results) => {
            if (err) return handleError(err);
            resolve(results);
        });
    });
}

router.get(BASE_URL + '/tracks/:album', async (ctx) => {
    try {
        ctx.body = await getTracks(ctx.params.album);
    } catch (err) {
        console.log(err)
    }
})

module.exports = router;
```

#### App Entrypoint

Create a `src/server/index.js` application entrypoint file as follows to initiate the app, routes and  configure the MongoDB connection.

##### index.js

```javascript
const Koa = require('koa');
const mongoose = require('mongoose');
const indexRoutes = require('./routes/index');
const artistRoutes = require('./routes/chinook/artist');
const albumRoutes = require('./routes/chinook/album');
const trackRoutes = require('./routes/chinook/track');

/**
 * Koa app */
const app = new Koa();
const PORT = process.env.PORT || 1337;
const server = app.listen(PORT, () => {
    console.log(`Server listening on port: ${PORT}`);
});

/**
 * MongoDB connection */
const connStr =  'mongodb://mongo:27017/default';
mongoose.connect(connStr);
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
    console.log('connected');
});

app.use(indexRoutes.routes());
app.use(artistRoutes.routes());
app.use(albumRoutes.routes());
app.use(trackRoutes.routes());

module.exports = server;
```

### [npm-run-script](https://docs.npmjs.com/cli/run-script) 

To build the respective `dev` or `prod` versions of the api server, in the `package.json` file under `scripts`, define the `dev` and `start` commands. These commands are executed when the Docker container is started based on the settings in the `docker-compose.yml`.

##### package.json

```javascript
...

"scripts": {
    "dev": "nodemon ./src/server/index.js",
    "start": "node ./src/server/index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
}
``` 

Since `nodemon` is needed to watch and rebuild our api app in `dev` mode, let's install it and save it as a dev dependency.

    npm i nodemon -D


### [Docker Compose](https://docs.docker.com/compose/)

To install the docker images, create our containers and start up our environment, add this `docker-compose.yml` file to the project root. Note that the volume paths map the project files to their paths within the Docker containers. For example, the Hugo publish directory `www/public` maps to the nginx server path for html, `/usr/share/nginx/html`.
 
```
version: "3"

services:

  app:
    image: node:alpine
    container_name: "${DEV_PROJECT_NAME}_node"
    user: "node"
    working_dir: /home/node/app
    labels:
      - 'traefik.backend=${DEV_PROJECT_NAME}_node'
      - 'traefik.frontend.rule=Host: ${DEV_PROJECT_HOST}; PathPrefix: /api'
    environment:
      - NODE_ENV=production
    volumes:
      - ./api:/home/node/app
      - ./api/node_modules:/home/node/node_modules
    expose:
      - "1337"
    # command: "node ./src/server/index.js"
    command: "npm run dev"
    depends_on:
      - mongo

  mongo:
    build:
      context: ./docker
      dockerfile: mongo.dockerfile
    container_name: "${DEV_PROJECT_NAME}_mongo"
    labels:
      - 'traefik.backend=${DEV_PROJECT_NAME}_mongo'
    ports:
      - "27017:27017"
    volumes:
      - mongodata:/data/db

  nginx:
    image: nginx
    container_name: "${DEV_PROJECT_NAME}_nginx"
    labels:
      - 'traefik.backend=${DEV_PROJECT_NAME}_nginx'
      - 'traefik.frontend.rule=Host: ${DEV_PROJECT_HOST}'
    volumes:
      - ./www/public:/usr/share/nginx/html

  traefik:
    image: traefik
    container_name: "${DEV_PROJECT_NAME}_traefik"
    command: -c /dev/null --docker --logLevel=INFO
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  mongodata:
```

I like to use an `.env` file to configure docker-compose variables. In the project root, create this `.env` file.

```
### PROJECT SETTINGS

DEV_PROJECT_NAME=hkm
DEV_PROJECT_HOST=localhost
```

In the project root, run `docker-compose up -d` which starts the containers in the background and leaves them running. The `-d` is for detached mode.

![nginx 403 webpage screen capture](https://thepracticaldev.s3.amazonaws.com/i/a8w4o3oc9c5y4mrblsmb.png)

If you get a 403 Forbidden nginx server message, it's because we didn't publish the Hugo site. 

```
cd www

hugo
```

To see the published Hugo site, restart the services in the project root using `docker-compose`. The `-d` switch is for disconnected mode, for example,

```
docker-compose down

docker-compose up -d
```

### API Test

Load [localhost/api/chinook/artists](http://localhost/api/chinook/artists) in a browser to see the json response.

> For troubleshooting, view the docker conatainer logs or spin up in connected mode by omitting the `-d` switch, e.g., `docker-compose up`.

* * *

All of [the source code](https://github.com/jimfrenette/hugo-koa-mongo/tree/part-1) for this tutorial is available on GitHub.

<h5><i class="icon-github"></i> <a href="https://github.com/jimfrenette/hugo-koa-mongo/tree/part-1">Source Code</a></h5>

Originally published at [jimfrenette.com/2019/05/hugo-plus-nodejs-koa-app-mongodb-connection](https://jimfrenette.com/2019/05/hugo-plus-nodejs-koa-app-mongodb-connection/)