Reducing Mental Fatigue: NestJS + ObjectionJS
Published: 2019-07-03 12:27:55

## ▶️ Introduction

Most of the article you’ll find me ranting on what helps me enjoy my job.

But you’ll also figure out how to start using Objection with Nest and what they are all about. I won’t explain in detail how Nest or Objection work. I think those two have wonderful documentation which is fun and worth being explored, BUT I’ll show you how one can start using them together 😌 

## ☝️ Prerequisites

You need to have PostgreSQL available locally. This can be achieved in various ways but I would suggest two approaches:

1. You can install docker [[Windows](https://docs.docker.com/docker-for-windows/install/), [macOS](https://docs.docker.com/docker-for-mac/install/), [Ubuntu](https://linuxize.com/post/how-to-install-and-use-docker-on-ubuntu-18-04/)] and execute a `package.json`  script  described later in the article . Which essentially runs PostgreSQL in a docker container and destroys it once you hit `ctrl+c` in your terminal

2. You can install it directly on your machine (though I’d recommend option #1 if you don’t have it already installed)

## 💆 Mental fatigue

Lot's of programmers nowadays are dealing with mental fatigue in software development. And it's not only because of new tools, approaches, and paradigms that pop up every minute.

With every new software development project (let's assume back-end app, cause I'm more of a back-end person) a programmer or a team should decide on a variety of things:

- Web framework
- Project structure
- Linting rules, code formatting (based on Google, Airbnb, Microsoft or homegrown conventions)
- Data storage (SQL, NoSQL)
- Deployment platform (Amazon, Google Cloud, Azure, Netlify, etc.)
- CI/CD
- Testing tools and strategies
- Documentation
- And so on and so forth ...
 
As you can see this list can grow on and on. A whole lot of high-level decisions involved in this process, not talking about millions of small ones.

Every decision made [depletes our mental energy](https://www.scientificamerican.com/article/tough-choices-how-making) even if it's trivial. After lots of minor decisions, we are less capable to make a good major one.

A tools landscape that we have right now in a JavaScript world is a blessing and a curse at the same time. On one side, it causes us to make more decisions, on the other side there are gems that come with lot's of good decisions made for us, decisions trusted by thousands of developers.

So why can't we use this opportunity to reduce our mental fatigue, reduce the number of decisions we make by adopting well-proven opinions?

By choosing the right tools we have a chance to develop a project which is easy to reason about and onboard new people on.

Almost any back-end project gets built on top of a web framework and ORM of some kind that heavily influence future project architecture.

We’ll have a look at the way they can simplify mental models we built in mind and reduce the number of decisions we usually have to make. We are going to develop a simple (but enough to demonstrate the powers of Nest and Objection) back-end for a note-taking app.

## 🕸️ Web framework

There are plenty of web frameworks available in the Node.js land: *Express, Hapi, Koa, Fastify, Restify,* etc. They are flexible and time-tested folks that allow you to structure a project in many different ways.

So you need to decide on how you want to organize routes, handlers, views, authentication, services, repositories, etc. This gives you a lot of freedom but comes with a cost. You need to make plenty of decisions to properly architect the app, and the way project is organized will be different in any other project that gets built using the same framework, because developers of a new project made their decisions in a slightly different way.

You have to start over again and grasp the way this framework is used in that particular project. You’re loosing that feeling of familiarity and awareness you developed on the previous project, or in other words, the level of framework knowledge conversion is not that high.

For me personally, those frameworks are missing one important thing (though I think they are very powerful) - a shared conceptual base, on top of which you can start growing an actual business logic, this base would repeat from project to project and allow to quickly familiarize new developers with the codebase.

Such conceptual base allows the increase of the framework’s knowledge conversion and reduce the amount of mental effort needed to start using it.

What do I mean by **conceptual base**? It is a minimal set of concepts or building blocks that framework gives you. And if those building blocks get aligned well with what you need to develop, it becomes easy to reason about the project and to communicate its different parts to other team members (to both seasoned developers and newcomers).

For me, such a framework is [Nest](https://nestjs.com)! It’s written in Typescript and has good, concise documentation. So for those who don’t like to read lengthy manuals (I don’t), this documentation gives just enough information and examples to do the job - no more, no less.

Nest has modules systems heavily inspired by Angular, so Angular developers should be quite comfortable reading Nest code. Angular and Nest is usually a good combination because their conceptual bases have a high intersection, and you can transfer some of your Angular knowledge to Nest.

I don’t want to repeat the docs, and encourage you to have a look on your own.

Though I’ll describe Nest’s main building blocks:

- [Guard](https://docs.nestjs.com/guards) - protects system from unauthenticated/unauthorized access
- [Interceptor](https://docs.nestjs.com/interceptors) - intercepts incoming requests or outgoing responses
- [Controller](https://docs.nestjs.com/controllers) - processes the requests
- [Provider](https://docs.nestjs.com/providers) - this is basically a service that is dedicated to some set of tasks and can be injected into any other thing from this list thanks to Nest built-in dependency injection capabilities
- [Pipe](https://docs.nestjs.com/pipes) - transforms/validates incoming request body
- [Middleware](https://docs.nestjs.com/middleware) - the purpose of the middleware is to intercept the request, execute some logic and pass the control flow to the next middleware
- [Module](https://docs.nestjs.com/modules) - this thing helps to organize your application structure and it has the same purpose as Angular modules do

Also, I suggest reading on this series of articles on [Nest.js Step by Step](https://www.bhaidar.dev/2019/06/nestjs-step-by-step-part1).

> You can ask me “Why do I need other stuff listed above if I have middlewares?”.
> 
> Middlewares are too generic whereas Guards, Intercepts, etc. are dedicated to one particular task. So by hearing the word “Guard”, you already know what it is responsible for.
> 
> You might need middlewares if you want to implement something beyond those concepts listed here.

💥 **Big Bang!**
For the sake of brevity, I won’t be describing every file of our future project, but rather will be highlighting key concepts along the way.

We’re starting with the Nest application, which has all the plumbings but no database.
This way it’ll be easy to talk about Nest stuff and then gradually move to Objection.
As I already said we are going to develop a toy notes API. Our notes can have theme and tags.

This is what our app structure looks like initially:

![Initial app structure](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199673077_Image.jpg)


Go ahead and investigate [the code we have so far](https://github.com/rychkog/nest-objection-article/tree/31d178d4991f5fce3e76c90c39e83981b0d6307c) (*the codebase is on the* [*initial commit*](https://github.com/rychkog/nest-objection-article/commit/31d178d4991f5fce3e76c90c39e83981b0d6307c) *at the moment*). We're gonna start building on top of it.
 
Just by looking at names we have in the codebase it becomes immediately clear the purpose and responsibilities of different classes.

Let’s have a look at `notes` folder in more details (cause `tags` and `themes` works exactly in the same vein).

The first thing is `NotesModule`

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199752208_Image+10.jpg)


In `NotesModule` we’ve registered `NotesService`. Here how it looks

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199798161_Image+12.jpg)


`NotesService` is used by `NotesController` and is injected by Nest once it discovers that the latter is dependent on the former.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199822090_Image+9.jpg)


You might have noticed that `NotesService` (as well as other services) is just a stub at the moment and does nothing. We’re going to fix that soon, after a small conversation about ORMs.

## 🦖 ORM

Historically, the purpose of ORMs was to remove [object-relational impedance mismatch](https://en.wikipedia.org/wiki/Object-relational_impedance_mismatch). They do this by abstracting out RDBMS and relational concepts as much as possible, they are especially good at hiding SQL from you and forcing you to use their DSL, which still sucks because it is a prominent example of a [leaky abstraction](https://en.wikipedia.org/wiki/Leaky_abstraction).

I remember lot’s of situations when I was struggling with such DSL for hours, trying to mimic a query which I’d already written in SQL (and spent minutes on this). Even if we’ve managed to write a proper DSL it still might be converted into monstrous (not always performant) SQL you have no control over.

The true power of relational databases comes with SQL and its declarative expressiveness. In reality, the majority of my colleagues are quite good with the RDBMS concepts. It’s comfortable for them to think in terms of SQL queries and more often than not developers have an intuitive understanding of how DB record should be represented as an object (dictionary, map, you name it) in their language of choice.

It simply makes no sense to hide SQL from developers in ORMs because you still have to know it for fetching at least something from the DB, but apart from that you need to enable a compiler in your head that converts DSL to SQL in order to understand what kind of query will be generated eventually and whether it’ll give you what you want.

It’s double work that puts extra pressure on your brain which is trying to keep and reconcile a million of other little things about the project you are working on.

If you are already proficient with SQL, why do you need to learn another language (DSL) to fetch/update stuff from/in the database?
Wouldn’t it be better for ORMs to implement an API that is as close to SQL as possible, allowing to transfer existing developer’s SQL knowledge to that API and flattening learning curve?
Such API would take advantage over the language features like auto-completion and static code analysis while still being close to generated SQL.

The solutions like Hibernate, TypeORM and similar ones are overloaded, heavyweight and over-complicated in my opinion.

And here is where [Objection](https://vincit.github.io/objection.js) comes in. Comparing to other ORMs it doesn’t try to put SQL and relational model behind the curtains. Here is how Objection developers describe their product:

> Objection.js is an [ORM](https://en.wikipedia.org/wiki/Object-relational_mapping) for [Node.js](https://nodejs.org/) that aims to stay out of your way and make it as easy as possible to use the full power of SQL and the underlying database engine while still making the common stuff easy and enjoyable.

## 🍽️ Integrating Objection with Nest

**TLDR;** If you just need to know what should be done to have Objection support in Nest, here is [the diff](https://github.com/rychkog/nest-objection-article/commit/5644b3295c8d5064606eff5e7c9296f90f3f2143) which shows changes that should be applied on top of our initial commit.

1️⃣ **Installing required dependencies**
`npm i @types/dotenv dotenv objection knex pg`


- **dotenv** populates `process.env` with environment variables defined in the `.env` file
- **objection** - the ORM
- **knex** is a SQL query builder that Objection uses under the hood. It also provides *migrations* and *data-seeding* support (we’ll talk about this a bit later)
- **pg** is a client for PostgreSQL database.

2️⃣ **Relational model**
Next step is to define our relation model (for now just get comfortable with the tables we are about to build)

![Note app relational model](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560075747688_postgres.png)

- Notes might have a theme
- Notes can have multiple tags
- One tag can belong to multiple notes

> **knex_migrations** and **knex_migrations_lock** are tables created and managed by Knex.
> They are not relevant for our data model.

3️⃣ **Extending package.json with helper scripts**
Before we start creating the migrations, let’s add a couple of commands to our `package.json`
No worries, their purpose will become clear in later sections.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199864056_Image+13.jpg)

4️⃣ **Knexfile**
In the `package.json` excerpt above you might be noticed `--knexfile knexfile.ts`. This is an argument that points to Knex configuration file, so let’s create it at the root of the project.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199900514_Image+5.jpg)

**knexSnakeCaseMappers** converts *camelCase* names in code to *snake_case* names in the database.
So in our database model, we have a **themes** table with **font_family** column. In order to update this column from the code, you can refer to it using **fontFamily** and mappers will do the job by transforming **font_family → fontFamily** and vice versa automatically.

The *purpose of migrations* is to create a database schema and subsequent changes to that schema that might come up over time. This allows versioning your database and rollback schema to its previous state when needed.

*Seeds* are useful in the development environment when you need to populate your database with some data.

**migration.stub** and **seed.stub** are *template files* which Knex uses to generate our migrations and seeds. Put those under the `database` folder as specified in the config

![migration.stub](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199940403_Image+6.jpg)

![seed.stub](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560199985691_Image+14.jpg)

5️⃣ **Migrations**
Now when we have `knexfile.ts` created we can start using Knex commands we’ve added previously to the `package.json`.

- `npm run migrate:make CreateTags`
- `npm run migrate:make CreateThemes` 
- `npm run migrate:make CreateNotes` 
- `npm run migrate:make CreateNoteTags` 

Those will generate migration files under the `database/migrations` folder using our `migration.stub`.

It’s time to define our tables. Let’s do it together for `CreateNotes` migration for others please have a look at the [final solution](https://github.com/rychkog/nest-objection-article/tree/master/src/database/migrations).

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200028805_Image+3.jpg)

6️⃣ **Connect our models with Objection**
In order to reflect relational tables in our code, we need to create a bunch of appropriate classes called *models*. For now, they are just plain Typescript classes located under `database/models` directory, so let’s sprinkle some *Objection* on them.

**BaseModel** ( `base.model.ts` )

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200062684_Image+2.jpg)

**TagModel** ( `tag.model.ts` )

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200081764_Image+15.jpg)

**NoteTagModel** ( `note-tag.model.ts` )

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200097152_Image+7.jpg)

**ThemeModel** ( `theme.model.ts` )

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200134180_Image+16.jpg)


7️⃣ **Mapping relations**
Especially interesting for us is how Objection handles relations between tables and the way we can express them in code.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200146666_Image+8.jpg)


8️⃣ **Connecting models to database and database.module.ts**

Each model class can be used to perform various SQL queries, but for that, we need to wire those classes with a Knex database connection.

Once they are wired we can expose those classes as injectable service to other modules.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200173729_Image+4.jpg)

DatabaseModule needs to be registered under the main ApplicationModule, so all its exported services are available to other modules.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200194065_Image+1.jpg)


9️⃣ **Implementing.service.ts files**
To start manipulating data we have in the database, we need to implement methods defined in the **.service.ts** files.

Each service relies on the model class(-es) we’ve exposed through the module’s *exports* above.
Here is NotesService implementation (`notes.service.ts`):

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200245958_Image+11.jpg)


As you can see the `.query()`  method is a gateway for building [rich queries](https://vincit.github.io/objection.js/guide/query-examples.html#find-queries).
In the example above we also have a transaction example, so any error thrown in the `transaction` callback will cause database changes triggered inside of that callback to roll back.

🔟 **Loading Note relations**
Let’s have a look at `findOne` method in [NotesController](https://github.com/rychkog/nest-objection-article/blob/master/src/notes/notes.controller.ts#L15):

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200685418_findOne.png)

The notable change is `$loadRelated` invocation. Here we’re asking Objection to load relations for this particular note:

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200709346_findOne-zoom.png)

`tags` and `theme` are names of the relations defined in the **NoteModel** class. This is how Objection knows how to fetch them.

All fetched relations get transformed into appropriate model instances.
Once fetched, Objection will create **tags** and **theme** fields for this particular note instance.

So all the relations get loaded only on demand by default.

In case you want to fetch lots of objects with loaded relations there is another way [you can use](https://github.com/rychkog/nest-objection-article/blob/master/src/notes/notes.controller.ts#L11):

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200725807_findAll.png)

In here, once all notes are loaded - Objection will loads `tags` relation for all of them.

1️⃣ 1️⃣ **Seeds**
Now we’re ready to generate seed files:

- `npm run seed:make 01-Tags`
- `npm run seed:make 02-Themes` 
- `npm run seed:make 03-Notes` 
- `npm run seed:make 04-NoteTags`

Seeds get generated under the `database/seeds` folder using our `seed.stub`.

Seed files get executed by Knex in order, so we have to ensure that it is correct. This is the reason we’ve prefixed seed files with numbers: we want to have tags created before note-tags because the latter depends on the former.

Let’s have a look at  `02-Themes.ts`  seed implementation

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560092333289_02-Themes.png)

1️⃣ 2️⃣ **dotenv**
[dotenv](https://github.com/motdotla/dotenv) is a library that loads environment variables from **.env** file into `process.env`
We’re going to utilize it for defining `DATABASE_URL` env var, which then will be used throughout the app including migrations and seed scripts.

All you need is to 

1. create a **.env** file at the root of the app and put there this single line
    DATABASE_URL=postgres://postgres:docker@localhost:5432/postgres

This connection string gets constructed based on the command we have in `package.json`. Postgres uses `postgres` as a name for a default user and database.

![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200837212_run-docker.png)

2. Add **dotenv** import at the very top of **knexfile.ts** and **main.ts**
![](https://paper-attachments.dropbox.com/s_1F921D6055F211537012522AC3AED06A165C5605936BC0BA49129723816F6EFE_1560200943290_dotenv.png)

1️⃣ 3️⃣ **Running PostgreSQL**
At this point we need to start our PostgreSQL instance:

    `npm run run:pg-docker`

Then create the schema (by executing migrations) and populate it with data (by executing seeds):

    `npm run migrate && npm run seed`


## 🚀 Playing with the app

Now you should have a fully working Nest application with the Objection support.

You can run it using 

    `npm run start`

**README.md** contains example http requests (using ***curl***) which you can modify and execute against the server.

And we’re done 🎉 

## ✍️ Summary

In the article I shared my thoughts on mental fatigue and that with the right tools it can be reduced by utilizing clear and intuitive concepts that help to communicate and share knowledge with others.

Nest does this by providing a conceptual base, which is great not only for reasoning about the project but also for communicating the way it works to other developers.

Objection gives you a framework that allows thinking in SQL terms, and avoid wasting time debugging esoteric DSLs. I would call it the “ORM without a pain”.

I hope you’ve enjoyed the article and get some understanding of how to start using Objection with Nest.

You can find the [full project](https://github.com/rychkog/nest-objection-article) on my github.

*This post written by Georgii Rychko of [ThisDot](https://www.thisdot.co/).*

*Need JavaScript consulting, mentoring, or training help? Check out our list of services at [This Dot Labs](thisdot.co).*