Data Migrations in NestJS
Tags: nestjs, postgres, typeorm
Published: 2022-04-08 10:50:55

---
title: Data Migrations in NestJS 
published: true
description: Keeping data up to date when changing your database structure
tags: nestjs, postgres, typeorm
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rp38c6afcqw34kw5qr2l.jpg
---

_(Header image by [Patrick Coe on Flickr](https://flickr.com/photos/patrickcoe/2449353023/in/photolist-4Jrzjx-diqkR7-iDqAtE-7NBZ7f-bjZUxQ-2hRwpzH-AATpz-2QFNaJ-2mLgxD5-2mH8UL2-2mAJJFU-2mt6jEf-2ncBeqt-2mPUVjN-2hmpc3w-AfShjK-hm7PME-AATon-QS1wqo-utncsK-YL2A53-5ByZaQ-7ANpKs-mULkWr-py9J8-2n8W133-NgGeJZ-2nbi8wS-2m7Bi1L-2mYUF2m-2n9Yzry-LuxYj-2msSpMy-2naUmap-2mgQy9f-2n8HKLz-2mP6eFF-2mPz8ud-2iDKFBf-2n8qj8o-2mj3axq-2kyP7W6-Q4Aya5-2n2VtcZ-2ncqtLZ-2mAmtKB-bpMLcN-G61CkE-hy1awa-71RPB1/))_

Coming to NestJS from a Ruby on Rails Background, there's a lot to like, and a lot that's familiar. There's also some stuff I miss, but I have a great time trying to replicate those missing features.

Take, for example, data migrations. When working on a live project, it's common that you'll want to change aspects of the data structure. A simple example might be if you have a `books` database with a single `author`.

Then, one day, your client informs you that some books have multiple authors, and has asked if you can make it possible for a `book` to have multiple `author`s. 

For a non-technical client, this seems like a simple request, but for us devs, we know it's not that simple. We'll have to set up a new relation (probably through a join table) and transfer that data. Setting up the new relation is easy enough, but how do we move that data from the old relation to the old relation?

A naïve solution might be to make that change directly in the database migration, but, dear reader, here be dragons. Writing our data migrations in this way means that they stay around forever. If something in our codebase changes and a new developer runs the whole migration suite, this could cause the migrations to fail. The full pitfalls of this approach are outlined in [this great blog post by Elle Meredith at Thoughtbot](https://thoughtbot.com/blog/data-migrations-in-rails).

Another solution might be to add automated data migrations, that run in a similar way to database migrations, running at deploy time, and updating a migrations table to let the codebase know when a migration has run.

In the past, we've used [data-migrate](https://github.com/ilyakatz/data-migrate) for this, but this comes with pitfalls. What happens if the deploy fails? We can often end up with strange errors and changes in state, so it's nicer to have a bit more control.

The approach I've taken to prefer recently is having small, short-lived scripts in the codebase that we run manually after deploying. In the example of changing our `books` table to have multiple `authors`, we would do the following:

- Add a new `authors` relation
- Write a data migration to move the `author` relation to the `authors` relation
- Deploy the code
- Run the migration in the production environment
- Remove the old relation
- Deploy that code

In Rails, we've added a little generator that generates a file in `db/data_migrate` that we can then run with `rails runner`. The beauty of `rails runner` is that any code that is run with that command has access to the whole of the Rails environment, so we have access to the database and the rest of the application.

In NestJS, we don't have the same luxury. However, I have found a neat solution that works just as well. For the uninitiated, the main entrypoint to a Nest app is `main.ts`. This is where the application is loaded, binded to Express and booted:

```typescript
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}
bootstrap();
```

We can steal this approach for our data migrations, which could then look like the following.

```typescript
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function migrate() {
  const application = await NestFactory.createApplicationContext(AppModule);

  // Add migration code here

  await application.close();
  process.exit(0);
}
migrate();
```

Notice the use of `NestFactory.createApplicationContext`, this gives us a reference to the application, so we can fetch things like services from the application like so:

```typescript
const application = await NestFactory.createApplicationContext(AppModule);

const booksService = application.get(BooksService);
```

In our `books` example above, we could then do something like this:

```typescript
async function migrate() {
  const application = await NestFactory.createApplicationContext(AppModule);

  const bookService = application.get(BooksService);
  const books = await bookService.getAll();

  console.log(`Updating ${books.length} books`);

  for (const book of books) {
    book.authors = [book.author];
    bookService.save(book);
  }

  console.log('Done!');

  await application.close();
  process.exit(0);
}
migrate();
```

We can then save that file to a known location, and run it with `ts-node`:

```typescript
ts-node --transpile-only path/to/your/migration.ts
```

(Here we're adding `transpile-only` to prevent type information from being produced, which has a lower memory footprint and prevents any weirdness in production)

So, this is all great, but copying and pasting boilerplate code is boring. How about we add a generator to generate our data migrations in a known place, with some guidance on what to do next? This could look something like this:

```typescript
import * as fs from 'fs';
import { format } from 'date-fns';

async function generateDataMigration() {
  const timestamp = format(new Date(), 'yyyymmddHMS');
  const name = process.argv[2]
    .replace(/[A-Z]/g, (m) => '-' + m.toLowerCase())
    .substring(1);

  const filename = `${timestamp}-${name}`;
  const path = __dirname + `/../src/db/data/${filename}.ts`;

  const body = `// Run me with \`ts-node --transpile-only ./src/db/data/${filename}.ts\`
import { NestFactory } from '@nestjs/core';
import { AppModule } from '../../app.module';

async function migrate() {
  const application = await NestFactory.createApplicationContext(AppModule);

  // Add migration code here

  await application.close();
  process.exit(0);
}

migrate();`;

  fs.writeFile(path, body, { flag: 'a+' }, function (err) {
    if (err) {
      return console.error(err);
    }
    console.log(`Data migration ${filename} created!`);
  });
}

generateDataMigration();
```

This generates a boilerplate migration in a known place with a timestamp. We can then run it in development with (assuming your script is at ` ./utils/generate-data-migration.ts`):

```typescript
ts-node ./utils/generate-data-migration.ts
```

You could even add a `data:migrate:generate` to your package.json like so:

```json
"data:migrate:generate": "ts-node ./util/generate-data-migration.ts",
```

And that's it! Now you can safely and carefully create data migrations on your project whenever you need to!

I'm still pretty new to Nest / Typescript, so please let me know in the comments if this is not the best approach, or share any alternative ideas. I'm all for learning.