Use SQL Query Builder Knex.js with Node.js and Express.js
By: Lu Wang
Tags: Nodejs, Sql, Knexjs, Expressjs, Programming

I first learned about Knex.js from a technical interview, it was the first time I used another abstraction to interact with the database besides ORM Sequelize.js, which I learned during my study at the Grace Hopper program. Knex.js is an SQL query builder for JavaScript, a thin abstraction that sits on top of the database driver for relational databases including PostgreSQL, MySQL, SQLite2 and Oracle. It supports transactions, connection pooling, streaming queries, a thorough test suite, and the ability to run in the browser. Knex can be used in both Node.Js and the browser, limited only by WebSQL’s constraints.

I got curious and spent some time playing around with it. It was fun so I decided to write an article about it as a reference if anyone wants to try it out themselves.

Step 1. Install Knex.js and migration CLI

One of the best features of Knex is its robust migration support.

Install Knex.js globally to get the migration CLI. The migration CLI is bundled with the knex install, and is driven by the node_liftoff module.

Run the $ npm install knex -g command. This installs the package and the CLI.

Step 2. Database configuration

Run $ knex init to create a knexfile.js. The file will be pre-populated with some default database configurations. Knex migration tool uses knexfile.js to create migration files in the specified directory so you will need this file to run migration commands.

Open knexfile.js, enter the configurations for your database matching different development stages. It comes with 3 default stages, development, staging and production.

const pkg = require('./package.json')

const path = require('path')

const databaseName = pkg.name + (process.env.NODE_ENV === 'test' ? '-test' : '') module.exports = {

development: {

client: 'pg',

connection: `postgres://localhost:5432/${databaseName}`,

migrations: {

directory: path.join(__dirname, './server/db/models/migrations'),

}

},

production :{

client: 'pg',

connection: process.env.DATABASE_URL,

pool: {min: 2, max: 10},

migrations: {

directory: path.join(__dirname, './server/db/models/migrations'),

}

}

}

Make sure to specify the path of the migrations directory. This is where migration files will live.

Step 3. Migrate using migration CLI

Next, run the $ knex migrate:make migration_name command.

Now look inside the migrations directory you specified earlier, you should see a new file created with a series of numbers in the beginning of the file name similar to 20201202222750_user.js . They represent the year, date and time the file is created, and the migration name. Because order matters, a good rule of thumb is that your first migration should be the table not relying on other tables or data because migration runs in chronological order and this timestamp preserves that order.

Step 4. Define tables

In the migration file, there are two functions, the exports.up function and the exports.down function. Inside the exports.up function is where we specify changes to the database such as creating a table using knex.schema.createTable() , the exports.down function is where we can rollback or undo the changes exports.up made such as dropping the table using knex.schema.dropTable() . Here you can also insert, remove columns etc. See below the user table I created for my project:

exports.up = function (knex) {

return knex.schema.createTable('user', function (table) {

table.increments('id').primary()

table.string('firstName').notNullable()

table.string('lastName').notNullable()

table.string('email').notNullable()

table.string('password')

table.string('googleId')

table.timestamps(true, true)

})

} exports.down = function (knex) {

return knex.schema.dropTable('user')

}

Step 5. Update the database using migration CLI

Run the $ knex migrate:latest command to update the database. Check your database, you should see a newly created user table.

Similarly, to rollback to the last batch of migrations: run the $ knex migrate:rollback command, it should drop the table you just created and roll back to the previous state.

Step 6. Application configuration and session store

Create a db.js file and a database instance by requiring the configurations from the knexfile. We’ll be using this file throughout our application whenever we need to interact with the database.

const environment = process.env.NODE_ENV || 'development';

const configuration = require('../knexfile')[environment];

const db = require('knex')(configuration); module.exports = db

Let’s set up our main application file app.js . Here we will be using another node module connect-session-knex with express session for user sessions. Run $ npm install connect-session-knex , import the package as KnexStore , initialize a session store with an instance of KnexStore , the parameter should take the database instance for its knex property. A basic set up looks something like this but feel free to add more middleware and functions:

const express = require('express')

const session = require('express-session')

const KnexStore = require('connect-session-knex')(session)

const db = require('./db')

if (process.env.NODE_ENV !== 'production') require('dotenv').config()

const PORT = process.env.PORT || 8080

const sessionStore = new KnexStore({knex: db})

const app = express()

module.exports = app if (process.env.NODE_ENV === 'test') {

after('close the session store', () => sessionStore.stopExpiringSessions())

} app.use(

session({

secret: process.env.SESSION_SECRET || 'my best friend is Marley',

store: sessionStore,

resave: false,

saveUninitialized: false,

})

) app.listen(PORT, () => console.log(`Mixing it up on port ${PORT}`))

Step 7. Create some API routes

Now let’s write some API routes using Express Router. Knex.js query syntax is similar to raw SQL but we are actually writing JavaScript. Here is a useful cheat sheet. We will assign req.session.user here and encrypt user passwords using bcrypt (sorry, not within the scope of this article but this medium article is a good reference.)

Shown below, we created 2 post routes, /signup and /login . The main difference is that for /signup , the user doesn’t exist in the database yet so we need to use .insert() to create the user in the database. Knex’s .insert() doesn’t automatically return the newly created record, we need to include .returning() or pass a returning array parameter to get the inserted data back which inconveniently gives us an array. From there, you can either spread it or in our case, we destructured it because we need to assign req.session.user to user.

In the /login route, notice we use .first('*') instead of .select('*') to query the database from the user table? .select('*') resolves with an array of records while .first('*') resolves with the first record of the query which gives us an object.

const router = require('express').Router()

const db = require('../db')

const bcrypt = require('bcrypt') module.exports = router router.post('/signup', async (req, res, next) => {

try {

const {email, password} = req.body

const hash = await bcrypt.hashSync(password, 10)

const [user] = await db('user').insert({email, hash}).returning('*')

req.session.user = user

res.json(user)

} catch (err) {

if (err) {

res.status(401).send('User already exists')

} else {

next(err)

}

}

}) router.post('/login', async (req, res, next) => {

try {

const {email, password} = req.body

const user = await db('user').first('*').where({email})

if (!user) {

console.log('No such user found:', req.body.email)

res.status(401).send('Wrong username and/or password')

} else {

const validPass = await bcrypt.compare(password, user.hash)

if (validPass) {

req.session.user = user

res.json(user)

} else {

console.log('Incorrect password for user:', email)

res.status(401).send('Wrong username and/or password')

}

}

} catch (err) {

next(err)

}

})

That’s pretty much it. Now you can build your application on top of the boilerplate we just created using Knex.js, Node.js and Express.js in the backend.