A short tour of ExpressJS implementation (Part 1)
Tags: webdev, javascript, node, express
Published: 2023-09-02 09:47:37

## Introduction

ExpressJS is a web framework that most of us know about. I really like how easy and fast it is to set up a new server using Express.

Under the hood, Express is a lightweight wrapper around the Node HTTP module. It abstracts out some complexity of the HTTP module and provides us with a simpler way to build our backend server.

In this article, I want to understand how Express is implemented and see if I can learn something from that.

## Start reading the codes

The source code of ExpressJS is available at: https://github.com/expressjs/express

First, let's check the directory structure:

```
├── index.js
├── lib
│   ├── application.js
│   ├── express.js
│   ├── middleware
│   │   ├── init.js
│   │   └── query.js
│   ├── request.js
│   ├── response.js
│   ├── router
│   │   ├── index.js
│   │   ├── layer.js
│   │   └── route.js
│   ├── utils.js
│   └── view.js
```

The main codes live in the `lib` folder, with a few modules such as: application, middleware, request, response, router… It is indeed simple, there are only a few files compared to other web frameworks.

Now, we can jump right into the `express.js` and read the codes. But I want to *approach the reading process in a different way*.

Let’s say that we have this tiny application that uses Express.

```jsx
const express = require('express')
const myapp = express()
const port = 3000

myapp.use(whateverLogger())

myapp.use('/hello', (req, res) => {
  res.send('Hello World!')
})

myapp.post('/world', (req, res) => {
  res.send('ok')
})

myapp.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
```

We can separate this application into 2 phases:

- **Phase 1: Registering the handlers**
    - In this phase, the handlers (middleware) have been set as follows: `whateverLogger`, and the handlers for `/hello` and `/world`.
- **Phase 2: Processing the requests**
    - This phase is where the server receives the requests and processes them

Let’s start reading the source codes following each phase.

## Phase 1: Registering handlers

The main method that we need attention to are the two:

- `myapp = express()`
- `myapp.use()`

For the `express()` my first guess is to read the `express.js`

### express.js

```jsx
...
var proto = require('./application');
var Route = require('./router/route');
var Router = require('./router');
...

exports = module.exports = createApplication;

function createApplication() {
  var app = function(req, res, next) {
    app.handle(req, res, next);
  };
  mixin(app, EventEmitter.prototype, false);
  mixin(app, proto, false);
  ...
  
  return app
}
```

(omitted other details for clarity)

The function call returns an `app` which is also a function. The `mixin(app, proto, false)` indicates that most of the logic of this `app` will reside in the  `./application`

We can skip the mixin logic and assume that its purpose is to add methods and properties to the `app`

Our next hop is the `application.js`

### application.js

At line 194, we can see the `app.use` is defined:

```jsx
...
var Router = require('./router');
...

app.use = function use(fn) {
  ...
  var router = this._router;
  var fns = flatten(slice.call(arguments, offset));
  ...

  fns.forEach(function (fn) {
    if (!fn || !fn.handle || !fn.set) {
      return router.use(path, fn);
    }

    ...
  }, this);

  return this;
};
```

There are many activities in this method but the first thing that should catch our eyes is the `fns.forEach` that calls the `router.use(path, fn)`

`app.use` receives a list of functions (that are our handlers for a path). Then register each of those functions using the `router`. 

Now let’s jump to the `router.js`

### router.js

A lot of interesting things happen in this file.

First, the constructor codes show a `stack` is defined.

Whenever I see a stack (or queue), I immediately think of an array that has some sort of order when we insert or access the array.

```jsx

var proto = module.exports = function(options) {
  ...
  router.stack = [];

  return router;
};
```

Continue, we find the definition of `router.use`

The sweet spot is this for loop where it creates a new `Layer` and pushes it to the stack

```jsx
proto.use = function use(fn) {
  ...
  var callbacks = flatten(slice.call(arguments, offset));
  ...

  for (var i = 0; i < callbacks.length; i++) {
    var fn = callbacks[I];
    ...
    var layer = new Layer(path, {
      sensitive: this.caseSensitive,
      strict: false,
      end: false
    }, fn);

    layer.route = undefined;

    this.stack.push(layer);
  }

  return this;
};
```

- Each item on the stack is a `Layer`
- A `Layer` can only receive a `path` and a `fn` callback

Now at this point, I can imagine how this setup is stored in the stack:

```jsx
myapp.use('/hello', handler1, handler2)
myapp.use('/world', handler3, handler4)
```

this will become:

```jsx
router.stack = [
  Layer('/world', handler4),
  Layer('/world', handler3),
  Layer('/hello', handler2),
  Layer('/hello', handler1) // <-- index: 0
]
```

I believe we now have a basic understanding of how the registration process works, and what is the data structure behind the scence

Before moving on to Phase 2, here's a diagram to summarize everything.

![Summary of phase 1: registering handlers](https://res.cloudinary.com/khanhhuy/image/upload/v1693645817/codediagram/Screenshot_2023-09-02_at_16.10.03_kegut9.png)
https://www.codediagram.io/app/shares?token=5dce1a36

On **Phase 2: Processing the requests**, we’ll focus on how a request is processed through the middleware by looking into the implementation of the `next()` function

Please stay tune 😄