How you can build your own web framework for Node.js
Tags: webdev, tutorial, javascript, node
Published: 2020-06-25 12:16:59

---
title: How you can build your own web framework for Node.js
published: true
description: This article covers building your own web framework by wrapping the http module.
tags: webdev, tutorial, javascript, node
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/u21viylubqliso0b44kl.jpg
---

Follow me on [Twitter](https://twitter.com/chris_noring), happy to take your suggestions on topics or improvements /Chris

TLDR; this article teaches you to implement the framework Express to a certain degree. Great for your own learning but don't use in production unless you got issues with space or bandwidth doing an NPM install. Hope it's helpful

The reason I write these kinds of articles is not that I want people to reinvent the wheel but to learn from the experience. I bet if you search npmjs you would find 100s of implementations looking more or less like one of the big known frameworks, Express, Nest, Koa, or Fastify. So what would creating one more framework do? Isn't that a waste of time? I don't think so and the reason is that you can learn a lot by trying to implement it yourself. You can acquire skills that help you in your everyday web dev life. It can also set you up nicely for OSS work as you now *see the Matrix*.

## Implementing the Express framework

For this article, I've chosen to try to implement a part of the Express framework. What parts is that exactly?

- **Routes**, Express has a way of associating specific routes and have specific code run if a route is hit. You are also able to differentiate routes based on HTTP Verb. So a GET to `/products` is different from a `POST` to `/products`.
- **Middleware**, is a piece of code that can run before or after your request and even control what should happen to the request. Middleware is how you could inspect a header for an auth token and if valid return the asked for resources. If the token is not valid then the request stops there and a suitable message can be sent back.
- **Query parameters**, this is the end part of the URL and is able to help further filter down what you want the response to look at. Given a URL looking like so `/products?page=1&pagesize=20`, the query parameters are everything that happens after `?`.
- **Sending data with a Body**, data can be sent from the client to the server application. It can be sent either over the URL or through a body. The body can contain different things, everything from JSON to simple form fields to even files.

## An example express App

Let's look at a few lines of implementing an Express app. There are a lot of things happening even with a few lines:

```javascript
const express = require('express')
const app = express();
app.get('/products/:id', (req, res) => {
  res.send(`You sent id ${req.params.id}`)
})

app.listen(3000, () => {
  console.log('Server up and running on port 3000')
})
```

## A Vanilla HTTP app

How would we go about implementing that? Well, we have the HTTP module at our disposal. So let's look at a very small implementation to understand what's missing:

```javascript
const http = require('http');
const PORT = 3000;

const server = http.createServer((req, res) => {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('hello world');
});

server.listen(PORT, () => {
  console.log(`listening on port ${PORT}`)
})
```

The HTTP module only has a very basic sense of routing. If you navigate toward such an app with URL `http://localhost:3000/products` the `req.url` will contain `/products` and `req.method` will contain the string `get`. That's it, that's all you have. 

## Implementing routing and HTTP Verbs

We are about to implement the following

- **HTTP Verb methods**, we need methods like `get()`, `post()` etc. 
- **Routing and route parameters**, we need to be able to match `/products` and we need to be able to break out the route parameter id from an expression looking like this `/products/:id`.
- **Query parameters**, we should be able to take a URL like so `http://localhost:3000/products?page=1&pageSize=20` and parse out the parameters `page` and `pageSize` so that they are easy to work with.
 
### HTTP Verb methods

Let's create a `server.js` and start implementing our server like so:

```javascript
// server.js
const http = require('http')

function myServer() {
  let routeTable = {};
  http.createServer((req, res) => {
    
  });   
  
  return {
    get(path, cb) {
      routeTable[path] = { 'get': cb }
    }
  }
}
```

Let's leave the code like that and continue to implement the routing.

### Parsing route parameters 

Implementing `/products` is easy, that's just string comparison with or without RegEx. Digging out an `id` parameter from `/products/:id` is slightly more tricky. We can do so with a RegEx once we realize that `/product/:id` can be rewritten as the RegEx `/products/:(?<id>\w+)`. This is a so-called named group that when we run the `match()` method will return an object containing a `groups` property with content like so `{ id: '1' }` for a route looking like so `/products/1`. Let's show such an implementation:

```javascript
// url-to-regex.js

function parse(url) {
  let str = "";

  for (var i =0; i < url.length; i++) {
    const c = url.charAt(i);
    if (c === ":") {
      // eat all characters
      let param = "";
      for (var j = i + 1; j < url.length; j++) {
        if (/\w/.test(url.charAt(j))) {
          param += url.charAt(j);
        } else {
          break;
        }
      }
      str += `(?<${param}>\\w+)`;
      i = j -1;
    } else {
      str += c;
    }
  }
  return str;
}

module.exports = parse;
```   

And to use it:

```javascript
const parse = require('./url-to-regex');

const regex = parse("/products/:id")).toBe("/products/(?<id>\\w+)");
const match = "/products/114".match(new RegExp(regex);
// match.groups is { id: '114' }     
```

### Adding routing to the server

Let's open up our `server.js` file again and add the route management part. 

```javascript
// server.js
const http = require('http')
const parse = require('./regex-from-url')

function myServer() {
  let routeTable = {};
  http.createServer((req, res) => {
    const routes = Object.keys(routeTable);
    let match = false;
    for(var i =0; i < routes.length; i++) {
       const route = routes[i];
       const parsedRoute = parse(route);
       if (
         new RegExp(parsedRoute).test(req.url) &&
         routeTable[route][req.method.toLowerCase()]
       ) {
         let cb = routeTable[route][req.method.toLowerCase()];
         
         const m = req.url.match(new RegExp(parsedRoute));
         
         req.params = m.groups;
         
         cb(req, res);
         
         match = true;
         break;
       }
    }
    if (!match) {
      res.statusCode = 404;
      res.end("Not found");
    }
  });   
  
  return {
    get(path, cb) {
      routeTable[path] = { 'get': cb }
    }
  }
}
```

What we are doing is looping through all the routes in our route dictionary until we find a match. The comparison looks like this:

```javascript
if (
  new RegExp(parsedRoute).test(req.url) &&
  routeTable[route][req.method.toLowerCase()]
)
```

Note also how the router params are parsed and placed on the `params` property like so:

```javascript
const m = req.url.match(new RegExp(parsedRoute));
req.params = m.groups;
```

### Query parameters

We already know that using the HTTP module, the URL will contain our route, like so `/products?page=1&pageSize`. The next step is to dig out those parameters. That can be accomplished by using a RegEx like and the below code:

```javascript
// query-params.js
  
function parse(url) {
  const results = url.match(/\?(?<query>.*)/);
  if (!results) {
    return {};
  }
  const { groups: { query } } = results;

  const pairs = query.match(/(?<param>\w+)=(?<value>\w+)/g);
  const params = pairs.reduce((acc, curr) => {
    const [key, value] = curr.split(("="));
    acc[key] = value;
    return acc;
  }, {});
  return params;
}

module.exports = parse;
```

Now we need to tie that into the server code. That's just a few lines, fortunately:

```javascript
const queryParse = require('./query-params.js')

// the rest omitted for brevity
ress.query = queryParse(req.url);
```

### Sending data with a Body

Reading the body can be done by realizing that the input parameter `req` is of type stream. It's good to know that data arrives in small pieces, so-called chunks. By listening to the event `end` the client is letting is now that the transmission is complete and no more data will be sent.

You can listen to incoming data by listening to the event `data`, like so:

```javascript
req.on('data', (chunk) => {
  // do something
})

req.on('end', () => {
  // no more data
})
```

To implement listening to data being transmitted from a client we can, therefore, create the following helper method:

```javascript
function readBody(req) {
    return new Promise((resolve, reject) => {
      let body = "";
      req.on("data", (chunk) => {
        body += "" + chunk;
      });
      req.on("end", () => {
        resolve(body);
      });
      req.on("error", (err) => {
        reject(err);
      });
    });
  }
```

and then use it in our server code like so:

```javascript
res.body = await readBody(req);
```

The full code at this point should look like this:

```javascript
// server.js

const http = require('http')
const queryParse = require('./query-params.js')
const parse = require('./regex-from-url')

function readBody(req) {
    return new Promise((resolve, reject) => {
      let body = "";
      req.on("data", (chunk) => {
        body += "" + chunk;
      });
      req.on("end", () => {
        resolve(body);
      });
      req.on("error", (err) => {
        reject(err);
      });
    });
  }

function myServer() {
  let routeTable = {};
  http.createServer(async(req, res) => {
    const routes = Object.keys(routeTable);
    let match = false;
    for(var i =0; i < routes.length; i++) {
       const route = routes[i];
       const parsedRoute = parse(route);
       if (
         new RegExp(parsedRoute).test(req.url) &&
         routeTable[route][req.method.toLowerCase()]
       ) {
         let cb = routeTable[route][req.method.toLowerCase()];
         
         const m = req.url.match(new RegExp(parsedRoute));
         
         req.params = m.groups;
         req.query = queryParse(req.url);
         req.body = await readBody(req);
         
         cb(req, res);
         
         match = true;
         break;
       }
    }
    if (!match) {
      res.statusCode = 404;
      res.end("Not found");
    }
  });   
  
  return {
    get(path, cb) {
      routeTable[path] = { 'get': cb }
    },
    post(path, cb) {
      routeTable[path] = { 'post': cb }
    }
  }
}
```

At this point you should be able to call your code like so:

```javascript
const server = require('./server')
const app = server();

app.get('/products/:id', (req, res) => {
  // for route /products/1, req.params has value  { id: '1' }

})
app.get('/products/', (req, res) => {
  // for route /products?page=1&pageSize=10, req.query has value  { page: '1', pageSize: '10' }
})
app.post('/products/', (req, res) => {
  // req.body should contain whatever you sent across as client
})
```

### Response helpers

At this point, a lot is working. But how do you actually return data back to the client? Because you are implementing the HTTP module the `res` parameter can be used. By calling its `end()` you can send data back. Here's an example:

```javascript
res.end('some data')
```

However, if you look at how Express does it, it has all sorts of helpers for this like `send()`, `json()`, `html()` and so on. You can have that too with a few lines of code:

```javascript
function createResponse(res) {
  res.send = (message) => res.end(message);
  res.json = (message) => {
    res.setHeader("Content-Type", "application/json");
    res.end(JSON.stringify(message));
  };
  res.html = (message) => {
    res.setHeader("Content-Type", "text/html");
    res.end(message); 
  }
  return res;
}
```

and make sure you add it in the server code:

```javascript
res = createResponse(res);
```

### Middleware

Having middleware allows us to run code before or after the request, or even control the request itself. Have a look at the following code:

```javascript
server.get("/protected", (req, res, next) => {
  if (req.headers["authorization"] === "abc123") {
    next();
  } else {
    res.statusCode = 401;
    res.send("Not allowed");
  }
 }, (req, res) => {
   res.send("protected route");
 });
```

The second argument is the middleware. It inspects `req.headers` for an `authorization` property and checks its value. If everything is ok it invokes `next()`. If it's not ok then the request stop here and `res.send()` is invoked and status code set to `401`, not allowed. 

The last argument is the route response you want the client to see providing they send you an ok header value.

Let's implement this. Create the following function in `server.js`:

```javascript
function processMiddleware(middleware, req, res) {
  if (!middleware) {
    // resolve false
    return new Promise((resolve) => resolve(true));
  }

  return new Promise((resolve) => {
    middleware(req, res, function () {
      resolve(true);
    });
  });
}
```

Above the `middleware` param is being called and you can see how the last argument to it is a function that resolves a Promise like so:

```javascript
middleware(req, res, function () {
  resolve(true);
});
```

For the server code to use this, there are a few steps we need to take:

1. Ensure we register the middleware
2. Get a hold of the middleware when we have a matching request
3. Call the middleware

**Register middleware**

We need to change slightly how we register routes by first adding this helper method:

```javascript
function registerPath(path, cb, method, middleware) {
    if (!routeTable[path]) {
      routeTable[path] = {};
    } 
    routeTable[path] = { ...routeTable[path], [method]: cb, [method + "-middleware"]: middleware };
  }
```

So trying to register a route like so:

```javascript
server.get('/products', (req, res, next) => {}, (req, res) => {})
```

leads to the middleware callback being saved on a property `get-middleware`

Then when we register the route we do something like this instead:

```javascript
return {
    get: (path, ...rest) => {
      if (rest.length === 1) {
        registerPath(path, rest[0] , "get");
      } else {
        registerPath(path, rest[1], "get", rest[0]);
      }
    },
```

**Get a reference to the middleware**

To get a reference to the middleware we can use this code:

```javascript
let middleware = routeTable[route][`${req.method.toLowerCase()}-middleware`]; 
```

**Process middleware**

Finally, to run the middleware write the below code:

```javascript
const result = await processMiddleware(middleware, req, createResponse(res));
if (result) {
  cb(req, res);
} 
```

## Summary

The full code is available at this repo:

> https://github.com/softchris/mini-web

and it can also be used via NPM by calling:

```bash
npm install quarkhttp
```

That was a lot, routing, routing parameters, query parameters, body parsing, and middleware. Hopefully, you can now understand what's going on. Remember that there are great libraries out there for you to use that are well tested. However, understanding how things are implemented can be really beneficial for your understanding.