Project structure for an Express REST API when there is no "standard way"
Tags: node, javascript, express
Published: 2018-11-06 13:20:30

---
title: Project structure for an Express REST API when there is no "standard way"
published: true
canonical_url: https://www.coreycleary.me/project-structure-for-an-express-rest-api-when-there-is-no-standard-way/
description: 
tags: node,javascript,express
---
<em><strong>Originally published at <a href="https://www.coreycleary.me/project-structure-for-an-express-rest-api-when-there-is-no-standard-way/">coreycleary.me</a></strong>. This is a cross-post from my content blog. I publish new content every week or two, and you can <a href="https://www.coreycleary.me/about/">sign up to my newsletter</a> if you'd like to receive my articles directly to your inbox! I also regularly send cheatsheets, links to great tutorials by other developers, and other freebies!</em>

Do any GitHub or Google search for REST API structures using Node + Express and you'll find very few of them follow the same organization.

What's even worse is, while there are lots of good tutorials out there, many of them have no structure at all. They just put everything into a 200-line server.js file and call it a day...

One of the <strong>best</strong> things about Node can also one of the most painful - there are <em>few</em> (if any) true conventions.

Sure, there are recommend ways of doing things. But it's such a flexible platform that you can often pick <em>any</em> way of doing something and it will likely work.

But even without conventions, developers want to know the best way to do things. And when it comes to REST API's (and Node projects in general...), everyone seems to feel like they're flying blind when it comes to structuring the project!

<h3>No "right way"</h3>

Ultimately, I don't believe there is one <strong>best</strong> project structure for Express projects.

Instead of asking:
<blockquote>"What’s the best way to structure my files and folders?"</blockquote>
I think it’s better to ask:
<blockquote>"What places do my different types of logic go?"</blockquote>
There are clearer answers to <em>that</em> question, and things we can follow.

And by doing a good job of separating our logic by <strong>layers</strong>, a project structure will naturally emerge. A structure that is flexible to how you choose to organize it, whether that is by more traditional MVC (or MVC-ish) or by the cool new kid, components. That's because these structures will be a layered approach anyways! You can simply group the routes, controllers, services, etc. into a component folder.

As long as logic is in the "right" place, the structure/organization becomes not that big a deal.

It's easier to refactor without having logic in weird places, it's easier to test without having logic in weird places, and once you've got your code checked into source control, it's easy to make changes anyways! Want to try out a components structure instead? Easy to make the changes!

<h3>"OK, I'm on board... but now what?"</h3>

<em>A quick note on scope of this post: obviously all projects can include many different things. In order to make this post understandable and not overwhelm you, I'm going to leave out request/model validation and authentication. Both of those are animals on their own, but expect posts in the future addressing them.</em>

Also, this is not intended to be a "boilerplate" project where you copy the repo, enter `npm start` in the console and instantly have a full-fledged application. Although, you will get a <strong>running app</strong> if you follow along. But a boilerplate wouldn't really achieve the goal of explaining layers in an app, where to put logic, and how to arrive at a structure based on that.

Since we're dealing with a REST API and an API receives a <strong>request</strong> and returns a <strong>response</strong>, let's follow that request from the point it hits our application, travels through our layers, and a response gets returned by the app. Along the way, we’ll figure out where our different types of logic should go.

<h3>Layers?</h3>

Let's take a look at an architecture diagram first:
<img src="https://www.coreycleary.me/wp-content/uploads/2018/11/Express-REST-API-Struc.png" alt="" width="741" height="362" class="aligncenter size-full wp-image-803" />

Ever peeled an onion? After you peel off the first outer layer, there are a couple layers underneath that.

"Layers" in this context is much the same, meaning we've got:

a <strong>HTTP layer</strong> --> which is "outside" the <strong>service layer</strong> --> which is "outside" the <strong>database access layer</strong> --> which is... you get the picture

<h3>Where does the logic go?</h3>

We'll be using an example of a blog application to demonstrate logic separation and our resulting structure. 

When I mentioned "types of logic" I was referring to the two "main" categories REST API logic falls into - HTTP logic and business logic. Of course, you can split "types of logic" down as much as you’d like, but these two are the main categories.

| Main Layers | Type | What logic goes here? |
| ----------- | ---- | --------------------- |
| HTTP logic layer | Routes + Controllers | Routes - handle the HTTP requests that hits the API and route them to appropriate controller(s); Contollers - take request object, pull out data from request, validate, then send to service(s) |
| Business logic layer | Services + Data Access | Contains the business logic, derived from business and technical requirements, as well as how we access our data stores** |

**The data access layer logic is often the more "technical" business logic, and I’ve grouped it in with the business logic as requirements often drive the queries you’ll need to write and the reports you’ll need to generate.

<h3>Routes</h3>

```javascript
const express = require('express')

const { blogpost } = require('../controllers')

const router = express.Router()

router.post('/blogpost', blogpost.postBlogpost)

module.exports = router
```
As you can see from the code above, no logic should go in your `routes/routers`. They should only chain together your `controller` functions (in this case, we only have one). So `routes` are pretty simple. Import your controller(s) and chain together the functions.

I <em>usually</em> just have one controller per route, but there are exceptions, of course. If you have a controller that handles authentication, and have routes that need authentication, you would obviously need to import that as well and hook it up to your route.

Unless you have a ton of `routes`, I usually put them all in one `index.js` file. If you <em>do</em> have a ton of routes, you can put them into individual route files, import them all into one `index.js` file and export that.

If you want to understand how to avoid manually prepending '/api' to each individual route, <a href="https://www.coreycleary.me/avoid-manually-prepending-api-to-every-express-route-with-this-simple-method/" rel="noopener" target="_blank">check out this other post I wrote on that</a>.

<h3>Controllers</h3>

```javascript
const { blogService } = require('../services')

const { createBlogpost } = blogService

/*
 * call other imported services, or same service but different functions here if you need to
*/
const postBlogpost = async (req, res, next) => {
  const {user, content} = req.body
  try {
    await createBlogpost(user, content)
    // other service call (or same service, different function can go here)
    // i.e. - await generateBlogpostPreview()
    res.sendStatus(201)
    next()
  } catch(e) {
    console.log(e.message)
    res.sendStatus(500) && next(error)
  }
}

module.exports = {
  postBlogpost
}
```
I think of `controllers` as "orchestrators". They call the `services`, which contain more "pure" business logic. But by themselves,`controllers` don't really contain any logic other than handling the request and calling `services`. The `services` do most of the work, while the `controllers` orchestrate the service calls and decide what to do with the data returned.

And if it’s not obvious already, they take the HTTP request forwarded from the route and either return a response, or keep the chain of calls going. They handle the HTTP status codes as part of this response too.

<h3>Why Express/HTTP context should end here</h3>

Something I see fairly frequently is the Express `req` object (which is our HTTP "context") passed through beyond the `routes` and `controllers` to the `services` or even `database access layer`. But the problem with that is that, now the rest of the application depends on not only the request object, but on Express as well. If you were to swap out frameworks, it would be more work to find all the instances of the `req` object and remove them.

It also makes testing more difficult and this does not achieve a separation of concerns that we strive for in designing our applications.

Instead, if you use destructuring to pull out what pieces of data you need from `req`, you can simply pass those on to the services. The Express <em>logic</em> "ends" right there in the controllers.

If you need to make a call to an external API from one of your services, that is ok though, and we'll discuss that more when we cover what logic goes in `services`. But for now know that those calls are outside the HTTP context of <em>your</em> application.

And with that, we know where to put our "initial" logic that the REST API will handle (routes + controllers). On to the business logic layer...

<h3>Services</h3>

```javascript
const { blogpostDb } = require('../db')

/*
  * if you need to make calls to additional tables, data stores (Redis, for example), 
  * or call an external endpoint as part of creating the blogpost, add them to this service
*/
const createBlogpost = async (user, content) => {
  try {
    return await blogpostDb(user, content)
  } catch(e) {
    throw new Error(e.message)
  }
}

module.exports = {
  createBlogpost
}
```
`Services` should contain the majority of your business logic: - logic that encapsulates your business requirements, calls your data access layer or models, calls API's external to the Node application. And in general, contains most of your algorithmic code.

You can certainly call external API's from within your `controllers` as well, but think about if that API is returning something that should be part of a "unit". `Services` should ultimately return a cohesive resource, and so if what that external API call returns is needed to augment your business logic, keep the logic there.

For example, if part of creating the blogpost was also posting the link to Twitter (an external API call), you would put it in the service above.

<em>Why not call the models/data layer directly from the `controllers` if that's all this service is doing?</em>

While our example above is simple in that all it does is access the database through our data access layer function - `blogpostDb` - as more business requirements get added, you add that Twitter API call, requirements change, etc. it will get complex <strong>fast</strong>.

If your controller handled all that logic, plus the request handling logic it already takes care of, it would start to become really hard to test, really fast. And remember, controllers can make multiple different service calls. So if you pulled all that logic out of other services and put it in the same controller, it would get even more unmanageable. You'd end up with the dreaded "fat controller" nightmare.

<h3>Data Access Layer/Models</h3>

```javascript
const blogpostDb = (user, content) => {
  /*
   * put code to call database here
   * this can be either an ORM model or code to call the database through a driver or querybuilder
   * i.e.-
    INSERT INTO blogposts (user_name, blogpost_body)
    VALUES (user, content);
  */
  return 1 //just a dummy return as we aren't calling db right now
}

module.exports = {
  blogpostDb
}
```
In the code above, rather than set up a full database connection, I just pseudo-coded it but adding it is easy enough. When you have your logic isolated like this, it's easy to keep it limited to just data access code.

If it's not obvious, "Data Access Layer" means the layer that contains your logic for accessing persistent data. This can be something like a database, a Redis server, Elasticsearch, etc. So whenever you need to access such data, put that logic here.

"Models" is the same concept but used as part of an ORM.

Even though both are different they contain the same type of logic, which is why I recommend putting either kind in a `db` folder so that its general enough. Whether you’re using models from an ORM or you’re using a query builder or raw SQL, you can put the logic there without changing the name of directory.

<h3>Utils</h3>

The last type of logic we'll cover is that of common logic functions that are not necessarily specific to your business logic or domain, or even a REST API in general. A good example of a <strong>utility function</strong> would be a function that converts milliseconds to minutes and/or seconds, or one that checks two arrays to see if they contain similar items. These are general enough - and <em>reusable</em> enough - that they deserve to go in their own folder.

My preferred method is just putting these all into an `index.js` file and exporting each function. And I leave it at that as they don't really have a bearing on the rest of the project structure.

<h3>app.js / server.js</h3>
```javascript
const express = require('express')
const bodyParser = require('body-parser')
const app = express()
const routes = require('./routes')

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.get('/', (req, res) => res.send('App is working'))

app.use('/api', routes)

app.listen(3000, () => console.log('Example app listening on port 3000!'))

module.exports = {
  app
}
```

And just to tie it all together, I included an example entrypoint (usually named `app.js` or `server.js`) that goes in the root of your project structure. You can add middleware here (such as `bodyParser`) and import your routes file. 

<h3>Now, the structure that emerges</h3>

<img src="https://www.coreycleary.me/wp-content/uploads/2018/11/role-structure.png" alt="" width="248" height="632" class="aligncenter size-full wp-image-801" />

When you separate your logic like I've described in this post, the structure seems to "naturally" fall into place like the above. This is why I like separating my logic in Express applications like this, it's easy to figure out where to put things!

And you can, of course, add more directories to the structure as you see fit (maybe a `config` folder, for example). But this is a great base to start from and 90% of your code will fall into one of these folders if you separate your logic as described.

<h3>Last but not least, tests!</h3>

Now that we've covered the structure that following this pattern will emerge out of, it's worth pointing out where tests go. I don't think this is as strict a rule, but I tend to keep my tests in one root `tests` folder and mimic the structure of the rest of the application.

<img src="https://www.coreycleary.me/wp-content/uploads/2018/11/tests-struc.png" alt="" width="298" height="285" class="aligncenter size-full wp-image-802" />

<em>In case you noticed, `routes` is missing! That's because if you separate out your logic like I've done, you don't really need to test the routes. You can use something like `supertest` if you want to, but the core logic - things that can break more easily with code changes! - will already be tested in your controllers, services, etc.</em>

As an alternative, you could also add a tests folder within each "layer" folder, i.e. - a tests directory within the controllers, one within the services, etc. Just depends on preference, don't fret about it.

Also, some developers like to separate test directories by <strong>unit tests</strong> and by <strong>integration tests</strong>. My thoughts on that are that if you have a an application where there is a clear delineation and you have a <em>lot</em> of integration tests, it might be good to separate them. But more often than not I include them in the same directory.

<h3>Wrapping up</h3>

Like I stated at the beginning of this post, I don't believe there is a "best structure". It's much more helpful to make sure you've separated your logic into roles instead. <strong>THAT</strong> will give you the structure as a by-product, as well as giving you the flexibility you need to easily make changes later.

So if you're starting a new Express project and wasting time deciding which folders you should create, what you should name them, what should go in them - or if you're refactoring an existing Express app - use the approach I've described here to get you unstuck and get the ball rolling. And don't worry about it from there.

Remember, you can always change it later as long as your separation of logic is sound!

<h3>One more thing!</h3>

There's only so much I could cover here without it being overwhelming and you closing the window fast. I'm going to supplement this with additional structure/separation of logic articles coming soon.

If you want those additional articles emailed directly to you, <a href="https://www.coreycleary.me/about/">here's that link again</a> to subscribe to my newsletter! I send out new articles every week or two, in addition to cheatsheets, quick tips, and more.