Building Backend with Composable Services
Tags: architecture, typescript, api
Published: 2022-11-16 22:12:47

## About Composable Systems
Working inside a well-structured backend code feels like playing with Lego blocks. Many backend developers love opinionated frameworks (like Spring, Ruby on Rails, NestJS) because these frameworks make it easier to split their code into simple components.

In this post, we'll explore the essence of composable systems, and apply it to an (unopinionated) ExpressJS project.

To build a composable system, each component should have a simple & clear boundary. There are two main offenders that hurt composability of a component:

### Global & Top-level Variable
Global variables or module-level variables smudge the boundary between components. Environment variables are a good example of this pattern. It is so easy to access `process.env` from anywhere, so we tend to use them without thinking too much about it. This makes it harder to check which function depends on which environment variable. In other words, the function's dependency gets blurry.

### Big Interface
Sometimes, interfaces are very clear but too big. This is as bad as blurry dependencies because it is difficult to swap out a component with big interface. ORMs (object-relational mapping) and database adapters are good examples of this pattern.

By using an ORM, developers can access database with a clean interface; however, most ORMs have extremely big interfaces. It is near impossible to swap out an ORM with another component.

## Refactoring for Simpler Structure
Ok, so we have identified the problem, but how do we fix it? Let's look at some examples and refactoring techniques.

### Global & Top-level Variable
#### Example
Environment variables are often used like this:

```ts
// env.ts
import dotenv from "dotenv";
dotenv.configure();
export const SESSION_SECRET = process.env.SESSION_SECRET;
export const DATABASE_URI = process.env.DATABASE_URI;
```

```ts
import { DATABASE_URI } from "./env";
mongoose.connect(DATABASE_URI);
```

Top-level variables in this example blur the boundary between components as mentioned above.

#### Refactoring
Instead of exposing `DATABASE_URI` as top-level variable, we can put it in a function scope. Let's call the wrapper function `EnvService`.

```ts
// env.ts
import dotenv from "dotenv";
export const EnvService = () => {
  dotenv.configure();
  return {
    SESSION_SECRET: process.env.SESSION_SECRET,
    DATABASE_URI: process.env.DATABASE_URI,
  };
};
export type EnvService = ReturnType<typeof EnvService>;
```

To express that our database depends on environment variables, we can write a wrapper function for database as well.

```ts
import { EnvService } from "./env";
export const DatabaseService = (env: EnvService) => {
  mongoose.connect(env.DATABASE_URI);
  ...
};
```

Now it is super clear that our database depends on environment variables. We don't need global variables anymore.

### Big Interface

Then what about big interfaces?

#### Example

It is common to use ORM classes directly inside request handlers.

```ts
app.get("/user/:id", async (req, res, next) => {
  try {
    const user = await UserModel.findById(req.params.id);
    //                 ^ like this one
    res.send({ user });
  } catch (err) {
    next(err);
  }
});
```

There are two problems with this implementation.
1. `UserModel` is a top-level class. The "/user/:id" handler depends on `UserModel`, but this dependency is not explicit.
2. `UserModel` has big interface. Most ORM libraries expose huge interfaces. It is unavoidable because an ORM needs to support a lot of database features & configurations.

In short, the request handler is tightly coupled with `UserModel`. There is no simple way to slice them apart cleanly.

#### Refactoring

We can attempt the same approach to make dependencies explicit. Let's start from the ORM.

```ts
// user.ts
export const UserService = () => {
  return {
    getById: (userId: string) => UserModel.findById(userId),
  };
};
export type UserService = ReturnType<typeof UserService>;
// type: { getById: (userId: string) => Promise<User> }
```

We have wrapped the complex top-level `UserModel` in a `UserService` function. `UserService` has much smaller interface compared to `UserModel`.

It's time to express the dependency between `UserService` & the request handler.

First, extract the handler into a separate function.

```ts
const userHandler: RequestHandler = async (
  req, res, next
) => {
  try {
    const user = await UserModel.findById(req.params.id);
    res.send({ user });
  } catch (err) {
    next(err);
  }
};
app.get("/user/:id", userHandler);
```

Then, express the dependency by wrapping the request handler with a function.

```ts
const UserHandler = (
  userService: UserService
): RequestHandler => async (req, res, next) => {
  try {
    const user = await userService.getById(req.params.id);
    res.send({ user });
  } catch (err) {
    next(err);
  }
};
app.get("/user/:id", UserHandler(userService));
```

Nice, now we have a clean interface between the request handler and the service!

## Final Touch

Wrapping top-level variables & expressing dependencies as parameters are excellent ways to simplify boundaries between components. (This technique is called "inversion of control")
{% embed https://en.wikipedia.org/wiki/Inversion_of_control %}

But you may have noticed that we still don't have a way to provide required services to request handlers.

```ts
app.get("/user/:id", UserHandler(userService));
// Where does `userService` come from?
```

We can create a meta service for this purpose. A service that provides other services.

```ts
import { EnvService } from "./env";
import { SessionService } from "./session";
import { UserService } from "./user";

type ServiceMap = {
  user: UserService;
  env: EnvService;
  session: SessionService;
};

// Meta service
export const ServiceProvider = () => {
  // Initialize services.
  const env = EnvService();
  // A service can depend on another service,
  const user = UserService(env);
  // or multiple services.
  const session = SessionService(env, user);

  const serviceMap: ServiceMap = {
    user,
    env,
    session,
  };

  /**
   * Get service by service name.
   */
  const getService = <TServiceName extends keyof ServiceMap>(
    serviceName: TServiceName
  ) => serviceMap[serviceName];

  return getService;
};

export type ServiceProvider = ReturnType<typeof ServiceProvider>;
```

This meta service can be used like this:

```ts
const service = ServiceProvider();
const env = service("env"); // EnvService
```

Let's express that our app depends on `ServiceProvider`:

```ts
// app.ts
import express from "express";
import { ServiceProvider } from "./service";
import { UserHandler } from "./handlers";

export const App = (service: ServiceProvider) => {
  const app = express();
  app.get("/user/:id", UserHandler(
    // Provide required service with `ServiceProvider`.
    service("user"),
  ));
  return app;
};
```

Finally, initialize `ServiceProvider` and pass it to `App`!

```ts
// index.ts
import { App } from "./app";
import { ServiceProvider } from "./service";

const service = ServiceProvider();
const app = App(service);
app.listen(service("env").PORT);
```

## What's Good About This Pattern?

By organizing functionalities into services, we get very straightforward project structure to reason about.

- Modifying service implementation is easier when the interface is small & explicit. Swapping out Firestore with MongoDB? That only affects the service that directly interacts with the database. The rest of the codebase is largely unaffected. The same can't be said if our request handlers are using ORM classes directly. Of course we don't change database every week, but the point is that changes are easier to make.
- Testing gets much easier. This is related to the first point, because we replace services with mocks during testing. Creating a test double (a.k.a. mock, fake, stub) is easier when the component has small & explicit interface. Also, checking which component to mock is effortless with inversion of control. All dependencies are expressed as parameters, so we just need to check those parameters.

## Summary

Here's a summary of the refactoring technique we used.

1. Wrap top-level variables/functions into a service function. The service function's interface should be small.
1. Extract request handlers as separate functions.
1. Express dependencies as parameters (inversion of control).
1. Create a meta service (`ServiceProvider`).
1. Provide services to request handlers via `ServiceProvider`.

If you want to take a look at this pattern in action, please check out my personal project:
- [service function](https://github.com/0916dhkim/spotify-filter/blob/655cdd4013bffbd8910036f5f393fcfe89aa6b04/server/src/services/database/index.ts)
- [request handler](https://github.com/0916dhkim/spotify-filter/blob/655cdd4013bffbd8910036f5f393fcfe89aa6b04/server/src/handlers/profile.ts)
- [ServiceProvider](https://github.com/0916dhkim/spotify-filter/blob/655cdd4013bffbd8910036f5f393fcfe89aa6b04/server/src/services/index.ts)
- [test code](https://github.com/0916dhkim/spotify-filter/blob/655cdd4013bffbd8910036f5f393fcfe89aa6b04/server/test/api.test.ts)