How does TypeScript's explicit resource management work?
Tags: typescript, webdev, using, resources
Published: 2023-10-23 09:30:55

TypeScript has a [fun new `using` keyword](https://devblogs.microsoft.com/typescript/announcing-typescript-5-2/#using-declarations-and-explicit-resource-management) which lets us play with [explicit resource management](https://github.com/tc39/proposal-explicit-resource-management)! In this context a "resource" could be a database connection, a file handle, a worker thread - anything that we might need to clean-up in the context of our application.

The release notes provide all the info you need about how consuming `using` works, but I thought it'd be fun to look at how it [_desugars_](https://en.wiktionary.org/wiki/desugar) to reverse engineer how it works... 

## Resource management 🤡

Say an application needs to temporarily write some data to disk. It might use `fs.openSync` to get a handle to the file and then use the handle to actually save the data.

```typescript
import * as fs from 'node:fs';
import { tmpdir } from 'node:os';
import * as path from 'node:path';

export function tmpData(tmp: unknown): void {
  const tmpPath = path.join(tmpdir(), `${Math.random()}`);
  const file = fs.openSync(tmpPath, 'w+');
  fs.writeSync(file, JSON.stringify(tmp));
}
```

Before you say anything - yeah, this code doesn't correctly clean up after itself! 🔥

But doing this "correctly" is actually pretty hard, especially if there are multiple resources in use! Even if we're super careful and make sure that all the possible different code paths dispose of the resources, it can be super fragile and susceptible to breaking on future changes.

## New shiny ✨

The [explicit resource management](https://github.com/tc39/proposal-explicit-resource-management) proposal tries to make it a bit easier for us, by allowing the resource to declare how it should be managed, rather than expecting us to clean everything up when we use the resource. We get a new keyword `using` to define a variable (rather than `const` or `let`), which tells the runtime to clean up the resource at the end of the function.

It looks something like this:

```typescript
import * as fs from 'node:fs';
import { tmpdir } from 'node:os';
import * as path from 'node:path';

export function tmpData(tmp: unknown): void {
  const tmpPath = path.join(tmpdir(), `${Math.random()}`);
  using file = fs.openSync(tmpPath, 'w+');
  fs.writeSync(file, JSON.stringify(tmp));
}
```

Doesn't seem like much difference, but if we look at the compiled JavaScript we can see just how much this is doing for us!

> Don't feel the need to decipher it all yet, we're gonna break it down I promise 👻

```javascript
var __addDisposableResource = (this && this.__addDisposableResource) || function (env, value, async) {
  if (value !== null && value !== void 0) {
    if (typeof value !== "object" && typeof value !== "function") throw new TypeError("Object expected.");
    var dispose;
    if (async) {
      if (!Symbol.asyncDispose) throw new TypeError("Symbol.asyncDispose is not defined.");
      dispose = value[Symbol.asyncDispose];
    }
    if (dispose === void 0) {
      if (!Symbol.dispose) throw new TypeError("Symbol.dispose is not defined.");
      dispose = value[Symbol.dispose];
    }
    if (typeof dispose !== "function") throw new TypeError("Object not disposable.");
    env.stack.push({ value: value, dispose: dispose, async: async });
  }
  else if (async) {
    env.stack.push({ async: true });
  }
  return value;
};

var __disposeResources = (this && this.__disposeResources) || (function (SuppressedError) {
  return function (env) {
    function fail(e) {
      env.error = env.hasError ? new SuppressedError(e, env.error, "An error was suppressed during disposal.") : e;
      env.hasError = true;
    }
    function next() {
      while (env.stack.length) {
        var rec = env.stack.pop();
        try {
          var result = rec.dispose && rec.dispose.call(rec.value);
          if (rec.async) return Promise.resolve(result).then(next, function(e) { fail(e); return next(); });
        }
        catch (e) {
          fail(e);
        }
      }
      if (env.hasError) throw env.error;
    }
    return next();
  };
})(typeof SuppressedError === "function" ? SuppressedError : function (error, suppressed, message) {
  var e = new Error(message);
  return e.name = "SuppressedError", e.error = error, e.suppressed = suppressed, e;
});

import * as fs from 'node:fs';
import { tmpdir } from 'node:os';
import * as path from 'node:path';

export function tmpData(tmp) {
  const env_1 = { stack: [], error: void 0, hasError: false };
  try {
    const tmpPath = path.join(tmpdir(), `${Math.random()}`);
    const file = __addDisposableResource(env_1, fs.openSync(tmpPath, 'w+'), false);
    fs.writeSync(file, JSON.stringify(tmp));
  }
  catch (e_1) {
    env_1.error = e_1;
    env_1.hasError = true;
  }
  finally {
    __disposeResources(env_1);
  }
}
```

## Transpiler output 💻:

So that's quite a lot of code that's generated for us by just using (😅) the `using` keyword! TypeScript has prepended some helper functions (`__addDisposableResource` and `__disposeResources`) that do the actual `using` magic, and then modified the code where we used `using`. Let's look at those changes first:

```typescript
// Before:
export function tmpData(tmp: unknown): void {
  const tmpPath = path.join(tmpdir(), `${Math.random()}`);
  using file = fs.openSync(tmpPath, 'w+');
  fs.writeSync(file, JSON.stringify(tmp));
}
```

```javascript
// After:
export function tmpData(tmp) {
  const env_1 = { stack: [], error: void 0, hasError: false };
  try {
    const tmpPath = path.join(tmpdir(), `${Math.random()}`);
    const file = __addDisposableResource(env_1, fs.openSync(tmpPath, 'w+'), false);
    fs.writeSync(file, JSON.stringify(tmp));
  }
  catch (e_1) {
    env_1.error = e_1;
    env_1.hasError = true;
  }
  finally {
    __disposeResources(env_1);
  }
}
```

First, we can see that the transpiler has added a new `env_1` variable. It initially has an empty `stack` array, an `error` property that starts undefined, and `hasError` set to false. This seems like a function-level context object for the entire `using` scope, something like:

```typescript
export type DisposalScope = {
  stack: Array<unknown>; // Don't know what this is yet
  error: unknown; // You can throw anything in JavaScript
  hasError: boolean;
}
```

Next, the entire body of the function has been wrapped in `try`/`catch`/`finally`. This guarantees that no matter what we do in the function, the execution will be guarded by the "resource management" functionality. 

> **Question**: What happens if we add a return statement before the `using` declaration?


```typescript
// Before:
export function tmpData(tmp: unknown): void {
  if (Math.random() > 0.5) {
    return;
  }
  // ...
}
```

```javascript
// After:
export async function tmpData(tmp) {
  const env_1 = { stack: [], error: void 0, hasError: false };
  try {
    if (Math.random() > 0.5) {
      return;
    }
  }
  // ...
}
```

> **Answer**: Nothing different, seems like the `try`/`catch`/`finally` will always wrap the full function body. Makes sense, and seems like the simplest solution!

The `catch` block is kind of interesting - we can see that it uses `env_1` to capture information about the error that was caught. The `error` is stored so that it can be referenced in the `finally` block - and rethrown later. You might wonder why `error` and `hasError` are both needed - I did! But then I remembered that `throw undefined` is totally valid in JavaScript, so you can't use `hasError = !!error`.

We can also see that the declaration expression has been wrapped in `__addDisposableResource`:

```typescript
// Before:
using file = fs.openSync(tmpPath, 'w+');
```

```javascript
// After:
const file = __addDisposableResource(env_1, fs.openSync(tmpPath, 'w+'), false);
```

So we can see that `using` becomes a `const`, which means we can't later assign a different handle to the variable. Seems reasonable, and like the least magical option, but I can't think of a reason right now that `let` wouldn't work - let me know if you know why! 

`__addDisposableResource` takes three arguments, firstly the `env_1` scope context, then the actual handle to the resource, and lastly a boolean flag (called `async`) in the code.

> **Question**: How do we change `async` from `true` to `false`?

> **Answer**:
> This doesn't change it: 

```typescript
// Before:
using file1 = await fs.openSync(tmpPath, 'w+');
```

```javascript
// After:
const file = __addDisposableResource(env_1, await fs.openSync(tmpPath, 'w+'), false);
```

> But this does:

```typescript
// Before:
await using file1 = fs.openSync(tmpPath, 'w+');
```

```javascript
// After:
const file = __addDisposableResource(env_1, await fs.openSync(tmpPath, 'w+'), true);
```

> So we have some additional `async` semantics that we need to learn more about! I guess this is for cases where you have to do asynchronous clean-up. (I wonder if there is a way to trigger this in a sync function with `then`?)

Finally (lol), `__disposeResources` is called in the `finally` block. It just takes one argument, the `env_1` function scope. Since `env_1` has access to any thrown errors, `__disposeResources` must be responsible for handling any errors thrown in the function body.

## Helper functions 💪🏻:

### `__addDisposableResource` 🗑️:

Let's look at `__addDisposableResource` first:

```javascript
var __addDisposableResource = (this && this.__addDisposableResource) || function (env, value, async) {
  if (value !== null && value !== void 0) {
    if (typeof value !== "object" && typeof value !== "function") throw new TypeError("Object expected.");
    var dispose;
    if (async) {
      if (!Symbol.asyncDispose) throw new TypeError("Symbol.asyncDispose is not defined.");
      dispose = value[Symbol.asyncDispose];
    }
    if (dispose === void 0) {
      if (!Symbol.dispose) throw new TypeError("Symbol.dispose is not defined.");
      dispose = value[Symbol.dispose];
    }
    if (typeof dispose !== "function") throw new TypeError("Object not disposable.");
    env.stack.push({ value: value, dispose: dispose, async: async });
  }
  else if (async) {
    env.stack.push({ async: true });
  }
  return value;
};
```

The first line looks a bit weird:

```javascript
var __addDisposableResource = (this && this.__addDisposableResource) || function (env, value, async) { 
  // ...
}
```

This just prevents an existing `__addDisposableResource` function from being overwritten (like if you've concatenated multiple files containing this transpiler output). So what we're really looking at is:

```javascript
function __addDisposableResource (env, value, async) {
  if (value !== null && value !== void 0) {
    if (typeof value !== "object" && typeof value !== "function") throw new TypeError("Object expected.");
    var dispose;
    if (async) {
      if (!Symbol.asyncDispose) throw new TypeError("Symbol.asyncDispose is not defined.");
      dispose = value[Symbol.asyncDispose];
    }
    if (dispose === void 0) {
      if (!Symbol.dispose) throw new TypeError("Symbol.dispose is not defined.");
      dispose = value[Symbol.dispose];
    }
    if (typeof dispose !== "function") throw new TypeError("Object not disposable.");
    env.stack.push({ value: value, dispose: dispose, async: async });
  }
  else if (async) {
    env.stack.push({ async: true });
  }
  return value;
}
```

The outer control flow handles a few different cases:

```javascript
if (value !== null && value !== void 0) {
  // ...
} else if (async) {
  env.stack.push({ async: true });
}
return value;
```

So if `value` is `null` or `undefined` (`using foo = null`), just return `value`. And if `value` is `null` or `undefined` and `async` is `true` (`await using foo = null`), first push `{ async: true }` to the stack and *then* return `value`.

The inner control flow handles validation of `value` and tries to find the `dispose` clean-up function.

```javascript
if (typeof value !== "object" && typeof value !== 
"function") throw new TypeError("Object expected.");
var dispose;
if (async) {
  if (!Symbol.asyncDispose) throw new TypeError("Symbol.asyncDispose is not defined.");
  dispose = value[Symbol.asyncDispose];
}
if (dispose === void 0) {
  if (!Symbol.dispose) throw new TypeError("Symbol.dispose is not defined.");
  dispose = value[Symbol.dispose];
}
if (typeof dispose !== "function") throw new TypeError("Object not disposable.");
env.stack.push({ value: value, dispose: dispose, async: async });
```

The validation rules are as follows:

* `value` must be an `object` (or a `function` which is also an object).
* If we're trying to attach an asynchronous clean-up function, the runtime must support the new `Symbol.asyncDispose` named `symbol`.
* If we're trying to attach a synchronous clean-up function, the runtime must support the new `Symbol.dispose` named `symbol`.
* If there is a value assigned to `value[Symbol.asyncDispose]` or `value[Symbol.dispose]` it must be a function.

Assuming all the validation passes, then the `value`, the `dispose` function, and the `async` flag are all pushed onto the `stack` as a kind of disposal object. Its shape is something like:

```typescript
export type Disposal = {
  value: unknown;
  dispose: () => void | () => Promise<void>;
  async: boolean;
}
```

> One interesting thing I noticed is that the disposal object always uses `{ value: value, dispose: dispose, async: async }`, even if the transpile target is ES2022. I'd have thought it could be `{ value, dispose, async }` when targeting modern JS. I wonder how many bytes of JS the would save globally?

So this means that we can figure out the type of a disposable resource. It's something like this:

```typescript
export type Disposable<T extends object> = T & ({
  [Symbol.asyncDispose](): Promise<void>; 
} | {
  [Symbol.dispose](): void; 
});
```

So if we try something like this, it should work!

```typescript
const Resource: Disposable<{}> = {
  [Symbol.dispose]() { return }
}

async function useResource () {
  using foo = Resource;
}
```

But if I try this in an IDE we get an error:

![TypeScript Error: The initializer of a 'using' declaration must be either an object with a 'Symbol.dispose' method, or be 'null' or 'undefined'](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hqp823nf9hq5gbuphij5.png)

As usual, TypeScript is right - our object might not have `[Symbol.dispose]`! If we change it to `await using foo = Resource;` it works though - a `Disposable` will definitely have either `[Symbol.dispose]` or `[Symbol.asyncDispose]`. So the typings have to be slightly different depending on whether we use `using` or `await using`. They should be more like:

```typescript
export type Disposable = {
  [Symbol.dispose](): void;
};
export type AsyncDisposable = {
  [Symbol.asyncDispose](): Promise<void>; 
};
```

When using `using`, only a `Disposable` is valid. When using `await using` either a `Disposable | AsyncDispoable` can be assigned.

This is confirmed by the global types defined in the latest TypeScript node types:

```typescript
interface SymbolConstructor {
  /**
   * A method that is used to release resources held by an object. Called by the semantics of the `using` statement.
   */
  readonly dispose: unique symbol;

  /**
   * A method that is used to asynchronously release resources held by an object. Called by the semantics of the `await using` statement.
   */
  readonly asyncDispose: unique symbol;
}

interface Disposable {
  [Symbol.dispose](): void;
}

interface AsyncDisposable {
  [Symbol.asyncDispose](): PromiseLike<void>;
}
```

> Note the use of `PromiseLike` instead of just `Promise`, which means the `dispose` function can return an 3rd-party `Promise` implementation. Also `interface` over `type` lol fight me.

Cool, so we can understand how *defining* a disposable resource works, now onto actually disposing of thing!

### `__disposeResources` 🚮:

We know that the `__disposeResources()` function is called inside the `finally` block that wraps our `using` code. The `finally` block is called at the end of the function execution no matter what, so it can handle errors, and even manipulate the return value of the function.

Here's the full code for `__disposeResources` again:

```javascript
var __disposeResources = (this && this.__disposeResources) || (function (SuppressedError) {
  return function (env) {
    function fail(e) {
      env.error = env.hasError ? new SuppressedError(e, env.error, "An error was suppressed during disposal.") : e;
      env.hasError = true;
    }
    function next() {
      while (env.stack.length) {
        var rec = env.stack.pop();
        try {
          var result = rec.dispose && rec.dispose.call(rec.value);
          if (rec.async) return Promise.resolve(result).then(next, function(e) { fail(e); return next(); });
        }
        catch (e) {
          fail(e);
        }
      }
      if (env.hasError) throw env.error;
    }
    return next();
  };
})(typeof SuppressedError === "function" ? SuppressedError : function (error, suppressed, message) {
  var e = new Error(message);
  return e.name = "SuppressedError", e.error = error, e.suppressed = suppressed, e;
});
```

We can again see the weird overwrite protection:

```javascript
var __disposeResources = (this && this.__disposeResources) || (function (SuppressedError) { 
  // ... 
});
```

This time there's another layer of function calls though, and something called `SuppressedError`:

```javascript
var __disposeResources = (function (SuppressedError) {
  return function (env) {
    // ...
  };
})(typeof SuppressedError === "function" ? SuppressedError : function (error, suppressed, message) {
  var e = new Error(message);
  return e.name = "SuppressedError", e.error = error, e.suppressed = suppressed, e;
});
```

So there's an immediately-invoked function expression (IIFE), which expects something called `SuppressedError`. Again there is some overwrite protection (`typeof SuppressedError === "function" ? SuppressedError : function () { /* ... */ }`), and then the `SuppressedError` definition:

```javascript
function (error, suppressed, message) {
  var e = new Error(message);
  return e.name = "SuppressedError", e.error = error, e.suppressed = suppressed, e;
}
```

It's not immediately clear what this special error type does, but let's keep moving. Our "real" `__disposeResources` function body is this bit:

```javascript
function __disposeResources (env) {
  function fail(e) {
    env.error = env.hasError ? new SuppressedError(e, env.error, "An error was suppressed during disposal.") : e;
    env.hasError = true;
  }
  function next() {
    while (env.stack.length) {
      var rec = env.stack.pop();
      try {
        var result = rec.dispose && rec.dispose.call(rec.value);
        if (rec.async) return Promise.resolve(result).then(next, function(e) { fail(e); return next(); });
      } catch (e) {
        fail(e);
      }
    }
    if (env.hasError) throw env.error;
  }
  return next();
}
```

Looks like we have some [recursion](https://en.wikipedia.org/wiki/Recursion)! Fun 🥳! The `__disposeResources` function has access to the `env` scope, and seems to loop through the `stack`, taking each `Disposal` item from the list, processing it, and then calling `next()`. So what changes the length of `stack`?

We remember that inside `__addDisposableResource` we pushed a new `Disposal` object to the stack:

```javascript
env.stack.push({ value: value, dispose: dispose, async: async })`
```

So if we have:

```typescript
// Before:
const tmpPath = path.join(tmpdir(), `${Math.random()}`);
using file = fs.openSync(tmpPath, 'w+');
const tmpPath2 = path.join(tmpdir(), `${Math.random()}`);
using file2 = fs.openSync(tmpPath2, 'w+');
```

Then we get:

```javascript
// After:
const tmpPath = path.join(tmpdir(), `${Math.random()}`);
const file = __addDisposableResource(env_1, fs.openSync(tmpPath, 'w+'), false);
const tmpPath2 = path.join(tmpdir(), `${Math.random()}`);
const file2 = __addDisposableResource(env_1, fs.openSync(tmpPath2, 'w+'), false);
```

Since `env_1` is shared between both `__addDisposableResource` calls, the stack will also be shared, both `Disposal` objects will be used. Note that since `pop` is used, the resources will be cleaned-up in the reverse order that they were created!

The `next()` function contains what happens for each resouce. Let's look at the synchronous flow first:

```javascript
function next() {
  while (env.stack.length) {
    var rec = env.stack.pop();
    try {
      var result = rec.dispose && rec.dispose.call(rec.value);
    } catch (e) {
      fail(e);
    }
  }
  if (env.hasError) throw env.error;
}
```

Get the next `Disposal` object off the stack, try to call the `dispose` function (if it exists), with the `value` set to `this`. If anything goes wrong with the disposal, call `fail()`.

Even if the clean-up was fine, there could still be an error in our original function (`tmpData`), and `env.hasError` could be true! In that case, that error is throw and has to be handled back by whoever called `tmpData()`. But what happens if `disposal` throws an error? Let's look at `fail()`:

```javascript
function fail(e) {
  env.error = env.hasError ? new SuppressedError(e, env.error, "An error was suppressed during disposal.") : e;
  env.hasError = true;
}
```

Now `SuppressedError` makes a bit more sense! If `disposal` fails, but another error already happened, we construct a new `Error` that wraps both the `disposal` error and the original error!. If `env.hasError` is false, then only `disposal` threw, and just the `disposal` error is used. If multiple `disposal` functions fail, then you would get multiple layers of nested errors!

The `async` flow makes things look complicated:

```javascript
if (rec.async) return Promise.resolve(result).then(next, function(e) { fail(e); return next(); });
```

All this is doing is making sure that all the `disposal` functions are called sequentially. Each clean-up function will be completed before the next one starts, regardless of whether it is `async` or not. I guess that helps makes things a bit more predictable.

## Checking the spec 🔎

So we've looked through all the code, and we have a better understanding of the implementation - it's a good time to check out the actual spec proposal and see if we got it right!

> [The Explicit Resource Management proposal](https://github.com/tc39/proposal-explicit-resource-management)

From looking through, it's pretty much what would be expected from the code! There's a few interesting snippets, like this example footgun which would be fixed with the proposal:

```typescript
// Avoiding common footguns when managing multiple resources:
const a = ...;
const b = ...;
try {
  ...
}
finally {
  a.close(); // Oops, issue if `b.close()` depends on `a`.
  b.close(); // Oops, `b` never reached if `a.close()` throws.
}
```

And there's some syntactic structures I didn't even think of:

```typescript
for (await using x of y) ...

for await (await using x of y) ...
```

The proposal even has [example code](https://github.com/tc39/proposal-explicit-resource-management#using-declarations-with-explicit-local-bindings) that approximates the runtime semantics which pretty much lines up with the TS implementation.

All in all, seems about right ✅ ✅ ✅

## Wrap up 🎁

Cool that was fun! I think this spec seems like a good idea, there's a lot of places where this could be used (and lots of existing Web/Server JS APIs that would benefit from it!), so I hope it makes it into the language.

If you liked reading this, reach out and let me know, or [hit me up online](https://twitter.com/phenomnominal) with question/comments/corrections. 🥰🥰🥰