Mastering Type-Safe JSON Serialization in TypeScript
Tags: typescript, webdev, frontend, node
Published: 2024-02-26 19:01:21

Almost every web application requires data serialization. This need arises in situations like:

- Transferring data over the network (e.g. HTTP requests, WebSockets)
- Embedding data in HTML (for hydration, for instance)
- Storing data in a persistent storage (like LocalStorage)
- Sharing data between processes (like web workers or postMessage)

In many cases, data loss or corruption can lead to serious consequences, making it essential to provide a convenient and safe serialization mechanism that helps detect as many errors as possible during the development stage. For these purposes, it's convenient to use JSON as the data transfer format and TypeScript for static code checking during development.

TypeScript serves as a superset of JavaScript, which should enable the seamless use of functions like `JSON.stringify` and `JSON.parse`, right? Turns out, despite all its benefits, TypeScript doesn't naturally understand what JSON is and which data types are safe for serialization and deserialization into JSON. Let's illustrate this with an example.

## The Problem with JSON in TypeScript

Consider, for example, a function that saves some data to LocalStorage. As LocalStorage cannot store objects, we use JSON serialization here:

```tsx
interface PostComment {
  authorId: string;
  text: string;
  updatedAt: Date;
}

function saveComment(comment: PostComment) {
    const serializedComment = JSON.stringify(comment);
    localStorage.setItem('draft', serializedComment);
}
```

We will also need a function to retrieve the data from LocalStorage. 

```tsx
function restoreComment(): PostComment | undefined {
    const text = localStorage.getItem('draft');
    return text ? JSON.parse(text) : undefined;
}
```

What’s wrong with this code? The first problem is that when restoring the comment, we will get a `string` type instead of `Date` for the `updatedAt` field. 

This happens because JSON only has four primitive data types (`null`, `string`, `number`, `boolean`), as well as arrays and objects. It is not possible to save a `Date` object in JSON, as well as other objects that are found in JavaScript: functions, Map, Set, etc.

When `JSON.stringify` encounters a value that cannot be represented in JSON format, type casting occurs. In the case of a `Date` object, we get a string because the `Date` object implements the [toJson()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toJSON) method, which returns a string instead of a `Date` object.

```tsx
const date = new Date('August 19, 1975 23:15:30 UTC');

const jsonDate = date.toJSON();
console.log(jsonDate);
// Expected output: "1975-08-19T23:15:30.000Z"

const isEqual = date.toJSON() === JSON.stringify(date);
console.log(isEqual);
// Expected output: true
```

The second problem is that the `saveComment` function returns the `PostComment` type, in which the date field is of type `Date`. But we already know that instead of `Date`, we will actually receive a `string` type. TypeScript could help us find this error, but why doesn't it?

Turns out, in TypeScript's standard library, the `JSON.parse` function is typed as `(text: string) => any`. Due to the use of `any`, type checking is essentially disabled. In our example, TypeScript simply took our word that the function would return a `PostComment` containing a `Date` object.

This TypeScript behavior is inconvenient and unsafe. Our application may crash if we try to treat a string like a `Date` object. For example, it might break if we call `comment.updatedAt.toLocaleDateString()`.

Indeed, in our small example, we could simply replace the `Date` object with a numerical timestamp, which works well for JSON serialization. But in real applications, data objects might be extensive, types can be defined in multiple locations, and identifying such an error during development may be a challenging task.

What if we could enhance TypeScript's understanding of JSON?

## Dealing with Serialization

To start with, let's figure out how to make TypeScript understand which data types can be safely serialized into JSON. Suppose we want to create a function `safeJsonStringify`, where TypeScript will check the input data format to ensure it's JSON serializable.

```tsx
function safeJsonStringify(data: JSONValue) {
    return JSON.stringify(data);
}
```

In this function, the most important part is the `JSONValue` type, which represents all possible values that can be represented in the JSON format. The implementation is quite straightforward:

```tsx
type JSONPrimitive = string | number | boolean | null | undefined;

type JSONValue = JSONPrimitive | JSONValue[] | {
    [key: string]: JSONValue;
};
```

First, we define the `JSONPrimitive` type, which describes all the primitive JSON data types. We also include the `undefined` type based on the fact that when serialized, keys with the `undefined` value will be omitted. During deserialization, these keys will simply not appear in the object, which in most cases is the same thing.

Next, we describe the `JSONValue` type. This type uses TypeScript's ability to describe recursive types, which are types that refer to themselves. Here, `JSONValue` can either be a `JSONPrimitive`, an array of `JSONValue`, or an object where all values are of the `JSONValue` type. As a result, a variable of type `JSONValue` can contain arrays and objects with unlimited nesting. The values within these will also be checked for compatibility with the JSON format.

Now we can test our `safeJsonStringify` function using the following examples:

```tsx
// No errors
safeJsonStringify({
    updatedAt: Date.now()
});

// Yields an error:
// Argument of type '{ updatedAt: Date; }' is not assignable to parameter of type 'JSONValue'.
//   Types of property 'updatedAt' are incompatible.
//     Type 'Date' is not assignable to type 'JSONValue'.
safeJsonStringify({
    updatedAt: new Date();
});
```

Everything seems to function properly. The function allows us to pass the date as a number, but yields an error if we will pass the `Date` object.

But let's consider a more realistic example, in which the data passed to the function is stored in a variable and has a described type.

```tsx
interface PostComment {
    authorId: string;
    text: string;
    updatedAt: number;
};

const comment: PostComment = {...};

// Yields an error:
// Argument of type 'PostComment' is not assignable to parameter of type 'JSONValue'.
//   Type 'PostComment' is not assignable to type '{ [key: string]: JSONValue; }'.
//     Index signature for type 'string' is missing in type 'PostComment'.
safeJsonStringify(comment);

```

Now, things are getting a bit tricky. TypeScript won't let us assign a variable of type `PostComment` to a function parameter of type `JSONValue`, because "Index signature for type 'string' is missing in type 'PostComment'".

So, what is an index signature and why is it missing? Remember how we described objects that can be serialized into the JSON format?

```tsx
type JSONValue = {
    [key: string]: JSONValue;
};
```

In this case, `[key: string]` is the index signature. It means "this object can have any keys in the form of strings, the values of which have the `JSONValue` type". So, it turns out we need to add an index signature to the `PostComment` type, right?

```tsx
interface PostComment {
    authorId: string;
    text: string;
    updatedAt: number;

    // Don't do this:
    [key: string]: JSONValue;
};
```

In fact, doing so would imply that the comment could contain any arbitrary fields, which is not typically the desired outcome when defining data types in an application.

The real solution to the problem with the index signature comes from [Mapped Types](https://www.typescriptlang.org/docs/handbook/2/mapped-types.html), which allow to recursively iterate over fields, even for types that don't have an index signature defined. Combined with generics, this feature allows converting any data type `T` into another type `JSONCompatible<T>`, which is compatible with the JSON format.

```tsx
type JSONCompatible<T> = unknown extends T ? never : {
    [P in keyof T]:
        T[P] extends JSONValue ? T[P] :
        T[P] extends NotAssignableToJson ? never :
        JSONCompatible<T[P]>;
};

type NotAssignableToJson =
    | bigint
    | symbol
    | Function;
```

The `JSONCompatible<T>` type is a mapped type that inspects whether a given type `T` can be safely serialized into JSON. It does this by iterating over each property in type `T` and doing the following:

1. The `T[P] extends JSONValue ? T[P] : ...` conditional type verifies if the property's type is compatible with the `JSONValue` type, assuring it can be safely converted to JSON. When this is the case, the property's type remains unchanged.
2. The `T[P] extends NotAssignableToJson ? never : ...` conditional type verifies if the property’s type isn't assignable to JSON. In this case, the property's type is converted to `never`, effectively filtering the property out from the final type.
3. If neither of these conditions is met, the type is recursively checked until a conclusion can be made. This way it works even if the type doesn’t have an index signature.

The `unknown extends T ? never :...` check at the beginning is used to prevent the `unknown` type from being converted to an empty object type `{}`, which is essentially equivalent to the `any` type.

Another interesting aspect is the `NotAssignableToJson` type. It consists of two TypeScript primitives (bigint and symbol) and the `Function` type, which describes any possible function. The `Function` type is crucial in filtering out any values which aren't assignable to JSON. This is because any complex object in JavaScript is based on the Object type and has at least one function in its prototype chain (e.g., `toString()`). The `JSONCompatible` type iterates over all of those functions, so checking functions is sufficient to filter out anything that isn't serializable to JSON.

Now, let's use this type in the serialization function:

```tsx
function safeJsonStringify<T>(data: JSONCompatible<T>) {
    return JSON.stringify(data);
}
```

Now, the function uses a generic parameter `T` and accepts the `JSONCompatible<T>` argument. This means it takes an argument `data` of type `T`, which should be a JSON compatible type. Now we can use the function with data types without an index signature.

The function now uses a generic parameter `T` that extends from the `JSONCompatible<T>` type. This means that it accepts an argument `data` of type `T`, which ought to be a JSON compatible type. As a result, we can utilize the function with data types that lack an index signature.

```tsx
interface PostComment {
  authorId: string;
  text: string;
  updatedAt: number;
}

function saveComment(comment: PostComment) {
    const serializedComment = safeJsonStringify(comment);
    localStorage.setItem('draft', serializedComment);
}
```

This approach can be used whenever JSON serialization is necessary, such as transferring data over the network, embedding data in HTML, storing data in localStorage, transferring data between workers, etc. Additionally, the `toJsonValue` helper can be used when a strictly typed object without an index signature needs to be assigned to a variable of `JSONValue` type.

```tsx
function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
    return value;
}

const comment: PostComment = {...};

const data: JSONValue = {
    comment: toJsonValue(comment)
};
```

In this example, using `toJsonValue` lets us bypass the error related to the missing index signature in the `PostComment` type.

## Dealing with Deserialization

When it comes to deserialization, the challenge is both simpler and more complex simultaneously because it involves both static analysis checks and runtime checks for the received data's format.

From the perspective of TypeScript's type system the challenge is quite simple. Let's consider the following example:

```tsx
function safeJsonParse(text: string) {
    return JSON.parse(text) as unknown;
}

const data = JSON.parse(text);
//    ^?  unknown

```

In this instance, we're substituting the `any` return type with the `unknown` type. Why choose `unknown`? Essentially, a JSON string could contain anything, not just the data what we expect to receive. For example, the data format might change between different application versions, or another part of the app could write data to the same LocalStorage key. Therefore, `unknown` is the safest and most precise choice.

However, working with the `unknown` type is less convenient than merely specifying the desired data type. Apart from type-casting, there are multiple ways to convert the `unknown` type into the required data type. One such method is utilizing the [Superstruct](https://docs.superstructjs.org/) library to validate data at runtime and throw detailed errors if the data is invalid.

```tsx
import { create, object, number, string } from 'superstruct';

const PostComment = object({
    authorId: string(),
    text: string(),
    updatedAt: number(),
});

// Note: we no longer need to manually specify the return type
function restoreDraft() {
    const text = localStorage.getItem('draft');
    return text ? create(JSON.parse(text), PostComment) : undefined;
}
```

Here, the `create` function acts as a type guard, [narrowing](https://www.typescriptlang.org/docs/handbook/2/narrowing.html#using-type-predicates) the type to the desired `Comment` interface. Consequently, we no longer need to manually specify the return type.

Implementing a secure deserialization option is only half the story. It's equally crucial not to forget to use it when tackling the next task in the project. This becomes particularly challenging if a large team is working on the project, as ensuring all agreements and best practices are followed can be difficult.

[Typescript-eslint](https://typescript-eslint.io/) can assist in this task. This tool helps identify all instances of unsafe `any` usage. Specifically, all usages of `JSON.parse` can be found and it can be ensured that the received data's format is checked. More about getting rid of the `any` type in a codebase can be read in the article [Making TypeScript Truly "Strongly Typed"](https://dev.to/nodge/making-typescript-truly-strongly-typed-4fd6).

## Conclusion

Here are the final utility functions and types designed to assist in safe JSON serialization and deserialization. You can test these in the prepared [TS Playground](https://www.typescriptlang.org/play?target=99&useUnknownInCatchVariables=true&ts=5.3.3#code/C4TwDgpgBAUgygeQHIAUBOBLAth4GBu0AvFAM7CYB2A5lAD5SUCuWARhGvVKwPY8A2EAIaUuzfvy5NKAEwgAzDJQgyA3ACh1oSLERIAakP5Niu5Omy4C0BvGSHjEANoBdLgG91Ub1CcBrCBAALjIKJWoXELsDIxMNAF8NLXBoJB5gAEFSUgxqSiFWQQAVHhhSHlESLx8GVlylYChq7wZSEDYBJp8uADFpAGM8CqTtaGiAYR4sMCE8QogAHiKAPigSaT9KHgB3UQgAD2AIWVIoIqgAfkYIQk4Qz26nFCglKACQHnkzyK7unyKnm4DkcTmYYo5LmdAVAQs0-gCUEDDscZKc0plsrl8vMSmUKpDlLcYb8-mDJtNZhh5ktAcsEiMUmcIOQEKwAFYQQZrMlTGZzQQLdxQeR8EIAclYQjQYtUUBkswgIQAIgqoPE6eoAPSanwAPQumlGTPIFhweEI3ImvMp1PIVGoGu1eoNyR0RWZwAAqpRNjtKjyKfzFhstrtHTrvPrDYz3eRvb7dugeGBLXpyXyqQKhUIQiG-Wrw87NE75AMhqJSEJ5BA8ZQ4GEaBh5CAABTy4A5sEOEwASignid3TQEGATDQomiADo7eEm6320IexonfFNKXKIMMPjK9Xa-X7XOllBgSjTlbA5nFitlm3Zp2in2Hj5h6Px2Dpw3qHPbx2l+pV+oO41uUdaft+T7eCKPDipK0oADT-n+Wo6kBe5gc2LYDhGPhMGA7YqBkwAhMo2xQCqRwtj2yFqkhDQcPIQj9NAKA8OQ5JYMcjQQVAQhMMAAAWPBoAAkjIIQzjQGgAJBHIc4mfho3S4fhMiEcRLDsGg9KaHI-T8FK0D9BU5BQEZWAcZQRFQCxbFTBZwAaKhIH7rOGFmfZSHrpu+LAKUIHdlex7IqC54ZtS14tvgsSKmcPZRHoAX9nCL5jqIUWOAkQA).

```tsx
type JSONPrimitive = string | number | boolean | null | undefined;

type JSONValue = JSONPrimitive | JSONValue[] | {
    [key: string]: JSONValue;
};

type NotAssignableToJson = 
    | bigint 
    | symbol 
    | Function;

type JSONCompatible<T> = unknown extends T ? never : {
    [P in keyof T]: 
        T[P] extends JSONValue ? T[P] : 
        T[P] extends NotAssignableToJson ? never : 
        JSONCompatible<T[P]>;
};

function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
    return value;
}

function safeJsonStringify<T>(data: JSONCompatible<T>) {
    return JSON.stringify(data);
}

function safeJsonParse(text: string): unknown {
    return JSON.parse(text);
}
```

These can be used in any situation where JSON serialization is necessary.

I've been using this strategy in my projects for several years now, and it has demonstrated its effectiveness by promptly detecting potential errors during application development.

I hope this article has provided you with some fresh insights. Thank you for reading!

## Useful Links

- [Making TypeScript Truly "Strongly Typed"](https://dev.to/nodge/making-typescript-truly-strongly-typed-4fd6)
- [Improving TypeScript’s Standard Library Types](https://dev.to/nodge/unleashing-the-power-of-typescript-improving-standard-library-types-47i0)
- [The superstuct library](https://docs.superstructjs.org/)
- [TS playground](https://www.typescriptlang.org/play?target=99&useUnknownInCatchVariables=true&ts=5.3.3#code/C4TwDgpgBAUgygeQHIAUBOBLAth4GBu0AvFAM7CYB2A5lAD5SUCuWARhGvVKwPY8A2EAIaUuzfvy5NKAEwgAzDJQgyA3ACh1oSLERIAakP5Niu5Omy4C0BvGSHjEANoBdLgG91Ub1CcBrCBAALjIKJWoXELsDIxMNAF8NLXBoJB5gAEFSUgxqSiFWQQAVHhhSHlESLx8GVlylYChq7wZSEDYBJp8uADFpAGM8CqTtaGiAYR4sMCE8QogAHiKAPigSaT9KHgB3UQgAD2AIWVIoIqgAfkYIQk4Qz26nFCglKACQHnkzyK7unyKnm4DkcTmYYo5LmdAVAQs0-gCUEDDscZKc0plsrl8vMSmUKpDlLcYb8-mDJtNZhh5ktAcsEiMUmcIOQEKwAFYQQZrMlTGZzQQLdxQeR8EIAclYQjQYtUUBkswgIQAIgqoPE6eoAPSanwAPQumlGTPIFhweEI3ImvMp1PIVGoGu1eoNyR0RWZwAAqpRNjtKjyKfzFhstrtHTrvPrDYz3eRvb7dugeGBLXpyXyqQKhUIQiG-Wrw87NE75AMhqJSEJ5BA8ZQ4GEaBh5CAABTy4A5sEOEwASignid3TQEGATDQomiADo7eEm6320IexonfFNKXKIMMPjK9Xa-X7XOllBgSjTlbA5nFitlm3Zp2in2Hj5h6Px2Dpw3qHPbx2l+pV+oO41uUdaft+T7eCKPDipK0oADT-n+Wo6kBe5gc2LYDhGPhMGA7YqBkwAhMo2xQCqRwtj2yFqkhDQcPIQj9NAKA8OQ5JYMcjQQVAQhMMAAAWPBoAAkjIIQzjQGgAJBHIc4mfho3S4fhMiEcRLDsGg9KaHI-T8FK0D9BU5BQEZWAcZQRFQCxbFTBZwAaKhIH7rOGFmfZSHrpu+LAKUIHdlex7IqC54ZtS14tvgsSKmcPZRHoAX9nCL5jqIUWOAkQA)