Arrow functions break JavaScript parsers
Tags: javascript
Published: 2019-06-03 01:25:06

---
title: Arrow functions break JavaScript parsers
published: true
description: Esoteric musings on JavaScript's ambiguous arrow function
tags: javascript, meta
series: Blog-A-Day in June
---

This is an incredibly esoteric post! Enjoy. 🔨🤓

In writing a JavaScript parser in C—which is a post for another day, but you can try it out [via Web Assembly online here](https://bit.ly/2UATDIM)—I've discovered JavaScript's only real ambiguity.

Surprise! It's the arrow function, your favourite shorthand for writing methods and callbacks. A quick reminder of what it looks like:

```js
const method = (arg1, arg2) => {
  console.info('do something', arg1, arg2);
};
const asyncMethodAddNumber = async foo => (await foo) + 123;
```

## Why ➡️ At All?

Arrow functions take the `this` variable of the scope where they were declared. Here's a classic example:

```js
class Foo {
  constructor(things) {
    this.total = 0;
    things.forEach((thing) => this.total += thing);
  }
}
```

If you were to change the above to use `function (thing) { ... }`, accessing `this.total` would fail: `this` wouldn't be set properly. And in general, my advice is to write `() => ...` by _default_. I believe it has the [least surprise](https://www.google.com/search?q=least+surprise).

<small>(Here's [another post](https://dev.to/samthor/declaring-js-variables-in-2019-4572) on sensible defaults in JS, around `let`, `const` and `var`!)</small>

## The Ambiguity

So: in writing a parser, your goal is to work out what each token is, and what sort of statement or expression it makes up. JavaScript's syntax makes this _fairly_ easy, with most cases needing you to look at most one token "forward".

### Easy: Let It Go

Here's an example with `let`. Did you know—`let` is only _sometimes_ a keyword (used to define new variables), and sometimes a valid variable name itself?^

```js
let += 123;  // let is a symbol which I'm adding 123 to
let abc;     // I'm declaring a variable "abc"
```

<small>(note that dev.to's syntax highlighter is wrong here! 🤣)</small>

So `let` is a keyword if:

* you're at the top-level of execution (not inside brackets etc)
   * ... unless you're inside a "for" declaration, e.g.: `for (let ... `
* the next token is a literal (or `[` and `{`, for `let {x,y} = ...`)
* the next token is _NOT_ `in` or `instanceof`
   * ... as `let in foo` asks, is the variable contained in "let" a key of the object "foo"

### Hard: Arrow Functions

But this post is about the humble arrow function! Now, the beginning of an arrow function can take two few different forms. The first is simpler, and trivially determinable as an arrow function:

```js
foo => bar;
async foo => something + await blah;
```

When a parser encounters `foo` (or any named variable), we can look at the next token and ask if it's an arrow `=>`. We can similarly look ahead from `async`, because the only valid interpretation of `async variableName` is the start of an async arrow function. Hooray! 🎊

But in the case of parenthesis, like this `(foo, bar)`, our parser _can't_ know what to do. This could just be a list of expressions: think putting some math into brackets to ensure correct order of evaluation.

Arrow functions are even _more_ ambiguous with a prefix of `async`: because `async` can technically be the name of method call. Yes, that's right, the following JavaScript is valid: 🤮

```js
var y = 123;
var async = (x) => x * 2;  // assign 'async' to a function
console.info(async(y));    // call 'async' as a function!
```

I'll wait for you to copy and paste it into a console. 📥

<small>(again, the syntax highlighter is wrong and says `async` is a keyword! 😂)</small>

## The Solution

There's a couple of solutions. No matter what, we must look forward, _over_ the ambiguous bit. And it's important to remember that this might not be "fast".

Here's a contrived example:

```js
(arg=function() {
  // whole other program could exist here
}) => ...
```

If we want to work out whether the first `(` opens an arrow function, we could parse forward to find the following `=>`. Naïvely, we would then discard all that work and start parsing from the `(` again.

But if we're aiming for speed, we've just thrown away all that "work".

Instead, a better solution is to leave it intentionally ambiguous and come back to it later. The _way_ we parse what's inside the parenthesis–luckily!–doesn't change based on whether it's an arrow function or not. The same tokens, equals signs etc, are all allowed there.

So we could end up with a stream of tokens like this:

```
AMBIG_PAREN
PAREN       (
SYMBOL      que
OP          =
FUNC        function
...
CLOSE       )
ARROW       =>
```

We can now clarify our 'AMBIG_PAREN'—it started an arrow function declaration. This also only happens at most once per "depth" of your program: the _same_ ambiguity could happen inside the whole other program, but it'll be at an increased depth.

## Some Context

^
To be fair, some of JavaScript's ambiguities are solved when running in _strict mode_.

For instance, we can't use `let` as a variable name in this mode. But not all code is written or served this way—and strict mode doesn't change the behavior of `async` or arrow function ambiguity.

### Regular Slashes

There's another fun challenge in JavaScript parsers: whether the humble slash is division, or the start of a regular expression. For example:

```js
function foo() {} / 123 /g
```

Q: While the above code _is_ nonsensical, we have to ask: what does the "divide by 123, divide by g" get parsed as?

A: Turns out—it's a regular expression. This is because a top-level function is a declaration, not an expression. If we surrounded the entire line with `()`, it would be division.

However, unlike arrow functions, this isn't really a problem for a JavaScript parser. When walking left-to-right through code, we can just keep track of what we expect any upcoming slash to be. So it's not ambiguous. 🤷

## Fin

I told you this was esoteric! Thanks for reading this far. Personally, I would like to see JavaScript shed its ambiguities as it evolves, but I think its wide adoption is going to stymie fixing up what is arguably just mildly annoying idiosyncrasies in its syntax. 😄

3 👋