The Tinkering Hobbit (excerpt from The hidden language within the language)
Tags: typescript, metaprogramming, hobbit, underdogs
Published: 2022-04-25 01:35:51

---
description: The Tinkering Hobbit saw something in the edge of the metaphorical vision. Something so clear and yet so elusive, something that was there all along, hidden in plain sight, something that could change everything...
---

The night falls over The Shire and gray smoke billows from the chimneys as every house gets ready for a hot supper and some stories before bed.

Every house except one.

In the distance, off the beaten path, in a strange house (strange by hobbit standards) something else is going on. You could tell it was strange by the plumes of green and purple smoke coming out of its roof, or by the clinks and clanks coming from inside.

This is not an ordinary hobbit house, in here lives...

## The Tinkering Hobbit

Tinky the Tinkering Hobbit is the one responsible for all the cool hobbit contraptions you see out there. Adventuring hobbits have no time to learn how stuff actually _works_.

Back in high-school, some hobbit bullies called Tinky "non-canon" because in the 9 hours of chasing a ring all over the land (extended director's cut), plus some other 9 hours of stealing treasure from a mean dragon, there was absolutely no mention of Tinky.

The Tinkering Hobbit never paid any attention to those comments. Understanding the inner workings of things, that was the ultimate goal.

But Tinky knew understanding was not all, sharing the knowledge was equally important, and so, a community was born, almost like a "fellowship", although they preferred to be called "The Tinkerhood".

For months Tinky has been researching, tinkering, trying, failing, and trying again.

One day after knocking down half a sink with a headbutt and having flux capacitor flash forwards, Tinky saw something in the edge of the metaphorical vision. Something so clear and yet so elusive, something that was there all along, hidden in plain sight, something that could change everything.

But Tinky knew well that the key to truly understanding stuff was sharing the knowledge so a book was born. A book we partially reproduce here for all of you honorary members of The Tinkerhood. A book called...

## The hidden language within the language
_(excerpt from Tinky's book)_

Hi there, my name is Tinky, the Tinkering Hobbit. You know me, I've been tinkering since forever. Today I'd like to tell you about my recent explorations into the world of types, more specifically of _Typescript_.

Some of the rabbit holes in this field are deeper than the deepest pits of Moria, so we'll tread carefully and take it slowly.

Let's start with...

## Unions, really?

Typescript has something called "Union Types". You write union types using a `|` like this:

```ts
type Monster = Orc | GiantSpider;
```

You read this as "a `Monster` is either an `Orc` or a `GiantSpider`".

I'm sure there's an [occult field of magic](https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#working-with-union-types) where calling this a "union" makes sense, but if you are a simple tinkerer like myself, one perhaps familiar with set-union or the bitwise operator `|`, then this will be at least a little confusing.

If you inspect that `Monster` type, you'll see it has readily available only the parts that both an `Orc` _AND_ a `GiantSpider` have in common. Now if you think about sets, a set-union has the stuff that _EITHER_ or _BOTH_ of its components have.

I guess what I'm trying to say is: don't pay much attention to the name. These are called "union types" (even though they work like intersections or something) but it doesn't matter they could've been called something else.

Do know that in this book when we refer to "union types" we are talking about these things and not what you might intuitively think of as "unions".

## An objective view

Have you ever been into [object-oriented programming](https://dev.to/pzavolinsky/those-weird-humans-21ho)? In there you used to have classes and such. Some classes were abstract representations of other classes, like:

```ts
class Monster { /*... */ }

class Orc extends Monster { /* ... */ }
```

A `Monster` is an abstract representation of an `Orc`. Or an `Orc` is a more concrete type of `Monster`.

Now, in OOP, a recurring problem (and cause of great anxiety) is called "downcasting".

Downcasting is the act of taking a generic thing, like a `Monster` that you _know_ (how?) to be a specific thing, like an `Orc` and asserting that this thing is an `Orc`, like this:

```ts
function f(monster: Monster) {
  const orc = monster as Orc; // how do you know this, pray tell?
}
```

Note that the reverse problem, know as "upcasting" or treating something specific as something generic, just works:

```ts
const orc = new Orc();
const monster: Monster = orc;
```

Why are we talking about this, you ask?

Well...

## Narrow and wide

A very similar problem occurs with union types. Let's say we have:

```ts
type Monster = Orc | GiantSpider;
```

And we have a `monster` variable of type `Monster` that we know (?) it's an `Orc` and we need to assert that.

When dealing with union types, what we used to call "downcasting" is now called "narrowing" as in "you take a wider, more generic type and you narrow it into a more specific one".

That being said, narrowing and downcasting are pretty much the same and we could use a similar hack as we did with classes:

```ts
function f(monster: Monster) {
  const orc = monster as Orc; // how do you know this, pray tell?
```

But there are a couple of better options. We could use a [discriminating property](https://www.typescriptlang.org/docs/handbook/2/narrowing.html#discriminated-unions) like this:

```ts
interface Orc {
  type: 'orc';
}
interface GiantSpider {
  type: 'giant-spider';
}
type Monster = Orc | GiantSpider;
```

So that we can later use `monster.type` to check for `'orc'` and get type-checked narrowing:

```ts
function f(monster: Monster) {
  if (monster.type === 'orc') {
    // in here the type of `monster` is `Orc`
  }
}
```

## En garde!

Another, more advanced, narrowing alternative is to use something called "type guards" (also known as [type predicates](https://www.typescriptlang.org/docs/handbook/2/narrowing.html#using-type-predicates)). These are just functions that take monsters and return a fancy boolean stating that you just checked and yeap, this monster is an `Orc`.

```ts
const isOrc = (m: Monster): m is Orc =>
  // some logic here (is smelly, is clumsy, etc.)

// note the `m is Orc` part which is the "fancy boolean"
```

Type guards are the answer to those "how did you know that?" questions above.

To be fair, technically all these strategies for type-checked narrowing are called "type guards" (e.g. type predicates, discriminating properties, the `typeof` operator, etc.), but I grew up calling type predicates "type guards" and old habits die hard.

With all that warm up out of the way, it's time for a crucial choice.

## Red berry vs blue berry

I once read of a powerful wizard called Morpheus that once offered his apprentice N00b a choice between two berries: eat the red berry and your eyes will be open, but there will be no turning back..., or take the blue berry and enjoy blissful ignorance.

Dear reader, I'm giving you the same choice right now. Keep reading at your own peril or choose to walk away now oblivious to the power (and responsibility) ahead.

## Hidden in plain sight

Let's say that we have a function:

```ts
const someFn = (t: number): number =>
  /* do something with `t` */
```

This function takes a single argument `t` of type `number` and returns another `number`.

Now look at this:

```ts
type SomeFn<T> =
  /* do something with `T` */
```

Now squint your eyes and look at both together. Do you see it? Both look like functions, right?

`someFn` is a run-time function while `SomeFn` is a compile-time function!

We could say that `SomeFn` takes a _type_ argument `T` and returns some other _type_ value.

In other words the type alias `SomeFn` acts as a function over types instead of a function over values.

Let's see another example:

```ts
type MakeArray<T> = T[];
```

This `MakeArray` function takes a type argument `T` and returns a new type `T[]`.

So we already have variables like `T` above (capable of holding types) and we have functions (like `MakeArray` and `SomeFn`) capable of producing new types from existing types...

Do you see where I'm going with this?

## The big reveal

We should be able to write pretty complex programs just by combining the elements above (i.e. variables and functions). These are not regular programs, they are programs that operate on the type-system of our "real" (or should I say "other") programs.

We could craft our type-system programs to provide better domain-specific compilation (think DSLs, encode business rules in the type-system, etc.).

But for us to be able to do really cool stuff in this hidden language we need a couple more elements.

## Who types the types?

Did you spot it back in the `SomeFn` example? Here's again all together:

```ts
const someFn = (t: number): number =>
  /* do something with `t` */

type SomeFn<T> =
  /* do something with `T` */
```

See how we know `t` is a `number` but we have no clue what `T` is?

We went to all the trouble of using Typescript because we believe types are important and now that we are coding at the type-system level we have no types?

Outrageous!

...actually, there is a way for us to specify types in our type-system programs:

```ts
type SomeFn<T extends number> =
  /* do something with `T` */
```

So what you do with `:` in run-time land you can do with `extends` in compile-time land.

But what about...

## Narrowing^2

What if we have something like:
```ts
type F<T extends Monster> =
  // do something if T is an Orc
```

How can we narrow `T` from `Monster` to `Orc`? Well, the answer is something called [conditional types](https://www.typescriptlang.org/docs/handbook/2/conditional-types.html) but I like to think of these as good old ternary operators:

```ts
type F<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : // T is not an Orc
```

If you don't want to allow one of the branches of your conditional types you could use the handy type `never` (I shudder to think of a run-time analogy for this):

```ts
type OnlyForOrcs<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : never;
```


OK, that's more like it, we have functions, variables and conditional expressions. What else do we need?

## Unions as lists 

We talked a lot about union types, truth is, for our type-system programs a union can function pretty well as a list. You need to return a bunch of stuff, you could return a union type of that.

Let's see some examples. You're probably familiar with good old `Object.keys`:

```ts
const a = { a: 1, b: true };

const keysOfA = Object.keys(a); // ['a', 'b']
```
`Object.keys` takes an object and gives you back an array of its keys (strings, numbers or symbols).

Ready for another squinting eyes moment? Here it goes:

```ts
interface A { a: number; b: boolean }

type KeysOfA = keyof A; // 'a' | 'b'
```

By now you are an expert at this. `keyof T` is the type-system equivalent to `Object.keys(t)`, but instead of returning an array, it returns a union type.

Let's see the other obvious example, I'll throw it all together this time:

```ts
const a = { a: 1, b: true };

const valuesOfA = Object.values(a); // [1, true]

// -----

interface A { a: number; b: boolean }

type ValuesOfA = A[keyof A]; // number | boolean
```

Clearly there's a pattern here.

## Unions and the Occult

One of my favorite run-time operations over arrays is without a doubt `.map`. Here's a refresher:

```ts
const a = [1, 2];

const b = a.map(value => value % 2 === 0) // [false, true]
```

`.map` takes a function and an array (via `this`), it then applies that function to every element of the array and returns a new array with these mapped values.

When dealing with this type of operations, some people with a magical proclivity like to use a spell called ["The Functor"](https://en.wikipedia.org/wiki/Functor_(functional_programming)). We tinkerers are not usually into magicks so we'll just stick to thinking of `map` as a function that can change every element in an array to something else, preserving order, cardinality, etc.

We said some time ago that union types could double as type-system arrays.

Wouldn't it be great to be able to _map_ over the types wrapped in a union type? Ha, you knew this was coming didn't you?

## Type-system map

The type-system version of `map` (i.e. map over _types_ rather than run-time values) requires a bit more work but it's perfectly doable:

```ts
type Monster = Orc | GiantSpider;

type MakePet<T> = T extends unknown
  ? WashCleanAndDress<T>
  : never;

type GoodMonster = MakePet<Monster>;
  //             =
  //               | WashCleanAndDress<Orc>
  //               | WashCleanAndDress<GiantSpider>
```
Let's go over all of this nonsense step by step!

We start with a `Monster` type that is a union of `Orc` and `GiantSpider` (remember that we think of unions as arrays for now).

Then we create a mapping function that is called `MakePet`. `MakePet` will unpack each monster type from the union, call `WashCleanAndDress` for each type and then neatly pack them back into a union type.

`MakePet` works like `map` because it does the unpacking / packing by exploiting something called [distributive conditional types](https://www.typescriptlang.org/docs/handbook/2/conditional-types.html#distributive-conditional-types).

This essentially means that whenever you do `T extends U ? SomeFn<U> : never` this will not only pattern match `T` and `U` but also, when `T` is a union type, "distribute" `SomeFn` over every type in `T` that extends `U`.

If you combine this knowledge that `extends` distributes with the fact that every type extends `unknown` you get the raw mapping pattern:

```ts
type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never; // won't happen, every T extends unknown
```

If you've been paying attention you might have noticed that the type-system map requires us to bind two type functions:

```ts
type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never; // won't happen, every T extends unknown

type MappingResult = ApplySomeMapping<A | B>;
```

This is necessary because if we tried to inline `A | B` in `ApplySomeMapping` we'd break distribution. Here's why, step by step:

```ts
// step 0 (this one works)
type X = A | B;

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never;

type Result = ApplySomeMapping<X>;

// -----

// step 1 (inline X in ApplySomeMapping, it's broken)
type X = A | B;

type Result = X extends unknown
  ? SomeMapping<X>
  : never;

// -----

// step 2 (inline X in Result to make it obvious it's broken)
type Result = (A | B) extends unknown
  ? SomeMapping<A | B>
  : never;
```

If we tried to merge `Result` with `ApplySomeMapping` we would be pattern matching (and distributing) `X`, but then we'd end up applying `SomeMapping` over the whole `X`. 

If this was run-time, it'd be the same as doing:

```ts
x.map(_ => someMapping(x));
```

When what you wanted probably was:

```ts
x.map(value => someMapping(value));
```

Wow, that was harder on the brain than a honey mead hangover. 

Maybe it's time for a recap...

## Cheatsheet of Typescript type-system programming

Let's summarize all this craziness.

### Functions
```ts
// run-time
const someFn = (t: number) => /* use `t` */

// compile-time
type SomeFn<T extends number> = /* use `T` */
```
### Narrowing
```ts
type Monster = Orc | GiantSpider;

// run-time
if (monster.type === 'orc') {
  // in here the type of `monster` is `Orc`
}
// or
const isOrc = (m: Monster): m is Orc =>
  // some logic here (is smelly, is clumsy, etc.)

// compile-time
type F<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : // T is not an Orc
// or
type OnlyForOrcs<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : never;
```
### Unions as lists
```ts
// run-time
const a = { a: 1, b: true };
const keysOfA = Object.keys(a); // ['a', 'b']
const valuesOfA = Object.values(a); // [1, true]

// compile-time
interface A { a: number; b: boolean }
type KeysOfA = keyof A; // 'a' | 'b'
type ValuesOfA = A[keyof A]; // number | boolean
```
### Mapping over unions
```ts
type Monster = Orc | GiantSpider;

// run-time
const pets = monsters.map(washCleanAndDress);

// compile-time
type MakePet<T> = T extends unknown
  ? WashCleanAndDress<T>
  : never;
type Pets = MakePet<Monster>;
//        =
//          | WashCleanAndDress<Orc>
//          | WashCleanAndDress<GiantSpider>
```

## Only returns void

Cool so now that we have that cheetsheet, it's time for an example and then a challenge.

Let's say I have an object type, and I really don't care about the keys but I do want every value to be a function that returns `void`.

You might be tempted to say:

```ts
type OnlyReturnsVoid = Record<string, () => void>;
```

But what I meant was this:
```ts
interface OnlyReturnsVoid {
  log: (text: string) => void;
  queue: (message: Message) => void;
  start: () => void;
}
```

See how functions in `OnlyReturnsVoid` take different numbers and types of arguments? We want to preserve that (i.e. if I call `onlyReturnsVoid.log(1)` it should complain that `1` is not a string).

How can we assert that future changes to our `OnlyReturnsVoid` don't break the rule? In other words how can we make this change break the build?

```ts
interface OnlyReturnsVoid {
  getLogLevel: () => string; // compilation error!
  log: (text: string) => void;
  queue: (message: Message) => void;
  start: () => void;
}
```

We can approach this problem by writing a type function to validate that a function returns void:

```ts
type IsVoidFn<T> = T extends (...args: any[]) => infer U
  ? U extends void
  ? T // This is OK (`T` is a fn returning void)
  : never  // `T` is a function that returns stuff
  : never // `T` is not a function
```
Note that here we are using `infer`. You can read more about it [here](https://www.typescriptlang.org/docs/handbook/2/conditional-types.html#inferring-within-conditional-types).

Now that we have that validation function we'll use another advanced type-system construct called [mapped types](https://www.typescriptlang.org/docs/handbook/2/mapped-types.html) to map over the keys of `OnlyReturnsVoid` and apply `IsVoidFn` to every value:

```ts
type Validator = {
  [K in keyof OnlyReturnsVoid]: IsVoidFn<OnlyReturnsVoid[K]>
};
//             = {
//                 getLogLevel: never;
//                 log: (text: string) => void;
//                 queue: (message: Message) => void;
//                 start: () => void;
//             };
```

See how our `Validator` type now has `never` for the invalid function? The last trick is to create a new interface that extends from both the `OnlyReturnsVoid` and our `Validator`:

```ts
interface OnlyReturnsVoidGuard extends
  OnlyReturnsVoid,
  Validator {}
```

You can play around with this [here](https://www.typescriptlang.org/play?#code/JYOwLgpgTgZghgYwgAgLIQM4bgcxQb2QAc4BPAGwHs4ATALmQzClB2QF8BYAKB9EliIUAeRDlSAJQhgArlBAYAapWA1k+HsmR4wAGUo5dEAG4RyDABQBKZAF4AfI2asA3MgD075AkoBbIsDkcGDAlCDI0FCUUACE3n6+EODIYAAWwBjI5KAolDJgKZTIvnAA1ijABQDu0aWaWQaWkAAeYAxMLCA4Ng7Ixio0LvUAjjIQY5aJWLgQDOjTeD2O-apD3FpMcFBtyNZ2ywNrXLzcYKREKACSSgMAYiAAPAAqjrbITxGtSTSZFgB0AK2OAwDDgIFIAG0ALpLZCgGDQZAAVXqAH5kZ9ICAfn0Bmj3h4vE90pkMshhABpXYAAye1LhmTgyBg4Sg0jkIFYuNUVnqDBAJkRhOQtPpZKZMBkIAQITCKVSwWQbNk8kyTBkMBgfOQAtMUGFooZOsoBQlUploRAPB4ZwuyEUcGyNGC0Ts6nqEKpoGQ5VIlBg5LEknZquUqihDGuYZo9weonEUhVCmjnqh9h47DWfHA0HgSEDCZDyYGAHEZFs1BAvtiMPV48GkzdVAAaeoOp0u-X4Y5AA).

Now the challenge: can you change this example so that `OnlyReturnsVoid` allows embedded objects that also return void, like this one?

```ts
interface OnlyReturnsVoid {
  getLogLevel: () => string; // compilation error!
  log: (text: string) => void;
  queue: (message: Message) => void;
  service: {
    getCount: () => number; // compilation error!
    start: () => void;
    stop: () => void;
  }
}
```

You can do it [here](https://www.typescriptlang.org/play?#code/JYOwLgpgTgZghgYwgAgLIQM4bgcxQb2QAc4BPAGwHs4ATALmQzClB2QF8BYAKB9EliIUAeRDlSAJQhgArlBAYAapWA1k+HsmR4wAGUo5dEAG4RyDABQBKZAF4AfI2asA3MgD075AkoBbIsDkcGDAlCDI0FCUUACEmshUOJaQAB5gDEwsIDg2DsjGKjQu8QCOMhDllr6Y2HgM6Fi4ELmOBarF3FoY0MbASAwanVra0gDClDLgli3IIDK+AEbQbp7efgFBIWERUFGx8V1gcFDpyNZ2rYUdw06URNMX+VfxXNyvPGCkRCgAkkqFADEQAAeAAqjlsyFBETSEBANAwZwAdCjjjgMAw4CBSABtAC6M1AMGgyAAqvEAPxkmGQeGIto0SlQjxeUEAC2AiM5yGEAGkzgADUEC5DcuDIGDhKDSOQgVhPVRWeIMEAmEks5BCkViiWTBBbcJgNnBZDS2TyRFMGQwGDK2ZqqAarWixEgShgZDimB6g08D5fFCKODkVTBaJ2dTxHH80DIADWEFIlBgPLEkhlFuUqjxDD+WZoQOBonEUnNCnz0bx9h47A6fHA0HgSFTJYz5cKAHEZMc1BBYXT4sX02X-qoADTxIMhmhhx34V5AA). Can you change the validator so that it checks nested objects of arbitrary depth?

_(the excerpt of the book ends here)_

## Final thoughts (and a shameless cliffhanger)

I don't know about you, dear reader, but I really love Tinky's explorations, to the point that I almost consider myself part of "The Tinkerhood".

About Tinky's current adventures little is known but it is said that a powerful wizard atop an evil tower once pointed an eye-shaped telescope in Tinky's direction and got a glimpse of a new book in the making.

The working title for that book appears to be: `Tunneling into the unknown: portaling between run-time and compile-time programs in Typescript`...

I can hardly wait!