The odyssey of asynchronous JavaScript
Tags: javascript, node, codenewbie, beginners
Published: 2019-10-30 16:34:08

# Table of Contents
* [The lead-up] (#lead-up)
* [Events, event-handlers and callbacks](#events-callbacks)
  * [Issue with callbacks](#callback-issues)
* [Promises](#promises)
* [Co-routines](#co-routines-title)
  * [Generators](#generators)
  * [Co-routines](#co-routines)
* [Async/await](#async-await)

## The lead-up <a name="lead-up"></a>
When I was learning JavaScript (about 1-2 years back), my mentor had me go through it step-by-step. So, first I spent some time to get comfortable with callbacks. Then I jumped onto Promises. And then, after months, I started using Async-await.

Due to this reason, I was exposed to a number of flow-control methods and practises that evolved around JavaScript; which I would've missed otherwise - simply because of the fact that I wasn't part of that generation.

Just like how our grandparents complain about how easy our generation has it due to the existence of internet, mobiles, electronic devices, etc. I strongly believe that in the next 2-3 years, we'll complain about how easy the JS-newcomers have it since they do not have to deal with callback-hell and all the other struggles of the _"pre-Promise" era_. To them, it'll probably just be a textbook paragraph on the history of JavaScript that no one really cares about; except for the compulsory 1 mark question that is asked from it.

When I was in college, I had no idea what _'asynchronous'_ meant. Coming from the world of C++, PHP, and Java, the word 'asynchronous' was completely alien to me. I had vague understanding of multi-threading in Java and I dreaded it. I've come a long way from there! 😌

My intention of writing this article is simple.
It is my humble attempt to immortalize the evolution of writing in JavaScript before it's too late and forgotten; in a way that even non-JS people can appreciate it. Even if they don't completely understand the specifics, as they're not familiar with JavaScript constructs, I'm trying to keep it so that they can at least get the general idea.
Yet, if something doesn't make sense, or you want to talk more about it, feel free to reach out.

## Events, event-handlers and callbacks. <a name="events-callbacks"></a>
This is from the pre-historic times.
If you have enough experience, you must've definitely come across event-driven systems - Visual Basic, `OnClickListener()` in Android, `onchange` behavior in HTML, etc.
Since node is primarily an event-based runtime environment, all it had initially was _events_ and _event-handlers_.
Event-handlers are just functions that are triggered once a certain event is fired/emitted. Just like the `onChange` behaviour in HTML.

> _Simply put, it means something that occurs after an unknown amount of time, so don't expect immediate results._

> _For example, "Mom, can I have five dollars?"_

> _Putting my hand out for money is me expecting her to immediately respond by giving me money (synchronous)._

> _Realistically, she will look at me for a moment or two, and then decide to respond when she wants to (asynchronous)._

> — [Kai (StackOverflow)][kai_answer_async]

Due to the asynchronous nature of JS, the system wouldn't wait while, say, you get some data from a database (It was really difficult to wrap my head around and get used to this initially).
{% gist https://gist.github.com/gaurang847/3d845f4c902adf350d50183955e8f19a %}

However, _events_ enable you to put your work on hold when Node.js realises that it is an _async task_ you're performing; and then lets you resume your work when the task has completed and data is available.

{% gist https://gist.github.com/gaurang847/ab7a258fff02398450c701b4ca15575a %}

In JavaScript, functions can be passed as arguments to other functions and functions can return functions. Such functions are called [_higher-order functions_][ho_func] - similar to how a person who manages other people under him is considered to be at a higher level or position.
Thus, a pattern emerged where a function will be passed as the last parameter to an asynchronous function; called a _callback function_.
Under the hood, this function would become the _event-handler_ for the concerned event.

{% gist https://gist.github.com/gaurang847/2309015f37f289f48d9adb3fccce7cd5 %}

#### Issue with callbacks. <a name="callback-issues"></a>
There are hardly any practical applications that may not involve async operations.
The advantage of using Node.js is that time-consuming async operations don't affect the performance of your server. The server won't hold off (or _starve_) one request till another one is completely processed and its response is sent.
As soon as Node.js realizes that an async operation is to be performed, it'll delegate a worker process to handle the operation and immediately start processing the next request. This gives a terrific boost to the speed of the system.
If your server is getting a lot of requests, and each request requires some async operation (say, database queries), this turns out to be significantly efficient.

However, this efficiency came at a great cost. Writing industry-grade applications with just events, event-handlers and callbacks is not easy.
[Callback-hell][callback_hell] is the biggest problem with callbacks that leads to decreased code-extensibility, reusability and manageability.
{% gist https://gist.github.com/gaurang847/1d62d9422476e3bd78993c7d3d10c070 %}
Coming from the object-oriented background of Java, I found it very difficult to get used to writing code involving callbacks - how you have to split the code into a separate function, the callback function.
The struggle was real during that time.

Frustrated by writing asynchronous code with callbacks, developers started finding creative ways to write better, cleaner code.
For example, we used to use [async.io][async_io] at my workplace. It has utility methods like [`async.series()`][async_series], [`async.parallel()`][async_parallel], [`async.waterfall()`][async_waterfall], etc.
`async.waterfall()` is the most interesting one according to me. It lets you chain async functions together so that one function's output is the next function's input - kind of like the human centipede but with functions. 😅

## Promises <a name="promises"></a>
Promises were introduced in [ES6 (2015)][es6-promises]. Until then, people only had callbacks.
Promises were the next step from callbacks. A major step that brought a revolution in the way we worked with Node.js. Consider it the industrial revolution of JavaScript.

> _The Promise object represents the eventual completion (or failure) of an asynchronous operation and its resulting value._
 — [MDN web docs][mdn_promise]

A promise is really just a wrapper around callbacks. An ingenious wrapper where we make a shift from using functions for storing the next code to using an object.
The next function to call (the callback), instead of passing it into a function, we attach it to an object - the promise object.
This object is then responsible to pass the callback function as an event handler to the concerned event.

You can instantiate a promise object from any callback based function. Thus, you can always go from a function-based approach to an object-based one.
{% gist https://gist.github.com/gaurang847/b2191b7a88547aeaa715f6f0be0268ee %}

The significance of this is that your code turns from nested blocks of callbacks to a linear chain of `.then`-ables.
{% gist https://gist.github.com/gaurang847/527617922dc4872eb88555f1cf14c9bb %}
It is lot more easy to make modifications to your code when it is written in linear sequential manner (the very reason we love synchronous code) than when it is written in nested blocks.
Your code instantly becomes readable, predictable and 200x more manageable.

Read this article for more information on Promises:
{% medium https://medium.com/javascript-scene/master-the-javascript-interview-what-is-a-promise-27fc71e77261 %}
If the Promise object sounded like magic, and you are interested in understanding its internal working, you might be interested in [this article][promise_working].

## Co-routines <a name="co-routines-title"></a>
#### Generators <a name="generators"></a>
Generators were introduced in [ES6 (2015)][es6-generators] along with promises. But, I believe not many people know about them or use them often.
They are functions that return _generator objects_.
A generator object is an _iterator_.
An iterator is anything that implements [_the iterator protocol_][iterator_protocol].

The iterator protocol says that an object can be called an _iterator_ if it has the `next()` method that is supposed to do a very specific job; get the next value of iteration/sequence. If you're familiar with [Scanner][java_scanner] in Java, it's an Iterator (Although it breaks [Java design priniciples][java_scanner_design])
{% runkit %}
//Simplest example of a custom iterator
function myIterator(){
    let a = 0;
    return {next: function(){return a++}}
}
let it = myIterator();
console.log(it.next());
console.log(it.next());
console.log(it.next());
{% endrunkit %}
So, a generator object is basically an object that has this `next()` method
And generator functions are just functions that return generator objects. If you've ever used [`xrange()`][xrange] in Python 2.x, that's literally a generator.
A very good example for generators will be a Fibonacci generator.
{% gist https://gist.github.com/gaurang847/c8cf43e22ed956ecff2fc1c13effea6b %}
Read the [Mozilla docs][generator_iterator] for more information on generators and iterators.
Also, this in-detail post on generators on Medium:
{% medium https://medium.com/dailyjs/a-simple-guide-to-understanding-javascript-es6-generators-d1c350551950 %}

#### Co-routines <a name="co-routines"></a>
Now that we know what generators are, we make _coroutines_ simply by adding promises to the mix.
{% gist https://gist.github.com/gaurang847/a6804f916f570333e6fdf1c764c6c2f9 %}
Please note that the code has started looking very similar to its synchronous equivalent. It just needs some supplementary part.
To take care of that, people came up with a few coroutine libraries such as [CO][co].

This part might have been pretty difficult to wrap your head around. It is pretty convoluted. But you might want to read this article if you're interested:
{% medium https://medium.com/front-end-weekly/modern-javascript-and-asynchronous-programming-generators-yield-vs-async-await-550275cbe433 %}

## Async/await <a name="async-await"></a>
Soon, in [ES8 (2017)][es8_async], [async-await][async_await] was announced and that made writing coroutines redundant.
Co-routines died out before they could become a thing. Many people today probably don’t even know about them.

Async-await is just a wrapper around Promises. And again, a promise is just a wrapper around callbacks. So, in reality, promises and async-await are all just glamour. Under the skin, it's still callbacks everywhere!
And yet, JS code now looks so clean, intuitive and manageable, that it's orgasmic!
6 years back, nobody would've imagined we could write such clean code in JavaScript.
{% gist https://gist.github.com/gaurang847/b8deb3befe38434fb977eb78b3d3e54a %}
This code looks exactly similar to the synchronous equivalent. And I’m awe-struck when I think about how much we hated callbacks, and how much we love structure that it led us from callbacks to async-await.
I'm mesmerized by the transitions that happened around Node.js in such short span of time and I needed to talk about it.

Now, the code looks really simple. Write your code using functions and when you're going to perform an async task, just use the `async` and `await` keywords. Anybody can easily write asynchronous code in JavaScript now.
But some times, things don't work as expected. Things that look simple and straight-forward often give unexpected results. And without enough understanding of the problem and the inherent system, one can go nuts in the process of debugging such errors.
Happened with me once.
{% twitter 1120390303440044033 %}{% twitter 1120390582239682560 %}
My mentor probably understood that well. And that is why, he set me up on this journey to find and feel the true essence of Node.js.

JS-veterans, if you find any inconsistencies in this piece, or would like to add more. Or simply want to talk, feel free to comment or DM me.
JS-newbies and JS-virgins, I hope I've sparked an interest for the JS community in your minds. Feel free to reach out in case of any doubts.

[ho_func]: https://hackernoon.com/effective-functional-javascript-first-class-and-higher-order-functions-713fde8df50a
[callback_hell]: http://callbackhell.com
[es6-promises]: https://www.ecma-international.org/ecma-262/6.0/#sec-promise-objects
[mdn_promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
[promise_working]: https://www.mattgreer.org/articles/promises-in-wicked-detail/
[es6-generators]: https://www.ecma-international.org/ecma-262/6.0/#sec-generator-objects
[iterator_protocol]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols
[java_scanner_design]: https://stackoverflow.com/questions/31153006/why-does-scanner-implement-iteratorstring
[java_scanner]: https://docs.oracle.com/javase/7/docs/api/java/util/Scanner.html
[xrange]: https://www.geeksforgeeks.org/range-vs-xrange-python/
[generator_iterator]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators
[co]: https://github.com/tj/co
[es8_async]: https://www.ecma-international.org/ecma-262/8.0/#sec-async-function-definitions
[async_await]: https://javascript.info/async-await
[async_io]: https://caolan.github.io/async/v3/
[async_series]: https://caolan.github.io/async/v3/docs.html#series
[async_parallel]: https://caolan.github.io/async/v3/docs.html#parallel
[async_waterfall]: https://caolan.github.io/async/v3/docs.html#waterfall
[kai_answer_async]: https://stackoverflow.com/a/4559139/5968979