The Journey of JavaScript: from Downloading Scripts to Execution - Part I
Tags: javascript, scripts, parsing
Published: 2019-02-25 11:31:07

---
title: The Journey of JavaScript: from Downloading Scripts to Execution - Part I
published: true
tags: javascript, scripts, parsing 
canonical_url: https://www.telerik.com/blogs/journey-of-javascript-downloading-scripts-to-execution-part-i
---

This article will help you understand the internals of JavaScript - even the weird parts. Every line of code that you write in JavaScript will make complete sense once you know how it has been interpreted by the underlying engine. You'll learn multiple ways of downloading scripts based on the use case, and how the parser generates an Abstract Syntax Tree and its heuristics while parsing the code. Let's dive deep into the internals of JavaScript engines - starting from downloading scripts.

JavaScript is one of the most popular languages today. Gone are the days when people would use JavaScript merely for handling DOM event listeners and for a few undemanding tasks. Today, you can build an entire application from the ground up using JavaScript. JavaScript has taken over the winds, lands and the seas. With Node.js invading the gamut of server-side technologies and the advent of rich and powerful client-side libraries and frameworks like React, Angular and Vue, JavaScript has conquered the web. Applications are shipping a lot of JavaScript over the wires. Almost all of the complicated tasks of an application are now implemented using JavaScript.

While this is all great, it is disheartening to see that most of these applications lack even the minimal user experience. We keep adding functionalities to our application without taking into effect its performance implications. It is important that we follow proper techniques to deliver optimized code.

In this series of tutorials, we’ll first understand what is wrong with the conventional techniques and then we’ll dig deeper to learn some of the techniques that’ll help us write optimized code. We’ll also understand how our code gets parsed, interpreted and compiled by the underlying JavaScript engine and what works best for our engines. While the syntax of JavaScript is pretty easy to grasp, understanding its internals is a more daunting task. We’ll start from the very basics and eventually take over the beast. Let’s get going.

## Understanding the Script Tag

Let’s consider a simple HTML file:

```
<!DOCTYPE html>
<html>
    <head>
        <script src='./js/first.js'></script>
        <script src='./js/second.js'></script>
        <script src='./js/third.js'></script>
        <script src='./js/fourth.js'></script>
    </head>
    <body>
        <div>Understanding the script tag</div>
    </body>
</html>

```

first.js includes the following code:

```
console.log('first.js file')

```

second.js includes the following code:

```
console.log('second.js file')

```

I’ve set up an express server for demonstrating the concepts explained in the article. If you want to experiment along the way, please feel free to clone my [GitHub repository](https://github.com/ankita1910/js-internals).

Let’s see what happens when we open this HTML file in the browser:

 ![loading-scripts](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/loading-scripts.png?sfvrsn=9c36ac48_1 "loading-scripts") 

The browser starts parsing the HTML code. When it comes across a script tag in the head section, the HTML parsing is paused. An HTTP request is sent to the server to fetch the script. The browser waits until the entire script is downloaded. It then does the work of parsing, interpreting and executing the downloaded script (we’ll get into the details of the entire process later in the article). This happens for each of the four scripts.

Once this is done, the browser resumes its work of parsing HTML and creating DOM nodes. The user, who is patiently staring at the screen waiting for something to load, doesn’t know most of his time is spent executing JavaScript code (even the code that may not be required during the startup). Script tags are blocking in nature. They block the rendering of the DOM. Your high school teacher might have told you, “Always put the script tags below body.” Now that you know script tags block rendering of the DOM, it makes sense to put them below the HTML. It is better to show non-interactive content (for a few milliseconds until the JavaScript code gets ready) than nothing at all.

Imagine you have a very big chain of DOM nodes — tens of thousands of them. According to what we’ve learned so far, in this case, the user would see a lot of content but he won’t be able to interact even with the tiniest piece. I’m sure you have visited websites that show you the entire content almost instantly but don’t let you scroll down or even click on any element. The page doesn’t seem to move for a few seconds. Isn’t that frustrating? The next obvious question is: when should we load the scripts — at the start before parsing of HTML or at the end after the HTML? Let’s analyze the problem a bit more.

Our end goal is clear — to load assets instantly during the startup. Our first approach of parsing scripts first and then the HTML renders a good user experience, but it eats up a lot of the user’s time by showing him blank screen while the content is getting executed. The problem with this approach is that it gets worse with an increase in the number of scripts since waiting time (load time) is directly proportional to the number of scripts. For every script, we make a ride to the server and wait until it gets downloaded.

Can we dump all of the JavaScript code in one file? This would reduce the number of rides we make to the server. That would mean dumping tens of thousands of lines of JavaScript into one file. I’m definitely not going for this. This would mean compromising with my code ethics.

Heard of Gulp, webpack? They are nothing but module bundlers in simple terms. Module bundlers, eh? You write your JavaScript code in any number of files (as many modules as you wish). Module bundlers bundle all of your JavaScript files and static assets in one big chunk, and you can simply add this one big file in your HTML.

Certainly, we reduced the number of HTTP requests to the server. Are we not still downloading, parsing and executing the entire content? Can we do something about it? There’s something called as code splitting. With webpack, you can split your code into different bundles. Dump all the common code in one bundle (like Vendor.js, which has all the common libraries to be used across the project) and others that are specific to modules.

For example, let’s say you are building an eCommerce website. You have different modules for Store, Transactions History and Payment. It doesn’t make sense to load your payment-specific code on the store-specific page. Bundlers have solved our problem by making fewer HTTP requests to the server.

Now, let’s consider one use case here. I’ve added Google Analytics to gain insights into how users are interacting with my eCommerce website. Google Analytics script is not required during the startup. We may want to load the app-specific stuff first and then other secondary scripts.

## Downloading Scripts Asynchronously

When you add the _async_ keyword in your script tag, the browser downloads that script asynchronously. The browser doesn’t pause the parsing of DOM when it comes across a script tag with _async_ keyword. The script is downloaded in another thread without disturbing the main thread, and, once it is downloaded, the browser pauses the parsing of HTML and gets busy in parsing this script code. Once the parsing of this JavaScript code is completed, it is executed in another thread and the browser resumes its work of parsing HTML. We’ve saved the waiting time of the browser while the script is getting downloaded.

Let’s say we want to download two of our scripts asynchronously:

```
<!DOCTYPE html>
<html>
    <head>
        <script async src='./js/first.js'></script>
        <script async src='./js/second.js'></script>
        <script src='./js/third.js'></script>
        <script src='./js/fourth.js'></script>
    </head>
    <body>
        <div>Understanding the script tag</div>
    </body>
</html>
```

## Deferring the Execution of Scripts

When you add _defer_ keyword in your script tag, the browser doesn’t execute that script until the HTML parsing is completed. Defer simply means the execution of the file is deferred or delayed. The script is downloaded in another thread and is executed only after the HTML parsing is completed.

```
<!DOCTYPE html>
<html>
    <head>
        <script defer src='./js/first.js'></script>
        <script defer src='./js/second.js'></script>
        <script src='./js/third.js'></script>
        <script src='./js/fourth.js'></script>
    </head>
    <body>
        <div>Understanding the script tag</div>
    </body>
</html>
```

![defer-scripts](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/defer-scripts.png?sfvrsn=8df6a065_1 "defer-scripts")

As we can see in the above screenshot, third.js and fourth.js were executed before first.js and second.js.

Here’s a brief overview of the three techniques of adding scripts:

![comparison](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/comparison.jpg?sfvrsn=63483a10_1 "comparison")

Until now, we’ve understood how scripts are downloaded and what the most effective ways of downloading scripts are. Let’s understand what happens after a script is downloaded. (We’re considering Chrome browser, although almost all of the popular browsers follow similar steps.)

Chrome uses V8 as the underlying JavaScript Engine. It consists of the following components.

![js-engine](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/js-engine.png?sfvrsn=dc2f64eb_1 "js-engine")

1. _Parser_ - JavaScript is fed into a Parser, which generates an Abstract Syntax Tree
2. _Interpreter_ - Abstract Syntax Tree is the input for the V8 Ignition Interpreter, which generates the ByteCode
3. _Compiler_ - The Turbofan Compiler of the V8 Engine takes in the ByteCode and generates machine code
4. _Optimizing Compiler_ - It takes ByteCode and some profiling data as the input and generates optimized machine code

We’ll get into the details of each of these components.

## Parsing JavaScript Code

The JavaScript source code is first converted to tokens. Tokens represent the alphabet of a language. Every unit in the source code is identified by the grammar of the language that you’re using.

So, something like `var a = 1` is a valid JavaScript statement. It can be broken down to tokens (‘var’, ‘a’, ‘=’, ‘1’) that match with the language grammar. However, something like `variable a = 2` is not a valid JavaScript statement because its grammar doesn’t specify anything related to the _variable_ keyword. Now, with the help of these tokens, the parser generates an Abstract Syntax Tree (AST) and scopes. AST, in simple terms, is a data structure that is used for representing the source code. Scopes are also data structures, used for identifying the scope of variables in their defined blocks. For example, a local variable would be accessible in the local scope and not in global scope. These constraints are defined in these scopes data structures.

Consider this simple JavaScript code snippet -  
  
`var a = 2`  
  
I refer [AST Explorer](https://astexplorer.net/) to check the AST generated for my code. The AST for the above code would look something like this:

```
{
  "type": "Program",
  "start": 0,
  "end": 9,
  "body": [
    {
      "type": "VariableDeclaration",
      "start": 0,
      "end": 9,
      "declarations": [
        {
          "type": "VariableDeclarator",
          "start": 4,
          "end": 9,
          "id": {
            "type": "Identifier",
            "start": 4,
            "end": 5,
            "name": "a"
          },
          "init": {
            "type": "Literal",
            "start": 8,
            "end": 9,
            "value": 2,
            "raw": "2"
          }
        }
      ],
      "kind": "var"
    }
  ],
  "sourceType": "module"
}

```

Let’s try to make sense of the above AST. It’s a JavaScript object with properties as _type_, _start_, _end_, _body_ and _sourceType_. _start_ is the index of the first character, and _end_ is the length of your code, which is `var a = 2` in this case. _body_ contains the definition of the code. It’s an array with a single object since there is only one statement of the type `VariableDeclaration` in our program. Inside `VariableDeclaration`, it specifies the identifier `a` and its initial value as `2`. Check `id` and `init` objects. The kind of declaration is `var`. It can also be `let` or `const`.

Let’s consider one more example to get better understanding of ASTs:

```
function foo () {
    let bar = 2
    return bar
}

```

And its AST is as follows -

```
{
  "type": "Program",
  "start": 0,
  "end": 50,
  "body": [
    {
      "type": "FunctionDeclaration",
      "start": 0,
      "end": 50,
      "id": {
        "type": "Identifier",
        "start": 9,
        "end": 12,
        "name": "foo"
      },
      "expression": false,
      "generator": false,
      "params": [],
      "body": {
        "type": "BlockStatement",
        "start": 16,
        "end": 50,
        "body": [
          {
            "type": "VariableDeclaration",
            "start": 22,
            "end": 33,
            "declarations": [
{
                "type": "VariableDeclarator",
                "start": 26,
                "end": 33,
                "id": {
                  "type": "Identifier",
                  "start": 26,
                  "end": 29,
                  "name": "bar"
                },
                "init": {
                  "type": "Literal",
                  "start": 32,
                  "end": 33,
                  "value": 2,
                  "raw": "2"
                }
}
            ],
            "kind": "let"
          },
          {
            "type": "ReturnStatement",
            "start": 38,
            "end": 48,
            "argument": {
"type": "Identifier",
"start": 45,
"end": 48,
"name": "bar"
            }
          }
        ]
      }
    }
  ],
  "sourceType": "module"
}

```

Again, it has properties — _type_, _start_, _end_, _body_ and _sourceType_. _start_ is 0, which means the first character is at position 0, and _end_ is 50, which means the length of the code is 50. _body_ is an array with one object of the type `FunctionDeclaration`. The name of the function `foo` is specified in the `id` object. This function doesn’t take any arguments hence _params_ is an empty array. The body of the `FunctionDeclaration` is of type `BlockStatement`. `BlockStatement` identifies the scope of the function. The body of the `BlockStatement` has two objects for `VariableDeclaration` and `ReturnStatement`. `VariableDeclaration` is same as we saw in the previous example. `ReturnStatement` contains an argument with name `bar`, as `bar` is being returned by the function `foo`.

This is it. This is how ASTs are generated. When I heard of ASTs the first time, I thought of them as big scary trees with complicated nodes. But now that we’ve got a good hold on what ASTs are, don’t you think they are just a group of nicely designed nodes representing the semantics of a program?

Parser also takes care of Scopes.

```
let globalVar = 2
function foo () {
    let globalVar = 3
    console.log('globalVar', globalVar)
}

```

Function `foo` would print 3 and not 2 because the value of `globalVar` in its scope is 3. While parsing the JavaScript code, the parser generates its corresponding scopes as well.

When a `globalVar` is referred in function `foo`, we first look for `globalVar` in the functional scope. If that variable is not found in the functional scope, we look up to its parent, which in this case is the _global_ object. Let’s consider one more example:

```
let globalVar = 2
function foo () {
    let localVar = 3
    console.log('localVar', localVar)
    console.log('globalVar', globalVar)
}
console.log('localVar', localVar)
console.log('globalVar', globalVar)

```

The console statements inside function `foo` would print 3 and 2 while the console statements outside function `foo` would print _undefined_ and 3. This is because `localVar` is not accessible outside function `foo`. It is defined in the scope of function `foo` and so a lookup for `localVar` outside of it results in _undefined_.

## Parsing in V8

V8 uses two parsers for parsing JavaScript code, called as Parser and Pre-Parser. To understand the need of two parsers, let’s consider the code below:

```
function foo () {
    console.log('I\'m inside function foo')
}

function bar () {
    console.log('I\'m inside function bar')
}

/* Calling function foo */
foo()

```

When the above code gets parsed, the parser would generate an AST representing the function _foo_ and function _bar_. However, the function _bar_ is not called anywhere in the program. We’re spending time in parsing and compiling functions that are not used, at least during the startup. _bar_ may be called at a later stage, maybe on click of a button. But it is clearly not needed during the startup. Can we save this time by not compiling function _bar_ during the startup? Yes, we can!

Parser is what we’re doing till now. It parses all of your code, builds ASTs, scopes and finds all the syntax errors. The Pre-Parser is like a fast parser. It only compiles what is needed and skips over the functions that are not called. It builds scopes but doesn’t build an AST. It finds only a restricted set of errors and is approximately twice as fast as the Parser. V8 employs a heuristic approach to determine the parsing technique at runtime.

Let’s consider one example to understand how V8 parses JavaScript code:

```
(function foo () {
    console.log('I\'m an IIFE function')

    function bar () {
        console.log('I\'m an inner function inside IIFE')
    }

})()

```

When the parser comes across the opening parenthesis, it understands that this is an IIFE and it would be called immediately, so it parses the `foo` function using full parser or eager parser. Inside `foo`, when it comes across the function `bar`, it lazily parses or pre-parses the function `bar` because, based on its heuristics, it knows that the function `bar` won’t be called immediately. As the function `foo` is fully parsed, V8 builds its AST as well as scopes while it doesn’t build an AST for function `bar`. It builds only scopes for function `bar`.

Have you encountered this situation ever while writing JavaScript code:

![parser-error](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/parser-error.png?sfvrsn=f08c6652_1 "parser-error")

The code throws an error only when you call the function `fnClickListener`. This is because V8 doesn’t parse this function on the first load. It parses the function `fnClickListener` only when you call it.

Let’s consider a few more examples to better understand the heuristics followed by V8.

```
function toBeCalled() {}
toBeCalled()

```

The function `toBeCalled` is lazily parsed by the V8 engine. When it encounters the call to function `toBeCalled`, it now uses a full parser to parse it completely. The time spent in lazily parsing the function `toBeCalled` is actually wasted time. While V8 is lazily parsing function `toBeCalled`, it doesn’t know that the immediate statement would be a call to this function. To avoid this, you can tell V8 which functions are to be eagerly-parsed (fully-parsed).

```
(function toBeCalled () {})
toBeCalled()

```

Wrapping a function in parentheses is an indicator to V8 that this function is to be eagerly-parsed. You can also add an exclamation mark before the function declaration to tell V8 to eagerly-parse that function.

```
!function toBeCalled () {}
toBeCalled()
```

## Parsing of Inner Functions

```
function outer () {
    function inner () {}
}

```

In this case, V8 lazily parses both the functions, `outer` and `inner`. When we call `outer`, the `outer` function is eagerly/fully-parsed and `inner` function is again lazily parsed. This means `inner` function is lazily parsed twice. It gets even worse when functions are heavily nested.

```
function outer () {
    function inner () {
        function insideInner () {}
    }
    return inner
}

```

Initially, all the three functions `outer`, `inner` and `insideInner` are lazily parsed.

```
let innerFn = outer()
innerFn()

```

When we call function `outer`, it is fully-parsed and functions `inner` and `insideInner` are lazily parsed. Now, when we call `inner`, `inner` is fully parsed and `insideInner` is lazily parsed. That makes `insideInner` get parsed thrice. **Don’t use nested functions when they are not required. Use nested functions appropriately!**

## Parsing of Closures

```
(function outer () {
    let a = 2
    let b = 3
    function inner () {
        return a
    }
    return inner
})

```

In the above code snippet, since the function `outer` is wrapped in parentheses, it is eagerly parsed. Function `inner` is lazily parsed. `inner` returns variable a, which is in the scope of its `outer` function. This is a valid case for closure.

```
let innerFn = outer()
innerFn()
```

`
innerFn` very well returns a value of 2 since it has access to variable a of its parent scope. While parsing the function `inner`, when V8 comes across the variable a, it looks up for variable a in the context of `inner` function. Since a is not present in the scope of `inner`, it checks it in the scope of function `outer`. V8 understands that the variable a is to be saved in the function context and is to be preserved even after `outer` function has completed its execution. So, variable a is stored in the function context of `outer` and is preserved until its dependent function `inner` has completed execution. Please note, variable b is not preserved in this case as it is not used in any of the inner functions.

When we call function `innerFn`, the value of a is not found in the call stack, we then look up for its value in the function context. Lookups in function context are costly as compared to lookups in the call stack.

Let’s check the parsed code generated by V8.

```
function fnCalled () {
    console.log('Inside fnCalled')
}

function fnNotCalled () {
    console.log('Inside fnNotCalled')
}

fnCalled()

```

As per our understanding, both of these functions will be lazily parsed and when we make a function call to `fnCalled`, it would be fully parsed and print _Inside fnCalled_. Let’s see this in action. Run the file containing the above code as `node --trace_parse parse.js`. If you’ve cloned my [GitHub repository](https://github.com/ankita1910/js-internals), you’ll find this file under public/js folder. `parse.js` is the name of the file, and `--trace_parse` serves as an indicator to the runtime of nodejs to print the parsed output. This command would generate a dump of parsing logs. I’ll save the output of this command in a file [parsedOutput.txt](https://github.com/ankita1910/js-internals/blob/master/parsedOutput.txt). For now, all that makes sense is the below screenshot of the dump.

![parsed-output](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/parsed-output.png?sfvrsn=4c64e1d6_1 "parsed-output")

Function `fnCalled` is parsed, but function `fnNotCalled` is not parsed. Try searching for `fnNotCalled` in the [dump](https://github.com/ankita1910/js-internals/blob/master/parsedOutput.txt).

## Script Streaming

Now that we know how parsing works in V8, let’s understand one concept related to Script Streaming. Script Streaming is effective from Chrome version 41.

From what we’ve learned till now, we know it’s the main thread that parses the JavaScript code (even with async and defer keywords). With Script Streaming in place, now the parsing can happen in another thread. While the script is still getting downloaded by the main thread, the parser thread can start parsing the script. This means that the parsing would be completed in line with the download. This technique proves very helpful for large scripts and slow network connections. Check out the below image to understand how the browser operates with Script Streaming and without Script Streaming.

![streaming](https://d585tldpucybw.cloudfront.net/sfimages/default-source/default-album/streaming.png?sfvrsn=709711b0_1 "streaming")

In this tutorial, we learned multiple ways of downloading scripts based on the use case. We learned how the parser generates an Abstract Syntax Tree and its heuristics while parsing the code. Later in the article, we learned about Script Streaming. In the next article, we’ll learn how parsing code gets compiled by the V8 compiler.

## For More on Building Apps with jQuery:
Want to learn more about creating great user interfaces with jQuery? Check out [Kendo UI for jQuery](https://www.telerik.com/kendo-jquery-ui) - our complete UI component library that allows you to quickly build high-quality, responsive apps. It includes all the components you’ll need, from grids and charts to schedulers and dials.