JSON Data in a Local File, IIFEs and the 'mjs' File Extension
Tags: javascript, node
Published: 2020-03-05 18:34:47

There may be times when you have to work with some 'data' that is stored in a local JSON file as opposed to having a RESTful API. One such occurrence of this might be if someone exported a CSV from some spreadsheet and then it was [converted to JSON](https://www.csvjson.com/csv2json). Now, as a JS Dev you might be tasked with performing some sort of data transformation.

For this article, we will have some 'catalog data' in JSON like so:

```JSON
[
      {
        "name": "Hammer",
        "desc": "A 🔨",
        "price": 1.5
      },
      {
        "name": "Chainsaw",
        "desc": "Cut up 🧟‍♂️s.",
        "price": 11.5
      },
      {
        "name": "Frying Pan",
        "desc": "For 👨🏽‍🍳ing 🥘.",
        "price": 10.5
      },
      {
        "name": "Spatula",
        "desc": "Use it for grilling.",
        "price": 7.5
      },
      {
        "name": "Airplane",
        "desc": "For flying around!",
        "price": 100000000.5
      },
      {
        "name": "Car",
        "desc": "For driving.",
        "price": 10000.5
      }
    ]
```

Our task is to *read* this data **from** the JSON file, iterate over this data and add an `id` 🔑to each 'item.' We then need to write the updated JSON back to the file. In order to accomplish this task (and for the purposes of understanding the rest of this post), we need to already be somewhat familiar with:

1. [Using `forEach()` to *iterate and mutate* *Arrays.*](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach)
2. *functions* written in *[arrow syntax.](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions#Syntax)*

We'll also be using [`import`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [reading and writing to the file using NodeJS](https://nodejs.org/api/fs.html), but as long as you understand how to work with *Promises* and `async`, it should be 🙆🏽‍♂️.

Along the way, I'll explain IIFEs and show when we need to use the 'mjs' file extension.

## Read The Data from the Local JSON File

Our data is in './db.json.' Let's read this data with some help from Node!

### `import` the '*Promises*-based' `fs` from Node

We'll need to `import` Node's *File System* modules to allow us to read/write the JSON from 'db.json.' We'll specify that we want to use the more modern 'Promise-based' `'fs'`.  [An explanation of *asynchronicity* and *promises*](https://youtu.be/mEBcloBq8n4) is well beyond the scope of this post. 

Suffice to say that we will [use *Promises*](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises) via the keywords: `async` and `await` to await the results of our read/write operations without blocking any other operations that may need to take place in the meanwhile. Generally, this approach is favored over *traditional callbacks* in modern JS, but was unavailable n Node until recently.

For convenience we are going to 'rename' that `import` `fs` (that's what `as` does). 

`import { promises as fs } from "fs";`

Summarily, this says, "👋🏽JS! Go Look in Node's `fs` *module* and only give me the part named `promises`. Also, when you `import` that in, I want to refer to that *module* as `fs` directly and not `promises`"

### Immediately Invoked Function Expression (IIFE)

As soon as we run our script, we want our `function` to get to work (be *invoked*)...immediately. And, since we are going to be using `await`, we need to specify that our *function expression* will be running *asynchronously* by prefacing it with the keyword: `async`. 

`(async function() {})();`

This *function* is *anonymous* (no name specified/needed) and it's *body* is currently empty. There is no code (yet) inside of its *scope* (established by the `{}`s).

### Read in the JSON and `parse()` it into a JS Object

We're going to read the JSON in *asynchronously* using `async` `await`. As soon as the file is read, we'll use `JSON.parse()` to turn the ['raw' *JSON* into a 'real' JS Object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON), and assign the results to a *variable* `data`. We'll `log` that to confirm that it works. 

```js
(async function() {
      const data = JSON.parse(await fs.readFile("./db.json"));
      console.log(data);
    })();
```

### 'mjs' file vs 'js'

If we were to run this from our command line with: `node index.js`, we'll get yelled for trying to use an `import`: `SyntaxError: Cannot use import statement outside a module`. Since we are building up a whole app here, the simplest way to solve this is to rename 'index.js' to 'index.mjs.' This will allow us to work with 'experimental Node stuff' like `import`s. 

---

```bash
[
      { name: 'Hammer', desc: 'A 🔨', price: 1.5 },
      { name: 'Chainsaw', desc: 'Cut up 🧟‍♂️s.', price: 11.5 },
      { name: 'Frying Pan', desc: 'For 👨🏽‍🍳ing 🥘.', price: 10.5 },
      { name: 'Spatula', desc: 'Use it for grilling.', price: 7.5 },
      { name: 'Airplane', desc: 'For flying around!', price: 100000000.5 },
      { name: 'Car', desc: 'For driving.', price: 10000.5 }
    ]
```

---

## Mutate our Data

We'll use `forEach` to *iterate* over `data` and add a new 🔑, `id` to each one. Here, that `id` will be 1 more than the *index* of the item. So, the first item's `id` will be `1`, and so on. 

`data.forEach((d, i) => {(d.id = i + 1)});`

`forEach` takes a [*callback function*](https://developer.mozilla.org/en-US/docs/Glossary/Callback_function) with the first *parameter*, `d` representing each individual item inside of our `data` *Array.* The second parameter, `i` represents the current *index* of each item in the *Array* as we *iterate.* This starts at `0`, so that's why we add `1` to each `i`. 

We *iterate* over `data` one item at a time (`forEach`) ( `d`) and also look at its index, `i`. We then add a new 🔑, `id` and set (`=`)its *value* to be...one more than the current index with: `i + 1`.

We are using *[arrow syntax](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions#Syntax),* we can omit the `function` keyword.

---

```bash
[
      { name: 'Hammer', desc: 'A 🔨', price: 1.5, id: 1 },
      { name: 'Chainsaw', desc: 'Cut up 🧟‍♂️s.', price: 11.5, id: 2 },
      {
        name: 'Frying Pan',
        desc: 'For 👨🏽‍🍳ing 🥘.',
        price: 10.5,
        id: 3
      },
      { name: 'Spatula', desc: 'Use it for grilling.', price: 7.5, id: 4 },
      {
        name: 'Airplane',
        desc: 'For flying around!',
        price: 100000000.5,
        id: 5
      },
      { name: 'Car', desc: 'For driving.', price: 10000.5, id: 6 }
    ]
```
---

## Write The Data Back to the Local JSON File

Now we need to get this data written back to './db.json.' But we want to write **JSON** back - not a *JS Object.* To 'convert' our *Object* into JSON, we use `JSON.stringify()`.

```js
fs.writeFile("./db.json", JSON.stringify(data))
          .then(() => {
            console.log("Rote new data!");
          })
          .catch(error => {
            console.error(`Error riting data: ${error}`);
          });
```

`writeFile` needs to know where to write to `"./db.json"` and what we want to write (the 'string-ified' `data`).

As mentioned previously 👆🏽, we are using *Promises*. We don't need to assign our results to any *variable*, so rather than `await`ing, we'll chain a typical `then()` and `catch()` and just 'print' some feedback to the console.

🏃🏽‍♂️our script with `node index.js` updates './db.json':


```JSON
 [{"name":"Hammer","desc":"A 🔨","price":1.5,"id":1},{"name":"Chainsaw","desc":"Cut up 🧟‍♂️s.","price":11.5,"id":2},{"name":"Frying Pan","desc":"For 👨🏽‍🍳ing 🥘.","price":10.5,"id":3},{"name":"Spatula","desc":"Use it for grilling.","price":7.5,"id":4},{"name":"Airplane","desc":"For flying around!","price":100000000.5,"id":5},{"name":"Car","desc":"For driving.","price":10000.5,"id":6}]
```   

### Format JSON

We can improve the readability of our JSON by utilizing `stringify()`'s *optional parameters:* `JSON.stringify(data, null, 2)`. The second *argument* `null` just accepts the 'default implementation' of the method, which 'converts' all of our 'data.' The *third argument,* `2` specifies a '2 space tab' in our resulting JSON, cleaning things up! 🎨

Now, after re-🏃🏽‍♂️our script, './db.json' looks like:

```JSON
[
      {
        "name": "Hammer",
        "desc": "A 🔨",
        "price": 1.5,
        "id": 1
      },
      {
        "name": "Chainsaw",
        "desc": "Cut up 🧟‍♂️s.",
        "price": 11.5,
        "id": 2
      },
      {
        "name": "Frying Pan",
        "desc": "For 👨🏽‍🍳ing 🥘.",
        "price": 10.5,
        "id": 3
      },
      {
        "name": "Spatula",
        "desc": "Use it for grilling.",
        "price": 7.5,
        "id": 4
      },
      {
        "name": "Airplane",
        "desc": "For flying around!",
        "price": 100000000.5,
        "id": 5
      },
      {
        "name": "Car",
        "desc": "For driving.",
        "price": 10000.5,
        "id": 6
      }
    ] 
```
---
Here's the code for this post: [https://github.com/manavm1990/read-transform-write-json-file](https://github.com/manavm1990/read-transform-write-json-file)

In the [repo](https://github.com/manavm1990/read-transform-write-json-file) I wrap up our `await` in a `try`-`catch` as is a common practice for `catch`ing any `error`s. That's not overly relevant to the purpose of this post, so I won't mention it again.