Node.js - File System
Tags: development, javascript, node
Published: 2020-12-18 16:41:11

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/tvigldg8iwg7st1w0uq2.jpeg)

#### Overview 
This article is **Part 5** of [Working With NodeJS](https://dev.to/meddy672/series/20844), a series of articles to help simplify learning Node.js. In this topic I will cover working with the file system.

#### Introduction
To be able to work with the file system using Node.js you need to understand the [fs](https://nodejs.org/dist/latest-v15.x/docs/api/fs.html#fs_file_system) and [path](https://nodejs.org/dist/latest-v15.x/docs/api/path.html#path_path) modules. These  modules provide the means to interact with a file system. The **path** module is important for path manipulation and normalization across platforms, and the **fs** module provides APIs for reading, writing, file system meta-data, and file system watching.


#### Paths
When navigating a file system it is helpful to have a reference point. The **__dirname** and **__filename** variables can provide such a point. These two variables are defined whenever Node.js executes a [modules code](https://nodejs.org/dist/latest-v15.x/docs/api/modules.html#modules_the_module_wrapper). The *__filename* variable holds the absolute path to the currently executing file, and *__dirname* holds the absolute path to the directory where the currently executing file resides.
```javascript
console.log(__dirname); // Path to current working directory
console.log(__filename); // Path to current executing file
```
With these reference points, we can build paths for files and directories programmatically. The most commonly used method to build a path is the `join` method. The [join](https://nodejs.org/dist/latest-v15.x/docs/api/path.html#path_path_join_paths) method joins all given path segments together using the platform-specific separator as a delimiter, then normalizes the resulting path.
```javascript
const { join } = require('path');
console.log(join(__dirname, 'out.txt'));
```
The above code snippet will output the absolute path for *out.txt* to the console. You can add as many segments to the path as you need. Lets say you wanted to construct the path for *out.txt* so the path leads to a subdirectory called **text**.
```javascript
const { join } = require('path');
console.log(join(__dirname, 'text', 'out.txt'));
```
##
In the above code snippet, the segment *text* is now apart of the absolute path for out.txt. Its important to know the `join` method does not create the directory *text*, but only a path which leads to it. There are other methods within the path module which can be used to construct a path, however, the `join` method is the most commonly used. In addition to path builders, there are path deconstructors. A path deconstructor will break down a path into its properties.
```javascript
const { parse } = require('path');
console.log(parse('/home/user/dir/file.txt'));
```
##### output
```javascript
{ root: '/',
  dir: '/home/user/dir',
  base: 'file.txt',
  ext: '.txt',
  name: 'file' }
```

In the above code snippet, the `parse` method is used to deconstruct the path into the segments which compose it. The important thing to understand about the [path](https://nodejs.org/dist/latest-v15.x/docs/api/path.html#path_windows_vs_posix) module is that its platform specific. This means if you are working on a different operating system all the methods within the module are still applicable.


#### Files - Reading and Writing
The [fs](https://nodejs.org/dist/latest-v15.x/docs/api/fs.html#fs_file_system) module provides methods for interacting with file system. The module contains both synchronous and asynchronous methods. All the names of synchronous methods in the fs module end with **Sync**. This important to remember because synchronous methods will block the main process until the operation has completed.
```javascript
'use strict';
const { readFileSync } = require('fs');
const contents = readFileSync(__filename);
console.log(contents);
```
##### output
```
<Buffer 27 75 73 65 20 73 74 72 69 63 74 27 0a 63 6f 6e 73
74 20 7b 20 72 65 61 64 46 69 6c 65 53 79 6e 63 20 7d 20 3d
20 72 65 71 75 69 72 65 28 27 66 73 27 ... 66 more bytes>
```
In the above code snippet, the `readFileSync` is used to read its own file contents. If no encoding is specified then a Buffer is returned. The encoding can be set by configuring an options object on `readFileSync`.
```javascript
'use strict';
const { readFileSync } = require('fs');
const contents = readFileSync(__filename, {encoding: 'utf8'});
console.log(contents);
```
In the above code snippet, the encoding was set to *utf8* which will allow a string to be returned instead of a Buffer. Some encoding options you can set for read methods are: *ascii, base64, hex, utf8, and binary*.

Writing synchronously can be accomplished using the `writeFileSync` method. At a minimum the method takes a filename and the data to be written to the file.
```javascript
'use strict';
const { join } = require('path');
const { writeFileSync } = require('fs');
const out = join(__dirname, 'out.txt');
writeFileSync(out, 'Hello');
```
If the file already exists then the operation will replace the file, and if not the file will be created. Like the read method, `writeFileSync` can be configured with options as well. If instead of replacing the entire file you wish to add to it, you can configure `writeFileSync` with the **flag** option and set it to append.
```javascript
'use strict';
const { join } = require('path');
const { writeFileSync } = require('fs');
const out = join(__dirname, 'out.txt');
writeFileSync(out, 'Hello', {
    flag: 'a'
});
```
For a full list of supported flags, see [File System Flags](https://nodejs.org/dist/latest-v12.x/docs/api/fs.html#fs_file_system_flags) section of the Node.js Documentation. It is important to remember that synchronous read and write operations should be surrounded with `try/catch` blocks for error handling.
```javascript
'use strict';
const { readFileSync } = require('fs');
try {
    const contents = readFileSync('noFile.txt');
    console.log(contents);
} catch (err) {
    console.error(err.message);
    return;
}
```
##### output
```
ENOENT: no such file or directory, open 'noFile.txt'
```

Up to this point, the reading and writing methods have been synchronous operations. However, Node.js is single threaded and therefore works best with asynchronous operations. The **fs** module provides both callback and promise based methods to perform asynchronous operations.
```javascript
'use strict';
const { readFile } = require('fs');
readFile(__filename, {encoding: 'utf8'}, (err, contents) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log(contents);
});
```
In the above code snippet, the `readFile` method is used instead of the `readFileSync` method. Both methods perform the same operation, however, `readFile` is an asynchronous operation. Let see an asynchronous writing operation.
```javascript
'use strict'
const { join } = require('path');
const { writeFile } = require('fs');
const out = join(__dirname, 'out.txt');
writeFile(out, 'Hello', { flag: 'a' }, (err) => {
    if (err) { console.error(err); }
    else console.log('Write Successful');
});
```
In the above code snippet, the `writeFile` method performs the same operation as the `writeFileSync`, but now its an asynchronous operation. The **important** thing to remember is that Node.js is single threaded so you want to avoid using operations that block further execution of code. All methods within the **fs** module that will block further execution of code end in **Sync**.

The **fs** module also supports promise based asynchronous operations. To use the promise based operations append the *.promises* on the `require('fs')` function.
```javascript
const { readFile, writeFile } = require('fs').promises;
```
In the above code snippet, the both `readFile` and `writeFile` are asynchronous operations, but now they now return promises. This is useful because now `async/await` can be used. This gives the code a cleaner and traditional look. Lets see an example. I'll use the `readFile` method to read its own content and then use the `writeFile` method to write the content to the file *out.txt*.
```javascript
'use strict';
const { join } = require('path');
const { readFile, writeFile } = require('fs').promises;
async function run () {
  const contents = await readFile(__filename, {encoding: 'utf8'});
  const out = join(__dirname, 'out.txt');
  await writeFile(out, contents);
}

run().catch((err) => {
    console.error(err);
});
```
##### out.txt
```
'use strict';
const { join } = require('path');
const { readFile, writeFile } = require('fs').promises;
async function run () {
  const contents = await readFile(__filename, {encoding: 'utf8'});
  const out = join(__dirname, 'out.txt');
  await writeFile(out, contents);
}

run().catch((err) => {
    console.error(err);
});
```

#### File Streams
Remember the **fs** module has four API types: *Synchronous, Callback-based, Promise-based, and Stream-based*. The **fs** module has `createReadStream` and `createWriteStream` methods which allow us to read and write files in chunks. Streams are ideal when handling very large files that can be processed incrementally.
```javascript
'use strict';
const { pipeline } = require('stream');
const { join } = require('path');
const { createReadStream, createWriteStream } = require('fs');
pipeline(
    createReadStream(__filename),
    createWriteStream(join(__dirname, 'out.txt')),
    (err) => {
        if (err) {
            console.error(err);
        }
        console.log('Finished writing');
    }
);
```
In the above code snippet, its the same as before, we read the content of the file and write it to *out.txt*, but now we are using streams. Quick note, notice that we didn't pass any content to `createWriteStream` like we did for `writeFile`. This is because we're using a `pipeline` and the data is automatically passed from one phase of the pipe to the next.

Before moving on to directories lets take a look at some other useful file methods within the **fs** module.
##### change the permissions of a file
```javascript
'use strict';
const { chmodSync, readFileSync } = require('fs');
chmodSync('out.txt', 0o000);
try {
    const content = readFileSync('out.txt');
    console.log(content);
} catch (error) {
    console.error(error.message);
}
```
The above code snippet uses the `chmodSync` to change the permissions of the file *out.txt*. When the `readFileSync` method is used an error is thrown *EACCES: permission denied, open 'out.txt'*. To learn more about `chmod` or `chmodSync` see [Node.js Documentation](https://nodejs.org/dist/latest-v15.x/docs/api/fs.html#fs_fs_chmod_path_mode_callback)

##### change the owner of a file
```javascript
'use strict';
const { chown } = require('fs');
chown('out.txt', 6101, 120, (err) => {
    if (err) {
        console.error(err);
    }
    console.log('Owner changed');
});
```
##### copying a file
```javascript
const { copyFile } = require('fs');

copyFile('out.txt', 'out.backup.txt', (err) => {
    if (err) {
        console.error(err.message);
    }
    console.log('File copied successful');
});
```

##### deleting a file
```javascript
const { rm } = require('fs');
rm('out.backup.txt', (err) => {
    if (err) {
        console.error(err.message);
    }
    console.log('File deleted');
});
```

#### Directories
The *fs* module provides the means for working with directories. Similar to files, the *fs* module provides three main operations to read the content of a directory.
1. Synchronous
2. Callback
3. Promise

To read the content of a directory you can use the `readdirSync`, `readdir` which is callback based, or its asynchronous counterpart `readdir` which is promise-based obtained through `require('fs').promises`. Lets see some examples.

##### Synchronous
```javascript
'use strict'
const { readdirSync } = require('fs');

try {
    const content = readdirSync(__dirname);
    console.log(content);
} catch (error) {
    console.error(error);    
}
```

##### Callback
```javascript
'use strict'
const { readdir } = require('fs');

readdir(__dirname, (err, files) => {
    if (err) {
        console.error(err);
        return;
    }
    console.log('callback', files);
});
```

##### Promise
```javascript

'use strict';
const { readdir } = require('fs').promises;

async function run () {
    const files = await readdir(__dirname);
    console.log('promise', files);
}
  
run().catch((err) => {
    console.error(err);
})
```
In the above code snippets, all three snippets do the same thing, that is read the content of the directory of the currently executing file. 

#### File Meta Data
Metadata about files can be obtained with the following methods:
* fs.stat, fs.statSync, fs.promises.stat
* fs.lstat, fs.lstatSync, fs.promises.lstat

The only difference between the stat and lstat methods is that stat follows symbolic links, and lstat will get meta data for symbolic links instead of following them. Lets see an example of how we could read the content of a directory and determine wether if the content returned was a file or directory.
```javascript
const { readdirSync, statSync } = require('fs');

const files = readdirSync(__dirname);

for (name of files) {
    const stat = statSync(name);
    const typeLabel = stat.isDirectory() ? 'dir: ' : 'file: ';
    console.log(typeLabel, name);
}
```

#### Watching Files and Directories
The **fs** module has the capabilities for watching changes to a file or directory using the `watch` method. Whenever changes are made the `watch` method is triggered and its callback function is executed. Let see a simple example.
```javascript
'use strict';
const { watch } = require('fs');

watch(__dirname, (event, filename) => {
    console.log(event, filename);
});
```
The callback function of the `watch` method takes an **event** and **filename** as parameters. Both parameters correspond to the file or directory in which the change happened. For the final example we'll use a combination of what we learned in conjunction with the `watch` method. We'll create a small program(index.js) that will monitor the content of our working directory.
##### index.js
```javascript
'use strict';
const { join, resolve } = require('path');
const { watch, readdirSync, statSync } = require('fs');

const cwd = resolve(__dirname);
const files = new Set(readdirSync(__dirname));
watch(__dirname, (evt, filename) => {
    try {
        const { ctimeMs, mtimeMs } = statSync(join(cwd, filename));
        if (files.has(filename) === false) {
            evt = 'created';
            files.add(filename);
        } else {
            if (ctimeMs === mtimeMs) evt = 'content-updated';
            else evt = 'status-updated';
        }
    } catch (err) {
        if (err.code === 'ENOENT') {
            files.delete(filename);
        } else {
            console.error(err);
        }
    } finally {
        console.log(evt, filename);
    }
});
```
In the above code snippet, we use `Set`, (a unique list), initializing it with the array of `files` already present in the current workings directory. To get the current working directory the `resolve(__dirname)` method is used. Alternatively, one could have used `resolve('.')` to achieve the same effect. Next, we begin to `watch` our current working directory. If a change is emitted within our working directory, we'll extract the `ctimeMs` and `mtimeMs` properties of the file using the `statSync` method. If the file is not apart of our list then we set the `evt` variable to **created** and add the filename using the `add` method. However, if the file is apart of our list then we'll check to see if the `ctimeMs` and `mtimeMs` are equal. If both properties are equal `evt` is set to **content-updated** and if not `evt` is set to **status-updated**. Finally, we log the **event** and **filename** to the console with within the `finally` block.

There is a lot more to discover within the [path](https://nodejs.org/dist/latest-v15.x/docs/api/path.html#path_path) and [fs](https://nodejs.org/dist/latest-v15.x/docs/api/fs.html#fs_file_system) modules of Node.js and you should take some time to explore the full power of its APIs. As always thank you for taking time to read this article and if you found it helpful subscribe to the series because more is on the way. Take care.