5 examples of reading, writing and pipelining streams in NodeJS
Tags: node, javascript, stream
Published: 2022-08-11 15:56:40

## 1. Piping input to output stream (file examples)

```js
const fs = require('fs');
let input = fs.createReadStream('/var/www/examples/test.txt');
let output = fs.createWriteStream('/tmp/out.txt');
input.pipe(output);
```

- `require('fs')` - library to work with file system,
- `fs.createReadStream(` - create stream to read data from (file in our case),
- `fs.createWriteStream` - open file and create writing stream from it,
- `/var/www/examples/test.txt` - path to file to stream read,
- `/tmp/out.txt` - path to file to write stream to,
- `.pipe(` - pipe input stream to output (in our case - read from input file and write to output).

[Open original](https://onelinerhub.com/nodejs/how-to-pipe-streams) or [edit on Github](https://github.com/Onelinerhub/onelinerhub/tree/main/nodejs/how-to-pipe-streams.md).

## 2. Preferred way to pipeline streams

```js
const fs = require('fs');
const { pipeline } = require('stream');

let input = fs.createReadStream('/var/www/examples/test.txt');
let output = fs.createWriteStream('/tmp/out.txt');

pipeline(input, output, (err) => console.log(err));
```

- `require('fs')` - library to work with file system,
- `fs.createReadStream(` - create stream to read data from (file in our case),
- `fs.createWriteStream` - open file and create writing stream from it,
- `/var/www/examples/test.txt` - path to file to stream read,
- `/tmp/out.txt` - path to file to write stream to,
- `pipeline` - pipelines given streams (left to right) and properly destroys all objects after processing,
- `(err) => console.log(err)` - last argument of pipeline() is always error callback.

[Open original](https://onelinerhub.com/nodejs/how-to-pipeline-streams) or [edit on Github](https://github.com/Onelinerhub/onelinerhub/tree/main/nodejs/how-to-pipeline-streams.md).

## 3. How to read input stream to Buffer

```js
const fs = require('fs');
let input = fs.createReadStream('/var/www/examples/test.txt');

input.on('data', buf => {
  console.log(buf);
});
```

- `require('fs')` - library to work with file system,
- `fs.createReadStream(` - create stream to read data from (file in our case),
- `/var/www/examples/test.txt` - path to file to stream read,
- `.on('data'` - handle date reading from stream,
- `buf` - buffer gets data chunk read from stream.

[Open original](https://onelinerhub.com/nodejs/how-to-read-stream-to-buffer) or [edit on Github](https://github.com/Onelinerhub/onelinerhub/tree/main/nodejs/how-to-read-stream-to-buffer.md).

## 4. How to read input stream to string

```js
const fs = require('fs');
let input = fs.createReadStream('/var/www/examples/test.txt');

const chunks = [];
input.on('data', buf => chunks.push(buf));
input.on('end', () => console.log( Buffer.concat(chunks).toString() ))
```

- `require('fs')` - library to work with file system,
- `fs.createReadStream(` - create stream to read data from (file in our case),
- `/var/www/examples/test.txt` - path to file to stream read,
- `.on('data'` - handle date reading from stream,
- `chunks.push(buf)` - push each chunk into array,
- `Buffer.concat(chunks)` - join all chunks into single buffer,
- `.toString()` - convert Buffer to string,
- `input.on('end'` - fire when stream reading was finished.

[Open original](https://onelinerhub.com/nodejs/how-to-read-stream-to-string) or [edit on Github](https://github.com/Onelinerhub/onelinerhub/tree/main/nodejs/how-to-read-stream-to-string.md).

## 5. How to transform streams using `Transform` interface

```js
const fs = require('fs');
const { Transform } = require("stream");

let input = fs.createReadStream('/var/www/examples/test.txt');

const my_transform = new Transform({
  transform(chunk, encoding, callback) {
    callback(null, 'TRANSFORMED: ' + chunk.toString());
  },
});

my_transform.on('data', buf => console.log(buf.toString()));

input.pipe(my_transform);
```

- `require('fs')` - library to work with file system,
- `fs.createReadStream(` - create stream to read data from (file in our case),
- `new Transform(` - create new stream that transforms data,
- `transform(chunk, encoding, callback)` - chunk transformation function,
- `'TRANSFORMED: ' + chunk.toString()` - return transformed chunk to callback (we add TRANSFORMED: text to chunk as transformation example),
- `.on('data'` - handle date reading from stream,
- `.pipe(` - pipe object to the given stream (read from file and pipe to transforming stream).

[Open original](https://onelinerhub.com/nodejs/how-to-transform-streams) or [edit on Github](https://github.com/Onelinerhub/onelinerhub/tree/main/nodejs/how-to-transform-streams.md).