How to do Piping/Streaming of JavaScript objects in Node.js ? - GeeksforGeeks

20 Nov, 2021
This article gives brief information about the piping/stream implementation in node.js and explains the working of various parts as buffers, streams and how to implement piping in node.js
Buffers: These are the temporary storage spot for a chunk of data that is being transferred from one place to another, the buffer is filled with data then passed along. Buffer transfers the small chunks of data at a time. Buffers are very useful as we don’t have to wait for the complete transfer process because we fill up the buffer which is the temporary storage spot for the data which can then be passed altogether.
Streams: They are the data handling method that is used for the transfer of the data. It increases the performance as the data is processed even before the complete transfer of the data. It collects the small chunks of buffer together and when the buffer is filled then it passes the data down the stream to be processed and sent to the client. They are a data-handling method and are used to read or write input into output following a proper sequence. Buffer and streams together while playing an online video you don’t need to wait for the complete video to be processed rather is small sections transferred with the help of buffers and streams bit by bit.
Readable Stream: Readable streams are used to extract information from any source that they are used to read the data that is present on any source, and then we can use that data to transfer to any source or to store in our system. Readable streams emit an event in the form of chunks (received from the buffers) that is a small part of the data that needs to be processed.
Example: Create a read.txt file in the current project directory with some sample text, in this case, we have the following text.
THIS IS READ.TXT FILE
Filename: index.js
Javascript
// Requiring file system modulevar fs = require('fs'); // Creating a read streamvar myReadStream = fs.createReadStream('read.txt', 'utf8'); myReadStream.on('data', function (chunk) {  console.log("new chunk received");   // Printing the chunk  console.log(chunk);});
Output:
new chunk received 
THIS IS READ.TXT FILE
Writable Streams: Writable stream lets you send the data to any destined source that can rather be any file or to stream online. These streams also emit the events to ensure proper functioning at regular intervals of time. 
Example: Create an index.js file with the following code.
Filename: index.js
Javascript
// Requiring file system modulevar fs = require('fs');       // Creating a write stream// It will create a txt file in provided locationvar myWriteStream =    fs.createWriteStream(__dirname + '/write.txt'); // Writing on streammyWriteStream.write("Greetings from GeeksforGeeks");
Output: A new file named write.txt will be generated in the current directory which will contain the following data.
Greetings from GeeksforGeeks
Piping is a method of transferring data from one stream to other directly with the help of buffers so that the data transfer process can start as soon as it fills up a buffer. In other words, piping is used to process streamed data in various steps. This is a module method to pipe between streams forwarding errors and properly cleaning up and provide a callback when the pipeline is complete.
Example: Create a read.txt file in the current project directory with some sample text, in this case, we have the following text.
THIS IS READ.TXT FILE
Filename: index.js
Javascript
var fs = require('fs');const zlib = require('zlib');const { pipeline } = require('stream'); // Constructing promisifyconst { promisify } = require('util');const pipelineAsync = promisify(pipeline); // Creating a read streamvar myReadStream = fs.createReadStream(      __dirname + '/read.txt', 'utf8'); // Creating a write streamvar myWriteStream = fs.createWriteStream(      __dirname + '/write.txt'); // Creating transform streamconst transform = zlib.createGzip(); (async function run() {   try {       // Pipelining three streams      await pipelineAsync(          // Reading from read stream         myReadStream,          // Transforming the file         transform,          // Writing on write stream         myWriteStream      );      console.log("Pipeline accomplished");   }   catch (err) {      console.error('pipeline failed with error:', err);   }})();
Output: We will see the following output and the write.txt file will be generated in the current directory.
Pipeline accomplished
Note: The pipeline should be used instead of a pipe, as the pipe is unsafe, but implementation on the pipe is given below:
Filename: index.js
Javascript
// Requiring file system modulevar fs = require('fs');       // Creating a read streamvar myReadStream = fs.createReadStream(        __dirname + '/read.txt','utf8'); // Creating a write streamvar myWriteStream = fs.createWriteStream(        __dirname + '/write.txt'); // Piping using pipemyReadStream.pipe(myWriteStream);  console.log("Pipeing accomplished");
Output: We will see the following output and the write.txt file will be generated in the current directory.
Pipeing accomplished
 
clintra
javascript-object
Node.js-Methods
NodeJS-Questions
Picked
Node.js
Web Technologies
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Comments
Old Comments
How to build a basic CRUD app with Node.js and ReactJS ?
How to connect Node.js with React.js ?
Mongoose Populate() Method
Express.js req.params Property
Mongoose find() Function
Top 10 Front End Developer Skills That You Need in 2022
Top 10 Projects For Beginners To Practice HTML and CSS Skills
How to fetch data from an API in ReactJS ?
How to insert spaces/tabs in text using HTML/CSS?
Difference between var, let and const keywords in JavaScript