Unraveling the power of Headless browser with Puppeteer
By: Nishla Shakya
Tags: Nodejs, JavaScript, Puppeteer, Web Development, Automated Testing

Being a web developer, I spend half of my day in a browser.

Browser to me is always something I see, with all the layouts, forms, developers tools etc. When I got to know about the Headless browser I started to dig deeper on it. Turns out it’s such a powerful thing that I have been unaware of all these years. It fascinated me so much that it brought me here to do some write ups. But before moving ahead let’s know what a headless browser is;

A headless browser is a web browser without a graphical user interface. It provides automated control of a web page in an environment similar to popular web browsers, but are executed via a command-line interface.

Simply, it just means there’s no graphical user interface(GUI). But what’s the point of a browser without a GUI? Here we go!

We can unravel the power of headless browser with many different tools, but I am here with Puppeteer.

With Puppeteer we will be taking a screenshot of a web browser, generate PDF of a web page and automate a form submission headlessly. But what exactly is Puppeteer?

Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol. Puppeteer runs headless by default, but can be configured to run full (non-headless) Chrome or Chromium.

TLDR;

Let’s start by setting up headless chrome and puppeteer assuming that the node environment has already been set up in your device.

mkdir puppeteer-project

cd puppeteer-project

yarn

yarn add puppeteer

Okay, now that we are all set and configured, let us begin by taking a screenshot of the website! Create an index.js file inside puppeteer-project directory with following code

const puppeteer = require('puppeteer'); (async () => {

const browser = await puppeteer.launch();

const page = await browser.newPage();

await page.goto('https://facebook.com');

await page.screenshot({path: 'facebook.png'});

console.log('Screenshot saved!'); await browser.close();

})();

We will then run this file with node and check the puppeteer-project the folder where our screenshot facebook.png will be saved.

node index.js

Great! You have screenshot the browser window, but you might be thinking it’s cropped out? I bet you did. You can set the viewpoint in puppeteer as follows:

page.setViewport({width: 2000, height: 2000});

How about generating a pdf for the same page, not much of a big deal. We’ll use the browser’s pdf method to execute it. So, now in index.js instead of using the screenshot method we’ll use the pdf method as follows:

const puppeteer = require('puppeteer'); (async () => {

const browser = await puppeteer.launch();

const page = await browser.newPage();

await page.goto('https://facebook.com', {waitUntil: 'networkidle2'});

await page.pdf({path: 'facebook.pdf', format: 'A4'}); await browser.close();

})();

Run the file again to see facebook.pdf inside our folder. I discovered puppeteer while searching for packages or tools to download content from the web in as pdf for the application (Mero Resume) we made for a Hackathon.

Lastly, automating form submission. Since we are working with facebook.com let’s try to log in to facebook headlessly.

const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch();

const page = await browser.newPage();

const email = 'youremail@pastehere.com' await page.setViewport({ width: 2000, height: 2000});

await page.goto('https://www.facebook.com', { waitUntil: 'networkidle2' }); await page.type('#email', email)

await page.type('#pass', '*********') // enter your password here

await page.click('input[type="submit"]');

await page.waitForNavigation({ waitUntil: 'domcontentloaded' });

page.screenshot({ path: 'loggedIn.png' }); })();

Check the loggedIn.png to see your logged-in homepage. Like this, we can automate any form submission. I mostly use this while I have to register a new user frequently for testing purpose. You can try this in any other forms and if you want to see what’s happening under the hood you can launch puppeteer in a browser window like below:

const browser = await puppeteer.launch(({ headless: false }));

These are a few things we can do with Puppeteer, you can discover more cool things and not to forget there are many other tools and frameworks to automate browsers like Selenium which supports browsers other than Chrome. Do try it if you haven’t already. :)