Browser Automation With Node Red
Tags: nodered, automation, playwright, pupeteer
Published: 2021-03-29 03:31:55

---
title: Browser Automation With Node Red
published: true
description: 
tags: nodered, automation, playwright, pupeteer
//cover_image: https://direct_url_to_image.jpg
---

[Node Red](https://nodered.org/) is a fantastic, open source, ["low code"](https://en.wikipedia.org/wiki/Low-code_development_platform) automation framework. I have been using it for years as the automation engine for [Home Assistant](https://www.home-assistant.io/) as well as a few other tasks. If you're coding-inclined, you can use Node Red's function nodes to run whatever JavaScript you want. Today I'm going to show you how to use these function nodes to run browser automation tasks using [Playwright](https://playwright.dev/) or [Puppeteer](https://pptr.dev/).

*Note: I'm using [Docker](https://www.docker.com/) to run Node Red and [Browserless](https://www.browserless.io/) as a browser instance I can access via API from any machine on my network, but similar steps can be followed if you're running Node Red locally or installed on a server another way.*

## Step 1: Install the Playwright/Puppeteer module
Navigate to the Node Red data directory. On most systems that defaults to `$HOME/.node-red`, and in the Docker container it's `/data`. If you're running Node Red in Docker, you can connect by doing `docker exec -it node-red-container-name /bin/bash` or if you have NodeJS installed on the host, you can just navigate to wherever the Node Red data directory is mapped.

Now run the appropriate command for the tool(s) you want to install:

- Playwright: `npm i playwright`  
- Playwright Core: `npm i playwright-core` (if you're using Browserless)  
- Puppeteer: `npm i puppeteer`  
- Puppeteer Core: `npm i puppeteer-core` (if you're using Browserless)

## Step 2: Add the tools to Node Red's global context
Inside Node Red's data directory should be a file named settings.js. Open that in your editor of choice and find the `functionGlobalContext` property. Add whatever tool(s) you installed and then restart Node Red so that the changes are picked up.

```js
functionGlobalContext: {
    env: process.env,
    playwright: require("playwright-core")
    // os:require('os'),
    // jfive:require("johnny-five"),
    // j5board:require("johnny-five").Board({repl:false})
}
```

*Note: I have a Browserless instance running in Docker, so I just installed `playwright-core` since I don't need to install the browsers typically installed with Playwright.*

## Step 3: Use your tool(s) in Function nodes

```js
const playwright = global.get('playwright');

(async () => {
    node.warn('Started');
    const browser = await playwright.chromium.connect({
        wsEndpoint: "ws://browserless:3000/playwright"
    });
    const context = await browser.newContext();
    const page = await context.newPage();
    await page.goto('https://iknowwhatyoudownload.com/en/peer/');
    node.warn(page.url());
    const downloads = await page.$$('td a');
    msg.payload = downloads.length;
    await browser.close();
    node.send(msg);
})();
```

*Note: if not using Browserless you would just instantiate a browser with `const browser = await playwright.chromium.launch();` or use `firefox` or `webkit`.*

For those that may be new to [Async/Await](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function), it allows for cleaner asynchronous programming without callbacks. Javascript doesn't allow the `await` keyword unless you're inside an `async` function, so that's why I wrapped my code in:

```js
(async () => {
  //...
})();
```

Also, since we're using asynchronous code, we have to use `node.send(msg);` instead of `return msg;`.

When creating a new browser automation, you're definitely better off testing things on a non-headless machine. When creating a browser instance for testing, you can run it non-headless and in "slowmo" to allow you to watch what is happening. Here's how to do it in Playwright:

```js
const browser = await chromium.launch({
  headless: false,
  slowMo: 100
});
```

## Adding other NPM modules
You can add other [npm](https://www.npmjs.com/) modules using this same process to make your life a little easier when writing custom functions and is much easier than developing a custom module. Some potentially handy modules might be: [date-fns](https://www.npmjs.com/package/date-fns), [mathjs](https://www.npmjs.com/package/mathjs), [cheerio](https://www.npmjs.com/package/cheerio), [execa](https://www.npmjs.com/package/execa).