How to use Puppeteer in a Chrome extension
Tags: puppeteer, automation, chrome, tutorial
Published: 2020-02-03 10:36:42

At my current job I would sometimes input client details into our CRM platform and would subsequently input these same details while setting them up on another third party platform. As such, there is a lot of copying data around and manual labor. 

Automating this process via a Chrome Extension seemed like a good idea because I could also share it with non-technical colleagues that weren't comfortable with running scripts. Also at this company, Chrome is the default browser for all of the machines which run on Windows. With all the stars aligned, I managed to develop a working solution, although a bit hacky but works for my needs. This post was inspired by this [thread](https://stackoverflow.com/questions/54647694/how-to-run-puppeteer-code-in-any-web-browser/54654516).

# Some prerequisite knowledge
- basic understanding of Chrome Extension structure
- basic understanding of Puppeteer

I had no prior experience working with Chrome Extensions, so I highly recommend [The Coding Train's Playlist](https://www.youtube.com/playlist?list=PLRqwX-V7Uu6bL9VOMT65ahNEri9uqLWfS) which is an excellent primer to developing them. It gave me everything I needed to get up and running.

With Puppeteer, the [docs](https://github.com/puppeteer/puppeteer/blob/master/docs/api.md) is quite thorough.

# Bringing it all together

The trick is to use the [bundled version of puppeteer](https://github.com/puppeteer/puppeteer/tree/6522e4f524bdbc1f1b9d040772acf862517ed507/utils/browser). This link points to an old commit as this is no longer within the scope of their project. S

1. Bundle the repository per above documentation
2. Place it in your chrome extension folder
3. Reference the newly bundled folder in your `popup.html` with something like
```html
<script src="./puppeteer/utils/browser/puppeteer-web.js"></script>
```

This is the hacky part. You'll then need to take advantage of Chrome's remote debugging functionality as puppeteer-web can't start its own instance via `puppeteer.launch()` and can only use `puppeteer.connect()` to connect to an already existing chrome instance. Add `--remote-debugging-port=9222` to the end of the target field of your `chrome.exe` short cut. To learn more about this, read [here](https://stackoverflow.com/questions/51563287/how-to-make-chrome-always-launch-with-remote-debugging-port-flag).

Once remote debugging is activated you'll be able to see the `webSocketDebuggerUrl` property by visiting http://localhost:9222/json/version on your browser. This is the `browserWSEndpoint` the connect method will invoke.

Example response when visiting http://localhost:9222/json/version

```json
{
   "Browser": "Chrome/79.0.3945.130",
   "Protocol-Version": "1.3",
   "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36",
   "V8-Version": "7.9.317.33",
   "WebKit-Version": "537.36 (@e22de67c28798d98833a7137c0e22876237fc40a)",
   "webSocketDebuggerUrl": "ws://127.0.0.1:9222/devtools/browser/343998a5-ce82-456f-944c-a214ebb59e83"
}
```


You will also need to add the port address to the `permissions` array in the Chrome Extension's `manifest.json` file otherwise ajax requests won't work and you'll suffer from cors errors.

Example `manifest.json` file.

```json
{
  "name": "Chrome Extension",
  "version": "0.1",
  "content_scripts": [
    {
      "matches": [
        "https://url-that-you-want-to-scrape/*"
      ],
      "js": ["content.js"]
    }
  ],
  "background": {
    "scripts": ["background.js"],
    "persistent": true
  },
  "permissions": [ "tabs" , "identity","http://localhost:9222/*"],
  "browser_action": {
    "default_title": "Chrome Extension",
    "default_popup": "popup.html"
  }
   "content_security_policy": "script-src 'self' 'unsafe-eval' https://apis.google.com/; object-src 'self'",
   "manifest_version": 2
}
```

Example `popup.html` file

```html
<!DOCTYPE html>
<html>
  <head>
    <title>Example popup</title>
    <link rel="stylesheet" type="text/css" href="style.css">
  </head>
  <body>
    <div>
      <button id='puppeteer-button'>Do Puppeteer Things</button>
      <script src="./puppeteer/utils/browser/puppeteer-web.js"></script>
      <script type="module" src="popup.js"></script>
    </div>
  </body>
</html>
```

Example `popup.js` file

```javascript
let browserWSEndpoint = '';
const puppeteer = require("puppeteer");

async function initiatePuppeteer() {
  await fetch("http://localhost:9222/json/version")
    .then(response => response.json())
    .then(function(data) {
        browserWSEndpoint = data.webSocketDebuggerUrl;
      })
    .catch(error => console.log(error));
}

initiatePuppeteer();

// Assign button to puppeteer function

document
  .getElementById("puppeteer-button")
  .addEventListener("click", doPuppeteerThings);

async function doPuppeteerThings() {
  const browser = await puppeteer.connect({
    browserWSEndpoint: browserWSEndpoint
  });
  const page = await browser.newPage();

  // Your puppeteer code goes here

}
```

Hopefully this is of help to someone. Happy automating.

Edit: Upon reflection, I'm pretty sure this can be accomplished with just basic javascript and dom manipulation via chrome.tabs.create and chrome.tabs.executeScript API and editing `.value` of input fields and what not. In my use case, it involved logging into multiple platforms and clicking various buttons through various states which may have been messy with plain javascript and I've already committed to puppeteer. Although if you just had a form that needed to be inputted and submitted, that's a different story.