A better puppeteer.evaluate
Tags: node, javascript, framework
Published: 2019-10-28 21:41:56

### Execute JavaScript with puppeteer
To execute JavaScript code on your puppeteer instance, you can use the provided `evaluate` method. e.g.
```javascript
const title = await page.evaluate(() => document.title);
```
This returns the document title from the puppeteer instance back to the node process.

But the way this works is actually quite limited and error prone.
This already errors at runtime:
```javascript
const selector = `p`;
const paragrapahs = await page.evaluate(() => {
    const elements = document.body.querySelectorAll(selector); //runtime error
});
```
(╯°□°)╯︵ ┻━┻

### Why?
Because the callback that you provide in the `evaluate` method gets **serialized**.
Meaning by the time it arrives at the browser it will have lost all information about closures/imports etc. What a bummer.

If you're like me and want to separate your evaluate code to an external file, use node modules etc, you're out of luck. It just doesn't work.

Worse, you will get no indication that this will not work, the IDE says all good, go ahead run it, only to be hit with a runtime error :(

To be fair though, the above *fairly simple* example can be made to work, as you can also specify arguments:
```javascript
const selector = `p`;
const paragrapahs = await page.evaluate(selector => {
    const elements = document.body.querySelectorAll(selector);
}, selector /* <- pass selector as argument */);
```
Still, this is very cumbersome to work with, as you just expect closures to be available, after all, that's how JavaScript works.

### A better puppeteer evaluate
For that reason I published a little library(my first) that lets me do just that: 

Aptly named [puppeteer-evaluate2](https://www.npmjs.com/package/puppeteer-evaluate2), it allows me to write my puppeteer callback code like any other JavaScript code, where closures/imports are available.

It's signature looks like this:
```typescript
export function evaluate2<T = any>(page: puppeteer.Page, jsPath: string): Promise<T>
```
You just pass in your puppeteer page object as the 1st parameter and the path to your JavaScript file as the 2nd parameter.

The library then does the rest, it will create a JS bundle, taking the 2nd parameter as the entry point. The finished bundle is then passed on to the actual `page.evaluate` method. Closures and imports are then available.

Here's an example importing lodash:
```javascript
//code.js
export default function() {
    const chunk = require("lodash/chunk");

    return chunk([1, 2, 3, 4], 2);
}
```
```javascript
let response = await evaluate2(page, `./code.js`);
console.log(response); //[[1, 2], [3, 4]]
```
voila :tada:

*The only requisite is that your entry file must export a default function!*
Thanks to the TypeScript API there are checks in place that tell you if you're missing a default function export :muscle:

Creating a bundle, entry file,.. this sounds a lot like webpack, and you're right! It is actually using the webpack API under the hood to create the bundle, but in memory and on the fly.

### Final
If you're asking why did I import lodash with `require` and not via `import`? Because I still consider it more of a POC, hence it only has a `beta` version right now, as I also want to have TypeScript support. But I wanted to share it early to get some feedback, so let me know what you think :)