Request interception with Puppeteer and Playwright
Tags: javascript, webdev, puppeteer, playwright
Published: 2020-11-05 11:46:07

---
title: Request interception with Puppeteer and Playwright
published: true
tags: 
  - javascript
  - webdev
  - puppeteer
  - playwright
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/6vyaraboikxhd6n3oqqn.png
---

When we browse the web, a series of HTTP requests and responses are exchanged between our browser and the pages we are visiting. There are scenarios in which it is useful to monitor or manipulate this traffic, instead of letting it happen as-is.

## Request interception

Request interception enables us to observe which requests and responses are being exchanged as part of our script's execution. For example, this is how we could print them out when we load our [test website](https://danube-webshop.herokuapp.com):

With Puppeteer:

{% gist https://gist.github.com/ragog/8fc27f223215a08c68800c9ce7dc769b %}

With Playwright:

{% gist https://gist.github.com/ragog/7d80a73c40da0af366e45bd0fdafdd13 %}

We might want to intervene and filter the outgoing requests. For example, when [scraping web pages](https://theheadless.dev/posts/basics-scraping/), we might want to block unnecessary elements from loading in order to speed up the procedure and lower bandwidth usage.

In the following snippet we are going to abort all requests for images on our test website. We will identify them based off of their [`resourceType`](https://pptr.dev/#?product=Puppeteer&version=v5.2.1&show=api-httprequestresourcetype), while letting all other requests through without modification.

For Puppeteer:

{% gist https://gist.github.com/ragog/a9cb9c61d2fddec37df640024376d5f9 %}

For Playwright:

{% gist https://gist.github.com/ragog/bd1188eaec3e453d37367bb9c9f8d023 %}

As a result, you will see the website logo not being loaded.

![test site without images](https://theheadless.dev/request-interception-image.png)

Similarly, switching the `resourceType` to `stylesheet` would result in the target website loading without any CSS styling.

![test site without css](https://theheadless.dev/request-interception-css.png)

## Response interception

Isolating one or more software components from their dependencies makes them easier to test. We can do so by substituting interactions with such dependencies with simulated, simplified ones. This is also known as _stubbing_. 

Puppeteer makes it easy for us, as for every request we can intercept we also can stub a response. This functionality is [not yet available in Playwright](https://github.com/microsoft/playwright/issues/1774).

Every time we load it, our test website is sending a request to its backend to fetch a list of best selling books. For our example, we are going to intercept this response and modify it to return a single book we define on the fly.

For Puppeteer:

{% gist https://gist.github.com/ragog/03b5c528915876dd6b51153e83e21737 %}

Here is what the homepage will look like with our stubbed response:

![test site with stubbed response](https://theheadless.dev/response-interception.png)

On macOS/Linux and Windows, you can run the above examples as follows: `node request-interception.js`

## Takeaways

1. Puppeteer and Playwright give us control over outgoing HTTP requests.
2. With Puppeteer we can easily stub HTTP responses. 

## Further reading

1. Official documentation on this topic from [Puppeteer](https://pptr.dev/#?product=Puppeteer&version=v5.2.1&show=api-class-httprequest) and [Playwright](https://playwright.dev/#version=v1.3.0&path=docs%2Fnetwork.md&q=handle-requests).
2. [Mocks Aren't Stubs](https://martinfowler.com/articles/mocksArentStubs.html) by Martin Fowler.

_This article was originally posted on [theheadless.dev](https://theheadless.dev)_