A Chrome app to rule them all (Paywalls) : Part 1
Tags: python, paywall, javascript, extension
Published: 2020-04-27 19:21:37

They’ve been telling us internet will make us free, that we will have the knowledge at the reach of our browser, infinite possibilities and all that stuff… until you hit the (pay)wall:

![Medium paywall](https://miro.medium.com/max/700/1*4GYUPIaMThPCt_pyrPjjpA.png)

When you find this message, you have 2 options, you pay the member fee or you can use all your knowledge to bypass it¹… Let’s go that way.
There are various types of paywall control techniques:

* By cookie (we will focus on this one today)
* By redirecting to another site
* By hiding content using JavaScript

***

# Paywall by cookie

Medium use cookies to track us, so the first option would be to delete the cookie that triggers this paywall (field **sid** in the medium cookie by the way). The problem is I would have to login again then, because maybe after reading it i want to bookmark it o follow the author (my problem is with the paywall not the content).
But why bother, just open the article in private/incognito mode and voilà, no paywall at all.

![We are in](https://miro.medium.com/max/700/1*dgDctn009yth5a0GCEGDSg.png)

So how can we achieve this with a less friction? Let’s say we build a **Chrome extension** that can detect we were stopped by the paywall and automatically open the site in private mode, pretty simple no?
If you want a quick guide in how to make a default extension see [this Google’s guide](https://developer.chrome.com/extensions/getstarted), i will focus on the paywall bypass, download the [template app](https://github.com/ipacheco-uy/gandalf) and let’s start with that.

# The extension

Our app consist of mainly of :
* **manifest.json** holds the permissions and declare the other files
* **background.js** for communicating the extension
* **hidden.js** detects the paywall warning and triggers the private window
* **display.js** we will use it after in this series

Let’s dive into manifest:
{%gist https://gist.github.com/ipacheco-uy/9eaedf977a2bd4887c1600dfc4073b90 %}

1. Link to the script that holds the extension logic, add listeners and define some constants.
2. The extension need some permissions, webRequest and webRequestBlocking for blocking a request (for the next chapter), tabs for creating a new window or tab. <all_urls> let us operate in all websites, you can change this if you only want specific sites.
3. Just a title for our extension and popup.html is what we render when someone click on the extension icon.
4. Content scripts can change the site content, this one apply to all urls and invoke hidden.js.

So we have defined that for all urls, hidden.js will be executed. This script can access the site body, check if paywall is present and send a message (**internal messaging** is the way our content scripts communicates with the extension).

{%gist https://gist.github.com/ipacheco-uy/a1e0c5e67eaa26c67c301d78a9692400 %}

For medium we can use the id paywall-background-color to check if we have to open in incognito; now its time to check the background script:

{%gist https://gist.github.com/ipacheco-uy/483ef2e1fa740540e7f28c0df20b767b %}

Now that we have our extension ready, we can [load it](https://developer.chrome.com/extensions/getstarted) to the browser and start reading non-stop.
Next in the series we will try to beat the redirection type of paywalls.

***

### Resources

* [Google’s extension develop guide](https://developer.chrome.com/extensions/getstarted)
* [Template app repo](https://github.com/ipacheco-uy/gandalf)

***

[1] All the extension is doing is removing some friction from the user (that could always copy the link, enter in private mode and paste it), by no means we are hacking or altering medium behaviour.