Plotting Bookmarks with Flask, Matplotlib, and OAuth 2.0
Tags: twitter, python, showdev, flask
Published: 2022-06-30 01:51:50

---
title: Plotting Bookmarks with Flask, Matplotlib, and OAuth 2.0
published: True
description: Learn how to plot your Bookmarks with Flask, Matplotlib, and OAuth 2.0.
tags: twitter, python, showdev, flask
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
---

I don’t Bookmark Tweets very often, and I was wondering how constantly I use the Bookmark functionality on Twitter. To figure out exactly how often I bookmark Tweets, I built a [demo application](https://plot-bookmarks.onrender.com/) in Python using [Flask](https://flask.palletsprojects.com/en/2.1.x/).

This application will authorize your account using the [OAuth 2.0 Authorization Code Flow with PKCE](https://developer.twitter.com/en/docs/authentication/oauth-2-0/authorization-code) and plot up to your last 100 Bookmarked Tweets. The full code sample can be found [on our GitHub](https://github.com/twitterdev/Plot-Bookmarks/).

## Step 1: Getting set up with the Twitter API

If you don’t already have access to the Twitter API, [you can sign up for a developer account.](http://t.co/signup) Once you have an approved developer account, you must create a Project in the [developer portal](https://developer.twitter.com/en/portal/dashboard). Each Project contains an [App](https://developer.twitter.com/en/docs/basics/apps/overview) with which you can generate the credentials required to use the Twitter API. You will also need to have OAuth 2.0 turned on in your App’s settings. In our [platform overview](https://developer.twitter.com/en/docs/platform-overview), you can learn more about getting started with the Twitter API.

## Step 2: Setting up your environment

After obtaining access to the Twitter API, you are ready to set up your `requirements.txt` file.

First, you’ll need to create a directory for this project in your terminal and navigate into it. After making a `requirements.txt` file, you can open it in your chosen code editor. In your terminal, run the following command:

```bash
mkdir plot-bookmarks
cd plot-bookmarks
touch requirements.txt
```

Add these packages to `requirements.txt` file

```
certifi==2022.6.15
charset-normalizer==2.0.12
click==8.1.3
cycler==0.11.0
Flask==2.1.2
fonttools==4.33.3
gunicorn==20.1.0
idna==3.3
importlib-metadata==4.11.4
itsdangerous==2.1.2
Jinja2==3.1.2
kiwisolver==1.4.3
MarkupSafe==2.1.1
matplotlib==3.5.2
numpy
oauthlib==3.2.0
packaging==21.3
pandas==1.3.5
Pillow==9.1.1
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2022.1
requests==2.28.0
requests-oauthlib==1.3.1
six==1.16.0
urllib3==1.26.9
Werkzeug==2.1.2
zipp==3.8.0
```

To install all the packages in your requirements.txt file, in your terminal, run the following command:

```
pip install -r requirements.txt
```

## Step 3: Creating your templates

Now that you have your environment set, you are ready to define your templates. Templates in Flask allow you to define the HTML of your pages. For this example, you’ll define two templates, `base.html`, and `index.html`.

Inside of the `plot-bookmarks` directory, you will want to make a new directory in your terminal and two files for templates (`base.html` and `index.htm`l). In your terminal, run the following command:

```
mkdir templates
cd templates
touch base.html
touch index.html
```

The `base.html` file contains the framework for all HTML pages in the application and can be extended to apply to other pages.

Add the following code to `base.html`:

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Plot Bookmarks!{% block title %}{% endblock %}</title>
    <link
      rel="stylesheet"
      href="https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/css/bootstrap.min.css"
    />
  </head>

  <body>
    <div class="container">
      <h1>Plot Bookmarks by Date</h1>
      {% block containercontent %}{% endblock %}
      <hr />
      <div class="footer">
        <p class="text-muted">
          <small
            >This is a basic demo created by the
            <a href="https://twitter.com/TwitterDev">@TwitterDev</a> team. It is
            written in Python, using
            <a href="http://flask.pocoo.org/">Flask</a>. Basic CSS is provided by
            <a href="http://getbootstrap.com/">Bootstrap</a>. The source code
            for this project can be found at
            <a href="https://github.com/twitterdev/plot-bookmarks"
              >github/twitterdev/plot-bookmarks</a
            >.</small
          >
        </p>
      </div>
    </div>
  </body>
</html>
```

The `index.html` template extends from the `base.html` template and will be used in your code later on to introduce the application to the user. Add this code to `index.html`.

```html
{% extends "base.html" %}
{% block containercontent %}
<p class="lead">This app lets you plot your Bookmarked Tweets by Date.</p>
<div class="hero-unit">
	<p>To get started, we'll need you to authorize Hello world twitter app to access your bookmarks on Twitter using OAuth 2.0. Learn more information about <a href="https://developer.twitter.com/en/docs/authentication/oauth-2-0">OAuth 2.0</a></p>
	<p>Here's how it works:</p>
	<ol>
		<li>We will redirect you to Twitter.com for you to authorize our app.</li>
		<li>While at Twitter.com, you will authorize bookmarks-to-Pinterest to access your bookmarks. Upon authorization, we will get your recent 100 bookmarks.</li>
		<li>Next we will display your Bookmarked Tweets by date.</li>
	</ol>
</div>
<p>Ready to give this a shot?</p>
<a href="/start" class="btn btn-small btn-success">Let's Begin</a>
{% endblock %}
```

## Step 4: Creating your app.py file

Now that you have your templates set, you will create an `app.py` file which will be the main file for your application.

To get started, navigate to the `plot-bookmarks` directory in your terminal and create an `app.py` file by running the following command:

```
cd ..
touch app.py
```

You will also set up environment variables to ensure you aren’t passing in your secrets directory. To achieve this, you will define environment variables. In your terminal, run the following command:

```
export CLIENT_ID=’xxxxxxxxxxxxxx’
export CLIENT_SECRET=’xxxxxxxxxxx’
export REDIRECT_URI=’http://127.0.0.1:5000/oauth/callback’
```

You can obtain your own OAuth 2.0 Client ID and Secret inside of your App’s settings in the developer portal. Use your Client ID and Secret instead of the `xxxxxxxxxxxxxx`.

I am currently using the type of App as a public client, a single-page App. Inside your App’s OAuth settings in the Developer Portal, ensure that you also have the above redirect URI defined for locally testing your application.

You are ready to start editing your `app.py` file in your chosen code editor. First, you will import the packages you’ll need for your application. You will use `base64`, `hashlib`, and `re` to create your code verifier and code challenge. The `os` package will be used to set environment variables and randomize values. You will use the `requests` package to make HTTP requests to the Twitter API and `pandas` to create a data frame so you can plot values. You will use io and the `matplotlib` features for creating your bookmarks plot and `flask` for creating the web framework.

Add this code to your `app.py` file.

```python
import base64
import hashlib
import os
import re
import requests
import pandas as pd
import io

from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure
from requests_oauthlib import OAuth2Session
from flask import Flask, request, redirect, session, render_template, Response
```

First, you will need to set a variable for your app, which is the start of every Flask app. After, you’ll need to create a secret key for your app to be a random string using the os package.

Add this code to below your import statements.

```python
app = Flask(__name__)

app.secret_key = os.urandom(50)
```

To bring the environment variables into your code, you can use the os package to get the variables you set in your terminal.

Add this code after the `app = Flask(__name__)` code block.

```python
client_id = os.environ.get("CLIENT_ID")
client_secret = os.environ.get("CLIENT_SECRET")
```

Additionally, you will obtain the environment variable you set for your redirect URI and create a variable for the authorization and token URLs for generating an OAuth 2.0 token.

Add this code below the `client_secret` variable.

```python
redirect_uri = os.environ.get("REDIRECT_URI")
auth_url = "https://twitter.com/i/oauth2/authorize"
token_url = "https://api.twitter.com/2/oauth2/token"
```

Scopes are what sets the permissions for an application. With OAuth 2.0, there are more fine-grained permissions, so you only need to ask the user for what you need. For this application, you will need the following scopes:

* `tweet.read`, which gives you access to read Tweets.
* `users.read` allowing you to obtain information about users
* `bookmark.read` giving you access to an authenticated user’s bookmarks

Add this code below the `token_url` variable.

```python
scopes = ["tweet.read", "users.read", "bookmark.read"]
```

Since Twitter’s implementation of OAuth 2.0 is PKCE compliant, you will need to set a code verifier, a secure random string. The code verifier is used to create the code challenge. The code challenge is a base64 encoded string of the SHA256 hash of the code verifier.

Add this code below the `scopes` variable.

```python
code_verifier = base64.urlsafe_b64encode(os.urandom(30)).decode("utf-8")
code_verifier = re.sub("[^a-zA-Z0-9]+", "", code_verifier)

code_challenge = hashlib.sha256(code_verifier.encode("utf-8")).digest()
code_challenge = base64.urlsafe_b64encode(code_challenge).decode("utf-8")
code_challenge = code_challenge.replace("=", "")
```

At this point, you’ll want to set a function for getting an authenticated user’s Bookmarks back from the Twitter API. For debugging, you’ll want to add a `print` statement so that you know when you are requesting your server. This function uses the `requests` package to request the [Bookmarks lookup endpoint](https://developer.twitter.com/en/docs/twitter-api/tweets/bookmarks/introduction). Since you will want to use the date of the Bookmark for the plot later, make sure that you pass in a `tweet.field` of `created_at` as a `param`.

```python
def get_bookmarks(user_id, token):
    print("Making a request to the bookmarks endpoint")
    params = {"tweet.fields": "created_at"}
    return requests.request(
        "GET",
        "https://api.twitter.com/2/users/{}/bookmarks".format(user_id),
        headers={"Authorization": "Bearer {}".format(token["access_token"])},
        params=params,
    )
```

Now, you’ll want to set up the landing page of your application. This will render the template from the index.html file you created earlier. The @`app.route("/")` indicates this is the first page at the beginning of the application a user will be taken to.

```python
@app.route("/")
def hello():
    return render_template("index.html")
```

After clicking the “Let’s Begin” button on the application you will be directed to a page for the user to log into Twitter and authorize your App to make requests on the behalf of the user.

```python
@app.route("/start")
def demo():
    global twitter
    twitter = OAuth2Session(client_id, redirect_uri=redirect_uri, scope=scopes)
    authorization_url, state = twitter.authorization_url(
        auth_url, code_challenge=code_challenge, code_challenge_method="S256"
    )
    session["oauth_state"] = state
    return redirect(authorization_url)
```

After the user logs in, they will be directed to the callback. The callback will generate an object called `token` that contains a series of tokens that includes an access token. The access token is what is used to make a request to the Bookmarks lookup endpoint. You will print out this token object to debug on the server.

```python
@app.route("/oauth/callback", methods=["GET"])
def callback():
    code = request.args.get("code")
    token = twitter.fetch_token(
        token_url=token_url,
        client_secret=client_secret,
        code_verifier=code_verifier,
        code=code,
    )
    print(token)
```

After getting a token, you will want to make a request to the authenticated user lookup endpoint. Since the Bookmarks lookup endpoint requires you to have the ID of the user you are obtaining bookmarks on behalf of. You will pass the `user_id` and `token` object into the `get_bookmarks` function you defined earlier in your code. You’ll save your Bookmarks into a JSON object so that you can easily convert into a data frame.

```python
    user_me = requests.request(
        "GET",
        "https://api.twitter.com/2/users/me",
        headers={"Authorization": "Bearer {}".format(token["access_token"])},
    ).json()
    print(user_me)
    user_id = user_me["data"]["id"]
    bookmarks = get_bookmarks(user_id, token).json()
```

Now you can create a data frame in Pandas for your Bookmarks, and adjust the date so that it is easy to work with. You’ll also plot a count of Bookmarks per day. To do so, you can use the `groupby` function of Pandas. You will define your `labels` as the date the Tweet was created and the count as the `values`.

```python
    df = pd.DataFrame(bookmarks["data"])
    df["created_at"] = pd.to_datetime(df["created_at"])
    df["created_at"] = df["created_at"].dt.date
    counts = pd.DataFrame({"count": 
    df.groupby(["created_at"]).size()}).reset_index()
    labels = counts["created_at"]
    values = counts["count"]
```

To plot the labels and values you will use [Matplotlib](https://matplotlib.org/). Once you have your graph you can save it and render in a view.

```python
    fig = Figure()
    ax = fig.subplots()
    ax.set_title("Number of Bookmarks by Date")
    ax.bar(labels, values, color="orange", width=26)
    output = io.BytesIO()
    FigureCanvas(fig).print_png(output)
    return Response(output.getvalue(), mimetype="image/png")

if __name__ == "__main__":
    app.run()
```

To run the file locally, run the following line in your terminal:

```
python app.py
```

For further testing or if you run into any errors you may want to consider turning on Flask’s [debug mode](https://flask.palletsprojects.com/en/2.1.x/debugging/).

## Step 5: Deployment

I currently have this project deployed using [Render](https://render.com/). I used a similar method to what’s described in their [documentation on flask deployment](https://render.com/docs/deploy-flask).

To serve the correct URI redirect, I had to deploy the application first to figure out what my URL would be and set it up in my environment variables in Render and the developer portal.

The redirect URI for my application was as follows:

https://plot-bookmarks.onrender.com/oauth/callback

I also set up environment variables for Client ID and Client Secret in my environment variable in my settings in Render.

## Next steps

Hopefully, this can be a starting point for you to get started with the Bookmarks endpoint, plotting in Python and OAuth 2.0. This code sample can be easily extended to be part of a complete application that includes more graphs and dynamic content. 

Be sure to inform us on the [forums](https://twittercommunity.com/) if you run into any troubles along the way, or Tweet us at @TwitterDev if this tutorial inspires you to create anything.