Strava: Export all activities to JSON file
Date: 2020-12-20 00:21:00
In this blog post we'll learn how export all activities from Strava to a JSON file

In my continued playing around with the http://developers.strava.com/[Strava API^], I wanted to write a script to download all of my Strava activities to a JSON file.

As I mentioned in https://markhneedham.com/blog/2020/12/15/strava-authorization-error-missing-read-permission/[a previous blog post^], the approach to authenticating requests has changed in the last two years, so we first need to generate an access token via the OAuth endpoint.
Luckily Odd Eirik Igland https://stackoverflow.com/questions/60873575/how-to-access-authentication-by-strava-api-using-python[shared a script^] showing how to solve most of the problem, and I've adapted it to do what I want.

== Pre-requisites

First we need to install the following libraries:

[source,bash]
----
pip install stravalib fastapi uvicorn jsonlines
----

And the script that we're going to execute next relies on the `CLIENT_ID` and `CLIENT_SECRET` environment variables.
We can get those values from the https://www.strava.com/settings/api[Strava API application page^], and set them by running the following commands:

[source,bash]
----
export CLIENT_ID="client_id"
export CLIENT_SECRET="client_secret"
----

== Generate access token

Now we're ready to generate a Strava access token, using the following script:

.https://github.com/mneedham/materialize-sandbox/blob/main/strava/authenticate.py[authenticate.py^]
[source,python]
----
import os
import pickle

from fastapi import FastAPI
from fastapi.responses import RedirectResponse
from stravalib.client import Client

CLIENT_ID = os.getenv("CLIENT_ID")
CLIENT_SECRET = os.getenv("CLIENT_SECRET")
REDIRECT_URL = 'http://localhost:8000/authorized'

app = FastAPI()
client = Client()


def save_object(obj, filename):
    with open(filename, 'wb') as output:  # Overwrites any existing file.
        pickle.dump(obj, output, pickle.HIGHEST_PROTOCOL)


@app.get("/")
def read_root():
    authorize_url = client.authorization_url(client_id=CLIENT_ID, redirect_uri=REDIRECT_URL)
    return RedirectResponse(authorize_url)


@app.get("/authorized/")
def get_code(state=None, code=None, scope=None):
    token_response = client.exchange_code_for_token(client_id=CLIENT_ID, client_secret=CLIENT_SECRET, code=code)
    access_token = token_response['access_token']
    refresh_token = token_response['refresh_token']
    expires_at = token_response['expires_at']
    client.access_token = access_token
    client.refresh_token = refresh_token
    client.token_expires_at = expires_at
    save_object(client, 'auth/client.pkl')
    return {"state": state, "code": code, "scope": scope}
----

If we run the following command, it will launch a web server on port 8000:

[source,bash]
----
uvicorn authenticate:app --reload
----

.Output
[source,text]
----
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [850840] using statreload
INFO:     Started server process [850842]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
----

Next we need to navigate to http://localhost:8000, from where we'll be redirected to the Strava login page:

image::{{<siteurl>}}/uploads/2020/12/strava-1.png[title="Strava Authorisation"]

Once we click on 'Authorize', the web browser will redirect to http://localhost:8000/authorized, and we'll see something like the following:

[source,json]
----
{
"state": "",
"code": "<our-code>",
"scope": "read,activity:read"
}
----

The script will also save a Strava client containing an access token to `auth/client.pkl`.

== Download Strava activities

We're now ready to download our Strava activites, which we'll do using the following script:

.https://github.com/mneedham/materialize-sandbox/blob/main/strava/download_activities.py[download_activities.py]
[source,python]
----
import os
import pickle
import time

import jsonlines

CLIENT_ID = os.getenv("CLIENT_ID")
CLIENT_SECRET = os.getenv("CLIENT_SECRET")


def check_token():
    if time.time() > client.token_expires_at:
        refresh_response = client.refresh_access_token(client_id=CLIENT_ID, client_secret=CLIENT_SECRET,
                                                       refresh_token=client.refresh_token)
        access_token = refresh_response['access_token']
        refresh_token = refresh_response['refresh_token']
        expires_at = refresh_response['expires_at']
        client.access_token = access_token
        client.refresh_token = refresh_token
        client.token_expires_at = expires_at


def load_object(filename):
    with open(filename, 'rb') as input:
        loaded_object = pickle.load(input)
        return loaded_object


try:
    client = load_object('auth/client.pkl')
    check_token()
    athlete = client.get_athlete()
    print("For {id}, I now have an access token {token}".format(id=athlete.id, token=client.access_token))

    with jsonlines.open('data/activities-all.json', mode='w') as writer:
        for activity in client.get_activities():
            writer.write({
                "id": activity.id,
                "distance": activity.distance.get_num(),
                "moving_time": activity.moving_time.seconds,
                "elapsed_time": activity.elapsed_time.seconds,
                "total_elevation_gain": activity.total_elevation_gain.get_num(),
                "elev_high": activity.elev_high,
                "elev_low": activity.elev_low,
                "average_speed": activity.average_speed.get_num(),
                "max_speed": activity.max_speed.get_num(),
                "average_heartrate": activity.average_heartrate,
                "max_heartrate": activity.max_heartrate,
                "start_date": str(activity.start_date)
            })

except FileNotFoundError:
    print(
        "No access token stored yet, run uvicorn authenticate:app --reload and visit http://localhost:8000/ to get it")
    print("After visiting that url, a pickle file is stored, run this file again to download your activities")
----

This script does the following:

* Loads the Strava client object from `auth/client.pkl`
* Uses the refresh token to generate a new access token if the old one has expired
* Iterates through all of our activities and writes them to `data/activities-all.json`

We can run the script like so:

[source,python]
----
python download_activities.py
----

After it's finished running, `data/activities-all.json` has the following contents:

[source, bash]
----
head -n3 data/activities-all.json
----

.Output
[source,json]
----
{"id": 4489569395, "distance": 13751.9, "moving_time": 4150, "elapsed_time": 4155, "total_elevation_gain": 96.9, "elev_high": 79.3, "elev_low": 25.0, "average_speed": 3.314, "max_speed": 4.6, "average_heartrate": 149.8, "max_heartrate": 176, "start_date": "2020-12-19 07:05:51+00:00"}
{"id": 4485841116, "distance": 12881.6, "moving_time": 3890, "elapsed_time": 3946, "total_elevation_gain": 90.7, "elev_high": 74.3, "elev_low": 22.9, "average_speed": 3.311, "max_speed": 4.4, "average_heartrate": 155.0, "max_heartrate": 191, "start_date": "2020-12-18 05:35:08+00:00"}
{"id": 4478032915, "distance": 13148.8, "moving_time": 3998, "elapsed_time": 4039, "total_elevation_gain": 100.4, "elev_high": 69.9, "elev_low": 22.9, "average_speed": 3.289, "max_speed": 5.3, "average_heartrate": 156.6, "max_heartrate": 183, "start_date": "2020-12-16 05:35:09+00:00"}
----