A fully realized OAuth2 Flow in Bash
Tags: bash, oauth2, shell
Published: 2023-07-25 12:03:28

In this article, I describe how to implement a fully realized OAuth2 flow in a bash script. The goal is to minimize external dependencies while still maintaining full support. Features include:

- Open the browser to initiate the third party authentication flow.
- Automatically retrieve the token from the oauth2 callback.
- Store the refresh token for future script invocations.

The dependencies are minimal and available by default on most modern Linux distros:

- `nc` or `netcat` - used to listen for the HTTP callback from the oauth flow
- `xgd-open` or equivalent - opens the browser
- `jq` - parse the JSON responses
- `grep` and `cut` - extract values from strings
- `curl` or `wget` - make HTTP requests to the OAuth2 backend

For the purposes of this article I will demonstrate using the Google Data API OAuth2 flow but this technique should be adaptable to any standard OAuth2 flow.

## Preamble

To start we will need an OAuth2 client ID and to register for API access through the API console. This process will obviously differ greatly for each API and every context you use an API in.

To follow along here you can get access to the Google Data APIs through the [Google Cloud Developer Console](https://console.cloud.google.com/). You'll need to create a new project (or select an existing one), access _APIs and Services_, go to _Credentials_, and create a new OAuth 2.0 client ID. The application type will be "Desktop". You will have to create a new consent screen if you do not have one already, the default values are fine here. This client ID will be for a test app with limited access, which is fine for testing. Google provides an explanation of OAuth2 and their specific flow [here](https://developers.google.com/identity/protocols/oauth2).

Once you have an OAuth2 client ID and client secret, we have everything we need to start. For this demo, we will store these in environment variables in the script:

```bash
CLIENT_ID="<...>.apps.googleusercontent.com"
CLIENT_SECRET="<...>"
```

I have elided my client id and secret here as they are secret values which provide access to your specific app for purposes of security, auditing, and billing. How do distribute these keys with an app is an open question I do not have a satisfactory answer to. A few options I've seen so far.

1. **Encrypt the tokens within the app:** this is of limited value, while it avoid the tokens appearing as plain text, the encryption key is inside a self contained app, ergo, it can still be reverse engineered.

2. **Store the tokens securely on a server and provide access with a web service:** this provides good security (or at least as good as the security you provide on your web service) but now makes what could be a self contained app dependent on your web service being online.

3. **Require the end user to provide their own app credentials:** this is obviously highly inaccessible to all but the most motivated and technically capable end users.

For a bash script, option 3 might be possible, and certainly this is not an issue if you are building a tool for your own use or within your own team or organization, but I will not address a specific solution in this article.

## Invoke the browser

Now that we have the OAuth2 client secret and id, we can generate the OAuth2 authentication URL and open the browser at that location.

```bash
AUTH_URL="https://accounts.google.com/o/oauth2/auth\
?client_id=${CLIENT_ID}\
&redirect_uri=http://localhost:8000\
&scope=https://www.googleapis.com/auth/youtube.readonly\
&response_type=code\
&access_type=offline"

xdg-open $AUTH_URL 2>/dev/null
```

Here we create our authentication URL. It takes several parameters:

- `CLIENT_ID` and `CLIENT_SECRET`: for the developer app we created earlier.
- `redirect_uri`: where to pass the eventual authentication code back to.
- `scope`: the specific pieces of data which we are requesting permission to access.
- `response_type`: the specific authentication token we want returned. In this case we are asking for an authentication `code` which we can exchange for our access and refresh tokens later.
- `access_type`: the type of access we request, in this we request `offline` in order to get long-lived tokens that can be stored between invocations.

Finally we use `xdg-open` to invoke the URL. This will request that the OS open the URL in the default application (hopefully a web browser) and redirect any warnings to `/dev/null` just to make the script interface cleaner.

## Retrieve the Authentication Code

In the OAuth2 flow, we invoke a browser to load the third party service's authentication page. In this case, we open a Google website, where the user will see a sign in page hosted by Google. They will then be presented with the permissions page to approve the request for data that we embedded in our `scope` parameter. Once the user approves the permissions, the Google page must pass the authentication code back to our script. This is where the `redirect_uri` parameter from earlier comes into play. The Google site will send a redirect to the URI that we specify but with the addition of the authentication code and other information added as query parameters in the URL.

So we need to do something to capture this callback request. The first thing we do is specify a `redirect_uri` of `http://localhost:8000` meaning the request will come back (encrypted) to the machine we are running on at a port which is usually available (an enhancement to this script might be to verify the port is available before running the script and selecting a new port at random if the port is in use--this would increase reliability, but would increase dependencies).

Next we must listen for this request. A typical way to do this is to spin up a http server to process the request. This is rather expensive though and would be a significant dependency. But we don't actually need a full HTTP server. We don't even need to process an HTTP request exactly. We just need to extract the provided authentication code from the URL and then pass back a canned response. We can do this with `nc` or `netcat`.

```bash
RESPONSE=$(\
  echo -e "HTTP/1.1 200 OK\r\n\r\n\
    Authorization code received. You can close this page now." \
  | nc -N -l -p 8000)

AUTH_CODE=$(echo "$RESPONSE" \
  | grep -o "code=.*" \
  | cut -d'=' -f2 \
  | cut -d'&' -f1)
```

Here we run `netcat` or `nc` with the command `nc -N -l -p 8000` to open port 8000 and to listen on that port using `-l`. We also tell `nc` to close the port after receiving an `EOF` on the input with `-N`. This ensures that `nc` terminates on it's own after the HTTP request is received and the response is sent.

Next, we pipe the HTTP response we wish to send back to the user into `nc` using the `echo -e "HTTP/1.1 200 OK\r\n\r\nAuthorization code received. You can close this page now."`. In this case, we're sending a simple plain text HTTP response starting with the HTTP status code 200 and OK message, with a simple plain text message instructing the user to close the browser window.

Finally, we capture the output of the `nc` command into the `RESPONSE` variable. That content will consist of the HTTP request that the browser made to the `redirect_uri`. This text will include the `url` that the browser loaded, which includes the query parameter `?code=<...>`, we extract that query parameter using `grep` and `cut`, but it would also be possible to use other tools like `sed` or `awk` for this.

Now we have the authentication code.

## Exchange the Authentication Code for Tokens

Once we have the authentication code, we must exchange it for the access and refresh tokens. The access token is the token we will use to make actual API requests, while the refresh token is the long lived token that we can store use to get a new access token as needed.

```bash
TOKEN_RESPONSE=$(curl -s --request POST \
  --url "https://accounts.google.com/o/oauth2/token" \
  --header 'content-type: application/x-www-form-urlencoded' \
  --data-urlencode "code=$AUTH_CODE" \
  --data-urlencode "client_id=$CLIENT_ID" \
  --data-urlencode "client_secret=$CLIENT_SECRET" \
  --data-urlencode "redirect_uri=http://localhost:8000" \
  --data-urlencode "grant_type=authorization_code")

ACCESS_TOKEN=$(echo "${TOKEN_RESPONSE}" \
  | jq -r '.access_token')

REFRESH_TOKEN=$(echo "${TOKEN_RESPONSE}" \
  | jq -r '.refresh_token')
```

Here we are using `curl` to make an HTTP request to the google endpoint which exchanges the authentication code for the access and refresh tokens. In this case, we use a `POST` request to end the endpoint url-encoded form data. We pass the authentication code, client id, client secret, and tell the endpoint we are using an `authorization_code`. The `redirect_uri` is required, but not used in this case.

The response to this request is captured and the we use `jq` to parse the JSON response and extract the relevant fields.

Finally we have the access token that we need to make API requests to the Google Data APIs. We can pass the token to `curl` to authenticate requests to the API either through an explicit header:

```bash
RESPONSE=$(curl -s --request GET \
    --url "https://www.googleapis.com/youtube/v3/playlists?..." \
    --header "Authorization: Bearer $ACCESS_TOKEN")
```

Or through the `--oauth2-bearer` parameter:

```bash
response=$(curl -s --request GET \
    --url "https://www.googleapis.com/youtube/v3/playlists?..." \
    --oauth2-bearer="$ACCESS_TOKEN")
```

## Store and Reuse the Refresh Token

While we have an access token that we can use to make API requests to Google, that token will expire quickly. Thus, this authentication flow must be repeated every time the script is invoked. This is inconvenient for the user.

To improve on this, we will do is store the refresh token in a config file so that we can re-authenticate the script in subsequent invocations without needing to go through the OAuth2 authentication flow. First, lets set some paths.

```bash
TOKENS_DIR="$HOME/.config/<myscript>"
TOKENS_FILE="$TOKENS_DIR/tokens"
```

Here we just use the standard config directory on any Linux system, specifically in the users `$HOME` we store in a hidden directory `.config` and under that directory, we store in a directory named for our script. Our config file is simply named `tokens`.

Next, we store our refresh token in that directory.

```bash
mkdir -p "$TOKENS_DIR"
echo "REFRESH_TOKEN=$REFRESH_TOKEN" > "$TOKENS_FILE"
```

First we ensure the directory exists with `mkdir -p`, then we echo a string to set a variable and pipe it into our file.

Finally, as the first thing we do in our script, we can `source` this file and then check if the `REFRESH_TOKEN` variable is set. If it is, we use it to get a new access token, otherwise we do the full authentication flow.

```bash
source "$TOKENS_FILE"

if [[ -n $REFRESH_TOKEN ]]; then
  RESPONSE=$(curl -s --request POST \
    --url="https://oauth2.googleapis.com/token" \
    --header 'content-type: application/x-www-form-urlencoded' \
    --data-urlencode="client_id=$CLIENT_ID" \
    --data-urlencode="client_secret=$CLIENT_SECRET" \
    --data-urlencode="refresh_token=$REFRESH_TOKEN" \
    --data-urlencode="grant_type=refresh_token" )

  ACCESS_TOKEN=$(echo "$RESPONSE" \
    | jq -r '.access_token')
fi

if [[ -z $ACCESS_TOKEN || "$ACCESS_TOKEN" == "null" ]]; then
  #Full authentication flow
fi
```

Here we again use `curl` to send an HTTP POST request to the Google tokens endpoint with all the url-encoded form fields in the request. And again we use `jq` to parse the JSON response for the relevant field. Finally, we check if we have a valid access token and if we do not, we invoke the full OAuth2 authentication flow. (A quirk of `jq` means that if the `access_token` field is not present in the JSON response--perhaps because of an error or authentication expiry--the `ACCESS_TOKEN` field will receive the string value `"null"` which we must separately check for).

Each of these snippets can be combined together into a full script that implements the fully ergonomic OAuth2 flow a user would expect, but some judicious use of bash functions would improve re-usability and readability.

I hope you find it useful.