Monitoring GitHub Pull Requests with Prometheus
Tags: github, hacktoberfest, devops, monitoring
Published: 2021-08-10 23:13:06

## The problem

Have you ever wanted to track your open source contributions? Or perhaps monitor contributions made by multiple users for some sort of event? Well I had the exact the same problem 😊. 
With [Hacktoberfest](https://hacktoberfest.digitalocean.com/) just around the corner, I wanted a way to automatically track open source contributions to be able to incentivize participation via prizes or similar. It's fairly difficult to track who opens pull requests to what projects on GitHub at scale within a large organization, but boy do I have the solution for you.

## The solution

I built a [prometheus exporter](https://github.com/circa10a/github-pr-exporter) that takes in a config file with a list of users and will use the GitHub search API to find pull requests created by said users. This exporter exposes some useful data such as `user`, `created_at`, `status` and `link`.

## What is a prometheus exporter?

[Prometheus](https://prometheus.io/) is an open source time series database that uses a "pull" model where it reaches out to configured clients (basically plugins) called exporters. Then ingests the data from the exporters via configured interval which is typically 15 seconds.

### I want to see some users' pull requests, what now?

If you're familiar with prometheus, you can view the github-pr-exporter docs [here](https://github.com/circa10a/github-pr-exporter).

Maybe you're not familiar with prometheus, follow along for how to get started!

## How does it work?

The exporter is written in [Go](https://golang.org/) and utilizes a [GitHub client library](https://github.com/google/go-github) to execute searches using the GitHub search API. The exporter is basically a CLI that takes in configuration options via arguments then runs a web server in a loop that looks for new pull request data periodically.

### Searches

The exporter uses the search API and runs this search for every user: `"type:pr author:<user> created:>=<calculated timeframe>"`

The search is executed once per user instead of a bulk search due to the 256 character limit of the search API, but because the search API has a rate limit of 10 searches per minute for unauthenticated clients, there is a hard 6 second wait between collecting pull request data for each user to avoid said rate limit. This isn't a huge deal since this would process around 1000 users per 90 minutes which completely fine for this kind of data.

### Configuration Options

- `--config` YAML config file with usernames
- `--days-ago` How many days prior to the current to get pull requests from. Defaults to `90`
- `--ignore-user-repos` Ignore pull requests that the user made to their own repositories (no cheating!). Defaults to `false`.
- `--interval` How long (in seconds) to sleep between checking for new data. Defaults to `21600` (6 hours)
- `--port` Which port to run the web server on. Defaults to `8080`

## Deploying github-pr-exporter, prometheus, and grafana

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w5h1127mckkb6nz1t2zv.png)

There are 3 components needed to get up and running to make use of the data.

- [github-pr-exporter](https://github.com/circa10a/github-pr-exporter)
- [Prometheus](https://prometheus.io/)
- [Grafana](https://grafana.com/)

You will also need docker, docker-compose, and git installed. This will run the 3 components above in the form of containers and connect them together via a docker network.


- Start by cloning the repo and cd'ing into the directory

```bash
git clone https://github.com/circa10a/github-pr-exporter.git
cd github-pr-exporter
```

- Start the containers

```bash
docker-compose up -d
```

- Profit!

You should now be able to access grafana at http://localhost:3000 and the preconfigured dashboard at http://localhost:3000/d/h_PRluMnk/pull-requests?orgId=1

> The default admin login is username: `admin` password: `admin`

## Monitoring your own user or others

To change the default configuration, simply update `examples/config.yaml` to include your GitHub username or others then recreate the containers by running:

```bash
docker-compose down
docker-compose up -d
```

Then check out the new details on the dashboard! Please note that it can take around 2 minutes for the dashboard to reflect the new data so just try refreshing or turn on auto-refresh.