How I used Chisel to pull stats on Gitlab pipelines
Tags: showdev, javascript, tutorial, opensource
Published: 2020-05-09 13:44:53

I built [chisel.cloud](https://chisel.cloud) in my spare time to automate something I did to derive insights about my Gitlab pipeline times.

In this blog post I’m going to show you how I did it in the hope that it might be useful to you too.

[![chisel app main screen](https://alediaferia.com/wp-content/uploads/2020/05/Screenshot-2020-05-04-at-15.24.59-1024x693.png)](https://chisel.cloud)

As you can see from the picture above, Chisel is still pretty early stage. I decided to publish it anyway because I’m curious to know whether something like this can be useful to you too or not.

## Understanding deployment time

The goal of this exercise was for me to better understand the deployment time (from build to being live in production) of my project and have a data-driven approach as to what to do next.

Since the project in question uses Gitlab CI/CD, I thought of taking advantage of its API to pull down this kind of information.

## Gitlab Pipelines API

The [Gitlab pipelines API](https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines) is pretty straightforward but a few differences between the `/pipelines` and the `/pipelines/:id` APIs means that you have to do a little composition work to pull down interesting data.

Here’s how I did it.

### 1. Pull down your successful pipelines

First thing I did was fetching the successful pipelines for my project.

![chisel app first screen](https://alediaferia.com/wp-content/uploads/2020/05/chisel-1-1024x768.png)

As you can see, this API returns minimal information about each pipeline. What I needed to do next in order to understand pipeline times was to fetch further details for each pipeline.

#### Chisel – Transform

Chisel provides a handy transformation tool that uses [JMESPath](https://jmespath.org/) to help you manipulate the JSON returned by the API you are working with. I used it to extract the pipeline IDs from the returned response.

![chisel app second screen](https://alediaferia.com/wp-content/uploads/2020/05/chisel-2-1024x768.png)

Chisel shows you an live preview of your transformation. Something as simple as `[*].id` is enough for now. The result is an array of pipeline IDs.

Right after obtaining all the IDs I need I can apply another transformation to turn those IDs into pipeline objects with all the relevant information I need for my stats.

Chisel has another kind of transformation type called **Fetch** that helps you transform the selected values into the result of something fetched from a URL.

![chisel app third screen](https://alediaferia.com/wp-content/uploads/2020/05/chisel-3-1024x768.png)

In particular, you can use the `${1}` placeholder to pass in the _mapped_ value. In my case, each ID is being mapped to the `/pipelines/${1}` API.

The result is pretty straightforward.

![chisel app fourth screen](https://alediaferia.com/wp-content/uploads/2020/05/chisel-4-1024x768.png)

### 2. Filter out what you don’t need

As you can see, some of the returned pipelines have a `before_sha`of value `0000000000000000000000000000000000000000`. Those are pipelines triggered outside of merges into `master` so I’m not interested in them.

Filtering those out is as simple as `[?before_sha != '0000000000000000000000000000000000000000`]

![chisel app fifth screen](https://alediaferia.com/wp-content/uploads/2020/05/chisel-5-1024x768.png)

#### The transformation history

As you can see, on the right of the screen there’s a little widget that shows you the transformations you have applied. You can use it to go back and forth in the transformation history and rollback/reapply the modifications to your data.

![chisel app transformation history](https://alediaferia.com/wp-content/uploads/2020/05/transform-history.png)

### 3. The last transformation

The last transformation I need to be able to start pulling out useful information has to turn my output into a set of records.

![chisel app sixth screen](https://alediaferia.com/wp-content/uploads/2020/05/chisel-6-1024x768.png)

I’m selecting only a few fields and turning the result into an array of array. This is the right format to be able to export it as a CSV.

![chiel app csv download screen](https://alediaferia.com/wp-content/uploads/2020/05/csv-export.png)

## Google Sheets

Finally, I can upload my CSV export to [Google Sheets](https://sheets.new) and plot the information I need.

![google sheets import](https://alediaferia.com/wp-content/uploads/2020/05/gsheets-grab.png)

## Conclusion

_Chisel_ is still at its earliest stage of development and it is pretty much tailored on my specific use case but if you see this tool can be useful to you too, please head to the [Github](https://github.com/chiselcloud/chisel) repo and suggest the improvements you’d like to see.

If you liked this post and want to know more about Chisel, **[follow me on Twitter](https://twitter.com/alediaferia)**!

* * *

Featured image by [Dominik Scythe](https://unsplash.com/@drscythe?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/s/photos/chisel?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)

The post [How I used Chisel to pull Gitlab pipelines stats](https://alediaferia.com/2020/05/08/how-used-chisel-pull-gitlab-pipelines-stats/) appeared first on [Alessandro Diaferia](https://alediaferia.com).

{% github chiselcloud/chisel no-readme %}