Using code coverage to debug large Go application
Tags: go, testing
Published: 2023-08-15 12:45:25

Code coverage is typically used as a code quality metric. 

One can run

```bash
go test -coverpkg ./... -coverprofile test.cov ./...
```

and receive a `test.cov` file containing information on how often particular parts of code were triggered during execution.

This is a great tool to keep your tests relevant.

However, code coverage can also help you to debug large codebases!

Imagine a situation when a seemingly innocent change in code leads to broken tests. Getting to the root cause might be non-trivial if there is a lot of code involved in execution. 

In smaller cases, you could walk through all the statements with a debugger and compare state and conditions, but when you have thousands of statements, this approach is not really feasible.

Code coverage can help you to narrow down the scope of debugging.

Here is the recipe. The prerequisite is that you have a test that passes in original code, and fails in new.

### Collect coverage

Collect coverages of both failure and pass. 

It is important to collect coverage across all packages to have a wholistic picture. You can use `./...` or something like `my-module/...`, `github.com/foo/my-module/...` for `-coverpkg`.


Apply the change and run the test.

```bash
go test -cover -coverpkg ./... -coverprofile new.cover -run ^Test_Foo$ ./path/to/tested/package
```

Then revert the change and collect coverage again.

```bash
go test -cover -coverpkg ./... -coverprofile orig.cover -run ^Test_Foo$ ./path/to/tested/package
```

After that, you will end up having two files: `orig.cover` and `new.cover`.

They might be big, but hopefully not very different.

Coverage file may look like this.

```
mode: set
github.com/swaggest/jsonschema-go/camelcase.go:14.31,21.22 6 1
github.com/swaggest/jsonschema-go/camelcase.go:21.22,22.27 1 1
github.com/swaggest/jsonschema-go/camelcase.go:22.27,24.4 1 1
github.com/swaggest/jsonschema-go/camelcase.go:26.3,26.27 1 1
github.com/swaggest/jsonschema-go/camelcase.go:26.27,28.4 1 1
github.com/swaggest/jsonschema-go/camelcase.go:30.3,30.27 1 1
.....
```

The first line says which mode of coverage collection was used. 

All the other lines describe a span of code (`file:start_line.start_col,end_line.end_col`) with a total number of statements in that span and a number of statements executed.

### Create coverage diff

Coverage files are ordered alphabetically, so they are friendly to `diff`.

```bash
diff orig.cover new.cover > diff.cover
```

The resulting file may look somewhat similar to this.

```diff
305c305
< github.com/swaggest/jsonschema-go/helper.go:75.46,78.16 2 1
---
> github.com/swaggest/jsonschema-go/helper.go:75.46,78.16 2 0
307c307
< github.com/swaggest/jsonschema-go/helper.go:82.2,84.46 2 1
---
> github.com/swaggest/jsonschema-go/helper.go:82.2,84.46 2 0
309c309
< github.com/swaggest/jsonschema-go/helper.go:88.2,88.15 1 1
```

You can already spot that the lines here are mostly different in the number of executed statements. This is the clue for us that those places in code are the best candidates to have a deeper look with a debugger.

Let's make this diff more convenient to follow and check.

### Coverage reporting

Lines starting with `< ` represent the original code, lines starting with `> ` are about the new code.

We can build filtered coverage reports from the diff.

For that, we need to restore the `mode: set` and remove `"< "` or `"> "` from the lines.

```bash
echo "mode: set" > orig_flt.cover && grep "< " diff.cover | sed 's/< //' >> orig_flt.cover

echo "mode: set" > new_flt.cover && grep "> " diff.cover | sed 's/> //' >> new_flt.cover
```

Now, there are two coverage files that only contain differences between two runs.

We can conveniently inspect them with standard `go tool cover`.

```bash
go tool cover -html=orig_flt.cover -o orig_flt.html
go tool cover -html=new_flt.cover -o new_flt.html
```


![Coverage diff screenshot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4ac5idxpo6xoexk3crej.png)

You can see which parts of code were executed differently, this can lead to an idea of what could be the root cause or at least can give a hint where debugger break point should be set.

I hope you've enjoyed reading this as much as I enjoyed digging into a huuuuge code base with this approach. :)