False-positive Go code coverage
Tags: go, testing, todayilearned, tutorial
Published: 2021-03-09 10:01:43

Go offers tools for code coverage directly in the main installation package, no need for addons or any other tool. But with the initial settings, the coverage results could be much higher than a real coverage.

> 🇨🇿 V češtině si lze článek přečíst na [kutac.cz](https://www.kutac.cz/pocitace-a-internety/falesne-pozitivni-code-coverage-v-go)

---

To understand why the code coverage can be much higher is mandatory to understand, how the coverage is measured. Go has some internal tools, which modify the code before executing tests and then counting occurrences in each path. But is counting occurrences only in the currently tested package.

Let's say you have a package `A` and `B`. Package `A` is fully covered by tests and is using some functions from package `B`, which has zero tests. The result will be that 100% of statements are covered. But if you add a test into package `B`, overall coverage can drop.

# Example
For example purposes I created repo [arxeiss/go-false-positive-coverage](https://github.com/arxeiss/go-false-positive-coverage), which contains 2 completely separated packages `unary` and `binary`.

## Step 1 - False hope I have 100% coverage

Clone the repo mentioned above and try to run the following commands. In the result, you can see a total coverage of 100%. It is only because there are no tests in `unary` package. Then there are no records in `coverage.out` file about other packages, so `go tool cover` knows nothing about them.

```bash
go test -covermode=count -coverprofile=coverage.out ./...
# ?   	github.com/arxeiss/go-false-positive-coverage	[no test files]
# ok  	github.com/arxeiss/go-false-positive-coverage/binary	0.002s	coverage: 100.0% of statements
# ?   	github.com/arxeiss/go-false-positive-coverage/unary	[no test files]

# Count total coverage and coverage per function.
go tool cover -func coverage.out
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:3:	Add		100.0%
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:7:	Sub		100.0%
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:11:	Mul		100.0%
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:15:	Div		100.0%
# total:									(statements)	100.0%
```

## Step 2 - Adding more tests drops overall coverage

There is a prepared test in the `unary` package. Just rename the file `operations_test.go.bak` and remove `.bak` suffix. Now run the same commands as above and total coverage will be 71.4%!.

## Step 3 - Switch `-coverpkg` will come to help

To get the real overall score, it is mandatory to use the `-coverpkg` switch and define all packages to test. This switch has 2 effects. 

1. Will insert records about packages without tests into `coverage.out`.
1. Will measure the coverage across packages. So package without a single test can be fully covered if it is used by other packages.

```bash
go test -covermode=count -coverprofile=coverage.out -coverpkg=github.com/arxeiss/go-false-positive-coverage/... ./...
# ?   	github.com/arxeiss/go-false-positive-coverage	[no test files]
# ok  	github.com/arxeiss/go-false-positive-coverage/binary	0.002s	coverage: 44.4% of statements in github.com/arxeiss/go-false-positive-coverage/...
# ok  	github.com/arxeiss/go-false-positive-coverage/unary	0.002s	coverage: 11.1% of statements in github.com/arxeiss/go-false-positive-coverage/...

go tool cover -func coverage.out
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:3:	Add		100.0%
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:7:	Sub		100.0%
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:11:	Mul		100.0%
# github.com/arxeiss/go-false-positive-coverage/binary/operations.go:15:	Div		100.0%
# github.com/arxeiss/go-false-positive-coverage/main.go:10:		main		0.0%
# github.com/arxeiss/go-false-positive-coverage/unary/operations.go:5:	Abs		100.0%
# github.com/arxeiss/go-false-positive-coverage/unary/operations.go:9:	Sin		0.0%
# github.com/arxeiss/go-false-positive-coverage/unary/operations.go:13:	Cos		0.0%
# total:									(statements)	55.6%
```

## Step 4 - Problems with generated code

The main problem of the `-coverpkg` switch is, it will include also generated code, like Proto files, etc. Those are mostly not tested explicitly, because they are generated. To ignore them, you have to filter out those records from `coverage.out` file with some shell magic. See more in [StackOverflow](https://stackoverflow.com/questions/50065448/how-to-ignore-generated-files-from-go-test-coverage/50177379#50177379).

```bash
cat coverage.out | grep -v ".pb.go" > coverage.filtered.out
go tool cover -func coverage.filtered.out
```