The Myth Of 100% Code Coverage
Tags: programming, agile, webdev, javascript
Published: 2023-05-13 05:46:01

---
title: The Myth Of 100% Code Coverage
published: true
tags: programming,agile,webdevelopment,javascript
canonical_url: https://fagnerbrack.com/the-myth-of-100-code-coverage-c7d4c789700d
---

#### The code only exists if you have a test for it

![](https://cdn-images-1.medium.com/max/1024/1*p750KoWY861JKRDS_UmFUw.jpeg)<figcaption>The design artwork of two mythical <a href="https://en.wikipedia.org/wiki/Headless_men">Headless Men</a> in the right corner of the picture facing a group of soldiers in the left. Each soldier is on a horse. The Headless Men lacks a head, and have their facial features on their chest</figcaption>

Code coverage doesn’t show all the picture. This post shows how the myth of 100% code coverage can irreversibly damage the design of your code.

Let's begin.

Tests for [Data Transfer Objects](https://en.wikipedia.org/wiki/Data_transfer_object) increase the code coverage. However, they don't exercise any meaningful business behavior of the application. They're only technical details. For example, changes to the name of an accessor method are more likely to require changes to multiple tests and incur unnecessary maintenance while not providing any business value.

<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjsfiddle.net%2Ffagnerbrack%2Fv89bwp2o%2Fembedded%2F&amp;url=https%3A%2F%2Fjsfiddle.net%2Ffagnerbrack%2Fv89bwp2o%2F&amp;image=https%3A%2F%2Fwww.gravatar.com%2Favatar%2F0bc8a8a53f215ebcbdc39479f182c803%3Fs%3D80&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=jsfiddle" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a4304525d08116c04fdbe0392d436a0/href">https://medium.com/media/9a4304525d08116c04fdbe0392d436a0/href</a></iframe>

In the example above, changes to the userId() accessor method of PurchaseBooksCommand breaks the test for the PurchaseBooksCommand and the test for BookStoreCommandHander.

<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjsfiddle.net%2Ffagnerbrack%2Fkjo5smxp%2Fembedded%2F&amp;url=https%3A%2F%2Fjsfiddle.net%2Ffagnerbrack%2Fkjo5smxp%2F&amp;image=https%3A%2F%2Fwww.gravatar.com%2Favatar%2F0bc8a8a53f215ebcbdc39479f182c803%3Fs%3D80&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=jsfiddle" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc2a816d31141c177d80bde7552abccd/href">https://medium.com/media/dc2a816d31141c177d80bde7552abccd/href</a></iframe>

Likewise, tests for the internal functions of a React component increase the code coverage. However, they also add coupling between the test code and the internal code of the component. Changes to the internal structure of the component [**without changes to the behavior**](https://fagnerbrack.com/how-to-refactor-a-public-interface-317ed18d38a3) are more likely also to require changes to the tests, which is unnecessary work.

<iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3133ddaf4a5e88d59c01081e032ec001/href">https://medium.com/media/3133ddaf4a5e88d59c01081e032ec001/href</a></iframe>

> The number of lines of code covered by tests is not a useful measure of code coverage.

Tests for a class that uses a [Data Transfer Object](https://en.wikipedia.org/wiki/Data_transfer_object) as an argument for an essential feature of your application, say to purchase books, may not increase the code coverage for the DTO. However, they ensure the tests **exercise** only the code that belongs to the [Business Domain](https://en.wikipedia.org/wiki/Business_domain).

"Exercise" means the test executes the code and asserts a behavior. It's an essential term in the context of this post. If the test only executes the code without an assertion, it increases code coverage but doesn't give any assurance that you'll see a test failure when the behavior of the code changes.

<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjsfiddle.net%2Ffagnerbrack%2Faupesbzg%2Fembedded%2F&amp;url=https%3A%2F%2Fjsfiddle.net%2Ffagnerbrack%2Faupesbzg%2F&amp;image=https%3A%2F%2Fwww.gravatar.com%2Favatar%2F0bc8a8a53f215ebcbdc39479f182c803%3Fs%3D80&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=jsfiddle" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d1949853f606ec00988aeb20964f1f4/href">https://medium.com/media/7d1949853f606ec00988aeb20964f1f4/href</a></iframe>

Likewise, tests that use the rendered state of a React component may not increase the code coverage for internal function calls. However, asserting against the rendered state ensures that when you [refactor](https://fagnerbrack.com/how-to-refactor-a-public-interface-317ed18d38a3) the internal code of the component, the tests are less likely to break.

<iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35fa5737314fce5deb815d0e31ac7345/href">https://medium.com/media/35fa5737314fce5deb815d0e31ac7345/href</a></iframe>

> The number of meaningful behavior covered by the tests is a better measure of code coverage.

According to [Goodhart’s law](https://en.wikipedia.org/wiki/Goodhart%27s_law), when an objective measurement becomes a target in a system, people tend to game that measurement to pretend they are achieving better outcomes:

> When a measure becomes a target, it ceases to be a good measure

> — [Goodhart’s Law](https://en.wikipedia.org/wiki/Goodhart%27s_law)

If you use lines of code as an objective measurement for test coverage, people tend to game that and try to write the most optimal code to satisfy that metric. The code ends up worse than if the system didn't have any test at all.

So how do you quickly verify if all the tests are covering the right things or not? I wish there was a [black and white](https://en.wikipedia.org/wiki/Splitting_(psychology)) response. I wish there was a magical tool you could throw at the code, and it would highlight **all** the test problems.

Unfortunately, there isn't.

The closest thing you can have to validate code coverage for the behavior of the application is [Mutation Testing](https://rachelcarmena.github.io/2017/09/01/do-we-have-a-good-safety-net-to-change-this-legacy-code.html). However, that can't highlight 100% of the testing problems.

The test coverage is heavily dependant on the design of the application, and the design of the application is heavily dependant on the type of [problem you're trying to solve](https://dev.to/treyhuffine/the-problem-you-solve-is-more-important-than-the-code-you-write-2b0g-temp-slug-2836748).

You design for humans to understand the code and you test for humans not to screw up. Machines don't have these problems, they always execute the code correctly, and they never screw up. Therefore, no [Static Analysis](https://en.wikipedia.org/wiki/Static_program_analysis) tool can understand your design; only a human brain with software design skills can look at the code, understand the design, and tell if the tests are covering the right things or not.

If you only write the tests to optimize coverage for machines, you'll end up with useless unreadable code that only machines can understand.

> There’s a big difference between 100% test coverage for business behavior and 100% test coverage for lines of code.

A good example of meaningful code coverage is when you design your models to [isolate the side-effects of HTTP requests](https://fagnerbrack.com/how-to-mock-the-network-with-nock-and-javascript-to-test-parts-of-a-system-you-need-a-good-design-9432c9c05a30). In that case, you need to design the code in a way that there's less coupling between the test code and the application code:

<iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/db04233c460077fda200b822c058f86f/href">https://medium.com/media/db04233c460077fda200b822c058f86f/href</a></iframe>

Another example of meaningful code coverage is when [you design your models to reduce the number of nested imports](https://fagnerbrack.com/the-test-should-verify-if-the-code-solves-the-problem-not-if-it-runs-afea37a3a6e). In that case, you need to design the code in a way that there's less direct coupling between the components of the application. It's better to pass the dependencies as function arguments or class constructors so that you can test the model without the need to mock imports.

If the tests simulate how the application uses the code under test and you mock imports in the test code, you should do the same in the application code. If that sounds ridiculous because nobody mocks imports in the application code, that’s because it is! Mocking imports is a [Bad Code Smell](https://fagnerbrack.com/code-smell-92ebb99a62d0) that the API is not pluggable enough for the application to **consume** it and it's a hint you should redesign:

<iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6eb045fb25f7deb1e4416fc5271fae6f/href">https://medium.com/media/6eb045fb25f7deb1e4416fc5271fae6f/href</a></iframe>

In the context of tests, a **client** is any piece of code that consumes your model. The tests are the “first client” of your model that simulates how the application uses it. The application is the “second client.”

The code is a piece of text that only becomes useful when a machine executes it. Without a **client** machine that can prove its usefulness, the code is just a bunch of text without a purpose. For all practical means, it doesn't exist. The machine which executes the code should run in your computer, not in production. That allows quick and early feedback.

> The problem is not 100% code coverage. Crap 100% is.

Proper 100% coverage means achieving 100% of the business use cases. It's to cover the external API of your models, not technical details like the methods of [Data Transfer Objects](https://en.wikipedia.org/wiki/Data_transfer_object) or the internals of React components.

Although 100% test coverage of business requirements is the metric that matters, there's no way to retrieve that information through static analysis tools. Code coverage is for humans to have control of the code, not machines.

Excellent design skills and the mindset of solving problems instead of technical details are the first steps toward meaningful test coverage.

After all, only with a good design, you can achieve 100%.

![](https://cdn-images-1.medium.com/max/460/1*QaWOCawSnRIKvd4pgsi50g.gif)

Thanks for reading. If you have some feedback, reach out to me on [Twitter](https://twitter.com/FagnerBrack), [Facebook](https://www.facebook.com/fagner.brack) or [Github](http://github.com/FagnerMartinsBrack).

Thanks to [Rachel M. Carmena](https://twitter.com/bberrycarmen) and [Jeff Grigg](https://twitter.com/JeffGrigg1) for their insightful inputs to this post.

* * *