Approval Tests For PDF Document Generation
Tags: unittests, tdd, testing, softwaredesign
Published: 2021-10-07 11:32:50

A while back I was confronted with a part of a legacy system that generates PDF documents. This legacy system used a well known library for generating the requested PDF files. The good news was that there were a decent amount of tests available. The not so good news was that these tests made heavy use of test doubles for swapping out most of the types provided by the third-party API. I strongly believe that using test doubles in such cases is not a good choice. In the past I already wrote about why to [avoid using test doubles for types that you don’t own](https://principal-it.eu/2020/05/test-double-heuristics/).

Tests like these might become a large impediment whenever we upgrade to a newer version of the third-party library. Major versions quite often introduce breaking changes to existing API’s. Whenever tests are strongly coupled to such an API, they basically need to be rewritten during the upgrade.

In order to reduce the coupling of these tests, I decided to replace them with [Approval Tests](https://approvaltests.com) instead. This technique is also known as _“Golden Master”_ or _“Characterization Test”_. The idea behind an _Approval Test_ is that after the first test run, some output needs to be visually verified and approved. During subsequent test runs, the approved output will be compared to current output. When there’s a difference in the output, the test will fail.

This is especially useful whenever we have to deal with code of a legacy system. You can check out this video from Emily Bache where she demonstrates the [Gilded Rose refactoring kata using Approval Tests](https://www.youtube.com/watch?v=zyM2Ep28ED8). Highly recommended!

Usually the output of _Approval Tests_ is captured in plain text files, which has nothing to do with PDF files. So I decided to extend the [Java Version of an Approval Test library](https://github.com/approvals/ApprovalTests.Java)to support PDF documents as well. Let’s have a look at some example code to demonstrate this extension.

Suppose that we have a small application that generates a PDF document. A generated document contains the refrain of a well-known song lyric. The following code shows a possible implementation.

```java
public class SingAlongPdfGenerator {

    public ByteArrayOutputStream generate(SingAlongData pdfData) {

        var outputStream = new ByteArrayOutputStream();
        PdfDocument pdf = new PdfDocument(new PdfWriter(outputStream));

        Document document = new Document(pdf);
        document.add(new Paragraph("Let's sing-a-long:"));

        List list = new List()
            .setSymbolIndent(12)
            .setListSymbol("\u2022");

        for(var refrainLine : pdfData.getRefrainLines()) {
           list.add(new ListItem(refrainLine));
        }

        document.add(list);
        document.close();

        return outputStream;
    }
}

public class SingAlongData {

    private final List<String> refrainLines;

    public SingAlongData(List<String> refrainLines) {

        this.refrainLines = refrainLines;
    }

    public List<String> getRefrainLines() {
        return refrainLines;
    }
}
```

The `SingAlongPdfGenerator` class provides a method named `generate` that accepts a `SingAlongData` instance as its only parameter. The `SingAlongData` class is merely a DTO that provides a list of refrain lines. The `generate`method creates a new document containing a paragraph of text and a list of the refrain lines.

Let’s have a look at the code of the corresponding _Approval Test_.

```java
public class SingAlongPdfGeneratorTests {
    
    @Test
    public void generateRickRollPdf() {

        var refrainLines = Arrays.asList(
            "Never gonna give you up",
            "Never gonna let you down",
            "Never gonna run around and desert you",
            "Never gonna make you cry",
            "Never gonna say goodbye",
            "Never gonna tell a lie and hurt you"
        );
        var data = new SingAlongData(refrainLines);

        var pdfGenerator = new SingAlongPdfGenerator();
        var result = pdfGenerator.generate(data);

        PdfApprovals.verify(result);
    } 
}
```

First we create an instance of the `SingAlongData` DTO, specifying some test data. Next we create an instance of the Subject Under Test, which in this case is the `SingAlongPdfGenerator` class and call the `generate` method. This returns a `ByteArrayOutputStream` containing the data of the PDF document. Then we verify the result by calling the `PdfApprovals.verify` method. Notice that the anatomy of an _Approval Test_ is identical to any other type of test as it also adheres to the _Arrange, Act, Assert_ pattern.

A new _Approval Test_ always fails the very first time that it gets executed. After the initial test run, a PDF file is generated that needs to be visually verified and approved. So when we first run the `generateRickRollPdf` test, a file with the name `SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf` appears which has the following content:

![A generated PDF file that needs to be approved](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6vmjsoisen16vyvvkyyb.png)

After we verified the PDF document, we approve it using the following command:

```
mv ~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf 
~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf
```

At this point we have a PDF file named `SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf`. When we now execute our test again, it passes as the newly generated PDF document matches the approved PDF document.

Note that we also have to make sure to commit the approved PDF file alongside our test code. Otherwise we have to approve the output of the test again when executed on another machine.

Let’s say that we want to make a change to the implementation of our `SingAlongPdfGenerator` class. For example, we’re going to make a small change to the text inside the paragraph.

```java
document.add(new Paragraph("Let's sing-a-long shall we?"));
```

When we execute our test again, it fails due to the change that we’ve made.

```
Failed Approval
  Approved:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf
  Received:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf
```

The test created a “received” PDF file again alongside the existing PDF file that we’ve approved earlier. We now have to visually compare the received and the approved file side-by-side. Needless to mention that this is going to be quite cumbersome. For this reason I’ve added the capability that in case of failing test, a third PDF file is generated that contains the annotated differences between the two PDF files.

![A PDF file that indicated the differences](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/obs5100vygspq1mglu5h.png)

The purple markers indicate where the changes are located in the document. The green colon indicates what is expected (approved). The red text that we’ve added to the paragraph shows the actual text found in the document (received). We now have to decide whether we want to approve the changes that we’ve made or not. Let’s say that we are happy with the change that we’ve made. To do that we simply run the approval command again as shown earlier. And that’s it.

With just a handful of these _Approval Tests_ I was able to eliminate all the tightly coupled tests. Generating PDF files is more often than not an infrastructure concern. [Sociable tests](https://principal-it.eu/2019/10/taxonomy-of-tests/) tests are much more appropriate in this case compared to [solitary tests](https://principal-it.eu/2019/10/taxonomy-of-tests/). Using _Approval Tests_ this way turned out to be a very valuable approach.