JUnit 5: link tests with task tracker issues
Tags: java, documentation, testing, githubactions
Published: 2023-04-21 18:42:15

In this guide, I'm telling you:

1. How you can link [JUnit 5](https://junit.org/junit5/) tests with issues in your task tracker systems?
1. How to generate documentation based on it automatically?
1. How to host the result documentation on [GitHub Pages](https://pages.github.com/)?

You can find the entire repository with code examples on GitHub [by this link](https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example). The generated documentation also [available here](https://simonharmonicminor.github.io/junit-pioneer-issue-doc-generation-example/).

![Article meme cover](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/83myyth9ncf99t4fem4o.jpg)

## Issue Annotation

There is a cool library called [JUnit Pioneer](https://junit-pioneer.org/). It's an extension pack that includes some features that vanilla JUnit lacks. These are [cartesian product tests](https://junit-pioneer.org/docs/cartesian-product/), [JSON argument parameterized source](https://junit-pioneer.org/docs/json-argument-source/), [retrying tests](https://junit-pioneer.org/docs/retrying-test/) and many others. But I'm particularly interested in [Issue annotation](https://junit-pioneer.org/docs/issue/). Look at the code example below:

```java
class TestExample {
    @Test
    @Issue("HHH-16417")
    void testSum() {
        ...
    }

    @Test
    @Issue("HHH-10000")
    void testSub() {
        ...
    }
}
```

> I put actual task IDs from [Hibernate task tracker](https://hibernate.atlassian.net/jira/projects) to make result documentation more concise.

As you can see, we can add the `@Issue` annotation with task ID that is associated with the test. So, every time you notice a test failure, you know what you have broken.

## Setting up service loaders

JUnit Pioneer provides an API that allows to get information about tests that marked with `@Issue` annotation. Meaning that we can combine the information in the HTML report and share it with other team members. For example, QA engineers might find it beneficial. Because now they are aware what tests do the project contains, and what bugs do they check.

There is a special interface `IssueProcessor`. Its implementation acts like a callback. Look at the code snippet below:

```java
public class SimpleIssueProcessor implements IssueProcessor {
    @Override
    public void processTestResults(List<IssueTestSuite> issueTestSuites) {
        ...
    }
}
```

However, we also to need to set up `SimpleIssueProcessor` as [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html). Otherwise, JUnit runner won’t register it. Create a new file with a name of `org.junitpioneer.jupiter.IssueProcessor` in `src/test/resources/META-INF/services` directory. It has to contain one row with the fully qualified name of the implementation (in our case, `SimpleIssueProcessor`). Look at the code snippet below:

```
org.example.SimpleIssueProcessor
```

Besides, there is another service loader to register. It’s the one provided by JUnit Pioneer library that does the complex logic of parsing information and delegating control to `IssueProcessor` implementation. Create a new file with a name of `org.junit.platform.launcher.TestExecutionListener` in the same directory. Look at the required file content below:

```
org.junitpioneer.jupiter.issue.IssueExtensionExecutionListener
```

Now we’re ready. You can put `println` statement in your `IssueProcessor` implementation to check that the framework invokes it after tests’ execution.

## Creating meta-information JSON file

The documentation generation process consists of two steps:

1. Generate documentation in JSON format (because it's easy to parse).
1. Put the information into HTML template.

Look at the `SimpleIssueProcessor` code below:

```java
public class SimpleIssueProcessor implements IssueProcessor {
    @Override
    @SneakyThrows
    public void processTestResults(List<IssueTestSuite> issueTestSuites) {
        writeFileToBuildFolder(
            "test-issues-info.json",
            new ObjectMapper().writeValueAsString(
                issueTestSuites.stream()
                    .map(issueTestSuite -> Map.of(
                        "issueId", issueTestSuite.issueId(),
                        "tests", issueTestSuite.tests()
                                     .stream()
                                     .map(test -> parseTestId(test.testId()))
                                     .toList()
                    ))
                    .toList()
            )
        );
    }
    ...
}
```

> The `writeToBuildFolder` method creates a file by path `build/classes/java/test/test-issues-info.json`. I use Gradle, but if you prefer Maven, your path will differ a bit. You can check out the source code of the function by [this link](https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/org/example/SimpleIssueProcessor.java#L68).

The result JSON is an array. Look at the generated example below:

```json
[
  {
    "tests": [
      {
        "testId": "TestExample.testSum",
        "urlPath": "org/example/TestExample.java#L12"
      }
    ],
    "issueId": "HHH-16417"
  },
  {
    "tests": [
      {
        "testId": "TestExample.testSub",
        "urlPath": "org/example/TestExample.java#L18"
      }
    ],
    "issueId": "HHH-10000"
  }
]
```

There is an issue ID and set of tests that reference to it (theoretically, there might be several tests pointing the same issue).

Now we need to parse the required information from the supplied `List<IssueTestSuite>`. Look at the `parseTestId` function below.

```java
@SneakyThrows
private static Map<String, Object> parseTestId(String testId) {
    final var split = testId.split("/");
    final var className = split[1].substring(7, split[1].length() - 1);
    final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
    final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);

    final var classPool = ClassPool.getDefault();
    classPool.appendClassPath(new ClassClassPath(clazz));
    final var methodLineNumber = classPool.get(className)
                                     .getDeclaredMethod(method)
                                     .getMethodInfo()
                                     .getLineNumber(0);
    return Map.of(
        "testId", lastArrayElement(className.split("\\.")) + "." + method,
        "urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
    );
}
```

Let's deconstruct this code snippet step by step.

The library puts `testId` as the string pattern below:

```
// [engine:junit-jupiter]/[class:org.example.TestExample]/[method:testSum()]
```

Firstly, we get fully qualified class name and method name. Look at the code below:

```java
final var split = testId.split("/");
// [class:org.example.TestExample] => org.example.TestExample
final var className = split[1].substring(7, split[1].length() - 1);
// [method:testSum()] => testSum
final var method = split[2].substring(8, split[2].length() - 1).replaceAll("\\(.*\\)", "");
```

Afterwards, we determine the line number of the test method. It’s useful to set links that point to the particular line of code. Look at the snippet below:

```java
// Load test class
final Class<?> clazz = Thread.currentThread().getContextClassLoader().loadClass(className);

final var classPool = ClassPool.getDefault();
classPool.appendClassPath(new ClassClassPath(clazz));

final var methodLineNumber = classPool.get(className)
                                 .getDeclaredMethod(method)
                                 .getMethodInfo()
                                 .getLineNumber(0);
```

> `ClassPool` comes from [Javaassist](https://www.javassist.org/) library. It gives convenient API to retrieve the line number of Java method.

Here we perform these steps:

1. Get the `Class` instance of the test suite.
1. Initialize `ClassPool`.
1. Append a test class to the pool
1. Get the line number of the test method.

And finally, we put together the information chunks into `java.util.Map` that we eventually convert to JSON. Look at the last piece of code below:

```java
return Map.of(
    // TestExample.testSum
    "testId", lastArrayElement(className.split("\\.")) + "." + method,
    // org/example/TestExample.java#L11
    "urlPath", className.replace(".", "/") + ".java#L" + methodLineNumber
);
```

The `testId` property is just a combination of a simple class name and test method name. Whilst `urlPath` is part of the link on GitHub pointing to the specific line where we declared the test.

## Generating documentation

Finally, it’s time to compose the generated JSON into a nicely laid out HTML page. Look at the entire snippet below. Then I’m explaining each part to you.

```js
const fs = require('fs');

function renderIssues(issuesInfo) {
  issuesInfo.sort((issueLeft, issueRight) => {
    const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
    return parseIssueId(issueRight) - parseIssueId(issueLeft);
  })
  return `
            <table>
                <tr>
                    <th>Issue</th>
                    <th>Test</th>
                </tr>
                ${issuesInfo.flatMap(issue => issue.tests.map(test => `
                    <tr>
                        <td>
                            <a target="_blank" href="https://hibernate.atlassian.net/browse/${issue.issueId}">${issue.issueId}</a>
                        </td>
                        <td>
                            <a target="_blank" href="https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/${test.urlPath}">${test.testId}</a>
                        </td>
                    </tr>
                `)).join('')}
            </table>
        `
}

console.log(`
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <title>List of tests validation particular issues</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
    </head>
    <body>
        <h1>List of tests validation particular issues</h1>
        <h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
        ${renderIssues(JSON.parse(fs.readFileSync('./build/classes/java/test/test-issues-info.json', 'utf8')))}
    </body>
    </html>
`)
```

> I'm using Javascript and NodeJS runtime environment.

The `renderIssues` function does the entire job. Let's deconstruct it step by step.

```js
function renderIssues(issuesInfo) {
  issuesInfo.sort((issueLeft, issueRight) => {
    const parseIssueId = issue => Number.parseInt(issue.issueId.split("-")[1])
    return parseIssueId(issueRight) - parseIssueId(issueLeft);
  })
  ...
}
```

The `issuesInfo` is an array that we generated previously with `IssueProcessor`. Therefore, each element has `issueId` and tests belonging to it.

As long as each issue id has a format of `MMM-123` we can sort them by number. In that case, we get issues sorted in descending order.

Look at the remaining portion of the function below:

```js
const issueBaseUrl = "https://hibernate.atlassian.net/browse/";
const repoBaseUrl = "https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example/blob/master/src/test/java/"
  return `
            <table>
                <tr>
                    <th>Issue</th>
                    <th>Test</th>
                </tr>
                ${issuesInfo.flatMap(issue => issue.tests.map(test => `
                    <tr>
                        <td>
                            <a target="_blank" href="${issueBaseUrl}${issue.issueId}">${issue.issueId}</a>
                        </td>
                        <td>
                            <a target="_blank" href="${repoBaseUrl}${test.urlPath}">${test.testId}</a>
                        </td>
                    </tr>
                `)).join('')}
            </table>
        `
```

Each present combination of issue and test transforms into a table row. Also, those snippets aren’t just plain text but links. You can open issue description and test declaration by clicking on it. Cool, isn’t it?

Then comes the output. Look at the final part of the script below:

```js
console.log(`
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <title>List of tests validation particular issues</title>
        <meta charset="UTF-8">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/yegor256/tacit@gh-pages/tacit-css-1.5.5.min.css"/>
    </head>
    <body>
        <h1>List of tests validation particular issues</h1>
        <h3>Click on issue ID to open it in separate tab. Click on test to open its declaration in separate tab.</h3>
        ${renderIssues(JSON.parse(
    fs.readFileSync('./build/classes/java/test/test-issues-info.json',
        'utf8')))}
    </body>
    </html>
`)
```

> I write the output to console because later I redirect it to file.

> The style sheet is called [Tacit CSS](https://github.com/yegor256/tacit). This is a set of CSS rules applied automatically. If you need to format an HTML page but don’t want to deal with complex layout, that’s a perfect solution.

The idea is to put the generated HTML table into a predefined template.

## Setting up GitHub Pages

The documentation is no use until you can examine it. So, let's host it on GitHub Pages. Look at the pipeline below:

```yaml
name: Java CI with Gradle

on:
  push:
    branches: [ "master" ]

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-and-deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 17
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'temurin'
      - name: Build with Gradle
        run: ./gradlew build
      - name: Set up NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 16
      - name: Run docs generator
        run: ./docsGeneratorScript.sh
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          path: public/
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1
```

The steps are:

1. Set up JDK 17
1. Build the project
1. Set up NodeJS
1. Generate documentation with the previously shown JS program
1. Deploy the result to GitHub Pages

The `docsGeneratorScript.sh` file is a trivial bash script. Look at its definition below:

```sh
mkdir -p public
touch ./public/index.html
node ./generateDocs.js > ./public/index.html
```

And that’s it! Now the documentation [is available](https://simonharmonicminor.github.io/junit-pioneer-issue-doc-generation-example/) and being updated automatically each time somebody merges a pull request.

## Conclusion

That’s all I wanted to tell you about linking tests with issues and generating documentation for it. If you have questions or suggestions, leave your comments down below. Thanks for reading!

## Resources

1. [JUnit 5](https://junit.org/junit5/)
1. [GitHub Pages](https://pages.github.com/)
1. [The repository with source code](https://github.com/SimonHarmonicMinor/junit-pioneer-issue-doc-generation-example)
1. [The result generated documentation](https://simonharmonicminor.github.io/junit-pioneer-issue-doc-generation-example/)
1. [JUnit Pioneer](https://junit-pioneer.org/)
1. [Issue annotation from JUnit Pioneer](https://junit-pioneer.org/docs/issue/)
1. [Hibernate task tracker](https://hibernate.atlassian.net/jira/projects)
1. [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html)
1. [Javaassist library](https://www.javassist.org/)
1. [Tacit CSS](https://github.com/yegor256/tacit)