The Code Speaks for Itself – Generating API docs for Spring Applications
Tags: java, api, rest
Published: 2020-09-25 11:18:09

## **Introduction**

Programming is mostly about communication, and one of the most time-consuming parts of this aspect of development is the communication of how service APIs function. If this is done poorly, then the documents can get out of date, or be so vague that the developers will spend too much time answering questions about how their API works.

This post outlines a process that we in Sonalake have found to automate the creation of REST API documentation. It’s done in such a way that it won’t require too much in the way of manual effort once it’s started, because most of the documentation detail will come from work you’re already doing to test the service. We have provided a working example of this in the [**sonalake-autodoc-example**](https://github.com/sonalake/sonalake-autodoc-example) project.

What drove the creation of this process was the aim to provide a good developer experience (DX) to our own developers, and our clients and partners, by delivering good documentation that:

1.  Describes what, specifically, is in the API
2.  Provides examples of how to use the API
3.  Contains a changelog for how the API has evolved between versions

Tools like [**Swagger**](https://swagger.io) do a great job on automating documentation for point 1, but when it comes to points 2 and 3, these types of documentation are generally written manually (or more likely, not written at all).

By generating documentation from the source, we have found that it allows for significant portions of the API documentation to be automatically generated. By generating documented examples from unit tests, we can ensure that these examples always align with the reality of the application.

It also allows for developers to keep documentation up-to-date without having to leave the development environment, and for documents to be released and published the same way as any other development artifact.

## **How do we do this?**

Some parts of the documentation are written manually by the developers in [**AsciiDoc**](https://asciidoc.org/). These parts of the documentation are not expected to change much between releases, and are limited to things like:

*   Introducing what the API is
*   Describing how to authenticate
*   Outlining a generic set of use case steps, without any actual code samples (the code samples will be auto-generated during the build, using the data passed to unit tests).

The rest of the process will generate the following sections, also in [**AsciiDoc**](https://asciidoc.org/) format.

*   [**Swagger**](https://swagger.io) documentation concerning the paths and entities
*   Code samples for the use case steps, generated from the unit tests
*   Changelog history of differences between published versions of the swagger.json

Finally, the AsciiDoc files are collated and published in a single PDF file using [**Asciidoctor PDF**](https://asciidoctor.org/docs/asciidoctor-pdf/).

At a high level, the main steps are as follows:


<table width="100%">
<tbody>
<tr>
<th>Step</th>
<th>Comment</th>
</tr>
<tr>

<td align="left"> <strong>Define Theme</strong> </td>

<td align="left">The theme in the above project is a simple, clean layout, , suitable for rendering most documents, and contains the standard document tracking elements such as document versions.

This uses the standard <a href="https://github.com/asciidoctor/asciidoctor-pdf/blob/v1.5.0.beta.7/docs/theming-guide.adoc" target="_blank" rel="noopener noreferrer"><strong>AsciiDoctor-PDF</strong></a> theme configurations.

</td>

</tr>

<tr>

<td align="left"><strong>Generate Example Code Snippets</strong></td>

<td align="left">Use <a href="https://github.com/spring-projects/spring-restdocs" target="_blank" rel="noopener noreferrer"><strong>spring-restdocs</strong></a> to document the inputs/outputs for REST queries by writing unit tests that exercise the APIs. We’ll embed these snippets in the final documentation later on.</td>

</tr>

<tr>

<td align="left"><strong>Generate swagger.json</strong></td>

<td align="left">Use a SpringBootTest to spin up the app in-memory and pull down the swagger.json to a local directory.  
You can use the test from the previous step to do this.</td>

</tr>

<tr>

<td align="left"><strong>Generate Changelog</strong></td>

<td align="left">Use Sonalake’s <a href="https://plugins.gradle.org/plugin/com.sonalake.swagger-changelog" target="_blank" rel="noopener noreferrer"><strong>swagger-changelog</strong></a> plugin to parse any previously published API specs, compare it to the current dev version, and produce a changelog in AsciiDoc format.</td>

</tr>

<tr>

<td align="left"><strong>Author Hand-written Content</strong></td>

<td align="left">A document containing:

*   Hand-written content that won’t change too often. For example, an introduction.
*   A code examples document of simple text, referencing the generated snippets.
*   Write a single framing document that will link to both the hand-written, and generated content.

</td>
</tr>
</tbody>
</table>


We have a developed sample project to showcase all of these steps: sonalake-autodoc-example. This is a very simple Spring Boot application with a trivial REST API with two GET methods. The rest of the project is solely dedicated to automating the documentation. Let’s walk through it.

## **Define Theme**  

The main tool for the AsciiDoctor-to-PDF generation is [**AsciiDoctor-PDF**](https://github.com/asciidoctor/asciidoctor-pdf/blob/v1.5.0.beta.7/docs/theming-guide.adoc) and it comes with a full set of theming options. The [**simple-theme.yml**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/src/docs/asciidoc/theme/simple-theme.yml) sample provides a simple, clean professional layout, that you can probably re-use by just changing the logo image.

## **Generate Example Code Snippets**

This part of the pipeline generates snippets in AsciiDoc format from unit tests. The output contains examples of REST calls, with request bodies and responses that will always be accurate for the current version of the code base.

In the sample project this all happens in [**BaseWebTest**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/src/test/java/com/sonalake/autodoc/api/BaseWebTest.java). It takes advantage of [**spring-restdocs**](https://github.com/spring-projects/spring-restdocs) and acts as a base class for all other web-based unit tests.

API calls would be tested in the normal way:
```java
mockMvc.perform(
    get("/api/endpoint-a")
    .contentType(MediaType.APPLICATION_JSON)
    .accept(MediaType.APPLICATION_JSON)
    .characterEncoding(StandardCharsets.UTF_8.name())
).andExpect(status().isOk()).andReturn();
```
A unit test of the form above generates the following snippets to  
`build/generated-snippets/${test-class-name}/${test-method-name}`

*   `curl-request.adoc`
*   `http-request.adoc`
*   `http-response.adoc`
*   `httpie-request.adoc`
*   `request-body.adoc`
*   `response-body.adoc`

For example, a http-request for a POST might look like:

```java
[source,http,options="nowrap"]
----
POST /api/endpoint-a HTTP/1.1
Content-Type: application/json;charset=UTF-8
Accept: application/json
Content-Length: 52
Host: autodoc.sonalake.com
{
  "fieldA" : "sample A",
  "fieldB" : "sample B"
}
----
```

These files can be referenced in your examples documents, with the result that examples will always be up-to-date.

## **Generate swagger.json**

This part of the pipeline generates an up-to-date view of the REST paths and entities in AsciiDoc format. First by generating a swagger.json, and then translating this into AsciiDoc.

The sample project contains a single test, [**GenerateDocumentationTest.java**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/src/test/java/com/sonalake/autodoc/GenerateDocumentationTest.java), that starts up the application as `@SpringBootTest` in the `test` profile, and pulls down the `swagger.json` generated by the [**SwaggerConfig.java**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/src/main/java/com/sonalake/autodoc/config/SwaggerConfig.java). It then runs swagger.json through the [**swagger2markup-gradle-plugin**](https://github.com/Swagger2Markup/swagger2markup-gradle-plugin) to convert it to AsciiDoc format.

This produces the following sets of files:

*   Overview.adoc

Contains some metadata from application.yml such as title text and version information for inclusion on the main page
*   security.adoc

a simple page describing how authentication, for example HTTP headers, should be configured
*   paths.adoc

A list of all the REST calls and responses the application will accept and respond with
*   definitions.adoc

a list of all the entities the application will accept and respond with

## **Make Documentation Easier to Follow**

[**Tags**](https://swagger.io/docs/specification/grouping-operations-with-tags/) are an optional, but useful, tool for collecting related endpoints together, even when they are implemented in different classes. By default, Swagger will name the resources after their controller classes, but tags allow you to give them a different name.

For example:

```java
@Api(tags = {"Section A"})
@Description("Some operations in section A")
public class ControllerA1 {
```


## **Generate Changelog**

The last part of the automated process is related to how to create a changelog. It assumes that previously released versions of the Swagger are published under Nexus. All of this configuration is contained in [**build.gradle**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/build.gradle).

1.  Publish Swagger Spec as a Nexus Artifact using the maven-publish and maven-publish-auth plugins.
2.  Generate changelog from Nexus history using the Sonalake swagger-changelog Gradle plugin.  
    The plugin will retrieve any previously published RELEASE versions of the Swagger spec, and will produce the following:
    *   A file of the form `change-log-0.0.1-0.0.2-SNAPSHOT.adoc` for each version
    *   An index file, `change-log.adoc`, listing all versions

## **Author Hand-written Content**

Writing the following documents will round out the process.

*   introduction.adoc – a simple one or two paragraph description of what the applications is for
*   security.adoc – a quick description of how to authenticate, and what, if any, roles exist in the application. Do note, that if you want to, you can easily write other tests that will print out a list of such roles in AsciiDoc format, and include them in this file.

A special case of a hand-written document is the[ **examples.adoc**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/src/docs/asciidoc/examples.adoc) where a high-level description of the overall flow of REST calls would be written. For example, to on-board a new user, you need to call X, then Y, and the Z. However, this document would not include any actual REST calls or parameters. Rather, it would refer to the results of the unit tests that you have written to test these endpoints.
```java
[[examplesscheme]]
== Examples
What follows are some examples of the API usage
=== Endpoint A
A get call
include::{snippets}/endpoint-a-test/test-get-value/http-request.adoc[]
Returns this
include::{snippets}/endpoint-a-test/test-get-value/http-response.adoc[]
```


Since the overall flow of your application isn’t likely to change – even if the URLs and request/responses change – this document will remain relatively unchanged over time. The only thing you are likely to have to update are your unit tests, but you’d be doing that anyway. Right?

## **Tying it All Together**

All this work is done in [**build.gradle**](https://github.com/sonalake/sonalake-autodoc-example/blob/develop/build.gradle) – it dictates where to write the files in the build directory.

```java
ext {
 asciiDocOutputDir = file("${buildDir}/asciidoc/generated")
 swaggerOutputDir = file("${buildDir}/swagger")
 snippetsOutputDir = file("${buildDir}/generated-snippets")
}
```

The following tells Gradle to pass system properties down to the test tool, so the generate documentation task can know the current document version.

```java
test {
 systemProperties = System.properties
 systemProperty 'sg.api.version', version
 
 useJUnitPlatform()
}
```

Then use [**swagger2markup**](http://swagger2markup.github.io/swagger2markup/1.3.1/) to convert the swagger.json into AsciiDoc format

```java
convertSwagger2markup {
 dependsOn test
 swaggerInput "${swaggerOutputDir}/swagger.json"
 outputDir asciiDocOutputDir
 config = [
   'swagger2markup.pathsGroupedBy'                          : 'TAGS',
   'swagger2markup.extensions.springRestDocs.snippetBaseUri': snippetsOutputDir.getAbsolutePath()
 ]
}
```

Next, the following tells the swagger changelog plugin from where to pull version information, and to where to write the diff files.

```java
swaggerChangeLog {
 groupId = "${rootProject.group}"
 artifactId = "${rootProject.name}-API"
 
 // where to find the nexus repo
 nexusHome = 'http://atlanta.sonalake.corp:8081/nexus'
 
 // where to store the changelog
 targetdir = "${buildDir}/asciidoc/generated/changelog"
 
 // if we’re building a snapshot version, then include it as the
 // end of the changelog
 snapshotVersionFile = "${buildDir}/swagger/swagger.json"
}
```

Finally, this is where the [**AsciiDoctor-PDF**](https://asciidoctor.org/docs/asciidoctor-pdf/) Gradle plugin takes all the AsciiDoc files we have created, and converts them into a pdf.

Note that the baseDirFollowsSourceDir setting, all paths are relative to the main index file. This is done because it allows for references within the AsciiDoc file structure to not have to worry about where they are on the file system.

```java
// create a PDF from the asciidoc
asciidoctorPdf {
 dependsOn convertSwagger2markup
 dependsOn generateChangeLog
 
 baseDirFollowsSourceDir()
 
 sources {
   include 'api-guide.adoc'
 }
 attributes = [
   doctype        : 'book',
   toc            : 'left',
   toclevels      : '3',
   numbered       : '',
   sectlinks      : '',
   sectanchors    : '',
   hardbreaks     : '',
   generated      : '../../../build/asciidoc/generated',
   resources      : '../../../src/main/resources',
   snippets       : '../../../build/generated-snippets',
   changes        : '../../../build/asciidoc/generated/changelog',
   imagesdir      : 'theme',
   'pdf-stylesdir': 'theme',
   'pdf-style'    : 'simple-theme.yml',
   revnumber      : version
 ]
}
```

That’s it. You can take the code from the sample project into any Spring Boot project in about an hour, and produce professional, clean documents. We hope you find it as useful as we do!