Unit Testing Backward Compatibility of Message Format
Tags: java, tutorial, testing, programming
Published: 2023-02-18 08:50:27

Do you apply [Apache Kafka](https://kafka.apache.org/) or [RabbitMQ](https://www.rabbitmq.com/) in your software project? If so, then you definitely have some message schemas. Have you ever encountered in backward compatibility issue? An accidental message format changing and your entire system is no longer functioning? I bet you have such an unpleasant experience at least once in your career.

Thankfully, there is a solution. In this article, I'm telling you:

1. Why backward compatibility is crucial in message driven systems?
2. How can you validate the backward compatibility automatically with simple unit tests in Java?
3. A few words about forward compatibility.

You can find code examples and the entire project setup of unit testing backward compatibility in [this repository](https://github.com/SimonHarmonicMinor/unit-testing-backward-compatibility-of-messages-example).

![Meme cover](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u7cjw8py4c629xmexxrt.jpg)

## Why backward compatibility matters?

Suppose there are two services. The `Order-Service` transfers a message about shipment details to the `Shipment-Service`. Look at the schema below.

![Message transferring system](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p21il5tmbrxqiupooob8.png)

The `Shipment-Service` (the consumer) defines the message schema (i.e. the contract) with two fields: `orderId` and `address`. And the `Order-Service` (the producer) sends the message according to the specified schema.

Everything works well. But suddenly the new requirement came into play. Now we have to set the complex address instead of just putting the city name (e.g. country, city, postcode, etc.). No problem, we just need to update the contract accordingly. Look at the fixed schema below.

```ts
{
  "orderId": Long
  "address": {
    "country": String,
    "city": String,
    "postcode": Integer
  }
}
```

Now there are two likely outcomes:

1. The producer had updated the contract before the consumer did.
1. Vice versa.

Assuming that the producer is the first one to make changes. What happens in such situation? Look at the diagram below to understand the consequences.

![The producer had updated the contract before the consumer did](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9055ckg6ga01112buf7q.png)

Here is what happens:

1. The `Order-Service` updates the contract.
2. The `Order-Service` sends the message with the new format.
3. The `Shipment-Service` tries to deserialize the received message with the old contract.
4. Deserialization process fails.

But what if the `Shipment-Service` updates the contract firstly? Let's discover this scenario on the schema below.

![The consumer had updated the contract before the producer did](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h6zkvpk3tjyonr55ubbz.png)

The problem remains the same. As a matter of fact, the `Shipment-Service` cannot deserialize received message.

So, we can conclude that altering the message schema **has to be** backward compatible.

> Some of you may ask whether is it possible to update the `Shipment-Service` and the `Order-Service` contracts simultaneously or not? That would eliminate the problem with the backward compatibility, right? Well, it’s unbearable to perform such update in real life. Besides, even if you did, there’s a possibility that the broker (e.g. Apache Kafka) still contains the messages with the old format that you need to process somehow. So, backward compatibility is always a concern.

## The idea of unit testing backward compatibility

Let's get to the code. I'm using [Java Jackson](https://www.baeldung.com/jackson) for de/serialization. But the idea remains the same for any other library.

Look at the first version of `OrderCreated` message schema below.

```java
@Data
@Builder
@Jacksonized
public class OrderCreated {
    private final Long orderId;
    private final String address;

    public OrderCreated(Long orderId, String address) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
    }
}
```

> The [Jacksonized annotation](https://projectlombok.org/features/experimental/Jacksonized) tells Jackson to use the builder generated by the [Builder annotation](https://projectlombok.org/features/Builder) as the deserialization entry point. Quite useful because you don't have to repeat [@JsonCreator and @JsonProperty](https://www.baeldung.com/jackson-annotations) usage while dealing with immutable classes.

How do we start with the backward compatibility check? Let’s add the message example to the `resources/backward-compatibility` directory. Look at the Json message example and the folder structure below.

```json
{
  "orderId": 123,
  "address": "NYC"
}
```

![Backward compatibility files folder structure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jcf40bwsnzshv8h06q58.png)

Each schema altering should come up with adding a new file to the `backward-compatibility` directory. And the content should describe the **increment** made to the schema.

> Each new file name should be lexically greater than the previous one. In that case, the tests run will be deterministic. The easiest way is just setting the incrementing natural number. But what if you need something more complicated? Then I recommend you to read my article about [Flyway migrations naming strategy in a big project](https://dev.to/kirekov/flyway-migrations-naming-strategy-in-a-big-project-51fp). I describe there the similar principle.

So, there are two rules we have to follow:

1. Each new schema update should come up with creating a new file in `resources/backward-compatibility` directory.
2. You should never delete the previously added files to ensure backward compatibility consistency.

> Anyway, there are cases when you **want** to break the backward compatibility. For example, no one uses this field anymore and you just need to eliminate it. In that case, you can remove some backward compatibility data files. However, don’t treat it as a regular situation. It’s an exception but not a valid common scenario.

## The automation process

Now we need the test that reads the described files and generates the validation cases. Look at the code example below:

```java
class BackwardCompatibilityTest {
    private final ObjectMapper objectMapper = new ObjectMapper();

    @ParameterizedTest
    @SneakyThrows
    void shouldRemainBackwardCompatibility(String filename, String json) {
        final var orderCreated = assertDoesNotThrow(
            () -> objectMapper.readValue(json, OrderCreated.class),
            "Couldn't parse OrderCreated for filename: " + filename
        );
        final var serializedJson = assertDoesNotThrow(
            () -> objectMapper.writeValueAsString(orderCreated),
            "Couldn't serialize OrderCreated to JSON from filename: " + filename
        );
        final var expected = new JSONObject(json);
        final var actual = new JSONObject(serializedJson);
        JSONAssert.assertEquals(
            expected,
            actual,
            false
        );
    }
}
```

I parametrised the backward compatibility test because the number of files will grow (therefore, the true number of test cases as well). Here is what happens step by step:

1. The test receives the json content and the latest combined filename (I describe the algorithm later in the article). We the latter parameter for a more informative assertion message in case of failures.
1. Then we try to parse the supplied json to the `OrderCreated` object. If this step fails, then we definitely have broken the backward compatibility.
1. Afterwards, we serialize the parsed `OrderCreated` object back to json. Usually, this operation doesn’t fail. Anyway, we should always be prepared for a dangerous scenario.
1. And finally, we check that the supplied json equals to the one we got on step 3 (I use [JSONAssert library](https://github.com/skyscreamer/JSONassert) here). The `false` boolean parameter tells to check only overlapping fields. For example, if the `actual` result contains the `addressV2` field and the `expected` object doesn’t, it won’t trigger the failure. That’s a normal situation because backward compatibility data is static while the `OrderCreated` might grow with new parameters.

It's time to provide the input parameters to the `shouldRemainBackwardCompatibility` test. Look at the code example below.

```java
private static Stream<Arguments> provideMessagesExamples() {
    final var resourceFolder = Thread.currentThread().getContextClassLoader().getResources("backward-compatibility").nextElement();
    final var fileInfos = Files.walk(Path.of(resourceFolder.toURI()))
                                  .filter(path -> path.toFile().isFile())
                                  .sorted(comparing(path -> path.getFileName().toString()))
                                  .map(file -> new FileInfo(
                                      file.getFileName().toString(),
                                      Files.readString(file))
                                  )
                                  .toList();
    ...
}

private record FileInfo(String name, String content) {}
```

In the beginning, we read all the files from the `backward-compatibility` directory, sort them by their names, and assign the result tuples list `[(filename, content), …]` to the `fileInfos` variable. Look at the next step below.

```java
final var argumentsList = new ArrayList<Arguments>();
for (int i = 0; i < fileInfos.size(); i++) {
    JSONObject initialJson = null;
    for (int j = 0; j <= i; j++) {
        if (j == 0) {
            initialJson = new JSONObject(fileInfos.get(i).content);
        }
        final var curr = fileInfos.get(j);
        deepMerge(new JSONObject(curr.content), initialJson);
        if (j == i) {
            argumentsList.add(Arguments.arguments(curr.name, initialJson.toString()));
        }
    }
}
return argumentsList.stream();
```

And here comes the algorithm of creating the backward compatibility data list itself. Suppose we have 3 files: `1.json`, `2.json` and `3.json`. Therefore, you should validate these combinations to ensure that newly added changes haven't broken the backward compatibility:

1. The content of the `1.json`
2. The content of the `1.json` + `2.json` (the latter one rewrites existing fields).
3. The content of the `1.json` + `2.json` + `3.json` (the latter one rewrites existing fields).

The loop above does this by calling the `deepMerge` method. The first parameter is the file with the current index in the loop. And the `initialJson` values are the target to merge all the latter changes.

Look at the `deepMerge` method declaration below.

```java
private static void deepMerge(JSONObject source, JSONObject target) {
    final var names = requireNonNullElse(source.names(), new JSONArray());
    for (int i = 0; i < names.length(); i++) {
        final String key = (String) names.get(i);
        Object value = source.get(key);
        if (!target.has(key)) {
            // new value for "key":
            target.put(key, value);
        } else {
            // existing value for "key" - recursively deep merge:
            if (value instanceof JSONObject valueJson) {
                deepMerge(valueJson, target.getJSONObject(key));
            } else {
                target.put(key, value);
            }
        }
    }
}
```

> I didn't write this code example by myself but took it from [this StackOverflow question](https://stackoverflow.com/questions/2403132/merge-concat-multiple-jsonobjects-in-java).

There are two arguments: `source` and `target`. The `source` is the one to read new values. And the `target` is the JSONObject to put them or replace existing ones.

Finally, to supply the computed data set just add the `@MethodSource` annotation to the `shouldRemainBackwardCompatibility` test method. Look at the code example below.

```java
@ParameterizedTest
@MethodSource("provideMessagesExamples")
void shouldRemainBackwardCompatibility(String filename, String json) { ... }
```

Here is the test run result.

![Test results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kzr33s7ybvv1ve4x8dbj.png)

As you can see, an example with a single file works as expected.

## Evolving data schema

Let’s get to the initial point of the article. There is an `address` field of `String` type and we want to store `city`, `country`, and `postcode` separately. As we already discussed, we cannot just replace the existing field with the new value type. Therefore, we should add the new field and mark the previous one as deprecating. Look at the updated version of `OrderCreated` class below:

```java
@Data
@Builder
@Jacksonized
public class OrderCreated {
    private final Long orderId;
    @Deprecated(forRemoval = true)
    private final String address;
    private final Address newAddress;

    public OrderCreated(Long orderId, String address, Address newAddress) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
        this.newAddress = notNull(newAddress, "Complex address cannot be null");
    }

    @Builder
    @Data
    @Jacksonized
    public static class Address {
        private final String city;
        private final String country;
        private final Integer postcode;

        public Address(String city, String country, Integer postcode) {
            this.city = notNull(city, "City cannot be null");
            this.country = notNull(country, "Country cannot be null");
            this.postcode = notNull(postcode, "Postcode cannot be null");
        }
    }
}
```

I deprecated the `address` field and added the `newAddress` field that is a complex object containing `country`, `city`, and `postcode`. All right, now we need to add the new `2.json` file with the example of filled `newAddress`. Look at the code snippet below.

```json
{
  "newAddress": {
    "city": "NYC",
    "country": "USA",
    "postcode": 123456
  }
}
```

Let's run the test to check the backward compatibility.

![Test results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yoxbouidng62zg65c4wc.png)

We did break the backward compatibility! Why is that? Because we marked the `newAddress` field as mandatory and checked it for non-nullability. No surprise that the `1.json` content failed the test because it has no mention of the `newAddress` field.

So, here comes an important conclusion. If you add a new field to the existing schema, you **cannot** check it for _non-nullability_. There is no guarantee that the producer will start sending messages with the newly stated field instantly. Meaning that the solution is the default values usage. Look at the fixed `OrderCreated` class declaration below.

```java
@Data
@Builder
@Jacksonized
public class OrderCreated {
    private final Long orderId;
    @Deprecated(forRemoval = true)
    private final String address;
    private final Address newAddress;

    public OrderCreated(Long orderId, String address, Address newAddress) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
        this.newAddress = requireNonNullElse(newAddress, Address.builder().build());
    }

    @Builder
    @Data
    @Jacksonized
    public static class Address {
        @Builder.Default
        private final String city = "";
        @Builder.Default
        private final String country = "";
        @Builder.Default
        private final Integer postcode = null;
    }
}
```

We don’t want to deal with `null` value. That’s why we assign `Address.builder().build()` instance, if the supplied `newAddress` equals to `null` (which basically means it’s not present in the provided json).

> The `@Builder.Default` Lombok annotation does the same thing as the `requireNonNullElse` function but without specifying the constructor manually.

Let's run the backward compatibility test again.

![Backward compatibility test result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/43fzj8urt3l7h7tyz6et.png)

Now we’re ready. The new field presence has not broken the backward compatibility and we can update the contracts safely on both sided in any order.

## A few words about forward compatibility

I’ve seen that developers tend to care about forward compatibility much less than the backward one. Though it’s crucial as well.

> I’ve covered the topic of forward compatible enums in [this article](https://dev.to/kirekov/forward-compatible-enum-values-in-api-with-java-jackson-532p). If you’re dealing with message driven systems, I strongly recommend you to check it out.

What if somebody put additional properties that aren't present in the message schema? Look at the example below:

```json
{
  "orderId": 123,
  "address": "NYC",
  "newAddress": {
    "city": "NYC",
    "country": "USA",
    "postcode": 123456
  },
  "unknown_field": "unknown_value"
}
```

Nothing should break, right? Anyway, there is no field to map the value. However, let's add the backward compatibility test to ensure consistency. Look at the new `3.json` file declaration below.

```json
{
  "unknown_field": "unknown_value"
}
```

Let's run the test to see the results.


![Backward compatibility test result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/drx30dncrzdvo0io9q29.png)

Wow, something went wrong. Look at the output message below.

```
Caused by: UnrecognizedPropertyException: Unrecognized field "unknown_field"
```

By default, Jackson treats unknown property presence as an error. Sometimes that can be really annoying. Especially if there are many producers (and some may not be the part of your project). Thankfully, the fix is a piece of cake. You just need to add [JsonIgnoreProperties](https://fasterxml.github.io/jackson-annotations/javadoc/2.6/com/fasterxml/jackson/annotation/JsonIgnoreProperties.html) annotation with the proper value. Look at the code snippet below.

```java
@Data
@Builder
@Jacksonized
@JsonIgnoreProperties(ignoreUnknown = true)
public class OrderCreated { ... }
```

Let's run the test again.

![Backward compatibility test result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ysqt9flaeqsm8g1g447s.png)

It still fails! Though the message is different.

```
Expected: unknown_field
     but none found
```

The assertion checks that the input and the serialized json values are equal. But in this situation, that’s not the case. The input json contains the `unknown_field`. But the `@JsonIgnoreProperties(ignoreUnknown = true)` annotation presence just erases it. So, we lost the value.

Again, this might be acceptable under particular circumstances. But what if you still want to guarantee that we won’t lose the user provided fields even if there is no business logic on these values? For example, your service might act as a middleware that consumes messages from a Kafka topic and produces them to another one.

Jackson has a solution for this case as well. There is [JsonAnySetter and JsonAnyGetter](https://www.concretepage.com/jackson-api/jackson-jsonanygetter-and-jsonanysetter-example) annotations that helps to store unknown values and serialize them lately. Look at the final version of `OrderCreated` class declaration below.

```java
@Data
@Builder
@Jacksonized
@JsonIgnoreProperties(ignoreUnknown = true)
public class OrderCreated {
    private final Long orderId;
    @Deprecated(forRemoval = true)
    private final String address;
    private final Address newAddress;
    @JsonAnySetter
    @Singular("any")
    private final Map<String, Object> additionalProperties;

    @JsonAnyGetter
    Map<String, Object> getAdditionalProperties() {
        return additionalProperties;
    }

    public OrderCreated(Long orderId, String address, Address newAddress, Map<String, Object> additionalProperties) {
        this.orderId = notNull(orderId, "Order id cannot be null");
        this.address = notNull(address, "Address cannot be null");
        this.newAddress = requireNonNullElse(newAddress, Address.builder().build());
        this.additionalProperties = requireNonNullElse(additionalProperties, emptyMap());
    }

    @Builder
    @Data
    @Jacksonized
    public static class Address {
        @Builder.Default
        private final String city = "";
        @Builder.Default
        private final String country = "";
        @Builder.Default
        private final Integer postcode = null;
    }
}
```

> I marked the `getAdditionalProperties` as package-private so the users of this class cannot access the unknown properties values.

Let's run the backward compatibility test again.

![Backward compatibility test results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ewbqoops9qya37i3zwix.png)

Everything works like a charm now. The morale is that forward compatibility might be as important as backward one. And if you want to maintain it, then make sure that you don’t simply erase unknown values but transmit them further to be serialized in the expected json.

## Conclusion

That’s all I wanted to tell you about backward (and a bit of forward) compatibility. Hope this knowledge will be useful to keep your contracts consistent to avoid unexpected deserialization errors on consumer side.

If you have questions or suggestions, leave your comments down below. Thanks for reading!

## Resources

1. [The repository with the whole setup of unit testing backward compatibility](https://github.com/SimonHarmonicMinor/unit-testing-backward-compatibility-of-messages-example)
1. [Apache Kafka](https://kafka.apache.org/)
1. [RabbitMQ](https://www.rabbitmq.com/)
1. [Java Jackson library](https://www.baeldung.com/jackson)
1. [Jacksonized Lombok annotation](https://projectlombok.org/features/experimental/Jacksonized)
1. [Builder Lombok annotation](https://projectlombok.org/features/Builder)
1. [JsonCreator and JsonProperty Jackson annotations](https://www.baeldung.com/jackson-annotations)
1. [My article about Flyway migrations naming strategy in a big project](https://dev.to/kirekov/flyway-migrations-naming-strategy-in-a-big-project-51fp)
1. [JSONAssert library](https://github.com/skyscreamer/JSONassert)
1. [StackOverflow question with example of merging two JSON objects](https://stackoverflow.com/questions/2403132/merge-concat-multiple-jsonobjects-in-java)
1. [My article about forward compatible enum values in API with Java Jackson](https://dev.to/kirekov/forward-compatible-enum-values-in-api-with-java-jackson-532p)
1. [JsonIgnoreProperties Jackson annotation](https://fasterxml.github.io/jackson-annotations/javadoc/2.6/com/fasterxml/jackson/annotation/JsonIgnoreProperties.html)
1. [JsonAnySetter and JsonAnyGetter Jackson annotations](https://www.concretepage.com/jackson-api/jackson-jsonanygetter-and-jsonanysetter-example)