JSON Merge Patch in .NET Core
Tags: csharp, dotnet, webdev, javascript
Published: 2020-08-18 02:51:35

The other day, I was looking at how I would implement a REST API in .NET Core that accepts PATCH requests.

## The many ways to patch a resource

I know of two RFCs that describe a format to represent a partial JSON update:
- [JSON Patch - RFC 6902](https://tools.ietf.org/html/rfc6902)
- [JSON Merge Patch - RFC 7386](https://tools.ietf.org/html/rfc7386)

While the similar name is prone to confusion, they describe different structures to represent a partial JSON update.

Given the following sample JSON representation of a person:
```JSON
{
  "name": "Joe",
  "email": "joe@example.com",
  "physicalAttributes": { "weight": 75, "height": 175 },
  "favoriteColors": ["blue", "red"]
}
```

A JSON Merge Patch request body to remove the email address and to add "black" as a favourite colour would look like:
```JSON
{
  "email": null,
  "favoriteColors": ["blue", "red", "black"]
}
```

The same with JSON Patch would be:
```JSON
[
   { "op": "remove", "path": "/email" },
   { "op": "add", "path": "/favoriteColors/-", "value": "black" }
]
```

It is interesting to note that JSON Merge Patch doesn't support partial updates of array items. Arrays must instead be entirely replaced.

On the other hand, JSON Patch is designed to allow the mutation of arrays.

Because the JSON Merge Patch structure mimics the target resource, it makes it easier for client applications to craft a JSON Merge Patch request than a JSON Patch request.

Most popular public REST APIs tend to align more closely to the JSON Merge Patch spec. 

## Patching a resource in .NET Core

Let us come back to partial JSON updates in the context of .NET Core.

Last year, The .NET Core team announced their own JSON serializer as part of [System.Text.Json](https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to), moving away from the popular NewtonSoft.Json library.

If you search for *"PATCH ASP.NET Core"*, one of the first results is [JSONPatch](https://docs.microsoft.com/en-us/aspnet/core/web-api/jsonpatch?view=aspnetcore-3.1).

However, [JSONPatch](https://docs.microsoft.com/en-us/aspnet/core/web-api/jsonpatch?view=aspnetcore-3.1) still relies on the *NewtonSoft.Json* package, and, as its name implies, implements the JSON Patch spec.
 
To support JSON Merge Patch, the first approach that comes to mind is to define a [DTO](https://en.wikipedia.org/wiki/Data_transfer_object) class and to use model binding to deserialize the request into a DTO instance.

This is what is traditionally done for PUT and POST (Create) requests.

Using the prior sample request, we would define the following:

```csharp
public class Person
{
  public string Name { get; set; }
  public string Email { get; set; }
  public PhysicalAttributes PhysicalAttributes { get; set; }
  public List<string> FavoriteColors { get; set; }
}

public class PhysicalAttributes
{
   public decimal? Weight { get; set; }
   public decimal? Height { get; set; }
}
```

We can then call it a day, right?

...not so quick. 

This works for the most part but there is an aspect of the spec that is not fulfilled.

> `null` values in the merge patch are given special meaning to indicate the removal of existing values in the target.

This information gets lost after deserializing the request. 
A `null` property on the C# model can represent either a `null` JSON value or the absence of the JSON property.

In the preceding patch example, the intent is to remove the `email` but such an implementation would instead ignore it, as if the property was absent from the request.

## JSON Merge Patch with System.Text.Json

I stumbled upon [this issue](https://github.com/dotnet/runtime/issues/31433) in the [dotnet/runtime Github repo](https://github.com/dotnet/runtime). 

All credits for the solution below go to [Ahson Khan](https://github.com/ahsonkhan).

As I was explaining, the naive initial approach was losing information about what properties were explicitly sent with `null`. 

One thing we can do is to parse the patch document and extract all those `null` properties. 

Combined with a DTO that contains all the non-null properties, there is now enough information to correctly patch the resource.

This can be done with the few lines of code below:
```csharp
public static List<string> ExtractNullProperties(string patch)
{
  using var patchDoc = JsonDocument.Parse(patch);
  if (patchDoc.RootElement.ValueKind != JsonValueKind.Object)
  {
    throw new InvalidOperationException($"The patch JSON document must be an object type. Instead it is {patchDoc.RootElement.ValueKind}.");
  }

  return ExtractNullPropertiesFromObject(patchDoc.RootElement).ToList();
}

private static IEnumerable<string> ExtractNullPropertiesFromObject(JsonElement patch)
{
  Debug.Assert(patch.ValueKind == JsonValueKind.Object);
  foreach (var property in patch.EnumerateObject())
  {
    if (property.Value.ValueKind == JsonValueKind.Null)
    {
      yield return property.Name;
    }
    else if (property.Value.ValueKind == JsonValueKind.Object)
    {
      foreach (var field in ExtractNullPropertiesFromObject(property.Value))
      {
        yield return String.Join('.', property.Name, field);
      }
    }
  }
}
```
Here, we recursively identify all `null` properties and return a list of property names, with nested property names separated by `.` i.e `physicalAttributes.weight`.

What I like about this solution is that it doesn't preclude the API from using any other JSON serializer for deserializing the non-null properties.

However, notice here that the request will need to be read twice 
- once to populate the DTO with the serializer of choice 
- once to extract all `null` properties.

The API is then responsible for taking into account the list of properties that need to be deleted.

The second approach is to serialize the original resource, apply the patch, and then to deserialize the JSON result into an object that represents the patched resource.

```csharp
public static T MergeModel<T>(T original, string patch, JsonSerializerOptions options = null)
{
    var originalJson = JsonSerializer.Serialize(original, options);
    return JsonSerializer.Deserialize<T>(Merge(originalJson, patch), options);
}

public static string Merge(string original, string patch)
{
    var outputBuffer = new ArrayBufferWriter<byte>();
    using (var originalDoc = JsonDocument.Parse(original))
    using (var patchDoc = JsonDocument.Parse(patch))
    using (var jsonWriter = new Utf8JsonWriter(outputBuffer))
    {
        var originalKind = originalDoc.RootElement.ValueKind;
        var patchKind = patchDoc.RootElement.ValueKind;
        if (originalKind != JsonValueKind.Object)
        {
            throw new InvalidOperationException($"The original JSON document to merge new content into must be an object type. Instead it is {originalKind}.");
        }
        if (patchKind != JsonValueKind.Object)
        {
            throw new InvalidOperationException($"The patch JSON document must be an object type. Instead it is {originalKind}.");
        }
        if (originalKind != patchKind)
        {
            return original;
        }
        MergeObjects(jsonWriter, originalDoc.RootElement, patchDoc.RootElement);
    }
    return Encoding.UTF8.GetString(outputBuffer.WrittenSpan);
}

private static void MergeObjects(Utf8JsonWriter jsonWriter, JsonElement original, JsonElement patch)
{
    Debug.Assert(original.ValueKind == JsonValueKind.Object);
    Debug.Assert(patch.ValueKind == JsonValueKind.Object);
    jsonWriter.WriteStartObject();
    // Write all the properties of the original document.
    // If a property exists in both documents, either:
    // * Merge them, if they are both objects
    // * Completely override the value of the original with the one from the patch, if the value kind mismatches (e.g. one is object, while the other is an array or string)
    // * Ignore the original property if the patch property value is null
    foreach (var property in original.EnumerateObject())
    {
        if (patch.TryGetProperty(property.Name, out JsonElement patchPropValue))
        {
            if (patchPropValue.ValueKind == JsonValueKind.Null)
            {
                continue;
            }
            jsonWriter.WritePropertyName(property.Name);
            var propValue = property.Value;
            if (patchPropValue.ValueKind == JsonValueKind.Object && propValue.ValueKind == JsonValueKind.Object)
            {
                MergeObjects(jsonWriter, propValue, patchPropValue); // Recursive call
            }
            else 
            {
                patchPropValue.WriteTo(jsonWriter);
            }
        }
        else
        {
            property.WriteTo(jsonWriter);
        }
    }
    // Write all the properties of the patch document that are unique to it (beside null values).
    foreach (var property in patch.EnumerateObject())
    {
        if (!original.TryGetProperty(property.Name, out JsonElement patchPropValue) && patchPropValue.ValueKind != JsonValueKind.Null)
        {
            property.WriteTo(jsonWriter);
        }
    }
    jsonWriter.WriteEndObject();
}
```

Example:
```csharp
var originalModel = new Person { Email = "joe@example.com", Name = "Joe", PhysicalAttributes = new PhysicalAttributes (75, 175), FavoriteColors = new List<string> { "blue", "red" } };

string patch = @"{ 
""favoriteColors"": [""black""], 
""email"": null, 
""physicalAttributes"": {
    ""weight"": 80
}}";

var options = new JsonSerializerOptions
{
  PropertyNameCaseInsensitive = true,
  PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};

var patchedModel = JsonMergeUtils.MergeModel(originalModel, patch, options);
// patchedModel { Name = "Joe", Email = null, FavoriteColors = ["black"], PhysicalAttributes = { Weight = 80, Height = 175 } }
```

`MergeModel` uses the *System.Text.Json* serializer here but it could easily be swapped with another JSON serializer.

There it is, you now have the building blocks to process JSON Merge requests. 

The code above, along with some async overrides, can be found [in this gist](https://gist.github.com/onlyann/fba43a44fc62d3b5559677011983e427).