System.Text.Json in action with practical example in VB.net
Tags: vbnet, json, net, wikibase
Published: 2020-08-16 11:35:19

XML is almost 25 years old and it is now a format that is widely-known to the point that also non tech-savvy people have at least a vague notion of it. The format was originally meant to encode documents in a format that is, of course, machine-readable, but that is *friendly* enough to also be *human*-readable.
It has been criticised for its complexity and the difficulty to exchange data structure between application.

For this reason, the format known as *JavaScript Object Notation* (JSON) came to prominence as a less verbose (albeit less human-readable as well) alternative for data interchange.
In this article I am going to show the creation of a JSON data structure to be used to programmatically create (or modify) an item on a WikiBase instance using the [WikiBase API](https://www.mediawiki.org/wiki/Wikibase/API) with VB.net.

## JSON in .NET
Before *NET Core 3.0* and *.NET Framework 4.7.2* using JSON in .NET was much more complicated than, say, XML. The choices were to use the somewhat limited [`JavaScriptSerializer` class](https://docs.microsoft.com/en-us/dotnet/api/system.web.script.serialization.javascriptserializer), that was specifically designed for AJAX-enabled applications, or move to the popular [Json.net](https://www.newtonsoft.com/json) library.
Starting with the above-mentioned versions, however, .NET *finally* started to offer a serious implementation of classes intended for JSON manipulation.
This has been announced in the .NET Blog, in a post called [Try the new System.Text.Json APIs](https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis/) that I suggest you read if you like to know more about the implementation.
In the rest of this article I will show you a practical use of this new component.

## The problem
For a project of mine, I need to massively create and edit items into a WikiBase-enabled MediaWiki installation. WikiBase is the extension that powers [WikiData](https://wikidata.org), but can be used on custom wiki site as well.
Any WikiBase-equipped site (WikiData included, of course, if you are wondering) allows interaction through [WikiBase-specific API](https://www.mediawiki.org/wiki/Wikibase/API).
To programmatically access and edit MediaWiki content I already used the excellent [DotNetWikiBot framework](https://sourceforge.net/projects/dotnetwikibot/) so I decided to expand it, using its already existing infrastructure to add WikiBase-specific functions (work is still underway on this project, but I will eventually release it as opensource, ideally [integrating it](https://sourceforge.net/p/dotnetwikibot/discussion/572544/thread/0b21ca9609/) into existing code).

Data into WikiBase are easily accessible as a JSON formatted data by accessing the *EntityData* special page like, for instance:
```
https://www.wikidata.org/wiki/Special:EntityData/Q96482610.json
```

When using the API, JSON data are of paramount importance to create or edit entities or part of them. Thus, handling JSON into my .NET application was crucial.

My first task was to create a set of Properties into the WikiBase instance, so I needed to use the [`wbeditentity`](https://www.wikidata.org/w/api.php?action=help&modules=wbeditentity) module following the example
> Create a new property containing the json data, return full entity structure
`api.php?action=wbeditentity&new=property&data={"labels":{"en-gb":{"language":"en-gb","value":"Propertylabel"}},"descriptions":{"en-gb":{"language":"en-gb","value":"Propertydescription"}},"datatype":"string"}`

The `data` parameter containing the JSON data.

## The solution
My application is a form-based one as I need to easily interact with the interface and be able to read and store results in various controls.
So first of all I needed to add the `System.Text.Json` (in contrast, this library is already part of the .NET Core 3.0) via the [NuGet package](https://www.nuget.org/packages/System.Text.Json).

This library allows object serialization, but this is too complex for my needs, as I would prefer manipulating the DOM without the need of creating corresponding classes in my code. While the solution might seem less elegant than having the full JSON structure created as .NET structure in classes and then simply *serialize* them into JSON with a single command, the reality is that the sheer code needed to create all the classes would have been much more than the one needed to directly handling the JSON DOM.
That being said, it remains the fact that JSON serializer is very useful and might be object of a future article.

### First steps
So the first steps to me were to familiarize with the [`Utf8JsonWriter`](https://docs.microsoft.com/en-us/dotnet/api/system.text.json.utf8jsonwriter) method so I started converting the [C♯ example](https://gist.github.com/terrajobst/336da6ae0a327bef113c1325710164f0#file-sample9-cs) to the following VB.net example:
 
{% gist https://gist.github.com/lucamauri/4b491b195f2ef12566d6bd62b40514a3 %}

Pay special attention at the `Writer.Flush()` method that can easily be overlooked, but that is critical for the stream to be filled with actual data.
As this worked as intended, I could proceed with the final version of the subroutine

### The practical code
In order to create a *Property* in Wikibase you need to provide JSON data as a parameter to the API call to set the `labels` (in the proper language), the related `descriptions` and `[datatype](https://www.wikidata.org/wiki/Help:Data_type)` defining the type of the data the property will contain.

An example would be:
```json
{
  "datatype": "string",
  "labels": {
    "en": {
      "language": "en",
      "value": "Label"
    },
    "it": {
      "language": "it",
      "value": "Etichetta"
    }
  },
  "descriptions": {
    "en": {
      "language": "en",
      "value": "Description"
    },
    "it": {
      "language": "it",
      "value": "Descrizione"
    }
  }
}
```
With labels and descriptions in English and Italian.

I started by creating an helper class like this:
```vb
Class LabelsWithDescription
    Public ReadOnly Property Culture As String
    Public ReadOnly Property Label As String
    Public ReadOnly Property Description As String

    Sub New(Culture As String, Label As String, Description As String)
        Me.Culture = Culture
        Me.Label = Label
        Me.Description = Description
    End Sub
End Class
```
Please **note** that this class cannot be serialized as it is: it holds all the necessary information, but not in the right structure. So it will *only* help creating the structure. As mentioned above, creating the proper structure in VB and then serialize it would have been more complicated and time consuming in comparison to writing the DOM directly. At least this is the case for the current example, of course this depends on the situation and the reader must evaluate this on a per-case basis.

Let's start to initialize the objects:
```vb
Sub New(IndentOutput As Boolean)
    Dim WriterOption As JsonWriterOptions

    With WriterOption
        .Indented = IndentOutput
    End With

    MemStream = New System.IO.MemoryStream
    JSONWriter = New Utf8JsonWriter(MemStream, WriterOption)
End Sub
```
We are creating here
* a `Utf8JsonWriter` that actually format and writes data
* its `WriterOption` (needed to chose if indent the cose for readability, or minify-it with no spaces) and
* the `MemoryStream` holding result.

An now, finally, the function doing the job:
```vb
Function PrepareNewItemData(ItemLabels As List(Of LabelsWithDescription), DataType As String) As String
    With JSONWriter
        .WriteStartObject()
        .WriteString("datatype", DataType)

        .WriteStartObject("labels")
        For Each Label As LabelsWithDescription In ItemLabels
            .WriteStartObject(Label.Culture)
            .WriteString("language", Label.Culture)
            .WriteString("value", Label.Label)
            .WriteEndObject()
        Next
        .WriteEndObject()

        .WriteStartObject("descriptions")
        For Each Label As LabelsWithDescription In ItemLabels
            .WriteStartObject(Label.Culture)
            .WriteString("language", Label.Culture)
            .WriteString("value", Label.Description)
            .WriteEndObject()
        Next
        .WriteEndObject()

        .WriteEndObject()
        .Flush()
    End With
    Return Encoding.UTF8.GetString(MemStream.ToArray())
End Function
```
The first couple of commands basically start writing
```json
{
  "datatype": "string",
```
The tricky parts comes afterwards where the method `.WriteStartObject("labels")` output the fragment
```json
  "labels": {
```
in the JSON structure, allowing other content to be added to this "node".
The two `For Each` basically uses the same strategy to create an Object with the name of the language and the related *Labels*, in the first cycle, and *Descriptions*, in the second cycle.

The final `.WriteEndObject()` basically closes the whole data fragment and the very *important* `.Flush()` commits actual data to the stream that can be converted to string with `.GetString`.

Now we can put everything to work with:
```vb
Dim JSONTest As New JSONEditor(False)
Dim LabelsList As New List(Of JSONEditor.LabelsWithDescription)

LabelsList.Add(New JSONEditor.LabelsWithDescription("en", "Label", "Description"))
LabelsList.Add(New JSONEditor.LabelsWithDescription("it", "Etichetta", "Descrizione"))

MessageBox.Show(JSONTest.PrepareNewItemData(LabelsList, "string"))
```
The output will be
```json
{"datatype":"string","labels":{"en":{"language":"en","value":"Label"},"it":{"language":"it","value":"Etichetta"}},"descriptions":{"en":{"language":"en","value":"Description"},"it":{"language":"it","value":"Descrizione"}}}
```
our intended JSON payload as above, but in minified format.

## Conclusion
`System.Text.Json` is intended to be a easy-to-use, fast and integrated alternative to third-party JSON editors. This first adventure using it was a bit bumpy mainly because of the lack of detailed documentation and real examples on the internet, basically, I guess, because it is very new and not yet much used.
Nevertheless it seems to me using is is really worth the minor pain of searching and experimenting.
With this little story of mine I hope having helped other users to get start with it a little faster than I originally did.