Create C# Classes from Avro IDL
Tags: dotnet, avro, kafka
Published: 2021-01-24 19:53:52

---
series: Avro and .NET
---

I was recently involved in a project where we built a .NET service which consumed messages from various Kafka topics, the messages themselves were serialised using [Apache Avro](https://avro.apache.org/).

This is the first time I had come across Avro, and there aren't many tools in the .NET ecosystem around this. In this series, I'll cover how to generate C# classes from Avro schemata, automate the steps, and finally how to integrate it with your project's build.

# Avro Schema

Avro schemata are defined as JSON in file with the extension `.avsc`. An example schema of a message which contains a single string would be:

```json
{
  "type" : "record",
  "name" : "ExampleMessage",
  "namespace" : "example.avro",
  "fields" : [ {
    "name" : "body",
    "type" : "string"
  } ]
}
```

You can generate C# classes from `.avsc` files using `avrogen`, which comes as part of https://www.nuget.org/packages/Apache.Avro.Tools/.

# Avro IDL

In our project, the messages were already defined and all we had to do was create the corresponding C# classes to hydrate into.

There was an extra catch for us: the schemata themselves were defined using the higher-level [Avro IDL](https://avro.apache.org/docs/current/idl.html) language and there currently isn't a tool to generate C# classes from Avro IDL.

The Avro IDL version of above schema would be:

```java
@namespace("example.avro")
protocol ExampleMessage {
  record ExampleMessage {
    string body;
  }
}
```



# Avro IDL to C# Class

Avro IDL files have the extension `.avdl`. To turn these files into C# classes we had to:

- Turn AVDL into AVSC using [the Java tool avro-tools](https://mirrors.ukfast.co.uk/sites/ftp.apache.org/avro/avro-1.10.0/java/avro-tools-1.10.0.jar) - this requires Java
- Turn AVSC into a C# class using the `avrogen` tool mentioned earlier

## Steps

Take the example Avro IDL above and save it to a file called `ExampleMessage.avdl`.

Turn it into an `.avsc` file with:

```bash
java -jar /path/to/avro-tools-1.10.0.jar idl2schemata ExampleMessage.avdl .
```

This produces `ExampleMessage.avsc`, which we then turn into a C# class with:

```bash
avrogen -s ExampleMessage.avsc .
```

The final file created is `./example/avro/ExampleMessage.cs`:

```csharp
namespace example.avro
{
    using System;
    using System.Collections.Generic;
    using System.Text;
    using Avro;
    using Avro.Specific;
    
    public partial class ExampleMessage : ISpecificRecord
    {
        public static Schema _SCHEMA = Avro.Schema.Parse("{\"type\":\"record\",\"name\":\"ExampleMessage\",\"namespace\":\"example.avro\",\"fields\":[{\"n" +
                "ame\":\"body\",\"type\":\"string\"}]}");
        private string _body;
        public virtual Schema Schema
        {
            get
            {
                return ExampleMessage._SCHEMA;
            }
        }
        public string body
        {
            get
            {
                return this._body;
            }
            set
            {
                this._body = value;
            }
        }
        public virtual object Get(int fieldPos)
        {
            switch (fieldPos)
            {
            case 0: return this.body;
            default: throw new AvroRuntimeException("Bad index " + fieldPos + " in Get()");
            };
        }
        public virtual void Put(int fieldPos, object fieldValue)
        {
            switch (fieldPos)
            {
            case 0: this.body = (System.String)fieldValue; break;
            default: throw new AvroRuntimeException("Bad index " + fieldPos + " in Put()");
            };
        }
    }
}
```

# Summary

While this works for a single file the experience doesn't really scale when we start dealing with multiple files.

For Java developers, there are plugins which handle this so all developers need to do is add AVDL files to their projects and the tooling generates the Java classes.

Ideally, we should have some equivalent in .NET, which is what I'll be covering in the rest of this series!