Building Microservices in Go: REST APIs - Custom JSON Types
Tags: go, rest, microservices
Published: 2021-06-11 18:22:09

> **Disclaimer:** This post includes Amazon affiliate links. If you click on one of them and you make a purchase I'll earn a commission. Please notice your final price **is not** affected at all by using those links.

---

## Implementing Custom JSON types

Creating a nice _User Experience_ for the customers of our REST API should be considered when implement our handlers because in the end they are the driving factor of the popularity and usage of what we created. In this post I cover how to build custom JSON types to enhance the way we define different types that make more sense when using JSON payloads in the requests and responses.

Specifically I'm talking about two types currently in our "To Do Microservice", the first one [`Priority`](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/todo.go#L24-L25):

```go
// Priority indicates how important a Task is.
type Priority int8
```

And the second one [`Dates`](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/todo.go#L40-L44):

```go
// Dates indicates a point in time where a task starts or completes, dates are not enforced on Tasks.
type Dates struct {
	Start time.Time
	Due   time.Time
}
```

Before that, let's talk about how we are receiving data from our clients and how we are sending data back to them.

## Introducing `encoding/json`

> The code used for this post is [available on Github](https://github.com/MarioCarrion/todo-api-microservice-example/tree/c7383ad410f64c4449fc52be512744b8b8e65e9c).

We implemented [HTTP handlers](https://mariocarrion.com/2021/04/18/golang-microservices-rest-api-http-handlers.html) in the first post of this series and decided to use JSON as the message format for receiving and sending data, we could have used any other format, like XML for example, but for our use case JSON made sense because it is well supported by multiple browsers and programming languages.

The way we generated those messages is by using the package [`encoding/json`](https://pkg.go.dev/encoding/json) which is included in the standard library. This package defines a lot of interesting types for interacting with JSON, for example the following interface types:

* [`json.Marshaler`](https://pkg.go.dev/encoding/json#Marshaler): a type implementing this interface indicates it knows how to convert itself into a valid JSON, and
* [`json.Unmarshaler`](https://pkg.go.dev/encoding/json#Unmarshaler): a type implementing this interface indicates it knows how to create an instance of itself from a valid JSON value.

Besides those, there are another function types used for _marshaling_ and _unmarshaling_:

* [`json.Marshal`](https://pkg.go.dev/encoding/json#Marshal): is used to _marshal_ types into a slice of bytes, if the type implements `json.Marshaler` then that one is used, and
* [`json.Unmarshal`](https://pkg.go.dev/encoding/json#Unmarshal): is used to _unmarshal_ a valid JSON, in the form of slice of bytes, into an instance of a Go type meant to represent that argument, if the type implements `json.Unmarshaler` then that method is used.

In both cases if the types don't implement the mentioned interface types then the default logic will be followed.

By putting all that knowledge in practice we could build [something like this](https://play.golang.org/p/5-HjHMfF6Cz):

```go
package main

import (
	"encoding/json"
	"fmt"
	"log"
)

type Number int

const (
	Unknown Number = iota
	One
	Two
)

func (n *Number) UnmarshalJSON(b []byte) error {
	var s string
	if err := json.Unmarshal(b, &s); err != nil {
		return err
	}

	switch s {
	case "one":
		*n = 1
	case "two":
		*n = 2
	default:
		*n = 0
	}

	return nil
}

func (n Number) MarshalJSON() ([]byte, error) {
	var s string

	switch n {
	default:
		s = "zero"
	case One:
		s = "one"
	case Two:
		s = "two"
	}

	return json.Marshal(s)
}

func main() {
	var n Number = 1

	b, _ := json.Marshal(n) // XXX: ignoring error
	fmt.Println(string(b)) // prints out: "one"

	//-

	_ = json.Unmarshal([]byte(`"two"`), &n) // XXX: ignoring error
	fmt.Println(n) // prints out "2"
}
```

The code above defines a type called `Number` that _marshals_ itself into a valid JSON string, and _unmarshals_ itself from a valid JSON string into the corresponding value of the `Number` type.

## Receiving data from our clients

Our handlers use another type called [`json.Decoder`](https://pkg.go.dev/encoding/json#Decoder) for decoding (or unmarshaling) the received payload. For example if we use the [`create` handler](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest/task.go#L64), its implementation looks like this:

```go
func (t *TaskHandler) create(w http.ResponseWriter, r *http.Request) {
	var req CreateTasksRequest
	if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
		// ... some code ...
	}

	// ... some other code ...
}
```

What this `json.NewDecoder(r.Body).Decode(&req)` call is doing is:

1. Reading the body of the request: `r.Body`, then 
1. Transforming the JSON, and finally
1. Assigning the transformed JSON values into the `req` variable of the type `CreateTasksRequest`.

Internally [`json.*Decoder.Decode()`](https://pkg.go.dev/encoding/json#Decoder.Decode) uses the logic defined in `json.Unmarshal` for doing that conversion, and depending if the fields are implementing `json.Unmarshaler` or not then that method would be used.

## Sending data to our clients

Writing values using the received `http.ResponseWriter` in our handler is what allows our users to get data back. For example if we use the [`task` handler](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest/task.go#L96), its implementation looks like this:

```go
func (t *TaskHandler) task(w http.ResponseWriter, r *http.Request) {
	// ... some code meant to get a task from somewhere ...

	renderResponse(w,
		&ReadTasksResponse{
			Task: Task{
				ID:          task.ID,
				Description: task.Description,
				Priority:    NewPriority(task.Priority),
				Dates:       NewDates(task.Dates),
			},
		},
		http.StatusOK)
}
```

In this case the [`renderResponse` function](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest/rest.go#L17) is the one writing the data:

```go
func renderResponse(w http.ResponseWriter, res interface{}, status int) {
	w.Header().Set("Content-Type", "application/json")

	content, _ := json.Marshal(res)

	// ... some error validation ...

	_, err = w.Write(content) // XXX: explicitly ignoring error
}
```

Which then again happens to be using the `json.Marshal` function as well as any implementation of the `json.Marshaler` in the types defined as fields.

## Defining custom JSON type

If we take a similar approach to what was implemented in the first example, we can define new types in the [`rest`](https://github.com/MarioCarrion/todo-api-microservice-example/tree/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest) package equivalent to the domain types mentioned in the beginning, however those **will need** to implement both `json.Unmarshaler` and `json.Marshaler` to handle any custom logic.

For example for [`rest.Priority`](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest/priority.go#L11-L12) we could define it as:

```go
// Priority indicates how important a Task is.
type Priority string

const (
	priorityNone   Priority = "none"
	priorityLow    Priority = "low"
	priorityMedium Priority = "medium"
	priorityHigh   Priority = "high"
)

// Validate ...
func (p Priority) Validate() error {
	switch p {
	case "none", "low", "medium", "high":
		return nil
	}

	return errors.New("unknown value")
}

// MarshalJSON ...
func (p Priority) MarshalJSON() ([]byte, error) {
	if err := p.Validate(); err != nil {
		return nil, fmt.Errorf("convert: %w", err)
	}

	b, err := json.Marshal(string(p))
	if err != nil {
		return nil, fmt.Errorf("json marshal: %w", err)
	}

	return b, nil
}

// UnmarshalJSON ...
func (p *Priority) UnmarshalJSON(b []byte) error {
	var s string
	if err := json.Unmarshal(b, &s); err != nil {
		return fmt.Errorf("json unmarshal: %w", err)
	}

	if err := Priority(s).Validate(); err != nil {
		return fmt.Errorf("convert: %w", err)
	}

	*p = Priority(s)

	return nil
}
```

The [complete code](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest/priority.go) include more details besides the Marshaling/Unmarshaling logic; similar steps could be taken for [`Dates`](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/rest/dates.go) as well.

---

## Parting words

When building JSON-based APIs we should consider defining custom types to make our API easier to understand, like what we covered in this post, using humanized values like `"none"`, `"low"`, `"medium"` or `"high"` instead of [constant integers](https://github.com/MarioCarrion/todo-api-microservice-example/blob/c7383ad410f64c4449fc52be512744b8b8e65e9c/internal/todo.go#L10-L22) improves the readability and usage of our APIs, it's an investment we can make and it will pay off as well sooner than later when defining documentation using OpenAPI 3 because those options could be use as a concrete list of supported enum values.

In the next post I will cover OpenAPI 3 giving you more details about how we can build upon all the decisions we made already and how those are starting to make more sense for a final REST API.

## Recommended Reading

If you're looking to sink your teeth into more REST and Web Programming I recommend the following books:

* [The Design of Web APIs (2019)](https://amzn.to/32yFwFe)
* [Go Web Programming (2016)](https://amzn.to/30euJ1U)
* [REST in Practice (2010)](https://amzn.to/3n6ztRZ)