Repository: go
Issue: proposal: encoding/json: `nilasempty` to encode nil-slices as `[]`
Labels: Proposal

https://go-review.googlesource.com/c/go/+/205897

This is an improvement over @pjebs PR/Proposal (which can be found here: https://github.com/golang/go/issues/27589) for the following reasons:
- The previous PR had a naming issue which was unresolved. As such I spent a long time stewing over the name of the option, and ended up going with `nilasempty` which I think works very well to describe what it does, and even gives a clue as to why it exists
- The encoder functions are entirely responsible for the encoding of their nil versions resulting in a cleaner implementation
- There are separate tests

This is the cleanest, most straightforward, most complete change that could implement this feature in the existing `encoding/json` package. There are good, clean, readable tests too.

If the maintainers agree, I really think it'd be great to get this merged so that we can start benefiting from it's presence in the next suitable release! Let's put this shortcoming to bed 👍 

### The issue 

I have an json API contract that says my response must look like this:
```json
{
  "foo": ["some"],
  "bar": ["values", "for the consumer"], 
  "baz": []
}
```
(empty-array expected over `null`)

So naturally I create a type to help me satisfy the contract
```go
type MyResponse struct {
  Foo: []string `json:"foo"`,
  Bar: []string `json:"bar"`,
  Baz: []string `json:"baz"`,
}
```

Then someone else comes along and creates a mock response for testing
```go
myMock := MyResponse{
  Foo: []string{"blah"},
  Bar: []string{"blah"},
}
```
The type checker is happy with a nil slice for `Baz`
```go
json.marshal(myMock)
```
for already understood reasons, this results in
```json
{
  "foo": ["blah"],
  "bar": ["blah"],
  "baz": null,
}
```
which, of course does not satisfy the contract as intended.

In conclusion of that, the type system is not doing anything wrong, but it's not helping us satisfy contracts we have in json.

### Moving forward

So what can we do? The obvious choice is a constructor to initialise `MyResponse` -- but Go doesn't have any way of enforcing constructors, so it has to be opt-in as far as usage goes, which _is_ going to be forgotten and the type checker still isn't going to complain to save us from ourselves. It's not a satisfactory solution.

### Digging deeper

I began thinking about where the fault lies - is it in Go's `nil` slices? is it in the json encoder? is it in our usage of types? 

I concluded that the error does lie in the choice the json encoder makes in choosing how to encode a nil-slice and I'll do my best to explain why I think that's the case.

The author of the encoder chose to encode a nil-slice as null -- but why? The code that returns `null` is _inside_ a function which knows it's encoding a `slice`. 

https://github.com/golang/go/blob/414c1d454e6d388443239209220fe0783d4dac71/src/encoding/json/encode.go#L840-L846

The reason that decision was made seems to be because it's `nil` under the hood, and `nil == null`.

So now I want to consider, _why is the zero value of a slice `nil` in the first place?_ Correct me if I'm wrong (I'm relatively new to Go) but it's _a memory allocation optimisation_, meaning Go only needs to allocate memory when it gets it's first values -- which is totally smart when designing a memory-efficient language!

So, I'll say that nil-slices are just an implementation detail of Go in it's aim of being memory efficient out-of-the-box as a programming language -- It looks like a slice, it quacks like a slice, but aha! it's a `nil` under the hood -- but it's _still_ *a slice*.

### Aha!

Now let's look at the perspective of the encoder - the encoder appears to be encoding the underlying `nil` to `null`. But now we understand that the slice being `nil` is just an implementation detail of Go lang. The encoder _should_ be encoding the values in the context of their `type` eg. `[]string` to `[]`, and definitely _not_ converting the underlying zero-value-of-a-slice-`nil` to it's closest json equivelent of `null`. In wonderful irony, representing the underlying `nil` (which is a memory allocation optimisation for Go) as `null` in json actually costs more bytes over-the-wire than the empty array version `[]`!

So how do we address this misstep? Ideally, mirroring `omitempty` and as the less common behaviour, `nullempty` would encode empty arrays as null. But to be backward compatible, something like `nilasempty` would be best.

### Summary 

In summary, nil-slices are an implementation detail of the Go language and nil-slices should be treated as plain-old slices by the json encoder - it's a slice and it's empty, and it should be represented in json as `[]` when specifying something like `nilasempty` to enable this new behaviour in a backwards compatible way.