What I learnt while working my first open source package for Go
Tags: showdev, go, news, todayilearned
Published: 2020-05-10 05:47:17

Recently I released a small package for Go. 
{% github akshaybharambe14/go-jsonc no-readme %}

While building this, I came across various problems and best practices. So I just wanted to share my learnings.

##**Make it compatible with the standard library**

This is my first priority when I build something for public. **This makes your library widely compatible and reusable**.

In this library, I made the [decoder](https://pkg.go.dev/github.com/akshaybharambe14/go-jsonc?tab=doc#Decoder) compatible with [io.Reader](https://golang.org/pkg/io/#Reader). Now, I don't have to care about the source. It can be an HTTP request, a file, etc. Anything that implements io.Reader, is supported. I recommend you to read [this](https://medium.com/@matryer/golang-advent-calendar-day-seventeen-io-reader-in-depth-6f744bb4320b) excellent article from [Mat Ryer
](https://twitter.com/matryer)

##**Reduce your API surface**

In simple words, avoid using external libraries for some simple tasks. **Try to use the standard library as much as you can**.

In the context of this library, I intentionally avoided some functionality, like validating resulting JSON, so that the importing library can reuse its existing dependencies. Stick to the original problem instead of providing some fancy functionality.

##**Unit Tests are must**

Writing unit tests is fun and I would say it makes your program more robust and easy to maintain. **Whenever you change some logic, tests make sure your program is working as intended or not**. If tests fail, then you know what to do!

This library provides **100% test coverage** and this helped me allot.

You can test your program with the following command.
```sh
go test -cover
```

##**Benchmark your program**

**Don't just get to the conclusion without benchmarking your programs**. Go tooling makes it super easy to benchmark your programs. You get the following information from benchmarks for a single function.
- Number of operations performed
- Time required for each operation in nanoseconds
- Data processed per second
- Memory required per operation
- Allocations made per operation

You can benchmark your program with the following command.
```sh
go test -bench=. -benchmem
```

Have a look at benchmarks from this package. As you can see, using io.Reader, memory is being reused resulting in almost **500% performance increase** in data processing than existing available library [jsonc](https://github.com/muhammadmuzzammil1998/jsonc)

```
goos: windows
goarch: amd64
pkg: github.com/akshaybharambe14/go-jsonc/benchmarks
BenchmarkOwnSmallJSONBytes-4              256599              4952 ns/op         353.00 MB/s           0 B/op          0 allocs/op
BenchmarkOwnSmallJSONBytesReader-4        206823              5832 ns/op         299.70 MB/s        6224 B/op          5 allocs/op
BenchmarkJSONCSmallJSONBytes-4            171474              6925 ns/op         252.41 MB/s        1792 B/op          1 allocs/op
BenchmarkOwnBigJSONBytes-4                 33517             35921 ns/op         462.26 MB/s           0 B/op          0 allocs/op
BenchmarkOwnBigJSONBytesReader-4          105244             11292 ns/op        1470.45 MB/s        6224 B/op          5 allocs/op
BenchmarkJSONCBigJSONBytes-4               19599             61422 ns/op         270.34 MB/s       18432 B/op          1 allocs/op
PASS
ok      github.com/akshaybharambe14/go-jsonc/benchmarks 26.250s
```

##**CPU profiling and memory profiling**

If you think your program is not fast enough, then you can profile your programs. **This really helps in finding the culprits**. 

In my case, I identified that my decode function was spending almost 40% time in just checking the byte can be added to resulting JSON or not. Results, I ended up gaining a 40% performance increase.

You don't need any separate tools to profile your programs. Benchmarks can be used for this also. You can generate profiles and read them with [pprof](https://blog.golang.org/pprof).  

```sh
go test -bench=. -benchmem -memprofile memprofile.out -cpuprofile profile.out
```

##**You can't avoid all allocations**

I was happy with the above tweaks, but still, I was trying to avoid those 5 allocations per operations in the case of io.Reader as input (See benchmarks). **But seriously stop being greedy and check it in the memory profile if those allocations are avoidable or not**. 

In my case, internal functions were making allocations to the heap and I can't control them. So I stopped there.

##**Work on feedbacks**

Don't stop here and work on the feedback provided by your colleagues, friends, and community. **File issues and work on them in your free time**.

That's it for now. It was a nice experience and I am looking forward to contributing to some opensource go projects.
{% github akshaybharambe14/go-jsonc no-readme %}

Give it a try. Your feedback is highly appreciated. I will surely work on your inputs, that's what I just learned. 

Happy coding!