Transforming json data with easygen
Tags: json, transform, easygen, generator
Published: 2021-12-31 04:00:35

---
title: Transforming json data with easygen
series: transforming json data to be more presentable to human beings
published: true
description: Nothing is more easier than that
tags: json,transform,easygen,generator
cover_image: https://www.imd.org/contentassets/11a6ae4d75ce4d4a83c9a8e87b3944e4/iai_07-18_big-teaser758x334.jpg
---

### Filtering

Take a look at the vast amount of data a github API may return, e.g.: https://api.github.com/search/repositories?q=go.

If there is only few key data that we want, then the first thing to do is to filter out the unwanted. Can `easygen` do that? Yes, of course. Given the power of Go template engine, everything is possible. However, such filtering is _imperative_ programming if done in `easygen`, especially when doing vertically filtering (skipping unwanted records). So it is better to use the _declarative_ approach, using tools like `jp` or `jq` instead. Since that is outside of the scope of this article, we'll skip it and move on to the next section.

### Presenting

Take a look at the end result first, at https://github.com/go-sqlparser/current/wiki/List:-Currently-Collected-Projects:

> ![image](https://user-images.githubusercontent.com/422244/147801349-ade37b3d-56c5-4276-a8d7-296d27362c9f.png)

Here is what happening behind the scene -- the mark down source code that renders above:

> ![image](https://user-images.githubusercontent.com/422244/147801476-97638ea8-28c6-4a1d-88f5-403e711f3214.png)

Note that 

- the project name keyword has been repeated five times in the mark down source code. 
- there is a TOC index that links directly to the detailed content. I.e., each project is repeated twice, first listed briefly in the TOC/index section, then full details in the following section.
- the PrimaryLanguage is therefore need to be repeated in two sections as well.

Such repetitive tasks are best  leave to tools to automate it, and `easygen` is such code/text auto generating tools. The mark down code generation template is as simple as this:

```sh
$ cat repos.tmpl 
## Currently Collected Projects

### List
{{range .Data.RepositoryOwner.Repositories.Edges}}
- [{{.Node.Name}} ({{.Node.PrimaryLanguage.PrimaryLanguage}})](#{{.Node.Name}}){{end}}

### Details
{{range .Data.RepositoryOwner.Repositories.Edges}}
<a name="{{.Node.Name}}"/>**{{.Node.Name}}** ({{.Node.PrimaryLanguage.PrimaryLanguage}})  
{{.Node.Url}}  
{{.Node.Description}}
{{end}}
```

### To put them together

So in order to come up with the above end result, I need to

- download `json` data via API
- filter the `json` data with the [`jp`](https://github.com/go-jsonfile/jp/releases/tag/v0.2.2)
- then present the `json` data in human friendly mark down format with the new `easygen` that can read from `stdin`.
- in the human friendly form, if I need to convert the long number of project size from bytes to the size in KB or MB, and I can make use of `easygen`'s built in [calculation in Go template](https://github.com/go-easygen/easygen/issues/25#cal) supports. 
- there are lots of other transformation support functions already builtin inside `easygen`, check out the full list with sample usage and results [here](https://github.com/go-easygen/easygen/issues/25).


**UPDATE:**

project size added to list details:

> ![image](https://user-images.githubusercontent.com/422244/147834144-4fbac511-9f50-4b66-946f-8f80e7b33da4.png)

and here is the updated code generation template:

```
$ cat repos.tmpl 
## Currently Collected Projects

### List
{{range .Data.RepositoryOwner.Repositories.Edges}}
- [{{.Node.Name}} ({{.Node.PrimaryLanguage.PrimaryLanguage}})](#{{.Node.Name}}){{end}}

### Details
{{range .Data.RepositoryOwner.Repositories.Edges}}{{$mb := sprintf "%.2f" (divide .Node.Size 1024)}}
<a name="{{.Node.Name}}"/>**{{.Node.Name}}** ({{.Node.PrimaryLanguage.PrimaryLanguage}}, {{.Node.Size}}KB/{{$mb}}MB)  
{{.Node.Url}}  
{{ coalesce .Node.Description "No description" }}
{{end}}
```

### Credit

The cover image is obtained from [imd.org](https://www.imd.org/contentassets/11a6ae4d75ce4d4a83c9a8e87b3944e4/iai_07-18_big-teaser758x334.jpg).