Golang Data Connector Part 2
Tags: go
Published: 2021-05-02 17:42:48

---
title: Golang Data Connector Part 2
published: true
date: 2021-05-02 17:39:54 UTC
tags: go
canonical_url: https://shindakun.dev/golang-data-connector-part-2/
---

### Intro
 ![Golang Data Connector Part 2](https://shindakun.dev/content/images/2021/05/attemptinggo_w.png)

[Last time](https://dev.to/shindakun/golang-data-connector-part-1-5a5f) we started our small data collector. The idea is we want to move data from a Ghost blog over to _something else_. This time we're going to extend our starting point to actually make use of the data that comes back from the API call. That means we need to take a closer look at the incoming data... and with that in mind let's get started.

### Adding Struct-ure

Currently, we are receiving our data and converting it from `[]byte` to `string` so we can print it to the screen. That's cool and all but not very useful. Instead, we need to see if we can parse the data and make it usable for the next stage of this project.

To do this we'll need to convert the JSON into a `struct`. OK, so how do we do that? There are a couple of different ways but the easiest I've found is to use the site [JSON to Go](https://mholt.github.io/json-to-go/). This site converts a JSON object into what is a matching Golang `struct`.

![Golang Data Connector Part 2](https://shindakun.dev/content/images/2021/05/Screen-Shot-2021-04-21-at-1.21.53-PM-1.png)

We can simply paste the raw output from the call to `https://demo.ghost.io/ghost/api/v4/content/posts/?key=22444f78447824223cefc48062` into the form. It will spit out the exact `struct` we need:

```go
type AutoGenerated struct {
	Posts []struct {
		ID string `json:"id"`
		UUID string `json:"uuid"`
		Title string `json:"title"`
		Slug string `json:"slug"`
		HTML string `json:"html"`
		CommentID string `json:"comment_id"`
		FeatureImage string `json:"feature_image"`
		Featured bool `json:"featured"`
		Visibility string `json:"visibility"`
		CreatedAt time.Time `json:"created_at"`
		UpdatedAt time.Time `json:"updated_at"`
		PublishedAt time.Time `json:"published_at"`
		CustomExcerpt string `json:"custom_excerpt"`
		CodeinjectionHead interface{} `json:"codeinjection_head"`
		CodeinjectionFoot interface{} `json:"codeinjection_foot"`
		CustomTemplate interface{} `json:"custom_template"`
		CanonicalURL interface{} `json:"canonical_url"`
		EmailRecipientFilter string `json:"email_recipient_filter"`
		URL string `json:"url"`
		Excerpt string `json:"excerpt"`
		ReadingTime int `json:"reading_time"`
		Access bool `json:"access"`
		OgImage interface{} `json:"og_image"`
		OgTitle interface{} `json:"og_title"`
		OgDescription interface{} `json:"og_description"`
		TwitterImage interface{} `json:"twitter_image"`
		TwitterTitle interface{} `json:"twitter_title"`
		TwitterDescription interface{} `json:"twitter_description"`
		MetaTitle interface{} `json:"meta_title"`
		MetaDescription interface{} `json:"meta_description"`
		EmailSubject interface{} `json:"email_subject"`
		Frontmatter interface{} `json:"frontmatter"`
		Plaintext string `json:"plaintext,omitempty"`
	} `json:"posts"`
	Meta struct {
		Pagination struct {
			Page int `json:"page"`
			Limit int `json:"limit"`
			Pages int `json:"pages"`
			Total int `json:"total"`
			Next interface{} `json:"next"`
			Prev interface{} `json:"prev"`
		} `json:"pagination"`
	} `json:"meta"`
}

```

Let's start by updating `AutoGenerated` to say `Ghost`. From there we can add the entire `struct` to the top of our previous code - right above `main()`. That is if we want to just added. In our imagined receiving system the entire `struct` isn't necessary so I've left out a bit. I've also updated several of the `interface{}` entries to be strings since that's what we'll actually be getting if the value is set.

```go
package main

import (
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
	"time"
)

type Ghost struct {
	Posts []struct {
		ID string `json:"id"`
		UUID string `json:"uuid"`
		Title string `json:"title"`
		Slug string `json:"slug"`
		HTML string `json:"html"`
		FeatureImage string `json:"feature_image"`
		Featured bool `json:"featured"`
		Visibility string `json:"visibility"`
		CreatedAt time.Time `json:"created_at"`
		UpdatedAt time.Time `json:"updated_at"`
		PublishedAt time.Time `json:"published_at"`
		CustomExcerpt string `json:"custom_excerpt"`
		URL string `json:"url"`
		Excerpt string `json:"excerpt"`
		ReadingTime int `json:"reading_time"`
		Access bool `json:"access"`
		OgImage string `json:"og_image"`
		OgTitle string `json:"og_title"`
		OgDescription string `json:"og_description"`
		TwitterImage string `json:"twitter_image"`
		TwitterTitle string `json:"twitter_title"`
		TwitterDescription string `json:"twitter_description"`
		MetaTitle string `json:"meta_title"`
		MetaDescription string `json:"meta_description"`
		Plaintext string `json:"plaintext,omitempty"`
	} `json:"posts"`
	Meta struct {
		Pagination struct {
			Page int `json:"page"`
			Limit int `json:"limit"`
			Pages int `json:"pages"`
			Total int `json:"total"`
			Next interface{} `json:"next"`
			Prev interface{} `json:"prev"`
		} `json:"pagination"`
	} `json:"meta"`
}

func main() {
    
    ...snip...

```

* * *
<!--kg-card-begin: html-->

| Enjoy this post? |
| --- |
| [How about buying me a coffee?](https://ko-fi.com/shindakun) |

<!--kg-card-end: html-->
* * *

### More Printing

Alright, we're slowly getting somewhere! Now, we are going to replace our `fmt.Println()` with code to actually use our `struct`. We'll read the body in using the same manner as last time.

```go
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

```

But this time instead of printing we'll create a variable `p` of the type `Ghost`.  Using `p` we then `json.Unmarshal` our received body.

```go
	var p Ghost
	err = json.Unmarshal(body, &p)
	if err != nil {
		log.Fatal(err)
	}

```

From here we now have access to `range` over the entirety of `p.Posts` and once again we print them out to the screen.

```go
	for i := range p.Posts {
		fmt.Printf("%v", p.Posts[i])
	}

```

### Next Time

That's all for this quick post, you can see the full code listing below. Next time we'll be taking one more step forward and actually use the `http` package to send our posts to another service.

### Code Listing

```go
package main

import (
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
	"time"
)

type Ghost struct {
	Posts []struct {
		ID string `json:"id"`
		UUID string `json:"uuid"`
		Title string `json:"title"`
		Slug string `json:"slug"`
		HTML string `json:"html"`
		FeatureImage string `json:"feature_image"`
		Featured bool `json:"featured"`
		Visibility string `json:"visibility"`
		CreatedAt time.Time `json:"created_at"`
		UpdatedAt time.Time `json:"updated_at"`
		PublishedAt time.Time `json:"published_at"`
		CustomExcerpt string `json:"custom_excerpt"`
		URL string `json:"url"`
		Excerpt string `json:"excerpt"`
		ReadingTime int `json:"reading_time"`
		Access bool `json:"access"`
		OgImage string `json:"og_image"`
		OgTitle string `json:"og_title"`
		OgDescription string `json:"og_description"`
		TwitterImage string `json:"twitter_image"`
		TwitterTitle string `json:"twitter_title"`
		TwitterDescription string `json:"twitter_description"`
		MetaTitle string `json:"meta_title"`
		MetaDescription string `json:"meta_description"`
		Plaintext string `json:"plaintext,omitempty"`
	} `json:"posts"`
	Meta struct {
		Pagination struct {
			Page int `json:"page"`
			Limit int `json:"limit"`
			Pages int `json:"pages"`
			Total int `json:"total"`
			Next interface{} `json:"next"`
			Prev interface{} `json:"prev"`
		} `json:"pagination"`
	} `json:"meta"`
}

func main() {
	req, err := http.NewRequest(http.MethodGet, "https://demo.ghost.io/ghost/api/v4/content/posts/?key=22444f78447824223cefc48062", nil)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatal(err)
	}

	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	var p Ghost
	err = json.Unmarshal(body, &p)
	if err != nil {
		log.Fatal(err)
	}

	for i := range p.Posts {
		fmt.Printf("%v", p.Posts[i])
	}
}

```