Attempting to Learn Go - Building Dev Log Part 03
Tags: go, atlg, continuingeducation
Published: 2019-01-26 17:58:03

---
title: Attempting to Learn Go - Building Dev Log Part 03
published: true
description: "The fun continues as we extend our simple static site generator to process multiple files!"
tags: go, atlg, continuing-education
cover_image: https://dev.shindakun.net/img/attemptinggo_w.png
series: "Attempting to Learn Go"
---

# Building Out A Dev Blog

Here we [go again](https://dev.to/shindakun/attempting-to-learn-go---building-dev-log-part-02-179c)! If you've been following these posts, welcome back, and if not welcome aboard! This time we'll be continuing our work to extend the basic code we put together to create a static site generator. We actually jump quite a bit forward this week. We've now are at a point where we can convert an entire directory of well-formatted markdown files into HTML ready to host. Anyway, let's keep on with our attempts to learn Go...

***

## Diving In

I'm still toying around with a different way to show our updated code but, for now, I think the code base is small enough I can just start from the top and work my way down. I'm going to gloss over the bits that haven't changed since last week so, if you haven't read that I suggest you do!

Now then, let's jump down to our first real changes.

```go
package main

import (
  "bufio"
  "bytes"
  "fmt"
  "html/template"
  "io/ioutil"
  "os"
  "regexp"
  "strings"

  "github.com/microcosm-cc/bluemonday"
  "github.com/russross/blackfriday"
  yaml "gopkg.in/yaml.v2"
)

const delim = "---"

type post struct {
  Title       string
  Published   bool
  Description string
  Tags        []string
  CoverImage  string
  Series      string
  PostBody    template.HTML
}

var templ = `<!DOCTYPE html>
<html lang="en">
  <head>
    <title>{{.Title}}</title>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="HandheldFriendly" content="True">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="referrer" content="no-referrer-when-downgrade" />
    <meta name="description" content="{{.Description}}" />
  </head>
  <body>
    <div class="post">
      <h1>{{.Title}}</h1>
      {{.PostBody}}
    </div>
  </body>
  </html>
  `
```

## Naming Stuff Is Hard

Here we get to the first (small) change, the function is the same as last week but I've renamed it from `loadFile()` to `getContents()`. It seems to more accurately capture what it does.

```go
func getContents(f *string) ([]byte, error) {
  b, err := ioutil.ReadFile(*f)
  if err != nil {
    return nil, err
  }
  return b, nil
}
```

One thing you should notice throughout this section is I've tried to make sure we're are breaking out code into standalone functions. This should keep our `main()` nice and make it so we can write tests. Below we have `parseFM()` which wraps up our un-marshaling and returning the first stage of our front matter. It may be more correctly called `parseYAML()` but, we'll leave it as is for now. `isNil()` has not changed and still is not really used, I will see to that before the next post, possibly.

```go
func parseFM(b *[]byte) (map[string]interface{}, error) {
  m := make(map[string]interface{})
  err := yaml.Unmarshal(*b, &m)
  if err != nil {
    msg := fmt.Sprintf("error: %v\ninput:\n%s", err, b)
    return nil, fmt.Errorf(msg)
  }
  return m, nil
}

func isNil(i interface{}) bool {
  if i != nil {
    return false
  }
  return true
}
```

As before we need to split up the data we loaded from the markdown file. A call to `splitData()` will take care of it. I've fed in various bits of malformed data and the `if` statement seemed to work as expected each time. I also ran my current set of markdown [dev.to](https://dev.to/shindakun) posts through and each was converted without a problem. They can all be found over at [dev.shindakun.net](https://dev.shindakun.net).

```go
func splitData(f *[]byte) ([][]byte, error) {
  b := bytes.Split(*f, []byte(delim))
  if len(b) < 3 || len(b[0]) != 0 {
    return nil, fmt.Errorf("Front matter is damaged")
  }
  return b, nil
}
```

Once we've split the data and parsed out the front matter we call `makePost()` to build the post struct and the template. I'm passing back the `*post` because we're currently executing the template just before we write the file to disk, not in this function itself. It will also be handy when I build out something to create an `index.html`. Again, we're not really using `isNil()` as intended mostly because I don't like the `if/else` jumble that it will create. We'll get that taken care of one of these days.

```go
// makePost creates the post struct, returns that and the template HTML
func makePost(fm map[string]interface{}, contents []byte,
  s [][]byte) (*template.Template, *post) {
  p := &post{}

  if isNil(fm["title"]) {
    panic("isNil tripped at title")
  } else {
    p.Title = fm["title"].(string)
  }
  p.Published = fm["published"].(bool)
  p.Description = fm["description"].(string)

  // TODO: Strip space after comma prior to parse?
  tmp := fm["tags"].(string)
  p.Tags = strings.Split(tmp, ", ")

  p.CoverImage = fm["cover_image"].(string)
  p.Series = fm["series"].(string)

  pBody := contents[len(s[1])+(len(delim)*2):]

  out := blackfriday.Run(pBody)

  bm := bluemonday.UGCPolicy()
  bm.AllowAttrs("class").Matching(regexp.MustCompile("^language-[a-zA-Z0-9]+$")).OnElements("code")
  p.PostBody = template.HTML(bm.SanitizeBytes(out))

  tm := template.Must(template.New("msg").Parse(templ))
  return tm, p
}
```

There you go, we're down to our `main()` function! Right of the bat, you'll see that we're now using `ioutil.ReadDir()` to get a list of the contents of the current directory.

```go
func main() {
  d, err := ioutil.ReadDir(".")
  if err != nil {
    panic(err)
  }
```

We then `range` through the list of the current directory and immediately check to see if the current file ends with `.md`. If it does the address of our string gets passed into `getContents()` and if all goes well we'll get the contents back.

```go
  for _, f := range d {
    if t := f.Name(); strings.HasSuffix(t, ".md") {
      contents, err := getContents(&t)
      if err != nil {
        panic(err)
      }
```

We'll pass a pointer to our contents into `splitData()` and then pass the second chunk of `s`, that is `s[1]`, into `parseFM()`. This gives us the base building blocks for our post. Both `splitData()` and `parseFM()` could probably be moved into `makePost()`, then we could pass the contents directly into it and do all the work there. That's something to consider for when refactoring.

```go
      s, err := splitData(&contents)
      if err != nil {
        panic(err)
      }

      fm, err := parseFM(&s[1])
      if err != nil {
        msg := fmt.Sprintf("error: %v\ninput:\n%s", err, s[1])
        panic(msg)
      }

      tm, p := makePost(fm, contents, s)
```

In this current version of the code, we're going to take the current filename and lop off the extension. We'll then create a new file with the same name as the original and append `.html`. Again, this is all currently happening in the same directory and we're being really sloppy and not checking for existing files or anything. [`os.Create()`](https://golang.org/pkg/os/#Create) uses `0666` (or equivalent permissions in Windows) for the file permissions so in theory we should be able to write and overwrite with no issues. This is another case where we might not be doing what may be considered "correct" - instead, we are just going through an iterative process. This lets everyone see how the code has changed, warts and all.

File created, `o`, we then use [`bufio.NewWriter()`](https://golang.org/pkg/bufio/#NewWriter) to create a "Writer" with a buffer. We can then execute our template and write directly into the file which is closed after we flush the buffer.

```go
      fin := strings.TrimSuffix(t, ".md")
      o, err := os.Create(fin + ".html")
      if err != nil {
        panic(err)
      }
      defer o.Close()

      buf := bufio.NewWriter(o)

      err = tm.Execute(buf, p)
      if err != nil {
        panic(err)
      }
      buf.Flush()
    }
  }
}
```

If we have any files left we jump back up to the top of our loop and do the whole things again!

***

## Next Time

We have just about everything in place! But first, we need to create and write out `index.html`. I think I'm going implement that over the next day or so and put up a quick follow up post. After that, I'll need to decide if we want to do one more post in the "dev site" series and extend the code to use external templates. It shouldn't be that much more work, I suppose I'll wait and see how busy the next week looks on Monday. See you next time!

***

You can find the code for this and most of the other Attempting to Learn Go posts in the repo on GitHub.

{% github shindakun/atlg %}

***

| Enjoy this post? |
|:----:|
| [How about buying me a coffee?](https://ko-fi.com/shindakun) |

***