What is the io.Reader in Go?
Tags: go, codenewbie
Published: 2020-03-01 19:12:06

---
title: What is the io.Reader in Go?
published: true
description: 
tags: #go, #codenewbie
---

If you write Go programs, you'll come across the `io.Reader` interface quite a bit. It's used all over the place to represent reading data from lots of different sources, but its type definition is actually pretty humble.

```go
type Reader interface {
    Read(p []byte) (n int, err error)
}
```

Just one method! Which means if you make a type with a method called Read, and that method takes in a byte slice and returns an int plus any error that occurs, your type is an `io.Reader`. But what is the idea of this interface, and how do we use it in our code?

## Taking a look at the function signature

`Reader` is a little interface with a strong abstraction! What is that abstraction exactly? The idea with the `Read` method is that it represents reading bytes of data from **some source**, so that we can then use those bytes in our code. That source could be files, cameras, network connections, or just a plain old string. For example, if we're reading data from files, the `io.Reader` we would use is a `*os.File`.

Before we try this out, let's take a look at the parts of the function signature for `Reader.Read`:

```go
Read(p []byte) (n int, err error)
```

* `p []byte` is a byte slice we pass into the Read method. The Reader copies the data it reads from its data source (like a file) over to that byte slice.
* The returned `n int` tells you how many bytes we read in this call to `Read`.
* The returned `err error` is any error that might happen reading the data, like reaching the end of a file.

Let's give this a try with a file and see how we use it. Copy this text and save it to a file in a new directory called `sloth.txt`:

```
I'm a sloth and hibiscus flowers are tasty!
```

Now, let's write some Go code to read it using a File's Read method. Copy this code and save it to a file `main.go` in the same directory as sloth.txt:

```go
package main

import (
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("sloth.txt")
    if err != nil {
        log.Fatalf("error opening sloth.txt: %v", err)
    }
    defer file.Close()

    // Make a byte slice that's big enough to store a few words of the message
    // we're reading
    bytesRead := make([]byte, 33)

    // Now read some data, passing in our byte slice
    n, err := file.Read(bytesRead)
    if err != nil {
        log.Fatalf("error reading from sloth.txt: %v", err)
    }

    // Take a look at the bytes we copied into the byte slice
	log.Printf("We read \"%s\" into bytesRead (%d bytes)",
		string(bytesRead), n)
}
```

Run this with `go run main.go` and the output you should be getting is:

```
We read "I'm a sloth and hibiscus flowers " (33 bytes) into bytesRead
```

So what happens when we call `os.File.Read` is, Go reads data from the file, copying it to `bytesRead`, and since no error happened, the error returned is nil. We're passing our bytesRead slice into the `Read` function like we pass a cup into a soda fountain!

![My Gopher plushies, Gostavo and Gophelia, putting a cup under the Slurpee machine at 7-Eleven. They're reading slushies into the cup!](https://dev-to-uploads.s3.amazonaws.com/i/mmnrqjs4yj4kv4w4kk5w.jpeg)

So that's how we get our bytes, but why are we even returning how many bytes we read?

To see why, let's see what happens if we do a second call to `Read`. We want to see this sloth's opinion on hibiscus flowers! Add this to the end of the main function:

```go
    n, err = file.Read(bytesRead)
    if err != nil {
        log.Fatalf("error reading from sloth.txt: %v", err)
    }

    log.Printf("We read \"%s\" into bytesRead (%d bytes)",
        string(bytesRead), n)
```

Now the output to `go run main.go` should be:

```
2020/03/01 13:38:50 We read "I'm a sloth and hibiscus flowers " into bytesRead (33 bytes)
2020/03/01 13:38:50 We read "are tasty!
 and hibiscus flowers " into bytesRead (11 bytes)
```

As expected we read in that our sloth indeed loves hibiscus flowers, but why does the third line of our output say ` and hibiscus flowers`?

The reason is that while our Reader copied the 11 remaining bytes from the file to the beginning our `bytesRead` slice, it didn't touch the rest of the slice. The rest of the slice contains the bytes we read on the **previous call to `Read`**!

![Bytes being copied in the second call to file.Read. There are 11 bytes to read, so the part of the bytesRead slice saying "I'm a sloth" is replaced with the sloth's opinion on hibscus flowers, "are tasty!" pus a newline. Meanwhile, remaining 22 bytes of the slice saying "and hibiscus flowers", which we read in on the last call to file.Read stay the same since we don't have any more data to copy to that part of the slicd.](https://dev-to-uploads.s3.amazonaws.com/i/mol8op3w5hw5ukg18mq2.jpeg)

So what the returned integer `n` does for us, is it tells us **which** bytes in the byte slice contain the content we read. That way, code working with those bytes will know which bytes in the slice it should be paying attention to, and which bytes it shouldn't pay attention to.

## Handling errors with io.Reader

Let's call `file.Read` one more time:

```go
    n, err = file.Read(bytesRead)
    if err == io.EOF {
        log.Printf("End of file reached; we got %d bytes on the last read", n)
    } else {
        log.Fatalf("unexpected error reading from the file: %v\n", err)
    }
```

Run this with `go run main.go`, and the final output will be something like:

```
2020/03/01 13:44:35 We read "I'm a sloth and hibiscus flowers " into bytesRead (33 bytes)
2020/03/01 13:44:35 We read "are tasty!
 and hibiscus flowers " into bytesRead (11 bytes)
2020/03/01 13:44:35 End of file reached; we got 0 bytes on the last read
```

The error we got is [`io.EOF`](https://golang.org/pkg/io/#pkg-variables), which stands for "end of file", and that error is one we generally expect to get. When you encounter EOF, typically what you do is just have your code work with any bytes that were read in, and then stop reading data from that io.Reader since there isn't any data left.

However, EOF isn't the only kind of error that `io.Reader.Read` can return. If we get an error besides EOF, typically that means either there's a bug in our code, or something unexpected happened. For example, try copying this code to `close-early.go` that reads from a file we already closed:

```go
package main

import (
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("sloth.txt")
    if err != nil {
        log.Fatalf("error opening sloth.txt: %v", err)
    }
    file.Close()

    bytesRead := make([]byte, 0, 33)
    if _, err := file.Read(bytesRead); err != nil {
        log.Fatalf("error reading from sloth.txt: %#v", err)
    }
}
```

If you run that with `go run close-early.go`, you should get:

```
2020/03/01 13:45:45 error reading from sloth.txt: &os.PathError{Op:"read", Path:"sloth.txt",
  Err:(*errors.errorString)(0xc000010110)}
exit status 1
```

An `os.PathError` because we're trying to read from a file we already closed!

So in general, if you're running `io.Reader.Read`, there are three cases to work with:

* ✅ If we get a nil error, we have our code work with the bytes we copied from the Reader to `bytesRead[:n]`, and can keep reading more data.
* 🛑 If we get `io.EOF`, we should stop reading data, and if any bytes were copied to `bytesRead`, then we have your code work with those last `n` bytes in `bytesRead[:n]`
* ⚠️ If we get a different kind of error, it's probably unexpected. If any bytes were read in, then have your code work with them, then handle the error with whatever error handling logic you see fit.

Notice that even if we got a non-nil error, we might have still successfully read in some bytes our code we can work with, so the code working with `io.Reader` typically should still be processing those bytes.

## Building on top of io.Reader

While `io.Reader.Read` gives us a generalized way to read from a lot of data sources, in a Go project, often you won't be calling `Read` directly; instead we have code that builds on top of that method. Some of those are:

* If you want to read all the bytes from a source into one long byte slice, you can pass your Reader into `ioutil.ReadAll`!
* If you want to read a file line-by-line, you can have a `bufio.Scanner` read in a line of data with its `Scan` method, and the Scanner will handle looking for newlines so your code doesn't have to think about that for you!
* Working with image data? The image package will help you decode your image into a fancy `image.Image` interface!
* Deserializing data from JSON for a web app? `json.Decoder.Decode`'s got the JSON logic covered!

Why use this code instead of just `Read`? Because the `io.Reader` is a low-level interface; its job is to read data from a source, and copy it over to a byte slice, but it's not concerned with **what** bytes you read in. Actually working with those bytes is handled by the **code that calls Read**.

To see this in action, let's try deserializing some JSON! Save this Go code to `sloth-facts.go`:

```go
package main

import (
    "encoding/json"
    "fmt"
    "strings"
)

type AnimalFacts struct {
	CommonName     string   `json:"commonName"`
	ScientificName string   `json:"scientificName"`
	HeightInInches int      `json:"heightInInches"`
	FavoriteFoods  []string `json:"favoriteFoods"`
	CanSwim        bool     `json:"canSwim"`
}

func main() {
	jsonData := `
{
    "commonName":     "brown-throated three-toed sloth",
    "scientificName": "Bradypus variegatus",
    "heightInInches": 31,
    "favoriteFoods":  ["Cecropia leaves", "Hibiscus flowers"],
    "canSwim":        true
}`

	// an io.Reader that isn't a file!
	rdr := strings.NewReader(jsonData)

	var sloth AnimalFacts
	if err := json.NewDecoder(rdr).Decode(&sloth); err != nil {
		fmt.Printf("error deserializing JSON: %v", err)
		return
	}

	fmt.Printf("Hi! I'm a %s (%s)! I'm about %d\" tall, and love eating %v!\n",
		sloth.CommonName, sloth.ScientificName, sloth.HeightInInches,
		sloth.FavoriteFoods)
	if sloth.CanSwim {
		fmt.Println("By the way, I can swim!")
	}
}
```

Run this with `go run sloth-facts.go` and you should get this output:

```
Hi! I'm a brown-throated three-toed sloth (Bradypus variegatus)! I'm about 31"
tall, and love eating [Cecropia leaves Hibiscus flowers]!
By the way, I can swim
```

What's happening here?

1. We take our JSON data from a string, and make an `io.Reader` with it. But instead of using a file as our io.Reader, we're making a `strings.Reader`, which lets us read data from a plain old string!
2. `json.NewDecoder` takes in an `io.Reader`, so we pass our strings.Reader into that to set up a `json.Decoder`, a struct that reads JSON data from our Reader, into Go variables.
3. We pass a pointer to our `AnimalFacts` struct into `json.Decoder.Decode`, and our JSON data gets read into that struct!
4. We print our data out in a cute sloth facts message, ready for a zookeeper to put on the sign at the sloth exhibit!

If we had read our JSON data with plain calls to `io.Reader.Read`, we would have to keep track of curly braces and commas, which field of the JSON we were currently on, and more. But with `json.Decoder.Decode`, the Go Team has already handled thinking that through for us, building JSON decoding logic on top of `Read`!

As you can see, `io.Reader` is a really versatile interface! And that's why in Go code, you'll see `io.Reader`s everywhere, and you can use code that works with the `Read` method to work with any data format, an awesome Go interface abstraction!