What about doing this instead?

```go
info, exists := data[size]
if !exists {
    info = []os.FileInfo{}
}
data[size] = append(info, fileInfo)
```