Repository: go
Issue: cmd/go: consider making "tool pprof" work with the module cache
Labels: NeedsInvestigation

I was just debugging a server's large memory usage, so via its http pprof endpoints, I downloaded some profiles.

The binaries running on that server were built with `-trimprefix`, because it's a sane default for "production builds", and enables reproducible builds too.

Unfortunately that means that, if I grab the deployed binary and some of its profiles, trying to use `go tool pprof` mostly works, but trying to inspect any source code fails:

```
$ go tool pprof dvotenode heap
[...]
(pprof) list NewCache
Total: 2.20GB
ROUTINE ======================== github.com/dgraph-io/ristretto.NewCache in github.com/dgraph-io/ristretto@v0.1.0/cache.go
    2.41MB    21.05MB (flat, cum)  0.94% of Total
 Error: could not find file github.com/dgraph-io/ristretto@v0.1.0/cache.go on path /home/mvdan
```

Adding `-source_path=$(go env GOMODCACHE)` fixes those cases:

```
(pprof) list NewCache
Total: 2.20GB
ROUTINE ======================== github.com/dgraph-io/ristretto.NewCache in github.com/dgraph-io/ristretto@v0.1.0/cache.go
    2.41MB    21.05MB (flat, cum)  0.94% of Total
         .          .    164:	case config.MaxCost == 0:
         .          .    165:		return nil, errors.New("MaxCost can't be zero")
         .          .    166:	case config.BufferItems == 0:
         .          .    167:		return nil, errors.New("BufferItems can't be zero")
         .          .    168:	}
         .    18.15MB    169:	policy := newPolicy(config.NumCounters, config.MaxCost)
         .          .    170:	cache := &Cache{
[...]
```

This won't cover quite all cases, because the main module lacks a version, and might not even be in the module cache if I built from a `git clone`. Note the lack of `@module-version` in the error output, unlike my previous example involving a dependency:

```
(pprof) list LoadTree
Total: 2.20GB
ROUTINE ======================== go.vocdoni.io/dvote/census.(*Manager).LoadTree in go.vocdoni.io/dvote/census/census.go
         0     2.13GB (flat, cum) 97.15% of Total
 Error: could not find file go.vocdoni.io/dvote/census/census.go on path /home/mvdan/go/pkg/mod
```

I still think it would be neat for `go tool pprof` to come with a built-in `-source_path=$(go env GOMODCACHE)` default. As far as I can tell, it can't hurt. It won't be perfect with handling the main module either, but that's okay.

One alternative I do have, which will support listing all function sources, is to rebuild the same program with the same build flags but without `-trimpath`. Arguably that's the better solution if I want to be able to inspect all the source. However, that takes extra steps, and sometimes it might be difficult to figure out how to build a slight variant of exactly the same program at exactly the same version. So the sane(r) default that I propose here would probably be enough to perform a lot of `list` commands.

Yet another question is how I could tell `go tool pprof` that all the `go.vocdoni.io/dvote` module sources are in the directory `/home/mvdan/src/dvote`. I don't think `-source_path` is powerful enough for this right now, as I don't seem to be able to supply two paths. This also seems worth investigating in terms of UX, but it does seem like a separate idea.

cc @hyangah @aalexand @cherrymui