Repository: go
Issue: x/pkgsite/cmd/frontend: no longer loads stdlib in proxy mode
Labels: pkgsite,pkgsite/cmd

<!--
Please answer these questions before submitting your issue. Thanks!
-->

### What version of Go are you using (`go version`)?

<pre>
$ go version
go version go1.20.2 linux/amd64
</pre>

### Does this issue reproduce with the latest release?
Yes


### What operating system and processor architecture are you using (`go env`)?

<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/net_home/alatter/.cache/go-build"
GOENV="/net_home/alatter/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/net_home/alatter/go/pkg/mod"
GONOPROXY="*.<redacted>"
GONOSUMDB="*.<redacted>"
GOOS="linux"
GOPATH="/net_home/alatter/go"
GOPRIVATE="*.epic.com"
GOPROXY="<redacted>,direct"
GOROOT="/net_home/alatter/local/go1.20.2.linux-amd64"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/net_home/alatter/local/go1.20.2.linux-amd64/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/net_home/alatter/source/pkgsite/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1051915940=/tmp/go-build -gno-record-gcc-switches"

</pre></details>

### What did you do?

We run pkgsite and host it internally to provide documentation for our own packages. We run it in "direct" mode, and send it through our internal go-proxy (Athena), which has credentials to fetch our internal go-modules.

We run pkgsite with arguments like the following:

```
$ go run ./cmd/frontend -bypass_license_check -direct_proxy -proxy_url https://<redacted> -host :8080
```

After commit f159e616582641bdbeb2d6a0018ca4c8c472fbb5 we are no longer able to load godocs
for stdlib packages - for example, loading `http://localhost:8080/io` will return HTTP-500 with the following logs from pkgsite:

```
2023/09/26 15:11:41 Info: Listening on addr :8080
2023/09/26 15:11:46 Info: 0 /io map[requestType:request start]
2023/09/26 15:11:46 Info: FetchDataSource: fetching std@latest
2023/09/26 15:11:46 Info: FetchDataSource: fetched std@latest using *fetch.proxyModuleGetter in 226.3µs with error FetchModule("std", "latest"): proxy.Client.Info("std", "latest"): Client.readBody("std", "latest", "info"): Client.escapedURL("std", "latest", "info"): path: malformed module path "std": missing dot in first path element: invalid argument
2023/09/26 15:11:46 Error: serveUnitPage(ctx, w, r, ds, &{io std latest}): FetchDataSource.GetUnitMeta("io", "std", "latest"): FetchDataSource.findModule("io", "std", "latest"): FetchDataSource.getModule("std", "latest"): FetchModule("std", "latest"): proxy.Client.Info("std", "latest"): Client.readBody("std", "latest", "info"): Client.escapedURL("std", "latest", "info"): path: malformed module path "std": missing dot in first path element: invalid argument
2023/09/26 15:11:46 Info: 500 /io map[isRobot:false requestType:request end]
```

~~The error `malformed module path "std": missing dot in first path element: invalid argument` appears to be coming from
our go-proxy (Athena)~~. It is objected to a module named *std*, which seems like a reasonable thing to object-to.

(edit: on further testing the error is internal to pkgsite - the module for loading stdlib does not seem to be compatible with the direct-proxy mode)

If I swap the order of the getters used in *fetchdatasource.Options* in the direct-mode branch of frontend startup everything seems to work for me:

```go
	if *directProxy {
		sourceClient := source.NewClient(&http.Client{Transport: &ochttp.Transport{}, Timeout: 1 * time.Minute})
		ds := fetchdatasource.Options{
			Getters: []fetch.ModuleGetter{
				fetch.NewProxyModuleGetter(proxyClient, sourceClient), // <-- if I move this down one line my problem goes away
				fetch.NewStdlibZipModuleGetter(),
			},
			ProxyClientForLatest: proxyClient,
			BypassLicenseCheck:   *bypassLicenseCheck,
		}.New()
		dsg = func(context.Context) internal.DataSource { return ds }
	} else {

```

Bisect result:

```
f159e616582641bdbeb2d6a0018ca4c8c472fbb5 is the first bad commit
commit f159e616582641bdbeb2d6a0018ca4c8c472fbb5
Author: Michael Matloob <matloob@golang.org>
Date:   Wed Jul 5 14:02:46 2023 -0400

    internal/fetch: add two stdlib getters

    This breaks out the behavior of getting the stdlib into its own
    getter, stdlibZipModuleGetter. Most (but not all) of the special cases
    in the fetch code for the stdlib are in the stdlibZipModuleGetter,
    which is installed on the frontend. For the local pkgsite command, we
    add a new builder for the goPackagesModuleGetter that loads the
    packages from the stdlib given a GOROOT. The stdlibZipModuleGetter is
    also installed for pkgsite in as a fallback if the
    goPackagesModuleGetter doesn't successfully fetch the stdlib module
    (I'm not sure if that's possible).

    Fixes #60114

    Change-Id: Ida6a5367343643cc337c6d05e0d40095f79ca9e5
    Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/507883
    Reviewed-by: Jamal Carvalho <jamal@golang.org>
    TryBot-Result: Gopher Robot <gobot@golang.org>
    Run-TryBot: Michael Matloob <matloob@golang.org>
    kokoro-CI: kokoro <noreply+kokoro@google.com>
```

<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on go.dev/play is best.
-->



### What did you expect to see?



### What did you see instead?