Repository: go
Issue: x/tools/gopls: long initial workspace load durations for workspace with many packages
Labels: gopls,Tools,gopls/metadata

### gopls version

```
Build info
----------
golang.org/x/tools/gopls (devel)
    golang.org/x/tools/gopls@(devel)
    github.com/BurntSushi/toml@v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak=
    github.com/google/go-cmp@v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
    golang.org/x/exp/typeparams@v0.0.0-20221212164502-fae10dda9338 h1:2O2DON6y3XMJiQRAS1UWU+54aec2uopH3x7MAiqGW6Y=
    golang.org/x/mod@v0.21.0 h1:vvrHzRwRfVKSiLrG+d4FMl/Qi4ukBCE6kZlTUkDYRT0=
    golang.org/x/sync@v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ=
    golang.org/x/telemetry@v0.0.0-20240829154258-f29ab539cc98 h1:Wm3cG5X6sZ0RSVRc/H1/sciC4AT6HAKgLCSH2lbpR/c=
    golang.org/x/text@v0.18.0 h1:XvMDiNzPAl0jr17s6W9lcaIhGUfUORdGCNsuLmPG224=
    golang.org/x/tools@v0.21.1-0.20240508182429-e35e4ccd0d2d => ../
    golang.org/x/vuln@v1.0.4 h1:SP0mPeg2PmGCu03V+61EcQiOjmpri2XijexKdzv8Z1I=
    honnef.co/go/tools@v0.4.7 h1:9MDAWxMoSnB6QoSqiVr7P5mtkT9pOc1kSxchzPCnqJs=
    mvdan.cc/gofumpt@v0.7.0 h1:bg91ttqXmi9y2xawvkuMXyvAA/1ZGJqYAEGjXuP0JXU=
    mvdan.cc/xurls/v2@v2.5.0 h1:lyBNOm8Wo71UknhUs4QTFUNNMyxy2JEIaKKo0RWOh+8=
go: go1.23.1
```

### go env

```shell
$ bin/go env
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/rma/Library/Caches/go-build'
GOENV='/Users/rma/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/rma/go/pkg/mod'
GONOPROXY=<REDACTED>
GONOSUMDB=<REDACTED>
GOOS='darwin'
GOPATH='/Users/rma/go'
GOPRIVATE=<REDACTED>
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/rma/.cache/gocode/sdk/1.22'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/rma/.cache/gocode/sdk/1.22/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.6'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/rma/stripe/gocode/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/wc/_g6hd_8d3dnb87c04x67w0l40000gn/T/go-build681359692=/tmp/go-build -gno-record-gcc-switches -fno-common'
```

### What did you do?

My organization within the company is responsible for maintaining a large Go monorepo (~20k packages). In this monorepo, all packages fall under 1 Go module.

For the most part, we run a fairly off-the-shelf gopls setup without any customization of note. This is what our stats look like (from `bea7373d8a8268c2e3a260c1b8d41f96c4f7489e`):

```
$ gopls stats -anon
Initializing workspace...     done (3m26.109285333s)
Gathering bug reports...      done (1.781858458s)
Querying memstats...          done (1.284304292s)
Querying workspace stats...   done (701.076375ms)
Collecting directory info...  done (7.958301292s)
{
  "DirStats": {
    "Files": 288174,
    "TestdataFiles": 8243,
    "GoFiles": 77688,
    "ModFiles": 20,
    "Dirs": 64532
  },
  "GOARCH": "arm64",
  "GOOS": "darwin",
  "GOPACKAGESDRIVER": "",
  "GOPLSCACHE": "",
  "GoVersion": "go1.23.1",
  "GoplsVersion": "(devel)",
  "InitialWorkspaceLoadDuration": "3m26.109285333s",
  "MemStats": {
    "HeapAlloc": 2970803424,
    "HeapInUse": 4669145088,
    "TotalAlloc": 36820682064
  },
  "WorkspaceStats": {
    "Files": {
      "Total": 72996,
      "Largest": 9189430,
      "Errs": 0
    },
    "Views": [
      {
        "GoCommandVersion": "go1.22.0",
        "AllPackages": {
          "Packages": 23170,
          "LargestPackage": 624,
          "CompiledGoFiles": 89502,
          "Modules": 1031
        },
        "WorkspacePackages": {
          "Packages": 18127,
          "LargestPackage": 354,
          "CompiledGoFiles": 61173,
          "Modules": 1
        },
        "Diagnostics": 95
      }
    ]
  }
}
```

### What did you see happen?

The experience of using gopls is good (i.e. rpcs like jump-to-definition, refactor symbol, etc. are fast) once initial workspace load has occurred, but initial workspace loads are a poor experience for our users. This poor experience is owed to two behaviors:

Firstly, initial workspace loads (IWLs) are slow. This is partially owed to the performance of the expensive upfront `go list` call itself:

```
$ time go list -e -json=Name,ImportPath,Error,Dir,GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,SwigFiles,SwigCXXFiles,SysoFiles,TestGoFiles,XTestGoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap,TestImports,XTestImports,ForTest,DepsErrors,Module,EmbedFiles -compiled=true -test=true -export=false -deps=true -find=false -pgo=off -- /Users/rma/stripe/gocode/... builtin
[ ...A bunch of redacted `go list` output... ]
go list -e  -compiled=true -test=true -export=false -deps=true -find=false  -  31.41s user 48.00s system 120% cpu 1:06.01 total
```

…but the majority of the time is spent on calling `packages.Load` on all of the packages that `go list` returns.

Secondly, because [IWLs are blocking](https://github.com/golang/tools/blob/bea7373d8a8268c2e3a260c1b8d41f96c4f7489e/gopls/internal/cache/snapshot.go#L1331-L1333) and VSCode waits on code actions before the IDE is able to save files, users are unable to save files/exit their IDEs/etc:

> ```Saving 'main.go': Getting code actions from ''Go', 'ESLint', 'GitHub Copilot Chat'' (configure).```

…while IWL is incomplete. This is partially a corollary of the first issue. While it is true that making code actions non-blocking for saves would fix this UX issue, if IWL were fast, this UX issue would not be perceived by users in the first place.

We have dipped our toes in trying to patch this behavior. For example, an early attempt at hacking around this behavior (on a `v1.16.1` base) [no-ops the initialization routine altogether](https://gist.github.com/rma-stripe/c9c5573025209389678af3a9755137e6), relying solely on gopls loading in package data as files are manually opened by the user. As expected, this hack has some large tradeoffs associated with it, because initialization is a load bearing part of gopls: it breaks features like "Rename symbol" and "Find all references", and it occasionally causes the wrong imports to be pulled in. However, buggy as it is, this hack in essence describes what we think the desirable IWL behavior would be:

* The initial snapshot at gopls startup does not await IWL
* On startup, an async IWL routine runs `go list` and lazy loads package data into the currently "active" snapshot
* While this async routine has not completed, language server RPCs which rely on all workspace packages being loaded (such as "Find all references" are not available)

### What did you expect to see?

This issue is a feature request.

* Does the newly proposed workspace initialization approach align with the gopls roadmap?
* Is there a better approach (in the interim) to addressing the slow/blocking behavior of IWLs than the no-op/hack solution linked above?

We would appreciate any tips/guidance in writing an upstream-able patch, or a better hack.


### Editor and settings

_No response_

### Logs

_No response_