Repository: go
Issue: x/tools/gopls: decide on a long-term testing strategy for integration with older Go commands (and GOPACKAGESDRIVERs?)
Labels: gopls,Tools

As described in #65917, it has been a multi-year journey to finally be able to ensure that gopls is only (and need only be) built with the latest version of Go.

With that achieved, I think the following experience will still be common for our users:

- The user runs `go install golang.org/x/tools/gopls@latest` with go 1.21.5.
- The go command performs a toolchain switch to 1.23.1, and installs gopls.
- gopls starts, and finds go 1.21.5 in its PATH.

This means that while gopls is build with 1.23.1, it will still frequently need<sup>1</sup> to integrate with `go list` for older go versions such as 1.21.5. For example, https://telemetry.go.dev still shows 62% of gopls users on 1.22 or older, and 16% on 1.21 or older.

Right now, we continue to have test coverage for this integration, by virtue of our 1.21 and 1.22 TryBots, because they set `GOTOOLCHAIN=auto`, and because the gopls integration tests run on temporary directories outside of gopls' own 1.23.1 module. Therefore, the tests will be built with 1.23.1, but still integrate with the ambient go version.

However, in https://go.dev/cl/605000 the question came up about whether we should continue to maintain "legacy" go version builders for gopls -- at this point Go 1.21 is no longer a supported Go version, and so gopls' [support](https://github.com/golang/tools/tree/master/gopls#support-policy) of integrating with the last three Go versions (recently narrowed from 4) is inconsistent with that policy.

IMO, testing that gopls integrates with Go 1.21 is not quite the same as supporting 1.21. After all, we have at times considered running tests against other go/packages drivers, such as for Bazel, and doing so would of course not be the same as supporting Bazel itself. Put differently: at this point Go 1.21 is not going to change, so if an integration test starts failing it will be due to a change in gopls. We may want to avoid making a change in gopls that breaks users integrating with Go 1.21.

Therefore, we have at least a few options:
1. Stop supporting integration with older Go versions, and take a hard stance that users should upgrade to a supported version of Go, perhaps by popping up a message warning users of the lack of support.
2. Stop supporting integration with older Go versions, but remain silent when users try to use gopls in this way.
3. Ask the release team to continue maintaining support for legacy TryBots.
4. Something else (see below...)

Regarding (1), I think this seems too severe. (2) would be very similar to our approach with other drivers such as Bazel, yet we *have* inadvertently caused problems for Bazel users in the past, and I think we can do better. (3) seems like a non-starter, since it burdens the rest of the team.

**Here is a tentative proposal for "something else" (4):**

Using our [existing mechanisms](https://cs.opensource.google/go/x/tools/+/master:gopls/internal/test/integration/misc/shared_test.go;l=30;drc=594cdabebbd503e8d20eeb03faf99d3c1cabef6e) for running tests in various environments, we can "opt in" (or opt out) certain tests to run with older Go commands, on the longtest builder. In legacy mode, these tests would configure the Go command to run with GOTOOLCHAIN=<legacy go version>.

Then we will opt in integration tests that exercise "basic" functionality of gopls: loading a workspace, processing metadata-affecting changes, and basic features like diagnostics, navigation, and completion. Since longtest builders do not run as a presubmit, failures of these tests with older Go versions will not block changes to gopls. If tests start failing, we can decide whether to revert the gopls change, or opt-out the test.

In this way, we can at least detect when gopls may break users with older Go versions, and be deliberate about when this is acceptable.

CC @golang/tools-team @golang/release 

<sup>1</sup> while we could perhaps avoid this by setting GOTOOLCHAIN=1.23.1 before invoking `go list` from gopls, we have a general philosophical stance that it is least surprising if gopls's view of the workspace is consistent with the user's experience running the go command from a terminal).