Repository: go
Issue: cmd/go: mod tidy should join "require" sections if there are more than two
Labels: NeedsDecision,GoCommand,modules

Since https://github.com/golang/go/issues/45965, `go.mod` files now generally consist of two sections: one `require (...)` section with direct module dependencies, and one for `// indirect` module dependencies.

However, if a `go.mod` file somehow ends up with more than two of those sections, then `go mod tidy` does not join them back. I think it should. For example, if I join the normal two sections into one, `go mod tidy`, splits them again.

Here are two instances I recently found of such an unintentional "split" of sections:

* https://github.com/kortschak/gospel/blob/a5524a0364dead9e4d48f4952c2edff9007a9f09/go.mod
* https://github.com/vocdoni/vocdoni-node/pull/650#issuecomment-1294133282

Both were spotted manually and fixed, as they were indeed unintentional. Locally, I used `for f in **/go.mod; do n=$(grep -F 'require (' $f | wc -l); if [[ $n -gt 2 ]]; then echo $n $f; fi; done` with Bash to find any others. I couldn't find any.

Globally, I tried using cs.github.com, but it doesn't appear to support matching regular expressions across multiple lines, like PCRE. But Googe's code search does; https://cs.opensource.google/search?q=path%3Ago.mod+AND+pcre%3Ayes+AND+%28require%5C+%5C%28%28.%7C%5Cn%29*require%5C+%5C%28%28.%7C%5Cn%29*require%29+ found:

* https://cs.opensource.google/knative/eventing-rabbitmq/+/main:go.mod;drc=0932f28e1db80078d67fee4a3f9ecb62f2fff09e
* https://cs.opensource.google/knative/eventing-kafka-broker/+/main:go.mod;drc=efa69d6f58637c70e5f44e35728bb471fc30cf18

None of those four appear to be on purpose. I think they could have appeared unintentionally due to a number of reasons, such as:

1) Git merges. If two branches make changes to `go.mod`, depending on how the user resolves the conflicts manually, they could end up with more than two sections. I'm fairly sure that this is what happened in vocdoni-node, as [the PR in question](https://github.com/vocdoni/vocdoni-node/pull/529) lived for over a month and had conflicts to be resolved.
2) Manual editing. I've seen some users not fully grasp how `go get` works, and resorting to editing `go.mod` directly to update or add `require` lines. When doing it quickly or copy-pasting, I imagine it's tempting to just add a `require some-module v1.2.3` at the end of the file, which will work regardless of what the file looks like.

I personally can't think why anyone would want more than two `require` sections today. The fact that I could only find four examples today in ten minutes of research is a double-edged sword. On one hand it's proof that basically noone wants more than two sections. On the other, it also means that this problem appears rarely, so it might not be a high priority as an improvement for `go mod tidy`.

Still, I have encountered this problem myself twice now, and that's the magic number that tells me I should file a bug and propose an automated fix. This topic has been brought up a number of times on Slack ([January 2022](https://gophers.slack.com/archives/C9BMAAFFB/p1642695154098500), [February 2022](https://gophers.slack.com/archives/C9BMAAFFB/p1644227637391599), [October 2022](https://gophers.slack.com/archives/C9BMAAFFB/p1666908172925619)), so there are at least a couple of other people experiencing the problem and fixing it manually.

In the second of those Slack threads, @bcmills mentions that this might be a quirk with the `go mod tidy` upgrade from `go 1.16` to `go 1.17`. It could be that some of the third sections came about that way; it's hard to tell for sure. I'm a bit skeptical that the problem will go away with time, as `1.17` was released over a year ago and the extra sections still pop up. For example, that vocdoni PR was finalized in April 2022, and the `go.mod` file in master was already on `go 1.17` [since September 2021](https://github.com/vocdoni/vocdoni-node/commit/3f4d59e2fd93fcaa2ed018d49ba7ab29d4b737b3).

I think `go mod tidy` should join these extra sections. The only reason I see that as potentially risky is if, in the future, another proposal like https://github.com/golang/go/issues/45965 comes along and we want more than two sections. But presumably that shouldn't be a problem, because `go mod tidy` already complains if a `go.mod` file has a `go X.Y` version that's too new.

cc @bcmills @matloob @leitzler @seankhliao @dylan-bourque