Repository: go
Issue: x/tools/go/packages: overlay does not work properly with external module files
Labels: NeedsInvestigation,Tools

Consider:

```go
package main

import (
	"fmt"

	"golang.org/x/tools/go/packages"
)

func main() {
	fmt.Println(run())
}

func run() error {
	cfg := packages.Config{
		Mode: packages.LoadSyntax,
		Dir:  ".",
		Overlay: map[string][]byte{
			"/home/mateusz/go/pkg/mod/golang.org/x/tools@v0.28.0/go/packages/nonExistent.go": []byte(`package packages; func TestFunc() {}`),  // New file
			"/home/mateusz/go/pkg/mod/golang.org/x/tools@v0.28.0/go/packages/doc.go":         []byte(`package packages; func TestFunc2() {}`), // File exists, but overridden.

			// File exists, but overridden, additional import (unicode/utf8).
			"/home/mateusz/go/pkg/mod/golang.org/x/tools@v0.28.0/go/packages/loadmode_string.go": []byte(`// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package packages

import (
	"fmt"
	"strings"
	"unicode/utf8"
)

func TestFunc3(r rune) []byte {
	return utf8.AppendRune(nil, r)
}

var modes = [...]struct {
	mode LoadMode
	name string
}{
	{NeedName, "NeedName"},
	{NeedFiles, "NeedFiles"},
	{NeedCompiledGoFiles, "NeedCompiledGoFiles"},
	{NeedImports, "NeedImports"},
	{NeedDeps, "NeedDeps"},
	{NeedExportFile, "NeedExportFile"},
	{NeedTypes, "NeedTypes"},
	{NeedSyntax, "NeedSyntax"},
	{NeedTypesInfo, "NeedTypesInfo"},
	{NeedTypesSizes, "NeedTypesSizes"},
	{NeedForTest, "NeedForTest"},
	{NeedModule, "NeedModule"},
	{NeedEmbedFiles, "NeedEmbedFiles"},
	{NeedEmbedPatterns, "NeedEmbedPatterns"},
}

func (mode LoadMode) String() string {
	if mode == 0 {
		return "LoadMode(0)"
	}
	var out []string
	// named bits
	for _, item := range modes {
		if (mode & item.mode) != 0 {
			mode ^= item.mode
			out = append(out, item.name)
		}
	}
	// unnamed residue
	if mode != 0 {
		if out == nil {
			return fmt.Sprintf("LoadMode(%#x)", int(mode))
		}
		out = append(out, fmt.Sprintf("%#x", int(mode)))
	}
	if len(out) == 1 {
		return out[0]
	}
	return "(" + strings.Join(out, "|") + ")"
}
`),
		},
	}
	pkgs, err := packages.Load(&cfg, "./test")
	if err != nil {
		return err
	}
	packages.PrintErrors(pkgs)
	return nil
}
```

This program runs `packages.Load` on a `./test` package, with three overlay files on the `golang.org/x/tools/go/packages` package.

- `nonExistent.go` - File does not exist, so this is a new file with a new func `TestFunc`.
- `doc.go` - File already exists, it defines a new func `TestFunc2`. 
- `loadmode_string.go` - File already exists, it defines a new func `TestFunc3` and imports `unicode/utf8` (this package is not imported without overlays).

The `./test` package contains following file:

```go
package main

import (
	"golang.org/x/tools/go/packages"
)

func main() {
	packages.TestFunc()
	packages.TestFunc2()
	packages.TestFunc3('a')
}
```

```bash
$ go run .
-: # golang.org/x/tools/go/packages
/tmp/gocommand-708375563/3-loadmode_string.go:10:2: could not import unicode/utf8 (open : no such file or directory)
/home/mateusz/go/pkg/mod/golang.org/x/tools@v0.28.0/go/packages/loadmode_string.go:10:2: could not import unicode/utf8 (no metadata for unicode/utf8)
/tmp/aa/test/test.go:8:11: undefined: packages.TestFunc
<nil>
```

So:

- It is not able to import `unicode/utf8` (probably `go list` does not return it to `go/packages`).
- Newly added files are ignored (`nonExistent.go`).
- Overlay works, but only for files that already exist.

Opened this issue, because i think that this behavior is wrong, it should either work without any error for the example above, or just ignore the overlay completely.

CC @matloob (per https://dev.golang.org/owners) @adonovan