Repository: go
Issue: proposal: cmd/go/v2: prohibit in-package tests from extending a package's API
Labels: v2,Proposal

Currently in-package tests are free to add exported identifiers to the package that they are testing.
Specifically an in-package test:

 - can define new variables
 - can define new types
 - can implement methods on non-test types

This makes tooling harder, because any given package can have several possible variants,
depending on which package's tests are being run.

Consider the following example:

	-- go.mod --
	module example

	-- a/a.go --
	package a
	
	import (
		"fmt"

		"example/b"
	)
	
	var A fmt.Stringer = b.B{}

	-- b/b.go --
	package b
	
	type B struct{}
	
	-- b/b1_test.go --
	package b
	
	func (B) String() string { return "b" }

	-- b/b2_test.go --
	package b_test
	
	import (
		"testing"

		"example/a"
	)
	
	func Test(t *testing.T) {
		if got, want := a.A.String(), "b"; got != want {
			t.Fatalf("got %q want %q", got, want)
		}
	}

Running `go test example/b` succeeds because when b is being tested, the internal tests extended the B type so it implements fmt.Stringer. But `example/a` won't actually compile on its own because B doesn't implement fmt.Stringer without the test code added by the internal tests in `example/b`!

It's as if tests exist in a closely related but different version of the universe in which every package might not be *quite the same* as its usual version. And that version is potentially different for every package.

This "parallel universe" property of testing makes Go types harder to reason about and the Go tooling less efficient, because it's not possible to type-check a package once and for all in the general case.

## Where does this problem come from?

Currently in Go, test code can be internal to a package, extending that package for the duration of the test and allowing the test code access to unexported identifiers within the package, or it can be external, with access to exported identifiers only. A hybrid approach is also possible, where tests are largely external, but some package-internal test code, often in an `export_test.go` file, is used to provide access to selected parts of the internal API.

It's the final case that is problematic, because external test code is free to depend on other packages which themselves depend on the package being tested which has been extended with extra test code.

All three approaches are common in existing Go code.

I believe that any solution to this issue should continue to support almost all existing code (including the possibility of automatic code migration with `gofix`), otherwise substantial parts of the ecosystem will not be able to move forward with new Go features.

## Proposed solution

I propose that it should not be possible to write tests in the same package as that being tested. That is, all tests must be in the external test package (e.g. `foo_test` to test package `foo`).

A test file can use a special form of import statement to request access to unexported identifiers from the package being tested. If they use this special form, code within that file (only) may access unexported identifiers within the package without compiler errors.

A possible (straw man) spelling for the syntax is:

	import "."

That is, import the package in the current directory.

This form is currently invalid because relative import paths are not allowed, and "." is not a valid package name by itself, so wouldn't overlap with existing import path syntax. As there is only one package that can be imported in this way, there is no ambiguity problem. It also satisfies [issue 29036](https://golang.org/issue/29036), as the imported package name can be automatically derived from the current file's package identifier by stripping the `_test` suffix.

Another possibility might be to add an extra keyword, recognized only within an import block; for example:

	import "mypackagepath" internal

Whatever the form for the special import syntax, this solution seems to solve the initial problem. It allows both all-internal tests (always use `import "."`), all-external tests (never use `import "."`) and hybrid tests ("internal" test code can define selected functionality to be used by external tests, similarly to how `export_test.go` files are used today). It also reduces the overall complexity of the way that tests work in Go.

I believe that it should also be possible to automatically convert almost all existing code to the new rules. Some care would need to be taken to rename symbols that appear in both internal and external test code, but that's not an hard issue to solve. Perhaps that issue is rare enough that manual intervention could be required. More investigation is needed to see how common it is.

As an example, some test code that exposes selected functionality to external tests might look like this:

	-- b/b.go --
	package b

	type B struct {
		internalField string
	}
	
	func GetBValue() B {
		return B{
			internalField: "b",
		}
	}
	-- b/b1_test.go --
	package b_test
	import "."		// allow access to unexported identifiers within this file.
	
	func BInternalField(x b.B) string {
		return x.internalField
	}
	-- b/b2_test.go --
	package b_test
	
	import (
		"testing"

		"example/b"
	)
	
	func Test(t *testing.T) {
		x := b.GetBValue()
		if got, want := BInternalField(x), "b"; got != want {
			t.Fatalf("got %q want %q", got, want)
		}
	}


## Other possible solutions

We could disallow all tests that rely on direct access to unexported identifiers i.e. allow external tests only. This is an attractive option for some, but the change would be very disruptive for much code. I do not believe that it would be possible to migrate existing internal tests automatically, so I think this possibility is excluded.

We could continue to allow both internal and external tests, but treat internal test code as being in its own package, with access allowed to the tested package's unexported identifiers (and all symbols from the package available in global scope), but otherwise separate from it. External tests could use some kind of special import syntax (for example `import "thispackage@test"`) to access symbols defined in the internal tests. This was my original thought, but the model seems more complex than necessary - we have an extra pseudo-package for tests *and* a special import path syntax.

We could prohibit internal test code from defining methods on types defined by the testing package. This solves some of the more obvious problems with the current situation, but the "parallel universe" issue is still present, and tooling probably would not become significantly simpler.