Test in Go: trick of Cleanup
Tags: go, testing, unittest
Published: 2024-02-18 19:34:12

My team is using [testify](https://pkg.go.dev/github.com/stretchr/testify) as the testing framework for Go. We noticed that many methods provided by [testify/suite](https://pkg.go.dev/github.com/stretchr/testify/suite) can be easily achieved by the built-in [testing](https://pkg.go.dev/testing) package. For example, using [Cleanup](https://pkg.go.dev/testing#T.Cleanup) can do the same of [AfterTest](https://pkg.go.dev/github.com/stretchr/testify/suite#AfterTest) provided by `testify/suite`. It makes me wondering if we can just use `Cleanup` to replace as much methods from third-party packages as possible.

## TL;DR

`Cleanup` can replace methods that are not relevant to subtests, which are `AfterTest`, `TearDownAllSuite` and `TearDownTestSuite`.

`Cleanup` can’t replace `TearDownSubTest` because there is a risk that a subtest pollute its parent test.

## The experiment

I use a global variable `externalResource` as  the external resource. It’s value is `1`.

When preparing a subtest at `SetupSubTest`, `externalResource` is changed to `2`. At the same time, a cleanup function is registered to restore `externalResource` to `1`.

I expect that `externalResource` is set to `2` before the subtest and restored to `1` once the subtest is finished. So, from the point of parent test, `externalResource` is always `1`. From the point of subtest, `externalResource` is always `2`.

The code is

```go
var externalResource = 1

type TestUnexpectedCleanupOrder struct {
	suite.Suite
}

func (s *TestUnexpectedCleanupOrder) TearDownTest() {
	fmt.Println("TearDownTest")
}

func (s *TestUnexpectedCleanupOrder) SetupSubTest() {
	externalResource = 2
	fmt.Printf("SetupSubTest. The externalResoure is: %d\n", externalResource)
	s.T().Cleanup(func() {
		externalResource = 1
		fmt.Printf("SetupSubTest. The externalResoure is: %d\n", externalResource)
	})
}

// Test case
func (s *TestUnexpectedCleanupOrder) TestCaseA() {
	fmt.Printf("Start TestCase A. The externalResoure is: %d\n", externalResource)

	s.Run("Sub-TestCase A", func() {
		fmt.Printf("Finish Sub-TestCase A. The externalResoure is: %d\n", externalResource)
	})
	fmt.Printf("Finish TestCase A. The externalResource is: %d\n", externalResource)
}
```

## The result

The output of the above code is (with line number)

```
1 Start TestCase A. The externalResoure is: 1
2 	SetupSubTest. The externalResoure is: 2
3 	Finish Sub-TestCase A. The externalResoure is: 2
4 Back to TestCase A. The externalResource is: 2
5 TearDownTest
6 	Cleanup subTest. The externalResoure is: 1
```

Notice line 4 and line 6. The `externalResource` is still `2` when the subtest is finished and the parent test is resuming! `externalResource` is not cleaned until the parent test is finished.

According to the documentation of `Cleanup`:

> Cleanup registers a function to be called when the test (or subtest) and all its subtests complete. Cleanup functions will be called in last added, first called order.
> 

It only guarantee that a registered function is called **after** something.

It does **NOT** guarantee when a registered function is called **before** something.

## The solution

If we add a `TearDownSubTest` to reset `externalResource`, then its value is cleaned as we need

```go
func (s *TestUnexpectedCleanupOrder) TearDownSubTest() {
	externalResource = 1
	fmt.Printf("\tTeardown subTest. The externalResoure is: %d\n", externalResource)
}
```

```
Start TestCase A. The externalResoure is: 1
	SetupSubTest. The externalResoure is: 2
	Finish Sub-TestCase A. The externalResoure is: 2
	Teardown subTest. The externalResoure is: 1
Back to TestCase A. The externalResource is: 1
TearDownTest
	Cleanup subTest. The externalResoure is: 1
```

## Summary

We can use `Cleanup` to clean up resources for normal tests. However, we need to use `TearDownSubTest` to clean the resource held by subtests.

## The source code

[https://github.com/xuanyuwang/testify-examples/tree/main/unexpected-cleanup-order](https://github.com/xuanyuwang/testify-examples/tree/main/unexpected-cleanup-order)