GoPV: Detecting Blocking Concurrency Bugs Related to Shared-Memory Synchronization in Go
Creators: name: Song, Wei
name: Xu, Xiaofan
name: Huang, Jeff
Publisher: name: Zenodo
Year: 2025

description: # GoPV, a tool for detecting blocking concurrency bugs related to shared-memory synchronization in Go

 

## Contents

- [Environment](#environment)

- [Getting Started Guide](#getting-started-guide)

- [Verification](#verification)

- [Troubleshooting](#troubleshooting)

- [Comparison tools](#comparison-tools)

- [goleak](#goleak)

- [GCatch](#gcatch)

   - [GOAT](#goat)

   - [GoPie](#gopie)

   - [GFuzz](#gfuzz)

 

## Environment

 

- Ubuntu 20.04+

- go1.23.1, you could download it at https://golang.google.cn/. Then you should configure GOPATH and environment variables. 

- GoLand-2024.2.1(IDE, not necessary). If you are used to using an IDE, you could download it at https://www.jetbrains.com/go/.

 

## Getting Started Guide

 

- All you need to run GoPV is Go environment. So the first step is to install and configure Go environment. Suppose you install golang in the */usr/local* directory of ubuntu and create a directory named *goProject* in */usr/local* directory as GOPATH. Then you should create three directory *bin, pkg and src* under the GOPATH directory */usr/local*. After that, you should add the following information to the *~/.bashrc* file to configure Go environment variables. You can use the command `vim ~/.bashrc` to open and modify the file.

 

  ```

  #The absolute path of GOROOT

  export GOROOT=/usr/local/go

  #The absolute path of GOPATH

  export GOPATH=/usr/local/goProject

  export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

  ```

 

  Then you could use the command `source ~/.bashrc` to make the configuration effective.

 

- After installing Go and configuring Go environment successfully, you should place GoPV in your *GOPATH/src* directory. Then, follow these steps to use GoPV.

 

    1. Edit the install.sh in GoPV, setting *CURDIR* and *GOPATH* as your local absolute paths.

 

    2. Edit the run.sh in GoPV, setting *CURDIR* and *GoPV* as your local absolute paths. Depending on whether the project you want to detect uses GOPATH or GOMODULE for dependency management, add the corresponding command to run.sh.

 

       Use the command below to detect Go projects that using GOPATH for dependency management, such as Docker. Note that the *$GOPATH* in run.sh must be set to the parent directory of the *src* directory of the project to be detected.

 

       ```

       GO111MODULE=off $GoPV -path="$GOPATH"/src/github.com/docker -include=github.com/docker -checker=ctx:M_WG:Cond_V -r

       ```

 

       Use the command below to detect Go projects that using GOMODULE for dependency management. Note that after -mod-abs-path and -mod-module-path, both should be the absolute path of the project.

    

       ```

       GoPV -mod -mod-abs-path <-mod-abs-path> -mod-module-path <-mod-abs-path> -r -checker=ctx:M_WG:Cond_V

       ```

    

    3. After correctly modifying these two scripts, run GoPV in the terminal.

    

       3.1 Navigate to the directory of GoPV.

    

       3.2 Run the install.sh script by the command `sh install.sh`.

    

       3.3 Run the run.sh script by the command `sh run.sh`. You can save the output results to a file like this: `sh run.sh > result.txt`

    

- The output information of GoPV includes bug type, bug location and some other information to help users troubleshoot the bug.

 

## Verification

 

Reproducing the GoPV experimental results will take about three hours. The *ExperimentalResult.xlsx* file, located under the artifact package of GoPV, contains all experimental results, including TP, FP, developer report links, and runtime on the experimental dataset. You can reproduce GoPV experimental results and verify effectiveness, efficiency by following these steps:

 

1. Install Go, then configure GOPATH and environment variables. 

 

2. Place GoPV in your *GOPATH/src* directory.

 

3. Edit the install.sh in GoPV, setting *CURDIR* and *GOPATH* as your local absolute paths.

 

4. Edit the run.sh in GoPV, setting *CURDIR* and *GoPV* as your local absolute paths. And you don't need to modify *GOPATH* in the run.sh, which has been properly set up to detect Docker in *GoPV/testdata/real-GoProjects*. We have put commands to the run.sh script to detect all projects in the dataset, These projects are located in the *testdata* directory of GoPV , please replace the path of each project in the run.sh with your local absolute path.

 

5. Navigate to the directory of GoPV, run the install.sh script firstly, then run the run.sh script to detect all the projects in the dataset. 

 

## Troubleshooting

 

- If there are some problems when running the `run.sh`  script, you could try to check a project individually by running the corresponding command in the terminal.

- If there are some problems with downloading project's dependency packages, such as timeout, please replace *goproxy* with a proxy in your country.

- For further assistance or questions, please contact the developers.

 

## Comparison tools

 

By reproducing the experimental results of the comparison tools, you can verify the advantage of GoPV. The process takes approximately three days, largely due to the goleak and GFuzz experiments on real-world applications.

 

### goleak 

 

- You could download goleak by `go get -u go.uber.org/goleak` in your terminal or add "go.uber.org/goleak v1.1.0" in the *go.mod* file of a project.

 

- goleak detects bugs based on the unit test function in Go, which starts with Testxxx. You should add the statement `defer goleak.VerifyNone(t)` at the beginning of a test, and run the test to verify whether there are unexpected goroutines running or not. Here is an example of using goleak to test the gobench program *cockroach#13197*. You should add the statement `defer goleak.VerifyNone(t)` at the beginning of the test, then run the test.

 

  ```

  func TestCockroach13197(t *testing.T) {

  defer goleak.VerifyNone(t)

  db := &DB{}

  db.begin(context.Background()) // G1

  }

  ```

 

- We recommend using goleak in GoLand. You could download it at https://www.jetbrains.com/go/. After adding the statement `defer goleak.VerifyNone(t)`, it is very convenient to run a test by simply clicking the green button next to the unit test function. Additionally, you could use the command `go test -v ./...` to run all tests in the current package and all sub-packages.

 

- Specially, you should place *Docker* in the *GOPATH/sr*c directory. Because it still use GOPATH to manage dependencies. 

 

- You could refer to https://github.com/uber-go/goleak for more usage details of goleak.

 

### GCatch

 

- Firstly, you could download GCatch from https://github.com/system-pclub/GCatch.

 

- Secondly, you should run installZ3.sh and modify install.sh and run.sh. The install.sh and run.sh scripts in GCatch and GoPV are similar. Refer to GoPV for how to modify them.

 

- Then you could detect GoBench programs by running the two scripts.

 

  ```

  sh install.sh 

  sh run.sh

  ```

 

### GOAT

 

- Firstly, you could download GOAT from https://github.com/cs-au-dk/goat.

- GOAT requires an installation of Go 1.18 to run

- You could put GoBench programs in the *examples* directory in GOAT. And using the commands to build goat.

 

```

go install golang.org/x/tools/cmd/goimports@latest

go generate ./...

go build github.com/cs-au-dk/goat

```

 

- Then run commands to detect bugs. For examples, using the below command to detect the program that is numbered 862 in GoBench.

 

```

./goat -gopath examples -task collect-primitives -include-tests -metrics -psets gcatch 862 

```

 

### GoPie

 

You could download GoPie from https://github.com/CGCL-codes/GoPie.

 

1. `GoPie` has been implemented using `Go 1.19.1`.

 

2. Build the binary under `cmd` with `go build -o ./bin ./cmd/...`. There will be two binaries after compilation, the `inst` and `fuzz`.

 

   ```

   go build -o ./bin ./cmd/...

   ```

 

3. Instrument stubs and compile your project which is to be tested.

 

   ```

   ./bin/fuzz --task inst --path your_project_to_be_tested

   ```

 

4. Build test binaries, the test binaries will be placed into `./testbins`

 

   ```

   // install dependencies

   cd your_project_to_be_tested

   go mod tidy

   

   // compile the unit tests

   ./bin/fuzz --task bins --path your_project_to_be_tested

   ```

 

5. Start testing

 

   ```

   ./bin/fuzz --task full --path path_of_test_binaries

   ```

 

**Attention please:**

 

- You should run *patch.sh* in the *script* directory to patch some go files in Go standard library.

- You should use a copy of the dataset to do the GoPie experiments, because codes are affected with the instrumentation.

 

### GFuzz

 

- You could download GFuzz from https://github.com/system-pclub/GFuzz.

- We use GFuzz by fuzzing a Golang project with `fuzz-mount.sh` script.

- Please use the **go1.16**, and run the `GFuzz/scripts/patch.sh` script to patch Go library. If there are some problems when running the script, please follow each step in `patch.sh` script to ensure successful patch.

- You should use a copy of the dataset to do the GFuzz experiments, because codes are affected with the instrumentation.

 

`fuzz-mount.sh` will mount target Golang project folder to the docker container, instrument, and start fuzzing.

 

```

$ ./scripts/fuzz-mount.sh <GOMOD DIR> <OUTPUT DIR> [...optional parameters passed to fuzzer]

 

# Examples 

$ ./scripts/fuzz-mount.sh /AbsolutePathOnYourFileSystemToGolangProject /AbsolutePathOnYourFileSystem

 

# This script will do following things:

# 1. create a docker image

# 2. copy fuzzer and instrumentation tool source code and Golang runtime patches

# 3. compile fuzzer and instrumentation tool

# 4. patch Golang runtime

# 5. done building docker image

# 6. docker run (start fuzzing)

# 7. instrument target repository (only caring about .go files)

# 8. start fuzzing

```

 

For example, you could run GFuzz to detect GoBench by the command

 

```

./scripts/fuzz-mount.sh ./testdata/gobench/src/goking/blocking ./gobenchRes

```

 

**Attention please**, since docker does not support go module, GFuzz needs to build unit tests first and then conducts the fuzzing.

 

- You should place *Docker* in the *GOPATH/sr*c directory. Because it still use GOPATH to manage dependencies. 

- You could use `fuzz-mount.sh` to instrument docker firstly, then run the *docker/builder/entrypoint.sh* script, please comment out all the code except for the following part, which generates the *testbin* files of docker unit tests.

 

```

OUTPUT_DIR=<output dir>

export GO111MODULE=off

pkg_list=$(go list github.com/docker/docker/... | grep -vE "(integration)")

 

for pkg in $pkg_list

do

    echo "generating test bin for $pkg"

    name=$(echo "$pkg" | sed "s/\//-/g")

    go test -c -o $OUTPUT_DIR/moby/native/$name.test $pkg

done

export GO111MODULE=on

```

 

- After that, use the following command to do the fuzzing.

 

```

$ ./scripts/fuzz-testbins.sh <testbin dir> <output dir> [optional flags for fuzzer]

```