Repository: go
Issue: gccgo: go compile got error in aarch64, lld
Labels: NeedsInvestigation,arch-arm64


<!--
Please answer these questions before submitting your issue. Thanks!
-->

### What version of Go are you using (`go version`)?

<pre>
$ go version
</pre>

```
# go version
go version go1.19 linux/arm64

# gccgo --version
gccgo (GCC) 12.1.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```



### Does this issue reproduce with the latest release?

I didn't test golang/go master branch, but there should be.

### What operating system and processor architecture are you using (`go env`)?

<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
GO111MODULE="off"
GOARCH="arm64"
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOOS="linux"
GOROOT="/usr/GoRelease/go1.19"
GOTOOLDIR="/usr/GoRelease/go1.19/pkg/tool/linux_arm64"
GOVERSION="go1.19"
GCCGO="/usr/gcc-12.1.0-install/bin/gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
GOGCCFLAGS="-fPIC -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1375830659=/tmp/go-build -gno-record-gcc-switches
</pre>
</details>


Linux kernel 4.19  aarch64



### What did you do?

<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on go.dev/play is best.
-->

A simple helloworld.go:

```
package main
  
import "fmt"

func main() {
        fmt.Println("Hello World!")
}
```

export GCCGO="/usr/gcc-12.1.0-install/bin/gccgo"

(LD_LIBRARY_PATH also needs to be set)

Test results:

| command                                                      | result                                                       |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| go build -a -compiler=gccgo -gccgoflags="-fuse-ld=gold" helloworld.go | success                                                      |
| go build -a -compiler=gccgo -gccgoflags="-fuse-ld=lld" helloworld.go | ld.lld: error: Mixing split-stack objects requires a definition of __morestack_non_split<br/>collect2: error: ld returned 1 exit status |
| gccgo -fuse-ld=lld helloworld.go                             | success                                                      |
| gccgo -fuse-ld=gold helloworld.go                            | success                                                      |

And then I have some analysis, which **may not be correct**.

First, in aarch64, gccgo doesn't support `-fsplit-stack`.

```bash
# gccgo -fsplit-stack helloworld.go 
go1: error: ‘-fsplit-stack’ is not supported by this compiler configuration
```

I found go command create and compile a `_buildid.s`. Run

```bash
go build -a -x -work -compiler=gccgo -gccgoflags="-fuse-ld=lld" helloworld.go
```

I see:

```
echo '	.section .go.buildid,"e"' >> $WORK/b001/_buildid.s
...
echo '	.section .note.GNU-stack,"",@progbits' >> $WORK/b001/_buildid.s
echo '	.section .note.GNU-split-stack,"",@progbits' >> $WORK/b001/_buildid.s
echo '' >> $WORK/b001/_buildid.s
/usr/gcc-12.1.0-install/bin/gccgo -xassembler-with-cpp -I $WORK/b001/ -c -o $WORK/b001/_buildid.o -D GOOS_linux -D GOARCH_arm64 $WORK/b001/_buildid.s
ar rcD $WORK/b001/_pkg_.a $WORK/b001/_go_.o $WORK/b001/_buildid.o
```

I see  `_buildid.s` contains a special section `.note.GNU-split-stack`, it was added at [cmd/go/internal/work/buildid.go:335](https://github.com/golang/go/blob/2f103873c5081c7013cadfecb9b1c051d7c5520a/src/cmd/go/internal/work/buildid.go#L335).

See below:

```go
	if cfg.Goos != "solaris" && cfg.Goos != "illumos" && cfg.Goos != "aix" {
		secType := "@progbits"
		if cfg.Goarch == "arm" {
			secType = "%progbits"
		}
		fmt.Fprintf(&buf, "\t"+`.section .note.GNU-stack,"",%s`+"\n", secType)
		fmt.Fprintf(&buf, "\t"+`.section .note.GNU-split-stack,"",%s`+"\n", secType)
	}
```





In [lld/ELF/InputFiles.cpp:994](https://github.com/llvm/llvm-project/blob/1fbc6b26a237f01b8f1a5c5228450918bd055ca2/lld/ELF/InputFiles.cpp#L994) of llvm-project  `main` branch , if it detected a name `.note.GNU-split-stack` , it will set variable `splitStack` as `true`. 

See below:

```c++
    // Split stacks is a feature to support a discontiguous stack,
    // commonly used in the programming language Go. For the details,
    // see https://gcc.gnu.org/wiki/SplitStacks. An object file compiled
    // for split stack will include a .note.GNU-split-stack section.
    if (name == ".note.GNU-split-stack") {
      if (config->relocatable) {
        error(
            "cannot mix split-stack and non-split-stack in a relocatable link");
        return &InputSection::discarded;
      }
      this->splitStack = true;
      return &InputSection::discarded;
    }
```



In [lld/ELF/InputSection.cpp:654](https://github.com/llvm/llvm-project/blob/1fbc6b26a237f01b8f1a5c5228450918bd055ca2/lld/ELF/InputSection.cpp#L954) , the variable  `splitStack` is true, it will call function `adjustSplitStackFunctionPrologues`.

See below:

```c++
template <class ELFT>
void InputSectionBase::relocate(uint8_t *buf, uint8_t *bufEnd) {
  if ((flags & SHF_EXECINSTR) && LLVM_UNLIKELY(getFile<ELFT>()->splitStack))
    adjustSplitStackFunctionPrologues<ELFT>(buf, bufEnd);
  //...
}
```



In function [adjustSplitStackFunctionPrologues](https://github.com/llvm/llvm-project/blob/1fbc6b26a237f01b8f1a5c5228450918bd055ca2/lld/ELF/InputSection.cpp#L1196) , in aarch64, the variable  `needsMoreStackNonSplit` is alway true, so it always call function `switchMorestackCallsToMorestackNonSplit`.

See below:

```c++
// If a function compiled for split stack calls a function not
// compiled for split stack, then the caller needs its prologue
// adjusted to ensure that the called function will have enough stack
// available. Find those functions, and adjust their prologues.
template <class ELFT>
void InputSectionBase::adjustSplitStackFunctionPrologues(uint8_t *buf,
                                                         uint8_t *end) {
  //...

  if (target->needsMoreStackNonSplit)
    switchMorestackCallsToMorestackNonSplit(prologues, morestackCalls);
}
```



In function [switchMorestackCallsToMorestackNonSplit](switchMorestackCallsToMorestackNonSplit), it cannot found  `__morestack_non_split` symbol, so it throws a error of `mixing split-stack objects requires a definition of __morestack_non_split`.

See below:

```c++
// For each function-defining prologue, find any calls to __morestack,
// and replace them with calls to __morestack_non_split.
static void switchMorestackCallsToMorestackNonSplit(
  //...

  // If the target adjusted a function's prologue, all calls to
  // __morestack inside that function should be switched to
  // __morestack_non_split.
  Symbol *moreStackNonSplit = symtab->find("__morestack_non_split");
  if (!moreStackNonSplit) {
    error("mixing split-stack objects requires a definition of "
          "__morestack_non_split");
    return;
  }

  //...
}
```





When the `split-stack` mode is not supported, it still write section `.note.GNU-split-stack` into `_buildid.s` . I think, this led to the problem.

If `split-stack` is not supported, is it a correct fix to **not** write section `.note.GNU-split-stack` into `_buildid.s`?



Note: The cause of this problem is **not** the same as [another cgo compilation problem](https://github.com/golang/go/issues/54367).



### What did you expect to see?

In aarch64, run `go build -a -compiler=gccgo -gccgoflags="-fuse-ld=lld" helloworld.go` success.

### What did you see instead?

lld reports an error:

```
ld.lld: error: Mixing split-stack objects requires a definition of __morestack_non_split
```