Repository: caddyserver/caddy (language: go)

Issue:
Caddy Build Fails for Solaris/Illumos and Plan9
Initially reported by @Toasterson

> The main issue is that badger@1.5.3 uses golangs internal syscall library and that does not support many operatingsystems notably openbsd, netbsd, and illumos. Badger uses a constant for MADVISE and that only came into the go for these Unixes after the internal syscall library got frozen. Right now the internal syscall library only really works for linux and every software that wants to support something else than linux and uses the syscall library directly must use x/sys/unix instead. I've personally changed quite a few direct dependencies and pushed people to use x/sys/unix instead of syscall so that today this is mostly happening due the dependency chain. I could not upgrade badger to 1.6.1 as there where api changes but nosql bump worked. So until that is resolved caddy cannot be built on quite a few unix versions I suppose. I know it won't build for illumos right now. Are you monitoring cross-builds?
>

> Also here the exact error for documentation
> 
> ```
> # github.com/dgraph-io/badger/y
> ../../../pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:57:30: undefined: syscall.SYS_MADVISE
> ```


_Originally posted by @Toasterson in https://github.com/caddyserver/caddy/pull/3602#issuecomment-663876698_


Build errors:
- Solaris/Illumos
```
$ GOOS=solaris go build
# github.com/dgraph-io/badger/y
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:57:30: undefined: syscall.SYS_MADVISE
```

- plan9:
```
$ GOOS=plan9 go build
# github.com/dgraph-io/badger/y
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/file_dsync.go:24:21: undefined: unix.O_DSYNC
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:32:11: undefined: unix.PROT_READ
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:34:12: undefined: unix.PROT_WRITE
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:36:9: undefined: unix.Mmap
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:36:54: undefined: unix.MAP_SHARED
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:41:9: undefined: unix.Munmap
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:48:11: undefined: unix.MADV_NORMAL
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:50:11: undefined: unix.MADV_RANDOM
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:57:30: undefined: syscall.SYS_MADVISE
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:59:8: cannot use 0 (type untyped int) as type syscall.ErrorString
../../../../../go/pkg/mod/github.com/dgraph-io/badger@v1.5.3/y/mmap_unix.go:59:8: too many errors
# github.com/caddyserver/caddy/v2/cmd
../proc_posix.go:28:9: undefined: syscall.Kill
# go.etcd.io/bbolt
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/db.go:217:12: undefined: flock
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/db.go:354:12: undefined: mmap
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/db.go:376:12: undefined: munmap
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/db.go:456:12: undefined: fdatasync
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/db.go:501:14: undefined: funlock
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/db.go:861:37: undefined: fdatasync
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/tx.go:542:13: undefined: fdatasync
../../../../../go/pkg/mod/go.etcd.io/bbolt@v1.3.2/tx.go:579:13: undefined: fdatasync
# github.com/dgraph-io/badger/v2/y
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/file_dsync.go:24:21: undefined: unix.O_DSYNC
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:30:11: undefined: unix.PROT_READ
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:32:12: undefined: unix.PROT_WRITE
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:34:9: undefined: unix.Mmap
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:34:54: undefined: unix.MAP_SHARED
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:39:9: undefined: unix.Munmap
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:46:11: undefined: unix.MADV_NORMAL
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:48:11: undefined: unix.MADV_RANDOM
../../../../../go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-rc1.0.20200413122845-09dd2e1a4195/y/mmap_unix.go:50:9: undefined: unix.Madvise
# github.com/chzyer/readline
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/operation.go:234:4: undefined: ClearScreen
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:129:20: undefined: GetScreenWidth
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:132:22: undefined: DefaultIsTerminal
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:142:26: undefined: DefaultOnWidthChanged
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:324:2: undefined: DefaultOnWidthChanged
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:346:17: undefined: GetScreenWidth
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:362:16: undefined: DefaultIsTerminal
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/terminal.go:51:2: undefined: SuspendMe
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/utils.go:81:29: undefined: State
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/utils.go:241:9: undefined: State
../../../../../go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/terminal.go:51:2: too many errors
```

Known as of v2.1.1, but issue could be extending back in history. Exact version/commit is pretty much irrelevant.


Discussion / hints:
I have opened issues upstream:

- https://discuss.dgraph.io/t/build-failure-for-goos-solaris-illumos-plan9/9145?u=mholt
- https://github.com/etcd-io/bbolt/issues/231
- https://github.com/chzyer/readline/issues/188

I've also removed those platforms from our download page for the time being. Too bad. :(
> I have opened issues upstream:
> 
>     * https://discuss.dgraph.io/t/build-failure-for-goos-solaris-illumos-plan9/9145?u=mholt
> 
>     * [etcd-io/bbolt#231](https://github.com/etcd-io/bbolt/issues/231)
> 
>     * [chzyer/readline#188](https://github.com/chzyer/readline/issues/188)
> 
> 
> I've also removed those platforms from our download page for the time being. Too bad. :(

There's one item in this stream 🙂 
```
# github.com/caddyserver/caddy/v2/cmd
../proc_posix.go:28:9: undefined: syscall.Kill
```
It's a thoughie because there's no equivalent to `syscall.Kill` on Plan9. If all upstream issues are resolved, we might need to resort to shell command to issue kill command.
> There's one item in this stream 🙂

Yeah, but that one won't be too hard -- like you said, a shell command or something: https://en.wikipedia.org/wiki/Kill_(command)#Plan_9_from_Bell_Labs `kill caddy | rc`

Tbh I'm skeptical that the upstream ones will be resolved though.
Dgraph fixing builds for Plan9; looking into Solaris: https://discuss.dgraph.io/t/build-failure-for-goos-solaris-illumos-plan9/9145/3?u=mholt
Joyent's pkgsrc somehow manages to compile [caddy on illumos](https://github.com/joyent/pkgsrc/tree/trunk/www/caddy), but it is with no packages.
Looks like it's working everywhere except plan9 now! https://github.com/caddyserver/caddy/actions/runs/1152172382

Errors I see from trying to build it locally:

```
# go.etcd.io/bbolt
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:223:12: undefined: flock
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:360:12: undefined: mmap
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:382:12: undefined: munmap
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:462:12: undefined: fdatasync
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:507:14: undefined: funlock
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\db.go:867:37: undefined: fdatasync
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\tx.go:559:13: undefined: fdatasync
..\..\..\..\go\pkg\mod\go.etcd.io\bbolt@v1.3.5\tx.go:596:13: undefined: fdatasync
# github.com/chzyer/readline
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\operation.go:234:4: undefined: ClearScreen
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\readline.go:129:20: undefined: GetScreenWidth
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\readline.go:132:22: undefined: DefaultIsTerminal
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\readline.go:142:26: undefined: DefaultOnWidthChanged
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\remote.go:324:2: undefined: DefaultOnWidthChanged
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\remote.go:346:17: undefined: GetScreenWidth
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\remote.go:362:16: undefined: DefaultIsTerminal
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\terminal.go:51:2: undefined: SuspendMe
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\utils.go:81:29: undefined: State
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\utils.go:241:9: undefined: State
..\..\..\..\go\pkg\mod\github.com\chzyer\readline@v0.0.0-20180603132655-2972be24d48e\terminal.go:51:2: too many errors
# github.com/dgraph-io/badger/v2/y
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\file_dsync.go:24:21: undefined: unix.O_DSYNC
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:30:11: undefined: unix.PROT_READ
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:32:12: undefined: unix.PROT_WRITE
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:34:9: undefined: unix.Mmap
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:34:54: undefined: unix.MAP_SHARED
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:39:9: undefined: unix.Munmap
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:46:11: undefined: unix.MADV_NORMAL
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:48:11: undefined: unix.MADV_RANDOM
..\..\..\..\go\pkg\mod\github.com\dgraph-io\badger\v2@v2.2007.3\y\mmap_unix.go:50:9: undefined: unix.Madvise
```

Asked badger if they can backport their fixes for plan9 to their v2 branch: https://github.com/dgraph-io/badger/pull/1451#issuecomment-903453493

`github.com\chzyer\readline` Looks essentially abandoned so we shouldn't hold our breath on that one.
Getting closer, sort of. Still builds for solaris OK, latest attempt at plan9 is now failing with:

```
$ GOOS=plan9 go build
# go.etcd.io/bbolt
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:230:12: undefined: flock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:375:12: undefined: mmap
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:404:12: undefined: munmap
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:508:12: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:554:14: undefined: funlock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/db.go:914:37: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/mlock_unix.go:14:17: undefined: unix.Mlock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/mlock_unix.go:32:17: undefined: unix.Munlock
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/tx.go:558:13: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/tx.go:595:13: undefined: fdatasync
/home/matt/go/pkg/mod/go.etcd.io/bbolt@v1.3.6/tx.go:595:13: too many errors
# github.com/chzyer/readline
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/utils.go:81:29: undefined: State
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/utils.go:241:9: undefined: State
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/operation.go:234:4: undefined: ClearScreen
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:129:20: undefined: GetScreenWidth
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:132:22: undefined: DefaultIsTerminal
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/readline.go:142:26: undefined: DefaultOnWidthChanged
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:324:2: undefined: DefaultOnWidthChanged
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:346:17: undefined: GetScreenWidth
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/remote.go:362:16: undefined: DefaultIsTerminal
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/terminal.go:51:2: undefined: SuspendMe
/home/matt/go/pkg/mod/github.com/chzyer/readline@v0.0.0-20180603132655-2972be24d48e/terminal.go:51:2: too many errors
# github.com/tailscale/tscert/internal/paths
/home/matt/go/pkg/mod/github.com/tailscale/tscert@v0.0.0-20220316030059-54bbcb9f74e2/internal/paths/paths_unix.go:44:15: undefined: unix.Access
/home/matt/go/pkg/mod/github.com/tailscale/tscert@v0.0.0-20220316030059-54bbcb9f74e2/internal/paths/paths_unix.go:44:32: undefined: unix.O_RDWR
```
Welp, still the same result 2 years later. I don't think there's any hope of getting Caddy to build on Plan9 until the listed dependencies also support it. We could be more proactive about getting this done, but I think we'll only prioritize this if there's a real need and/or sponsorship to do so.

Fix patch:
diff --git a/.github/workflows/cross-build.yml b/.github/workflows/cross-build.yml
index 676607d0e6b..df67dc83472 100644
--- a/.github/workflows/cross-build.yml
+++ b/.github/workflows/cross-build.yml
@@ -17,14 +17,12 @@ jobs:
       matrix:
         goos: 
           - 'aix'
-          - 'android'
           - 'linux'
           - 'solaris'
           - 'illumos'
           - 'dragonfly'
           - 'freebsd'
           - 'openbsd'
-          - 'plan9'
           - 'windows'
           - 'darwin'
           - 'netbsd'
@@ -69,7 +67,3 @@ jobs:
         working-directory: ./cmd/caddy
         run: |
           GOOS=$GOOS GOARCH=$GOARCH go build -tags nobadger -trimpath -o caddy-"$GOOS"-$GOARCH 2> /dev/null
-          if [ $? -ne 0 ]; then
-            echo "::warning ::$GOOS Build Failed"
-            exit 0
-          fi


Test patch: