Repository: netty/netty
Issue: Throwing an exception in case glibc is missing instead of segfaulting the JVM

It seems Netty's native support depends on glibc (btw is this documented?).
When glibc is missing (for example using vanilla Alpine), the JVM could end up with a crash (`SIGSEGV`), see details in this issue: https://github.com/micrometer-metrics/micrometer/issues/2776

It also seems that `Epoll.isAvailable()` returns `true` even if glibc is missing.

### Expected behavior
Throwing an exception

### Actual behavior
JVM crash

### Steps to reproduce
Using Netty native with an environment that does not have glibc, e.g.: vanilla Alpine

### Minimal yet complete reproducer code (or URL to code)
This seems to be a known issue, please let me know if you really need a reproducer.

### Netty version
`4.1.68` (latest)

### JVM version (e.g. `java -version`)
Latest 11:
```bash
 ❯ docker run --rm 'azul/zulu-openjdk-alpine:11-jre' 'sh' '-c' 'java --version'
openjdk 11.0.12 2021-07-20 LTS
OpenJDK Runtime Environment Zulu11.50+19-CA (build 11.0.12+7-LTS)
OpenJDK 64-Bit Server VM Zulu11.50+19-CA (build 11.0.12+7-LTS, mixed mode)
```
Or also latest 17:
```bash
 ❯ docker run --rm 'bellsoft/liberica-openjdk-alpine-musl:17' 'sh' '-c' 'java --version'
openjdk 17 2021-09-14 LTS
OpenJDK Runtime Environment (build 17+35-LTS)
OpenJDK 64-Bit Server VM (build 17+35-LTS, mixed mode)
```
I assume this is an issue in every JRE that is currently supported.

### OS version (e.g. `uname -a`)
Both images above have the same output:
```bash
 ❯ docker run --rm 'azul/zulu-openjdk-alpine:11-jre' 'sh' '-c' 'uname -a'
Linux 1ee8d1090f14 5.10.47-linuxkit #1 SMP Sat Jul 3 21:51:47 UTC 2021 x86_64 Linux
```
I assume this is an issue in every OS where glibc is missing.

Issue comments:
The binary compatibility baseline for our native code is the GLIBC version that's used on CentOS 6. The work-around is to use the Nio transport.

It's not clear from the error reports how this is really showing up. If it's an ABI problem, then I'm not sure what we can do to mitigate it. If we are calling glibc specific functions, then I wonder if we can somehow check what libc we are using at runtime.
>It's not clear from the error reports how this is really showing up.

I'm not sure what your question is. The error happens when Netty native is used without having glibc.

>If it's an ABI problem...

I don't think it is an ABI problem since glibc is missing in Alpine, there is nothing to be compatible with (and based on [this comment ](https://github.com/netty/netty/issues/6841#issuecomment-307716433), other libc implementations are out of question).

I think checking if glibc is available or not and/or fixing `Epoll.isAvailable()` would fix the issue (it seems it returns true even if glibc is missing).
I put together a little test for this: https://github.com/jonatan-ivanov/netty-gh-11701

I called `Epoll.isAvailable()` and `Epoll.ensureAvailability()` on different distros to see if they return false/fail in case glibc is missing. It seems `isAvailable` does not return false and `ensureAvailability` does not fail even if glibc is missing.

Image | `isAvailable` | `ensureAvailability` | Comment
------ | ------------------- |  --------------------------- | -------------
bellsoft_liberica-openjre-debian:17 | `true` | `OK` | works
bellsoft_liberica-openjre-alpine:17  | `true` | `OK` | works since it contains glibc
**bellsoft_liberica-openjre-alpine-musl:17**  | `true` | `OK` | **does not work as expected**
azul_zulu-openjdk:17 | `true` | `OK` | works as expected
**azul_zulu-openjdk-alpine:17-jre**  | `true` | `OK` | **does not work as expected**


There's no way to detect the libc vendor from within a process, as far as I can see. And calling out to `lld`, or something like that, is quite awkward and can be a security liability. I think the best we can do for the moment is to document that only the Nio transport is supported on musl.

What do you think, @normanmaurer ?
I agree with @chrisvest here 
I'm not very familiar with this domain but do you think is it at least possible to throw an exception instead of crashing a JVM if such a thing occur?

Also can you use the `__GLIBC__ ` from [`features.h`](https://sourceware.org/git/?p=glibc.git;a=blob;f=include/features.h;h=d974eabfafc24ffb9ff4cd066b650134df32048b;hb=refs/heads/master#l477) to detect if you are dealing with glibc or not or is this only possible compile time?

@jonatan-ivanov That's only at compile time. At runtime you can feature detect by inspecting linked symbols, except you cant do that without calling into libc which in this case would cause a crash. Alternatively we shell out to a program to inspect libc, but that is also bad for multiple reasons, and we don't want to do that.
I have updated the documentation on the native transports to note that musl is not officially supported: https://github.com/netty/netty/wiki/Native-transports

I think that's the best we can do for now.
musl generally tries to support glibc-linked binaries invoking POSIX and some BSD extensions. errors related to GNU-specific extensions (and glibc header bastardizations) should be caught at load time, as in #6841 "__strndup: symbol not found". that particular issue can be fixed by installing gcompat package in alpine linux. segfaults are, as you say, caused by some ABI incompatibility.

i briefly glanced through netty c code and found some suspicious items: https://github.com/netty/netty/blob/6b3ec62e6e4e19a85304aecd8cb5331a3fcc70be/transport-native-unix-common/src/main/c/netty_unix_errors.c#L41 is suspicious: application code should never check feature test macros, because it should be defining it themselves. https://github.com/netty/netty/blob/51ebcbd9aba7eac854716e45f07765744f8dcdaf/transport-native-epoll/src/main/c/netty_epoll_native.c#L140 is suspicious: sysctl values should always be accessed using `read`/`write`, never stdio. some sysctls will not be read correctly if using stdio. neither of these should cause segfault at run time though.
@Hello71 The netty_unit_errors code is because we wish to use a thread-safe strerror variant, which is not immediately portable. (I'm not familiar with the other one you point out)
If you know of a way to improve it, we'd love to review a PR!
Fix reverted because it negatively impacts musl-based systems that also use the glibc compatibility layer.
@chrisvest Would you consider reopening this issue so that we can track another fix (if any)?
reopened 
fwiw if you want to do this in the dumbest possible way, you can check for `gnu_get_libc_version`. if your program is segfaulting and not just failing with symbol not found, then most likely there is some more serious issue with your program, not related to musl.
@Hello71 The JVM itself might rely on signals and tie them to internal semantics, i.e. rare NullPointerExceptions sometimes rely on segfaults to optimise a non-null fast path. So I wouldn't feel very confident trying to trap signals in JNI library code.
I think this would be fixed by https://github.com/netty/netty/pull/12272 ... 

Pull request review comments:

Patch that resolved the issue:
diff --git a/transport-native-epoll/pom.xml b/transport-native-epoll/pom.xml
index 1dec5ca0dbc..7707b7b100e 100644
--- a/transport-native-epoll/pom.xml
+++ b/transport-native-epoll/pom.xml
@@ -32,8 +32,9 @@
     <unix.common.lib.dir>${project.build.directory}/unix-common-lib</unix.common.lib.dir>
     <unix.common.lib.unpacked.dir>${unix.common.lib.dir}/META-INF/native/lib</unix.common.lib.unpacked.dir>
     <unix.common.include.unpacked.dir>${unix.common.lib.dir}/META-INF/native/include</unix.common.include.unpacked.dir>
-    <jni.compiler.args.cflags>CFLAGS=-O3 -Werror -fno-omit-frame-pointer -Wunused-variable -fvisibility=hidden -I${unix.common.include.unpacked.dir}</jni.compiler.args.cflags>
-    <jni.compiler.args.ldflags>LDFLAGS=-L${unix.common.lib.unpacked.dir} -Wl,--no-as-needed -lrt -ldl -Wl,--whole-archive -l${unix.common.lib.name} -Wl,--no-whole-archive</jni.compiler.args.ldflags>
+    <jni.compiler.args.cflags>CFLAGS=-O2 -pipe -Werror -fno-omit-frame-pointer -Wunused-variable -fvisibility=hidden -D_FORTIFY_SOURCE=2 -ffunction-sections -fdata-sections -I${unix.common.include.unpacked.dir}</jni.compiler.args.cflags>
+    <jni.compiler.args.ldflags>LDFLAGS=-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -Wl,--gc-sections -L${unix.common.lib.unpacked.dir}</jni.compiler.args.ldflags>
+    <jni.compiler.args.libs>LIBS=-Wl,--whole-archive -l${unix.common.lib.name} -Wl,--no-whole-archive -ldl</jni.compiler.args.libs>
     <nativeSourceDirectory>${project.basedir}/src/main/c</nativeSourceDirectory>
     <skipTests>true</skipTests>
   </properties>
@@ -154,6 +155,7 @@
                   <platform>.</platform>
                   <configureArgs>
                     <arg>${jni.compiler.args.ldflags}</arg>
+                    <arg>${jni.compiler.args.libs}</arg>
                     <arg>${jni.compiler.args.cflags}</arg>
                     <configureArg>--libdir=${project.build.directory}/native-build/target/lib</configureArg>
                   </configureArgs>
@@ -303,6 +305,7 @@
                   <platform>.</platform>
                   <configureArgs>
                     <arg>${jni.compiler.args.ldflags}</arg>
+                    <arg>${jni.compiler.args.libs}</arg>
                     <arg>${jni.compiler.args.cflags}</arg>
                     <configureArg>--libdir=${project.build.directory}/native-build/target/lib</configureArg>
                     <configureArg>--host=aarch64-linux-gnu</configureArg>


Test patch: