Repository: netty/netty
Issue: Enabled SecurityManager results in ClassNotFoundError during io.netty.util.NetUtil initialization
Labels: defect

Hey,

Netty version used: 4.0.27

Setup: An application that has the security manager enabled, spits an not really helpful `ClassNotFoundError` in case that the application is not allowed to read a file due to the security manager. The reason for this is the try to read the file `/proc/sys/net/core/somaxconn` in https://github.com/netty/netty/blob/4.0/common/src/main/java/io/netty/util/NetUtil.java#L238

Exception looks like this:

```
Caused by: io.netty.channel.ChannelException: Unable to create Channel from class class io.netty.channel.socket.nio.NioServerSocketChannel
    at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:455)
    at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:306)
    at io.netty.bootstrap.AbstractBootstrap.doBind(AbstractBootstrap.java:271)
    at io.netty.bootstrap.AbstractBootstrap.bind(AbstractBootstrap.java:267)
    at org.elasticsearch.http.netty.NettyHttpServerTransport$1.onPortNumber(NettyHttpServerTransport.java:261)
    at org.elasticsearch.common.transport.PortsRange.iterate(PortsRange.java:69)
    at org.elasticsearch.http.netty.NettyHttpServerTransport.doStart(NettyHttpServerTransport.java:257)
    ... 45 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class io.netty.util.NetUtil
    at io.netty.channel.socket.DefaultServerSocketChannelConfig.&lt;init&gt;(DefaultServerSocketChannelConfig.java:39)
    at io.netty.channel.socket.nio.NioServerSocketChannel$NioServerSocketChannelConfig.&lt;init&gt;(NioServerSocketChannel.java:189)
    at io.netty.channel.socket.nio.NioServerSocketChannel$NioServerSocketChannelConfig.&lt;init&gt;(NioServerSocketChannel.java:187)
    at io.netty.channel.socket.nio.NioServerSocketChannel.&lt;init&gt;(NioServerSocketChannel.java:85)
    at io.netty.channel.socket.nio.NioServerSocketChannel.&lt;init&gt;(NioServerSocketChannel.java:70)
    at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at java.lang.Class.newInstance(Class.java:442)
    at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:453)
    ... 51 more
```

if you need more infos, drop me a note, I am happy to help!


Issue comments:
@spinscale thanks for reporting. Let me fix this.

Actually the scope of this needs to be broaden as we need to also ensure property access works in the static initializer. So this will be a bigger change.

@trustin @Scottmitch FYI

@normanmaurer - Lets fix this one and any others we can think of.  If we suspect this is a systemic problem lets open another issue to catch the remainder.

@Scottmitch +1 @spinscale fine with this ?

@normanmaurer sure!

@spinscale cool! Was this the only problem you needed to fix to get it run with a SecurityManager ?

yes, mainly due to having specific permissions on how to access files, we dont do any property based security. You can see our policy file at https://github.com/elastic/elasticsearch/blob/master/dev-tools/tests.policy

@spinscale thanks! 

Fixed


Pull request review comments:

Patch that resolved the issue:
diff --git a/common/src/main/java/io/netty/util/NetUtil.java b/common/src/main/java/io/netty/util/NetUtil.java
index 00958c45b27..436da329a10 100644
--- a/common/src/main/java/io/netty/util/NetUtil.java
+++ b/common/src/main/java/io/netty/util/NetUtil.java
@@ -28,6 +28,8 @@
 import java.net.NetworkInterface;
 import java.net.SocketException;
 import java.net.UnknownHostException;
+import java.security.AccessController;
+import java.security.PrivilegedAction;
 import java.util.ArrayList;
 import java.util.Enumeration;
 import java.util.List;
@@ -230,38 +232,45 @@ public final class NetUtil {
         LOOPBACK_IF = loopbackIface;
         LOCALHOST = loopbackAddr;
 
-        // Determine the default somaxconn (server socket backlog) value of the platform.
-        // The known defaults:
-        // - Windows NT Server 4.0+: 200
-        // - Linux and Mac OS X: 128
-        int somaxconn = PlatformDependent.isWindows() ? 200 : 128;
-        File file = new File("/proc/sys/net/core/somaxconn");
-        if (file.exists()) {
-            BufferedReader in = null;
-            try {
-                in = new BufferedReader(new FileReader(file));
-                somaxconn = Integer.parseInt(in.readLine());
-                if (logger.isDebugEnabled()) {
-                    logger.debug("{}: {}", file, somaxconn);
-                }
-            } catch (Exception e) {
-                logger.debug("Failed to get SOMAXCONN from: {}", file, e);
-            } finally {
-                if (in != null) {
+        // As a SecurityManager may prevent reading the somaxconn file we wrap this in a privileged block.
+        //
+        // See https://github.com/netty/netty/issues/3680
+        SOMAXCONN = AccessController.doPrivileged(new PrivilegedAction<Integer>() {
+            @Override
+            public Integer run() {
+                // Determine the default somaxconn (server socket backlog) value of the platform.
+                // The known defaults:
+                // - Windows NT Server 4.0+: 200
+                // - Linux and Mac OS X: 128
+                int somaxconn = PlatformDependent.isWindows() ? 200 : 128;
+                File file = new File("/proc/sys/net/core/somaxconn");
+                if (file.exists()) {
+                    BufferedReader in = null;
                     try {
-                        in.close();
+                        in = new BufferedReader(new FileReader(file));
+                        somaxconn = Integer.parseInt(in.readLine());
+                        if (logger.isDebugEnabled()) {
+                            logger.debug("{}: {}", file, somaxconn);
+                        }
                     } catch (Exception e) {
-                        // Ignored.
+                        logger.debug("Failed to get SOMAXCONN from: {}", file, e);
+                    } finally {
+                        if (in != null) {
+                            try {
+                                in.close();
+                            } catch (Exception e) {
+                                // Ignored.
+                            }
+                        }
+                    }
+                } else {
+                    if (logger.isDebugEnabled()) {
+                        logger.debug("{}: {} (non-existent)", file, somaxconn);
                     }
                 }
+                return somaxconn;
             }
-        } else {
-            if (logger.isDebugEnabled()) {
-                logger.debug("{}: {} (non-existent)", file, somaxconn);
-            }
-        }
-
-        SOMAXCONN = somaxconn;
+        });
     }
 
     /**


Test patch: