Service Unavailable after making space accessible by anounymous users
Asked by Gus Johansson:

Hi,

I tried the Mention function in the comment section and our Confluence Server 6.14 just stopped working. Now I can not start it again.

I have reset the service and restarted the server and still same.

________________________________________

Edit:

I have now caused Confluence to crash a second time. This is caused by the settings that make a space available for anonymous users. I have a test space, I opened it up for anonymous access and site crashed. When I try to enter the site i get:

"**Http/1.1 Service Unavailable"**

Service is running. A reset of service does not work. A restart of server does not work.

**All I see in the logs are:**

04-Apr-2019 12:59:24.102 INFO \[https-jsse-nio2-8443-exec-23\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 12:59:24.133 INFO \[https-jsse-nio2-8443-exec-6\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 12:59:25.430 INFO \[https-jsse-nio2-8443-exec-8\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 13:00:13.709 INFO \[https-jsse-nio2-8443-exec-11\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'

**Here are logs from first incident, they dont seem to interrupt the service:**

03-Apr-2019 17:15:50.911 WARNING \[localhost-startStop-4\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[org.h2.Driver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

03-Apr-2019 17:15:50.942 WARNING \[localhost-startStop-4\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application \[ROOT\] appears to have started a thread named \[AtlassianEvent::CustomizableThreadFactory-1\] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:  
sun.misc.Unsafe.park(Native Method)  
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)  
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)  
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)  
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
java.lang.Thread.run(Thread.java:748)

....

03-Apr-2019 17:15:51.005 SEVERE \[localhost-startStop-4\] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application \[ROOT\] created a ThreadLocal with key of type \[com.atlassian.confluence.cache.TransactionalCacheFactory$1\] (value \[com.atlassian.confluence.cache.TransactionalCacheFactory$1@4b0f26a2\]) and a value of type \[com.atlassian.confluence.cache.TransactionalCacheFactory.DelegateCacheSynchronization\] (value \[com.atlassian.confluence.cache.TransactionalCacheFactory$DelegateCacheSynchronization@17b47241\]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.


Answers:
author: Gus Johansson
body: I have now caused Confluence to crash a second time. This is caused by the settings that make a space available for anonymous users. I have a test space, I opened it up for anonymous access and site crashed. When I try to enter the site i get:

"**Http/1.1 Service Unavailable"**

Service is running. A reset of service does not work. A restart of server does not work.

All I see in the logs are:

04-Apr-2019 12:59:24.102 INFO \[https-jsse-nio2-8443-exec-23\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 12:59:24.133 INFO \[https-jsse-nio2-8443-exec-6\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 12:59:25.430 INFO \[https-jsse-nio2-8443-exec-8\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 13:00:13.709 INFO \[https-jsse-nio2-8443-exec-11\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'

Please assist.


author: Gus Johansson
body: We are now up and running again. We restored the database prior to my Mention comment.

\* Could it be the mention comment that caused this? Very good question. We do not know.

Here are logs from the working startup:

<br />

04-Apr-2019 09:52:53.800 SEVERE \[https-jsse-nio2-8443-Acceptor-0\] org.apache.tomcat.util.net.Acceptor.run Socket accept failed  
java.util.concurrent.ExecutionException: java.io.IOException: The specified network name is no longer available.

at sun.nio.ch.PendingFuture.get(PendingFuture.java:185)  
at org.apache.tomcat.util.net.Nio2Endpoint.serverSocketAccept(Nio2Endpoint.java:339)  
at org.apache.tomcat.util.net.Nio2Endpoint.serverSocketAccept(Nio2Endpoint.java:56)  
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)  
at java.lang.Thread.run(Thread.java:748)  
Caused by: java.io.IOException: The specified network name is no longer available.

at sun.nio.ch.Iocp.translateErrorToIOException(Iocp.java:309)  
at sun.nio.ch.Iocp.access$700(Iocp.java:46)  
at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:399)  
... 1 more

04-Apr-2019 09:54:34.349 INFO \[https-jsse-nio2-8443-exec-12\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:53.040 INFO \[https-jsse-nio2-8443-exec-13\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:53.149 INFO \[https-jsse-nio2-8443-exec-12\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:53.399 INFO \[https-jsse-nio2-8443-exec-12\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc  
04-Apr-2019 09:54:53.415 INFO \[https-jsse-nio2-8443-exec-12\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.grammars.WadlGrammarsAdaptor  
04-Apr-2019 09:54:53.415 INFO \[https-jsse-nio2-8443-exec-12\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport  
04-Apr-2019 09:54:53.431 INFO \[https-jsse-nio2-8443-exec-13\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc  
04-Apr-2019 09:54:53.477 INFO \[https-jsse-nio2-8443-exec-13\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.grammars.WadlGrammarsAdaptor  
04-Apr-2019 09:54:53.477 INFO \[https-jsse-nio2-8443-exec-13\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport  
04-Apr-2019 09:54:53.977 INFO \[https-jsse-nio2-8443-exec-4\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:54.196 INFO \[https-jsse-nio2-8443-exec-16\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:54.602 INFO \[https-jsse-nio2-8443-exec-17\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:54.806 INFO \[https-jsse-nio2-8443-exec-1\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:55.618 INFO \[https-jsse-nio2-8443-exec-19\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:56.119 INFO \[https-jsse-nio2-8443-exec-7\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:56.243 INFO \[https-jsse-nio2-8443-exec-13\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:57.212 INFO \[https-jsse-nio2-8443-exec-10\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:54:57.368 INFO \[https-jsse-nio2-8443-exec-10\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc  
04-Apr-2019 09:54:57.368 INFO \[https-jsse-nio2-8443-exec-10\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.grammars.WadlGrammarsAdaptor  
04-Apr-2019 09:54:57.368 INFO \[https-jsse-nio2-8443-exec-10\] com.sun.jersey.api.wadl.config.WadlGeneratorLoader.loadWadlGenerator Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport  
04-Apr-2019 09:55:00.720 INFO \[https-jsse-nio2-8443-exec-24\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:55:12.455 INFO \[https-jsse-nio2-8443-exec-3\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:55:13.049 INFO \[https-jsse-nio2-8443-exec-5\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:55:13.127 INFO \[https-jsse-nio2-8443-exec-1\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:55:13.268 INFO \[https-jsse-nio2-8443-exec-17\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:55:18.248 INFO \[https-jsse-nio2-8443-exec-19\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:55:18.466 WARNING \[https-jsse-nio2-8443-exec-19\] com.sun.jersey.spi.container.servlet.WebComponent.filterFormParameters A servlet request, to the URI <https://confluence.xenit.se/rest/tinymce/1/content/1245203/comment?actions=true>, contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.  
04-Apr-2019 09:55:19.841 INFO \[https-jsse-nio2-8443-exec-5\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:56:03.485 INFO \[https-jsse-nio2-8443-exec-8\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'  
04-Apr-2019 09:59:21.608 INFO \[https-jsse-nio2-8443-exec-5\] com.sun.jersey.server.impl.application.WebApplicationImpl._initiate Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'


author: Gus Johansson
body: This is what I see in my logs when attempting to start the service:

04-Apr-2019 08:45:47.449 SEVERE \[https-jsse-nio2-8443-Acceptor-0\] org.apache.tomcat.util.net.Acceptor.run Socket accept failed  
java.util.concurrent.ExecutionException: java.io.IOException: The specified network name is no longer available.

at sun.nio.ch.PendingFuture.get(PendingFuture.java:185)  
at org.apache.tomcat.util.net.Nio2Endpoint.serverSocketAccept(Nio2Endpoint.java:339)  
at org.apache.tomcat.util.net.Nio2Endpoint.serverSocketAccept(Nio2Endpoint.java:56)  
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)  
at java.lang.Thread.run(Thread.java:748)  
Caused by: java.io.IOException: The specified network name is no longer available.

at sun.nio.ch.Iocp.translateErrorToIOException(Iocp.java:309)  
at sun.nio.ch.Iocp.access$700(Iocp.java:46)  
at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:399)  
... 1 more


Tags: confluence-server

server