Getting HTTP Status 500 error on opening Confluence login page
Asked by Jaydeep Sarkar:

Hello,

Please find the background and details of the problem below. I've seen similar problem being discussed in this community but even after trying out all of the recommendations, it did not get fixed. Not sure, if I've missed out any of the suggested steps.

**Background:**

1) The confluence application version 5.4.3 is running on Linux 2.6.32-279.el6.x86_64 on confluence user.

501 14491 1 0 Apr24 ? 00:02:01 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java -Djava.util.logging.config.file=/opt/atlassian-confluence-5.4.3/conf/logging.properties -Xms256m -Xmx512m -XX:MaxPermSize=256m -Djava.awt.headless=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/atlassian-confluence-5.4.3/endorsed -classpath /opt/atlassian-confluence-5.4.3/bin/bootstrap.jar -Dcatalina.base=/opt/atlassian-confluence-5.4.3 -Dcatalina.home=/opt/atlassian-confluence-5.4.3 -Djava.io.tmpdir=/opt/atlassian-confluence-5.4.3/temp org.apache.catalina.startup.Bootstrap start

**Problem:**

3) When I'm trying to launch loggin screen http://\<hostname\>.com:8090/

I'm getting the following error on the home page:

**HTTP Status 500 -**   

**type Exception report**

message

description The server encountered an internal error () that prevented it from fulfilling this request.

**exception**

com.atlassian.util.concurrent.LazyReference$InitializationException: java.lang.NullPointerException  
com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:149)  
com.atlassian.util.concurrent.LazyReference.get(LazyReference.java:112)  
com.atlassian.confluence.setup.ConfluenceEncodingFilter.getGlobalSettings(ConfluenceEncodingFilter.java:45)  
com.atlassian.confluence.setup.ConfluenceEncodingFilter.getEncodingInternal(ConfluenceEncodingFilter.java:35)  
com.atlassian.confluence.setup.ConfluenceEncodingFilter.getEncoding(ConfluenceEncodingFilter.java:27)  
com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:38)  
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)  
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)  
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:65)

**root cause**

java.lang.NullPointerException  
com.atlassian.spring.container.ContainerManager.getComponent(ContainerManager.java:33)  
com.atlassian.confluence.util.LazyComponentReference$Accessor.get(LazyComponentReference.java:46)  
com.atlassian.util.concurrent.Lazy$Strong.create(Lazy.java:85)  
com.atlassian.util.concurrent.LazyReference$Sync.run(LazyReference.java:321)  
com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:143)  
com.atlassian.util.concurrent.LazyReference.get(LazyReference.java:112)  
com.atlassian.confluence.setup.ConfluenceEncodingFilter.getGlobalSettings(ConfluenceEncodingFilter.java:45)  
com.atlassian.confluence.setup.ConfluenceEncodingFilter.getEncodingInternal(ConfluenceEncodingFilter.java:35)  
com.atlassian.confluence.setup.ConfluenceEncodingFilter.getEncoding(ConfluenceEncodingFilter.java:27)  
com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:38)  
com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)  
com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)  
com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:65)

**Log files:**

One checking confluence logs generated at /opt/atlassian-confluence-5.4.3/logs/

-rw-r--r-- 1 confluence confluence 2794106 Apr 26 12:22 catalina.out  
-rw-r--r-- 1 confluence confluence 34062 Apr 26 12:22 catalina.2017-04-26.log

It shows following errors:

Apr 26, 2017 12:22:59 PM org.apache.catalina.core.StandardHostValve custom  
SEVERE: Exception Processing ErrorPage\[errorCode=500, location=/500page.jsp\]  
java.lang.NullPointerException  
at com.atlassian.spring.container.ContainerManager.getComponent(ContainerManager.java:33)  
at com.atlassian.confluence.setup.sitemesh.PluginDecoratorMapper.getPluginAccessor(PluginDecoratorMapper.java:74)  
at com.atlassian.confluence.setup.sitemesh.PluginDecoratorMapper.getDecorator(PluginDecoratorMapper.java:23)  
at com.opensymphony.module.sitemesh.mapper.AbstractDecoratorMapper.getDecorator(AbstractDecoratorMapper.java:45)  
at com.opensymphony.module.sitemesh.mapper.InlineDecoratorMapper.getDecorator(InlineDecoratorMapper.java:41)  
at com.atlassian.confluence.util.profiling.ConfluenceDecoratorSelector.selectDecorator(ConfluenceDecoratorSelector.java:58)  
at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:83)  
at com.atlassian.confluence.util.profiling.ProfilingPageFilter.doFilter(ProfilingPageFilter.java:44)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:71)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)  
at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:211)  
at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:27)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)  
at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:150)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:71)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:71)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)  
at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:50)  
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)  
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:646)  
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:438)  
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:374)  
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:302)  
at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:421)  
at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:342)  
at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:286)  
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)  
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)  
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)  
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)  
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)  
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)  
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)  
at java.lang.Thread.run(Thread.java:745)

**What all things I've tried:**

1) I verified that I can connect to MySql DB.

A snapshot of mysql process.

root 29010 1 0 Apr21 ? 00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/inmyzd0033.in.dst.ibm.com.pid  
mysql 29224 29010 0 Apr21 ? 00:04:33 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/lib/mysql/inmyzd0033.in.dst.ibm.com.pid --socket=/var/lib/mysql/mysql.sock

2) I checked the permission level for all directories, sub directories and files under Confluence home /opt/ConfData/

<br />

drwxr-xr-x 3 confluence confluence 4096 Feb 27 2014 attachments  
drwxr-xr-x 2 confluence confluence 4096 May 30 2016 backups  
drwxr-xr-x 2 confluence confluence 4096 Feb 27 2014 bundled-plugins  
drwxr-xr-x 2 confluence confluence 4096 Feb 27 2014 config  
-rw-r--r-- 1 confluence confluence 2764 Apr 24 19:40 confluence.cfg.xml  
drwxr-xr-x 2 confluence confluence 4096 May 13 2015 imgEffects  
drwxr-xr-x 4 confluence confluence 4096 Apr 24 19:40 index  
drwxr-xr-x 3 confluence confluence 4096 Apr 21 17:19 logs  
drwxr-xr-x 2 confluence confluence 4096 Apr 24 19:40 plugins-cache  
drwxr-xr-x 5 confluence confluence 4096 Apr 24 17:34 plugins-osgi-cache  
drwxr-xr-x 2 confluence confluence 4096 Apr 22 01:48 plugins-temp  
drwxr-xr-x 2 confluence confluence 4096 Feb 27 2014 restore  
drwxr-xr-x 2 confluence confluence 4096 Apr 5 18:57 temp  
drwxr-xr-x 215 confluence confluence 4096 Mar 8 14:50 thumbnails  
drwxr-xr-x 3 confluence confluence 20480 Feb 3 15:46 viewfile  
drwxr-xr-x 2 confluence confluence 36864 Apr 24 19:39 webresource-temp

3) Cleared up cache and then restarted confluence app i.e. Cleared following directories -

bundled-plugins, plugins-cache, plugins-osgi-cache, plugins-temp

Appreciate if someone could assist me resolving this issues.

Thanks in advance.

- Jaydeep


Answers:
author: Nic Brough -Adaptavist-
body: There's lots of things that can cause that error. You have ruled a number of them out with your investigations, and you mention changes to storage which I suspect are the root cause.  

The usual culprit in my notes are:

1. Trying to run Confluence with the wrong version of Java
2. The user running Confluence does not have the right permissions on the confluence-install and/or confluence-home directories
3. You've moved the Confluence home directory
4. There's a corrupted cache
5. There's a corrupted index
6. The database is inaccessible or has the wrong permissions for the user

I reckon it's 2 or possibly 3 from what you've said.

comments: author: Jaydeep Sarkar
body: Thanks Nic for your response.

Aparently -

Reg your point # 2:

I checked the permission level of home directory:

drwxr-xr-x 3 confluence confluence 4096 Feb 27 2014 attachments  
drwxr-xr-x 2 confluence confluence 4096 May 30 2016 backups  
drwxr-xr-x 2 confluence confluence 4096 Feb 27 2014 bundled-plugins  
drwxr-xr-x 2 confluence confluence 4096 Feb 27 2014 config  
-rw-r--r-- 1 confluence confluence 2764 Apr 24 19:40 confluence.cfg.xml  
drwxr-xr-x 2 confluence confluence 4096 May 13 2015 imgEffects  
drwxr-xr-x 4 confluence confluence 4096 Apr 24 19:40 index  
drwxr-xr-x 3 confluence confluence 4096 Apr 21 17:19 logs  
drwxr-xr-x 2 confluence confluence 4096 Apr 24 19:40 plugins-cache  
drwxr-xr-x 5 confluence confluence 4096 Apr 24 17:34 plugins-osgi-cache  
drwxr-xr-x 2 confluence confluence 4096 Apr 22 01:48 plugins-temp  
drwxr-xr-x 2 confluence confluence 4096 Feb 27 2014 restore  
drwxr-xr-x 2 confluence confluence 4096 Apr 5 18:57 temp  
drwxr-xr-x 215 confluence confluence 4096 Mar 8 14:50 thumbnails  
drwxr-xr-x 3 confluence confluence 20480 Feb 3 15:46 viewfile  
drwxr-xr-x 2 confluence confluence 36864 Apr 24 19:39 webresource-temp

Do you notice any issue here?

Reg point 3:

Additional storage capacity was added to the partition hostingteh file system /dev/sda2.

And Confluence home directory still sitting in that FS.

\[confluence@inmyzd0033 ConfData\]$ df -h /opt/ConfData/  
Filesystem Size Used Avail Use% Mounted on  
/dev/sda2 2.0T 811G 1.2T 41% /

<br />

So not sure if 2 \& 3 can be responsible.

Do you think I should seomething else as well?

Thanks again.


author: Nic Brough -Adaptavist-
body: You might want to try "df -i" as well when checking the size of the disk.

I assume confluence is being run as a user also called "confluence", who is the owner you've got for the files in home, so that's ok, but you need to check the ownership and access of all the files beneath there AND, as I said before, the confluence installation.


author: Jaydeep Sarkar
body: Thanks Nic.

This is what I see.

\[confluence@inmyzd0033 \~\]$ df -i  
Filesystem Inodes IUsed IFree IUse% Mounted on  
/dev/sda2 133685280 306728 133378552 1% /  
tmpfs 749316 6 749310 1% /dev/shm  
/dev/sda1 50200 38 50162 1% /boot  
/dev/sdb1 67108864 11 67108853 1% /data


author: Jaydeep Sarkar
body: Also, the installation directory is /opt/atlassian-confluence-5.4.3 and it has following permission level:

drwxr-xr-x 11 confluence confluence 4096 Feb 12 2014 atlassian-confluence-5.4.3

<br />

And whatever comes inside thsi installation directory is as shown below. Permission wise I don't see any issue.

<br />

The ownnership of all files and folders inside home /opt/ConfData and installation was changed by

chown -R confluence:confluence \*

But still the problem persists.

In the fist response you said-

Trying to run Confluence with the wrong version of Java  
The user running Confluence does not have the right permissions on the confluence-install and/or confluence-home directories  
You've moved the Confluence home directory  
There's a corrupted cache  
There's a corrupted index  
The database is inaccessible or has the wrong permissions for the user

How do we address point # 5? Could it be because of this?

<br />

Thanks


author: Jaydeep Sarkar
body: Some additional information:

Please note that it was working fine till 5th/6th April. However after additional storage capacity was added, although the application is running but throwing those errors

Thanks!


Tags: http