Upgrade to 7.x throws 500 errors on all pages
Asked by Andrew Smith:

Hi guys,

I've spent the entire weekend trying to upgrade from 6.15.9 to 7.0.1 and no matter which way I go about it, it seems to upgrade the database over several minutes, then every page load (login.action, authenticate.action, pulp-render.action) return 500 errors.

It shows absolutely nothing in catalina.out

How do I progress with debugging this issue, as I obviously can't get to the control panel to extract a support payload either?

Thanks,

Andrew


Answers:
author: Ram Bhusal
body: We had similar issues after reboot. The following actions could help.

1. check MySQL or PostgreSQL is up and running.
2. check httpd is up and running. ---- This is what was missing on my end.
   1. Had done additional steps since this was not up after reboot.
      1. Check'
         1. systemctl is-enabled httpd
         2. service httpd status
      2. To Enable
         1. systemctl enable httpd
      3. check and start httpd if not running.
3. ping domain to ensure you are connected to network.
4. final step start confluence

Hope this helps

Good Luck. :)


author: Nic Brough -Adaptavist-
body: What does the application log say the problem is? (Jira home/logs/atlassian-jira.log is the main one although others in that directory might tell you something too)

Have you changed the old location of Jira in the server.xml with the "context" directive? has that been replaced in the server.xml file during upgrade?

comments: author: Andrew Smith
body: Yep all empty.

Tried upgrading with an unchanged server.xml, as well as trying with the modified (nginx https proxy), both failed and threw 500's no matter which page I navigated to


author: Nic Brough -Adaptavist-
body: If the application log is empty, then the service is not running. If the service starts and then halts without writing to the application log, then there will be an error written into the catalina.out, or an error thrown on to screen when you run the start up script.


author: Andrew Smith
body: I'll try again right now, sec.


author: Andrew Smith
body: Hi Nic,

I have upgraded to 7.0.1 once again, the atlassian-jira.log is now populated, I seem to be receiving "every plugin we have" throwing something that looks very similar to this

2019-09-29 11:11:59,281 ERROR \[ThreadPoolAsyncTaskExecutor::Thread 1\] \[internal.dependencies.startup.DependencyWaiterApplicationContextExecutor\] fail Unable to create application context for \[com.gliffy.plugin.confluence.gliffy-confluence-plugin\], unsatisfied dependencies: none  
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'diagramManager' defined in URL \[bundle://290.0:0/META-INF/spring/atlassian-plugins-components.xml\]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate \[com.gliffy.plugin.confluence.diagram.ConfluenceDiagramManager\]: Constructor threw exception; nested exception is java.lang.NullPointerException


author: Andrew Smith
body: 2019-09-29 11:06:50,917 ERROR \[Catalina-utility-1\] \[plugin.osgi.factory.OsgiPlugin\] enableInternal Detected an error (BundleException) enabling the plugin 'com.atlassian.confluence.plugins.confluence-questions' : Unable to resolve com.atlassian.confluence.plugins.confluence-questions \[267\](R 267.0): missing requirement \[com.atlassian.confluence.plugins.confluence-questions \[267\](R 267.0)\] osgi.wiring.package; (\&(osgi.wiring.package=com.atlassian.plugin.util.collect)(version\>=3.0.7)) Unresolved requirements: \[\[com.atlassian.confluence.plugins.confluence-questions \[267\](R 267.0)\] osgi.wiring.package; (\&(osgi.wiring.package=com.atlassian.plugin.util.collect)(version\>=3.0.7))\]. This error usually occurs when your plugin imports a package from another bundle with a specific version constraint and either the bundle providing that package doesn't meet those version constraints, or there is no bundle available that provides the specified package. For more details on how to fix this, see <https://developer.atlassian.com/x/mQAN>  
2019-09-29 11:06:50,917 WARN \[Catalina-utility-1\] \[atlassian.plugin.impl.AbstractPlugin\] enable Unable to enable plugin 'com.atlassian.confluence.plugins.confluence-questions'  
2019-09-29 11:06:50,918 WARN \[Catalina-utility-1\] \[atlassian.plugin.impl.AbstractPlugin\] enable Because of this exception  
com.atlassian.plugin.osgi.container.OsgiContainerException: Cannot start plugin: com.atlassian.confluence.plugins.confluence-questions  
at com.atlassian.plugin.osgi.factory.OsgiPlugin.enableInternal(OsgiPlugin.java:424)  
at com.atlassian.plugin.impl.AbstractPlugin.enable(AbstractPlugin.java:252)  
at com.atlassian.plugin.manager.PluginEnabler.actualEnable(PluginEnabler.java:120)  
at com.atlassian.plugin.manager.PluginEnabler.enable(PluginEnabler.java:97)  
at com.atlassian.plugin.manager.DefaultPluginManager.lambda$enableDependentPlugins$22(DefaultPluginManager.java:1246)  
at com.atlassian.plugin.manager.PluginTransactionContext.wrap(PluginTransactionContext.java:63)  
at com.atlassian.plugin.manager.DefaultPluginManager.enableDependentPlugins(DefaultPluginManager.java:1230)  
at com.atlassian.plugin.manager.DefaultPluginManager.lambda$addPlugins$20(DefaultPluginManager.java:1215)  
at com.atlassian.plugin.manager.PluginTransactionContext.wrap(PluginTransactionContext.java:63)  
at com.atlassian.plugin.manager.DefaultPluginManager.addPlugins(DefaultPluginManager.java:1115)  
at com.atlassian.plugin.manager.DefaultPluginManager.lambda$lateStartup$4(DefaultPluginManager.java:652)  
at com.atlassian.plugin.manager.PluginTransactionContext.wrap(PluginTransactionContext.java:63)  
at com.atlassian.plugin.manager.DefaultPluginManager.lateStartup(DefaultPluginManager.java:645)  
at com.atlassian.confluence.plugin.ConfluencePluginManager.lateStartup(ConfluencePluginManager.java:149)  
at com.atlassian.confluence.plugin.PluginFrameworkContextListener.contextInitialized(PluginFrameworkContextListener.java:81)  
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4682)  
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5143)  
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)  
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)  
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)  
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)  
at java.lang.Thread.run(Thread.java:748)


author: Andrew Smith
body: Caused by: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getConfluenceResourceTags' in class com.atlassian.confluence.setup.velocity.VelocityFriendlyPageBuilderService threw exception com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at /decorators/includes/header.vm\[line 92, column 24\]  
at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:342)  
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:284)  
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:262)  
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:342)  
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)  
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)  
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)  
at org.apache.velocity.runtime.directive.Parse.render(Parse.java:263)  
at com.atlassian.confluence.setup.velocity.ProfilingParseDirective.render(ProfilingParseDirective.java:21)  
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)  
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)  
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:212)  
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:247)  
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)  
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)  
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:212)  
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:247)  
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)  
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)  
at org.apache.velocity.Template.merge(Template.java:328)  
at org.apache.velocity.Template.merge(Template.java:235)  
at com.atlassian.confluence.util.velocity.VelocityUtils.renderTemplateWithoutSwallowingErrors(VelocityUtils.java:70)  
at com.atlassian.confluence.util.velocity.VelocityUtils.renderTemplateWithoutSwallowingErrors(VelocityUtils.java:76)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator$StreamingResponseWritingStrategy.renderToResponse(ConfluenceSitemeshDecorator.java:271)  
... 81 more


author: Andrew Smith
body: The log file seems to now be on a spam loop (hundreds) with the error above.


author: Andrew Smith
body: 2019-09-29 11:37:10,529 INFO \[http-nio-8090-exec-8\] \[atlassian.confluence.status.SystemErrorInformationLogger\] writeToLog  
Request Unique ID : 5ea50e0b-f81f-431d-b911-81f428d34256  
--------------------------  
JVM Stats  
--------------------------  
allocatedHeap = 1073741824  
freeAllocatedHeap = 486186568  
availablePermGen = 0  
maxPermGen = -1  
availableHeap = 486186568  
maxHeap = 1073741824  
usedHeap = 587555256  
usedPermGen = -1  
--------------------------  
Request Information  
--------------------------  
URL: <https://confluence.heffron.com.au/500page.jsp>  
Scheme: https  
Server: confluence.heffron.com.au  
Port: 443  
URI: /500page.jsp  
Context Path:  
Servlet Path: /500page.jsp  
Path Info: null  
Query String: null  
--------------------------  
Attributes  
--------------------------  
javax.servlet.forward.request_uri: /  
javax.servlet.forward.context_path:  
javax.servlet.forward.servlet_path: /index.action  
javax.servlet.forward.mapping: org.apache.catalina.core.ApplicationMapping$MappingImpl@404d8de2  
javax.servlet.error.status_code: 500  
com.atlassian.confluence.impl.profiling.DecoratorTimings: com.atlassian.confluence.impl.profiling.DecoratorTimings@7a6890f  
com.opensymphony.sitemesh.APPLIED_ONCE: true  
__cleanup_recursion_counter: 0  
webwork.request_uri: /  
atlas.webInterfaceContext: com.atlassian.confluence.plugin.descriptor.web.DefaultWebInterfaceContext@4cb3d914  
com.atlassian.confluence.util.message.MessagesDecoratorFilter__already_filtered__: true  
com.atlassian.confluence.web.filter.validateparam.RequestParamValidationFilter_already_filtered: true  
atlassian.core.seraph.original.url: /500page.jsp  
theme.resource.path: /download/resources/com.refinedwiki.confluence.plugins.theme.original%3Amain/  
com.atlassian.labs.botkiller.BotKillerFilter: true  
com.atlassian.gzipfilter.GzipFilter_already_filtered: true  
Confluence-Request-Time: 1569757030075  
com.atlassian.confluence.web.ConfluenceJohnsonFilter_already_filtered: true  
com.atlassian.seraph.auth.LoginReason: OK  
__sitemesh__page: com.opensymphony.sitemesh.compatability.Content2HTMLPage@60731d0d  
webwork.valueStack: com.opensymphony.xwork.util.OgnlValueStack@4c7dae7c  
confluence.themecontext: com.atlassian.confluence.themes.ThemeContext@7f20b80d  
webwork.view_uri: /dashboard.action  
javax.servlet.error.message:  
javax.servlet.error.servlet_name: action  
com.refinedwiki.confluence.plugins.theme.original.filters.StartSiteFilter_already_filtered: true  
com.opensymphony.sitemesh.USINGSTREAM: false  
com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator: true  
B3-TraceId: c5a569a595dd72  
loginfilter.already.filtered: true  
javax.servlet.error.request_uri: /  
com.atlassian.core.filters.HeaderSanitisingFilter_already_filtered: true  
com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFixupFilter: true  
javax.servlet.error.exception: java.lang.RuntimeException: javax.servlet.ServletException: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getConfluenceResourceTags' in class com.atlassian.confluence.setup.velocity.VelocityFriendlyPageBuilderService threw exception com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at /decorators/includes/header.vm\[line 92, column 24\]  
os_securityfilter_already_filtered: true  
com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter: true  
--------------------------  
Parameters  
--------------------------  
caused by: java.lang.RuntimeException: javax.servlet.ServletException: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getConfluenceResourceTags' in class com.atlassian.confluence.setup.velocity.VelocityFriendlyPageBuilderService threw exception com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at /decorators/includes/header.vm\[line 92, column 24\]  
at com.opensymphony.sitemesh.webapp.decorator.BaseWebAppDecorator.render(BaseWebAppDecorator.java:37)  
caused by: javax.servlet.ServletException: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getConfluenceResourceTags' in class com.atlassian.confluence.setup.velocity.VelocityFriendlyPageBuilderService threw exception com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at /decorators/includes/header.vm\[line 92, column 24\]  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator$ErrorSendingErrorHandlingStrategy.handleException(ConfluenceSitemeshDecorator.java:224)  
caused by: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getConfluenceResourceTags' in class com.atlassian.confluence.setup.velocity.VelocityFriendlyPageBuilderService threw exception com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at /decorators/includes/header.vm\[line 92, column 24\]  
at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:342)  
caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException  
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)  
caused by: java.lang.NullPointerException  
at com.comalatech.confluence.adhoclists.condition.ConfluenceBuildNumberCondition.shouldDisplay(ConfluenceBuildNumberCondition.java:25)

2019-09-29 11:37:10,601 WARN \[http-nio-8090-exec-8\] \[webresource\] evaluateSafely exception thrown in \`shouldDisplayImmediate\` during condition evaluation  
-- traceId: 39f48d4edf18096f  
java.lang.NullPointerException  
at com.comalatech.confluence.adhoclists.condition.ConfluenceBuildNumberCondition.shouldDisplay(ConfluenceBuildNumberCondition.java:25)  
at com.atlassian.plugin.webresource.condition.DecoratingLegacyCondition.shouldDisplayImmediate(DecoratingLegacyCondition.java:53)  
at com.atlassian.plugin.webresource.impl.CachedCondition.evaluateSafely(CachedCondition.java:49)  
at com.atlassian.plugin.webresource.impl.helpers.UrlGenerationHelpers.buildIncludedExcludedConditionsAndBatchingOptions(UrlGenerationHelpers.java:248)  
at com.atlassian.plugin.webresource.impl.helpers.UrlGenerationHelpers.resolve(UrlGenerationHelpers.java:204)  
at com.atlassian.plugin.webresource.assembler.DefaultWebResourceAssembler$1.resolve(DefaultWebResourceAssembler.java:87)  
at com.atlassian.plugin.webresource.assembler.DefaultWebResourceAssembler$1.drainOrPoll(DefaultWebResourceAssembler.java:76)  
at com.atlassian.plugin.webresource.assembler.DefaultWebResourceAssembler$1.drainIncludedResources(DefaultWebResourceAssembler.java:67)  
at com.atlassian.confluence.plugin.webresource.DefaultConfluenceWebResourceService.writeConfluenceResourceTags(DefaultConfluenceWebResourceService.java:129)  
at com.atlassian.confluence.setup.velocity.VelocityFriendlyPageBuilderService.getConfluenceResourceTags(VelocityFriendlyPageBuilderService.java:104)  
at sun.reflect.GeneratedMethodAccessor1015.invoke(Unknown Source)  
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
at java.lang.reflect.Method.invoke(Method.java:498)  
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:385)  
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:374)  
at com.atlassian.velocity.htmlsafe.introspection.AnnotationBoxingMethod.invoke(AnnotationBoxingMethod.java:24)  
at com.atlassian.velocity.htmlsafe.introspection.UnboxingMethod.invoke(UnboxingMethod.java:28)  
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:270)  
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:262)  
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:342)  
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)  
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)  
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)  
at org.apache.velocity.runtime.directive.Parse.render(Parse.java:263)  
at com.atlassian.confluence.setup.velocity.ProfilingParseDirective.render(ProfilingParseDirective.java:21)  
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)  
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)  
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:212)  
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:247)  
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)  
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)  
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:212)  
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:247)  
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)  
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)  
at org.apache.velocity.Template.merge(Template.java:328)  
at org.apache.velocity.Template.merge(Template.java:235)  
at com.atlassian.confluence.util.velocity.VelocityUtils.renderTemplateWithoutSwallowingErrors(VelocityUtils.java:70)  
at com.atlassian.confluence.util.velocity.VelocityUtils.renderTemplateWithoutSwallowingErrors(VelocityUtils.java:76)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator$StreamingResponseWritingStrategy.renderToResponse(ConfluenceSitemeshDecorator.java:271)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator.applyDecoratorUsingVelocity(ConfluenceSitemeshDecorator.java:198)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator.applyDecorator(ConfluenceSitemeshDecorator.java:138)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshErrorDecorator.renderInternal(ConfluenceSitemeshErrorDecorator.java:44)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator.lambda$render$0(ConfluenceSitemeshDecorator.java:100)  
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshDecorator.render(ConfluenceSitemeshDecorator.java:98)  
at com.atlassian.confluence.util.profiling.ConfluenceSitemeshErrorDecorator.render(ConfluenceSitemeshErrorDecorator.java:36)  
at com.opensymphony.sitemesh.webapp.decorator.BaseWebAppDecorator.render(BaseWebAppDecorator.java:33)  
at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:84)  
at com.atlassian.confluence.util.profiling.ProfilingSiteMeshFilter.doFilter(ProfilingSiteMeshFilter.java:50)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)  
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.lambda$doFilter$0(DelegatingPluginFilter.java:57)  
at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:56)  
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:62)  
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:43)  
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:50)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:242)  
at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.applyFilter(ConfluenceSecurityFilter.java:40)  
at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:29)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:148)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)  
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.lambda$doFilter$0(DelegatingPluginFilter.java:57)  
at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:56)  
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:62)  
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:43)  
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:50)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39)  
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.lambda$doFilter$0(DelegatingPluginFilter.java:57)  
at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:56)  
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:62)  
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)  
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:43)  
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:50)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:59)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:59)  
at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)  
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)  
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)  
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)  
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712)  
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)  
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:384)  
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)  
at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:394)  
at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:253)  
at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:348)  
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:173)  
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)  
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)  
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)  
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)  
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)  
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853)  
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)  
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)  
at java.lang.Thread.run(Thread.java:748)


author: Nic Brough -Adaptavist-
body: Ok, have you upgraded all the apps as well as the core? The errors are happening during the load of the add-ons, so I suspect the existing ones are out of date and crashing when loading.

If you can't get to them, then try to restart it without them. Stop Confluence, go to \<confluence home\>/plugins and find all the .jar files in there. Move them somewhere safe and outside Confluence (like a new directory in your home directory) and restart it, this should enable Confluence to start up ok, then you can use "manage add ons" to add the later versions back in (it will retain the data from the old ones)


author: Andrew Smith
body: Yep, definitely updated all the plugins to the latest prior to the upgrade.

Strangely there is no plugins directory here, nor is there on the datacentre shared home. The bundled-plugins folder is empty.

\[root@confluence01 confluence\]# pwd  
/var/atlassian/application-data/confluence  
\[root@confluence01 confluence\]# ls -lha  
total 60M  
drwx------ 21 confluence confluence 4.0K Sep 29 14:06 .  
drwxr-xr-x 3 confluence confluence 4.0K Sep 29 04:05 ..  
drwx------ 2 confluence confluence 4.0K Mar 22 2018 analytics-logs  
drwx------ 3 confluence confluence 4.0K Mar 22 2018 attachments  
drwxr-x--- 2 confluence confluence 4.0K Sep 29 12:45 bundled-plugins  
-rw-r----- 1 confluence confluence 6.2K Sep 29 14:06 confluence.cfg.xml  
drwx------ 2 confluence confluence 4.0K Aug 29 2017 export  
drwx------ 2 confluence confluence 4.0K Feb 2 2018 imgEffects  
drwx------ 4 confluence confluence 12K Sep 30 03:42 index  
drwx------ 2 confluence confluence 4.0K Sep 30 03:42 journal  
-rw------- 1 confluence confluence 0 Aug 14 2017 lock  
drwx------ 3 confluence confluence 4.0K Sep 29 12:12 logs  
drwx------ 5 confluence confluence 4.0K Mar 22 2018 originaltheme  
drwxr-x--- 2 confluence confluence 4.0K Sep 29 14:11 plugins-cache  
drwxr-x--- 5 confluence confluence 4.0K Sep 29 12:45 plugins-osgi-cache  
drwxr-x--- 2 confluence confluence 4.0K Sep 29 12:45 plugins-temp  
drwx------ 2 confluence confluence 4.0K Mar 22 2018 recovery  
drwx------ 21 confluence confluence 4.0K Sep 30 00:05 sandbox  
drwx------ 5 confluence confluence 4.0K Mar 22 2018 shared-home  
-rw-r----- 1 confluence confluence 60M Sep 29 14:05 synchrony-standalone.jar  
drwx------ 2 confluence confluence 16K Sep 30 02:01 temp  
drwx------ 14 confluence confluence 4.0K Mar 22 2018 thumbnails  
drwx------ 3 confluence confluence 4.0K Feb 2 2018 viewfile  
drwx------ 2 confluence confluence 20K Sep 30 03:42 webresource-temp


author: Nick Mancuso
body: For those like me who came to this exact post only to not find the solution... here it is!

If you are running Confluence 6.1 and later you can run the following to start confluence up without any plugins

1. stop you confluence instance

2. go to your /opt/atlassian/confluence/bin folder and run

    ./start-confluence.sh --disable-all-addons

When it starts up you can then go into the admin interface and see which plugins were causing the error

More details about disabling can be found here: <https://confluence.atlassian.com/confkb/how-to-manually-remove-malfunctioning-add-ons-113705118.html>


author: Pavel Junek
body: Hi [@Andrew Smith](/t5/user/viewprofilepage/user-id/2994877),

Welcome to Atlassian community!

I will try help you with your problem.

* Which database using?
* Is something written in {confluence-home}/log/atlassian-confluence.log file?
* Did you keep the upgrade running long enough?

Thank you!

Pavel

comments: author: Andrew Smith
body: Hi there,

Yes ran the upgrade for 6 hours. top showed Java using \< 1% cpu by then.

atlassian-confluence.log empty

postgresql


author: Pavel Junek
body: I'm still thinking - do you have folder permissions set to *confluence:confluence* for {cofluence-install} and {confluence-home} folder?


author: Andrew Smith
body: Most certainly do yes


Tags: confluence-server

server