Confluence failed to start after upgrade
Asked by Sascha Z?nker:

Hey there,

i've got Problems to start my confluence after upgrading from 8.13.0 to 8.14.1

The catalina.out log file tells me:

WARNING: An illegal reflective access operation has occurred  
WARNING: Illegal reflective access by com.atlassian.hibernate.adapter.proxy.BytecodeProviderImpl_ImplementV2Proxy (file:/usr/confluence/confluence/WEB-INF/lib/hibernate.adapter-1.0.3.jar) to field java.lang.reflect.Field.modifiers  
WARNING: Please consider reporting this to the maintainers of com.atlassian.hibernate.adapter.proxy.BytecodeProviderImpl_ImplementV2Proxy  
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations  
WARNING: All illegal access operations will be denied in a future release  
26-Jan-2021 20:20:40.042 SCHWERWIEGEND \[Catalina-utility-1\] org.apache.catalina.core.StandardContext.startInternal One or more Filters failed to start. Full details will be found in the appropriate container log file  
26-Jan-2021 20:20:40.043 SCHWERWIEGEND \[Catalina-utility-1\] org.apache.catalina.core.StandardContext.startInternal Context \[\] startup failed due to previous errors  
26-Jan-2021 20:20:40.165 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[com.mysql.jdbc.Driver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.166 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[com.mysql.fabric.jdbc.FabricMySQLDriver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.166 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[com.microsoft.sqlserver.jdbc.SQLServerDriver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.166 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[com.github.gquintana.metrics.sql.Driver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.166 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[org.h2.Driver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.166 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[org.postgresql.Driver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.167 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[net.sourceforge.jtds.jdbc.Driver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.167 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application \[ROOT\] registered the JDBC driver \[org.hsqldb.jdbc.JDBCDriver\] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.  
26-Jan-2021 20:20:40.168 WARNUNG \[Catalina-utility-1\] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application \[ROOT\] appears to have started a thread named \[mysql-cj-abandoned-connection-cleanup\] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:  
java.base@11.0.8/java.lang.Object.wait(Native Method)  
java.base@11.0.8/java.lang.ref.ReferenceQueue.remove(Unknown Source)  
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:80)  
java.base@11.0.8/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)  
java.base@11.0.8/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)  
java.base@11.0.8/java.lang.Thread.run(Unknown Source)  
26-Jan-2021 20:20:40.184 INFORMATION \[main\] org.apache.catalina.startup.Catalina.start Server startup in \[23447\] milliseconds  
26-Jan-2021 20:20:44.136 INFORMATION \[mysql-cj-abandoned-connection-cleanup\] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load \[\]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.  
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load \[\]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.  
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1385)  
at org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1038)  
at com.mysql.jdbc.AbandonedConnectionCleanupThread.checkThreadContextClassLoader(AbandonedConnectionCleanupThread.java:112)  
at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:79)  
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)  
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)  
at java.base/java.lang.Thread.run(Unknown Source)

Can you please help?

Thanks in advance


Answers:
author: Pavel Junek
body: Hi [@Sascha Z?nker](/t5/user/viewprofilepage/user-id/2673295),

You probably mistaken the version numbering for Jira and Confluence - Confluence is now in version 7.10.2. Version 8.14.1 fits me rather on Jira :)

Can you please attach the last extract from atlassian-confluence.log here? What DB do you use and what OS? Do you have Apache / NGINX before Confluence? Do you use SSL?

Thank you

Pavel

comments: author: Sascha Z?nker
body: Hi [@Pavel Junek](/t5/user/viewprofilepage/user-id/1157350)

Oh i did sorry\^\^

Well then i've got the 7.10.2 and before it was 7.8.0 i think.

Can you tell me where to find these Log File? I already searched for it but dont find some interesting Log Files. Im using MariaDB 10.5 (i know its not supported but it workes very well until today) and my OS is Debian 10 Buster. I also use Apache2 as ReverseProxy in Front of confluence and my Apache is also working fine with SSL with my other Stuff.


author: Daniel Ebers
body: The application log (atlassian-confluence.log) could also be interesting, please refer to [Working with Confluence logs](https://confluence.atlassian.com/doc/working-with-confluence-logs-108364721.html) in order to identify the place where they reside on your disk.  

Which database driver (JDBC) did you download/use for the database connection - which version number?


author: Sascha Z?nker
body: Hi [@Daniel Ebers](/t5/user/viewprofilepage/user-id/800113)

I will Check the logs Files soon as possible.

My JDBC Driver got the Version 5.1.49.


author: Sascha Z?nker
body: Well i found it. Seems to be a Database Error. I only post the last 30 Lines hope this is enoght:

at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeLocalJob(CaesiumSchedulerService.java:402)  
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:380)  
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)  
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)  
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)  
at java.base/java.lang.Thread.run(Unknown Source)  
Caused by: javax.persistence.PersistenceException: Failed to update database schema  
at com.atlassian.migration.agent.store.jpa.impl.LiquibaseSchemaUpgrader.upgrade(LiquibaseSchemaUpgrader.java:35)  
at com.atlassian.migration.agent.store.jpa.impl.DefaultSessionFactorySupplier.buildSessionFactory(DefaultSessionFactorySupplier.java:70)  
at com.atlassian.util.concurrent.Lazy$Strong.create(Lazy.java:85)  
at com.atlassian.util.concurrent.LazyReference$Sync.run(LazyReference.java:325)at com.atlassian.util.concurrent.LazyReference.getInterruptibly(LazyReference.java:143)  
at com.atlassian.util.concurrent.LazyReference.get(LazyReference.java:112)  
at com.atlassian.migration.agent.store.jpa.impl.DefaultSessionFactorySupplier.get(DefaultSessionFactorySupplier.java:65)  
at com.atlassian.migration.agent.store.jpa.impl.DefaultSessionFactorySupplier.get(DefaultSessionFactorySupplier.java:40)  
at com.atlassian.migration.agent.store.jpa.impl.DefaultPluginTransactionTemplate.on(DefaultPluginTransactionTemplate.java:29)  
at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.read(PluginTransactionTemplate.java:16)  
at com.atlassian.migration.agent.service.impl.DefaultStatisticsService.loadSpaceStatistics(DefaultStatisticsService.java:138)  
at com.atlassian.migration.agent.service.impl.DefaultStatisticsService.calculateServerStats(DefaultStatisticsService.java:167)  
at com.atlassian.migration.agent.service.impl.SingleJobExecutor.lambda$execute$0(SingleJobExecutor.java:33)  
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)  
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)  
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)... 1 more  
Caused by: liquibase.exception.LockException: Could not acquire change log lock. Currently locked by 172.17.0.1 (172.17.0.1) since 26.02.19 14:04  
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:230)  
at liquibase.Liquibase.update(Liquibase.java:184)  
at liquibase.Liquibase.update(Liquibase.java:179)  
at com.atlassian.migration.agent.store.jpa.impl.LiquibaseSchemaUpgrader.upgrade(LiquibaseSchemaUpgrader.java:30)  
... 16 more


author: Daniel Ebers
body: Is there any chance something else is having an open connection to DB?


author: Sascha Z?nker
body: Well i dont know why but after i stopped jira and then started confluence again everything seems to be fine.


author: Pavel Junek
body: Great, thanks for the info :)


Tags: atl-gruland

confluence-server

server

upgrade