Confluence is no longer starting up
Asked by Jamie Edwards:

Greetings

I powered down our VE and started it up again, and no Confluence is no longer starting up.

I have tried:

* Clearing the confluence-data caches/plugin files
* Checked permissions
* Database is fine

catalinia.out:

```
-bash-3.2# cat catalina.outMar 1, 2014 9:00:31 PM org.apache.tomcat.util.digester.SetPropertiesRule beginWARNING: [SetPropertiesRule]{Server} Setting property 'debug' to '0' did not find a matching property.Mar 1, 2014 9:00:31 PM org.apache.tomcat.util.digester.SetPropertiesRule beginWARNING: [SetPropertiesRule]{Server/Service/Engine} Setting property 'debug' to '0' did not find a matching property.Mar 1, 2014 9:00:31 PM org.apache.tomcat.util.digester.SetPropertiesRule beginWARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'debug' to '0' did not find a matching property.Mar 1, 2014 9:00:31 PM org.apache.tomcat.util.digester.SetPropertiesRule beginWARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property.Mar 1, 2014 9:00:31 PM org.apache.coyote.http11.Http11Protocol initINFO: Initializing Coyote HTTP/1.1 on http-8080Mar 1, 2014 9:00:31 PM org.apache.catalina.startup.Catalina loadINFO: Initialization processed in 409 msMar 1, 2014 9:00:31 PM org.apache.catalina.core.StandardService startINFO: Starting service Tomcat-StandaloneMar 1, 2014 9:00:31 PM org.apache.catalina.core.StandardEngine startINFO: Starting Servlet Engine: Apache Tomcat/6.0.322014-03-01 21:00:38,056 INFO [main] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 3.5.6 (build #2162)2014-03-01 21:00:42,585 INFO [main] [atlassian.plugin.manager.DefaultPluginManager] init Initialising the plugin system2014-03-01 21:00:43,024 INFO [main] [atlassian.plugin.manager.DefaultPluginManager] init Plugin system started in 0:00:00.437Mar 1, 2014 9:01:31 PM org.apache.catalina.core.StandardContext startSEVERE: Error listenerStartMar 1, 2014 9:01:32 PM org.apache.catalina.core.StandardContext startSEVERE: Context [] startup failed due to previous errorsMar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbcSEVERE: The web application [] registered the JDBC driver [org.hsqldb.jdbc.JDBCDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbcSEVERE: The web application [] registered the JDBC driver [org.postgresql.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbcSEVERE: The web application [] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.google.common.base.internal.Finalizer] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [Timer-3] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [] appears to have started a thread named [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2] but has failed to stop it. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [] created a ThreadLocal with key of type [com.atlassian.confluence.concurrent.ThreadLocalMap] (value [com.atlassian.confluence.concurrent.ThreadLocalMap@594b7042]) and a value of type [java.util.HashMap] (value [{}]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [] created a ThreadLocal with key of type [com.atlassian.core.logging.ThreadLocalErrorCollection$2] (value [com.atlassian.core.logging.ThreadLocalErrorCollection$2@2a5e8368]) and a value of type [java.lang.Boolean] (value [false]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Mar 1, 2014 9:01:33 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@2b41fedb]) and a value of type [org.dom4j.DocumentFactory] (value [org.dom4j.DocumentFactory@79c2f20c]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.log4j:ERROR LogMananger.repositorySelector was null likely due to error in class reloading, using NOPLoggerRepository.Mar 1, 2014 9:01:34 PM org.apache.coyote.http11.Http11Protocol startINFO: Starting Coyote HTTP/1.1 on http-8080Mar 1, 2014 9:01:34 PM org.apache.catalina.startup.Catalina startINFO: Server startup in 63083 msException in thread "Timer-3" java.lang.NullPointerException        at com.mchange.v2.log.log4j.Log4jMLog$Log4jMLogger.isLoggable(Log4jMLog.java:257)        at com.mchange.v2.resourcepool.BasicResourcePool$CullTask.run(BasicResourcePool.java:1934)        at java.util.TimerThread.mainLoop(Timer.java:512)        at java.util.TimerThread.run(Timer.java:462)
```

atlassian-confluence.log in confluence-data:

```
-bash-3.2# cat atlassian-confluence.log2014-03-01 21:07:49,359 INFO [main] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 3.5.6 (build #2162)2014-03-01 21:07:53,966 INFO [main] [atlassian.plugin.manager.DefaultPluginManager] init Initialising the plugin system2014-03-01 21:07:54,563 INFO [main] [atlassian.plugin.manager.DefaultPluginManager] init Plugin system started in 0:00:00.5942014-03-01 21:07:57,270 INFO [main] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization started2014-03-01 21:08:03,320 INFO [main] [com.atlassian.confluence.lifecycle] &lt;init&gt; Loading EhCache cache manager2014-03-01 21:08:17,758 ERROR [main] [confluence.search.lucene.ConfluenceSearcherInitialisation] initialise Error encountered while warming up searcher - The index has not been upgraded. Please reindex.2014-03-01 21:08:39,110 INFO [main] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization completed in 41839 ms2014-03-01 21:08:40,258 ERROR [main] [sf.hibernate.util.JDBCExceptionReporter] logExceptions Table 'confluence.BANDANA' doesn't exist2014-03-01 21:08:40,260 ERROR [main] [sf.hibernate.util.JDBCExceptionReporter] logExceptions Table 'confluence.BANDANA' doesn't exist2014-03-01 21:08:40,262 ERROR [main] [atlassian.event.internal.EventPublisherImpl] invokeListeners There was an exception thrown trying to dispatch event 'com.atlassian.confluence.event.events.admin.ConfluenceReadyEvent[source=com.atlassian.confluence.upgrade.impl.DefaultUpgradeManager@2d275595]' from the invoker 'com.atlassian.event.legacy.LegacyListenerHandler$LegacyListenerInvoker@2e807f85'.org.springframework.jdbc.BadSqlGrammarException: Hibernate operation: Could not execute query; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'confluence.BANDANA' doesn't existCaused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'confluence.BANDANA' doesn't exist        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)        at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)        at com.mysql.jdbc.Util.getInstance(Util.java:381)        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1051)        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3563)        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3495)        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959)        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2113)        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2693)        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2102)        at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2261)        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76)        at net.sf.hibernate.impl.BatcherImpl.getResultSet(BatcherImpl.java:89)        at net.sf.hibernate.loader.Loader.getResultSet(Loader.java:880)        at net.sf.hibernate.loader.Loader.doQuery(Loader.java:273)        at net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:138)        at net.sf.hibernate.loader.Loader.doList(Loader.java:1063)        at net.sf.hibernate.loader.Loader.list(Loader.java:1054)        at net.sf.hibernate.hql.QueryTranslator.list(QueryTranslator.java:854)        at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1552)        at net.sf.hibernate.impl.QueryImpl.list(QueryImpl.java:49)        at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao$1.doInHibernate(HibernateConfluenceBandanaRecordDao.java:53)        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:337)        at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao.getRecord(HibernateConfluenceBandanaRecordDao.java:45)        at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:30)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)        at $Proxy5.retrieve(Unknown Source)        at com.atlassian.confluence.setup.bandana.ConfluenceCachingBandanaPersister.retrieve(ConfluenceCachingBandanaPersister.java:61)        at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)        at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:24)        at com.atlassian.confluence.plugin.BandanaPluginStateStore.load(BandanaPluginStateStore.java:29)        at com.atlassian.confluence.plugin.ConfluencePluginManager.init(ConfluencePluginManager.java:186)        at com.atlassian.confluence.plugin.ConfluencePluginManager.processConfluenceReadyEvent(ConfluencePluginManager.java:306)        at com.atlassian.confluence.plugin.ConfluencePluginManager.onApplicationEvent(ConfluencePluginManager.java:209)        at org.springframework.context.event.SimpleApplicationEventMulticaster$1.run(SimpleApplicationEventMulticaster.java:77)        at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:75)        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:246)        at com.atlassian.event.legacy.SpringContextEventPublisher.handleEvent(SpringContextEventPublisher.java:22)        at com.atlassian.event.legacy.LegacyListenerHandler$LegacyListenerInvoker.invoke(LegacyListenerHandler.java:55)        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$2.run(AsynchronousAbleEventDispatcher.java:60)        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$1.execute(AsynchronousAbleEventDispatcher.java:28)        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.dispatch(AsynchronousAbleEventDispatcher.java:56)        at com.atlassian.event.internal.EventPublisherImpl.invokeListeners(EventPublisherImpl.java:151)        at com.atlassian.event.internal.EventPublisherImpl.publish(EventPublisherImpl.java:70)        at com.atlassian.event.legacy.LegacyEventManager.publishEvent(LegacyEventManager.java:36)        at com.atlassian.confluence.event.ConfluenceEventManager.publishEvent(ConfluenceEventManager.java:47)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)        at $Proxy10.publishEvent(Unknown Source)        at com.atlassian.confluence.upgrade.impl.DefaultUpgradeManager.upgrade(DefaultUpgradeManager.java:142)        at com.atlassian.confluence.upgrade.UpgradeLauncherServletContextListener.contextInitialized(UpgradeLauncherServletContextListener.java:28)        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4205)        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4704)        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)        at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)        at org.apache.catalina.core.StandardService.start(StandardService.java:525)        at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)        at org.apache.catalina.startup.Catalina.start(Catalina.java:595)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)2014-03-01 21:08:40,269 ERROR [main] [ContainerBase.[Standalone].[localhost].[/]] log Exception sending context initialized event to listener instance of class com.atlassian.confluence.upgrade.UpgradeLauncherServletContextListenerorg.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:626)        at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:314)        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)        at $Proxy10.publishEvent(Unknown Source)        at com.atlassian.confluence.upgrade.impl.DefaultUpgradeManager.upgrade(DefaultUpgradeManager.java:142)        at com.atlassian.confluence.upgrade.UpgradeLauncherServletContextListener.contextInitialized(UpgradeLauncherServletContextListener.java:28)        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4205)        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4704)        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)        at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)        at org.apache.catalina.core.StandardService.start(StandardService.java:525)        at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)        at org.apache.catalina.startup.Catalina.start(Catalina.java:595)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)2014-03-01 21:08:41,110 INFO [main] [atlassian.plugin.manager.DefaultPluginManager] shutdown Shutting down the plugin system2014-03-01 21:08:41,565 INFO [main] [com.atlassian.confluence.lifecycle] contextDestroyed Stopping Confluence
```


Answers:
author: Jamie Edwards
body: Database corruption. Who'd have thunk it? :)


Tags: confluence-administration