Could not open Hibernate Session for transaction; Hazelcast instance is not active!
Asked by Lara Weber:

Hello,

We have Confluence Data Center with 2 nodes. The second node was not reachable for a few hours. I got the the error message:

```
ERROR [ConfluenceSpringScheduler:thread-9] [security.denormalisedpermissions.impl.DenormalisedChangeLogListenerBase] onCronEvent Unable to process records: Could not open Hibernate Session for transaction; nested exception is com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!? ? at org.springframework.orm.hibernate5.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:598)? ? at com.atlassian.confluence.impl.hibernate.ConfluenceHibernateTransactionManager.doBegin(ConfluenceHibernateTransactionManager.java:31)? ? at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)? ? at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)? ? at jdk.internal.reflect.GeneratedMethodAccessor123.invoke(Unknown Source)? ? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)? ? at java.base/java.lang.reflect.Method.invoke(Method.java:566)? ? at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)? ? at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)? ? at com.sun.proxy.$Proxy107.getTransaction(Unknown Source)? ? at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:595)? ? at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:382)? ? at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)? ? at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)? ? at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:220)? ? at com.sun.proxy.$Proxy157.getSpaceServiceState(Unknown Source)? ? at com.atlassian.confluence.security.denormalisedpermissions.impl.space.DenormalisedSpaceChangeLogListener.isServiceEnabled(DenormalisedSpaceChangeLogListener.java:73)? ? at com.atlassian.confluence.security.denormalisedpermissions.impl.DenormalisedChangeLogListenerBase.onCronEvent(DenormalisedChangeLogListenerBase.java:95)? ? at jdk.internal.reflect.GeneratedMethodAccessor121.invoke(Unknown Source)? ? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)? ? at java.base/java.lang.reflect.Method.invoke(Method.java:566)? ? at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)? ? at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)? ? at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)? ? at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)? ? at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)? ? at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)? ? at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)? ? at java.base/java.lang.Thread.run(Thread.java:829)Caused by: com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!? ? at com.hazelcast.instance.HazelcastInstanceProxy.getOriginal(HazelcastInstanceProxy.java:321)? ? at com.hazelcast.instance.HazelcastInstanceProxy.getCluster(HazelcastInstanceProxy.java:219)? ? at jdk.internal.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)? ? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)? ? at java.base/java.lang.reflect.Method.invoke(Method.java:566)? ? at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)? ? at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)? ? at com.sun.proxy.$Proxy117.getCluster(Unknown Source)? ? at com.hazelcast.hibernate.HazelcastTimestamper.nextTimestamp(HazelcastTimestamper.java:38)? ? at com.hazelcast.hibernate.HazelcastLocalCacheRegionFactory.nextTimestamp(HazelcastLocalCacheRegionFactory.java:77)? ? at org.hibernate.cache.spi.AbstractCacheTransactionSynchronization.<init>(AbstractCacheTransactionSynchronization.java:19)? ? at org.hibernate.cache.spi.StandardCacheTransactionSynchronization.<init>(StandardCacheTransactionSynchronization.java:14)? ? at org.hibernate.cache.spi.AbstractRegionFactory.createTransactionContext(AbstractRegionFactory.java:145)? ? at org.hibernate.internal.AbstractSharedSessionContract.<init>(AbstractSharedSessionContract.java:159)? ? at org.hibernate.internal.AbstractSessionImpl.<init>(AbstractSessionImpl.java:29)? ? at org.hibernate.internal.SessionImpl.<init>(SessionImpl.java:230)? ? at org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl.openSession(SessionFactoryImpl.java:1334)? ? at org.hibernate.internal.SessionFactoryImpl.openSession(SessionFactoryImpl.java:486)? ? at com.atlassian.confluence.impl.core.persistence.hibernate.SwitchableCachingSessionFactory.openSession(SwitchableCachingSessionFactory.java:120)? ? at jdk.internal.reflect.GeneratedMethodAccessor226.invoke(Unknown Source)? ? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)? ? at java.base/java.lang.reflect.Method.invoke(Method.java:566)? ? at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)? ? at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)? ? at com.sun.proxy.$Proxy108.openSession(Unknown Source)? ? at org.springframework.orm.hibernate5.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:479)? ? ... 28 more
```

What can I do to avoid the system error?

Thank you and Kind Regards


Answers:
author: Shreya Gupta
body: Hi, so the fix we applied is - creating a clusterrole and clusterrolebinding for confluence and destinationrule setting the value of ttl to 0 for now and adding the httpcookieheader too.  

Post implying the cr and crb, got the issue resolved.

comments: author: Automation Group
body: Hi Shreya,

Great to know, you find the solution. Would be great if you elaborate more.

Regards,

Ankit Gupta


author: fivahij carspure
body: [@Shreya Gupta](/t5/user/viewprofilepage/user-id/5311750), hi

can you explain more detailed your fix, please?


author: Automation Group
body: Hi..!

Do we have any solution for it? if yes kindly suggest.

As per this article, we don't have any solution -

<https://jira.atlassian.com/browse/CONFSERVER-64166>

Regards,

Ankit Gupta


author: Shreya Gupta
body: Did you got the solution - i am also facing the same issue - for version 8.6.2


author: IKP Reddy
body: we also faced the same situation, As of now we confluence working on both nodes fine.

Now suddenly we see the above issue in node2. after restart it up.

But not sure why it suddenly gave the error. we are in 8.5 version.

Any one has know the root cause here ?

comments: author: Gita Meskauskas
body: Does anyone find couse of error? I get wthat error on version 8.5.4 too.


author: Joel Erickson
body: Also affecting ver. 8.5.9


author: Kenneth Epker
body: Experiencing the same error when attempting to add a node in confluence clustering using Kubernetes.


author: Martin Podhovnik
body: Hi!

If you run your DC cluster on Linux, I would have a look on the firewalld or selinux settings.

Just had the same issue when setting up another node - and reason was a forgotten port allowance.

BR Martin


author: Gita Meskauskas
body: Hi, We met the same situation too. Which version of Confluence you are running? We are on 7.19.14. Any ideas what's wrong?


author: Gonchik Tsymzhitov
body: Hi! I met with absolutely the same situation

comments: author: Gonchik Tsymzhitov
body: It has been by updating Confluence


Tags: questions-for-confluence

server