opened
jrudolph
username_0
Resurrection of https://github.com/akka/akka/issues/29185

https://jenkins.akka.io:8498/job/akka-artery-cluster-tests/2465/consoleFull

```
[info] [JVM-1] - must Scenario(lease-majority, 5) (on node 'node1', class akka.cluster.sbr.RandomizedSplitBrainResolverIntegrationSpecMultiJvmNode1) *** FAILED ***
[info] [JVM-1]   java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for 0
[info] [JVM-1]   at scala.Predef$.assert(Predef.scala:223)
[info] [JVM-1]   at akka.testkit.TestKitBase.expectMsg_internal(TestKit.scala:459)
[info] [JVM-1]   at akka.testkit.TestKitBase.expectMsg(TestKit.scala:445)
[info] [JVM-1]   at akka.testkit.TestKitBase.expectMsg$(TestKit.scala:445)
[info] [JVM-1]   at akka.testkit.TestKit.expectMsg(TestKit.scala:969)
[info] [JVM-1]   at akka.cluster.sbr.RandomizedSplitBrainResolverIntegrationSpec$DisposableSys.$anonfun$verify$2(RandomizedBrainResolverIntegrationSpec.scala:247)
```
Failed: RandomizedSplitBrainResolverIntegrationSpec
issue
created
johanandren
username_1
https://jenkins.akka.io:8498/job/akka-artery-cluster-tests/2487/consoleText
comment
created
johanandren
username_1
https://jenkins.akka.io:8498/job/akka-multi-node/4234/consoleText
comment
created
patriknw
username_2
https://jenkins.akka.io:8498/job/akka-nightly-jdk8/8096/consoleFull
```
[info] [JVM-1] - must Scenario(lease-majority, 3) (on node 'node1', class akka.cluster.sbr.RandomizedSplitBrainResolverIntegrationSpecMultiJvmNode1) *** FAILED ***
[info] [JVM-1]   java.lang.AssertionError: assertion failed: received unexpected message RealMessage(DoubleRegister(/system/sharding/Entity-1/1/1-default,trying to register akka://RandomizedSplitBrainResolverIntegrationSpec-1@localhost:42701, but akka://RandomizedSplitBrainResolverIntegrationSpec-1@localhost:37725 was already registered for /system/sharding/Entity-1/1/1-default),Actor[akka://RandomizedSplitBrainResolverIntegrationSpec/user/shardingRegistry-1#-1155572375]) after 0 millis
[info] [JVM-1]   at akka.testkit.TestKitBase.expectNoMsg_internal(TestKit.scala:767)
[info] [JVM-1]   at akka.testkit.TestKitBase.expectNoMessage(TestKit.scala:731)
[info] [JVM-1]   at akka.testkit.TestKitBase.expectNoMessage$(TestKit.scala:730)
[info] [JVM-1]   at akka.testkit.TestKit.expectNoMessage(TestKit.scala:969)
[info] [JVM-1]   at akka.cluster.sbr.RandomizedSplitBrainResolverIntegrationSpec$DisposableSys.$anonfun$verify$25(RandomizedBrainResolverIntegrationSpec.scala:356)
[info] [JVM-1]   at akka.remote.testkit.MultiNodeSpec.runOn(MultiNodeSpec.scala:375)
[info] [JVM-1]   at akka.cluster.sbr.RandomizedSplitBrainResolverIntegrationSpec$DisposableSys.$anonfun$verify$4(RandomizedBrainResolverIntegrationSpec.scala:353)
[info] [JVM-1]   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
```
comment
created
johanandren
username_1
To sum up the latest log, one node, JVM-3, is partitioned from the rest of the cluster, SBR hits on the "surviving" side and downs it, but before SBR hits on JVM-3 the partition heals up, JVM-3 then starts to be quarantined by the other nodes since it was downed and removed from that side of the cluster, but itself does not take a decision for another stable-after from there.

There has been previous work around this to try to down a node when it sees the other side quarantining it https://github.com/akka/akka/pull/25961 but there were some problems around that.
comment
closed
chbatey
username_3
issue