Bug: Test failure: compaction_test.py::TestCompaction::test_user_defined_compaction
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 23/Jun/22 12:50
Resolved: 23/Jun/22 17:44

The Pyton dtest {{compaction_test.py::TestCompaction::test_user_defined_compaction}} seems to be 1% flaky. There are intermittent failures at least in trunk:
 * [https://app.circleci.com/pipelines/github/adelapena/cassandra/1726/workflows/45429a7e-fac0-4ecb-91ad-2d533a529075]
 * [https://app.circleci.com/pipelines/github/adelapena/cassandra/1726/workflows/80a58c5f-391f-46d3-82d8-17ec049421de/jobs/18105]

{code:java}
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [[node1] 'ERROR [Native-Transport-Requests-2] 2022-06-23 11:44:45,036 StorageProxy.java:1632 - Failed to apply mutation locally : \njava.lang.RuntimeException: null for ks: keyspace1, table: standard1\n\tat org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1442)\n\tat org.apache.cassandra.db.CassandraTableWriteHandler.write(CassandraTableWriteHandler.java:40)\n\tat org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:653)\n\tat org.apache.cassandra.db.Keyspace.apply(Keyspace.java:506)\n\tat org.apache.cassandra.db.Mutation.apply(Mutation.java:228)\n\tat org.apache.cassandra.db.Mutation.apply(Mutation.java:248)\n\tat org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1626)\n\tat org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2541)\n\tat org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:124)\n\tat org.apache.cassandra.concurrent.SEPExecutor.maybeExecuteImmediately(SEPExecutor.java:214)\n\tat org.apache.cassandra.concurrent.Stage.maybeExecuteImmediately(Stage.java:127)\n\tat org.apache.cassandra.service.StorageProxy.performLocally(StorageProxy.java:1620)\n\tat org.apache.cassandra.service.StorageProxy.sendToHintedReplicas(StorageProxy.java:1527)\n\tat org.apache.cassandra.service.StorageProxy.lambda$static$0(StorageProxy.java:213)\n\tat org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:1359)\n\tat org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:868)\n\tat org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:1130)\n\tat org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:516)\n\tat org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:491)\n\tat org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:258)\n\tat org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:826)\n\tat org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:804)\n\tat org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:167)\n\tat org.apache.cassandra.transport.Message$Request.execute(Message.java:254)\n\tat org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:122)\n\tat org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:141)\n\tat org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:168)\n\tat org.apache.cassandra.transport.Dispatcher.lambda$dispatch$0(Dispatcher.java:82)\n\tat org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:81)\n\tat org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47)\n\tat org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57)\n\tat org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:120)\n\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: java.lang.NullPointerException: null\n\tat org.apache.cassandra.db.memtable.AbstractMemtableWithCommitlog.accepts(AbstractMemtableWithCommitlog.java:101)\n\tat org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:329)\n\tat org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1421)\n\t... 33 common frames omitted']
{code}
4.1 doesn't seem affected:
* https://app.circleci.com/pipelines/github/adelapena/cassandra/1725/workflows/6447e935-19de-4e87-89f1-d7e3972d3e79
* https://app.circleci.com/pipelines/github/adelapena/cassandra/1725/workflows/93b8f1db-4fb7-4a01-a0e2-fe1e6637b356

This was detected while running CircleCI [CI for CASSANDRA-17500|https://app.circleci.com/pipelines/github/adelapena/cassandra/1724/workflows/31962907-35eb-41a1-aa9c-1394ed944777/jobs/18069], and [reproduced on trunk with the multiplexer|https://app.circleci.com/pipelines/github/adelapena/cassandra/1726/workflows/45429a7e-fac0-4ecb-91ad-2d533a529075/jobs/18101]. The failure hasn't been reproduced on Jenkins yet.