av9ash/GitBugs huggingface dataset:
Bug: Test failure: org.apache.cassandra.distributed.test.log.RegisterTest.serializationVersionDisagreementTest-_jdk17
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Nov/23 10:13
Resolved: 24/Nov/23 10:16

Circleci Failure: https://app.circleci.com/pipelines/github/michaelsembwever/cassandra/256/workflows/c4fda8f1-a8d6-4523-be83-5e30b9de39fe/jobs/20500/tests

```
 java.lang.RuntimeException: java.lang.NoSuchFieldException: modifiers
 at org.apache.cassandra.distributed.test.log.RegisterTest.lambda$serializationVersionDisagreementTest$81c80a4a$2(RegisterTest.java:132)
 at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
 at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
 at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 at java.base/java.lang.Thread.run(Thread.java:833)
 Caused by: java.lang.NoSuchFieldException: modifiers
 at java.base/java.lang.Class.getDeclaredField(Class.java:2610)
 at org.apache.cassandra.distributed.test.log.RegisterTest.lambda$serializationVersionDisagreementTest$81c80a4a$2(RegisterTest.java:120)
```
Bug: Do not drop CQLTester.KEYSPACE content in @After asynchronously
Status: In Progress
Priority: Normal
Created: 15/Nov/23 11:12

CASSANDRA-18710 showed that removal of tables, views, functions, etc. in {{@After}} method asynchronously may span the next test case and modify that test behaviour in some unexpected way

Concretely, {{@After}} method includes dropping {{KEYSPACE_PER_TEST}} synchronously and the whole content of {{KEYSPACE}} asynchronously. I don't see a reason for dropping the content of {{KEYSPACE}} after each test because that keyspace is not expected to be clean for each test case. It is the case for {{KEYSPACE_PER_TEST}}. Therefore I propose removal of that async cleanup of {{KEYSPACE}} after each test case.
Bug: Refactor LocalLog to make intialisation more straightforward and consistent
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Nov/23 21:03
Resolved: 22/Nov/23 21:06

It should be possible to rationalise the multiple ways we currently have for obtaining an instance of {{LocalLog}}. We also want to ensure that the necessary steps for initialising the log, which may vary depending on context (i.e. starting up a node normally, starting up immediately after upgrade, forcing recovery from a metadata export, running unit & dtests), are performed exactly once and in the correct sequence.
Bug: Fix potential race condition in IndexViewManager during invalidation
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Oct/23 10:50
Resolved: 20/Nov/23 10:09

There is a potential race condition in the {{IndexViewManager.invalidate}} method:
{code:java}
public void invalidate()
{
    View currentView = view.get();

    for (SSTableIndex index : currentView)
    {
        index.markObsolete();
    }

    view.set(new View(context, Collections.emptyList()));
} {code}
We should {{getAndSet}} the view before marking the indexes as obsolete. This would avoid indexes potentially being made obsolete when being accessed.
Bug: SAI in-memory index does not include maximum term size check when adding terms
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Oct/23 10:39
Resolved: 20/Nov/23 16:12

The {{SSTableIndexWriter}} rejects terms that exceed a maximum term size with a no-spam warning, but the {{TrieMemoryIndex}} does not do this. 

We should check term sizes when rows are added and issue client warnings when this happens. This still needs to happen in the {{SSTableIndexWriter}} to handle terms during an initial index build.
Bug: Test failure: org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testReplayLogic-compression.jdk17.arch=x86_64.python2.7 
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 20/Oct/23 17:56
Resolved: 20/Nov/23 06:45

Seen here:

https://ci-cassandra.apache.org/job/Cassandra-5.0/71/testReport/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerCDCTest/testReplayLogic_compression_jdk17_arch_x86_64_python2_7/
h3.  
{code:java}
Error Message
Missing old CDCIndexData in new set after replay: CommitLog-7-1697673230997_cdc.idx,1747785 List of CDCIndexData in new set of indexes after replay: CommitLog-7-1697673230996_cdc.idx,3510561 CommitLog-7-1697673230998_cdc.idx,3509214

Stacktrace
junit.framework.AssertionFailedError: Missing old CDCIndexData in new set after replay: CommitLog-7-1697673230997_cdc.idx,1747785 List of CDCIndexData in new set of indexes after replay: CommitLog-7-1697673230996_cdc.idx,3510561 CommitLog-7-1697673230998_cdc.idx,3509214 at org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testReplayLogic(CommitLogSegmentManagerCDCTest.java:319) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
Bug: system_views.sstable_tasks queries not consistent with `nodetool compactionstats` output
Status: Triage Needed
Priority: Normal
Created: 18/Nov/23 16:51

Simple to reproduce: on any node with a series of compactions running, in one window run `watch nodetool compactionstats` and in another window run `watch 'cqlsh -e "select * from system_views.sstable_tasks;"'`.  The nodetool output will contain all the tasks (except those that complete) every iteratation, while the query will vary.
Bug: Consider adding a truncation record on table creation so that we can avoid force recycle on table drop
Status: Triage Needed
Priority: Normal
Created: 17/Nov/23 16:08

By adding a truncation record with the same timestamp a table is created we will have a marker that says all the data for that table prior to its creation should be considered stale. 

That should be more concise and more efficient approach than requiring recycle of all the commit log segments, which starts with flushing all dirty tables. 

The scenario in question is when a user recreates a table with the same ID and there is data in the commit log for that table, we may experience data resurrection in that new table.

The performance improvement is probably quite minor for production use as users usually do not drop tables that often. However it may positively impact our tests which do quite many schema changes. It will also reduce significantly the noise in the logs.
Bug: Test failure: org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
Status: Open
Priority: Normal
Created: 10/Oct/23 22:23

 
{code:java}
org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11 (from org.apache.cassandra.io.sstable.VerifyTest-.jdk11)

Failing for the past 1 build (Since #60 ) Took 0.42 sec.      Failed 1 times in the last 16 runs. Flakiness: 6%, Stability: 93% Stacktrace
java.io.EOFException at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:180) at org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:142) at org.apache.cassandra.io.util.RebufferingInputStream.readLong(RebufferingInputStream.java:231) at org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair(VerifyTest.java:538) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
Seen here: 

https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/org.apache.cassandra.io.sstable/VerifyTest/testMutateRepair__jdk11/
Bug: Bring PropertyFileSnitch & RackInferringSnitch into line with TCM
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:09
Resolved: 16/Nov/23 16:53

With TCM, all snitch implementations should only be responsible for the initial the topology configuration of the local node. For example, PFS should behave more like {{GossipingPropertyFileSnitch}} in that its configuration file specifies the DC & rack for the local node only. Each node in the cluster is responsible for updating its own topology info in {{ClusterMetadata}} when it is registered and the location of peers can be easily looked up there.

Modifying the DC & rack of a live node via GPFS/PFS config is desabled by default and strongly discouraged (CASSANDRA-10242/CASSANDRA-10243/CASSANDRA-9474) and there are {{StartupChecks}} in place to prevent changing location at startup. 

Given that modifying location of a joined node is a fundamentally unsafe operation, we should remove the ability to do so. The argument for allowing it has previously been that if a DC or cluster has no data, then the overhead of decommissioning and re-joining nodes to fix location is prohibitive. CEP-21 fixes that by making those operations more reliable, so the override becomes less valuable. 


Implementation can be found in:
https://github.com/beobal/cassandra/commit/8ec1ea705210b7e2e38d888768ba83faa53754d8
https://github.com/beobal/cassandra/commit/7b41f063e04f25b8215318f51d908732b1f21f5f
https://github.com/beobal/cassandra/commit/09cdc5351ad049d97c7eeb53c4e68701c088a711

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Refactor multistep operations
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:10
Resolved: 16/Nov/23 16:51

Make it easier to identify the current position in a sequence of steps.
Improve documentation and readability.

Implementation can be found in:
https://github.com/beobal/cassandra/commit/91a5cdde0c5e79063e2e90e8d5839736de20e076
https://github.com/beobal/cassandra/commit/791af4b59103b09e47aa2b13a285d8970a02ab53
https://github.com/beobal/cassandra/commit/2ff794db348f11196f857dafdef74561627d3453
https://github.com/beobal/cassandra/commit/cd8b4e29fe53e176874ab61e848c0512454806a9
https://github.com/beobal/cassandra/commit/c6254c161524ea364ff10ff68494f125f07161d9

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Improve CMS handoff and configuration
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:06
Resolved: 16/Nov/23 16:48

The intial mechanism for configuring membership of the CMS is rather simplistic. It requires the operator to select which nodes are members and to execute nodetool commands to perform every individual change to the membership. Handoff when a member is decommissioned or replaced also uses a naive strategy for selecting a replacement member.

This ticket enables the CMS membership to be configured more declaratively, by specifying the desired number of CMS members per-DC, much like replication using {{{}NetworkTopologyStrategy{}}}. 
Cassandra itself is responsible for selecting exactly which nodes should be CMS members based on this configuration and will attempt to maintain rack diversity if possible, with handoff of CMS membership handled in the same way.

When making changes to membership Cassandra will manage the entire transition from the starting state to end state using an in-progress sequence to compose the specific additions and removal operations.

Implementation can be found in:
[https://github.com/beobal/cassandra/commit/bcbb3f441d9df10207be38066c9752a08d62545b]

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

 
||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Test assigning the same token to multiple nodes
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:07
Resolved: 16/Nov/23 16:47

Add a test to ensure a node cannot join the cluster with a token already assigned to another peer

Implementation can be found in:
https://github.com/beobal/cassandra/commit/b9e70ffa12455119396a1bcdc60e3dfc64b4f795


CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Refactor StorageService to relocate functionality to more appropriate classes
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:08
Resolved: 16/Nov/23 16:46

Historically, {{StorageService}} contains much of the logic related to bootstrap, decommission, token movement and so on. Although much of this has already been refactored in the {{cep-21-tcm}} branch, there remains plenty of room for improvement.

Implementation can be found in:
https://github.com/beobal/cassandra/commit/525f46d7961e7ffaa126792d5d003e9a42ef6c63

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Add a mechanism to signal that a rejected Transformation should not be retried
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:10
Resolved: 16/Nov/23 16:46

In certain circumstances on the CMS node, it is useful to be able to cause a Transformation being applied to return a Rejected response without triggering catchup and retry. Adding an unchecked exception type for this enables it to be triggered from anywhere within the call stack, without polluting the interfaces.

Implementation can be found in:
https://github.com/beobal/cassandra/commit/8a84c0b562d59487a4c5fe7c8fb233cba3036216

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Use epoch based table ids by default
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:11
Resolved: 16/Nov/23 16:45

The initial implementation of CEP-21 depends on the use of deterministic table identifiers, as the table creation itself is performed by each node locally. This is suboptimal as recreating a previously dropped table can lead to the old data directory and files on disk, as well as CFIDs in the commit log being incorrectly associated with the new table.
As the epoch gives us a unique identifier for each creation event, we can base the table id on that to ensure uniqueness.

Implementation can be found in:
https://github.com/beobal/cassandra/commit/89a0a2bfb0d1646e8057d9bb69f8ce558ef3b483

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Simplify scope-local retries using Entry.Id
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:11
Resolved: 16/Nov/23 16:45

If a node submits a commit request to the CMS which succeeds but doesn't receive the response, it will automatically retry. However, the CMS node handling the resubmission most likely reject it as transformations are generally not idempotent. The submitter will need to then handle that rejection of the resubmission, which can currently mean inspecting the ClusterMetadata to see if the expected effects of the transformation are present. By using the entry id, the CMS node can identify the resubmission as such and return the same (success) response it sent (or should have sent) the first time.

Implementation can be found in:
https://github.com/beobal/cassandra/commit/c3c33ea1430e1bd14552f68de75bb9c4166fc986

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Remove unused GlobalPlacementDelta class
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Oct/23 12:13
Resolved: 16/Nov/23 16:44

This is a leftover from early development and has been unused for some time.

Implementation can be found in:
https://github.com/beobal/cassandra/commit/0ee6c6299c3ae391bea1c1efc4f33e60070f2f80

CI summary results attached. For right now, we've only been foccussing on unit tests, in-jvm & python dtests, so the failures in other suites are to be expected. Unfortunately, the archive containing the detailed results is too large to attach as a single file, so will look into the best way to work around that.
The intention is to fix all suites before merging the feature branch.

||Suite||Total||Passed||Skipped||Failed||Errors||
|jvm dtest|1508|1465|24|8|11|
|python dtest|1083|790|262|28|3|
|unit|12082|12006|61|6|9|
Bug: Python DTest does not release CQL connections, which can cause containers to hang for 2 hours trying to connect to a server thats already gone
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Nov/23 14:45
Resolved: 15/Nov/23 18:11

While debugging why upgrade tests were taking 2 hours we noticed that the test completed 2 hours early… yet the process did not exit and was logging cassandra.pool trying to connect to a cluster (but python dtest already cleaned it up)…

After investigating the code I saw that we create connections and store them in an array, but only 2 tests actually clean up!
Bug: Python DTest does not release CQL connections, which can cause containers to hang for 2 hours trying to connect to a server thats already gone
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 15/Nov/23 14:42
Resolved: 15/Nov/23 15:07

While debugging why upgrade tests were taking 2 hours we noticed that the test completed 2 hours early… yet the process did not exit and was logging cassandra.pool trying to connect to a cluster (but python dtest already cleaned it up)…

After investigating the code I saw that we create connections and store them in an array, but only 2 tests actually clean up!
Bug: Exclude jna transitive dependecy from the ohc library
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 01/Nov/23 20:25
Resolved: 15/Nov/23 09:15

The concern is related to the JNA dependency on the ohc library. We are now using version 0.5.1 of the ohc library in Cassandra. The ohc 0.5.1 depends on version 4.1.0 of the net.java.dev.jna . For the Cassandra project, JNA has been updated since the 4.0 release version up to JNA 5.6.0 in CASSANDRA-16212 to support running nodes on the arm64 architecture, so I assume that JNA 4.1.0 is no longer in Casssandra's classpath. 

The common practice in the Cassandra project is to exclude these kinds of transitive dependencies when they are explicitly mentioned in the main parent pom, since Ant may resolve them incorrectly. See comment:
https://issues.apache.org/jira/browse/CASSANDRA-14667?focusedCommentId=17765091&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17765091
Bug: BLOG - Apache Cassandra 5.0 Features: Storage Attached Indexes
Status: Patch Available
Priority: Normal
Created: 14/Nov/23 23:31

This ticket is to capture the work associated with publishing the blog "Apache Cassandra 5.0 Features: Storage Attached Indexes"

This blog can be published as soon as possible, but if it cannot be published within a week of the noted publish date {*}(November 14){*}, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).