av9ash/GitBugs huggingface dataset:
Bug: Compaction purger doesn't handle seeing random slices of state correctly
Status: Triage Needed
Priority: Normal
Created: 24/Aug/23 16:44

Compaction purger was written assuming it will be compacting the latest state of the command row, but this is far from the case and it will actually be compacting a random slice of command row columns and their versions based on whatever input sstables are selected for the compaction.

This is not very problematic to fix. A lot of the logic is based on just the value of the field, and the remaining can wait until enough of the command is visible to determine if cleanup is possible cleanup.

Eventually durability is `universal` and we can drop the row contents based on just the `txnid` so there is always a fallback no matter how badly we have torn up the state of a command by dropping portions. We don't want to rely on that so we do drop a tombstone in the `erase` case which we can later drop when durability is `universal` without any expensive tombstone handling checks.
Bug: BLOG - Town Hall Replay: Time Series Data Modeling for Massive Scale
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Aug/23 19:26
Resolved: 23/Aug/23 07:30

This ticket is to capture the work associated with publishing the blog "Town Hall Replay: Time Series Data Modeling for Massive Scale"

This blog can be published as soon as possible, but if it cannot be published within a week of the noted publish date *(August 23)*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).
Bug: JMXUtil.getJmxConnector should retry connection attempts 
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 04/Aug/23 22:14
Resolved: 21/Aug/23 13:08

We previously added a JMXUtil class that makes it easy for dtests to get a JMX connection. It ends up that occasionally the JMX server side needs more time to start up (especially when stopping and restarting instances, which we’re now doing more frequently, or when stopping and restarting the whole cluster). In these cases, JMXUtil.getJmxConnector can fail, when it would be possible to connect if a retry mechanism was added. We should add this capability to the in-jvm dtest framework.
Bug: Open File Descriptors not cleared post upgrade from 3.11.9 to 4.0.5
Status: Triage Needed
Priority: Normal
Created: 16/Aug/23 17:19



We recently upgraded our fleet of ~2500 Cassandra instances from 3.11.9 to 4.0.5.

After the upgrade, we are seeing a unique issue where the compacted SSTables's file descriptors are still present and are never cleared. This is causing false disk alerts. We have to restart nodes very often in order to reclaim the disk space.

We use the *Apache* version of C* along with CentOS Linux 7.9.2009 and Java 8.

We are seeing it for all the tables (Filter, Data ..etc ) when checked with `lsof` command 

{code:java}
<path>/nb-1-big-Statistics.db.tmp (deleted)
<path>/nb-3-big-Statistics.db.tmp (deleted)
<path>/nb-2-big-Data.db (deleted)
<path>/nb-2-big-Statistics.db.tmp (deleted)
<path>/nb-2-big-Index.db (deleted)
<path>/nb-2-big-Statistics.db.tmp (deleted)
<path>/nb-2-big-CompressionInfo.db (deleted)
<path>/nb-2-big-Filter.db (deleted)
<path>/nb-2-big-Summary.db (deleted)
<path>/nb-2-big-Digest.crc32 (deleted)
{code}
Bug: CEP-15 (C*) When a host replacement happens don't loose the peer mapping right away
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Aug/23 23:46
Resolved: 16/Aug/23 21:17

When a host replacement happens the latest epoch will no longer have the removed peer, but we may have in-flight messages that still reference it.  When C* restarts we need to also filter out these peers so we don’t wait for an epoch ack.
Bug: Fix flaky test: org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 16/Aug/23 16:51
Resolved: 16/Aug/23 16:57

{noformat}
java.lang.reflect.InaccessibleObjectException: Unable to make field private final jdk.management.jfr.StreamManager jdk.management.jfr.FlightRecorderMXBeanImpl.streamHandler accessible: module jdk.management.jfr does not "opens jdk.management.jfr" to unnamed module @6db9f5a4
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
	at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
	at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
	at org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
	at org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:121)
	at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644)
	at org.apache.cassandra.Util.flakyTest(Util.java:669)
	at org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}

Seen here: https://app.circleci.com/pipelines/github/driftx/cassandra/1203/workflows/087cb08a-d035-491f-bb90-208b25a5fd4b/jobs/44699/tests
Bug: Change the logging level from WARN to INFO when invoked from "insights" thread (Spinning trying to capture readers)
Status: Resolved
Priority: Normal
Resolution: Won't Fix
Created: 15/Aug/23 21:16
Resolved: 15/Aug/23 21:27

The metric collector for Apache Cassandra (MCAC) is trying to get stats about the sstables to report metrics while compaction has them locked.

Since the warning message is from the "{*}insights{*}" thread rather than the "{*}compaction{*}" thread, the message is considered harmless.

Since the message is harmless and doesn't require action when reported from the “{*}insights”{*} thread{*}.{*} The message logging level should be moved from WARN to INFO.

Sample WARN Message:
WARN [insights-6-1] 2022-09-21 12:00:53,416 NoSpamLogger.java:95 - Spinning trying to capture readers [BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-17753-big-Data.db'), BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-19976-big-Data.db'), BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-18476-big-Data.db'), BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-21870-big-Data.db'), BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-20638-big-Data.db'), BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-22723-big-Data.db'), BigTableReader(path='/cassandra/data/tokens/events-d111f320f8e011ec8ae317b93c6db81b/nb-21223-big-Data.db'),
Bug: BLOG - Town Hall Replay: Bad Partition Handling & Large Language Models
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Aug/23 22:19
Resolved: 16/Aug/23 10:25

This ticket is to capture the work associated with publishing the blog "Town Hall Replay: Bad Partition Handling & Large Language Models"

This blog can be published as soon as possible, but if it cannot be published within a week of the noted publish date *(August 16)*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).
Bug: Optimized default configuration
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Aug/23 09:43
Resolved: 15/Aug/23 10:46

We currently offer only one sample configuration file with Cassandra, and that file is deliberately configured to disable all new functionality and incompatible improvements. This works well for legacy users that want to have a painless upgrade, but is a very bad choice for new users, or anyone wanting to make comparisons between Cassandra versions or between Cassandra and other databases.

We offer very little indication, in the database packaging itself, that there are well-tested configuration choices that can solve known problems and dramatically improve performance. This is guaranteed to paint the database in a worse light than it deserves, and will very likely hurt adoption.

We should find a way to offer a very easy way of choosing between "optimized" and "compatible" defaults. At minimal, we could provide alternate yaml files. Alternatively, we could build on the {{storage_compatibility_mode}} concept to grow it into a setting that not only enables/disables certain settings, but also changes their default values.
Bug: We should offer an option for optimized default configuration
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Aug/23 09:41
Resolved: 15/Aug/23 10:46

We currently offer only one sample configuration file with Cassandra, and that file is deliberately configured to disable all new functionality and incompatible improvements. This works well for legacy users that want to have a painless upgrade, but is a very bad choice for new users, or anyone wanting to make comparisons between Cassandra versions or between Cassandra and other databases.

We offer very little indication, in the database packaging itself, that there are well-tested configuration choices that can solve known problems and dramatically improve performance. This is guaranteed to paint the database in a worse light than it deserves, and will very likely hurt adoption.

We should find a way to offer a very easy way of choosing between "optimized" and "compatible" defaults. At minimal, we could provide alternate yaml files. Alternatively, we could build on the {{storage_compatibility_mode}} concept to grow it into a setting that not only enables/disables certain settings, but also changes their default values.
Bug: Simplify helptopics.py using associative array
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 04/Aug/23 17:45
Resolved: 07/Aug/23 11:27

The ~200 lines of code in helptopics.py can be greatly simplified using an associative array to map topics to HTML anchors instead of class reflection. This will halve the lines of code and improve readability.

Functionality will not be changed.
Bug: BLOG - Town Hall Replay: Monitoring Apache Cassandra Without Implementation
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Aug/23 16:00
Resolved: 14/Aug/23 15:20

This ticket is to capture the work associated with publishing the blog "Town Hall Replay: Monitoring Apache Cassandra Without Implementation"

This blog can be published as soon as possible, but if it cannot be published within a week of the noted publish date *(Aug 2)*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).
Bug: [Doc] Add to virtual table documentation that a minimum cq;lsh version is needed
Status: Triage Needed
Priority: Normal
Created: 12/Jul/23 22:57

See [https://the-asf.slack.com/archives/CJZLTM05A/p1686771286554899] for details.

 

Add some warning on top of [https://cassandra.apache.org/doc/4.1/cassandra/new/virtualtables.html]  that protocol version v5 is needed
Bug: [Cassandra Analytics] Fix bulk writes with Buffered RowBufferMode 
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 25/Jul/23 17:49
Resolved: 08/Aug/23 21:42

When setting {{Buffered}} {{RowBufferMode}} as part of the {{WriterOption's}} , {{org.apache.cassandra.spark.bulkwriter.RecordWriter}} ignores that configuration and uses the batch size to finalize an SSTable and start using a new one if new rows are available.
Bug: Skip ColumnFamilyStore#topPartitions initialization when client or tool mode
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Jul/23 17:19
Resolved: 01/Aug/23 20:26

In {{org.apache.cassandra.db.ColumnFamilyStore}} the {{topPartitions}} is initialized when the keyspace is not a system keyspace. However, when running the cassandra library as client mode or tool mode, the initialization also happens. However, {{TopPartitionTracker}} performs queries to the {{system}} keyspace, which might not be available in most of the cases. For that reason, we should skip initialization of {{topPartitions}} when running on client mode or tool mode.

In utilities and external libraries, this can produce a warning that is displayed with a stack trace. This warning can be misleading for end users, and can cause confusion. But more importantly, the initialization of {{topPartitions}} is not required in this mode.

The warning is similar to this:
{code:java}
    WARN org.apache.cassandra.db.SystemKeyspace: Could not load stored top SIZES partitions for ...
    org.apache.cassandra.db.KeyspaceNotDefinedException: keyspace system does not exist
            at org.apache.cassandra.schema.Schema.validateTable(Schema.java:xxx) ~[?:?]
            at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:xxx) ~[?:?]
            at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:xxx) ~[?:?]
            at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:xxx) ~[?:?]
            at org.apache.cassandra.cql3.QueryProcessor.parseAndPrepare(QueryProcessor.java:xxx) ~[?:?]
            ... {code}
Bug: Test failure: StandaloneSplitterWithCQLTesterTest/testSplittingMultipleSSTables_cdc_jdk17
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 24/Jul/23 21:25
Resolved: 01/Aug/23 21:25

It seems like we can hit that path only sometimes and the test is flaky as I never saw it so far  failing in CircleCI.
Seen here:
https://ci-cassandra.apache.org/job/Cassandra-trunk-test-cdc/jdk=jdk_17_latest,label=cassandra,split=4/lastBuild/testReport/junit/org.apache.cassandra.tools/StandaloneSplitterWithCQLTesterTest/testSplittingMultipleSSTables_cdc_jdk17/


{code:java}
Stacktrace
junit.framework.AssertionFailedError: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object java.nio.channels.spi.AbstractInterruptibleChannel.closeLock accessible: module java.base does not "opens java.nio.channels.spi" to unnamed module @57536d79
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at org.apache.cassandra.utils.concurrent.Ref.getFields(Ref.java:654)
	at org.apache.cassandra.utils.concurrent.Ref.getFields(Ref.java:657)
	at org.apache.cassandra.utils.concurrent.Ref.getFields(Ref.java:657)
	at org.apache.cassandra.utils.concurrent.Ref$Visitor.traverse(Ref.java:611)
	at org.apache.cassandra.utils.concurrent.Ref$Visitor.run(Ref.java:566)
	at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)

	at org.apache.cassandra.tools.StandaloneSplitterWithCQLTesterTest.testSplittingMultipleSSTables(StandaloneSplitterWithCQLTesterTest.java:99)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
Bug: Python dtests upgrade tests mixed SHA runs
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 27/Jul/23 09:28
Resolved: 27/Jul/23 09:43

Hi adding a bug I noticed in jenkins:
A- Commit SHA1
B- Jenkins starts a test run and eventually starts python dtests
C- Before B completes commit SHA2 comes in

Python dtests in B will run mixed SHA1 and SHA2 versions of trunk. An example [https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest-upgrade/1213/jdk=jdk_11_latest,label=cassandra-dtest,split=11/consoleFull|http://example.com] where both 2a458ba01dae3b35b62ad2ef361e1acb567c97f0 and the later a565711056c859398d0b26081b46e71d2076de1d get mixed up.
Bug: User credentials are validated with LOCAL_QUORUM instead of with LOCAL_ONE
Status: Triage Needed
Priority: Normal
Created: 26/Jul/23 05:12

The documentation ([https://cassandra.apache.org/doc/latest/cassandra/operating/security.html]) tells us that users should be authenticated with LOCAL_ONE (except the default user).
From logging we see it is trying with LOCAL_QUORUM.INFO [Native-Transport-Requests-1] 2023-07-24 14:51:11,526 NoSpamLogger.java:105 - "Cannot achieve consistency level LOCAL_QUORUM" while executing SELECT salted_hash FROM system_auth.roles WHERE role = 'test' ALLOW FILTERING
This is detected on C* 4.1.2 (RHEL)
Bug: Use AssertJ-db for Cassandra to make testing easier
Status: Triage Needed
Priority: Normal
Created: 26/Jul/23 09:01

I noticed that there is assertj-db project as a sub-project of AssertJ.

The way it connects to data source is via JDBC. There is Cassandra JDBC library (2) (Apache 2.0) for which there is a maintained fork (3). 

The idea here is to implement some helper classes which would make this kind of stuff possible (4)

I am not completely sure how much compatible all parts are. 

This is so far just an idea if somebody wants to investigate what is possible.

(1) https://assertj.github.io/doc/#assertj-db
(2 )https://github.com/ing-bank/cassandra-jdbc-wrapper
(3) https://github.com/adejanovski/cassandra-jdbc-wrapper/
(4) https://assertj.github.io/doc/#statically-import-org-assertj-db-api-assertions-assertthat
Bug: generate.sh no longer works for repeated tests correctly
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 25/Jul/23 15:23
Resolved: 25/Jul/23 15:39

Using the examples:

{noformat}
$ .circleci/generate.sh -e DTEST_BRANCH=CASSANDRA-8272 -e DTEST_REPO=https://github.com/adelapena/cassandra-dtest.git -e REPEATED_DTESTS=cqlsh_tests/test_cqlsh.py::TestCqlshSmoke -e REPEATED_DTESTS_COUNT=500

Detecting new or modified tests with git diff --diff-filter=AMR trunk...HEAD:

Setting environment variables:
  DTEST_BRANCH=CASSANDRA-8272: 
  DTEST_REPO=https://github.com/adelapena/cassandra-dtest.git: 
  REPEATED_DTESTS=cqlsh_tests/test_cqlsh.py::TestCqlshSmoke: 
  REPEATED_DTESTS_COUNT=500: 
{noformat}

but no changes are actually made.
Bug: Avoid unnecessary index builds and validations
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 06/Jul/23 15:27
Resolved: 24/Jul/23 22:39

In the testing of CASSANDRA-18490 it was noted that the SSTableAddedNotification will trigger an index build that will always build the column index components even if they already exist and are valid. 

The same notification is also handled by the StorageAttachedIndexGroup that can validate the components. This can lead to the components being validated more than is necessary.

We should coordinate these notification handlers so there is no overlap of responsibility.
Bug: Unfinalise AbstractVirtualTable.select(..) for downstream patches
Status: Patch Available
Priority: Normal
Created: 06/Jul/23 22:02

In AbstractVirtualTable the select methods are final.  This prevents downstream C* engineers from implementing their own virtual tables where the select needs to be overridden.

This is not a C* API and is not intended for C* users and operators.  Extension of these methods should also clearly marked as experimental with no maintenance or compatibility provided from any release to another (including patch versions).

The original proposal for Virtual Tables (CASSANDRA-7622) was to have a table "backed by an API, rather than data explicitly managed and stored as sstables".  A number of people on the ticket supported the eventual idea of user-defined Virtual Tables.  The consensus was that an incremental approach should be taken, where this should not be part of the initial implementation, and that use-cases and careful consideration around API security and compatibility would be needed.

The next incremental approach can be to permit downstream patches to experiment against an explicitly labelled experimental (non-stable) internal code (so to protect the C* community from security and compatibility concerns).  Such experiments will help smoke out and promote more grounded discussions for further work, if so found and desired.

The patch is two lines: to remove the final keyword from both select(..) methods; and adding whatever comment/annotation we need to state their experimental/non-stable state and limited audience.
Bug: Concurrency is broken in the block balanced tree for queries
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Jul/23 11:05
Resolved: 17/Jul/23 11:09

Performance testing has highlighted that concurrent queries are broken against the block balanced tree because the mutable traversal state was pushed up to the walker that is linked to an SSTableIndex so is shared amongst all queries.

A new traversal state should be created for each query and marked as not threadsafe. We should add a test to show that concurrent queries work against the block balanced tree.
Bug: CCM does not select Java version properly
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 19/Jul/23 09:43
Resolved: 19/Jul/23 14:53

CASSANDRA-18106 added a new method to {{commons.py}} https://github.com/riptano/ccm/blob/master/ccmlib/common.py#L857 which is called when Cassandra version is >= 4.2


{code:python}
def get_supported_jdk_versions(install_dir):
    """
    Return the supported java versions from build.xml
    Only works in > 4.1
    """
     ...
                versions = ['8' if v == '1.8' else v for v in versions]
    ....
{code}

The problem is that this method returns a collection of strings, while later in {{commons.py}}:

{code:python}
...
        if current_java_version in versions:
            jvm_version = current_java_version
...
{code}

the condition expects the collection to consists of ints and the condition fails.

It is causing problems when you want to run the upgrade tests against Java 11 because it always goes to the next branch of the condition which results in selecting the first supported version, which is 8.