av9ash/GitBugs huggingface dataset:
Bug: CEP-42: Constraints Framework
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 23/Sep/24 18:49
Resolved: 31/Jan/25 09:56

Add new Constraints framework as described in CEP-42. This initial Jira ticket includes the core constraints length and numeric. Follow up tickets will include the rest of the constraints described in the CEP.

 

[https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-42%3A+Constraints+Framework]
Bug: [Analytics] Decouple Cassandra types from Spark types so Cassandra types can be used independently from Spark
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 05/Aug/24 19:53
Resolved: 11/Sep/24 19:55

The Cassandra types and Spark types are tightly coupled in the same classes, making it difficult to deserialize Cassandra types without pulling in Spark as a dependency, We can split out the Spark types into a separate module by introducing a new TypeConverter that maps Cassandra types to Spark types. This enables use of the Cassandra types without pulling in Spark and also opens the possibility of other TypeConverters in the future beyond Spark.
Bug: Commitlog leak leads to multi-node outage
Status: Open
Priority: Normal
Created: 18/Jul/24 22:44

After days of sustained write traffic, our nodes get into a state where hundreds of the below commitlog error are happening a second.  The node becomes basically unwritable.  This happens to multiple nodes at a time, effectively causing the entire cluster to be unusable.

java.lang.Error: Maximum permit count exceeded
    at java.base/java.util.concurrent.Semaphore$Sync.tryReleaseShared(Semaphore.java:198)
    at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1382)
    at java.base/java.util.concurrent.Semaphore.release(Semaphore.java:619)
    at org.apache.cassandra.db.commitlog.AbstractCommitLogService.requestExtraSync(AbstractCommitLogService.java:297)
    at org.apache.cassandra.db.commitlog.BatchCommitLogService.maybeWaitForSync(BatchCommitLogService.java:40)
    at org.apache.cassandra.db.commitlog.AbstractCommitLogService.finishWriteFor(AbstractCommitLogService.java:284)
    at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:330)
    at org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:100)
    at org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:54)
    at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:641)
    at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:525)
    at org.apache.cassandra.db.Mutation.apply(Mutation.java:228)
    at org.apache.cassandra.db.Mutation.apply(Mutation.java:248)
    at org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1652)
    at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2611)
    at org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
    at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:142)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:829)

This happens on large 80 core machines and with the following yaml settings.
commitlog_sync: batch
concurrent_writes: 640
native_transport_max_threads: 640

This happens on 4.1.5 and 5.0beta1.  I have not tested other branches.

I am NOT able to reproduce it with smaller 12 core machines.
I am NOT able to reproduce it with commitlog_sync: periodic

I added some small instrumentation to periodically printout the value of org.apache.cassandra.db.commitlog.AbstractCommitLogService.haveWork.permits().  Under sustained write traffic, this value continually grows (by a million or so a minute using my cassandra-stress workload) until it hits MAXINT and the error starts occuring.  If write traffic slows, the value of haveWork.permits() drops.
Bug: Spinning trying to capture readers
Status: Triage Needed
Priority: Normal
Created: 17/Jul/24 05:26

On a handful of clusters we are noticing Spin locks occurring. I traced back all the calls to the EstimatedPartitionCount metric (eg. org.apache.cassandra.metrics:type=Table,keyspace=testks,scope=testcf,name=EstimatedPartitionCount)

Using the following patched function:
{code:java}
    public RefViewFragment selectAndReference(Function<View, Iterable<SSTableReader>> filter)
    {
        long failingSince = -1L;
        boolean first = true;
        while (true)
        {
            ViewFragment view = select(filter);
            Refs<SSTableReader> refs = Refs.tryRef(view.sstables);
            if (refs != null)
                return new RefViewFragment(view.sstables, view.memtables, refs);
            if (failingSince <= 0)
            {
                failingSince = System.nanoTime();
            }
            else if (System.nanoTime() - failingSince > TimeUnit.MILLISECONDS.toNanos(100))
            {
                List<SSTableReader> released = new ArrayList<>();
                for (SSTableReader reader : view.sstables)
                    if (reader.selfRef().globalCount() == 0)
                        released.add(reader);
                NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.SECONDS,
                                 "Spinning trying to capture readers {}, released: {}, ", view.sstables, released);
                if (first)
                {
                    first = false;
                    try {
                        throw new RuntimeException("Spinning trying to capture readers");
                    } catch (Exception e) {
                        logger.warn("Spin lock stacktrace", e);
                    }
                }
                failingSince = System.nanoTime();
            }
        }
    }
 {code}
Digging into this code I found it will fail if any of the sstables are in released state (ie. reader.selfRef().globalCount() == 0).

See the extract.log for an example of one of these spin lock occurrences. Sometimes these spin locks last over 5 minutes. Across the worst cluster with this issue, I ran a log processing script that everytime the 'Spinning trying to capture readers' was different to previous one it would output if the released tables were in Compacting state. Every single occurrence has it spin locking with released listing a sstable that is compacting.

In the extract.log example its spin locking saying that nb-320533-big-Data.db has been released. But you can see prior to it spinning that sstable is involved in a compaction. The compaction completes at 01:03:36 and the spinning stops. nb-320533-big-Data.db is deleted at 01:03:49 along with the other 9 sstables involved in the compaction.
Bug: Cassandra 4.1.5 - Curl timeout on receiving metadata for RedHat repo
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Aug/24 06:52
Resolved: 23/Jan/25 14:36

Hi team,

On trying to install and download Cassandra 4.1.5 for RedHat we are getting the following timeout message:
{code:java}
120.5   - Curl error (28): Timeout was reached for https://redhat.cassandra.apache.org/41x/repodata/repomd.xml [Operation timed out after 30000 milliseconds with 0 out of 0 bytes received]
120.5 Error: Failed to download metadata for repo 'cassandra': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried {code}
On trying to download the metadata for the repo it is currently failing and we are unable to install Cassandra 4.1.x on our RedHat systems. Please help us address the same.
Bug: CREATE TABLE LIKE for table with UDT
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 29/Sep/24 05:13
Resolved: 22/Jan/25 18:08

if under same keyspace , we should do nothing with udt.

We reject the action of table copy in the following two scenarios:
# if the UDTs used by the source table does not exist in the target keyspace;
# if it exists but the UDTs' value is different

We are not going to create udt if the target keyspace do not have it.
udf and mv are not supported.
Bug: Add guardrail for enabling usage of VectorType
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/Sep/24 18:20
Resolved: 06/Sep/24 22:21

Add a guardrail to allow operators to toggle support for the new Vector type. This is a precaution that operators may desire when working with clusters than have a diverse set of clients, and some clients may not yet support the new vector type. Those clients may fail when fetching metadata ("unexpected column type") and prevent session establishment, or fail to query vector columns, or otherwise behave strangely.

 

Patch is ready here: https://github.com/apache/cassandra/compare/trunk...aratno:cassandra:CASSANDRA-X-guardrail-vector-enable
Bug: Create a new test kind focused on long running tests that rely on randomized input
Status: Open
Priority: Normal
Created: 21/Aug/24 22:31

Right now we have jvm-dtest that is a grab back of tests.  Within these tests there are tests similar to org.apache.cassandra.fuzz.harry.integration.model.ConcurrentQuiescentCheckerIntegrationTest that take a very long time to run (14 minutes).  This makes resource provisioning for these tests problematic as the distribution of run times are large (most takes seconds, some take 10+ minutes)…

For this reason we should create a new test kind “fuzz” that has 2 focuses: long run times, and randomized inputs.

With this new test kind we will be able to create a new job with its own resource needs (parallel workers) focused on long running tests only.

Another view point is that we really only care about long running tests, so “burn” kind already covers this… so we could drop the “randomized inputs” requirement and place these tests in “burn” instead
Bug: CEP-15: (Accord) sequence EpochReady.coordinating to allow syncComplete to be learned from newer epochs
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Jul/24 00:19
Resolved: 27/Sep/24 23:25

When a node is bootstrapping or doing a host replacement it sees several epochs before it actually joins the ring, but in Accord we only synchronize epoch knowledge to the nodes that have already joined; this means we won’t ever synchronize the epochs seen on the new nodes! This becomes a problem because it forces these nodes to include far more epochs than required (because they don’t know if the peers know the epoch), and may include stale epochs that are not possible to reach quorum (such 2 host replacements to the same range would cause that historic range to not be able to reach quorum).

By sequencing EpochReady.coordinating, we have the property that we only mark sync complete for epoch=N if and only if epoch=N-1 has done it as well.  With this, peers are able to recover the past data when a new epoch is seen.
Bug: Test Failure: org.apache.cassandra.distributed.test.tcmCMSPlacementAfterMoveTest.testMoveToCMS
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Aug/24 15:07
Resolved: 18/Dec/24 13:05

Seen [here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2731/workflows/b987651a-5280-4dd6-9895-3298456406c8/jobs/61576/tests#failed-test-0] and [here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2731/workflows/b987651a-5280-4dd6-9895-3298456406c8/jobs/61577/tests#failed-test-0]


{code:java}
junit.framework.AssertionFailedError: /127.0.0.4:7012 did not become a CMS member after 11 seconds
	at org.apache.cassandra.distributed.test.tcm.CMSPlacementAfterReplacementTest.lambda$assertInCMS$18759fb3$1(CMSPlacementAfterReplacementTest.java:104)
	at org.apache.cassandra.concurrent.FutureTask$2.call(FutureTask.java:124)
	at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)
{code}
Bug: CEP-44: Kafka integration for Cassandra CDC using Sidecar
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Sep/24 20:47
Resolved: 07/Dec/24 00:10

This is the initial commit for CEP-44 to introduce a standalone CDC module into the Analytics project. This module provides the foundation for CDC in the Apache Cassandra Sidecar.
    
    This module provides:
    - a standalone Cdc class as the entrypoint for initializing CDC.
    - pluggable interfaces for: listing and reading commit log segments for a token range, persisting and reading CDC state, providing the Cassandra table schema, optionally reading values from Cassandra.
    - read and deserialize commit log mutations.
    - reconcile and de-duplicate mutations across replicas.
    - serialize CDC state into a binary object for persistence.
    - a layer for converting Cassandra mutations into a standard consumable format.
Bug: Accord: Improve burn test stability
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Aug/24 09:58
Resolved: 04/Dec/24 09:33

Various fixes and improvements to get burn test running more cleanly. This includes removing the concept of non-participating home keys, simplifying the “covers” concept for validation, calculating InvalidIfNot per key rather than per shard, and a mix of minor bug fixes.
Bug: Accord: DefaultProgressLog
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Aug/24 12:04
Resolved: 04/Dec/24 09:33

Redesign progress mechanisms to be memory efficient, use fewer messages and to resolve dependency chains promptly.

The SimpleProgressLog had a number of problems:

# It polled for progress with no attempt to determine whether progress could realistically be made, so:
## as the number of pending transactions grew, the proportion of useful work dropped (as many would be unable to make progress without earlier transactions completing)
## each transaction in the chain could recover only on average 1/2 poll interval behind the last transaction to complete
# It requested full transaction state from every replica on each attempt
# It maintained a lot of in-memory state
# Polling happened en-masse, allowing for little per-transaction control
We also separately maintained fairly expensive per-command listener state that negatively affected our command loading and caching.

The new DefaultProgressLog makes use of several new features: LocalListeners, RemoteListeners, Timers and Await messages.
- LocalListeners provide a memory-efficient collection for managing each CommandStore’s transaction listeners, with dedicated record keeping for inter-transaction relationships.
- RemoteListeners provide a mechanism for request/response pairs that may be separated by longer than the normal Cassandra message timeout, and require minimal state on sender and recipient. This permits replicas to cheaply update their local state machine as soon as distributed information becomes available.

The DefaultProgressLog tracks each transaction with separate timers to handle per-transaction scheduling, backoff etc, and a succinct state machine. To reduce overhead correspondence is preferentially limited to a handful of replicas, and limited to the home shard where appropriate.
Bug: Accord: Refine epoch quorum intersections, sync points, conflict relations and outcome inference
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Sep/24 13:16
Resolved: 04/Dec/24 09:33

- ExclusiveSyncPoints must imply global cluster properties, by executing on all participating epochs so that across all healthy replicas we can know all lower TxnId are durably applied
- Conflict relations are managed at Token rather than Key level to improve memory occupancy for workloads with large keys
- Epoch quorum intersections have been updated: preaccept no longer needs to chase the execution epoch, and recovery now contacts unmigrated epochs for dependency calculation; fast path decisions only involve the proposal epoch. This behaviour is now partially configurable.
- Simplifies and fixes the inference of a transaction's outcome when some shards have truncated the transaction
- Refined internal representations of key ownership across epoch changes
Bug: CFK Serializer buffer underflow
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Aug/24 19:13
Resolved: 02/Dec/24 11:37

{code}
java.lang.AssertionError: 6884646163227752822 seed failed

	at org.apache.cassandra.service.accord.serializers.CommandsForKeySerializerTest.testOne(CommandsForKeySerializerTest.java:487)
	at org.apache.cassandra.service.accord.serializers.CommandsForKeySerializerTest.serde(CommandsForKeySerializerTest.java:359)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{code}
Bug: Fix NPE in InformDurableSerializers
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Aug/24 21:00
Resolved: 02/Dec/24 11:37

Currently, R/W concurrent workload may throw:
{code}
java.lang.NullPointerException: null
	at org.apache.cassandra.service.accord.serializers.CommandSerializers$TimestampSerializer.serialize(CommandSerializers.java:97)
	at org.apache.cassandra.service.accord.serializers.InformDurableSerializers$1.serializeBody(InformDurableSerializers.java:39)
	at org.apache.cassandra.service.accord.serializers.InformDurableSerializers$1.serializeBody(InformDurableSerializers.java:35)
	at org.apache.cassandra.service.accord.serializers.TxnRequestSerializer.serialize(TxnRequestSerializer.java:46)
	at org.apache.cassandra.service.accord.serializers.TxnRequestSerializer.serialize(TxnRequestSerializer.java:31)
	at org.apache.cassandra.net.Message$Serializer.serialize(Message.java:833)
	at org.apache.cassandra.distributed.impl.Instance.serializeMessage(Instance.java:445)
	at org.apache.cassandra.distributed.impl.Instance.lambda$registerOutboundFilter$5(Instance.java:388)
	at org.apache.cassandra.net.OutboundSink$Filtered.accept(OutboundSink.java:54)
	at org.apache.cassandra.net.OutboundSink.accept(OutboundSink.java:70)
	at org.apache.cassandra.net.MessagingService.send(MessagingService.java:534)
	at org.apache.cassandra.net.MessagingService.send(MessagingService.java:472)
	at org.apache.cassandra.service.accord.AccordMessageSink.send(AccordMessageSink.java:228)
	at accord.local.Node.send(Node.java:526)
	at accord.local.Node.lambda$send$8(Node.java:464)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at accord.local.Node.send(Node.java:464)
	at accord.coordinate.MaybeRecover.onDone(MaybeRecover.java:109)
	at accord.coordinate.ReadCoordinator.handle(ReadCoordinator.java:241)
	at accord.coordinate.ReadCoordinator.onSuccess(ReadCoordinator.java:145)
	at accord.coordinate.ReadCoordinator.onSuccess(ReadCoordinator.java:43)
{code}
Bug: 2i created in C* 2.2 with COMPACT STORAGE are not readable from 4.x
Status: Open
Priority: Normal
Created: 03/Sep/24 14:38

While attempting to read a historical secondary index created in Cassandra 2.2 and upgraded to 4.0, the index isn't readable.

The issue can be circumvene by dropping and recreating the 2i but considering COMPACT STORAGE is still covered in Cassandra 5.0 even though deprecated, I think this should be reviewed

Sample schema:
{code:java}
CREATE KEYSPACE testkeyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
USE testkeyspace;
CREATE TABLE repro01 (id text, c2 text, version text, primary key(id)) WITH COMPACT STORAGE;
INSERT INTO repro01 (id,c2,version) values ('a','b','v01');
CREATE INDEX idx_repro01 ON repro01 (c2);
{code}
 

Sample output in 2.2 based on script:
{code:java}
Waiting for Cassandra 2.2 to start...

TEST WITH COMPACT STORAGE
 id | c2 | version
----+----+---------
  a |  b |     v01

(1 rows) {code}
 

Output in 4.0
{code:java}
Waiting for Cassandra 4.0 to start...

TEST WITH COMPACT STORAGE
 id | c2 | version
----+----+--------- 

(0 rows){code}
 

Error extracted from log
{code:java}
ERROR [SSTableBatchOpen:1] 2024-09-03 14:19:52,118 CassandraDaemon.java:577 - Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.IllegalStateException: org.apache.cassandra.exceptions.UnknownColumnException: Unknown column value during deserialization
    at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:505)
    at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:372)
    at org.apache.cassandra.io.sstable.format.SSTableReader$2.run(SSTableReader.java:540)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.apache.cassandra.exceptions.UnknownColumnException: Unknown column value during deserialization
    at org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:337)
    at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:501)
    ... 8 common frames omitted{code}
Sample script was created with docker to reproduce the upgrade process and the shell script attached *_docker_cass_upgrade.sh_*
Bug: cassandra docs should default to 5.0
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Sep/24 17:42
Resolved: 26/Nov/24 08:19

Now that 5.0 is released we should default results from the doc search to 5.0 pages.  At the moment people get docs for 4.1.  5.0 docs don't appear to be indexed at all.  Searching for UCS for example, shows the glossery, and bti doesn't show anything.

 

!image-2024-09-17-10-40-06-432.png!
Bug: Introduce system property to prevent schema modifications
Status: Needs Committer
Priority: Normal
Created: 12/Aug/24 17:44

This is the first ticket as described here (1) in order to eventually merge CASSANDRA-19556.

(1) https://issues.apache.org/jira/browse/CASSANDRA-19556?focusedCommentId=17856432&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17856432

I created this ticket to have type "New Feature" but this needs to be merged to 4.0, 5.0 for the purposes of CASSANDRA-19556. "Task" would probably make it too.
Bug: Enhance CQLSSTableWriter to notify clients on sstable production
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Jul/24 22:14
Resolved: 22/Nov/24 23:04

Notifying when SSTables are produced is useful for CQLSSTableWriter clients to have a better control on processing the SSTables. For example, Cassandra Analytics can leverage the notification to determine when to import the sstables.