av9ash/GitBugs huggingface dataset:
Bug: trunk fails to start
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 24/May/24 18:43
Resolved: 26/Jul/24 23:05

On commit {{67bbbb01259bce91672a7c3ca9fb77ea7b040e9c}}, I get errors on startup.

Verified the build was successful:

{noformat}
    easy-cass-lab.amazon-ebs.ubuntu: BUILD SUCCESSFUL
    easy-cass-lab.amazon-ebs.ubuntu: Total time: 1 minute 41 seconds
{noformat}

Running on a new Ubuntu instance:

{noformat}
INFO  [main] 2024-05-24 18:31:16,397 YamlConfigurationLoader.java:103 - Configuration location: file:/usr/local/cassandra/trunk/conf/cassandra.yaml
ERROR [main] 2024-05-24 18:31:16,470 CassandraDaemon.java:900 - Exception encountered during startup
java.lang.NoSuchMethodError: 'void org.yaml.snakeyaml.LoaderOptions.setCodePointLimit(int)'
	at org.apache.cassandra.config.YamlConfigurationLoader.getDefaultLoaderOptions(YamlConfigurationLoader.java:433)
	at org.apache.cassandra.config.YamlConfigurationLoader$CustomConstructor.<init>(YamlConfigurationLoader.java:278)
	at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:135)
	at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:116)
	at org.apache.cassandra.config.DatabaseDescriptor.loadConfig(DatabaseDescriptor.java:403)
	at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:265)
	at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:250)
	at org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:781)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:724)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:878)
{noformat}

Running on Java 17:

{noformat}
ubuntu@cassandra0:~$ java -version
openjdk version "17.0.10" 2024-01-16
OpenJDK Runtime Environment (build 17.0.10+7-Ubuntu-122.04.1)
OpenJDK 64-Bit Server VM (build 17.0.10+7-Ubuntu-122.04.1, mixed mode, sharing)
{noformat}

Built with 11.

The only configs I changed:

{noformat}
cluster_name: "system_views"
num_tokens: 4
seed_provider:
  class_name: "org.apache.cassandra.locator.SimpleSeedProvider"
  parameters:
    seeds: "10.0.0.225"
hints_directory: "/mnt/cassandra/hints"
data_file_directories:
- "/mnt/cassandra/data"
commitlog_directory: "/mnt/cassandra/commitlog"
concurrent_reads: 64
concurrent_writes: 64
trickle_fsync: true
endpoint_snitch: "Ec2Snitch"
{noformat}
Bug: Add guardrail to block DDL/DCL queries and replace alter_table_enabled guardrail
Status: Needs Committer
Priority: Normal
Created: 11/Apr/24 18:08

Sometimes we want to block DDL/DCL queries to stop new schemas being created or roles created. (e.g. when doing live-upgrade)

For DDL guardrail current implementation won't block the query if it's no-op (e.g. CREATE TABLE...IF NOT EXISTS, but table already exists, etc. The guardrail check is added in apply() right after all the existence check)

I don't have preference on either block every DDL query or check whether if it's no-op here. Just we have some users always run CREATE..IF NOT EXISTS.. at startup, which is no-op but will be blocked by this guardrail and failed to start.

 

4.1 PR: [https://github.com/apache/cassandra/pull/3248]

trunk PR: [https://github.com/apache/cassandra/pull/3275]
Bug: Refactor ColumnCondition post CASSANDRA-19341
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/May/24 20:43
Resolved: 25/Jul/24 08:27

CASSANDRA-19341 added ColumnsExpression in restrictions. 
This ticket's target is to extend the kinds of ColumnsExpression and use them in conditions, too. 
Also, it should add a few other simplifications around ColumnCondition, which were suggested by [~blerer]
Bug: Website - Download section - Update / remove EOL dates
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Apr/24 11:17
Resolved: 20/Jul/24 14:16

Enterprise customers with on-prem Cassandra usage can be pretty nitpicking in terms of EOL, running unsupported Cassandra versions and they often refer to what is stated in https://cassandra.apache.org/_/download.html (as the only source available?) and don't really think about the dependency to 5.0 GA, but just reflecting EOL date information there.

As of April 11, 2024, the download section states the following information:
 !image-2024-04-11-13-15-52-317.png! 

According to that, 3.x is unmaintained, 4.0 soon to be EOL etc ...

Either remove these EOL estimates or keep them strongly maintained aligned with an updated 5.0 GA timeline.

Thanks!
Bug: Add auto_hints_cleanup_enabled to web documentation
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 05/Jun/24 21:50
Resolved: 19/Jul/24 17:30

{{auto_hints_cleanup_enabled flag in cassandra.yaml is not documented in the [Cassandra web documentation|https://cassandra.apache.org/doc/stable/cassandra/operating/hints.html].}}

Let’s add it there for completion.
Bug: CCM wrongly interprets CASSANDRA_USE_JDK11 for Cassandra 4.x
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/May/24 06:57
Resolved: 24/May/24 10:59

CCM interprets {{CASSANDRA_USE_JDK11}} only by its existence in the environment rather than by its actual value (true/false). 

I can see two solutions:
- make it interpret {{CASSANDRA_USE_JDK11}} properly
- do not take into account {{CASSANDRA_USE_JDK11}} in the current env and set it or unset it automatically when starting a node basing on which Java version was selected
Bug: LWT conditions behavior on collections is inconsistent
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/May/24 14:27
Resolved: 27/Jun/24 11:44

LWT conditions behaviour on collections is inconsistent in several ways around null values:

1)+Conditions comparing a collection column with a {{null}} value to a non-null have a different behaviour for frozen and non-frozen collection+.

 {code}UPDATE myTable SET l = ? WHERE k = 0 IF l < [1, 2]{code}
If {{l}} is null the previous query will return {{[false, null]}} for a frozen collection and {{[true]}} for a non-frozen collection. 

2) +Conditions on non-frozen collection treat empty differently from null+

Due to the way multi-cell collections are implemented, it is not possible to differentiate between {{null}} and empty collections like it is feasible for single cell (frozen) collections. Therefore an empty multi-cell collection will always be treated as {{null}}.
Unfortunately, the way LWT conditions handle that is not consistent with that.

For example for {{colA list<int>}} non null: {code}.. IF colA >= null{code} will throw an invalid request error whereas {code}..IF colA >= []{code} will returns {{true}}.
Moreover, if we insert an empty list through:
{code}INSERT INTO mytable (pk, colA) VALUES (1, []);{code}
and use {code}DELETE FROM mytable WHERE pk=1 IF colA = []{code} the returned results will be {code}{false, null}{code}. Which can be quite confusing.

The way to fix that behaviour to make it consistent with other operations is to consider empty multi-cell collection input as {{null}} and reject the {{null}} input for non {{=}} and {{!=}} operators.
Bug: Latency regression on 4.1 comparing to 4.0
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 04/Apr/24 18:38
Resolved: 11/Apr/24 14:39

When upgrading from Cassandra 4.0.10 to 4.1.3, I noticed an increase from application point of view latency from ~8ms to ~15ms when upgrading to Cassandra. The latency includes 3 simple queries (INSERT + SELECT (PK+CK) + UPDATE) plus application overhead.
It has been investigated in CASSANDRA-18766 to realize it is not related.
I tested to downgrade to 4.1alpha1 and the latency regression is still there with same value.

The version 4.1.4  has the same issue.

In a graph how it looks like 
!screenshot-1.png!
Bug: Test failure: org.apache.cassandra.cql3.validation.operations.DropRecreateAndRestoreTest.testCreateWithIdRestore
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 10/Apr/24 16:42
Resolved: 10/Apr/24 22:42

{noformat}
junit.framework.AssertionFailedError: Got less rows than expected. Expected 2 but got 1
	at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1256)
	at org.apache.cassandra.cql3.validation.operations.DropRecreateAndRestoreTest.testCreateWithIdRestore(DropRecreateAndRestoreTest.java:80)
{noformat}

As seen at https://app.circleci.com/pipelines/github/driftx/cassandra/1579/workflows/03c0b2f6-d84f-440d-baad-bf323810a292/jobs/84088
Bug: Flakey test upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_indev_4_0_x_To_indev_trunk::test_non_eq_conditional_update
Status: Open
Priority: Normal
Created: 03/Apr/24 16:47

{code}
ERROR [HintsDispatcher:1] 2024-04-03 06:28:19,398 JVMStabilityInspector.java:70 - Exception in thread Thread[HintsDispatcher:1,1,HintsDispatcher]
java.lang.IllegalArgumentException: serializedSize() called with non-matching version 14
	at org.apache.cassandra.hints.HintMessage$Serializer.serializedSize(HintMessage.java:99)
	at org.apache.cassandra.hints.HintMessage$Serializer.serializedSize(HintMessage.java:77)
	at org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1155)
	at org.apache.cassandra.net.Message.payloadSize(Message.java:1212)
	at org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:813)
	at org.apache.cassandra.net.Message.serializedSize(Message.java:1187)
	at org.apache.cassandra.net.OutboundConnections.connectionTypeFor(OutboundConnections.java:215)
{code}
Bug: IntergerIntervalsTest may fail due to integer overflow
Status: Triage Needed
Priority: Normal
Created: 10/Apr/24 13:40

{noformat}
junit.framework.AssertionFailedError
	at org.apache.cassandra.utils.IntegerInterval$Set.add(IntegerInterval.java:138)
	at org.apache.cassandra.utils.IntegerIntervalsTest.lambda$testSetAddMultiThread$5(IntegerIntervalsTest.java:252)
	at java.base/jdk.internal.util.random.RandomSupport$RandomIntsSpliterator.forEachRemaining(RandomSupport.java:1002)
	at java.base/java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:617)
	at org.apache.cassandra.utils.IntegerIntervalsTest.testSetAddMultiThread(IntegerIntervalsTest.java:252)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}

https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/1326/workflows/021d350a-4b62-44af-9650-f5a0eb105522/jobs/70420/tests
Bug: Paxos may re-distribute stale commits that predate a collectable tombstone
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/May/24 17:13
Resolved: 11/Jun/24 13:37

Note: this bug only affects {{paxos_state_purging: {gc_grace, repaired}}}, i.e. those introduced alongside Paxos v2.

There are two problems:
1) Purging is applied only on compaction, not on load, which can lead to very old commits being resurfaced in certain circumstances
2) PaxosPrepare does not filter commits based on paxos repair low bound

This permits surprising situations to arise, where some replicas purge a stale commit _and all newer commits_, but due to compaction peculiarities some other replica may purge only the newer commits, leaving a stale commit in some compaction "purgatory"\[1] to be returned to reads indefinitely. 

So long as there are no newer commits, the paxos coordinator will see this commit is not universally known and redistribute it - no matter how old it is. This can permit an insert to be reapplied after GC grace has elapsed and the tombstone has been collected.

For proposals this is not a problem, as we correctly filter proposals based on the last paxos repair time. This also does not affect clusters with the legacy (and default) paxos state purging using TTL. Problem (1) only applies also to the new {{gc_grace}} compatibility mode for purging.

\[1] Compaction purgatory can arise for instance because paxos purging allows whole sstables to be erased quite effectively, and if this is able to ordinarily prevent sstables being promoted to L1, then if for some abnormal reason sstables reach L1 (e.g. repairs being disabled for some time), those that collect may remain uncompacted for an extended period without purging being applied.
Bug: Consider adding exceptions among shared packages in in-jvm dtest api
Status: Triage Needed
Priority: Normal
Created: 27/Jun/24 19:52

This does not work

{code}
try (Cluster ignored = build(1)
                       .withConfig(c -> c.with(Feature.NETWORK, Feature.NATIVE_PROTOCOL, Feature.GOSSIP)
                                         .set("some_property_which_throws_when_misconfigured", value))
                       .start())
{
    fail("should throw ConfigurationException");
}
catch (Exception ex)
{
    assertEquals(ConfigurationException.class, ex.getClass());
}
{code}

Because ConfigurationException is not shared so it looks like it doesn't match.

this is workaround

{code}
        Predicate<String> EXTRA = className -> className.equals(ConfigurationException.class.getName());
        Cluster.Builder builder = build(1);
        assertThatThrownBy(() -> builder
                                 .withSharedClasses(EXTRA.or(builder.getSharedClasses()))
                                 .start())
        .isInstanceOf(ConfigurationException.class);
{code}

But to have it nice and clean like this, it would require us to share all exceptions, to add them to "DEFAULT_SHARED_PACKAGES". in "org.apache.cassandra.distributed.shared.InstanceClassLoader".

{code}
        assertThatThrownBy(() -> build(1).withConfig(c -> c.set("some_property_which_throws_when_misconfigured", value)).start())
        .isInstanceOf(ConfigurationException.class);
{code}

We could just put all package there.

This would require release of in-jvm dtest api.
Bug: Always repair the full range when repairing system_cluster_metadata
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Jun/24 12:54
Resolved: 10/Jul/24 15:06

Since system_cluster_metadata uses a custom partitioner we need to repair the full range whenever an operator uses -st and -et tokens to avoid breaking existing repair automations. This is fine since the amount of data in this keyspace should be small.

https://github.com/krummas/cassandra/commit/6ab8fcd1126f67b0117e1ee3c1fd1d4b40ac2362
Bug: Ignore repair requests for system_cluster_metadata
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Jun/24 12:59
Resolved: 10/Jul/24 15:07

Since system_cluster_metadata is not replicated like other keyspaces we might break existing repair automation if a {{nodetool repair}} is run against a node not in the CMS. Just ignore the request if so.

https://github.com/krummas/cassandra/commit/76437723acea35421ec5bf0412dcdee1411dcb6e
Bug: Avoid ClassCastException when verifying tables with reversed partitioner
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Jun/24 12:57
Resolved: 10/Jul/24 15:07

A few TCM tables use a custom partitioner, this causes class cast exception when running nodetool verify on them.

https://github.com/krummas/cassandra/commit/64897cb6382967f3e134752f5b9f223ff7daeb84
Bug: Use table-specific partitioners during Paxos repair
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Jun/24 13:42
Resolved: 10/Jul/24 15:05

Partition keys in the \{{system.paxos}} table are derived from the key involved in the paxos transaction. Initially, it was safe to assume that the paxos table itself used the same partitioner as the tables in the transactions as all distributed keyspaces and tables were configured with the global partitioner. This is no longer true as the \{{system_cluster_metadata.distributed_metadata_log}} has its own custom partitioner. 


Likewise, \{{PaxosRepairHistory}} and the \{{system.paxos_repair_history}} table which makes that history durable map token ranges in the transacted tables to ballots. Prior to CASSANDRA-19482 it was safe to assume that these ranges contained tokens from the global partitioner but as this is no longer the case, we must use the specific partitioner for the table in question when working with ranges during paxos repair.
Bug: wrap tracing logs in isTraceEnabled across the codebase
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/May/24 10:22
Resolved: 30/May/24 14:49

Our usage of logger.isTraceEnabled across the codebase is inconsistent. This would also fix issues similar in e.g. CASSANDRA-19429 as [~rustyrazorblade] suggested.

We should fix this at least in trunk and 5.0 (not critical though) and probably come up with a checkstyle rule to prevent not calling isTraceEnabled while logging with TRACE level.
Bug: Incorrect date format in SettingsGraph
Status: Triage Needed
Priority: Normal
Created: 22/May/24 07:02

            ? "cassandra-stress - " + new SimpleDateFormat("yyyy-mm-dd hh:mm:ss").format(new Date())
should be
            ? "cassandra-stress - " + new SimpleDateFormat("yyyy-MM-dd hh:mm:ss").format(new Date())
Bug: Remove sid from bullseye docker images
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Jun/24 14:41
Resolved: 05/Jul/24 16:32

sid is flakey, often broken and takes days for correct packages to be uploaded.

ref: https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/jdk=jdk_1.8_latest,label=cassandra/611/ 

sid is only used for jdk8

looks like replacing it with temurin might be a safer/stable choice.
Bug: Add feature to filter streaming source when bootstrapping
Status: Patch Available
Priority: Normal
Created: 28/Jun/24 04:14

h2. Proposed improvement

Right now, the stream sources for the bootstrap are chosen from the closest nodes of the bootstrapping node.

Add the following system properties so that when bootstrap happens, the node can choose where to stream data from.
 * {{cassandra.bootstrap.include_dcs}}
 * {{cassandra.bootstrap.exclude_dcs}}
 * {{cassandra.bootstrap.include_sources}}

{{cassandra.bootstrap.include_dcs and }}{{cassandra.bootstrap.exclude_dcs can take args like "dc1,dc2" to specify DCs or "dc1:rack1,dc1:rack2,..." to specify DC/Racks.}}

{{cassandra.bootstrap.include_sources is used to specify ip address/port of the specific nodes to stream from.}}
h2. Motivation

Currently, when the node failure happens in the middle of the major cluster upgrade, general advice given to the user is to complete the upgrade of the cluster until all nodes are in the same version, and replace/remove the failed node. This is because the streaming breaks when there is unsupported SSTable version streamed from newer to the older.

This approach can create availability issue when two nodes (the failed node from different rack and the node currently stopped for the upgrade) are down.

With this improvement, the user can replace the failed node using, for example, specific rack in the same DC or another DC, to eliminate the availability issue described above.
h2. Note

The user also needs to set "-Dcassandra.skip_schema_check=true" when replacing the node during the major upgrade to complete the replacement.
Bug: UnsupportedOperationException is thrown when no space for LCS
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Jun/24 00:30
Resolved: 02/Jul/24 20:12

In {{CompactionTask#buildCompactionCandidatesForAvailableDiskSpace}} with LCS, if node has limited disk space and can't remove any sstable from L0 or L1 in {{{}LeveledCompactionTask#reduceScopeForLimitedSpace{}}}, {{LeveledCompactionTask#partialCompactionsAcceptable}} will throw {{UnsupportedOperationException}}.

We should handle {{LeveledCompactionTask#partialCompactionsAcceptable}} more gracefully with {{return level <= 1}} or simply {{true}} since {{reduceScopeForLimitedSpace}} only removes sstable from L0 or L1.

Related https://issues.apache.org/jira/browse/CASSANDRA-17272
Bug: CEP-15: (Accord) When nodes are removed from a cluster, need to update topology tracking to avoid being blocked
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Jun/24 20:54
Resolved: 01/Jul/24 20:18

When doing a host replacement or decom nodes will leave the cluster and won't ever come back, this can put accord's topology sync logic into a bad state as its going to wait forever for those nodes to acknowledge the epoch or parts of the ranges.
Bug: ShallowInfoRetriever: cache offsets to void resetting of RandomAccessReader buffer
Status: In Progress
Priority: Normal
Created: 21/May/24 16:19

 Currently in org.apache.cassandra.io.sstable.format.big.RowIndexEntry.ShallowInfoRetriever#fetchIndex we do 2 seek/read operations: 1st is to find the offset for IndexInfo and the 2nd to read it. These are two quite distant regions of the file and for standard disk access mode we do not use a benefit from a buffer in RandomAccessReader due to jumping between the regions and reseting this buffer again and again. A possible improvement here can be to read and cache N first offsets (to limit the amount of memory to use) on the first read and do later only sequential reads of IndexInfo data. By caching of less than 1Kb we can reduce the number of syscalls even more, in my case: from few hundred to less than 10.
Bug: [Analytics] Bulk writer fails validation stage when writing to a cluster using RandomPartitioner
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Jun/24 22:27
Resolved: 27/Jun/24 22:51

In bulk writer after writing SSTables, written data to SSTables is read back again to perform validations before shipping the files to the Cassandra instances. The logic to validate the SSTables assumes {{{}Murmur3Partitioner{}}}. This validation fails however when a bulk writer job is running against a cluster using the {{RandomPartitioner}} with the following stacktrace:


{code}
java.lang.RuntimeException: java.lang.IllegalStateException: Partitioner in ValidationMetadata does not match TableMetaData: org.apache.cassandra.dht.RandomPartitioner vs. org.apache.cassandra.dht.Murmur3Partitioner
	at org.apache.cassandra.spark.utils.Throwing.lambda$function$2(Throwing.java:84)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.apache.cassandra.spark.data.BasicSupplier.openAll(BasicSupplier.java:44)
	at org.apache.cassandra.bridge.CassandraBridgeImplementation.getCompactionScanner(CassandraBridgeImplementation.java:248)
	at org.apache.cassandra.spark.data.DataLayer.openCompactionScanner(DataLayer.java:262)
	at org.apache.cassandra.spark.bulkwriter.SSTableWriter.validateSSTables(SSTableWriter.java:129)
	at org.apache.cassandra.spark.bulkwriter.SSTableWriter.close(SSTableWriter.java:113)
	at org.apache.cassandra.spark.bulkwriter.RecordWriter.finalizeSSTable(RecordWriter.java:224)
	at org.apache.cassandra.spark.bulkwriter.RecordWriter.write(RecordWriter.java:122)
	... 15 more
{code}
Bug: In-JVM dtest JMX support should support nodetool usage
Status: Triage Needed
Priority: Normal
Created: 27/Jun/24 17:19

As we continue to push the boundaries of what the in-jvm dtest framework can do, we’ve found an edge-case where nodetool can’t create a NodeProbe instance because the JVM memory and Runtime mbeans aren’t exposed via the IsolatedJmx class. Registering these when starting up an IsolatedJmx instance allows NodeProbe to be properly instantiated and the nodetool command-line tool to work correctly.