av9ash/GitBugs huggingface dataset:
Bug: Unit test org.apache.cassandra.tools.TopPartitionsTest#testServiceTopPartitionsSingleTable is still flaky
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Oct/24 21:33
Resolved: 10/Oct/24 17:17

org.apache.cassandra.tools.TopPartitionsTest#testServiceTopPartitionsSingleTable fails randomly and quite frequently on CI:

[https://ci-cassandra.apache.org/view/Cassandra%204.1/job/Cassandra-4.1/521/testReport/org.apache.cassandra.tools/TopPartitionsTest/]

[https://ci-cassandra.apache.org/job/Cassandra-4.0/726/testReport/junit/org.apache.cassandra.tools/TopPartitionsTest/]

Based on the source code - the most probably cause of this flakiness is async logic when we submit a sample here: org.apache.cassandra.metrics.Sampler#addSample - the submitted task can executed after we invoke finishLocalSampling(..) within the test.

The failure can be emulated by adding a synthetic delay to this logic like this:
{code:java}
public void addSample(final T item, final int value)
{
    if (isEnabled())
        samplerExecutor.submit(() -> {
            // ------ synthetic delay -----
            try
            {
                Thread.sleep(10000);
            } catch (Exception e) {};
            // ----------------------- -----
            insert(item, value);
        });
}{code}
Bug: Auto-delete snapshots at X% disk full
Status: Triage Needed
Priority: Normal
Created: 29/Oct/24 05:41

One of the possible enhancements that came out from the DISCUSSION as part of CEE-37 is to auto-delete snapshots at X% disk full.

Snippets from the ML discussion (attached)

[^image-2024-10-28-22-41-59-373.png]
Bug: Update Jacoco dependency for JDK21 support
Status: Resolved
Priority: Normal
Resolution: Later
Created: 31/Oct/24 18:59
Resolved: 31/Oct/24 19:08

Currently, 5.0 and trunk use JDK21 but Jacoco 0.8.8.
 
Jacoco 0.8.8 does not support JDK21: [https://github.com/apache/cassandra/blob/15ed18e9d49f48e88f40b90c156248b8b697c7e2/build.xml#L146]
 
Need to update to Jacoco 0.8.11+: [https://github.com/jacoco/jacoco/releases/tag/v0.8.11]
 
Might as well go to 0.8.12 (latest as of writing).
Bug: Memory leak in Bootstraping
Status: Triage Needed
Priority: Normal
Created: 30/Oct/24 21:05

We recently updated Cassandra to 4.1.7 from 4.0.3 and noticed that bootstrap started to consume a lot of memory.
I have a feeling that there is a memory leak somewhere, because even 32GB of heap is not enough and the node becomes unresponsive (long pauses on the GC).

!2024-10-31_01-23-24-resized.png!
A simple analysis (jmap -histo:live) shows that we have a constantly growing org.apache.cassandra.io.compress.CompressionMetadata$Chunk

If required, I am ready to provide additional information
Bug: Consider reintroducing RestrictionSet#iterator() optimization around multi-column restrictions
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Oct/24 22:19
Resolved: 29/Oct/24 19:54

Before CASSANDRA-19341, {{RestrictionSet#iterator()}} only worried about eliminating duplicate restrictions when multi-column restrictions were present. After that refactor, we always create a new {{LinkedHashSet}} to wrap the {{SingleRestriction}} values, and this is observable in allocation profiles for any write-heavy workload. (See attached flame graphs.) Specifically, we can see the extra allocations from codepaths starting at {{ModificationStatement#addFunctionsTo()}} and {{ModificationStatement#getMutations()}}.

We could fix this problem by simply reintroducing the {{hasMultiColumnRestrictions}} flag in {{RestrictionSet}}, but then we have to decide whether to also reintroduce {{DistinctIterator}} or leave it out and continue to use {{LinkedHashSet}}. Perhaps multi-column restrictions are not common enough to worry about it, but {{HashMap}} nodes and entries are not free, and this is a hot path.
Bug: Add support for Role's OPTIONS
Status: Changes Suggested
Priority: Normal
Created: 15/Oct/24 11:44

The Cassandra Query Language [https://cassandra.apache.org/doc/stable/cassandra/cql/security.html] / [https://cassandra.apache.org/doc/5.0/cassandra/developing/cql/security.html] specify that a role can have custom options defined as literal map.

The documentation shows a valid example of these custom options:

{{CREATE ROLE carlos WITH OPTIONS = \{ 'custom_option1' : 'option1_value', 'custom_option2' : 99 };  }}

However, the storage/retrieval of such custom options has not been implemented in Cassandra. See for instance, [https://github.com/apache/cassandra/blob/18960d6e3443bf002ef4f46c7f0e1f2ee99734e1/src/java/org/apache/cassandra/auth/CassandraRoleManager.java#L393-L396]

Storing custom options per role could have multiple usages, for instance, it could allow admins to specify fine-grain permissions that can be interpreted by custom authenticator/authorizer.

The goal of this task is to add support for Role custom options, by storing them in an additional table called {{role_options}} in the {{system_auth}} keyspace.

Creating a role with options should write the information in both the {{roles}} and the {{role_options}} tables. Creating a role with no options or having an empty map of options should not write any information in the {{role_options}} table.

Altering a role should behave as follows when executing an {{ALTER ROLE}} statement:
 * without specifying {{{}OPTIONS{}}}: no changes should be done in the {{role_options}} table. 
 * specifying {{OPTIONS}} altering a role with no previous custom options: we should insert the custom options in the {{role_options}} table.
 * specifying {{OPTIONS}} altering a role with previous custom options: we should replace the existent custom options
 * in the {{role_options}} table.

Dropping a role should drop information in both {{roles}} and {{{}role_options{}}}.
Bug: Suppress CVE-2024-45772 lucene-core-9.7.0.jar
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Oct/24 12:56
Resolved: 29/Oct/24 11:47

CVE in Lucene Replicator which the project doesn't use.

Trunk can also upgrade, while we're at it.

https://nvd.nist.gov/vuln/detail/CVE-2024-45772

Reproduced with `ant dependency-check`
Bug: METADATA_CHANGED is never set
Status: Open
Priority: Normal
Created: 25/Oct/24 15:26

CASSANDRA-10786 added the METADATA_CHANGED flag to the v5 spec (ROWS RESPONSE). Drivers are supposed to handle this flag and update their locally cached prepared statement object with the new metadata.

As we're working to implement v5 support on GoCQL we are trying to trigger a scenario where C* returns this flag but we are unable to. Instead we receive an UNPREPARED which causes the driver to re-prepare the statement. This flag was added to avoid this scenario so I don't really understand if there is a bug somewhere or if the test case we're creating is not valid?

I ran the following steps against C* 4.0-rc1, 4.0.0 and 5.0.2 with the same results:

{code}
CREATE KEYSPACE testks WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor': 1 }

CREATE TABLE testks.testtable (id uuid PRIMARY KEY, txt TEXT)

INSERT INTO testks.testtable(id, txt) VALUES (uuid(), 'test')

// PREPARED_ID: 2bea4b54ec20bd3533f7b91467abb618 
// RESULT METADATA ID: 1f685bd34ed0ba1e99ba6bd960da2545
SELECT * FROM testks.testtable

ALTER TABLE testks.testtable ADD (txt2 TEXT)

INSERT INTO testks.testtable(id, txt, txt2) VALUES (uuid(), 'test2', 'test222')

// this SELECT gets an UNPREPARED response from the server 
// instead of a ROWS response with the new metadata
//
// after RE-PREPARE: 
// PREPARED_ID: 2bea4b54ec20bd3533f7b91467abb618 
// RESULT METADATA ID: 87257e07eccd14a6c8e623012af9278a
SELECT * FROM testtable
{code}

Note that v5 support on gocql is still WIP so I imported gocql [from this branch|https://github.com/apache/cassandra-gocql-driver/pull/1822] to test this with gocql.

I believe [~lukasz.antoniak] was able to reproduce this with the java driver which has had v5 support for a while and got the same results, I'll ask him to post a comment on this ticket.


*UPDATE*

If the last SELECT is sent with the old metadata ID then we are able to see the METADATA_CHANGED flag but this would only occur if the driver does not update its local prepared statement object after the re-PREPARE. The expected behavior is that no re-PREPARE is needed in the first place.
Bug: Cassandra 5 min() and max() results incorrect when cluster order desc
Status: Open
Priority: Normal
Created: 24/Oct/24 19:02

Given:

CREATE TABLE min_max_bigint_desc(id text, created bigint, PRIMARY KEY (id, created)) WITH CLUSTERING ORDER BY (created DESC);

CREATE TABLE min_max_bigint_asc(id text, created bigint, PRIMARY KEY (id, created)) WITH CLUSTERING ORDER BY (created ASC);

SELECT * FROM min_max_bigint_asc;

 id | created

----+---------

  0 |       1

  0 |       2

  0 |       3

 

and data in both tables for both Cassandra 5 and 4, the same:

 

SHOW VERSION;SELECT id,min(created),max(created) FROM min_max_bigint_asc WHERE id='0';SELECT id,min(created),max(created) FROM min_max_bigint_desc WHERE id='0';

[cqlsh 6.2.0 | Cassandra 5.0.2 | CQL spec 3.4.7 | Native protocol v5]

 id | system.min(created) | system.max(created)

----+---------------------+---------------------

  0 |                   1 |                   3

 

 id | system.min(created) | system.max(created)

----+---------------------+---------------------

  0 |                   3 |                   1

 

[cqlsh 6.1.0 | Cassandra 4.1.7 | CQL spec 3.4.6 | Native protocol v5]

 id | system.min(created) | system.max(created)

----+---------------------+---------------------

  0 |                   1 |                   3

 

 id | system.min(created) | system.max(created)

----+---------------------+---------------------

  0 |                   1 |                   3

 

bigint and timestamp column types have the same behavior, and likely others but not explicitly verified.
Bug: Add vector type to protocol spec v6
Status: Open
Priority: Normal
Created: 23/Oct/24 15:23

I noticed there is not code for vector in (1). These codes are taken from the specification but looking into native_protocol_v5.spec for cassandra-5.0 (trunk too) there is not such code in section "4.2.5.2. Rows".

I am missing this code in native-protocol because I am integrating with this in another software.

After digging deeper, what we do in the driver is that it is done under CUSTOM type which is already in the spec but it is quite suspicious why we used CUSTOM when it should be a dedicated type of VECTOR.

[~absurdfarce] suggests that in v6 this should be a standalone type.

https://github.com/datastax/native-protocol/blob/1.x/src/main/java/com/datastax/oss/protocol/internal/ProtocolConstants.java#L118-L145
Bug: SAI (maybe) could support the blob CQL type
Status: Triage Needed
Priority: Normal
Created: 21/Oct/24 15:56

Hello,

we're currently exploring SAI for a use case, we have a table like this:

```
CREATE TABLE ucp.profile (
    project_key blob,
    profile_id blob,
    age int,
    PRIMARY KEY ((project_key, profile_id))
)
```

we tried using a SAI index but we get an error:
``` 
cqlsh:ucp> create custom index ucp_profile_project_key on ucp.profile (project_key) using 'sai';
InvalidRequest: Error from server: code=2200 [Invalid query] message="Unsupported type: blob"
```

The documentation is clear, it lists all the types that are supported, but it didn't say _why_ the `blob` type is not supported and it seemed a little weird to me that `blob` isn't supported because in my mind it should be the easiest type to support; I have no clue if that is _true_ but I started to investigate anyway.

I took a look at the code and stumbled upon [this line](https://github.com/apache/cassandra/blob/6bbe85ef51d2a5a52b9a585e7c1d3031a7fb0c80/src/java/org/apache/cassandra/index/sai/StorageAttachedIndex.java#L163-L168) which lists all the supported types. I thought "why not just add the BLOB type and see what happens ?"

So that's basically what I did and to my surprise it _seems_ to work from my limited testing:

```
cqlsh:ucp> CREATE TABLE ucp.profile (    project_key blob,    profile_id blob,   age int,    PRIMARY KEY ((project_key, profile_id)));
cqlsh:ucp> create custom index ucp_profile_profile_id on ucp.profile (profile_id) using 'sai';                                                                                                        
cqlsh:ucp> insert into profile(project_key, profile_id, age) values(0xcafe, 0xdeadbeef, 200);                                                                                                           cqlsh:ucp> select * from profile_id = 0xdeadbeef;                                                                                                                
cqlsh:ucp> select * from profile where profile_id = 0xdeadbeef;

 project_key | profile_id | age
-------------+------------+-----
      0xcafe | 0xdeadbeef | 200

(1 rows)
```

I didn't test further, but I'm left wondering if it is simply an oversight that blob are not supported with SAI today ? and if it is _not_ an oversight, I'd love to know the reasons.
Bug: Accord metrics are isolated which cause existing coordination metrics to be empty, should also populate there as well
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Oct/24 18:37
Resolved: 23/Oct/24 16:42

When using Accord the latency/timeout metrics are isolated to accord, which breaks existing monitoring as it looks like the cluster is idle due to no traffic… we should also populate the coordination metrics as well like how paxos does
Bug: ShortPaxosSimulationTest failure on seed 0x23a7663eab53fc38
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Oct/24 16:14
Resolved: 22/Oct/24 08:56

This appears to reproduce locally w/ {{--seed 0x23a7663eab53fc38}}:

{noformat}
[junit-timeout] Testcase: simulationTest(org.apache.cassandra.simulator.test.ShortPaxosSimulationTest)-_jdk11:	Caused an ERROR
[junit-timeout] Failed on seed 0x23a7663eab53fc38
[junit-timeout] org.apache.cassandra.simulator.SimulationException: Failed on seed 0x23a7663eab53fc38
[junit-timeout] Caused by: java.lang.AssertionError: (from replicasForPrimaryKey) [1, 4, 5] != [3, 4, 5] (predicted)
[junit-timeout] 	at org.apache.cassandra.simulator.cluster.ClusterActions.validateReplicasForKeys(ClusterActions.java:274)
[junit-timeout] 	at org.apache.cassandra.simulator.cluster.OnClusterChangeTopology.lambda$before$0(OnClusterChangeTopology.java:62)
[junit-timeout] 	at org.apache.cassandra.simulator.systems.NonInterceptible.execute(NonInterceptible.java:58)
[junit-timeout] 	at org.apache.cassandra.simulator.cluster.OnClusterChangeTopology.before(OnClusterChangeTopology.java:61)
[junit-timeout] 	at org.apache.cassandra.simulator.cluster.OnClusterLeave.performSimple(OnClusterLeave.java:53)
[junit-timeout] 	at org.apache.cassandra.simulator.Action.performAndRegister(Action.java:468)
[junit-timeout] 	at org.apache.cassandra.simulator.Action.perform(Action.java:486)
[junit-timeout] 	at org.apache.cassandra.simulator.ActionSchedule.next(ActionSchedule.java:379)
[junit-timeout] 	at org.apache.cassandra.simulator.paxos.PaxosSimulation$2.next(PaxosSimulation.java:217)
[junit-timeout] 	at org.apache.cassandra.simulator.paxos.PaxosSimulation.run(PaxosSimulation.java:189)
[junit-timeout] 	at org.apache.cassandra.simulator.paxos.PairOfSequencesPaxosSimulation.run(PairOfSequencesPaxosSimulation.java:351)
[junit-timeout] 	at org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:363)
[junit-timeout] 	at org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:344)
[junit-timeout] 	at org.apache.cassandra.simulator.paxos.PaxosSimulationRunner$Run.run(PaxosSimulationRunner.java:34)
[junit-timeout] 	at org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
[junit-timeout] 	at org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:101)
[junit-timeout] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit-timeout] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit-timeout] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}
Bug: During TCM upgrade, retain all properties of existing system tables
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Oct/24 08:41
Resolved: 22/Oct/24 08:32

When we upgrade to TCM and initialise schema, tables in the various system keyspaces are created or recreated if they already exist. If the parameters/options of any existing tables differ from the defaults we should preserve them.
Bug: Gossip settled to early for new joining node leading to data loss
Status: Triage Needed
Priority: Normal
Created: 18/Oct/24 03:49

Recently we found one issue with gossip settle to early leading to data loss on the joining node.

The new node joining the ring crashed a few times before it successfully join the ring. When the new node joins the ring, it will try to wait for gossip to settle, the issue we saw is that the gossip settled before it recognize the entire cluster. This leads to the new node requesting ranges to the wrong nodes and stream phase ended without getting any data because the requested range on the target nodes are not the real owner of the token ranges.

After checking the gossip settle code, I found that gossip may settle in 5 + 3 = 8 seconds if the new node local gossip statemap size is not changing. This may happen if the new node is busy with other gossip task and cannot populate all nodes to its local gossip state map.

Proposing a fix for this to add a env variable for bootstrapping node so that we will also check the minimum number of nodes needed for a node to consider gossip settle. PR will come later.
Bug: debian bash completion for nodetool/cqlsh is not working
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Oct/24 23:57
Resolved: 15/Oct/24 08:37

I tried to enable nodetool/cqlsh completion and got an error:
{code:java}
nodetool-completion:18: command not found: have {code}
Source for this line
{code:java}
    16    #
    17    #
 >> 18    have nodetool && have cqlsh &&
    19    {
    20
    21        show_keyspaces()
    22        {
    23            local ks=$(get_keyspaces)
    24            COMPREPLY=( $(compgen -W "$ks" -- "$1") )
    25        } {code}
I don't know from which debian package the `have` is, but I think it's better to use bash builtin function `type` 

Fix for 4.0: [https://github.com/apache/cassandra/pull/3613]
Bug: Add latest test config for dynamic_remote batchlog_endpoint_strategy and new auth parameterizedClass maps
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Oct/24 12:02
Resolved: 19/Oct/24 10:49

CASSANDRA-18120 and CASSANDRA-19946 introduced new cassandra_latest.yaml changes but (I) forgot to add them to test/conf/latest_diff.yaml and test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java so they actually are tested by the *latest tests.

Patch
https://github.com/apache/cassandra/compare/cassandra-5.0...thelastpickle:cassandra:mck/18120_19946__latest_tests_fix/5.0

CI
lint and all \*latest* test types
 [^CASSANDRA-20002-50_#65_ci_summary.html] &  [^CASSANDRA-20002-50_#65_results_details.tar.xz]
Bug: CRC mismatch
Status: Triage Needed
Priority: Normal
Created: 01/Oct/24 14:04

I got these errors during performance tests:
 * NoHostAvailable: ('Unable to connect to any servers', {'10.129.53.159:9042': ConnectionShutdown('{*}CRC mismatch on header 22f840. Received 85\", computed e3d12c.{*}')})
 * NoHostAvailable: ('Unable to connect to any servers', {'10.129.53.159:9042': ConnectionShutdown('{*}CRC mismatch on header 718a02. Received 85a8\", computed a7811f.{*}')})

I used this connection setting with {{ProtocolVersion.V5}} :
{code:java}
self._cluster = Cluster(contact_points = self._run_setup['ip'],
            port = self._run_setup['port'], 
            auth_provider = auth_provider,
            load_balancing_policy = load_balancing_policy, 
            control_connection_timeout = Setting.TIMEOUT, 
            idle_heartbeat_interval = Setting.TIMEOUT,
            connect_timeout = Setting.TIMEOUT, 
            protocol_version = ProtocolVersion.V5){code}
I did not get the mistake about CRC mismatch, when I used {{{}ProtocolVersion.V4{}}}. It seems on the first view, that {{ProtocolVersion.V5}} generates these issues.

BTW: I am probably able to simulate the issue repeatable in case of request (I got these issues in Cassandra version 5.0.0)
Bug: Extended TTL digest generation creates false digest mismatches during upgrades
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Oct/24 19:13
Resolved: 11/Oct/24 17:20

CASSANDRA-14227 added support for TTLs all the way up to the year 2106, and this is disabled in {{CASSANDRA_4}} storage compatibility mode. However, even with {{CASSANDRA_4}} in use, {{LivenessInfo#digest()}} writes the local expiration time as an 8-byte long, even when it could fit in a 4-byte int. During an upgrade, 5.0 nodes will generate slightly different digests and cause unnecessary read-repair traffic until the upgrade is complete and all nodes use the new digest calculation. This should be easy to fix if we just conditionally update the digest with int or long depending on the actual size of the local expiration time. (That should also avoid writing an extra 4 bytes we don't need to the digest in many cases.)
Bug: Enables IAuthenticator's to return own AuthenticateMessage
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Oct/24 09:49
Resolved: 11/Oct/24 12:37

Upon a client connection to the C*, the server sends a startup message with the full qualified class name (FQCN) of the authenticator that is used. The different drivers recognize this FQCN to enable scheme negotiation allowing different authentication methods. See for instance [https://github.com/apache/cassandra-java-driver/blob/4.x/core/src/main/java/com/datastax/dse/driver/api/core/auth/BaseDseAuthenticator.java#L76]

Using a custom authenticator becomes restricted to either follow the default C* implementation (which doesn't allow scheme negotiation) or use the exact same FQCN that is already known in the drivers. The later might be impractical due to the restriction of having multiple implementations with the same FQCN.

This change enables IAuthenticator implementation to control what's sent back to clients by creating their own AuthenticateMessage.
Bug: Commitlog  online replay tool 
Status: Open
Priority: Normal
Created: 11/Oct/24 06:02

We can add a tool that can replay commitlogs online. 
That may be useful when we restore from some archiving logs without restart the nodes.
I think the tools can have the options like :

1、log name with absolute path if we replay log using local files;
2、only absolute path , so that all file under the dir will be replayed in the order of file list;
3、filter some  keys that do not belong  to the nodes
4、ratelimit
5、log name and object storage url and bucket , together with information related to authentication，but I think  this can be implement in the future. 
6、keyspace and table filter, will replay all log data if not specified.
Bug: Change cassandra-gocql-driver module name
Status: Triage Needed
Priority: Normal
Created: 10/Oct/24 13:37

Change module name to github.com/apache/cassandra-gocql-driver/v2 (assuming that this goes into a new major - 2.0.0 - release).

This is a breaking change as it requires users to change the imports on their Go applications or to add a rewrite to go.mod.

Current major (v1) will keep the existing module name (github.com/gocql/gocql). Since the project only supports an active branch (trunk), we shouldn't create a v1 branch.