av9ash/GitBugs huggingface dataset:
Bug: CircleCI and Jenkins - add missing Java and Python upgrade tests
Status: Open
Priority: Normal
Created: 16/Jun/23 17:57

CASSANDRA-17869 changed the way we select upgrade tests to run.
Tests are run with the lowest overlapping JDK between Cassandra versions.
In preparation for dropping JDK 8, we added JDK 11 upgrade tests for trunk in CASSANDRA-18285 and removed the JDK8 jobs because we will drop JDK8 in CASSANDRA-18255.
Lower priority task we still need to implement as part of this ticket:
Both Jenkins and CircleCI - Add JDK11 jobs for Python upgrade tests with 4.0 and 4.1 (Java upgrade tests do not have any forward upgrade tests added now (4.0 or 4.1 to trunk)).
Bug: Dropped table causes node to get stuck in JOINING
Status: Triage Needed
Priority: Normal
Created: 14/Jun/23 01:12

To facilitate reproducibility of this issue I have created a patch.

[^cass_stream.patch]

It can also be reproduced without patch by dropping table while node is doing stream for join.
{code:java}
cd ~/dev/cassandra
git checkout 3.11.14
git apply ~/downloads/cass_stream.patch
ant clean
ant
ccm create -n 3 test
ccm node1 start
ccm node2 start
ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': '3'};"
# receive warning since nodes < RF
ccm node1 cqlsh -e "CREATE TABLE test.streamtest(stream_id int PRIMARY KEY, payload TEXT);"
ccm node1 cqlsh -e "INSERT INTO test.streamtest(stream_id, payload) VALUES(1, 'hello world')"
ccm flush
sed -i 's/auto_bootstrap: false/auto_bootstrap: true/'  ~/.ccm/test/node3/node.conf
sed -i 's/auto_bootstrap: false/auto_bootstrap: true/'  ~/.ccm/test/node3/conf/cassandra.yaml
sed -i 's/seeds: 127.0.0.1,127.0.0.2,127.0.0.3/seeds: 127.0.0.1,127.0.0.2/' ~/.ccm/test/node3/conf/cassandra.yaml
touch ~/.ccm/test/node3/logs/system.log
ccm node3 start
tail -f ~/.ccm/test/node3/logs/system.log{code}
The logs will display something like:
{noformat}
ERROR [STREAM-IN-/127.0.0.1:7000] 2023-06-14 11:08:37,399 StreamSession.java:609 - [Stream #fdd71710-0a4f-11ee-a768-c3d20ff8c2d1] Streaming error occurred on session with peer 127.0.0.1
java.io.IOException: streamtest 38f79d70-0a4f-11ee-9849-a999e71c8739 CF was dropped during compressed streaming

ERROR [main] 2023-06-14 11:08:37,405 StorageService.java:1630 - Error while waiting on bootstrap to complete. Bootstrap will have to be restarted.{noformat}
The problem is in this use case the customer is rotating out tables every few hours and so resuming the bootstrap keeps failing. In my opinion this is a bug and Cassandra should be robust enough to continue streaming if a table is dropped.
Bug:  UnknownColumnFamilyException reading from socket Error Post DB Version Upgrade from 3.11.10 to 4.1.0
Status: Triage Needed
Priority: Normal
Created: 15/Jun/23 00:14

There is a three Node Apache Cassandra Cluster with version: 3.11.10 with around 160GB data on each node and Keyspace RF is 3

On Day1, On first Node, DB Version was Upgraded from 3.11.10 to 4.1.0 and upgradesstables was run post start of db service from apache cassandra 4.1.0 binaries.

On Day2, On Second Node, Post start of binaries from 4.1.0, On third node which was with DB Version 3.11.10 below error was continuously observed.

When cfId d46780c2-2f1c-3db9-b4c1-b8d9fbc0cc23 was checked it was of Table system_auth.network_permissions

 

/*

WARN  [MessagingService-Incoming-/10.x.135.y] 2023-06-10 03:11:37,632 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId d46780c2-2f1c-3db9-b4c1-b8d9fbc0cc23. If a table was just created, this is likely due to the schema not being fully propagated.  Please wait for schema agreement on table creation.
WARN  [MessagingService-Incoming-/10.x.135.y] 2023-06-10 03:15:47,633 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId d46780c2-2f1c-3db9-b4c1-b8d9fbc0cc23. If a table was just created, this is likely due to the schema not being fully propagated.  Please wait for schema agreement on table creation.
WARN  [MessagingService-Incoming-/10.x.135.y] 2023-06-10 03:19:57,638 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId d46780c2-2f1c-3db9-b4c1-b8d9fbc0cc23. If a table was just created, this is likely due to the schema not being fully propagated.  Please wait for schema agreement on table creation.
WARN  [MessagingService-Incoming-/10.x.135.y] 2023-06-10 03:19:57,642 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId d46780c2-2f1c-3db9-b4c1-b8d9fbc0cc23. If a table was just created, this is likely due to the schema not being fully propagated.  Please wait for schema agreement on table creation.
WARN  [MessagingService-Incoming-/10.x.135.y] 2023-06-10 03:28:17,643 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId d46780c2-2f1c-3db9-b4c1-b8d9fbc0cc23. If a table was just created, this is likely due to the schema not being fully propagated.  Please wait for schema agreement on table creation.
WARN  [MessagingService-Incoming-/10.x.135.y] 2023-06-10 03:28:17,649 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing

*/
Bug: CQLSH formatting output is incorrect when querying the system properties virtual table
Status: Triage Needed
Priority: Normal
Created: 12/Jun/23 17:02

cqlsh produces incorrect output for the following query:
{code:java}
select * from system_views.system_properties;{code}
See the screenshot:
!image-2023-06-12-19-01-34-086.png|width=318,height=314!

The reason for this is that the {{java.class.path}} system property has a very long string representation value, so it breaks the formatting of the query result. Changing the output format from horizontal query output to vertical [1] doesn't help either.

The formatting must be fixed.



[1] [https://docs.datastax.com/en/dse/5.1/cql/cql/cql_reference/cqlsh_commands/cqlshExpand.html]
Bug: CEP 15: (Accord) push waiting for epoch into CommandStore rather than require every command to think about the edge case when the epoch isn't ready
Status: Open
Priority: Normal
Created: 09/May/23 19:54

We have a common pattern of scheduling work on a CommandStore, but this work has to double check the epoch exists, if not it has to wait on the fetch and reschedule again… this pattern could likely be pushed to CommandStore if the PreLoadContext includes the epochs required (just like we do for commands).

For things outside of CommandStores, they would have to maintain their current logic, the scope of this work is just tasks that run on CommandStores.
Bug: [Analytics] Circle CI Configuration for Cassandra Analytics
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Jun/23 16:04
Resolved: 09/Jun/23 21:55

We need to add the CircleCI configuration to build and test the Cassandra Analytics library. We need to test against all 4 profiles:

* cassandra-analytics-core-spark2-2.11-jdk8
* cassandra-analytics-core-spark2-2.12-jdk8
* cassandra-analytics-core-spark3-2.12-jdk11
* cassandra-analytics-core-spark3-2.13-jdk11
Bug: Align all system property names used by the project to use the 'cassandra' prefix
Status: Triage Needed
Priority: Normal
Created: 12/Jun/23 18:38

Some of the system properties related to the project don't have the correct prefix in their names, which in turn makes it difficult to identify the relationship of a property to the project itself and also adds confusion. 

The properties must be properly prefixed. For example, 
{{legacy-sstable-root}} -> {{cassandra.test.legacy_sstable_root}}
{{default.provide.overlapping.tombstones}} -> {{cassandra.default.provide.overlapping.tombstones}}

The {{@Replaces}} annotation could be used to achieve this behaviour. It is also worth mentioning the redesign of the system properties virtual table. Since all properties will have the correct prefix, we can add a new column to the virtual table, the {{property_type}}, which shows a property relationship, e.g. {{production, test, runtime_production}}, etc.
Bug: Fix shutting down IsolatedJmx 
Status: Resolved
Priority: Normal
Resolution: Abandoned
Created: 11/Jun/23 20:43
Resolved: 12/Jun/23 15:39

{code:java}
public class MyTest extends TestBaseImpl
{
    @Test
    public void test() throws Throwable
    {
        try (Cluster cluster = Cluster.build(2).withConfig(c -> c.with(Feature.values())).start())
        {
            ClusterUtils.stopUnchecked(cluster.get(2));
            System.out.println(cluster.get(1).nodetoolResult("status").getStderr());
        }
    }
}  {code}
This will fail on this:
{code:java}
java.net.BindException: Address already in use (Bind failed)
	at java.net.PlainSocketImpl.socketBind(Native Method)
	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
	at java.net.ServerSocket.bind(ServerSocket.java:390)
	at java.net.ServerSocket.<init>(ServerSocket.java:252)
	at javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
	at org.apache.cassandra.distributed.impl.CollectingRMIServerSocketFac
{code}

The problem is that IsolatedJMX clears whole map of TCPTransports in clearMapField as part of the stopJmx method. This results in JMX internals thinking the socket is not there so it tries to create it but it fails to do so because it is technically still bound.

The fix is consisting of cleaning just the bits belonging to that specific IsolatedJMX instance.
The investigation of the issue done by myself, the actual fix by [~drohrer]. Found while working on CASSANDRA-18572.
Bug: Mixed single-column and multi-column clustering restrictions are not merged correctly
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 08/Jun/23 10:15
Resolved: 08/Jun/23 10:28

During working on CEP-29, I found that single-column and multi-column clustering restrictions are not merged correctly and cause query failures or incorrect results.

The algorithm used in {{RestrictionSet.mergeRestrictions}} is incorrect, as its results depend on the (undefined) order of restrictions in a hashset.
In more complex scenarios, when there are more restrictions to merge, the algorithm results can even differ between the test runs.
Bug: [Analytics] Abstract mTLS provisioning via a SecretsProvider
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 23/May/23 17:42
Resolved: 06/Jun/23 22:12

This enhancement allows us to abstract the mTLS secrets provisioning through a {{SecretsProvider}} interface. This will allow custom implementations of the {{SecretsProvider}} to be able to hook into the secrets provisioning. We need to provide a default implementation {{SslConfigSecretsProvider}} which provides secrets via the {{SslConfig}} which is parsed from the reader options.
Bug: [Jenkins] Run Upgrade tests framework against all maintained JDKs
Status: Open
Priority: Normal
Created: 05/Jun/23 19:11

After CASSANDRA-17869, we have to run the upgrade test framework against both JDKs to ensure all tests (backward and forwards) are run. But no test should be run twice – each is either one JDK or the other, not both. Running a test with only one of the JDKs is enough for heavy Python upgrade tests.

This ticket should cover the addition in Jenkins.

The addition for CircleCI will be covered in CASSANDRA-18285.
Bug: Cannot read or repair data after dropping a column when one of the nodes is down
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 17/May/23 08:54
Resolved: 05/Jun/23 14:51

When a column is dropped while one of the nodes is down, when the node come back online, repair or querying the data fails.

This can be reproduced by running the following in-jvm dtest:


{code:java}
    @Test
    public void testDroppingColumnWhenOneNodeIsDown() throws IOException, ExecutionException, InterruptedException
    {
        try (Cluster cluster = init(Cluster.build(3).start()))
        {
            cluster.schemaChange("create table " + KEYSPACE + ".tab (id int primary key, v1 int, v2 int)");
            cluster.disableAutoCompaction(KEYSPACE);

            cluster.coordinator(1).execute("insert into " + KEYSPACE + ".tab (id, v1, v2) VALUES (?, ?, ?)", ConsistencyLevel.ALL, 1, 2, 3);
            cluster.coordinator(1).execute("insert into " + KEYSPACE + ".tab (id, v1) VALUES (?, ?)", ConsistencyLevel.ALL, 4, 5);
            cluster.coordinator(1).execute("insert into " + KEYSPACE + ".tab (id, v2) VALUES (?, ?)", ConsistencyLevel.ALL, 6, 7);

            cluster.get(3).shutdown(true).get();

            cluster.schemaChangeIgnoringStoppedInstances("alter table " + KEYSPACE + ".tab drop v1");

            cluster.get(3).startup();

            cluster.coordinator(1).execute("insert into " + KEYSPACE + ".tab (id, v2) VALUES (?, ?)", ConsistencyLevel.ALL, 1, 9);
            cluster.coordinator(1).execute("insert into " + KEYSPACE + ".tab (id, v2) VALUES (?, ?)", ConsistencyLevel.ALL, 2, 11);

            NodeToolResult repairResult = cluster.get(1).nodetoolResult("repair", KEYSPACE, "tab");
            assertThat(repairResult.getRc()).isZero();

            SimpleQueryResult rows = cluster.coordinator(1).executeWithResult("select * from " + KEYSPACE + ".tab", ConsistencyLevel.ALL);
            assertRows(rows.toObjectArrays(), row(1, 9), row(2, 11), row(4, null), row(6, 7));

            assertRows(cluster.coordinator(1).executeWithResult("select * from " + KEYSPACE + ".tab where id = ?", ConsistencyLevel.ALL, 1).toObjectArrays(), row(1, 9));
            assertRows(cluster.coordinator(1).executeWithResult("select * from " + KEYSPACE + ".tab where id = ?", ConsistencyLevel.ALL, 2).toObjectArrays(), row(2, 11));
            assertRows(cluster.coordinator(1).executeWithResult("select * from " + KEYSPACE + ".tab where id = ?", ConsistencyLevel.ALL, 4).toObjectArrays(), row(4, null));
            assertRows(cluster.coordinator(1).executeWithResult("select * from " + KEYSPACE + ".tab where id = ?", ConsistencyLevel.ALL, 6).toObjectArrays(), row(6, 7));
        }
    }
{code}
Bug: Anticompaction triggered during repair when it's not needed
Status: Triage Needed
Priority: Normal
Created: 15/May/23 06:42

I've been testing repair and different options and comparing Cassandra 4 to Cassandra 3.11.

What i've discovered is some strange behavior in Cassandra 4, mainly when it comes to anticompaction. Anticompaction seems to be triggered when "–full" is not provided.

I.e this will trigger anticompaction in Cassandra 4, but not in Cassandra 3.11:

nodetool repair --in-dc datacenter1

nodetool repair -st 5 -et 10

The easiest way to observe this is to:
 * Disable autocompaction for a table
 * Run repair for a table
 * Observe the diskspace, every time a repair is triggered the diskspace increases by like 30% (not the case when used with --full)

On top of that, even though anticompaction is triggered, the timestamp is not set on SSTables. It remains as 0. I don't really see any point in doing anticompaction, either we do anticompaction and set the timestamp or we don't do both.
Bug: Improve nodetool enable{audit,fullquery}log, CVE-2023-30601
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 25/May/23 06:38
Resolved: 25/May/23 14:45

The {{--archive-command}} parameter to {code}nodetool enable{audit,fullquery}log{code} allows an attacker to execute arbitrary commands as the user running cassandra.

Patch adds a configuration option which disallows using this parameter - for any existing users of --archive-command this can be re-enabled
Bug: Add JMX utility class to in-jvm dtest to ease development of new tests using JMX
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 19/May/23 00:44
Resolved: 19/May/23 06:06

While reviewing CASSANDRA-18511, some repetitive code was identified across the 4 branches, and 2 different tests, that would also be repeated for any new usages of the JMX support in the in-jvm dtest framework. Therefore, a utility class should be added to the dtest-api's `shared` package that will simplify some of this repetitive and error-prone code.
Bug: Fix flaky dtest: transient_replication_test.TestTransientReplicationRepairStreamEntireSSTable
Status: Open
Priority: Normal
Created: 19/May/23 19:33

As seen on https://app.circleci.com/pipelines/github/driftx/cassandra/1019/workflows/2c4502d8-52d2-4dc8-a871-ecad2a61645a/jobs/26149/tests :

{noformat}
failed on setup with "TypeError: 'list' object is not callable"
self = <transient_replication_test.TestTransientReplicationRepairStreamEntireSSTable object at 0x7fecd5e4bda0>
fixture_dtest_setup = <dtest_setup.DTestSetup object at 0x7fecd60f8dd8>

    @pytest.fixture(scope='function', autouse=True)
    def setup_cluster(self, fixture_dtest_setup):
>       self.tokens = self.tokens()
E       TypeError: 'list' object is not callable

transient_replication_test.py:206: TypeError
{noformat}

Looking at the code, I'm not sure how this has been passing.
Bug: Disable chronicle analytics
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 19/May/23 16:08
Resolved: 19/May/23 16:59

 At least chronicle-queue phones home to google analytics:  https://github.com/OpenHFT/Chronicle-Queue/blob/chronicle-queue-5.20.116/DISCLAIMER.adoc

but we can disable all chronicle analytics with a single flag: -Dchronicle.analytics.disable=true
Bug: Investigate preloading ccm repositories in the docker image
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Apr/23 11:06
Resolved: 19/May/23 02:23

In CASSANDRA-18391 it was discovered that to skip some upgrade tests, the ccm repository first needed to be populated with older versions.  While that case was solved, it may still be beneficial to preload the ccm repositories in the docker image so they don't need to be fetched at all.
Bug: BLOG - EOL Announcement for 3.x and 3.0.x
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/May/23 02:56
Resolved: 16/May/23 23:59

This ticket is to capture the work associated with publishing the May 2023 blog "Apache Cassandra 3.0.x and 3.11.x - End of Life Announcement"
Bug: Document that DecimalType violates property serialize(deserialize(bb)) != bb when used as a clustering column
Status: Open
Priority: Normal
Created: 15/May/23 23:15

Found by CASSANDRA-18504

When converting the BB to comparable bytes, converting back returns a different set of bytes for OSS42

{code}
java.lang.AssertionError: Property falsified after 18 example(s) 
Smallest found falsifying value(s) :-
Type:
decimal
Values: [-9.05674E-92]
Cause was :-
org.junit.ComparisonFailure: [fromComparableBytes(asComparableBytes(bb)) != bb; version OSS42] expected:<"0000006[1f22e36]"> but was:<"0000006[2ff75ce1c]">
{code}

EDIT:

After more testing and talking with [~blambov], this is expected behavior and been this way since at least 3.0 (didn't test further back). The new compare as bytes methods are only matching existing semantics, and the issue itself is that when we try to compare this type, we "normalize" it, which mutates the value; this normalization is not well documented and should be more explicit to users
Bug: StorageServiceMBean throws inconsistent exceptions
Status: Triage Needed
Priority: Normal
Created: 15/May/23 02:26

StorageServiceMBean has the following two methods:

 
{code:java}
public List<String> getTokens()
    {
        return ssProxy.getTokens();
    }

    public List<String> getTokens(String endpoint)
    {
        try
        {
            return ssProxy.getTokens(endpoint);
        }
        catch (UnknownHostException e)
        {
            throw new RuntimeException(e);
        }
    }

 {code}
The getTokens() method throws UnknownHostException, but the getTokens(String endpoint) throws RuntimeException. This is strange when the error is identical and the two methods have the same name.
Bug: "Connection refused" problem
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 10/May/23 10:39
Resolved: 15/May/23 14:21

Hello! I am trying to deploy The Hive 4 on a VMware Workstation 17 player VM to test Splunk integration with The Hive. 
I am following the guide at this [link|https://docs.thehive-project.org/thehive/installation-and-configuration/installation/step-by-step-guide/], but I encountered an error at one of the stages, namely when launching cassandra using the cqlsh localhost 9042 command:

Connection error: ('Unable to connect to any servers', \{'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})

I tried to solve the problem based on the information from [this|https://stackoverflow.com/questions/29121904/cassandra-cqlsh-connection-refused] site, but it didn't help me.

OS Version - Ubuntu 22.04.2 LTS

I'm new in this field, I can provide any information you need
Bug: yaml should explain behavior from CASSANDRA-13325
Status: Open
Priority: Normal
Created: 06/Apr/23 21:37

After CASSANDRA-13325, it is possible in the yaml to set the 'protocol' option in a given encryption_options to the csv list of acceptable protocols, as we do in [this test|https://github.com/apache/cassandra-dtest/blob/trunk/cqlsh_tests/test_cqlsh.py#L185] to limit it to TLSv1.2.  However, there is no way to know this from the yaml today, so some comments/example would be helpful.
Bug: CEP-15: (Accord) Join cluster without full transaction log
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/May/23 08:43
Resolved: 12/May/23 09:03

Joining replicas should not require the full transaction history to successfully start serving queries. This ticket introduces mechanisms for a replica to join (or catch up) with a data snapshot and all transactions that execute after that snapshot. This is a precursor for transaction state GC.
Bug: CEP-15: (Accord) Separate durable and transient listeners
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/May/23 08:46
Resolved: 12/May/23 09:03

Transient listeners should be handled differently and, ironically, should be more "persistent" in that they should not disappear when we evict state from cache. This patch separates listeners into `DurableAndIdempotent` and `Transient` with the latter being saved in a shared global register that also more easily permits us to ensure we do not invoke listeners redundantly (and for listeners themselves to know if we have done so). This is also a stepping stone to ensuring listeners survive cache eviction.