av9ash/GitBugs huggingface dataset:
Bug: Revert some accesses to internals after CASSANDRA-18329 lands
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Apr/23 15:50
Resolved: 30/Aug/23 14:59

The current version of JAMM in Cassandra does not handle properly the module system.

To workaround some of the failures we see when starting Cassandra and running tests, a numerous _-  -add-opens-_ and  _add-exports_ were added temporarily until Jamm is fixed.  

Those need to be revised and potentially reverted.

[https://github.com/apache/cassandra/blob/trunk/conf/jvm17-server.options#L94-L107]

(potentially also in the jvm11-server.options file too)

[https://github.com/apache/cassandra/blob/trunk/conf/jvm17-clients.options#L55-L67]

(again, potentially also in jvm11-client.options file too)

CASSANDRA-18329 should cover jamm upgrade which will happen when new jamm version is released.
Bug: compaction_tombstone_warning_threshold and compaction_large_partition_warning_threshold cause deprecation warnings
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Jun/23 16:09
Resolved: 28/Jun/23 20:58

If you start trunk without any changes you will see at startup:

{noformat}
WARN  [main] 2023-06-26 15:59:36,613 YamlConfigurationLoader.java:426 - [compaction_tombstone_warning_threshold, compaction_large_partition_warning_threshold] parameters have been deprecated. They have new names and/or value format; For more information, please refer to NEWS.txt
{noformat}

We should give these the same treatment as CASSANDRA-18617, removing them from the default yaml to stop the warnings, but still accepting them with converters and @Replaces tags.
Bug: Improvements to SSTable format configuration
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Apr/23 11:20
Resolved: 18/May/23 10:58

CEP-17 and CASSANDRA-17056 abstracted some interfaces for SSTable format implementations and defined a method of plugging in specific configurations. 

This method is brittle and asks users to specify format identifiers whose configuration does not provide value but can be the source of conflicts and problems. On the other hand it makes important choices non-obvious, as the selection of format to write is given by the order of configured interfaces.

An improved specification mechanism needs to be put in place before Cassandra 5 is released.
Bug: Fix org.apache.cassandra.transport.DriverBurnTest.measureLargeV4WithCompression-.jdk17 
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 06/Jun/23 22:50
Resolved: 05/Jul/23 23:53

h1.  
{code:java}
Regression
org.apache.cassandra.transport.DriverBurnTest.measureLargeV4WithCompression-.jdk17 (from org.apache.cassandra.transport.DriverBurnTest-.jdk17)

Failing for the past 1 build (Since #1590 ) Took 30 sec.      Failed 5 times in the last 30 runs. Flakiness: 24%, Stability: 83% Stacktrace
junit.framework.AssertionFailedError at org.apache.cassandra.transport.DriverBurnTest.perfTest(DriverBurnTest.java:425) at org.apache.cassandra.transport.DriverBurnTest.measureLargeV4WithCompression(DriverBurnTest.java:316) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 
{code}
The test is flaky since recently, failing every other time in Jenkins (burn tests are not running in CircleCI) First seen with run #1572 this commit - 

CASSANDRA-18025

CC [~stefan.miklosovic] and [~brandon.williams]
Bug: Disable the deprecated keyspace/table thresholds and convert them to Guardrails
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Jun/23 17:36
Resolved: 14/Jul/23 11:29

The non-guardrail thresholds 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' configuration settings were first added with CASSANDRA-16309 in 4.0-beta4 and have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when they were replaced/migrated to guardrails as part of CEP-3 (Guardrails).

These thresholds should now be removed from cassandra.yaml, while still allowed in existing yaml files.

The old thresholds will be disabled by removing their default values from Config.java, and any existing values for these thresholds will be converted to the new guardrails using the '@Replaces' tag on the corresponding guardrail values.

Since the old thresholds considered the number of system keyspace/tables in their values, the '@Replaces' conversion will subtract the current number of system tables from the old value and log a descriptive message.

See dev list discussion: https://lists.apache.org/thread/0zjg08hrd6xv7lhvo96frz456b2rvr8b
Bug: jackson-databind-2.13.2.2.jar vulnerability: CVE-2023-35116
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Jun/23 11:31
Resolved: 28/Jun/23 11:08

https://nvd.nist.gov/vuln/detail/CVE-2023-35116

{noformat}
 An issue was discovered jackson-databind thru 2.15.2 allows attackers to cause a denial of service or other unspecified impacts via crafted object that uses cyclic dependencies. NOTE: the vendor's perspective is that the product is not intended for use with untrusted input.
{noformat}
Bug: Limit the read size on the row level
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/May/23 06:51
Resolved: 12/Jun/23 09:37

When deserializing a row, we have a limit on amount of data we can read per cell. If an sstable is corrupted in a certain way, it may read many cells with extreme length leading to oom errors. When we read a row, it has a row size in the beginning. The idea is to limit the read size for a cell to the minimum of the max cell length and the remaining row size.
Bug: Upgrade guava on trunk
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Jun/23 22:12
Resolved: 07/Jul/23 14:14

Recently guava added JDK17 in CI and fixed some bugs down the road.

Upgrading before the major 5.0 release is something we should do. 

Also, the current version that Cassandra uses is from 2018.
Bug: Add support for JMX in the in-jvm dtest framework
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/May/23 20:33
Resolved: 31/May/23 19:24

In many cases, it would be useful to be able to enable JMX endpoints within the in-jvm dtest framework, including the existing JMX Getter test, which used to simply spin up a JMX registry and then leave it running.  There are quite a few JMX-related functions that don’t have tests today, and some external usages of the in-jvm dtest framework could also benefit from exposing JMX like we did Native before.
Bug: Assertion error when describing mv as table
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Jun/23 06:08
Resolved: 19/Jun/23 15:40

When describing materialized view as a table Cassandra gets an assertion error.

Steps to reproduce:
CREATE KEYSPACE test WITH replication = \{'class': 'NetworkTopologyStrategy', 'datacenter1': '3'} AND durable_writes = true;
CREATE TABLE test.table1 (key1 text,key2 int,value int,PRIMARY KEY (key1, key2));
CREATE MATERIALIZED VIEW test.table1_by_value AS SELECT key1, key2, value FROM test.table1 WHERE value IS NOT NULL AND key1 IS NOT NULL AND key2 IS NOT NULL PRIMARY KEY(value, key1, key2);
DESCRIBE MATERIALIZED VIEW test.table1;
DESCRIBE TABLE test.table1_by_value;
DESCRIBE TABLE test.non_existing;
 
From the above the "DESCRIBE TABLE test.table1_by_value;" throws an assertion error while "DESCRIBE TABLE test.non_existing;" returns a meaningful error msg.
Bug: Use WithProperties to ensure that system properties are handled
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Apr/23 12:59
Resolved: 26/May/23 16:32

The {{WithProperties}} is used to handle system properties to set and reset values during the test run, instead of try-catch it uses the try-with-resource approach which facilitates test development.

We need to replace all the try-catch clauses that work with system properties with {{WithProperties}} and try-with-resource for all the similar cases and where it is technically possible.

Example:
{code:java}
        try
        {
            COMMITLOG_IGNORE_REPLAY_ERRORS.setBoolean(true);
            testRecoveryWithGarbageLog();
        }
        finally
        {
            COMMITLOG_IGNORE_REPLAY_ERRORS.clearValue();
        }
{code}
Can be replaced with:
{code:java}
        try (WithProperties = new WithProperties().with(COMMITLOG_IGNORE_REPLAY_ERRORS, "true"))
        {
            testRecoveryWithGarbageLog();
        }
{code}
Bug: Integer(int), Long(long), Float(double) were deprecated in JDK9
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 13/Apr/23 21:59
Resolved: 02/May/23 13:27

Now when we are moving with Cassandra 5.0 to 11+17, it is good to declutter at least a bit our build log which  contains 76 deprecation warnings.

Half of them are for deprecation of Integer(int), Long(long) and Float(double).

Oracle advises to use factory methods which are likely to yield significantly better space and time performance too. The factory methods are also marked with 
@IntrinsicCandidate
Bug: jackson-core vulnerability: CVE-2022-45688
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Jun/23 11:53
Resolved: 06/Jul/23 14:54

This is failing owasp.

https://nvd.nist.gov/vuln/detail/CVE-2022-45688

{quote}
A stack overflow in the XML.toJSONObject component of hutool-json v5.8.10 allows attackers to cause a Denial of Service (DoS) via crafted JSON or XML data.
{quote}
Bug: Update tasks configuration to run checks locally when requested
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Jun/23 05:26
Resolved: 21/Jul/23 04:01

Currently CheckStyle and RAT are run with almost every single Ant target, which is annoying as when developing locally. The targets should be clear - "test" - runs the tests, "jar" - builds the project and creates jars, and then we should have a task "check" which runs all the static analysis, that is CheckStyle, RAT and Eclipse-Warnings (or whatever we decide to replace Eclipse-Warnings with).

Such goal should be include in "artifacts" and we should run it instead of "eclipse-warnings" on CircleCI. This way building, static analysis and testings are clearly separated.
Bug: Unit tests in org.apache.cassandra.cql3.EmptyValuesTest class occasionally failing with JDK17
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Apr/23 18:10
Resolved: 12/May/23 15:10

 

All of them failed with the below stack trace for the same assertion failing:
{code:java}
junit.framework.AssertionFailedError: at org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:90) at org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112) at org.apache.cassandra.cql3.EmptyValuesTest.testEmptyDecimal(EmptyValuesTest.java:192) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
 

Unfortunately I do not have a link to the CI run as this was seen last in private infra and not in CircleCI. Maybe we want to check with the multiplexer for flakiness.
Bug: Partial compaction can resurrect deleted data
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/May/23 10:19
Resolved: 18/May/23 17:43

If there isn't enough disk space available to compact all existing sstables, Cassandra will attempt to perform a partial compaction by removing sstables from the set of candidate sstables to be compacted, starting with the largest one. It is possible that the sstable removed from the set of sstables to compact contains data for which there are tombstones in another (more recent) sstable. Since the overlaps between sstables is computed when the {{CompactionController}} is created, and the {{CompactionController}} is created before the removal of any sstables from the set of sstables to be compacted this computed overlap will be outdated when checking which sstables are covered by certain tombstones. This leads to the faulty conclusion that the tombstones can be pruned during the compaction, causing the data to be resurrected.

The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the {{CompactionController}} after the set of sstables to compact has been reduced, and is thus not affected. {{trunk}} does not appear to support partial compactions at all, but instead refuses to compact when the disk is full.

This regression appears to have been introduced by CASSANDRA-13068.
Bug: Cassandra doesn't start on JDK17
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 05/Apr/23 19:44
Resolved: 06/Apr/23 19:06

CASSANDRA-17199 added a new call to jamm measureDeep which hits some JDK internals and prevents us from starting Cassandra with JDK17. This will be solved with CASSANDRA-18329.

Until new version of jamm lands we can workaround the problem either by adding add-opens or by replacing temporarily

{code:java}
public static final long IPV6_SOCKET_ADDRESS_SIZE = ObjectSizes.measureDeep(new InetSocketAddress(getIpvAddress(16), 42));
{code}

with

{code:java}
public static final long IPV6_SOCKET_ADDRESS_SIZE = 168;
{code}

measured with JOL.

Then we can switch back to the current call [here|https://github.com/apache/cassandra/commit/4444721b6de555352bf0ac3ef7e36f94dc832f41#diff-1122d7d3efe9721af7244d373e66378f7e90cb05fd65859a52e8a3ea58a7c8f9R45] later.
Bug: Paxos only repair is treated as an incremental repair
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 19/Apr/23 15:04
Resolved: 08/Sep/23 14:10

Paxos only repair tries to continue or is treated as an incremental repair. This happened on 4.1.0 and 4.1.1 when trying to run repair in preparation for enabling paxos_state_purging. The repair was in preparation mode triggered multiple anti-compactions on the nodes. Running the command with --full behaves in the expected way, ie. only the paxos data is repaired and it's finished within a few seconds.
{code:java}
nodetool repair --paxos-only // This does not behave as expected, does it complete quickly and seems to be waiting on anticompactions
{code}
{code:java}
nodetool repair --full --paxos-only // Completes within a few seconds as expected
{code}
Bug: Update Cassandra logos on ASF project logos site
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Jun/23 20:41
Resolved: 02/Sep/23 09:14

It appears that the Cassandra logos on the ASF project logos website are outdated, as they include lowercase letters and no trademark: https://www.apache.org/logos/#cassandra

The logo currently being used on the website is this one: https://github.com/apache/cassandra-website/blob/trunk/site-content/source/modules/ROOT/images/logo-white.svg

To ensure proper usage by contributors and third parties, I propose the creation of the following logo set based on the current website logo:

Version 1 - update and create black and white versions
Version 2- remove
Version 3 - update and create black and white versions
Version 4 - no change

If these already exist somewhere, then please upload them to the ASF project logos page (requires a committer): https://www.apache.org/logos/about.html

Otherwise I will be asking our designer to create hi-res versions to be uploaded.
Bug: jvm dtest logs should indicate specific test case name being run
Status: Open
Priority: Normal
Created: 27/Apr/23 08:40

Currently jvm dtest logs do not indicate which specific test case is being run.

I believe two things should be added:
 * the test logs (<main>/system.log) should indicate test cases
 * cluster logs should indicate test cases (e.g. test case name should be included in cluster id)
Bug: Test pipeline does not always copy to nightlies
Status: Open
Priority: Normal
Created: 02/Jun/23 16:49

Spotted here: https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/2483/console

{noformat}
04:15:37  For test report and logs see https://nightlies.apache.org/cassandra/devbranch/Cassandra-devbranch/2483/
04:15:38  Post stage
04:15:38  [Pipeline] sshPublisher
04:15:38  SSH: Current build result is [FAILURE], not going to run.
{noformat}

Even though the pipeline says to [always|https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_pipeline.groovy#L458] upload to nightlies, it appears to not be the case.
Bug: Test failure: pdtest: dtest-upgrade.upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_clustering_order_and_functions
Status: Open
Priority: Normal
Created: 04/Apr/23 20:03

Same failure on: {{dtest-upgrade.upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_2_1_x_To_indev_3_11_x.test_timeuuid}}, also on 3.0 branch
{{Failed 3 times in the last 30 runs. Flakiness: 3%, Stability: 90%}}

{code}
Error Message
cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Unknown function 'totimestamp'"
{code}

Maybe related to merge of CASSANDRA-18328?
Bug: j11 flaky test: org.apache.cassandra.cql3.MemtableSizeTest#testTruncationReleasesLogSpace
Status: Open
Priority: Normal
Created: 13/Apr/23 14:53

Seen here: https://app.circleci.com/pipelines/github/driftx/cassandra/973/workflows/9c320e3e-f6f2-4743-9e5c-d220f99b6fd4/jobs/20901/tests

Looks like some kind of jamm problem:

{noformat}
java.lang.reflect.InaccessibleObjectException: Unable to make field private final jdk.management.jfr.StreamManager jdk.management.jfr.FlightRecorderMXBeanImpl.streamHandler accessible: module jdk.management.jfr does not "opens jdk.management.jfr" to unnamed module @1ebd319f
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
	at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
	at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
	at org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
	at org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:121)
	at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644)
	at org.apache.cassandra.Util.flakyTest(Util.java:669)
	at org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}
Bug: WEBSITE - Replace homepage, Events page banners with C* Summit
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Apr/23 11:14
Resolved: 27/Jul/23 09:10

Cassandra Forward is in the past, so this addresses the fact by simply replacing the banner and the "register now" CTA with an equivalent thing for the December C* Summit.

The (very simple) PR is available for inspection here: [https://github.com/apache/cassandra-website/pull/219]

See images:

 !banner_on_index_page.png|width=300!

 !banner_on_events_page.png|width=300!
Bug: BLOG - Town Hall Replay: Cassandra Performance Tuning Like You’ve Been Doing it For Ten Years
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Jun/23 23:56
Resolved: 27/Jul/23 08:43

This ticket is to capture the work associated with publishing the June 2023 blog "Town Hall Replay: Cassandra Performance Tuning Like You’ve Been Doing it For Ten Years"

This blog can be published as soon as possible, but if it cannot be published within a week of the noted publish date *(July 27)*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).
Bug: Test failure: org.apache.cassandra.io.sstable.indexsummary.IndexSummaryTest.testLargeIndexSummary
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 04/Apr/23 10:32
Resolved: 26/Jul/23 18:31

Failed 6 times in the last 6 runs. Flakiness: 0%, Stability: 0%

{noformat}
unit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.
	at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.util.Vector.forEach(Vector.java:1394)
	at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.util.Vector.forEach(Vector.java:1394)
{noformat}

Note that only the 'default' version of this test is failing: https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/org.apache.cassandra.io.sstable.indexsummary/IndexSummaryTest/testLargeIndexSummary
Bug: CEP-21 Avoid NPE when getting dc/rack for not yet registered endpoints
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Apr/23 09:44
Resolved: 24/Jul/23 12:18

If a snitch is asked for location info for a node not yet added to the cluster, it should not NPE. In future, it may be desirable to fine tune the actual behaviour, but for now returning a default would be an improvement.
Bug: CEP-21 Ensure that ClusterMetadata::forceEpoch keeps component epochs consistent
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Apr/23 09:44
Resolved: 24/Jul/23 12:18

{{forceEpoch}} is used to make the application of multiple transformations in series appear as a single atomic update. It's primary use is in {{UnsafeJoin}} (i.e. join without bootstrap) to apply the usual sequence of start/mid/finish join in a single transformation. In such cases, we must ensure that no component of {{ClusterMetadata}} which maintains its own last-modified epoch, ends up with an epoch greater than the one of the enclosing {{ClusterMetadata}}.
Bug: CEP-21 Rewrite o.a.c.distributed.test.SchemaTest
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Apr/23 09:43
Resolved: 24/Jul/23 12:17

Several of the premises that this test is based on are no longer valid. Rewrite it so that instead of executing schema changes node-locally to prevent them propagating, we can have the non-cms node pause before enacting a schema change to enable us to verify that the expected exceptions are thrown. Also, local schema reset and pulling during startup are not relevant in TCM, so we can simplify the tests to ensure that a down node learns of any missed updates when it restarts.
Bug: CEP-21 During startup, don't open SSTables until local metadata log replay is complete
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Apr/23 09:43
Resolved: 24/Jul/23 12:17

Eagerly opening SSTables when their {{ColumnFamilyStore}} is first instantiated presents problems when replaying the local metadata log during startup. Schema modifications which _had_ been applied by the time an SSTable was written may not have been replayed yet, causing serialization errors. To address this, we can defer opening SSTables until after the local log replay is complete.