av9ash/GitBugs huggingface dataset:
Bug: Test Failure: org.apache.cassandra.distributed.test.CASTest.testSuccessfulWriteBeforeRangeMovement
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 21/Mar/22 12:47
Resolved: 21/Mar/22 17:34

Intermittent failure on {{org.apache.cassandra.distributed.test.CASTest#testSuccessfulWriteBeforeRangeMovement}} for trunk:

https://ci-cassandra.apache.org/job/Cassandra-trunk/1025/testReport/org.apache.cassandra.distributed.test/CASTest/testSuccessfulWriteBeforeRangeMovement/

{code}
Failed 3 times in the last 6 runs. Flakiness: 40%, Stability: 50%
Error Message
CAS operation timed out: received 1 of 2 required responses after 0 contention retries
Stacktrace
org.apache.cassandra.exceptions.CasWriteTimeoutException: CAS operation timed out: received 1 of 2 required responses after 0 contention retries
	at org.apache.cassandra.service.paxos.Paxos$MaybeFailure.markAndThrowAsTimeoutOrFailure(Paxos.java:546)
	at org.apache.cassandra.service.paxos.Paxos.begin(Paxos.java:1048)
	at org.apache.cassandra.service.paxos.Paxos.cas(Paxos.java:659)
	at org.apache.cassandra.service.paxos.Paxos.cas(Paxos.java:618)
	at org.apache.cassandra.service.StorageProxy.cas(StorageProxy.java:307)
	at org.apache.cassandra.cql3.statements.ModificationStatement.executeWithCondition(ModificationStatement.java:500)
	at org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:467)
	at org.apache.cassandra.distributed.impl.Coordinator.unsafeExecuteInternal(Coordinator.java:122)
	at org.apache.cassandra.distributed.impl.Coordinator.unsafeExecuteInternal(Coordinator.java:103)
	at org.apache.cassandra.distributed.impl.Coordinator.lambda$executeWithResult$0(Coordinator.java:66)
	at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)
{code}

Failures can also be hit with CircleCI test multiplexer:

https://app.circleci.com/pipelines/github/adelapena/cassandra/1391/workflows/90fb3315-394c-4c52-815f-0a35c47d2552

CircleCI results show failures with a ~30% flakiness when running the test isolated.

The same test looks ok in 4.0, as suggested by Butler and [this repeated Circle run|https://app.circleci.com/pipelines/github/adelapena/cassandra/1392/workflows/cff25b21-0d76-45cc-b7aa-08014390376e].
Bug: Performance regression in range queries with per partition limit
Status: Open
Priority: Normal
Created: 21/Mar/22 16:46

Since upgrading from 2.2 to 3.11, we've noticed a large number of timeouts on partition range queries. We use these range queries to retrieve all data from some small tables (usually a few hundred partitions, outliers to ~3k partitions), so that we can cache this data at the application level.

After investigation we noticed a large number of extra requests from short read protection, even though the initial range query had returned all data. It turns out that when a per partition limit is used, the fix for CASSANDRA-13911 causes a short read protection request for every partition that has less rows/cells than the specified limit.

I've attached a query trace of this occurring for a very small table (12 single-row partitions):
 * A RANGE_SLICE is executed, which returns all 12 live rows (and 0 tombstones).
 * For each partition, an extra query for 100 rows (the per-partition limit) is triggered from ShortReadRowsProtection.
 * An additional RANGE_SLICE query is executed from ShortReadPartitionsProtection in an attempt to fill the total limit (1M). 

So an additional 13 queries per node are executed, even though the initial query returned all available data. For larger tables we are seeing hundreds or thousands of extra queries.

We first encountered this issue in Thrift (which supports per-partition limits on 2.2), but confirmed it occurs in CQL as well when using PER PARTITION LIMIT.

This occurs whenever a partition contains less rows than the per-partition limit:
 * Less rows than the limit means that ShortReadRowsProtection.moreContents() is called.
 * None of the conditions for skipping short read protection are true:
 ** The per partition count is not NO_LIMIT.
 ** The current partition is not empty.
 ** The clustering is not equal to Clustering.EMPTY as it's a clustered table.
 ** lastQueried is 0.

From my understanding, the issue fixed in CASSANDRA-13911 can only occur when the data returned in a partition is limited by the total limit (which may or may not coincide with the per-partition limit for that partition). If rows in previous partitions are skipped while merging because of the per-partition limit and out-of-sync nodes, the single result counter for that node may undercount the actual returned data, causing isExhausted() to give a false positive. The patch in CASSANDRA-13911 fixes the false positives (which cause incorrect query responses in some edge cases) but introduces many false negatives (which trigger unnecessary short read protection queries).
Bug: Produce and verify BoundedReadCompactionStrategy as a unified general purpose compaction algorithm
Status: Open
Priority: Normal
Created: 14/Feb/22 15:34

The existing compaction strategies have a number of drawbacks that make all three unsuitable as a general use compaction strategy, for example STCS creates giant files that are hard to back up, mess with read performance and the page cache, and led to many of the early re-open bugs. LCS improved dramatically on this but also has various issues e.g. lack of performant full compaction or due to the strict leveling with e.g. bulk loading when writes exceed the rate we can do the L0 - L1 promotion.

In this [talk|https://github.com/ngcc/ngcc2019/blob/master/NextGenerationCassandraCompactionGoingBeyondLCS.pdf] I proposed a novel compaction strategy that aims to expose a single tunable that the user can control for the read amplification. Raise the min_threshold_levels and you tradeoff read/space performance for write performance. Since then a proof of concept [patch |https://github.com/jolynch/cassandra/tree/jolynch_bounded_read_final]has been published along with some rudimentary [documentation |https://gist.github.com/jolynch/9118465f32ad5298b4e39d03ccd4370e] but this is still not tracked in Jira.

The remaining work here is

1. Validate the algorithm is correct via test suites and performance testing stress testing and benchmarking with OSS tools (e.g. cassandra-stress, [tlp-stress|https://github.com/thelastpickle/tlp-stress], or [ndbench|https://github.com/Netflix/ndbench]). When issues are found (there likely will be issues as the patch is a PoC), devise how to adjust the algorithm and implementation appropriately. Key metric of success is we can run Cassandra stably for more than 24 hours while applying sustained load, with minimal compaction load (and also compaction can keep up).

2. Do more in depth experiments measuring performance across a wide range of workloads (e.g. write heavy, read heavy, balanced, time series, register update, etc ...) and in comparison with LCS (leveled), STCS (size tiered), and TWCS (time window). Key metrics of success are establishing that as we tune max_read_per_read we should get more predictable read latency under low system load (ρ < 30%) while not degrading at high system load (ρ > 70%), and we should match LCS performance under low load while doing better at high load.

Once this is validated a Cassandra blog post reporting on the findings (positive or negative) may be advisable.
Bug: Race in CompactionExecutorTest
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 05/Jan/22 23:39
Resolved: 06/Jan/22 21:46

CompactionExecutorTest has a race between the runnable/callable under test completing
and the {{afterExecute}} method stashing it for the test.  Replace the wait/sleep loop
with a {{SimpleCondition}} that is signaled once the test task throwable has been recorded.

This seems fairly hard to hit but has happened on CI.  It took about 2600 iterations on my MacBook to trigger, but you can artificially hit frequently by adding a sleep at the start of the afterExecute method.
Bug: CVE-2019-17571 from log4j 1.2.17
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 14/Mar/22 17:12
Resolved: 14/Mar/22 17:16

"cassandra:4.0.3" image scanning reveals critical vulnerability due to dependency on log4j 1.2.17:

CVE-2019-17571 / 	CWE-502 Deserialization of Untrusted Data

Could you please share your rationale of not upgrading log4j to 2.x

Thanks.
Bug: dropping of a materialized view does not create a snapshot with dropped- prefix
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/Mar/22 15:19
Resolved: 08/Mar/22 16:46

When auto_snapshot: true and MV is dropped, the name of the snapshot does not start with "dropped-" prefix as a normal table would. This is an issue for 3.11.x only. In 4.x, the code was refactored a lot and it does not happen there.
Bug: metrics.html with UI bug
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 09/Mar/22 09:41
Resolved: 09/Mar/22 09:58

Hi,
Looks that https://cassandra.apache.org/doc/3.11/cassandra/operating/metrics.html with some UI bugs. 
 !image-2022-03-09-11-41-28-785.png!
Bug: WEBSITE - March 2022 blog "Apache Cassandra’s Google Summer of Code program"
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Feb/22 19:11
Resolved: 08/Mar/22 06:11

This ticket is to capture the work associated with publishing the March 2022 blog "Apache Cassandra’s Google Summer of Code program"

If this blog cannot be published by the *March 7, 2022 publish date*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).
Bug: DOC - STCS page has LCS in title, broken links in compaction page
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Feb/22 15:32
Resolved: 05/Mar/22 00:34

The STCS page here, [https://cassandra.apache.org/doc/latest/cassandra/operating/compaction/stcs.html], says "Leveled Compaction Strategy" in the title where it should say "Size-tiered Compaction Strategy.

*EDIT:* This needs to be fixed in the following branches:
* {{trunk}} - https://cassandra.apache.org/doc/trunk/cassandra/operating/compaction/stcs.html
* {{4.0}} - https://cassandra.apache.org/doc/4.0/cassandra/operating/compaction/stcs.html
* {{3.11}} - https://cassandra.apache.org/doc/3.11/cassandra/operating/compaction/stcs.html

While looking into this ticket, I've also discovered that the links to the compaction strategy sub-pages are broken in the {{operating/compaction/index.adoc}} and will need to be fixed in the following branches:
* {{trunk}} - https://cassandra.apache.org/doc/trunk/cassandra/operating/compaction/index.html
* {{4.0}} - https://cassandra.apache.org/doc/4.0/cassandra/operating/compaction/index.html
* {{3.11}} - https://cassandra.apache.org/doc/3.11/cassandra/operating/compaction/index.html
Bug: a new SSTable created when single SSTable tombstone compact occurred in TWCS
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 22/Feb/22 08:39
Resolved: 22/Feb/22 12:54

we found a issue that a new SSTable created when single SSTable tombstone compact occurred. The cassandra version is *cqlsh 5.0.1 | Cassandra 3.10 | CQL spec 3.4.4,* we use *TWCS.*

The old SSTable, which Estimated droppable tombstones above 0.9, is the oldest SSTable in this table, it store oldest records, and it contains same partitions with newer SSTables, there is no expired SSTable deletion block about it.

when the old SSTable exists almost TTL+gc_grace_seconds, then it's deleted, but later I found a new SSTable created, from log we know the new SSTable is created by old one, the size 42.920MiB is old SSTable and 2.381MiB is new SSTable.

 
{code:java}
DEBUG [CompactionExecutor:44581] 
2022-02-21 11:11:15,429 CompactionTask.java:255 - Compacted 
(e99c1550-9306-11ec-8461-0bfbe41d7414) 1 sstables to 
[.../mc-317850-big,]
 to level=0. 42.920MiB to 2.381MiB (~5% of original) in 31,424ms. Read 
Throughput = 1.366MiB/s, Write Throughput = 77.602KiB/s, Row Throughput =
 ~4,311/s. 194 total partitions merged to 194. Partition merge counts 
were {1:194, } {code}
 

and weird data exist in new SSTable, all the fileds only contain deletion_info, the partition/clustering/xxxxx/yyyyy is same in old SSTable.

 
{code:java}
"cells" : [
          { "name" : "xxxxx", "deletion_info" : { "local_delete_time" : "2022-02-12T10:55:15Z" }
          },
          { "name" : "yyyyy", "deletion_info" : { "local_delete_time" : "2022-02-12T10:55:15Z" }
          },
...
}{code}
also, the old SSTable only contain part of data in new SSTable, we found 129426 rows in old and 94694 rows in new one.

 

 

also I found there are TTL min:0 in sstablemetadata but I dump all data from the old SSTable, then I can't find any record with ttl=0, all data is same as deletion_info records

 
{code:java}
Minimum timestamp: 1644740070072443
Maximum timestamp: 1644742695566429
SSTable min local deletion time: 1644740070
SSTable max local deletion time: 1645433895
Compressor: org.apache.cassandra.io.compress.LZ4Compressor
Compression ratio: 0.01234938023191464
TTL min: 0
TTL max: 691200
Estimated droppable tombstones: 0.9057755011460312 {code}
 

 

I guess it's not performed as design, when a SSTable live exceed TTL+gc, it should be deleted if Estimated droppable tombstones exceed threshold, this is what I thought. So create a new SSTable behavior should be removed.
Bug: Fix flaky test - test_cqlsh_completion.TestCqlshCompletion
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Feb/22 13:25
Resolved: 19/Feb/22 09:43

 Failed 4 times in the last 24 runs. Flakiness: 30%, Stability: 83%

A bunch of the test_completion_* tests fail occasionally with an eyebleed inducing mismatched output.
Bug: BLOG - Make minor corrections "Tightening Security part 3"
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Feb/22 21:37
Resolved: 24/Feb/22 06:01

We released a blog entry as part of https://issues.apache.org/jira/browse/CASSANDRA-17373 However we realized the need for a minor correction in it. 

This ticket is for the correction.

Blog post - [Tightening security for Apache Cassandra: Part 3|http://example.com|https://cassandra.apache.org/_/blog/Tightening-Security-for-Apache-Cassandra-Part-3.html]
Bug: Automate updating tickets with CI results after merge
Status: Patch Available
Priority: Normal
Created: 21/Jan/22 15:49

From [the wiki article|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280]
{quote}ci-cassandra run bot updating ticket in JIRA w/state of test run for the SHA (JIRA Pending; to be linked)
{quote}
We already run CI for every commit (see [example|https://ci-cassandra.apache.org/job/Cassandra-trunk/]). The goal is to have automation that'll tie the results of a commit to the original JIRA ticket and update it automatically w/a comment indicating:
 * The results of the CI run (duration, pass, fail, # failures)
 * Any potential regressions in CI from the merge w/links to job history ([example|https://ci-cassandra.apache.org/job/Cassandra-trunk/912/testReport/junit/dtest.cqlsh_tests.test_cqlsh_copy/TestCqlshCopy/test_bulk_round_trip_with_timeouts/history/])

From a workflow perspective we want to optimize for having as minimal friction as possible to see the impact of one's commit on ci-cassandra and rapidly verify if their change appears to have destabilized any tests on that infrastructure.
Bug: Allow specifying PARTITION and CLUSTERING keys explicitly on DDL
Status: Triage Needed
Priority: Normal
Created: 17/Feb/22 21:04

The distinction between primary key and partition/clustering key may be potentially confusing to users so I would like to propose an addition to the to *CREATE TABLE*  DDL to specify partition and clustering keys explicitly:

CREATE TABLE events (
sensor_id UUID,
event_time TIMESTAMP,
event TEXT,
PARTITION KEY (sensor_id),
CLUSTERING KEY (event_time)
)

This would be optional and supported in addition to the current *PRIMARY KEY ((partition_key), clustering_key)* construct.

One downside is it can get potentially confusing to users if we provide multiple ways to specify partition and clustering keys.
Bug: Test failure: TestCqlshOutput failing tests 
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Feb/22 20:24
Resolved: 17/Feb/22 19:28

The following TestCqlshOutput tests are failing consistently on 3.11 and as far as Butler goes, since the very beginning it started collecting data so I am not able to say at this point what exactly broke them:

Test_static_cf_output, test_boolean_output, test_count_output, test_string_output_ascii, test_null_output, test_columness_key_output, test_numeric_output
Bug: Fix flaky test - dtest.offline_tools_test.TestOfflineTools.test_sstableverify
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Jan/22 15:16
Resolved: 17/Feb/22 11:37

Failed 15 times in the last 16 runs. Flakiness: 13%, Stability: 6%
Error Message

IndexError: list index out of range
Bug: Fix flaky test - org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 04/Feb/22 17:44
Resolved: 15/Feb/22 20:26

Failed 5 times in the last 12 runs. Flakiness: 36%, Stability: 58%

Error Message

java.lang.AssertionError

Stacktrace

java.lang.RuntimeException: java.lang.AssertionError
	at org.apache.cassandra.distributed.impl.IsolatedExecutor.waitOn(IsolatedExecutor.java:218)
	at org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$sync$19(IsolatedExecutor.java:124)
	at org.apache.cassandra.distributed.test.GossipTest.nodeDownDuringMove(GossipTest.java:124)
Caused by: java.lang.AssertionError
	at org.apache.cassandra.locator.TokenMetadata.getTokens(TokenMetadata.java:568)
	at org.apache.cassandra.distributed.test.GossipTest.lambda$nodeDownDuringMove$805d71c$1(GossipTest.java:122)
	at org.apache.cassandra.distributed.impl.IsolatedExecutor.lambda$null$17(IsolatedExecutor.java:123)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:83)
	at java.lang.Thread.run(Thread.java:748)
Bug: dtest byteman errors
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Jan/22 09:15
Resolved: 26/Jan/22 13:17

Testing on another ticket I noticed 4.0 was failing 3 dtests on circle on missing btm files. I checked and the files _are there_. Also it doesn't repro locally. See 4.0 vanilla runs [here|https://app.circleci.com/pipelines/github/bereng/cassandra/555/workflows/21cfe9e4-2d17-45a2-8ea8-1593b7ed6d8b]
Bug: Test Failure: dtest.batch_test.TestBatch.test_batchlog_replay_compatibility_1 (from Cassandra dtests)
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Jan/22 18:57
Resolved: 14/Feb/22 18:54

https://ci-cassandra.apache.org/job/Cassandra-3.0/240/testReport/dtest.batch_test/TestBatch/test_batchlog_replay_compatibility_1/

Failed 1 times in the last 29 runs. Flakiness: 3%, Stability: 96%
Error Message
NameError: name 'mk_bman_path' is not defined

{code}
Stacktrace
self = <batch_test.TestBatch object at 0x7f770b9793a0>

    @ported_to_in_jvm('4.0')
    @since('3.0', max_version='3.x')
    def test_batchlog_replay_compatibility_1(self):
        """
            @jira_ticket CASSANDRA-9673, test that logged batches still work with a mixed version cluster.
    
            Here we have one 3.0/3.x node and two 2.2 nodes and we send the batch request to the 3.0 node.
            """
>       self._batchlog_replay_compatibility_test(0, 1, 'github:apache/cassandra-2.2', 2, 4)

batch_test.py:310: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <batch_test.TestBatch object at 0x7f770b9793a0>, coordinator_idx = 0
current_nodes = 1, previous_version = 'github:apache/cassandra-2.2'
previous_nodes = 2, protocol_version = 4

    def _batchlog_replay_compatibility_test(self, coordinator_idx, current_nodes, previous_version, previous_nodes, protocol_version):
        session = self.prepare_mixed(coordinator_idx, current_nodes, previous_version, previous_nodes,
                                     protocol_version=protocol_version, install_byteman=True)
    
        coordinator = self.cluster.nodelist()[coordinator_idx]
>       coordinator.byteman_submit([mk_bman_path('fail_after_batchlog_write.btm')])
E       NameError: name 'mk_bman_path' is not defined

batch_test.py:380: NameError
{code}
Bug: Dynamic Table TTL
Status: Triage Needed
Priority: Normal
Created: 10/Feb/22 19:55

One limitation of the {{default_time_to_live}} option is that it only applies to newly inserted data so an expensive migration is required when altering the table-level TTL, which is not an uncommon request due to changes in retention policies.

This seems to have been a deliberate design decision when adding the Table TTL feature on CASSANDRA-3974, due to the reasons stated [on this comment|https://issues.apache.org/jira/browse/CASSANDRA-3974?focusedCommentId=13427314&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13427314] so we should revisit and potentially address these concerns.

I would like to explore supporting dynamic TTL, which would reflect any updates to the table-level {{default_time_to_live}} immediately to all table data.
Bug: Revert CASSANDRA-17132 and instead deprecate the parameters and update NEWS.txt
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Feb/22 23:05
Resolved: 13/Feb/22 01:52

Follow up on [https://lists.apache.org/thread/z5zmv284cxfm5ljr0fogqr2fxbc9kwlw]

by revert and instead deprecate the parameters removed in CASSANDRA-17132.
Bug: Security Bug
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 31/Jan/22 10:28
Resolved: 31/Jan/22 12:00

The Cassandra 3.11.11 uses _netty-all-4.0.44.Final.jar_

This library has the following CVEs. {*}Part of these CVEs are critical{*}!
Please upgrade to 4.1.71.Final

CVE-2019-20445
CVE-2019-20444
CVE-2019-16869
CVE-2020-7238
CVE-2021-37136
CVE-2021-37137
CVE-2021-21409
CVE-2021-43797
CVE-2021-21295
CVE-2021-21290
Bug: Native protocol v5 specification references non-existing sections
Status: Open
Priority: Normal
Created: 08/Feb/22 21:37

Section 10 and Section 2.2.1 do not exist, this means I don't get correct links when I format the spec as html with https://github.com/martin-sucha/cqlprotodoc.

Section 2.4.1 is not linkable (there is no section header for it, only subsections), but that is probably not an issue since it's referenced from section 2.4.

See attached patch: [^sections.patch]