av9ash/GitBugs huggingface dataset:
Bug: Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Jan/23 19:43
Resolved: 31/Mar/23 15:44

On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store HostID in the new "me" SSTable version.

But SSTables flushed during CommitLogReplay miss this HostID info.

 

In the next Cassandra startup, if these SSTables were still present, system.log will show:


{{WARN Origin of 3 sstables is unknown or doesn't match the local node; commitLogIntervals for them were ignored}}

{{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node; commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }}

 

And debug.log will show a list of SSTables, witch can include "md" and "me" version (before upgradesstables):

 

{{Ignored commitLogIntervals from the following sstables: [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db, /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db, /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}}

 

https://issues.apache.org/jira/browse/CASSANDRA-16619
Bug: snakeyaml vulnerabilities: CVE-2021-4235, CVE-2022-1471, CVE-2022-3064
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Jan/23 21:29
Resolved: 24/Jan/23 18:30

The OWASP scan is reporting these for both snakeyaml-1.11 and snakeyaml-1.26.

These are similar to CASSANDRA-17907 in that they require access to the yaml to have any effect.
Bug: Test Failure: cqlsh_tests.test_cqlsh_copy.TestCqlshCopy.test_bulk_round_trip_non_prepared_statements
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 15/Jan/23 10:14
Resolved: 19/Nov/24 09:50

Flaky.
{noformat}
itf = ('127.0.0.3', 7000)

    def assert_socket_available(itf):
        info = socket.getaddrinfo(itf[0], itf[1], socket.AF_UNSPEC, socket.SOCK_STREAM)
        if not info:
            raise UnavailableSocketError("Failed to get address info for [%s]:%s" % itf)
    
        (family, socktype, proto, canonname, sockaddr) = info[0]
        s = socket.socket(family, socktype)
        s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    
        try:
>           s.bind(sockaddr)
E           OSError: [Errno 98] Address already in use

../venv/lib/python3.8/site-packages/ccmlib/common.py:631: OSError

During handling of the above exception, another exception occurred:

self = <cqlsh_tests.test_cqlsh_copy.TestCqlshCopy object at 0x7f36a7d70580>

    def test_bulk_round_trip_non_prepared_statements(self):
        """
            Test bulk import with default stress import (one row per operation) and without
            prepared statements.
    
            @jira_ticket CASSANDRA-11053
            """
>       self._test_bulk_round_trip(nodes=3, partitioner="murmur3", num_operations=100000,
                                   copy_from_options={'PREPAREDSTATEMENTS': False})

cqlsh_tests/test_cqlsh_copy.py:2475: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
cqlsh_tests/test_cqlsh_copy.py:2391: in _test_bulk_round_trip
    self.prepare(nodes=nodes, partitioner=partitioner, configuration_options=configuration_options)
cqlsh_tests/test_cqlsh_copy.py:116: in prepare
    self.cluster.populate(nodes, tokens=tokens).start()
../venv/lib/python3.8/site-packages/ccmlib/cluster.py:513: in start
    common.assert_socket_available(itf)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

itf = ('127.0.0.3', 7000)

    def assert_socket_available(itf):
        info = socket.getaddrinfo(itf[0], itf[1], socket.AF_UNSPEC, socket.SOCK_STREAM)
        if not info:
            raise UnavailableSocketError("Failed to get address info for [%s]:%s" % itf)
    
        (family, socktype, proto, canonname, sockaddr) = info[0]
        s = socket.socket(family, socktype)
        s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    
        try:
            s.bind(sockaddr)
            s.close()
            return True
        except socket.error as msg:
            s.close()
            addr, port = itf
>           raise UnavailableSocketError(
                "Inet address %s:%s is not available: %s; a cluster may already be running or you may need to add the loopback alias" % (
                addr, port, msg))
E           ccmlib.common.UnavailableSocketError: Inet address 127.0.0.3:7000 is not available: [Errno 98] Address already in use; a cluster may already be running or you may need to add the loopback alias

../venv/lib/python3.8/site-packages/ccmlib/common.py:637: UnavailableSocketError
{noformat}
Bug: Exception message for RTBoundValidator
Status: Open
Priority: Normal
Created: 23/Feb/23 18:15

From the exception RTBoundValidator its not clear what 'RT' refers to. 

1.) Exception subject

Instead of: 'has an illegal RT bounds sequence'

change to: 'has an illegal Range Tombstone bounds sequence'

2.) Why doesn't the (key: %s omdt: [%s]) display in the exception message?

Example:

2023-01-31 18:26:27,480 [ERROR] [ReadStage-9] cluster_id=5 ip_address=127.1.1.1 AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread Thread[ReadStage-9,5,main]
java.lang.IllegalStateException: MERGED UnfilteredRowIterator for rdhprod.instrument_alias_map has an illegal RT bounds sequence: unexpected end bound or boundary Marker INCL_END_BOUND(T, TBG)@1612002648782856/1612002648
at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120) ~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87) ~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:148) ~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) ~[apache-cassandra-3.11.11.jar:3.11.11]
Bug: Replace eclipse warnings based static code analysis with something better (CheckerFramework)
Status: Review In Progress
Priority: Normal
Created: 06/Feb/23 13:00

Eclipse warnings is used for static code analysis. However, it does not fit well into Cassandra code and practically we end up explicitly adding suppressions in many places just to satisfy that tool rather than fix the real issues.

This is an incomplete list of reasons to remove it:
 - not closed resources are detected incorrectly
 - does not recognize custom utility methods used to close the resources, which use use heavily in the code, like {{{}Throwables.close{}}}, {{{}FileUtils.close{}}}, {{{}closeQuietly{}}}...
 - because of the above, we cannot make important things like {{Ref}} to implement {{Closeable}} as it would make the tool to explode with tons of warnings
 - it complains about correct generics - something like "method X is not applicable for ..." when the code compiles successfully is not acceptable
 - it is old and not maintained

There are better tools like IntelliJ inspections for example, which can also be run in headless mode

As agreed on the [dev mailing list|https://lists.apache.org/thread/8ok01odwx79crxw45cnfh0n1j4nsf9cp] we will replace it in trunk with CheckerFramework.
Bug: Warn about unqualified prepared statement only if it is a select, update, delete, insert
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Mar/23 10:06
Resolved: 08/Aug/24 12:41

Hi, 

We get the following warnings when we use prepared statements with "create keyspace ... " or "drop keyspace" statements.

"

{{USE <keyspace>}} with prepared statements is considered to be an anti-pattern due to ambiguity in non-qualified table names. Please consider removing instances of {{{}Session#setKeyspace(<keyspace>){}}}, {{Session#execute("USE <keyspace>")}} and {{cluster.newSession(<keyspace>)}} from your code, and always use fully qualified table names (e.g. <keyspace>.<table>). Keyspace used: null, statement keyspace: null, statement id: 8153d922390fdf9a9963bfeda85b2f3b at 

"

Such statements are already full-qualified. So, why are we getting this warning? 

Regards
Mohammad
Bug: Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 14/Mar/23 02:03
Resolved: 06/Oct/23 10:44

 
{code:java}
assert not True
+ where True = <bound method Event.is_set of <threading.Event object at 0x7f5b16fda070>>()
+ where <bound method Event.is_set of <threading.Event object at 0x7f5b16fda070>> = <threading.Event object at 0x7f5b16fda070>.is_set
Stacktrace
self = <bootstrap_test.TestBootstrap object at 0x7f5b25f4f880>
def test_cleanup(self):
"""
@jira_ticket CASSANDRA-11179
Make sure we remove processed files during cleanup
"""
cluster = self.cluster
cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', 'True')
cluster.set_configuration_options(values=
{'concurrent_compactors': 4}
)
cluster.populate(1)
cluster.start()
node1, = cluster.nodelist()
for x in range(0, 5):
node1.stress(['write', 'n=100k', 'no-warmup', '-schema', 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', 'replication(factor=1)', '-rate', 'threads=10'])
node1.flush()
node2 = new_node(cluster)
node2.start(wait_for_binary_proto=True)
event = threading.Event()
failed = threading.Event()
jobs = 1
thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, len(node1.get_sstables("keyspace1", "standard1")), jobs, failed))
thread.setDaemon(True)
thread.start()
node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs))
event.set()
thread.join()
> assert not failed.is_set()
E assert not True
E + where True = <bound method Event.is_set of <threading.Event object at 0x7f5b16fda070>>()
E + where <bound method Event.is_set of <threading.Event object at 0x7f5b16fda070>> = <threading.Event object at 0x7f5b16fda070>.is_set
bootstrap_test.py:912: AssertionError
{code}
 

failed twice
Bug: Test failure: org.apache.cassandra.distributed.test.SchemaTest timeout
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 12/Jan/23 07:57
Resolved: 22/Sep/23 05:21

See here for another timeout of this test

https://app.circleci.com/pipelines/github/bereng/cassandra/836/workflows/dd9f0441-df59-40e7-b00b-c0788f0235a6/jobs/7597/tests#failed-test-0

It is unclear if it's an env issue or related to CASSANDRA-17819 or similar ones as this test has some history to it.
Bug: Flaky test: org.apache.cassandra.cql3.validation.operations.InsertUpdateIfConditionTest.testConditionalUpdate[1: clusterMinVersion=3.11]-compression.jdk1.8 on trunk
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Mar/23 19:52
Resolved: 04/Sep/23 07:25

Failed 1 times in the last 1 runs. Flakiness: 0%, Stability: 0%
{code}
Error Message
5.0.0-SNAPSHOT boolean:false

Stacktrace
junit.framework.AssertionFailedError: 5.0.0-SNAPSHOT boolean:false
	at org.apache.cassandra.cql3.validation.operations.InsertUpdateIfConditionTest.lambda$data$1(InsertUpdateIfConditionTest.java:70)
	at org.apache.cassandra.cql3.validation.operations.InsertUpdateIfConditionTest.beforeSetup(InsertUpdateIfConditionTest.java:95)
	at org.apache.cassandra.cql3.validation.operations.InsertUpdateIfConditionTest.before(InsertUpdateIfConditionTest.java:89)
{code}
Bug: Test failure: org.apache.cassandra.distributed.test.accord.AccordIntegrationTest
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 02/Feb/23 16:48
Resolved: 28/Mar/24 16:37

The accord integration tests are currently flaky due to Preempt happening (we convert that to a timeout).  We need to address this to make sure the tests are stable.

One solution may be make SimpleProgressLog’s frequency run different for jvm-dtest (likely to depend on CASSANDRA-18221 ), this would be similar to what we already do as timeouts maybe larger in jvm-dtest due to the extra complexity running multiple processes in the same JVM.  Another solution may be to have the client coordinator “attach” to the new coordinator, this would avoid the user being impacted by this preempt when the coordinator is healthy.  

Example: https://app.circleci.com/pipelines/github/dcapwell/cassandra/1822/workflows/0d92898f-ad7d-4a7d-9198-3a7ce844ee93/jobs/16464
Bug: Centralize all snapshot operations to SnapshotManager
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 17/Feb/23 16:32
Resolved: 20/Jun/24 08:32

Currently, there is a lot of places where snapshotting logic is in place, scattered across the codebase. We should centralize snapshotting logic under SnapshotManager so every place in the code interacts only with that. This should remove whole class of possible synchronization problems. Merely looking into the code, all these methods are potentially unsafe.
{code:java}
public Map<String, TableSnapshot> listSnapshots()
protected TableSnapshot buildSnapshot(String tag, SnapshotManifest manifest, Set<File> snapshotDirs) {
protected static SnapshotManifest maybeLoadManifest(String keyspace, String table, String tag, Set<File> snapshotDirs)
public List<String> listEphemeralSnapshots()
private List<File> listAllSnapshots()
protected Map<String, Set<File>> listSnapshotDirsByTag()
public boolean snapshotExists(String snapshotName)
public static void clearSnapshot(String snapshotName, List<File> tableDirectories, RateLimiter snapshotRateLimiter)
public static void removeSnapshotDirectory(RateLimiter snapshotRateLimiter, File snapshotDir)
public long trueSnapshotsSize()
{code}
Bug: Enhance nodetool compactionstats with existing MBean metrics
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Mar/23 20:45
Resolved: 23/Jun/23 09:20

Nodetool compactionstats reports only on active compactions, if nothing is active, you see only:
{quote}$nodetool compactionstats

pending tasks: 0
{quote}
but in the MBean Compaction/TotalCompactionsCompleted there are recent statistic in events/second for:
 * Count
 * FifteenMinueRate
 * FiveMinueRate
 * MeanRate
 * OneMinuteRate

1) It would be useful to see in addition:
{quote}pending tasks: 0

compactions completed: 20

    1 minute rate:    0/second

   5 minute rate:    2.3/second

  15 minute rate:   4.6/second
{quote}
2) Since compaction_throughput_mb_per_sec is a throttling parameter in cassandra.yaml (default 64 MBps), it would be nice to show the actual compaction throughput and be able to observe if you're close to the limit.  I.e., 
{quote}compaction throughput 13.2 MBps / 16 MBps (82.5%)
{quote}
3) for completness, compactionstats should list the number of concurrent compactors configured, perhaps simply add to existing 'pending tasks' line:
{quote}4 concurrent compactors, 0 pending tasks
{quote}
Bug: Push LocalSessions info logs to debug
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Mar/23 15:10
Resolved: 14/Mar/24 15:42

I don't think these are particularly useful to have at info, but the impetus for this ticket is specifically [this message|https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java#L456] during automatic cleanup about skipping deleting sessions that seems to confuse or alarm people.
Bug: Log hardware and container params during test runs to help troubleshoot intermittent failures
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/Jan/23 18:55
Resolved: 23/Apr/24 19:56

{color:#000000}We’ve long had flakiness in our containerized ASF CI environment that we don’t see in circleci. The environment itself is both containerized and heterogenous, so there are differences in both the hardware environment and the software environment in which it executes. For reference, see: [https://github.com/apache/cassandra-builds/blob/trunk/ASF-jenkins-agents.md#current-agents]{color}
{color:#000000} {color}
{color:#000000}We should log a variety of hardware, container, and software environment details to help get to the bottom of where some test failures may be occurring. As we don’t have shell access to the machines it’ll be easier to have this information logged / retrieved during test runs than to try and profile each host independently.{color}
Bug: Warning message on aggregation queries doesn't specify table name or aggregate type
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 01/Feb/23 22:08
Resolved: 07/Feb/23 15:02

The existing aggregation query warning messages in cassandra.log from SelectStatement.java:

     _'[WARN] Aggregation query used without partition key'_

     {_}'{_}{_}[WARN]{_} _Aggregation query used on multiple partition keys (IN restriction)'_

are missing the helpful details which would assist users in quickly identifying the offending query. The table name and type of aggregation would be useful additions in the WARN message.

In addition, following the example of the slow query logger and printing out the query string at DEBUG level would be especially helpful.
Bug: Investigate removing PriorityQueue usage from KeyRangeConcatIterator
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Jan/23 13:03
Resolved: 01/Apr/24 15:10

It has been identified during the review of CASSANDRA-18058 that the 
KeyRangeConcatIterator could potentially stop using a PriorityQueue to maintain it's active list of sorted KeyRangeIterators.

The code suggested by [~maedhroz] is as follows:

{code:java}
private int i = 0;
...
protected void performSkipTo(PrimaryKey primaryKey)
{
    while (i < toRelease.size())
    {
        RangeIterator currentIterator = toRelease.get(i);

        if (currentIterator.getCurrent().compareTo(primaryKey) >= 0)
            break;

        if (currentIterator.getMaximum().compareTo(primaryKey) >= 0)
        {
            currentIterator.skipTo(primaryKey);
            break;
        }

        i++;
    }
}
...
protected PrimaryKey computeNext()
{
    while (i < toRelease.size())
    {
        RangeIterator currentIterator = toRelease.get(i);
        
        if (currentIterator.hasNext())
            return currentIterator.next();
        
        i++;
    }

    return endOfData();
}
{code}

It was decided that this change would need performance and correctness testing in it's own right would not be included in the original SAI CEP ticket.
Bug: Harry model and in-JVM test for partition-restricted 2i queries
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 20/Feb/23 22:57
Resolved: 01/Mar/24 22:23

The smallest useful scope for Harry testing w/ 2i would be to cover queries within large partitions. Creating a model to do this should also be easier than it would be for the “global” query case, so it seems like a good place to start. The goal here should be to create something that can simply be run for a specified amount of time as an in-JVM test and be run going forward as part of our normal CI process.
Bug: Allow sharding of the SAI in-memory index 
Status: Triage Needed
Priority: Normal
Created: 01/Feb/23 16:41

The Memtable implementation allows it to be split into a number of shards. This reduces contention during writes. 

The MemtableIndex in SAI should follow this practice. It currently does not support sharding so all writes hit the same synchronized write block. The in-memory index search can also use the sharding to reduce the number of indexes that it searches based on the key range passed to the search.
Bug: CEP-15: Accord Protocol Correctness issue
Status: Open
Priority: Normal
Created: 24/Mar/23 17:38

There is a bug in the Accord protocol which might break linearizability, even in the single partition case.

The file in attachment details this issue following the pseudo-code in the white paper describing Accord [1]. If the Java implementation follows this pseudo-code, it should be subject to the very same problem.

This bug was found by Fedor Ryabinin,  Alexey Gotsman and Pierre Sutra.

[1] https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-15%3A+General+Purpose+Transactions?preview=/188744725/188744736/Accord.pdf
Bug: Test Failure: o.a.c.cql3.validation.operations.AlterTest#testDropListAndAddListWithSameName
Status: In Progress
Priority: Normal
Created: 24/Mar/23 11:03

The unit test {{org.apache.cassandra.cql3.validation.operations.AlterTest#testDropListAndAddListWithSameName}} is flaky at least on trunk. Flakiness seems lesser than 1%.

While I haven't seen it on Jenkins yet, it can easily be reproduced on CircleCI with the multiplexer:
* https://app.circleci.com/pipelines/github/adelapena/cassandra/2731/workflows/58edc2f6-9a21-4d09-b783-b7fb15e1b320/jobs/32235
* https://app.circleci.com/pipelines/github/adelapena/cassandra/2731/workflows/58edc2f6-9a21-4d09-b783-b7fb15e1b320/jobs/32234
* https://app.circleci.com/pipelines/github/adelapena/cassandra/2731/workflows/739a95d3-8e42-4447-93dd-122fc16fdd7d/jobs/32233/tests

Those runs show two types of errors:
{code}
junit.framework.AssertionFailedError: Invalid value for row 0 column 1 (content of type text), expected <first test> but got <st test>

	at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1506)
	at org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:102)
{code}
and: 
{code}
org.apache.cassandra.serializers.MarshalException: Invalid UTF-8 bytes 000000e0279515437f00
	at org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:46)
	at org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:29)
	at org.apache.cassandra.serializers.TypeSerializer.deserialize(TypeSerializer.java:37)
	at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:1494)
	at org.apache.cassandra.cql3.validation.operations.AlterTest.testDropListAndAddListWithSameName(AlterTest.java:102)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
The CircleCI config I used to reproduce the test failure can be generated with:
{code}
.circleci/generate.sh -p \
  -e REPEATED_UTESTS_COUNT=500 \
  -e REPEATED_UTESTS=org.apache.cassandra.cql3.validation.operations.AlterTest
{code}
Bug: Bypass row-awareness for small partitions
Status: Triage Needed
Priority: Normal
Created: 16/Jan/23 14:11

SAI supports row-awareness in that it indexes both the partition key and the clustering key of a row. This improves query performance significantly for wide partitions with many rows but it can impact performance for small partitions where it could make sense to bypass row-awareness post-filter the results (read the whole partition) or batch rows for a single partition.

However this is achieved it would be necessary for the index to have an idea of the size of the partition being read and to be aware of whether reading the whole partition is likely to improve read performance.  

SAI is aware of partition sizes during indexing so one option would be feed these sizes into a histogram in the index metadata and apply a set of rules to this metadata to decide whether we should attempt any optimisation.
Bug: Implicitly enable ALLOW FILTERING on virtual tables
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/Feb/23 11:38
Resolved: 09/Feb/23 13:41

Ticket to track work / discussion of this thread (1)

(1) https://lists.apache.org/thread/104fld746d6qtggq5132n3hqtjqqpkjp