av9ash/GitBugs huggingface dataset:
Bug: Unexpectedly ignored dtests
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Aug/21 15:37
Resolved: 14/Sep/21 11:21

An issue, which I was hit:

When one class in a dtest file is marked as resource intensive, then all tests in all classes are treated as resource intensive. For example, [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py] contains three classes and the last class is marked as resource intensive:
{code:java}
@pytest.mark.resource_intensive
class TestRepairDataSystemTable(Tester):
{code}
So if I try to run an unmarked class: 
{code:java}
pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair --collect-only --skip-resource-intensive-tests
{code}
then all tests are ignored
{code:java}
collected 36 items / 36 deselected 
{code}
This is because a test is treated to be marked if any class in the same file has the mark. This bug was introduced in the fix of CASS-16399. Before only upgrade tests had such behaviour, i.e., if a class is marked as upgrade test, then all tests are upgrade test in the file.

 

This bug, for example, means that if the same file contains one class marked with vnodes and another class with no_vnodes, then no tests will be executed in the file.

I also noticed another issue that If a test run is executed with the argument {{-only-resource-intensive-tests}} and there is no sufficient resources for resource intensive tests, then no tests were executed. Thus it was necessary to provide {{-force-resource-intensive-tests}} in addition.

Suggestions for the solutions:
 # Require to mark each class and remove the special case of upgrade tests. This will simplify the implementation and might be more obvious for new comers.
 # Treat {{-only-resource-intensive-tests}} in the same way as {{-force-resource-intensive-tests}}, so it will be enough to just specify it even with no sufficient resources.

*Update:* comments were provided to keep only the first suggestion and do not implement the second suggestion.
Bug: Sstableverify unit test operate on SSTables
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Sep/21 16:42
Resolved: 26/Jan/22 07:16

as part of https://issues.apache.org/jira/browse/CASSANDRA-16009, unit coverage is a bit lax and doesn't run through the verifier (based on my coverage results).

There should be a unit test that exercises the internal verifier both for a corrupt example and a working example.
Bug: List snapshots of dropped tables
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Aug/21 06:21
Resolved: 29/Apr/22 22:03

Auto snapshots from dropped tables don't seem to show up in {{nodetool listsnapshots}} (even though they do get cleared by {{nodetool clearsnapshot}}). This makes them kind of annoying to clean up, since you need to muck about in the data directory to find them.

Erick on the mailing list said that this seems to be an oversight and that clearsnapshot was fixed by [CASSANDRA-6418|https://issues.apache.org/jira/browse/CASSANDRA-6418].

I reproduced this both on 3.11.11 and 4.0.0.
Bug: Allow reverse iteration of resources during permissions checking
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Sep/21 15:55
Resolved: 08/Oct/21 12:51

When we perform authz checks for AuthenticatedUser on a given resource, we traverse the resource hierarchy until we either find the required permission or we exhaust the traversal. This goes from most specific resource (i.e. iResource we're interested in) to the broadest (the root for that iResource type).

Since permissions are a whitelist it isn't possible for a permission granted on a more specific resource to be negated or overridden by a grant on a resource lower in the hierarchy towards the root. For example, for DataResource, the hierarchy goes:
{code:java}
root -> keyspace -> table{code}
So for instance if we:

 
{code:java}
GRANT ALL ON KEYSPACE ks TO admin_user; 
{code}
 It is impossible to grant admin_user any set of permissions more restrictive than ALL on ks.

When a request comes in for a user with permissions on a ks for example, you can have a path of:
 # Validate SELECT on t1
 # Then check for SELECT on ks
 # Then check for permissions on 'root'

These unnecessary lookups can negate some of the benefits of caching (and pre-warming, which another ticket is in flight for), and lead to issues on bounces with timeouts.

Additionally, the permissions cache ends up storing far more entries than necessary, as a subsequent request for ks.t2 by user_x will go through the same process and create a second cache entry etc.

So all this said - this is something we should allow operators to opt-in to; impact and performance is going to be highly dependent on the pattern of authentication granting and usage on a cluster.
Bug: Verify correct ownership of attached locations on disk at C* startup
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Aug/21 15:39
Resolved: 05/Oct/21 16:15

There's two primary things related to startup and disk ownership we should mitigate.

First, an instance can come up with an incorrectly mounted volume attached as its configured data directory. This causes the wrong system tables to be read. If the instance which was previously using the volume is also down, its token could be taken over by the instance coming up.

Secondly, in a JBOD setup, the non-system keyspaces may reside on a separate volume to the system tables. In this scenario, we need to ensure that all directories belong to the same instance, and that as the instance starts up it can access all the directories it expects to be able to. (including data, commit log, hints and saved cache dirs)
Bug: Improve MV TTL error message
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Sep/21 06:28
Resolved: 20/Sep/21 07:52

Old MVs could have been created with a {{default_time_to_live}} before the time of CASSANDRA-12868.

A few years forward customers altering that MV for other reasons might get a very confusing message which can benefit from some clarification.

{code}
ALTER MATERIALIZED VIEW XXXXX_view WITH gc_grace_seconds = 10800;

Cannot set or alter default_time_to_live for a materialized view. Data in a materialized view always expire at the same time than the corresponding data in the parent table.
{code}
Bug: Catch read repair timeouts and add metrics to indicate they occurred
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Aug/21 16:04
Resolved: 01/Sep/21 15:48

When we fire off async read repairs onto their own executor they may time out and in doing so, we don't have anything that stops them from propagating that timeout exception the way up to CassandraDaemon's uncaught exception handler. When this happens we logs at ERROR.

Obviously a timeout isn't great, but it's not an ERROR, so we should trap them instead and add some metrics around this occurrance.
Bug: Actively update auth caches in the background
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Sep/21 14:12
Resolved: 09/Nov/21 20:35

Currently the guava cache backing the various auth caches refreshes its data lazily; you won't get an update on cached credentials until you try and read them and they're expired. For the PasswordCache in particular, this not only gives us a window of async "serve the old while you fetch the new" which isn't ideal, but also causes the cache to be invalidated and thus not adding value / perf after expiration period (24h I believe by default).

The expected behavior after this change is for the caches to auto-refresh themselves on an interval so you a) don't have stale data sitting around waiting to be served, and b) getting invalidated so having effectively a dead cache for intermittent users assuming you have the backing resources to serve them proactively.
Bug: Delete branch cassandra-4.0.0
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Jul/21 16:03
Resolved: 01/Aug/21 13:18

With 4.0.0 released, and the {{cassandra-4.0.0}} tag created, the temporary {{cassandra-4.0.0}} should now be deleted.

Changes are required first in cassandra-builds and cassandra repos, to prevent ci-cassandra.a.o and circleci breaking correspondingly.
Bug: Handle timestamp wraparound for internode messages
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Sep/21 08:29
Resolved: 30/Sep/21 09:29

In internode messaging we send the lower 4 bytes of the timestamp and then correct on receiving side if the local time has wrapped (every 50 days), but we currently don't correct the time if the sending side wrapped before us. 

This can cause us to use the wrong timestamp for the message and if the local System.nanoTime is < 50 days, we get a negative creation time which causes an AssertionError in MonitorableImpl
Bug: Remove windows-specific classes
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Sep/21 13:55
Resolved: 05/Feb/22 17:51

To continue the work CASSANDRA-16171 began, now that Windows support is no more there are some source files that can be removed.

Just doing a naive grep on the source directory I see:

{noformat}
src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
src/java/org/apache/cassandra/utils/WindowsTimer.java
{noformat}
Bug: CompactionTask#runMayThrow should not release new SSTables for offline transactions
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Sep/21 16:03
Resolved: 30/Sep/21 12:19

Right now, {{CompactionTask#runMayThrow}} releases new SSTables for offline transactions ([code|https://github.com/apache/cassandra/blob/f7c71f65c000c2c3ef7df1b034b8fdd822a396d8/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L227-L230]). This change was added in CASSANDRA-8962, prior to the introduction of lifecycle transactions in CASSANDRA-8568. I suspect that this behavior might be undesired and could have just fallen through the cracks.

To my knowledge, this code does not cause any known bugs solely because in-tree tools do not access the SSTables they produce before exiting. However, if someone is to write, say, offline compaction daemon, it might break on subsequent compactions because newly created SSTables will be released.
Bug: A user should be able to view permissions of role they created
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Aug/21 12:28
Resolved: 27/Oct/21 17:30

Currently users are denied to view permissions to see a role they created:
{code}
CREATE ROLE parent WITH PASSWORD = 'x' AND LOGIN = true;
GRANT CREATE ON ALL ROLES TO parent;
LOGIN parent;
CREATE ROLE child WITH PASSWORD = 'x' AND LOGIN = true;
LIST ALL PERMISSIONS OF 'child'; -- You are not authorized to view child's permissions
{code}
When a user creates a role they should get the {{DESCRIBE}} permission on that role by default.
Bug: Update from Jackson 2.9 to 2.12
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Aug/21 21:13
Resolved: 17/Sep/21 21:38

Given that Jackson 2.9 support has ended, it would be good to move at least to the next minor version (2.10, patch 2.10.5) or later – latest stable being 2.12.4.
 I can test to see if anything breaks, but looking at existing Jackson usage there shouldn't be many issues.

Assuming upgrade is acceptable there's the question of which branches to apply it to; I will first test it against 4.0.
Bug: Add TTL support to nodetool snapshots
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Jul/21 14:03
Resolved: 26/Aug/21 18:22

Add new parameter {{--ttl}} to {{nodetool snapshot}} command. This parameter can be specified in human readable duration (ie. 30mins, 1h, 300d) and should not be lower than 1 minute.

The expiration date should be added to the snapshot manifest in ISO format.

A periodic thread should efficiently scan snapshots and automatically clear those past expiration date. The periodicity of the scan thread should be 1 minute by default but be overridable via a system property.

The command {{nodetool listsnapshots}} should display the expiration date when the snapshot contains a TTL.
Bug: Create property to hold local maven repository location
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 04/Sep/21 14:25
Resolved: 17/Nov/21 10:03

In their current state, both build.xml and .build/build-resolver.xml refer to the local maven repository by "${user.home}/.m2/repository" in multiple places.
It would be nice instead to have a property store this location, and refer the property vs the actual location itself. 
If one needs to change this maven repository location for their environmental purposes, this change makes it easy since it will need to be changed only at one place (the value of the property) vs changing all the direct references to the repository location.
Bug: Add client warnings and abort to tombstone and coordinator reads which go past a low/high watermark
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Aug/21 18:46
Resolved: 26/Aug/21 20:41

We currently will abort queries if we hit too many tombstones, but its common that we would want to also warn clients (client warnings) about this before we get that point; its also common that different logic would like to be able to warn/abort about client options (such as reading a large partition).  To allow this we should add a concept of low/high watermarks (warn/abort) to tombstones and coordinator reads.

Another issue is that current aborts look the same as a random failure, so from an SLA point of view it would be good to differentiate between user behavior being rejected and unexplained issue.
Bug: PasswordObfuscator should not assume PASSWORD is the last item in the WITH clause
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 13/Jul/21 19:34
Resolved: 25/Jan/22 06:19

CASSANDRA-16669 introduced support for obfuscating passwords for audit log statements, but there are a few cases where the obfuscation logic can destroy some of the contents of the original/provided string.

ex. This is perfectly valid...

{noformat}
WITH LOGIN = false AND PASSWORD = 'bar' AND SUPERUSER = false
{noformat}

...but calling obfuscate() on it will produce...

{noformat}
WITH LOGIN = false AND PASSWORD *******
{noformat}

-We should be able to create a reasonable RegEx and use String#replaceAll() to both simplify and correct PasswordObfuscator#obfuscate().-
Bug: Truncation snapshots unnecessarily created on node startup
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Aug/21 12:46
Resolved: 11/Oct/21 17:12

When testing cassandra 4.0 on ccm I noticed that everytime I restart a node, truncation snapshots are created for the tables {{system.table_estimates}} and {{system.size_estimates}}:

{noformat}
$ ccm create -n 1 test -s
$ ccm node1 stop
$ ccm node1 start
$ ccm node1 stop
$ ccm node1 start
$ ccm node1 nodetool listsnapshots

Snapshot Details:
Snapshot name                           Keyspace name Column family name True size Size on disk
truncated-1628599001857-table_estimates system        table_estimates    0 bytes   13 bytes
truncated-1628599099560-table_estimates system        table_estimates    0 bytes   13 bytes
truncated-1628599001736-size_estimates  system        size_estimates     0 bytes   13 bytes
truncated-1628599057438-table_estimates system        table_estimates    6.16 KiB  6.19 KiB
truncated-1628599099458-size_estimates  system        size_estimates     0 bytes   13 bytes
truncated-1628599057340-size_estimates  system        size_estimates     5.73 KiB  5.76 KiB

Total TrueDiskSpaceUsed: 0 bytes
{noformat}

Not sure if this is expected behavior, but feels like a bug to me.

Reproduced on 4.0, not sure if it reproduces on lower versions.
Bug: WEBSITE - August 2021 updates #2
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Aug/21 23:22
Resolved: 04/Sep/21 08:34

Updates to be made to the website that include:

* Add blog post: Cassandra-on-Kubernetes-A-Beginners-Guide to pages/blog
* Added Cassandra-on-Kubernetes-A-Beginners-Guide card to blog index
* Changed Urban Airship to Airship on the case studies page
* Changed the contributor meetings learn more link on the community page
* Replace images/companies backblaze.png with the new logo attached with the same name
* Removed Comcast logo from main page index
* Removed Comcast from case studies page
* Removed Yelp from Apache Cassandra 4.0 is Here blog
Bug: CEP-10: Cluster and Code Simulations
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Sep/21 17:45
Resolved: 09/May/22 16:12

This is a tracking Jira for CEP-10: https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-10%3A+Cluster+and+Code+Simulations
Bug: DOC - Jumbled contents in doc/4.0/cassandra/configuration/cass_yaml_file.html
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 23/Aug/21 10:18
Resolved: 31/Mar/22 09:09

The page at [https://cassandra.apache.org/doc/4.0/cassandra/configuration/cass_yaml_file.html#commitlog_directory] seems to have published a portion of a Git merge annotation which renders the formatting incorrect.
Bug: DOC - Broken link to Snitch page in site nav menu
Status: In Progress
Priority: Normal
Created: 29/Jul/21 07:46

There are three issues since website has been recently updated.
 * Links that used have latest in url path is not working [https://cassandra.apache.org/doc/latest/operating/snitch.html]
 * -Website table column name and values are not matching, they are off by one. Example - [https://cassandra.apache.org/doc/3.11/cassandra/operating/hints.html]- (fixed by CASSANDRA-17346)
 * Website typeahead search also stopped working.
Bug: Single partition reads can read more SSTables than required 
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Sep/21 13:08
Resolved: 28/Sep/21 16:14

For some scenarios involving row deletions, range deletions or static columns, the logic of {{SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder}} might trigger more SSTables reads that expected. 

For row deletions and range deletions the reasons is that the logic do not take them into account. Once we hit a deleted row (caused by a row deletion or a range deletion) with a timestamp higher than the one of the next SStable we know that we can stop reading more SSTables.

For static columns the problems seems to have been introduced by the changes in CASSANDRA-16671.
Bug: The clustering order logic in Materialized view creation changed in 4.0
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Aug/21 13:19
Resolved: 31/Aug/21 15:05

Before 4.0, when a Materialized view was created with no {{CLUSTERING ORDER}} specified the clustering order was based on the base table clustering order (see [3.11 code|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/cql3/statements/CreateViewStatement.java#L108]). In 4.0 this behaviour changed and the clustering order was defaulted to ASC for all columns.
Bug:  Multiple full sources can be select unexpectedly for bootstrap streaming
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Sep/21 16:06
Resolved: 12/Nov/21 00:30

With CASSANDRA-14404, the RangeStreamer selects all endpoints as sources when using strict consistency and RF > endpoints count. In such case, the bootstrapping node is almost guaranteed to fail in the validation later. 

It is a change of behavior. Before the patch, in such case (i.e., higher RF) the bootstrapping node just pick one endpoint as source, since consistent range movement cannot be established. I'd propose to restore the old behavior.
Bug: NPE when running in-jvm upgrade dtest in CircleCI
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Jul/21 00:11
Resolved: 19/Jul/21 21:32

CASSANDRA-16649 changed the upgrade dtest to test all upgrade paths. When running from trunk, the dtest jar of the lower version should present. 
 
CircleCI config is not updated to build the cassandra-4.0 dtest jar, so that most of the upgrade dtest fail. Add the 4.0 version to the dtest jar build task should fix.
Bug: Ignore repair requests in mixed mode
Status: Review In Progress
Priority: Normal
Created: 18/Aug/21 06:02

Repairs in mixed mode are not supported. On 4.0 we have CASSANDRA-13944 but <4.0 there is nothing preventing NPE and other scares in the logs. These cause unnecessary noise to operators.

{noformat}
java.lang.RuntimeException: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:423)
at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:93)
{noformat}
Bug: Unable to build from source - generate-jflex-java task fails with exception regarding JFlex
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 08/Aug/21 11:14
Resolved: 08/Aug/21 13:57

I'm unable to build Cassandra from source by running {{ant}} in the root directory on Windows 10, using AdoptOpenJDK 8 and Ant 1.10.11. The build fails with the following:
{code:bash}
generate-jflex-java:

BUILD FAILED
C:\Users\Jonathon\IdeaProjects\cassandra\build.xml:438: java.lang.NoSuchMethodError: jflex.core.LexParse.getSymbolFactory()Ljava_cup/runtime/SymbolFactory;
        at jflex.core.LexParse$CUP$LexParse$actions.CUP$LexParse$do_action_part00000000(LexParse.java:1087)
        at jflex.core.LexParse$CUP$LexParse$actions.CUP$LexParse$do_action(LexParse.java:2257)
        at jflex.core.LexParse.do_action(LexParse.java:598)
        at java_cup.runtime.lr_parser.parse(lr_parser.java:569)
        at jflex.generator.LexGenerator.generate(LexGenerator.java:74)
        at jflex.anttask.JFlexTask.execute(JFlexTask.java:78)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
        at org.apache.tools.ant.Task.perform(Task.java:350)
        at org.apache.tools.ant.Target.execute(Target.java:449)
        at org.apache.tools.ant.Target.performTasks(Target.java:470)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1401)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1374)
        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1264)
        at org.apache.tools.ant.Main.runBuild(Main.java:818)
        at org.apache.tools.ant.Main.startAnt(Main.java:223)
        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284)
        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)
{code}
I've also tried doing it through Windows Subsystem for Linux (WSL 2) with the same versions of Java and Ant. I get the same error.

My environment:
 * Windows 10 Pro (Version 20H2, build 19042.1110)
 * 
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)
 * 
Apache Ant(TM) version 1.10.11 compiled on July 10 2021

Note: I should probably mention that I'm new to Cassandra and I've never been able to build it before. I'm trying to run an initial build before I run the script to generate Intellij IDEA files (since generate-jflex-java is executed during the generate-idea-files task, it fails there with the error above too). If I'm doing something wrong, please let me know.
Bug: BLOG - Changelog #8 July 2021
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Jul/21 08:34
Resolved: 08/Jul/21 10:16

Draft by [~Calico] - [C* Blog: Cassandra Changelog #8|https://docs.google.com/document/d/1aL-1C2ckMYvT9ch8pQUsAik_uA5SohAiI2POe4BvE20/edit?usp=sharing].

Highlights:
 * Apache Cassandra 4.0-rc2 released
 * say hello to our Google Summer of Code intern
 * new community intro to Cassandra videos