av9ash/GitBugs huggingface dataset:
Bug: Diff Job retry bug fixes in reading previous run's job parameters & marking the job status
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Sep/21 18:19
Resolved: 22/Sep/21 17:43

Diff job retry with same jobId should avoid running diffs on the partitions that were successfully diffed previously. The retry failed because previous run failed to mark the job as not running on exit. This is due to a bug in the resource try catch block where session object Is closed before marking the job as not running. Also there is another bug in the way we get job parameters during rerun of a failed diff job.
Bug: Timestamp String displayed for partition compaction warnings is not correct
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 16/Sep/21 11:15
Resolved: 16/Sep/21 12:02

When compaction encounters a large partition, it outputs a warning in the logs e.g.:
 (Apologies, had to redact some information)

WARN [CompactionExecutor:343] 2021-09-16 09:28:43,539 BigTableWriter.java:211 - Writing large partition XXX/XXXX:sourceid:{color:#de350b}*2021-09-16 05\:00Z*{color} (1.381GiB) to sstable /mnt/var/lib/cassandra/data/segment/message-336c5ff04db211ebbffc2980407d44d6/md-58982-big-Data.db

i.e [https://github.com/apache/cassandra/blob/cassandra-3.11.5/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java#L211]

*Example Table/insert*

CREATE TABLE myks.mytable (
 sourceid text,
 {color:#de350b}*messagehour timestamp,*{color}
 messagetime timestamp,
 messageid text
 PRIMARY KEY ((sourceid, messagehour), messagetime, messageid)
 ) ;

 

insert into myks.mytable (sourceid, messagehour, messagetime, messageid) values ('sourceid', '{color:#de350b}*2021-09-16 05:00Z'*{color}, '2021-09-16 05:00:31Z', '123ABC');

If I then need to try and work out which nodes in the cluster contain the replica data for this partition (from the logs), I will get the token via CQL

eg:
 select distinct token(sourceid,messagehour) from myks.mytable where sourceid='sourceid' and messagehour='{color:#de350b}*2021-09-16 05:00Z*{color}';

system.token(sourceid, messagehour)
 -------------------------------------
 {color:#de350b}*7663675819538124697*{color}

I then run nodetool to get the endpoints for this token/ks/table

eg
 nodetool getendpoints myks mytable {color:#de350b}*7663675819538124697*{color}
 172.31.10.187
 172.31.12.193
 172.31.13.91

And *the list of endpoints is not correct* as the value outputted in the timestamp warning log entry, I suspect, is missing additional information/precision so obviously will give back the wrong token and hence the wrong endpoints.

Possibly this warning log statement should output the actual partition key token in addition to the other information to avoid confusion and the string representation of the timestamp be correct.
Bug:  cassandra local_quorum query is inconsistent
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 09/Sep/21 09:51
Resolved: 09/Sep/21 11:39

The version number of this issue is wrong：https://issues.apache.org/jira/browse/CASSANDRA-16919

update the version: cassandra version: 2.1.15
Number of nodes: dc1: 80, dc2: 80
problem:
Our copy strategy is as follows:
WITH REPLICATION = \{'class':'NetworkTopologyStrategy','dc1': 3,'dc2': 3};
We encountered a problem with cassandra, and it was inconsistent when querying with local_quorum. We will only read and write in dc1.
We also use local_quorum for writing, and then use local_quorum for queries.
But there is a phenomenon, use the following statement:
select count(*) from table where partitionKey=?
The results of the query were initially inconsistent and eventually consistent.

Assuming that the first is 10000, the second is 9998, and the third is 9997, it may remain at 10001 in the end(Maybe it was triggered to read repair, which led to the final stabilization) .
During this period, we have done a large-scale expansion. And make sure that every machine is cleaned up. And we also found that the results of using getEndpoint <keyspace> <table> <key> on different machines are inconsistent. In the end, we found that the result of getEndpoint has 4 machines in dc1.

Then we executed getSstable on the corresponding 4 machines, only 3 machines showed the results, and the other machine did not show the results. At the same time, we encountered a similar problem with another partitionKey, but this partitionKey was only queried once, because we recorded the total number of partitionKey in another place, and we can confirm that the total number of partitionKey is incorrect.

After we restarted each machine of dc1 one by one, this problem was solved.
The total number of partitionKey is consistent with the result recorded by us, and if the same query is done multiple times, the result will not change.
Therefore, I suspect that the gossip synchronization node information is too slow, which may lead to inconsistent final results when selecting nodes for query.
Bug: cassandra local_quorum query is inconsistent
Status: Resolved
Priority: Normal
Resolution: Won't Fix
Created: 08/Sep/21 01:44
Resolved: 08/Sep/21 02:05

cassandra version: 2.0.15
Number of nodes: dc1: 80, dc2: 80
problem:
Our copy strategy is as follows:
WITH REPLICATION = \{'class':'NetworkTopologyStrategy','dc1': 3,'dc2': 3};
We encountered a problem with cassandra, and it was inconsistent when querying with local_quorum. We will only read and write in dc1.
We also use local_quorum for writing, and then use local_quorum for queries.
But there is a phenomenon, use the following statement:
select count(*) from table where partitionKey=?
The results of the query were initially inconsistent and eventually consistent.

Assuming that the first is 10000, the second is 9998, and the third is 9997, it may remain at 10001 in the end(Maybe it was triggered to read repair, which led to the final stabilization) .
During this period, we have done a large-scale expansion. And make sure that every machine is cleaned up. And we also found that the results of using getEndpoint <keyspace> <table> <key> on different machines are inconsistent. In the end, we found that the result of getEndpoint has 4 machines in dc1. 

Then we executed getSstable on the corresponding 4 machines, only 3 machines showed the results, and the other machine did not show the results. At the same time, we encountered a similar problem with another partitionKey, but this partitionKey was only queried once, because we recorded the total number of partitionKey in another place, and we can confirm that the total number of partitionKey is incorrect. 

After we restarted each machine of dc1 one by one, this problem was solved. 
The total number of partitionKey is consistent with the result recorded by us, and if the same query is done multiple times, the result will not change. 
Therefore, I suspect that the gossip synchronization node information is too slow, which may lead to inconsistent final results when selecting nodes for query.
Bug: Optimize heavily-used `String.format()`-with-constant formatter use cases
Status: Triage Needed
Priority: Normal
Created: 30/Aug/21 23:59

For background, I did some benchmarking to compare performance of two simple ways to concatenate 2 Strings with a separator (used in f.ex `QualifiedName.java` for "keyspace:name" concatenation):
 # String.format("%s.%s", keyspace, name);
 # new StringBuilder().append(keyspace).append('.').append(name).toString()

Difference is somewhat significant (for me, factor of 40x), as per:

https://cowtowncoder.medium.com/measuring-performance-of-java-string-format-or-lack-thereof-2e1c6a13362c

and so it seems worthwhile replacing at least some of usage from Cassandra. With a quick look it seems that classes like:
 * QualifiedName.java, FieldSelector.java, Selectable.java (in cql3)

could benefit from this. And the second implementation can be packaged as a simple utility method (concatenateStringsWithChar?) so that code should be as readable as before.

I can provide a patch, or if anyone else wants to have a go, feel free to.
Bug: nodetool assassinate of a node that failed bootstrap may lead to IndexOutOfBoundsException
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Aug/21 03:40
Resolved: 08/Sep/21 16:56

User [Yorick|https://the-asf.slack.com/team/U026YMYS2QN] reported on ASF Slack that a node which failed to bootstrap is stuck in {{hibernate}} status and persists in {{gossipinfo}}:
{noformat}
/10.x.x.108
  generation:1625493756
  heartbeat:86
  STATUS:2:hibernate,true
  LOAD:79:96124.0
  SCHEMA:14:59adb24e-f3cd-3e02-97f0-5b395827453f
  DC:10:DC1
  RACK:12:RAC3
  RELEASE_VERSION:6:3.11.10
  INTERNAL_IP:8:10.x.x.108
  RPC_ADDRESS:5:10.x.x.108
  NET_VERSION:3:11
  HOST_ID:4:5b254d51-fc58-4ca2-856f-fe7878752131
  TOKENS:1:<hidden> {noformat}
Attempts to assassinate the node returns {{IndexOutOfBoundsException}}:
{noformat}
ERROR [GossipStage:1] 2021-08-11 09:10:03,440 CassandraDaemon.java:244 - Exception in thread Thread[GossipStage:1,5,main]
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[na:1.8.0_292]
        at java.util.ArrayList.get(ArrayList.java:435) ~[na:1.8.0_292]
        at com.google.common.collect.Iterables.get(Iterables.java:728) ~[guava-18.0.jar:na]
        at org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.makeTokenString(VersionedValue.java:156) ~[apache-cassandra-3.11.10.jar:3.11.10]
        at org.apache.cassandra.gms.VersionedValue$VersionedValueFactory.left(VersionedValue.java:178) ~[apache-cassandra-3.11.10.jar:3.11.10]
        at org.apache.cassandra.gms.Gossiper.lambda$assassinateEndpoint$1(Gossiper.java:695) ~[apache-cassandra-3.11.10.jar:3.11.10]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_292]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_292]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_292]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_292]
        at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) [apache-cassandra-3.11.10.jar:3.11.10]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292] {noformat}
Bug: Delay is not applied in the dtest to test bootstrap delay
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 03/Sep/21 15:12
Resolved: 08/Sep/21 14:24

Test [test_bootstrap_waits_for_streaming_to_finish|https://github.com/apache/cassandra-dtest/blob/trunk/bootstrap_test.py#L267] supposed to delay bootstrap of {{node2}} by using byteman :
{code:java}
node2 = new_node(cluster, byteman_port='4200')
node2.update_startup_byteman_script('./byteman/bootstrap_5s_sleep.btm')
{code}
where [byteman code|https://github.com/apache/cassandra-dtest/blob/trunk/byteman/bootstrap_5s_sleep.btm] is:
{code:java}
RULE Sleep 5s when finishing bootstrap 
CLASS org.apache.cassandra.service.StorageService 
METHOD bootstrapFinished 
AT ENTRY
IF NOT flagged("done") 
DO
    flag("done");
    Thread.sleep(5000) 
ENDRULE
{code}
However, I found that this byteman rule is not applied.

For example, I changed the rule body into:
{code:java}
...
IF TRUE
DO
    asfa;adfa;
    flag("done");
    throw new RuntimeException("Test");
    Thread.sleep(50000)
ENDRULE{code}
So my conclusion is that the delay is not applied. I haven't investigated if it is the issue in calling {{update_startup_byteman_script}}, which is implemented in CCM, or in the implementation inside CCM.

This issue might exist in other similar tests.
Bug: Check if size of object being added to RowCache and KeyCache is bigger than cache capacity
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Aug/21 16:41
Resolved: 03/Sep/21 13:47

We don't check if the size of an object being added to the RowCache/KeyCache itself exceeds the max configured size of the cache.

For instance, if a RowCache object is ~5GB due to IndexInfo objects, but the cache is configured to have a max capacity of 100MB, we will still add the 5GB object into the cache and then need to wait for the eviction thread in the cache to come around, realize we're over capacity, and remove the object from the cache.

We could check the size of the object with jamm and ensure it's smaller than the max size of the cache. If it exceeds the size of the cache don't cache it at all.
Bug: DROP permission behavior does not match documentation
Status: Open
Priority: Normal
Created: 02/Sep/21 00:14

There seems to be a mismatch between the documentation and reality with respect to the DROP permission.

Specifically, the matrix in [the documentation|https://cassandra.apache.org/doc/4.0/cassandra/cql/cql_singlefile.html#permissions] says that you can only {{DROP KEYSPACE}} if you have the {{DROP}} permission on {{ALL KEYSPACES}}; however, in testing, it appears that you can drop a keyspace if you merely have the {{DROP}} permission on that keyspace itself.

I believe this is a documentation error; for example, having {{ALTER}} permission on a keyspace is documented to let a role alter the keyspace itself in addition to altering tables within it. I don't particularly like this behavior and would prefer to have a way to grant roles the ability to alter/drop/create all objects within a keyspace without altering the keyspace itself.

Demo on a stock cassandra.yaml with the only changes being setting {{authenticator}} to {{PasswordAuthenticator}} and {{authorizer}} to {{CassandraAuthorizer}}:

{code}
$ ./bin/cqlsh -u cassandra -p cassandra
Connected to Bug Report Cluster at 127.0.0.1:9042
[cqlsh 6.0.0 | Cassandra 4.0.0 | CQL spec 3.4.5 | Native protocol v5]
Use HELP for help.
cassandra@cqlsh> CREATE KEYSPACE demo_keyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
cassandra@cqlsh> CREATE ROLE nonsuper WITH PASSWORD='nonsuper' AND LOGIN=true;
cassandra@cqlsh> GRANT DROP on KEYSPACE demo_keyspace TO nonsuper;
cassandra@cqlsh> 

 $ ./bin/cqlsh -u nonsuper -p nonsuper
Connected to Bug Report Cluster at 127.0.0.1:9042
[cqlsh 6.0.0 | Cassandra 4.0.0 | CQL spec 3.4.5 | Native protocol v5]
Use HELP for help.
nonsuper@cqlsh> DROP KEYSPACE demo_keyspace;
nonsuper@cqlsh> CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
Unauthorized: Error from server: code=2100 [Unauthorized] message="User nonsuper has no CREATE permission on <all keyspaces> or any of its parents"
nonsuper@cqlsh> DROP KEYSPACE foo;
Unauthorized: Error from server: code=2100 [Unauthorized] message="User nonsuper has no DROP permission on <keyspace foo> or any of its parents"
nonsuper@cqlsh>
{code}

Given the documented state that {{DROP KEYSPACE}} requires the {{DROP}} permission on {{ALL KEYSPACES}}, I would expect the cal to {{DROP KEYSPACE demo_keyspace}} to fail with a 2100 error.

Two things that seem like action item to me:

* The documentation should be updated to indicate that having the {{DROP}} permission on a keyspace allows you to {{DROP KEYSPACE}} and {{DROP TABLE in <keyspace>}}
* It sure would be swell if one could do something like {{GRANT DROP ON ALL TABLES IN KEYSPACE <ks> TO <role>}} in order to avoid this whole issue (the way we can already grant on {{ALL FUNCTIONS IN KEYSPACE <ks>}})...
Bug: cqlsh broken after upgrade from 3.11.10 to 3.11.11
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 01/Sep/21 16:15
Resolved: 01/Sep/21 18:49

After upgrading Cassandra from 3.11.10 to 3.11.11 on a CentOS 7 via the repository, cqlsh is no longer working, throwing the error below:

# cqlsh
Traceback (most recent call last):
 File "/usr/bin/cqlsh.py", line 169, in <module>
 from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling, cqlshhandling
ImportError: No module named cqlshlib

 

The only cqlshlib found on the file system is located at /usr/lib/python3.6/site-packages/cqlshlib, so it would appear to be an issue with the deployed python libraries and those expected by cqlsh.
Bug: Weak visibility guarantees of Accumulator can lead to failure to recognize digest mismatches
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 25/Aug/21 17:51
Resolved: 31/Aug/21 21:43

The context for this problem is largely the same as CASSANDRA-16807. The difference is that for 4.0+, CASSANDRA-16097 added an assertion to {{DigestResolver#responseMatch()}} that ensures the responses snapshot has at least one visible element (although of course only one element trivially cannot generate a mismatch and short-circuits immediately). In 3.0 and 3.11, this assertion does not exist, and when the underlying problem occurs (i.e. zero responses are visible on {{Accumulator}} when there should be 2), we can silently avoid the digest matching entirely. This seems like it would make it both impossible to do a potentially necessary full data read to resolve the correct response and prevent repair.

The fix here should be similar to the one in CASSANDRA-16807, although there might be some test infrastructure that needs porting in order to make that work.
Bug: Remove duplicate 'lib.download.sha' entries in build-resolver.xml
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Aug/21 13:18
Resolved: 30/Aug/21 19:49

As reported by [~jasonstack] [here|https://the-asf.slack.com/archives/CK23JSY2K/p1630325518017000].
Bug: Ensure off-heap Merkle trees are release on remote validation failures
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 20/Aug/21 20:32
Resolved: 27/Aug/21 20:06

Currently it's possible for off-heap merkle trees to be orphaned and we lean on the leak detector to make sure they're properly released. We should actively clean up orphaned merkle trees on remote validation failure and abort.
Bug: Java 11 support - remove the experimental flag
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 27/Aug/21 12:50
Resolved: 27/Aug/21 12:52

As per this [thread. |https://lists.apache.org/thread.html/r06a4768bae215dc19469491a1faec64dd90e1c3d4d10ed34b85ba248%40%3Cdev.cassandra.apache.org%3E]the goal of this ticket is to remove the experimental flag for Java 11 support.
Bug: Add GPG key for samt@apache.org to to dist/cassandra/KEYS
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Aug/21 13:30
Resolved: 26/Aug/21 17:02

I need to add my public key to the project's KEYS file in order to sign/publish release artifacts.

Fingerprint: F8B7FD00E05C932991A2CD6150EE103D162C5A55
Bug: High priority internode messages which exceed the large message threshold are dropped
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 23/Aug/21 17:05
Resolved: 25/Aug/21 16:47

Currently, there is an assumption that internode messages whose verb has priority P0 will always fit within a single messaging frame. While this is usually the case, on occasion it is possible that this assumption does not hold. One example is gossip messages during the startup shadow round, where in very large clusters the digest ack can contain all states for every peer. In this scenario, the respondent fails to send the ack which may lead to the shadow round and, ultimately, the startup failing.
 
We could tweak the shadow round acks to minimise the message size, but a more robust solution would be to permit high priority messages to be sent on the large messages connection when necessary.
Bug: L0 compactions never recover when thousands unleveled
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 24/Aug/21 16:11
Resolved: 24/Aug/21 18:34

While this is somewhat mitigated by incremental repair and CASSANDRA-14388, we can still find ourselves in a scenario where we have thousands of small sstables living in L0 on STCS and get into a heavily degraded state.

We should add a JMX flag to allow operators to flip the behaviour when doing STCS in L0 from picking the ‘hottest’ bucket to picking the largest one - this should allow us to get down to reasonable numbers of sstables in L0 quicker in this pathological case. If this proves to work particularly well we could consider making this the default in a future version.
Bug: Create config.yml.MIDRES for 3.0 and 3.11
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Jul/21 22:21
Resolved: 20/Jul/21 16:41

[config.yml.MIDRES|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml.MIDRES] was created for 4.0 some time ago but not for 3.0 and 3.11.

This task is to facilitate that backport
Bug: CircleCI MIDRES might be missing some changes in 3.11
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Aug/21 17:34
Resolved: 17/Aug/21 14:24

I've noticed that when we run {{.circleci/generate.sh}} in 3.11 the file {{config.yml.MIDRES}} is modified, when I'd say it shouldn't be changed. If I'm right running that script should be noop when there hasn't been changes in {{config-2_1.yml}}, and indeed the file isn't modified in the other branches.

Not sure yet, but I think that this might be the result of not having generated a new {{config.yml.MIDRES}} with the script when merging 3.0 into 3.11 during CASSANDRA-16804.
Bug: Following documentation for setting flake8 hook in dtests breaks environment
Status: Open
Priority: Normal
Created: 12/Aug/21 12:19

The [contributing page of cassandra-test|https://github.com/apache/cassandra-dtest/blob/trunk/CONTRIBUTING.md] contains instructions to setup git hook to run flake8 on commit:

 
{code:java}
flake8 --install-hook
git config flake8.strict true
git config flake8.ignore E501,F811,F812,F821,F822,F823,F831,F841,N8,C9
{code}
I insulted flake8 and followed the instruction and after I committed changes to {{conftest.py}}, I've got many E501 errors:

 

 
{code:java}
conftest.py:21:80: E501 line too long (85 > 79 characters)
conftest.py:31:80: E501 line too long (98 > 79 characters)
conftest.py:32:80: E501 line too long (96 > 79 characters)
conftest.py:33:80: E501 line too long (96 > 79 characters)
conftest.py:34:80: E501 line too long (96 > 79 characters)
conftest.py:35:80: E501 line too long (90 > 79 characters)
conftest.py:39:80: E501 line too long (85 > 79 characters)
...{code}
This means that the ignore configuration is not used, and I don't see if there is any workaround except removing the commit hook manually.

According to flake8 documentation the instructions are applicable to an old version of flake8, e.g, [2.6.0|https://flake8.pycqa.org/en/2.6.0/vcs.html]. This page doesn't exist in the latest version of flake8. The latest documentation has [instructions|https://flake8.pycqa.org/en/3.9.2/user/using-hooks.html?highlight=hooks], which suggests to use [pre-commit package|https://pre-commit.com/], and doesn't recommend to use the old way, since it is not tested well.

Will it be good idea to update instructions?
Bug: Support for JDK 11 in Apache Cassandra 4.0
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 11/Aug/21 21:21
Resolved: 11/Aug/21 21:36

I understand that the JAVA 11 is listed as "Experimental" in the new version of Cassandra 4.0, but from what I have been reading in the user mailing list is that soon the "Experimental" word will be removed and the support for JAVA 11 for production use will officially be announced. 
If that is true what is the timeline for it and if that's not the case, is there any plan to support JAVA 11 before a new minor version is release ?
Bug: CassandraVersion compareTo does not order SNAPSHOT versions properly
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Jul/21 09:57
Resolved: 12/Jul/21 11:16

{{CassandraVersion::compareTo}} does not order {{SNAPSHOT}} versions properly in several scenario.
It return wrong results for the 2 following usecases:

1. {{new CassandraVersion("4.0-rc2").compareTo(("4.0.0-SNAPSHOT")}}
2. {{new CassandraVersion("4.0-rc2").compareTo(("4.0-rc2-SNAPSHOT")}}