av9ash/GitBugs huggingface dataset:
Bug: Importing cassandra driver causes modern pytest to fail in certain circumstances
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 27/Mar/23 17:11
Resolved: 27/Mar/23 17:22

When collecting on running test without exclusion on Pytest test name, importing cassandra driver causes a very strangely looking error:

 
{quote}# from cassandra.cluster import Cluster
# cassandra/cluster.py:48: in init cassandra.cluster
# ???
# cassandra/connection.py:40: in init cassandra.connection
# ???
# cassandra/protocol.py:698: in genexpr
# ???
# cassandra/protocol.py:698: in genexpr
# ???
# E KeyError: '@py_builtins'{quote}
This happened for Apache Airflow when we wanted to enable pytest collection to include all files, rather than only "test_*.py" files, because we found that there were at least few modules in airlfow that did not start with "test". 

It turned out that the culprit was Assert Rewrite done by pytest that tried to rewrite "type_codes.py" module of Cassandra.

One of the ways to avoid the problem is to disable assert rewrite by "–assert=plain".  However then asserts are not as useful. Better solution (PR is coming) is to add PYTEST_DONT_REWRITE in the docstring:

https://docs.pytest.org/en/latest/how-to/assert.html#disabling-assert-rewriting

For now we are mitigating that issue in Airflow by manually patching cassandra code, however better solution will be to add the PYTEST_DONT_REWRITE directive directly in Cassandra's Python driver's repository.

Fix is coming.
Bug: Redhat 40x repo signature is not valid
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Mar/23 12:14
Resolved: 27/Mar/23 15:05

Looks like part of the packaging upload troubles we had with the 4.0.8 release:

{noformat}
gpg --verify repomd.xml.asc repomd.xml
gpg: Signature made Wed 19 Oct 2022 20:19:43 AEDT
gpg:                using RSA key A4C465FEA0C552561A392A61E91335D77E3E87CB
gpg: BAD signature from "Michael Semb Wever <mick@thelastpickle.com>" [unknown]
{noformat}
Bug: Separate release scripts into discrete steps
Status: Open
Priority: Normal
Created: 27/Mar/23 11:57

Currently the release scripts are two monolithic scripts: prepare_releash.sh, and finish_release.sh.  If any step in either of the scripts fails, you are left with a state that requires manual resolution, generally by script dissection.  It would be easier to handle these scenarios if each step were defined in discrete files that the prepare/finish wrapper then called.
Bug: hinted_handoff_enabled=false is not honored
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/Mar/23 23:14
Resolved: 24/Mar/23 16:19

I've had some dtests with disabled hints failing.

After investigation it seems that CASSANDRA-17164 moved hint submission on timeout from [RequestCallbacks.onExpired|https://github.com/apache/cassandra/commit/d2923275e360a1ee9db498e748c269f701bb3a8b#diff-b73c13ea8cae91a215495917fe5e90d55c9f4a283f9e053110992bc9a60004b8L176] to [AbstractWriteResponseHandler.onFailure|https://github.com/apache/cassandra/commit/d2923275e360a1ee9db498e748c269f701bb3a8b#diff-3b202de0d077638bede7bf4076a15eb4d483b717f955f11e743efb8d27c6eb1dR285], but it no longer checks if {{CallbackInfo.shouldHint}} which checks for {{StorageProxy.shouldHint}} which ultimately checks if {{{}hinted_handoff_enabled=true{}}}.

This could cause some tests which expect hints to be disabled to fail intermittently.
Bug: Test failure: cqlsh_tests.test_cqlsh.TestCqlsh.test_list_queries
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Mar/23 15:14
Resolved: 24/Mar/23 15:24

Looks like this needs to be updated for DDM: https://app.circleci.com/pipelines/github/driftx/cassandra/930/workflows/0dc6fa5b-65eb-4ea8-9238-2d78b889d9bc/jobs/16238/tests
Bug: Make nodetool describecluster print all configuration parameters of topology strategies
Status: Triage Needed
Priority: Normal
Created: 22/Mar/23 14:20

If there is a custom implementation of topology strategy, that strategy can accept more options, not only "class" and "replication factors for datacenters".

I can have a replication strategy like this:

{code}
CREATE KEYSPACE ks WITH replication = {'class': 'CustomTopologyStrategy', 'datacenter1': '1', 'some_other_option': 'myoptionvalue'};
{code}

There is nothing which prevents users to put whatever options they want into a strategy as it is a map. Currently, we treat all options except "class" as names of datacenters and the current code is pretty much opinionated about that (KeyspaceParams class).

However, when I create a keyspace like this:

{code}
CREATE KEYSPACE ks WITH replication = {'class': 'CustomTopologyStrategy', 'datacenter1': '1'};
{code}

nodetool describecluster command prints this:

{code}
Keyspaces:
        system_schema -> Replication class: LocalStrategy {}
        system -> Replication class: LocalStrategy {}
        test -> Replication class: SimpleStrategy {replication_factor=1}
        system_auth -> Replication class: SimpleStrategy {replication_factor=1}
        system_distributed -> Replication class: SimpleStrategy {replication_factor=3}
        system_traces -> Replication class: SimpleStrategy {replication_factor=2}
        testks -> Replication class: CustomTopologyStrategy {datacenter1=1}
{code}

But it should print (or whatever the default is for some_other_option)

{code}
testks -> Replication class: CustomTopologyStrategy {datacenter1=1, some_other_option=somedefaultvalue}
{code}

The reason this is happening is that params are saved to system_schema.keyspaces into "replication" column and if I do not specify all parameters upon keyspace creation, they will not be persisted so describing it via nodetool will print only that.

I want to see all options, even I have not set them explicitly in CQL CREATE KEYSPACE ...

Another option would be to modify CQL like this: 

{code}
CREATE KEYSPACE ks WITH replication = {'class': 'CustomTopologyStrategy', 'datacenter1': '1'} AND options = {'some_other_option': 'myoptionvalue'}
{code}

I would start with simply adding other options to that output. We can still modify it later to have "AND options ..." in CQL, it is OK to expand CQL like that if we decide. This ticket does not prevent that. Adding of "options" is more complicated. We would need to have another column for that in system_schema.keyspaces and (Java) drivers would need to be accommodated too. Too much work ...

PR here: https://github.com/apache/cassandra/pull/2239/files
Bug: Test Failure: repair_tests.deprecated_repair_test.TestDeprecatedRepairNotifications.test_deprecated_repair_error_notification
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Jan/23 09:05
Resolved: 22/Mar/23 10:51

Failing since https://ci-cassandra.apache.org/job/Cassandra-3.0/313/testReport/dtest.repair_tests.deprecated_repair_test/TestDeprecatedRepairNotifications which is https://github.com/apache/cassandra/commit/13d495aa7d5b7a7c121fcc9e105f79107c5c2a1c from CASSANDRA-17254 

but can be reproduced earlier in circleci, e.g. https://app.circleci.com/pipelines/github/michaelsembwever/cassandra/64/workflows/f99abc05-aa9b-42de-a424-44ab7c247e76/jobs/3175 

Only two commits are in this time-range (10th to 30th November) in cassandra-dtest
 - https://github.com/apache/cassandra-dtest/commit/7f2b8eda5c52fb6f637aa7166e2d48cd34a64eec (CASSANDRA-17679)
 - https://github.com/apache/cassandra-dtest/commit/e0d3cc90558a17b63634d15ee6df339ceb87b225 (CASSANDRA-15402)
Bug: Add package-info files for each package
Status: Triage Needed
Priority: Normal
Created: 21/Mar/23 03:21

- To make the whole codebase more readable(especially for C* beginner), add package-info files for each package to describe what each package did may be a good choice.
- Look at an example from Netty:
https://github.com/netty/netty/blob/4.1/codec-redis/src/main/java/io/netty/handler/codec/redis/package-info.java
https://github.com/netty/netty/blob/4.1/handler-proxy/src/main/java/io/netty/handler/proxy/package-info.java
https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/package-info.java
Bug: Invalid result when a non existent row is assigned to a variable
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Feb/23 14:50
Resolved: 16/Feb/23 18:38

{noformat}
cqlsh:ks> CREATE TABLE tbl3 (k INT, c INT, v INT, PRIMARY KEY (k, c));
cqlsh:ks> 
BEGIN TRANSACTION 
  LET row = (SELECT * FROM tbl3 WHERE k=1 LIMIT 1); 
  SELECT row.k, row.c, row.v; 
  IF row.c > 10 THEN 
    INSERT INTO ks.tbl3 (k, c, v) VALUES (1, 10, 110); 
  END IF
COMMIT TRANSACTION;

 row.k | row.c | row.v
-------+-------+-------
     1 |  null |  null

cqlsh:ks> SELECT * FROM tbl3;

 k | c | v
---+---+---

(0 rows)
{noformat}

as discussed on Slack, the transaction should return:
{noformat}
 row.k | row.c | row.v
-------+-------+-------
  null |  null |  null
{noformat}
Bug: Race in sstable tracking can cause sstables to never be compacted
Status: Triage Needed
Priority: Normal
Created: 17/Mar/23 14:30

The sstable tracking by the compaction strategies permits a race between an sstable is added and removed that can cause a removed sstable to remain in the tracked list (when the add notification is received after the removal).

This can cause reference leaks, because such {{SSTableReader}} s remain in the list forever, but if a strategy does not filter its sstable list through the live list (e.g. LCS in some code paths) it can also cause sstables that end up being selected to compact with these zombies to never compact, without even triggering the "Could not acquire references for compacting SSTables..." message in some cases (e.g. as LCS will not select the same compaction immediately).
Bug: Drop support for sstable formats before `na`
Status: Triage Needed
Priority: Normal
Created: 09/Mar/23 13:04

From 5.0 sstable formats before {{\-na\-}} are unsupported.

This patch cleans up code related to the older formats.
https://github.com/apache/cassandra/compare/trunk...thelastpickle:cassandra:mck/18312/trunk 

Raises the question whether backward compatibility to the previous major is defined by the Cassandra version or by the internal component version (sstable format).

Also raises the question on whether downstream forks are wanting to support such upgrade paths, e.g. offline, and to what versions the {{sstableupgrade}} tool supports. This also touches on recent discussions around [Downgradability|https://lists.apache.org/thread/tcp339k5ph8ql35wxr085to4qgp8tpg7].
Bug: Accord simulator fails with "available partitions are empty!"
Status: Open
Priority: Normal
Created: 16/Mar/23 21:14

With the change to allow accord to run with multiple partitions in simulator we can hit an edge case where all partitions are used in queries, so the current query isn’t able to find a partition to use, causing this error

{code}
junit.framework.AssertionFailedError: available partitions are empty!
	at org.apache.cassandra.simulator.paxos.AbstractPairOfSequencesPaxosSimulation.consume(AbstractPairOfSequencesPaxosSimulation.java:252)
	at org.apache.cassandra.simulator.paxos.AbstractPairOfSequencesPaxosSimulation.access$000(AbstractPairOfSequencesPaxosSimulation.java:72)
	at org.apache.cassandra.simulator.paxos.AbstractPairOfSequencesPaxosSimulation$2.get(AbstractPairOfSequencesPaxosSimulation.java:211)
	at org.apache.cassandra.simulator.paxos.AbstractPairOfSequencesPaxosSimulation$2.get(AbstractPairOfSequencesPaxosSimulation.java:207)
	at org.apache.cassandra.simulator.Actions.next(Actions.java:145)
	at org.apache.cassandra.simulator.Actions.lambda$streamNextSupplier$3(Actions.java:154)
	at org.apache.cassandra.simulator.Actions$LambdaAction.performSimple(Actions.java:63)
	at org.apache.cassandra.simulator.Action.performAndRegister(Action.java:468)
	at org.apache.cassandra.simulator.Action.perform(Action.java:486)
	at org.apache.cassandra.simulator.ActionSchedule.next(ActionSchedule.java:378)
	at org.apache.cassandra.simulator.paxos.PaxosSimulation$2.next(PaxosSimulation.java:255)
	at org.apache.cassandra.simulator.paxos.PaxosSimulation.run(PaxosSimulation.java:227)
	at org.apache.cassandra.simulator.paxos.AbstractPairOfSequencesPaxosSimulation.run(AbstractPairOfSequencesPaxosSimulation.java:295)
	at org.apache.cassandra.simulator.paxos.PairOfSequencesAccordSimulation.run(PairOfSequencesAccordSimulation.java:62)
	at org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:374)
	at org.apache.cassandra.simulator.paxos.AccordSimulationRunner$Run.run(AccordSimulationRunner.java:39)
	at org.apache.cassandra.simulator.paxos.AccordSimulationRunner$Run.run(AccordSimulationRunner.java:30)
	at org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:355)
	at org.apache.cassandra.simulator.paxos.AccordSimulationRunner.main(AccordSimulationRunner.java:76)
	at org.apache.cassandra.simulator.test.ShortAccordSimulationTest.simulationTest(ShortAccordSimulationTest.java:32)
{code}

See https://app.circleci.com/pipelines/github/dcapwell/cassandra/1947/workflows/05ee1183-36a0-41de-a798-e49b4e1ec413/jobs/19052
Bug: Fix AIOOBE and improve validation messages for transaction statements
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/Mar/23 13:25
Resolved: 16/Mar/23 17:45

Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown from asCql method when validation transaction statement. In addition, asCql does not return precisely the query user entered so the whole error message can be misleading.
Bug: Fix link to .tar.gz download in documentation
Status: Open
Priority: Normal
Created: 14/Mar/23 11:45

On [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball] the reader is instructed to:
 
{{curl -OL [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}}

However, that mirror no longer exists.

It would be nicer to use a URL such as [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz], but that requires having the current Cassandra version available as a variable to asciidoc and inserting in into a code snippet, which I'm not sure is easy to do.

Alternatively we could point people to [https://cassandra.apache.org/_/download.html] instead of showing a curl command.
Bug: Change "Cleaned up" log message slightly
Status: Triage Needed
Priority: Normal
Created: 15/Mar/23 09:54

Might seem like a very small change, but it leads me to confusion each and every time. 🤣

I propose to change current message like:



{{Cleaned up to /var/db/cassandra/data/…/me-19423-big-Data.db. 571.512GiB to 502.648GiB (~87% of original) for 11,708,552 keys. Time: 62,321,057 ms.}}



To only 2 decimal places, to disambiguate the fact that the "." is a decimal separator and not a thousands separator.

{{Cleaned up to /var/db/cassandra/data/…/me-19423-big-Data.db. 571.51GiB to 502.65GiB (~87% of original) for 11,708,552 keys. Time: 62,321,057 ms.}}

I guess this happens less in the USA or countries where "." is always and only a decimal separator, but e.g. in Italy we use numbers usually like this "1.000,345" and thus, while accustomed to many values in the "international" format the period is a bit overloaded in my brain and I always am like "is that 571 and a half GiB, again?"

Changing to two digits would help in that regard and, I think, only removes extra detail which was kinda overkill in the first place.
Bug: Enhance nodetool tablehistograms with ignore options
Status: Open
Priority: Normal
Created: 03/Mar/23 08:34

Add some options for nodetool tablehistograms to filter the output 
of System tables and data tables.
As we know if we nodetool tablehistograms all tables will be output ,but system table 's number are too much.
Add options to filter only output data tables or system tables, together with the already exist ks.tb lists, the output options are just fine.

It seems that tablestats got the -i option that can ignore keyspace, just add this first, and then to see wether this can meet our needs
Bug: Debian package repository misconfiguration
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Mar/23 08:54
Resolved: 14/Mar/23 12:38

Debian apt is failing on current jfrog repository access for 40x releases with:

{code}
W: Conflicting distribution: https://debian.cassandra.apache.org 40x InRelease (expected 40x but got 40)

E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its 'Codename' value from '40x' to '40'

N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
{code}

This is caused by the typo in [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] containing
{code}
Codename: 40
{code}
but it is expected to be
{code}
Codename: 40x
{code}
Bug: CEP-15 (C*) No commandsForKey in context for key PartitionKey
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Feb/23 22:12
Resolved: 10/Mar/23 21:51

See https://app.circleci.com/pipelines/github/maedhroz/cassandra/688/workflows/84cb0f28-d010-43c6-97cb-55af4a31a3ce/jobs/6554/tests#failed-test-0

{code}
ERROR [CommandStore[25]:1] node1 CS:[25] OP:0xe17e624f 2023-02-02 19:46:44,764 Operation AsyncOperation{RUNNING}-0xe17e624f failed
java.lang.IllegalArgumentException: No commandsForKey in context for key PartitionKey{tableId=dbb35709-2b11-38b3-9fcb-9a197512080e, key=DecoratedKey(5414964737888519087, 242cad05)}
	at org.apache.cassandra.service.accord.AccordCommandStore.getCommandsForKeyInternal(AccordCommandStore.java:442)
	at org.apache.cassandra.service.accord.AccordCommandStore$SafeAccordCommandStore.commandsForKey(AccordCommandStore.java:204)
	at org.apache.cassandra.service.accord.AccordCommandStore$SafeAccordCommandStore.lambda$register$1(AccordCommandStore.java:190)
	at accord.primitives.Routables$Helper.foldl(Routables.java:282)
	at accord.primitives.Routables.foldl(Routables.java:134)
	at org.apache.cassandra.service.accord.AccordCommandStore$SafeAccordCommandStore.register(AccordCommandStore.java:190)
	at accord.local.Command.set(Command.java:1015)
	at accord.local.Command.commit(Command.java:377)
	at accord.messages.Commit.apply$unsync(Commit.java:152)
	at accord.messages.Commit.apply(Commit.java)
	at accord.messages.Commit.apply(Commit.java:40)
	at org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:239)
	at org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:226)
	at org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
	at org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
	at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
	at org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
	at org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)
{code}
Bug: CEP-15: (C*) Accord adds a accord.messages.Defer which isn't supported for persisting, causing the operation to fail
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Feb/23 22:13
Resolved: 10/Mar/23 21:51

See https://app.circleci.com/pipelines/github/maedhroz/cassandra/688/workflows/84cb0f28-d010-43c6-97cb-55af4a31a3ce/jobs/6554/tests#failed-test-0

{code}
ERROR [CommandStore[25]:1] node1 CS:[25] OP:0xf01a5a4b 2023-02-02 19:46:44,701 Operation AsyncOperation{RUNNING}-0xf01a5a4b failed
java.lang.RuntimeException: Unhandled non-transient listener: accord.messages.Defer@3d36718e
	at org.apache.cassandra.service.accord.AccordCommand.maybeWrapListener(AccordCommand.java:711)
	at org.apache.cassandra.service.accord.AccordCommand.addListener(AccordCommand.java:717)
	at accord.messages.Defer.add(Defer.java:67)
	at accord.messages.Commit.apply$unsync(Commit.java:163)
	at accord.messages.Commit.apply(Commit.java)
	at accord.messages.Commit.apply(Commit.java:40)
	at org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:239)
	at org.apache.cassandra.service.accord.async.AsyncOperation$ForFunction.apply(AsyncOperation.java:226)
	at org.apache.cassandra.service.accord.async.AsyncOperation.runInternal(AsyncOperation.java:154)
	at org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:194)
	at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
	at org.apache.cassandra.concurrent.SyncFutureTask.run(SyncFutureTask.java:68)
	at org.apache.cassandra.simulator.systems.InterceptingExecutor$AbstractSingleThreadedExecutorPlus.lambda$new$0(InterceptingExecutor.java:584)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)
{code}
Bug: Store min and max partition key in sstable stats metadata rather than in some index component
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 26/Jan/23 12:59
Resolved: 09/Mar/23 07:24

Currently min and max partition key is stored in the index summary.

Firstly, that informat better fits stats metadata as there are other similar statistics (like min and max clusterings)

Secondly, opening index summary is costly. Though, index summary and the index itself are loaded upon opening an sstable just because we need to read min and max partition keys. Min and max partition keys need to be know for an sstable so that when some data are queried, we can select the sstable which may contain that data - that is, whether the queried partition is included in the min/max key range of the sstable.

With the proposed solution, we could postpone loading index components to the time when the data from such sstable is really requested. It will be enough to read lightweight stats metadata as it will be sufficient to know everything about data ranges included in that sstable. It will also let to save memory used by those components until data from those sstables are requested.