av9ash/GitBugs huggingface dataset:
Bug: Improvements in circle CI default config
Status: Open
Priority: Normal
Created: 02/Apr/20 19:52

I have been looking at and played around with the [default CircleCI config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml], a few comments/questions regarding the following topics:
 * Python dtests do not run successfully (200-300 failures) on {{medium}} instances, they seem to only run with small flaky failures on {{large}} instances or higher
 * Python Upgrade tests:
 ** Do not seem to run without many failures on any instance types / any parallelism setting
 ** Do not seem to parallelize well, it seems each container is going to download multiple C* versions
 ** Additionally it seems the configuration is not up to date, as currently we get errors because {{JAVA8_HOME}} is not set
 * Unit tests do not seem to parallelize optimally, number of test runners do not reflect the available CPUs on the container. Ideally if # of runners == # of CPUs, build time is improved, on any type of instances.
 ** For instance when using the current configuration, running on medium instances, build will use 1 junit test runner, but 2 CPUs are available. If using 2 runners, the build time is reduced from 19min (at the current main config of parallelism=4) to 12min.
 * There are some typos in the file, some dtests say "Run Unit Tests" but they are JVM dtests (see [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077], [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])

So some ways to process these would be:
 * Do the Python dtests run successfully for anyone on {{medium}} instances? If not, would it make sense to bump them to {{large}} so that they can be run successfully?
 * Does anybody ever run the python upgrade tests on CircleCI and what is the configuration that makes it work?
 * Would it make sense to either hardcode the number of test runners in the unit tests with `-Dtest.runners` in the config file to reflect the number of CPUs on the instances, or change the build so that it is able to detect the appropriate number of core available automatically?

Additionally, it seems this default config file (config.yml) is not as well maintained as the [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml] (+its lowres/highres) version in the same folder (from CASSANDRA-14806). What is the reasoning for maintaining these 2 versions of the build? Could the better maintained version be used as the default? We could generate a lowres version of the new config-2_1.yml, and rename it {{config.yml}} so that it gets picked up by CircleCI automatically instead of the current default.
Bug: Simplify the documentation
Status: Triage Needed
Priority: Normal
Created: 13/Apr/20 20:08

Can you fix the documentation, [https://github.com/apache/cassandra/blob/trunk/doc/source/new/streaming.rst]

In the zero copying section, it says "Pre-4.0, during streaming Cassandra reifies the SSTables into objects." 

The word reifies makes no sense. Its not easy to understand and as end user if you can update it with some simple English would help. 

I think it "It serializes the data, the meta data needs to be re-built on the receiving node" but a better and simple explanation would help.
Bug: Table of content links for  https://cassandra.apache.org/doc/old/CQL-3.0.html are broken
Status: Triage Needed
Priority: Normal
Created: 11/Apr/20 19:38

Table of content links of the https://cassandra.apache.org/doc/old/CQL-3.0.html page is redirecting to non-existing [https://cassandra.apache.org/doc/old/CQL.html#createKeyspaceStmt] page instead of to itself.

Expected behavior:

Table of content link "https://cassandra.apache.org/doc/old/CQL.html#createKeyspaceStmt" should become "https://cassandra.apache.org/doc/old/CQL-3.0.html#createKeyspaceStmt"
Bug: Cassandra commitlog dump 
Status: Open
Priority: Low
Created: 15/May/20 04:15

when we do backup using commitlog or other cases using commitlog ,we may want to know the data contains in commitlog . For other nosql like hbase, they also support the tool to show the contents in WAL log(Hlog). so I think a tool like sstabledump is needed for commitlog . 

For me , I think the input is the commitlog's absolute path, the out put format may be json depends on the commitlog version.
Bug: DOC - Review Download page
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 07/Apr/20 09:56
Resolved: 20/Jun/23 19:25

h2. Background
With updates to the [Installing Cassandra|http://cassandra.apache.org/doc/latest/getting_started/installing.html] page in CASSANDRA-15466, there's an opportunity to review and tidy up the [Downloading Cassandra|http://cassandra.apache.org/download/] page.

h2. Scope
Retire sections of the document relating to C* installation and direct users to the Installation page instead so we don't have to main installation instructions in multiple places.
Bug: Document how to run Cassandra on Windows
Status: Open
Priority: Low
Created: 19/Jun/20 13:05

The "Getting Started" section on the website only has instructions about installing Cassandra on Linux.

It would help us drive Cassandra adoption if we had instructions for developers that want to run Cassandra on their Windows development environment.

We should include instructions on how to use the existing powershell scripts to run Cassandra on native Windows but the docs should recommend users to prefer using WSL2/Docker before attempting to run it natively in my opinion.
Bug: Improve messaging on indexing frozen collections
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 30/Jun/20 00:48
Resolved: 14/Jul/20 16:10

When attempting to create an index on a frozen collection the error message produced can be improved to provide more detail about the problem and possible workarounds. Currently, a user will receive a message indicating "...Frozen collections only support full() indexes" which is not immediately clear for users new to Cassandra indexing and datatype compatibility.

Here is an example:
{code:java}
cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};

cqlsh> CREATE TABLE test.mytable ( id int primary key, addresses frozen<set<text>> );

cqlsh> CREATE INDEX mytable_addresses_idx on test.mytable (addresses);
 InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot create values() index on frozen column addresses. Frozen collections only support full() indexes"{code}
 

I'm proposing possibly enhancing the messaging to something like this.
{quote}Cannot create values() index on frozen column addresses. Frozen collections only support indexes on the entire data structure due to immutability constraints of being frozen, wrap your frozen column with the full() target type to index properly.
{quote}
Bug: Prevent repair from overrunning compaction
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 16/May/20 03:33
Resolved: 24/Aug/20 07:13

Repair can run and stream faster than a host can compact. At some point, if a host is sufficiently out of sync, or compaction is especially expensive, it makes sense to intentionally block repair so that compaction can catch up
Bug: DOC - Add known installation issues to the Troubleshooting page
Status: Open
Priority: Low
Created: 07/Apr/20 10:02

h2. Background
With updates to the [Installing Cassandra|http://cassandra.apache.org/doc/latest/getting_started/installing.html] page in CASSANDRA-15466, we should add known installation issues to the [Troubleshooting|http://cassandra.apache.org/doc/latest/troubleshooting/index.html] page.

h2. Topics
* GPG error unavailable public key with Debian installation
* C* service disabled on systemd distributions
* C* service does not automatically start on reboot
Bug: system_schema keyspace complain of schema mismatch during upgrade
Status: Resolved
Priority: Low
Resolution: Cannot Reproduce
Created: 12/May/20 14:48
Resolved: 31/Aug/21 19:04

When upgrading from 3.11.4 to 3.11.6, we got the following error:

{code:Plain Text}
ERROR [MessagingService-Incoming-/10.20.11.59] 2020-05-07 13:53:52,627 CassandraDaemon.java:228 - Exception in thread Thread[MessagingService-Incoming-/10.20.11.59,5,main]
java.lang.RuntimeException: Unknown column kind during deserialization
    at org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:464) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:419) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:675) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:658) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180) ~[apache-cassandra-3.11.4.jar:3.11.4]
    at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94) ~[apache-cassandra-3.11.4.jar:3.11.4]
{code}

I've noticed that system_schema.dropped_columns has a new column called "kind".
No issues arise from this error message, and the error disappeared after upgrading all nodes. But it still caused concerns due to the ERROR logging level, although "nodetool describecluster" reported only one schema version.

It makes sense for the system keyspaces to not be included for the "describecluster" schema version check, but it seems to me that these internal schema mismatches should be ignored if the versions are different between the nodes.
Bug: test org.apache.cassandra.config.DatabaseDescriptorRefTest#testDatabaseDescriptorRef fail when run with intellij ide
Status: Resolved
Priority: Low
Resolution: Cannot Reproduce
Created: 02/May/20 03:52
Resolved: 08/Jul/21 16:25

I run org.apache.cassandra.config.DatabaseDescriptorRefTest#testDatabaseDescriptorRef using ide intellij with mac

and it fails with following message:

    Thread #13: AsyncAppender-Worker-ASYNC
    Thread #12: Attach Listener
    Thread #10: logback-1
    Thread #5: Monitor Ctrl-Break
    Thread #4: Signal Dispatcher
    Thread #3: Finalizer
    Thread #2: Reference Handler
    Thread #1: main

    java.lang.AssertionError: thread started in clientInitialization 
    Expected :5
    Actual :8
Bug: cqlsh ALTER KEYSPACE completion show virtual table keyspaces
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 09/Apr/20 09:23
Resolved: 09/Apr/20 14:56

{code}
cqlsh> ALTER KEYSPACE
system_traces         system_virtual_schema system_auth           system_views          system_distributed
{code}

The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual tables keyspaces that cannot be altered and by consequence they should not appear in this list.
Bug: Add a debug line informing of building summary during sstable open
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 23/Apr/20 15:35
Resolved: 06/May/20 21:50

We've had a situation where our backup-restore flow did not include summary.db component for a large file with a lot of keys. This caused Cassandra on the restored node to take a long time to boot (while it was building the summary).

The last message was about opening a few sstables. It would be nice to also include a debug line about building summary, which then operators can use to check if a missing summary file is expected.
Bug: WEBSITE - October 2021 blog post for What the Future Holds for Apache Cassandra and Apache Cassandra Changelog #10 formatting errors fix
Status: Resolved
Priority: Urgent
Resolution: Fixed
Created: 23/Oct/21 01:09
Resolved: 26/Oct/21 21:16

This ticket is to capture the work associated with publishing the October 2021 blog post for "What the Future Holds for Apache Cassandra."

If this blog cannot be published by the October 26 publish date, please be sure to correct the date before going live.

Also added formatting errors fix for Apache Cassandra Changelog #10

We can close this once the blog post and fixes have gone live on the website.
Bug: WEBSITE - November 2021 blog post titled Inside Cassandra: an interview with Marcel Birkner at Instana
Status: Resolved
Priority: Urgent
Resolution: Fixed
Created: 17/Nov/21 00:16
Resolved: 17/Nov/21 09:56

This ticket is to capture the work associated with publishing the November 2021 blog post for "Inside Cassandra: an interview with Marcel Birkner at Instana."

If this blog cannot be published by the *November 17 publish date*, please be sure to *correct the date before going live*.
Bug: WEBSITE - December 2021 blog post titled "Using Arithmetic Operators in Cassandra 4.0"
Status: Resolved
Priority: Urgent
Resolution: Fixed
Created: 17/Dec/21 20:40
Resolved: 21/Dec/21 09:33

This ticket is to capture the work associated with publishing the December 2021 blog post for "Using Arithmetic Operators in Cassandra 4.0"

If this blog cannot be published by the *December 21, 2021 publish date*, please *correct the date before going live* (on blog.adoc and in the blog).
Bug: Dropping a column can break queries until the schema is fully propagated (TAKE 2)
Status: Resolved
Priority: Normal
Resolution: Abandoned
Created: 18/Oct/21 17:05
Resolved: 29/May/24 09:02

With a table like:
{code}
CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
{code}

and we drop {{v2}}, we get this exception on the replicas which haven't seen the schema change:
{code}
ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread Thread[SharedPool-Worker-1,5,node2]
java.lang.IllegalStateException: [ColumnDefinition{name=v1, type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, ColumnDefinition{name=v3, type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] is not a subset of [v1 v3]
	at org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) ~[main/:na]
	at org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) ~[main/:na]
	at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184) ~[main/:na]
	at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114) ~[main/:na]
	at org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102) ~[main/:na]
	at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132) ~[main/:na]
	at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87) ~[main/:na]
...
{code}

Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}

CASSANDRA-15899 tried to fix the problem when columns are dropped as well as when columns are added. Unfortunately the fix introduced an issue and had to be reverted in CASSANDRA-16735. 

If the scenario for ADDED columns is tricky, the original scenario for DROPPED columns can  be solved in a safe way at the {{ColumnFilter}} level. By consequence, I think that we should at least solve that scenario.

[~bdeggleston], [~samt], [~ifesdjeen] does my proposal makes sense to you?
Bug: Allow to filter using LIKE predicates
Status: Review In Progress
Priority: Normal
Created: 08/Dec/21 15:08

{{LIKE}} predicates can only be used with the SASI indices. In several usecases (e.g. querying the {{settings}} virtual table) it makes sense to support them for filtering.

+Additional information for newcomers:+

There are some checks in the {{StatementRestrictions}} constructor and on {{LikeRestriction}} that need to be removed for allowing filtering using LIKE on clustering and regular columns.
For filtering on partition columns the {{needFiltering}} methods in {{PartitionKeySingleRestrictionSet}} will need to be modified to return true when LIKE predicate are used.
The unit tests should go in {{SelectTest}}.
Bug: JNA 5.6.0 does not support arm64
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 01/Oct/21 09:39
Resolved: 07/Jan/22 19:07

Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native binding into the C library.

JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing cassandra to fail on bootstrap.
 Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the dependency be acceptable ?
Bug: Refactor schema management to allow for schema source pluggability
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Oct/21 13:38
Resolved: 11/Mar/22 10:06

The idea is decompose `Schema` into separate entities responsible for different things. In particular extract what is related to schema storage and synchronization into a separate class so that it is possible to create an extension point there and store schema in a different way than `system_schema` keyspace, for example in etcd. 

This would also simplify the logic and reduce the number of special cases, make all the things more testable and the logic of internal classes encapsulated.
Bug: Migrate thresholds for logged/unlogged batches to guardrails
Status: Triage Needed
Priority: Normal
Created: 06/Dec/21 17:09

Migrate the existing thresholds for logged/unlogged batches:
{code}
batch_size_warn_threshold_in_kb: 5
batch_size_fail_threshold_in_kb: 50
unlogged_batch_across_partitions_warn_threshold: 10
{code}
To a pair of new guardrails under the guardrails section, for example:
{code}
guardrails:
    ...
    batch_size:
        warn_threshold_in_kb: 5
        abort_threshold_in_kb: 50
    unlogged_batch_across_partitions:
        warn_threshold: 10
{code}
At the moment these thresholds are enabled by default, while guardrails are globally disabled by default, so we should consider whether we want to globally enable guardrails, or change these thresholds to disabled by default, or don't migrate the thresholds.
Bug: Refactor structure of caching metrics and expose auth cache metrics via JMX
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Oct/21 21:01
Resolved: 08/Feb/24 09:56

Unlike to other caches (row, key, counter), Auth Caches lack some monitoring capabilities. Here are a few particular changes to get this inequity fixed:
 # Add auth caches to _system_views.caches_ VT
 # Expose auth caches metrics via JMX
 # Add auth caches details to _nodetool info_
Bug: Update docs for CASSANDRA-17132
Status: Open
Priority: Normal
Created: 10/Nov/21 23:35

Some cleaning of old parameters was done in CASSANDRA-17132.

We need to update the docs respectively after the migration to ASCIIdoc is done:
 * -Remove any references to {_}otc_backlog_expiration_interval_ms, otc_coalescing_strategy, otc_coalescing_window_us_default{_}, _otc_coalescing_window_us,_ _otc_coalescing_enough_coalesced_messages, internode_application_timeout_in_ms._-  _Inform the users about deprecation of otc_backlog_expiration_interval_ms, otc_coalescing_strategy, otc_coalescing_window_us_default, otc_coalescing_window_us, otc_coalescing_enough_coalesced_messages._ 
_internode_application_timeout_in_ms - this one never existed internally so I am not sure whether we need to add it now to Config.java... It was only a commented property in cassandra.yaml that never got into the release_
 * The names of _internode_send_buff_size_in_bytes_ and _internode_recv_buff_size_in_bytes_ to be changed to 

_internode_socket_send_buffer_size_in_bytes_ and _internode_socket_recv_buffer_size_in_bytes_ everywhere in the docs
Bug: utests_system_keyspace_directory is not run in Jenkins
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Nov/21 15:33
Resolved: 15/Nov/23 03:05

It seems to me utests_system_keyspace_directory which we run with 4.0 and trunk in CircleCI do not run in Jenkins as they all fail in trunk now in CircleCI and nothing in Jenkins. The issue is not environmental and I don't see that group of tests under Stage View in Jenkins so they are probably not added there? We need to investigate that further.

CC [~mck]  and [~blerer] if they know anything about those - like work in progress to be added or a reason not to be added.

FYI [~bereng]
Bug: Enable full checkstyle on tests
Status: Open
Priority: Normal
Created: 08/Nov/21 21:16

-Currently, checkstyle only runs against the source, leaving the tests with violations.  This ticket is to clean up the test code so checkstyle can be run against it at build time.-

As of November 2023: a checkstyle_test.xml was added in CASSANDRA-17316, but it is a stopgap measure and doesn't cover everything that checkstyle.xml run on the code does.  This ticket is to enable full checkstyle on the tests, such that only one checkstyle config is needed and the code in the tests is equally compliant.
Bug: Broken test_rolling_upgrade - upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Nov/21 00:04
Resolved: 11/Apr/22 05:09

The tests "test_rolling_upgrade" fail with the below error. 
 
[https://app.circleci.com/pipelines/github/yifan-c/cassandra/279/workflows/6340cd42-0b27-42c2-8418-9f8b56c57bea/jobs/1990]
 
I am able to alway produce it by running the test locally too. 
{{$ pytest --execute-upgrade-tests-only --upgrade-target-version-only --upgrade-version-selection all --cassandra-version=4.0 upgrade_tests/upgrade_through_versions_test.py::TestUpgrade_indev_3_11_x_To_indev_4_0_x::test_rolling_upgrade}}
 
{code:java}
self = <upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x object at 0x7ffba4242fd0>
subprocs = [<Process(Process-1, stopped[SIGKILL] daemon)>, <Process(Process-2, stopped[1] daemon)>]

    def _check_on_subprocs(self, subprocs):
        """
            Check on given subprocesses.
    
            If any are not alive, we'll go ahead and terminate any remaining alive subprocesses since this test is going to fail.
            """
        subproc_statuses = [s.is_alive() for s in subprocs]
        if not all(subproc_statuses):
            message = "A subprocess has terminated early. Subprocess statuses: "
            for s in subprocs:
                message += "{name} (is_alive: {aliveness}), ".format(name=s.name, aliveness=s.is_alive())
            message += "attempting to terminate remaining subprocesses now."
            self._terminate_subprocs()
>           raise RuntimeError(message)
E           RuntimeError: A subprocess has terminated early. Subprocess statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting to terminate remaining subprocesses now.{code}