av9ash/GitBugs huggingface dataset:
Bug: Add multi-partition key commands
Status: Triage Needed
Priority: Normal
Created: 01/Jul/20 16:14

As it's mentioned in [this TODO comment|https://github.com/apache/cassandra/blob/dd255ffa07d0263521a1ca863fc2192db19bc04c/src/java/org/apache/cassandra/service/ReplicaFilteringProtection.java#L141] that was introduced in CASSANDRA-8272, replica filtering protection would benefit from a {{ReadCommand}} implementation able to internally read the rows identified by a group of primary keys residing in different partitions. 

It would be different from {{SinglePartitionReadCommand.Group}} in that it wouldn't need to be split in per-partition commands when sending it to a replica. We could call it {{MultiPartitionReadCommand}}, for example.

Such command would also be beneficial for the performance of regular IN CQL queries, since it would reduce the traffic between the coordinator and the replicas by grouping the queries sent to a replica.

An alternative way of getting that performance gain could be modifying how the current {{SinglePartitionReadCommand.Group}} is executed on the {{StorageProxy}} so, instead of sending each of its internal {{SinglePartitionReadCommand}} s to the replicas, we could split it in smaller {{Group}} commands targeted to specific replicas, if that makes sense.
Bug: SASI should handle ReversedType when using "instanceof" on AbstractType
Status: Open
Priority: Low
Created: 22/Jul/20 17:23

{code:java}
createTable("CREATE TABLE %s (pk int, ck text, v int, primary key(pk, ck)) WITH CLUSTERING ORDER BY (ck DESC);");
createIndex("CREATE CUSTOM INDEX ON %s (ck) USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS" +
            " = {'analyzer_class': 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 'case_sensitive': 'false'} ");
{code}
Bug: Improve docs about frozen types and invert UDT/Tuple order
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 09/Sep/20 20:12
Resolved: 08/Oct/20 12:21

Currently, there's no documentation regarding frozen/non-frozen types.

Also, a tuple is mentioned after the definition of UDT: "tuples can be though as anonymous UDT with anonymous fields". Since in the code base a UDT is an extension of a tuple, it would be nice to invert these in the docs.

This issue addresses both topics.
Bug: Change style guide to include @Override
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 02/Sep/20 17:40
Resolved: 02/Sep/20 18:37

@Override is used by javac to enforce that a method is in fact overriding from an abstract class or an interface and if this stops being true (such as a refactor happens) then a compiler error is thrown; when we default to excluding, it makes it harder to detect that a refactor catches all implementations and can lead to subtle and hard to track down bugs.

This ticket removes the comment in the docs that @Override should be avoided and updated the style guide to include them by default.
Bug: Add a getter for InstanceConfig parameters - in-jvm-dtests-api
Status: Resolved
Priority: Low
Resolution: Won't Fix
Created: 07/Aug/20 15:26
Resolved: 22/Oct/20 20:28

In order to change the way config will be loaded (for reference CASSANDRA-15234 ) a getter for the InstanceConfig parameters is needed 

CC [~maedhroz]
Bug: Additional information to be added to the Contributing to Cassandra section on Cassandra web site
Status: Open
Priority: Low
Created: 16/Sep/20 22:01

It would be nice to inform people about the CLA (some contributors never contributed to OSS before).

Also there is no mention about the CEPs. I think it would be good to add a link to the wiki. 

/CC [~lorina@datastax.com]
Bug: Circle CI for jdk11 doesn't run unit tests by default but runs jvm-dtests and runs with only 2 workers
Status: Open
Priority: Low
Created: 15/Sep/20 23:21

By default j11 builds and runs jvm-dtests (with 2 workers on HIGH, smaller than j8) but does not run the unit tests.  We should disable jvm-dtests by default and make them run with the same amount of workers as j8.
Bug: Circle CI start_ jobs should not require build
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 21/Jul/20 20:12
Resolved: 24/Jul/20 01:09

The Circle CI start_ jobs all depend on build, so can’t approve them until the build is done, but their downstream also depends on build.  Given this, we can remove this dependency which will allow someone to approve the downstream pipelines even if the build isn’t complete yet.
Bug: Cassandra fails to start on aarch64 with Java8 due to too small a stack size
Status: Resolved
Priority: Urgent
Resolution: Duplicate
Created: 13/Jul/21 11:42
Resolved: 06/Aug/21 15:54

Trying to start 4.0.0 on aarch64 with Java8 throws
{noformat}
The stack size specified is too small, Specify at least 328k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
{noformat}

Cong [reports|https://lists.apache.org/thread.html/r71d00644a3b488dc2718f99273df4903ee890dadd7e7e467cef612ae%40%3Cdev.cassandra.apache.org%3E] that

bq. It’s worked once I added the -Xss512k to the cassandra-env.sh (for the JVM_OPTS).
Bug: WEBSITE - September 2021 blog post on ApacheCon
Status: Resolved
Priority: Urgent
Resolution: Fixed
Created: 20/Sep/21 19:24
Resolved: 21/Sep/21 13:30

This ticket is to capture the work associated with publishing the September 2021 blog post on ApacheCon 2021.

We can close this once the blog post has gone live on the website.
Bug: Bump zstd-jni version to 1.5.0-4
Status: Resolved
Priority: High
Resolution: Fixed
Created: 25/Aug/21 18:55
Resolved: 26/Aug/21 22:11

The current zstd-jni version (1.3.8-5) was released in 04/12/2019. There has been a lot of development on the zstd library and the jni binding during the 2.5 years, including a fuzzer which detected a handful of corruption bugs and performance improvements. 

The version of zstd-jni maps with the one of the native library. The current native lib version is 1.5.0, hence 1.5.0-4 for zstd-jni. 

I am proposing bumping the zstd-jni version to the current.
Bug: Fix incorrect UI bundle URL in content Dockerfile
Status: Resolved
Priority: High
Resolution: Fixed
Created: 03/Sep/21 04:21
Resolved: 09/Dec/21 20:41

We need to change the UI bundle URL in the content container Dockerfile to point to a UI bundle that contains the correct site styling.

The URL to the bundle can be updated only after an initial version of the UI bundle is tagged and released on the cassandra-website.
Bug: Secure HTTPS link to the research paper in the website landing page
Status: Triage Needed
Priority: Normal
Created: 19/Aug/21 19:10

In [https://cassandra.apache.org/_/index.html] , under the Performant section, there's a link to a research paper - [http://vldb.org/pvldb/vol5/p1724_tilmannrabl_vldb2012.pdf] - "Solving Big Data Challenges for Enterprise Application Performance Management". But this link is a HTTP link. On clicking this, my browser shows me a security warning. This is how it looks like on Firefox -

!security-risk-warning.png|width=450,height=75!

and some more details -

!extra-details-about-security-risk.png|width=351,height=259!

I noticed that the same research paper has an alternative HTTPS URL. I think that could be used instead - [https://vldb.org/pvldb/vol5/p1724_tilmannrabl_vldb2012.pdf]

I would love to make this small fix in the docs
Bug: Blocking Concurrency Primitives
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Sep/21 17:48
Resolved: 07/Oct/21 11:28

To support CEP-10 it is necessary to support alternative implementations of the blocking concurrency primitives we use on the project. At the same time, the project is very inconsistent in its usage of these APIs, so this work includes a number of improvements to the coherency of the codebase. 

This ticket introduces new abstractions and standardises old ones, migrating all blocking concurrency operations besides Futures returned by Executors to these new APIs. This includes a migration of SimpleCondition to a new Conditioninterface, new CountDownLatch and Semaphore interfaces, and new static factory methods for creating these objects (as well as WaitQueue) that can be intercepted by byte weaving. Additionally the internal Netty Future implementation is improved to support more general use (though this is only fully realised in a later ticket), OpOrder is improved to more easily support mocking WaitQueue.

Finally we standardise the propagation of InterruptedExecption, with the new UncheckedInterruptedException, so that simulations may be terminated cleanly.
Bug: allow blocking IPs from updating metrics about traffic
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Aug/21 23:15
Resolved: 21/Aug/21 00:24

It is common that network scans happen for some companies, and when these scans happen they will send packets to the Cassandra ports which do not match our protocol; when this happens we fail and increment metrics to show this.  To avoid these scans polluting our metrics, we should allow users to block their own IP subnets from updating these metrics.
Bug: UDTs named after built-in data types
Status: Open
Priority: Normal
Created: 31/Aug/21 15:54

Working with Cassandra 4.0 I found UDTs can be named after some built-in data type, like map, list, and tuple, but not for other types, like text or set. This behavior should be consistent across all built-in types where Cassandra shouldn't allow a UDT to be named after any built-in type.

Having UDTs named after built-in types cause confusion working with table schemas and may introduce downstream issues when Cassandra works with these types.
Bug: Add support for IF EXISTS and IF NOT EXISTS in ALTER statements
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/Sep/21 15:42
Resolved: 04/Apr/22 09:40

It would make sense to add support for {{IF EXISTS}} and {{IF NOT EXISTS}} in the different {{ALTER}} statements. 

For example:
* {{ALTER TABLE IF EXISTS myTable ...}}
* {{ALTER TABLE myTable ADD IF NOT EXISTS ...}}
* {{ALTER TABLE myTable DROP IF EXISTS ...}}
* {{ALTER TYPE IF EXISTS myType ...}}
* {{ALTER TYPE myType ADD IF NOT EXISTS ...}}

+Additional info for newcomers:+

In order to implement this change you will need to change the {{Parser.g}} ANTLR file located in the src/antlr directory and the java classes corresponding to the different alter statements located in the {{org.apache.cassandra.cql3.statements.schema}} package. You can look at the CreateTableStatement class to see how it was done there.
The unit test for the CQL logic are located under {{org.apache.cassandra.cql3.validation}}
Bug: Scrub does not detect invalid partition keys
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Sep/21 20:35
Resolved: 27/Sep/21 14:14

The standalone scrubber [gets the key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202] from the file but never validates it, and this will propagate to the new sstable so it will be corrupted when read later.
Bug: add native_port_ssl to system_views.peers table
Status: Resolved
Priority: Normal
Resolution: Won't Fix
Created: 27/Sep/21 14:29
Resolved: 13/Feb/24 12:38

system.peers_v2 includes a “native_port” but has no notion of native_transport_port vs. native_transport_port_ssl.  Given this limited information, there’s no clear way for the driver to know that different ports are being used for SSL vs. non-SSL or which of those two ports is identified by “native_port”.

 

The issue we ran into is that the java driver, since it has no notion of the transport port SSL, the driver was only using the contact points and was not load balancing.

 

The customer had both set:

native_transport_port: 9042

native_transport_port_ssl: 9142

 

They were attempting to connect to 9142, but that was failing. They could only use 9042, and so their applications load balancing was failing. We found that any node that was a contact point was connecting, but the other nodes were never acting as coordinators.

 

There are still issues in the driver, for which I have created JAVA-2967, which also refers to JAVA-2638, but the system.peers and system.peers_v2 tables should both contain native_transport_port and native_transport_port_ssl.
Bug: Respect com.sun.management.jxmremote.host 
Status: Review In Progress
Priority: Normal
Created: 20/Sep/21 21:38

It seems we don't respect com.sun.management.jxmremote.host in [createJMXServer()|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/utils/JMXServerUtils.java#L64]
 The flow should be - when_ local _is true, we only bind localhost, otherwise we bind everything, unless com.sun.management.jmxremote.host is set, then we use that.
Bug: Test failure: test_compression_cql_options
Status: Open
Priority: Normal
Created: 17/Aug/21 21:51

While working on another ticket, I saw test_compression_cql_options failing (which is not the case in Jenkins) but the Circle CI multiplexer showed it as being flaky:

[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1082/workflows/8dacef5d-46f4-4b59-a8f7-08824bf1180f/jobs/6407/tests#failed-test-0]


{code:java}
        sstables = self.get_sstables(table='compression_opts_table', indexes=list())
        sstable_paths = self.get_table_paths('compression_opts_table')
        found = False
        for sstable_path in sstable_paths:
            sstable = os.path.join(sstable_path, sstables['compression_opts_table'][1])
            if os.path.exists(sstable):
                assert 'DEFLATE' == self._get_compression_type(sstable)
                found = True
>       assert found
E       assert False

compression_test.py:118: AssertionError
{code}
Bug: Dtest cluster reusage
Status: Open
Priority: Normal
Created: 14/Sep/21 08:22

Dtests are very heavy but in some instances most of the time is spent restarting nodes in between test methods. Not all of them, but many seem could benefit form reusing a common cluster sparing the restarts. Obviously that is not the case for tests that manipulate the nodes itself during the test. The ones that follow a setup node/do test seem to benefit greatly in terms of time execution.

Some classes run time can be cut form 10m to 1,5m. Others only from 30m to 25m. But taking a 5m shave and considering it will probably get ran * with/without vnodes * j8/j11/j8j11 * 4.0/trunk turns the 5m cut into a 60m cut. That should be a nice reduction in CI usage. Unfortunately run time will mostly remain the same until we have a majority of tests reusing nodes as the 'longest pole' will be the determining factor.

How it works? It's an opt-in. Annotate the first test with {{@reuse_cluster(new_cluster=True)}} and the following ones with {{@reuse_cluster}}. Best effort to reuse the cluster will be made. Stop using the annotation at any test method and it will start a new one.
Bug: Remove ephemeral snapshot marker file and introduce a flag to SnapshotManifest
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Sep/21 19:25
Resolved: 15/Jul/22 19:39

While creating an ephemeral snapshot, a marker file is created at disk in a respective snapshot directory. This is not necessary anymore as we have introduced SnapshotManifest in CASSANDRA-16789 so we can move this flag there. By putting the information if a snapshot is ephemeral or not into SnapshotManifest, we simplify and "clean up" the snapshotting process and related codebase.
Bug: DOC - Formulae in Evaluating DM page does not display correctly
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Aug/21 18:31
Resolved: 10/Apr/23 18:11

In the [Evaluating and Refining Data Models|https://cassandra.apache.org/doc/latest/cassandra/data_modeling/data_modeling_refining.html] page, all of formulas does not display correctly:

 !スクリーンショット 2021-08-31 18.03.57.png|width=500! 

h3. Environment
browser: chrome 92.0.4515.159
os: macOS Big Sur 11.5.2

h3. Slack 

[https://the-asf.slack.com/archives/CJZLTM05A/p1630400708095100]
[https://the-asf.slack.com/archives/CJZLTM05A/p1630400951095900]
Bug: Replace minor use of `json-simple` with Jackson
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Aug/21 00:13
Resolved: 02/May/23 20:08

Jackson library is used for most JSON reading/writing, but there are couple of places where older "json-simple" library is used, mostly for diagnostics output. Replacing those minor usages would allow removal of a dependency, one for which the last release was made in 2012.

Places where json-simple is used are:
 * src/java/org/apache/cassandra/db/ColumnFamilyStore.java
 * src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java
 * src/java/org/apache/cassandra/hints/HintsDescriptor.java
 * src/java/org/apache/cassandra/tools/nodetool/stats/StatsPrinter.java

(and some matching usage in couple of test classes)

I can take a stab at replacing these uses; it also looks like test coverage may be spotty for some (StatsPrinter json/yaml part has no tests for example).

It is probably best to target this for "trunk" (4.1?).
Bug: Deprecate org.apache.cassandra.hadoop code
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Sep/21 17:15
Resolved: 12/Mar/23 21:18

The code under org.apache.cassandra.hadoop is becoming a legacy and probably should be deleted.

The code does not have any example / test how to use it, it is not documented and it depends on Hadoop of version 1 which is hardly used nowadays.

This ticket should track its deletion.

Feel free to object and we might revisit this decision. If we remove it in 4.1, it will still be present in 4.0.x where we can still patch.
Bug: Copy from csv file with duration type fields fails to import
Status: Open
Priority: Normal
Created: 05/Jul/21 10:22

Getting error:
{code:java}
cqlsh> COPY users.user_credentials_by_email FROM '/home/ubuntu/users.csv' WITH HEADER = FALSE AND NULL='null';
Using 3 child processes

Starting copy of users.user_credentials_by_email with columns [email, la_duration].
Failed to make batch statement: Received an argument of invalid type for column "la_duration". Expected: <class 'cassandra.cqltypes.DurationType'>, Got: <type 'str'>; (DurationType arguments must be a Duration.)_
Failed to import 1 rows: TypeError - Received an argument of invalid type for column "la_duration". Expected: <class 'cassandra.cqltypes.DurationType'>, Got: <type 'str'>; (DurationType arguments must be a Duration.), given up without retries
Failed to process 1 rows; failed rows written to import_users_user_credentials_by_email.err
Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 2 rows/s
0 rows imported from 1 files in 0.431 seconds (0 skipped).
{code}
*To Reproduce:*
{code:java}
CREATE KEYSPACE users WITH replication = {'class': 'SimpleStrategy', 'replication_factor' : 1 } AND durable_writes = true;

CREATE TABLE users.user_credentials_by_email (
email text,
la_duration duration,
PRIMARY KEY(email)
);
{code}
create users.csv file with:
{code:java}
LA@LA.com,8m26s482ms
{code}
Run:
{code:java}
COPY users.user_credentials_by_email FROM 'users.csv' WITH HEADER = FALSE AND NULL='null';
{code}
Bug: Further restrict schema column drop/recreate conversions
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Aug/21 16:43
Resolved: 13/Sep/21 14:27

CASSANDRA-14948 took us a good bit of the way to preventing recreation issues with columns, however we should take the safety a bit further and disallow recreation on fixed/variable length mismatches and simple/complex mismatches.
Bug: Fix flakiness in upgrade_tests/thrift_upgrade_test.py in dtests
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Aug/21 10:59
Resolved: 06/Sep/21 07:12

I noticed that tests for dtest in upgrade_tests/thrift_upgrade_test.py are flaky.

The reason this flakiness happens is that we are stopping and starting a node too fast without waiting for its full initialisation and then next attempt to connect to rpc port fails and whole test fails.

The fix is rather easy, we just need to wait until it is full started.
Bug: WEBSITE - PMC Governance link on Community page is broken
Status: Open
Priority: Normal
Created: 21/Sep/21 23:01

The link to *CASSANDRA PMC GOVERNANCE OVERVIEW* on the [Community|https://cassandra.apache.org/_/community.html] page is broken:

https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+Project+Governance

 !community-broken_PMC_governance_link.png|width=300! 

The wiki page was renamed and the link needs to be updated to use the permalink:

https://cwiki.apache.org/confluence/x/9hB4CQ
Bug: Introduce Determinism (and other) Configuration Options
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Sep/21 17:52
Resolved: 07/Oct/21 15:32

In order to support CEP-10 we must enable deterministic execution within Cassandra, support disabling certain optional features that are incompatible with simulation, and support overriding some existing features. 

Some determinism will be enabled by the Simulator itself, but there are aspects of Cassandra that must be made compliant as well. This includes: compression system-wide, memtable overhead computation, specifying the number of processors used for computation/decisions, specifing all cache sizes explicitly, enabling deterministic table IDs, and supporting deterministic node IDs for TimeUUID creation.

We also want to be able disable sstable activity tracking, and to be able to happily ignore problems with native filesystem methods and SIGAR. We also want to be able to override the migration delay, batch sync interval and system auth replication factor.

For pluggability we also want to be able to specify a custom FailureDetector.
Bug: Make creation timestamp consistent for all tables of a snapshot
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Sep/21 13:38
Resolved: 16/Sep/21 20:28

This mostly affects listsnapshots command as well as removal of a snapshot when it is on TTL.

Currently, there is not the same timestamp for "created_at" field in listsnapshots output so what might happen when TTL deletion kicks in is that only part of tables might be removed because other tables do not have to make it into that TTL threshold but they will be removed on the next SnapshotManager's removal round.
Bug: Close native transport port before hint transfer during decommission
Status: Open
Priority: Normal
Created: 10/Aug/21 13:14

New hints can be generated on a node when it's decommissioning which is a problem if the node has already started hint transfer because any hints that come in after the transfer has begun will remain on-disk and not be transferred to a peer.

You can work around this problem by manually closing the native transport port before starting the decommission with {{nodetool disablebinary}} but it feels like something we might want to do automatically.
Bug: Flaky failure of test_view_metadata_cleanup - materialized_views_test.TestMaterializedViews
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 10/Sep/21 08:22
Resolved: 24/Feb/22 09:50

When I run one of [java11 build on dtests with vnodes in CircleCI|https://app.circleci.com/pipelines/github/k-rus/cassandra/11/workflows/65abc408-edac-4c75-85dc-495f5c9af848/jobs/101] on Cassandra trunk with no relevant changes in dtests, {{test_view_metadata_cleanup}} of {{materialized_views_test.TestMaterializedViews}} has failed with
{code:java}
Unexpected error found in node logs (see stdout for full details). Errors: [WARN  [epollEventLoopGroup-5-18] 2021-09-10 07:45:42,765 ExceptionHandlers.java:134 - Unknown exception in client networking
io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer, WARN  [epollEventLoopGroup-5-18] 2021-09-10 07:45:42,765 ExceptionHandlers.java:134 - Unknown exception in client networking
io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer]
{code}