av9ash/GitBugs huggingface dataset:
Bug: Simplify the almost duplicated code for calculating serialization size and serialization
Status: Resolved
Priority: Normal
Resolution: Abandoned
Created: 20/Oct/20 20:14
Resolved: 29/Oct/20 16:38

The current pattern of counting the serialization size and the actual serialization in the codebase is error-prone and hard to maintain. Those 2 code paths have almost the same code repeated. 
 
The pattern can be found in {{org.apache.cassandra.net.Message#Serializer}} and numerous locations that use {{org.apache.cassandra.transport.CBCodec}}. 
 
I am proposing a new approach that lets both methods share the same code path. The code basically looks like the below (in the case of {{org.apache.cassandra.net.Message#Serializer}}).

{code:java}
// A fake DataOutputPlus that simply increment the size when invoking write* methods
public class SizeCountingDataOutput implements DataOutputPlus
{
 private int size = 0;

 public int getSize()
 {
   return size;
 }

 public void write(int b)
 {
   size += 1;
 }

 public void write(byte[] b)
 {
   size += b.length;
 }
 ...
}
{code}

Therefore, in the size calculation, we can supply the fake data output and call serialize to get the size.

{code:java}
private <T> int serializedSize(Message<T> message, int version)
{
 SizeCountingDataOutput out = new SizeCountingDataOutput();
 try
 {
   serialize(message, out, version);
 }
 catch (IOException exception)
 {
   throw new AssertionError("It should not happen!", exception);
 }
 return out.getSize();
// The original implementation
// return version >= VERSION_40 ? serializedSizePost40(message, version) : serializedSizePre40(message, version);
}
{code}
Bug: SSTableLoader issue when restoring a table named backups
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 29/Oct/20 14:44
Resolved: 02/Nov/20 16:02

"backups" is a keyword used to parse sstable path (Descriptor.fromFilename())..
If sstable file's parent directory name is "backups", it takes "backups"'s parent directory name as table name.
Bug: ArrayClustering does not properly handle null clustering key elements left over from tables created WITH COMPACT STORAGE
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 02/Nov/20 16:12
Resolved: 10/Nov/20 20:46

The only way we can produce null clustering key elements is leaving them empty on insert while a table is still compact. If we subsequently DROP COMPACT STORAGE, those null elements linger, and {{ArrayClustering}} does not handle them appropriately on compaction. 

If you run the test [here|https://github.com/maedhroz/cassandra/commit/e247b7868cae383168153bbe8bbbaa47a660f76b], you should be able to observe an exception that looks roughly like this:

{noformat}
java.lang.NullPointerException
	at java.base/java.nio.ByteBuffer.wrap(ByteBuffer.java:422)
	at org.apache.cassandra.db.AbstractArrayClusteringPrefix.getBufferArray(AbstractArrayClusteringPrefix.java:45)
	at org.apache.cassandra.io.sstable.metadata.MetadataCollector.finalizeMetadata(MetadataCollector.java:246)
	at org.apache.cassandra.io.sstable.format.SSTableWriter.finalizeMetadata(SSTableWriter.java:315)
	at org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$200(BigTableWriter.java:52)
	at org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:415)
	at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:168)
	at org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:283)
	at org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:380)
	at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:168)
	at org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:118)
	at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:168)
	at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:179)
	at org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:128)
	at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:225)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
{noformat}

There are already numerous places where we respect the fact that clustering elements may be null, so this should be pretty straightforward to fix, and the tests that accompany it will probably be more complex than the fix itself.
Bug: Fix flaky test testTPStats - org.apache.cassandra.tools.NodeToolGossipInfoTest
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 05/Nov/20 22:49
Resolved: 13/Nov/20 17:02

https://app.circleci.com/pipelines/github/dcapwell/cassandra/764/workflows/6d7a6adc-59d1-4f3c-baae-1f8329dca9b7/jobs/4363

{code}
junit.framework.AssertionFailedError
	at org.apache.cassandra.tools.NodeToolGossipInfoTest.testTPStats(NodeToolGossipInfoTest.java:128)
{code}
Bug: Extend the exclusion of replica filtering protection to other indices instead of just SASI
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/Dec/20 16:09
Resolved: 11/Dec/20 18:40

There was a check introduced in CASSANDRA-8272 telling if an index is a SASI index and if it is, replica filtering protection was not triggered.

There might be other custom index implementations which also do not support filtering protection and they do not have to be SASI indices neccessarily, however it is not possible to exclude them.

 

PR trunk

[https://github.com/apache/cassandra/pull/844]

PR 3.11

[https://github.com/apache/cassandra/pull/847]

PR 3.0

https://github.com/apache/cassandra/pull/848
Bug: Bump native protocol framing from v5 to v6
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 09/Dec/20 13:12
Resolved: 15/Dec/20 17:49

CASSANDRA-15299 revised the wire format of CQL native protocol to add a framing layer around the existing CQL messages. This was originally targetted at protocol v5, which is (still) currently in beta. There's a complication with this though; while v5-beta is supported in Cassandra 3.11.x and 4.0.0, the new wire format is only committed to trunk (4.0.0). This means that if clients upgrade to a version of their driver which supports the new wire format (3.10.0 for the java driver, python driver is not yet offically released), connections to Cassandra 3.11.x nodes will break if the client specifies v5-beta as the preferred protocol version. 

Without the wire format changes the v5 changelog is pretty minimal, but pulling v5 support from 3.11.x would make for a pretty bad user experience, despite it being "only" a beta.

I propose finalizing CASSANDRA-14973 to promote v5 from beta to a final version and bumping the new wire format changes to v6-beta, making both of these changes in 4.0 and 3.11. Because v5 requires the beta flag to be set in the frame header if using a beta version, this requires that users upgrade clusters to a version with v5-final before upgrading to a driver with v5-final. We should include this in NEWS.txt

The changes required to Cassandra itself are minor, as are the corresponding patches to the java and python drivers.
Bug: python upgrade tests include tests which are not impacted by the version under test
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 10/Dec/20 18:34
Resolved: 14/Dec/20 18:22

When we try to run the upgrade tests for trunk, we get tests such as upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x to run, which do not include the trunk source.  Since these tests are costly and can be flaky, running the unneeded tests for a release increases the cost and can make it harder to see if any new failures.

See https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests
Bug: Does open source cassandra supports PITR(point in time recovery) ?
Status: Resolved
Priority: Normal
Resolution: Not A Bug
Created: 18/Dec/20 12:23
Resolved: 18/Dec/20 13:32

Does open source cassandra (3.11.3 ) supports PITR(point in time recovery) ?
 or it will only work on enterprise edition?
 If so how to achieve i am not getting any proper documentation.
 i have read about commitlog_archiving.properties and enabled the commit logs archiving
 but not able to restore to some point in time.
 Does PITR works only in case of node failure.
 if i mistakenly deleted the data or keyspace can i use PITR in that to revert to previous before  the issue,

have some queries like apart from node failure data if i want to restore to any point is that possible?
 wt is restore command used for?
 wt is restore directories used for?

by placing archive commit logs in commit log directory and restart the node that will get automatic replayed?
 if i copy archive logs and mention point in time in file does get data before that time?

Do we have any example configuration how to use this settings?
Bug: Tracing continues after session completed
Status: Open
Priority: Normal
Created: 08/Dec/20 08:47

We saw the system_trace.events table increasing in size continuously without any trace requests being issued.

I traced the issue back to a specific version and patch. I believe we have removed the call to reset the trace flag in CASSANDRA-15041 which causes tracing to continue in the thread even after it is finished with the request.

Reproduced like follows:
1. ccm test -n 1 -v 3.11.9

2. Enable authentication/authorization

3. Set permissions_update_interval_in_ms: 1000 (It works if this value is default value. I am guessing this is because the update is done in the calling thread)

4. select * from some table a bunch of times until PermissionRoleCache is refreshed

5. Watch system_traces.events grow
Bug: Fix upgrade python dtest test_crc_check_chance_upgrade - upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Dec/20 02:12
Resolved: 15/Dec/20 17:37

https://app.circleci.com/pipelines/github/dcapwell/cassandra/846/workflows/bee81d86-f720-4fc2-b3e7-7ed19ec3c9f0/jobs/4988/tests

test_crc_check_chance_upgrade - upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade

{code}
>       self.verify_old_crc_check_chance(node1)

upgrade_crc_check_chance_test.py:58: 
{code}

{code}
self = <upgrade_crc_check_chance_test.TestCrcCheckChanceUpgrade object at 0x7f2d1d3b4be0>
node = <ccmlib.node.Node object at 0x7f2d1d335a90>

    def verify_old_crc_check_chance(self, node):
        session = self.patient_exclusive_cql_connection(node)
        session.cluster.refresh_schema_metadata(0)
        meta = session.cluster.metadata.keyspaces['ks'].tables['cf1']
>       logger.debug(meta.options['compression_parameters'])
E       NameError: name 'logger' is not defined

upgrade_crc_check_chance_test.py:107: NameError
{code}
Bug: Add metric for checking the token-awarness of the users
Status: In Progress
Priority: Normal
Created: 10/Dec/20 17:37

In order to increase the speed of the interaction with a Cassandra database, especially when the consistency level and the replication factor settings make one node enough for responding, having knowledge of the token allocation among the nodes and querying the specific node which holds the data requested might be a good solution. Basically, in these conditions, the internode communication latency is eliminated.

A metric which counts the number of queries that resolve on the coordinator node, and the number of those which do not, might be useful for the client applications which are designed to be token-aware as this way it would be easy to see if they work as expected (the number of remote executed queries is low) or not.
Bug: Can't build Cassandra 3.11 from source
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 04/Dec/20 13:45
Resolved: 04/Dec/20 16:40

Version of Cassandra is 3.11.9/3.11.10
{code:bash}
java -version
openjdk version "1.8.0_275"
OpenJDK Runtime Environment (build 1.8.0_275-b01)
OpenJDK 64-Bit Server VM (build 25.275-b01, mixed mode)
{code}
{code:bash}
Buildfile: .../cassandra/build.xml
   [script] Warning: Nashorn engine is planned to be removed from a future JDK release

init:

maven-ant-tasks-localrepo:

maven-ant-tasks-download:

maven-ant-tasks-init:

maven-declare-dependencies:

maven-ant-tasks-retrieve-build:

init-dependencies:
     [echo] Loading dependency paths from file: .../cassandra/build/build-dependencies.xml

init-dependencies:
     [echo] Loading dependency paths from file: .../cassandra/build/build-dependencies-sources.xml
    [unzip] Expanding: .../cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar into .../cassandra/build/lib/jars

check-gen-cql3-grammar:

gen-cql3-grammar:

generate-cql-html:

generate-jflex-java:

rat-init:

rat-report:

build-project:
     [echo] apache-cassandra: .../cassandra/build.xml
    [javac] Compiling 2 source files to .../build/classes/main
    [javac] .../cassandra/src/java/org/apache/cassandra/utils/Throwables.java:80: error: unreported exception Exception; must be caught or declared to be thrown
    [javac]         perform(Stream.of(actions));
    [javac]                ^
    [javac] .../cassandra/src/java/org/apache/cassandra/utils/concurrent/Locks.java:46: error: cannot find symbol
    [javac]             unsafe.monitorEnter(object);
    [javac]                   ^
    [javac]   symbol:   method monitorEnter(Object)
    [javac]   location: variable unsafe of type Unsafe
    [javac] .../cassandra/src/java/org/apache/cassandra/utils/concurrent/Locks.java:52: error: cannot find symbol
    [javac]             unsafe.monitorExit(object);
    [javac]                   ^
    [javac]   symbol:   method monitorExit(Object)
    [javac]   location: variable unsafe of type Unsafe
    [javac] Note: .../cassandra/src/java/org/apache/cassandra/utils/Throwables.java uses unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 3 errors

BUILD FAILED
.../cassandra/build.xml:863: Compile failed; see the compiler error output for details.

{code}
Bug: Cloud/AWS - use local instance stores for reads and mirror sstables to EBS
Status: Triage Needed
Priority: Normal
Created: 10/Dec/20 17:22

Using instance stores for cassandra in AWS is difficult because instance restart involves a node reconstruction.

EBS has gotten better, but still suffers from restrictions in IOPS and more importantly, overall network bandwidth to the EBS subsystem.

If we could view the instance store as an actively managed sstable cache that is replicated to a mirrored ebs volume, we could utilize it for improved read performance. 

Flushing: a flush would flush to both the instance local ssd and to the ebs volume. Commitlogs are only removed once the flush to the ebs volume is completed.

Compaction: compaction would perform the sstable merge work on the instance local store with its vastly higher throughput. Once calculated, the sstables are only switched over once we have the compacted table mirrored. 

Read path: redirect all reads to instance local store sstables for servicing.

Write path is normal to the commitlog, and is impacted with the additional flush mechanics.

On instance restart, cassandra has to know that the instance store is cleared and needs to be warmed. This would trigger either a throttled copy and/or as sstables are read they are mirrored to the instance local store. 

I'm thinking this may become particularly important as extremely high performance SSDs like Intel's XPoint develops.
Bug: Connection failure to down node spams debug.log
Status: Open
Priority: Normal
Created: 04/Dec/20 14:30

When a node becomes down a periodic attempt is made to try to reconnect to the node. However, the connection error [spams|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L546] the debug.log of all nodes in the cluster with the following message:
{noformat}
DEBUG [MessagingService-Outgoing-/XX.YY.ZZ.WW-Gossip] 2020-12-04 09:13:58,999 OutboundTcpConnection.java:546 - Unable to connect to /XX.YY.ZZ.WW
java.net.ConnectException: Connection refused
	at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_172]
	at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_172]
	at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_172]
	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648) ~[na:1.8.0_172]
	at org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:146) ~[apache-cassandra-3.11.3.jar:3.11.3]
	at org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:132) ~[apache-cassandra-3.11.3.jar:3.11.3]
	at org.apache.cassandra.net.OutboundTcpConnection.connect(OutboundTcpConnection.java:434) [apache-cassandra-3.11.3.jar:3.11.3]
	at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:262) [apache-cassandra-3.11.3.jar:3.11.3]
{noformat}
I've seen cases where this rolled the entire debug.log which can difficult troubleshooting.

We should probably guard this in a NoSpamLogger.
Bug: Having issues creating a table with name profiles
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 02/Nov/20 06:50
Resolved: 02/Dec/20 15:49

Whenever I try to create a table with name profiles, it always gets created with additional quotes surrounding it. Attaching the screenshot.
I am on Cassandra 3.7
I tried creating the table in another keyspace. I also tried creating new virtual machines with the same AMI and same Cassandra version, but to no avail.
If I try to create a table with any other name, there are no issues at all. It's just with the name profiles.
I am on Ubuntu 18.04 by the way.




!image-2020-11-02-12-13-16-999.png!
Bug: Stress testing a mixed cluster with C* 2.1.0 (seed) and 2.0.0 causes NPE
Status: Resolved
Priority: Normal
Resolution: Won't Fix
Created: 11/Nov/20 22:23
Resolved: 12/Nov/20 19:27

Steps to reproduce: 
 # setup a mixed cluster with C* 2.1.0 (seed node) and C* 2.0.0
 # run the stress testing tool, e.g.,

{code:java}
/cassandra/tools/bin/cassandra-stress write n=1000 -rate threads=50 -node 250.16.238.1,250.16.238.2{code}
NPE: 
{code:java}
ERROR [InternalResponseStage:2] 2020-07-22 08:29:36,170 CassandraDaemon.java (line 186) Exception in thread Thread[InternalResponseStage:2,5,main]
java.lang.NullPointerException
  at org.apache.cassandra.serializers.BooleanSerializer.deserialize(BooleanSerializer.java:33)
  at org.apache.cassandra.serializers.BooleanSerializer.deserialize(BooleanSerializer.java:24)
  at org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:142)
  at org.apache.cassandra.cql3.UntypedResultSet$Row.getBoolean(UntypedResultSet.java:106)
  at org.apache.cassandra.config.CFMetaData.fromSchemaNoColumnsNoTriggers(CFMetaData.java:1555)
  at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1642)
  at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:305)
  at org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:270)
  at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:183)
  at org.apache.cassandra.service.MigrationTask$1.response(MigrationTask.java:66)
  at org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:46)
  at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
{code}
Root cause: incompatible data

In the `CFMetaData` class of version 2.0.0, there is a boolean field named `replicate_on_write`. In the same class of version 2.1.0, however, this field no longer exists. When serializing this class in function `toSchemaNoColumnsNoTriggers`, it will first write all of its fields into a `RowMutation` (in 2.0.0) / `Mutation` (in 2.1.0) class, and then serialize this “Mutation” like class in the same way. In 2.0.0 the `replicate_on_write` field gets serialized at [https://github.com/apache/cassandra/blob/03045ca22b11b0e5fc85c4fabd83ce6121b5709b/src/java/org/apache/cassandra/config/CFMetaData.java#L1514] .

When deserializing this class in function `fromSchemaNoColumnsNoTriggers`, it reads all its fields from a map-like class `UntypedResultSet.Row`. In 2.0.0 the `replicate_on_write` field gets deserialized at [https://github.com/apache/cassandra/blob/03045ca22b11b0e5fc85c4fabd83ce6121b5709b/src/java/org/apache/cassandra/config/CFMetaData.java#L1555] .

The problem is that the existence of the key is not checked, and the map returns a `null` value because the message from 2.1.0 doesn’t contain the `replicate_on_write` key, which leads to the NullPointerException.
Bug: org.apache.cassandra.streaming.LongStreamingTest fail with NPE
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 24/Nov/20 16:01
Resolved: 26/Nov/20 09:48

{code}
java.lang.NullPointerException
	at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.awaitTermination(AbstractCommitLogSegmentManager.java:509)
	at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.stopUnsafe(AbstractCommitLogSegmentManager.java:427)
	at org.apache.cassandra.db.commitlog.CommitLog.stopUnsafe(CommitLog.java:467)
	at org.apache.cassandra.SchemaLoader.cleanupAndLeaveDirs(SchemaLoader.java:731)
	at org.apache.cassandra.streaming.LongStreamingTest.setup(LongStreamingTest.java:56)
{code}
Bug: unable to find authenticator class CassandraAuthorizer
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 23/Nov/20 08:26
Resolved: 24/Nov/20 14:52

Referral code for the implementation of LDAP authentication of the Cassandra DB

[https://github.com/instaclustr/cassandra-ldap] 

Issue Reported :- 

[https://github.com/instaclustr/cassandra-ldap/issues/16] 

 
h3. What version of Cassandra are you using? - 2.2
h3. What version of Cassandra LDAP are you using? cassandra-ldap-2.2-1.0.0
h3. What LDAP server you are using? Any specifics? - No
h3. What did you do?

Placed the jar(cassandra-ldap-2.2-1.0.0) in the /lib path
which resolved the issue unable to find authenticator class org.apache.cassandra.auth.Cassandra22LDAPAuthenticator

Current issue that i am facing is "unable to find authenticator class CassandraAuthorizer"

Tried with this resolution by placing of the jar cassandra-all-2.2.14.jar inside the /lib

Still the error exists.
h3. What did you expect to see?

Normal start of the Cassandra server
h3. What did you see instead?
h2. unable to find authenticator class CassandraAuthorizer

If you are having connectivity related issues please share the following additional information
h3. Describe your Cassandra cluster

please provide the following information
 * output of {{nodetool status}} node tool is not there
 * output of {{SELECT peer, rpc_address FROM system.peers}} both the columns are empty
 * output of {{describe keyspace system_auth}} having with simple strategy class and replication factor is 1
Bug: Update NetBeans project file for dependency changes since 11th Feb 2020
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 29/Oct/20 11:48
Resolved: 20/Nov/20 08:23

A number of dependencies have been add/removed/updated to the project.
The netbeans project file needs an update to be in-sync.

Causing tickets:
 - CASSANDRA-15677
 - CASSANDRA-16064
 - CASSANDRA-12995
 - CASSANDRA-15867
 - CASSANDRA-12197
 - CASSANDRA-15556
 - CASSANDRA-15868
 - CASSANDRA-16150
 - CASSANDRA-15631
 - CASSANDRA-15851
 - CASSANDRA-16148
 - CASSANDRA-14655
 - CASSANDRA-15867
 - CASSANDRA-15388
 - CASSANDRA-15564
 - CASSANDRA-16127