av9ash/GitBugs huggingface dataset:
Bug: Performance degradation when the data size grows
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 07/Feb/22 22:18
Resolved: 07/Feb/22 22:35

Hello Everyone, 

I am noticing a huge perf drop (spike in latency and CPU utilization) for the counter type tables when the data size grows. To better understand/simulate I have done the following perf test with `cassandra-stress` instead of my use-case and I can reproduce the performance issue consistently. When using the counter type tables, when the datasize grows the read latency and cpu spikes to very high number.

 

*Test Setup:*
 # Setup a cluster with 3 nodes.
 # Run a test with cassandra-stress and I see the latency and CPU are okay without much spike.
 # Send a lot of counter traffic using `cassandra-stress` tool (Replication Factory = 3)
 # Now the data size on the cluster is ~300G. 
 # Now run another test with cassandra-stress with 3:1 read write mixed workload.
 # At this point I see the CPU spikes to double (32 on a 16 core CPU) and the latency reaches ~1 seconds (which earlier was < 5ms).
 # Another interesting observation is the disk reads goes to a higher number and it keeps going higher with the increase in the disk size. 
 # It pretty much looked like a disk bottleneck issue but the same result shows very low disk reads, cpu, latency with less amount of data.
 # Below is the configuration I have used for testing this.

 
{quote}C* Version: 3.11.9

CPU: 16

Memory: 64G

Heap: 16G

GC: G1GC

Disk: 500G GCP Persistent disk 

 
{quote}
I understand that, with growth in disk the number of lookup grows high, but this looked to be a big performance drop.

Please let me know if you need more details. Also let me know this is known limitation with the counter type and if there is a work around.
Bug: Harry: Update Model validations to return ModelResult instead of void
Status: Triage Needed
Priority: Normal
Created: 08/Feb/22 05:54

From this PR discussion: [https://github.com/apache/cassandra/pull/1382#discussion_r791222990]

 

Currently, Model validations return void, so they cannot be used in assertions. There are situations where it would be useful to know what a validation actually did, in terms of queries run, partitions validated, etc.
Bug: Harry: refactor usage of primitive long to use specialized types
Status: Triage Needed
Priority: Normal
Created: 08/Feb/22 05:53

From this PR discussion: [https://github.com/apache/cassandra/pull/1382#discussion_r801209270]

 

Currently, Harry uses primitive long types extensively (logical timestamps, operation IDs, partition descriptors, clustering descriptors, etc). These are allocated on the stack so they are performant, but they are not type-safe. There's nothing preventing an accident like trying to inflate a clustering descriptor where a partition descriptor is expected.

 

To get around this, we could migrate these types to be non-primitive, with a negative impact on performance. When Value Objects arrive (with JEP 169), there may be new possibilities with different trade-offs.
Bug: WEBSITE - update various pages to reflect CI process
Status: Triage Needed
Priority: Normal
Created: 21/Jan/22 16:28

See [wiki article|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280]

See [dev list discussion|https://lists.apache.org/thread/bq470ml17g106pwxpvwgws2stxc6d7b9]

We have a few new sites on the confluence wiki that need to at least be linked from the official site, or preferentially have the content updated over there to reflect what we've updated here.

We should also leave a bread crumb / trail as comments there / notes in the cwiki to keep the content in sync between the two sources. Given how low friction wiki updates are this shouldn't represent significant content creation overhead; the risk is stagnation / staleness.

Pages:
1. Build Lead: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199527692
2. Patching, versioning, and LTS releases: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530302
3. Cassandra CI process: https://cwiki.apache.org/confluence/display/CASSANDRA/Cassandra+CI+Process
Bug: Investigate and remove Windows-specific threading-related code in copyutils.py
Status: Triage Needed
Priority: Normal
Created: 05/Feb/22 14:38

There are bits of the code to be removed or refactored related to how Windows were treating threading in copyutils.py as Windows is not longer supported.

[~Bowen Song] put it best so I just copy it here from GitHub PR for 16956, this code relates to FilesReader, FeedingProcess and ChildProcess Python classes.

We agreed on the fact that 16956 may be merged without this being addressed as it requires further investigation in the matter which would unncessarily postpone and delay it.

Bowen's take on this:

I believe that we should move the code from on_fork() to __init__(), and may also remove the on_fork() method if it's no longer used.

The problem with Windows and Python multiprocessing is that Windows doesn't support fork(), therefore Python implemented a workaround. On Windows, Python multiprocessing library uses pickle to serialise everything in memory, spawn a new process, and then restores the memory content from the serialised data. The ReceivingChannel and SendingChannel objects are not serialisable because they have file descriptors (which I believe it's called a file handle on Windows) in them, therefore the code has to handle them after the fake fork().

However, I'm concerned that moving the code from on_fork() to __init__() may have other unintended side effects. For example, the file descriptors (FDs) will be opened before the fork, in some edge cases the fork may never happen (e.g.: an exception raised in or just after the init, but before the fork). Where's the code responsible for closing the FDs on the parent process side? Will this cause any FD leak? This clearly requires further work to find out.

To be honest, I don't think removing the comments without addressing the above is a wise move. Future developers wouldn't have the opportunity to understand why the code is written in this way if the comment is removed.
Bug: Fix flaky test - 	TestThriftHSHA.test_6285
Status: Open
Priority: Normal
Created: 02/Feb/22 13:16

Failed 1 times in the last 18 runs. Flakiness: 5%, Stability: 94%
Error Message

test teardown failure

Stacktrace

Unexpected error found in node logs (see stdout for full details). Errors: [WARN  [MessagingService-Incoming-/127.0.0.1] 2021-11-11 01:11:53,939 IncomingTcpConnection.java:107 - UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId 59bb8940-428c-11ec-9980-6b05561bb8b2. If a table was just created, this is likely due to the schema not being fully propagated.  Please wait for schema agreement on table creation.
Bug: Fix flaky test - upgrade_tests.thrift_upgrade_test.TestUpgradeSuperColumnsThrough.test_dense_supercolumn 
Status: Open
Priority: Normal
Created: 31/Jan/22 16:09

Failed 3 times in the last 9 runs. Flakiness: 50%, Stability: 66%
Error Message

thrift.transport.TTransport.TTransportException: Transport not open
Bug: Ahead of Time compilation
Status: Resolved
Priority: Normal
Resolution: Won't Fix
Created: 28/Jan/22 06:14
Resolved: 28/Jan/22 12:09

*Ahead of Time compilation* for the Cassandra service (Which does some JIT Compilation by default).

I see that Cassandra has been written in Java and it is open-source which has clear instructions to build and run the service from the source ([https://blog.pythian.com/how-to-build-your-very-own-cassandra-4-0-release/])

I also can see there are a few popular options that can help achieve Ahead of Time Compilation for Java Programs (Ex: [https://www.graalvm.org/22.0/reference-manual/native-image/] a very popular choice)

Automated integration of Graalvm for Cassandra
Bug: In Jenkins, replace PostBuildTask plugin with PostBuildScript plugin
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 25/Jan/22 13:03
Resolved: 25/Jan/22 13:08

PostbuildTask loads the build log into memory, causing OutOfMemoryException.
https://issues.jenkins.io/browse/JENKINS-12830

We started hitting this problem this week, see discussion here:
https://the-asf.slack.com/archives/CK23JSY2K/p1643063068079200?thread_ts=1643052183.068300&cid=CK23JSY2K

Fix is to use the PostBuildTask plugin instead, which does not perform any reading off or matching on the build log.

PR: https://github.com/apache/cassandra-builds/pull/61
Bug: Add prompt to accept website/document sources before generating HTML
Status: Open
Priority: Normal
Created: 24/Jan/22 15:01

Provide a summary of the repository and the branches/tags that will be used as sources for Antora before any HTML generation is started. In addition, prompt the user to accept the listed sources.

There are a number of defaults that are used which determine what is generated by the documentation tooling. It is unclear what will be generated exactly if the defaults are used when the script is executed.

We will need to have the ability to automatically accept changes; that is a non-interactive option so CI can operate still.
Bug: WEBSITE - January 2022 changelog #11
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 13/Jan/22 23:56
Resolved: 18/Jan/22 14:56

This ticket is to capture the work associated with publishing the January 2022 changelog blog #11

If this blog cannot be published by the *January 18, 2022 publish date*, please contact me, suggest changes, or correct the date yourself when possible in the pull request for the appropriate time that the blog will go live (on blog.adoc and in the blog).
Bug: Broken link for contribute to cassandra on community page
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Jan/22 05:15
Resolved: 21/Jan/22 07:47

When visiting the community page of cassandra and going to `How to contribute` session the link is not rendered properly.

!image-2022-01-21-10-45-44-885.png!

 

We can see that _contribute to Cassandra_ is a simple text instead of a blank link tag.

 

Cause for the issue was in content we were using the URL macro but instead have to use  link macro.
Bug: Finalize and formalize canonical set of pre-commit tests for circleci
Status: Triage Needed
Priority: Normal
Created: 21/Jan/22 15:55

See [wiki article|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280]

See [dev list discussion|https://lists.apache.org/thread/bq470ml17g106pwxpvwgws2stxc6d7b9]

While a lot of good work went into this on CASSANDRA-16882 and subsequent tickets, there is still some dissent on the right set of canonical jobs we should require on circleci before a merge, when certain job suites might be acceptable to skip if ever, and the configuration output itself.

Success criteria here:
1. Ratify the canonical circleci job suites to be run on all commits pre-merge if using circleci
2. Determine JDK's that need to be run pre-merge
3. Update the circleci config files to match this by default
4. Determine a majority consensus on whether circle should auto-run on push or not, and integrate a "one-command" way to toggle to the alternate version + correct provisioning size (low, mid, high) to smooth out dev workflows and satisfy multiple competing personas
5. Document the above in the [Cassandra confluence wiki|https://cwiki.apache.org/confluence/display/cassandra/] and [testing|https://cassandra.apache.org/_/development/testing.html] sites
Bug: python-driver - There are "DeprecationWarning" during the run
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 19/Jan/22 10:39
Resolved: 19/Jan/22 11:32

1.
{code:java}
cassandra/io/asyncioreactor.py:178: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead {code}
2.  
{code:java}
tests/integration/standard/test_cluster.py:153: DeprecationWarning: invalid escape sequence \( 08:27:34  with self.assertRaisesRegexp(NoHostAvailable, "OperationTimedOut\('errors=Timed out creating connection \(1 seconds\)"):{code}
 

3. A user that runs the tests via Pytest will see the following warning:
{code:java}
tests/integration/__init__.py:1022
PytestCollectionWarning: cannot collect test class 'TestCluster' because it has a __new__ constructor (from: tests/integration/standard/test_authentication_misconfiguration.py)
08:27:34      class TestCluster(object): {code}
 

My PR will fix those issues

[https://github.com/datastax/python-driver/pull/1118]
Bug: sstableloader ignores the storage port option
Status: Triage Needed
Priority: Normal
Created: 17/Jan/22 11:49

sstableloader silently ignores the storage port option and connects to the storage port of the server.

E.g.
{code:java}
sstableloader -v -d 192.168.1.24 -p 32181 -sp 32182 -u USER -pw PASSWORD backups/test-workspace/test-table-9150b730742911ec9d011fb0ef4f5206 {code}
Here sstableloader will ignore the storage port 32182 and try to connect to port 7000, if the server reports itself listening on port 7000.

 

This seems natural (server listens on port 7000 so just use it), but it does not work, when Cassandra runs in a Docker container with ports mapped - a normal cases for orchestrators (Kubernetes, Mesos).

E.g.
{code:java}
docker run -it \
  -p 32181:9042 \
  -p 32182:7000 \
cassandra:4.0.1{code}
Here the storage port will be 7000 _inside_ the container (and Cassandra server exposes it), but _outside_ the container the port is 32182.

 

The code ({*}NativeSSTableLoaderClient{*} - [link|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java#L83]):
{code:java}
Set<Host> endpoints = metadata.getReplicas(Metadata.quote(keyspace), tokenRange);
...
for (Host endpoint : endpoints)
{
    int broadcastPort = endpoint.getBroadcastSocketAddress().getPort();
    // use port from broadcast address if set.
    int portToUse = broadcastPort != 0 ? broadcastPort : storagePort;
... {code}
It effectively ignores the {*}storagePort{*}, taken from CLI option. I believe, CLI option should have a priority here - let the user to decide what he wants.
Bug: Add diagnostic events for unavailable errors
Status: Patch Available
Priority: Normal
Created: 04/Jan/22 13:57

Errors from unavailables during query execution lack important details on the context why the error occurred. Emitting diagnostic events on unavailables will allow you to attach to the instance during troubleshooting and observe the type of operation (what kind of reads/writes), keyspace replication settings and down nodes as seen by the instance throwing the error.
Bug: Revise use of pytz in Python >= 3.9 
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 14/Mar/22 15:36
Resolved: 05/Jun/23 19:21

PEP 615 – Support for the IANA Time Zone Database in the Standard Library class ZoneInfo

PEP 615 (Python 3.9) has obsoleted the 3rd party Olsen tz library 'pytz' with support for the IANA Time Zone Database.  The code which imports this in cqlshmain.py and tests in [test_cqlsh_output.py|https://github.com/apache/cassandra/pull/1493/files/9c658a20c669d11a54ecc6b42ba083da13de0103#diff-f5a3955faadf50a1292df481044b83cefc44b2dac46676022c80bad076491a50] should be updated accordingly.

This can be done with something like:

if sys.version_info >= (3,9):

    try:

        import zoneinfo

else:

    try:

        import pytz

        ...
Bug: Trunk warnings: Unsafe is internal proprietary API and may be removed in a future release
Status: Open
Priority: Low
Created: 03/Jan/22 14:22

We see on trunk the following warnings:

 
{code:java}
_simulator-bootstrap_build:
    [mkdir] Created dir: /Users/ekaterina.dimitri/IdeaProjects/april/cassandra/build/classes/simulator-bootstrap
    [javac] Compiling 1 source file to /Users/ekaterina.dimitri/IdeaProjects/april/cassandra/build/classes/simulator-bootstrap
    [javac] /Users/ekaterina.dimitri/IdeaProjects/april/cassandra/test/simulator/bootstrap/org/apache/cassandra/simulator/systems/InterceptorOfSystemMethods.java:29: warning: Unsafe is internal proprietary API and may be removed in a future release
    [javac] import sun.misc.Unsafe;
    [javac]                ^
    [javac] /Users/ekaterina.dimitri/IdeaProjects/april/cassandra/test/simulator/bootstrap/org/apache/cassandra/simulator/systems/InterceptorOfSystemMethods.java:192: warning: Unsafe is internal proprietary API and may be removed in a future release
    [javac]         public static Unsafe getUnsafe()
    [javac]                       ^
    [javac] /Users/ekaterina.dimitri/IdeaProjects/april/cassandra/test/simulator/bootstrap/org/apache/cassandra/simulator/systems/InterceptorOfSystemMethods.java:196: warning: Unsafe is internal proprietary API and may be removed in a future release
    [javac]                 Field field = Unsafe.class.getDeclaredField("theUnsafe");
    [javac]                               ^
    [javac] /Users/ekaterina.dimitri/IdeaProjects/april/cassandra/test/simulator/bootstrap/org/apache/cassandra/simulator/systems/InterceptorOfSystemMethods.java:198: warning: Unsafe is internal proprietary API and may be removed in a future release
    [javac]                 return (Unsafe) field.get(null);
    [javac]                         ^
 

    [javac] 4 warnings
{code}
Bug: Fail starting when the same parameter exists more than once in cassandra.yaml 
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 14/Feb/22 02:06
Resolved: 29/Apr/22 08:40

The way that SnakeYAML works, if someone has added the same parameter more than once - the last occurrence will be the one that will take precedence. 

Now after CASSANDRA-15234 we can even add the parameter with the old name and with the new name and the occurrences will replace each other. Again, whatever is last in cassandra.yaml will take precedence. Example:

If you add the following in cassandra.yaml
{code:java}
hinted_handoff_enabled: true
enabled_hinted_handolff: false
{code}
you will get loaded in Config - 
{code:java}
hinted_handoff_enabled: false{code}
 //there is backward compatibility from the old name to load into the new one

Currently Cassandra prints in the logs what config was loaded but it is good also to detect the case mentioned and emit a warning for the user so they can verify that the value they wanted was loaded in config.

To do that you might want to look at the PropertiesChecker and the way we emit other warnings in [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] in YamlConfigurationLoader.
Bug: Fail build when Ant is not of a certain version
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 10/Mar/22 12:01
Resolved: 14/Mar/22 19:11

As of writing this ticket, trunk is known to be buildable with Ant 1.10 and users have reported build failures on Ant 1.9 (see CASSANDRA-16831).

There should be a check which fails the build if Ant used to build Cassanra source code was not at least 1.10.
Bug: Add support for short form of version to CQLSH
Status: Resolved
Priority: Low
Resolution: Fixed
Created: 04/Jan/22 16:46
Resolved: 09/Feb/22 18:13

Today we do only support the `–version` long option/form for cqlsh and this enhancement Jira is to request that we also offer a shorter version `-v` to cqlsh. This will have consistency benefits with other tools and even match with what we have at `bin/cassandra -v` option for instance.

Today, `cqlsh` does support `--v` to get the version which is different than the single dashed short form that is available at many other tools. Thanks to Ekaterina for finding this. It looks like this is stemming from Python's parse mechanism which is detailed here, [https://docs.python.org/2.7/library/optparse.html#printing-a-version-string].

 

[https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L194-L196]
{quote}parser = optparse.OptionParser(description=description, epilog=epilog,
                               usage="Usage: %prog [options] [host [port]]",
                               version='cqlsh ' + version)
{quote}
 

{{$ bin/cqlsh --v}}
{{cqlsh 6.0.0}}

This looks like a weird implementation at Python. Both (\-\-help) and (\-\-version) options are stemming from here, [https://github.com/python/cpython/blob/2.7/Lib/optparse.py#L1248-L1256] and they did decide to ignore the short form option for version and it somehow automatically takes the (--v) option to spit the version info.
Bug:  Log client address when detecting unknown exception in client networking
Status: Resolved
Priority: Urgent
Resolution: Fixed
Created: 10/Nov/24 10:46
Resolved: 11/Nov/24 12:28

We are getting hundreds of the below warnings in Cassandra logs: 
{code:java}
WARN  [epollEventLoopGroup-5-7] 2024-11-10 00:26:24,296 ExceptionHandlers.java:139 - Unknown exception in client networking
io.netty.channel.StacklessClosedChannelException: null
        at io.netty.channel.AbstractChannel.close(ChannelPromise)(Unknown Source)
{code}
This means the client dropped the connection abruptly or abnormally. The issue could be a network or application issue on the client side. 
Since there could be many client servers, Cassandra must log the client IP and the port so we can check what's the problem with that server. 

I'll attached the fix for this small change.
Bug: snakeyaml1.26 still exists in apache-cassandra-5.0.2
Status: Resolved
Priority: Urgent
Resolution: Duplicate
Created: 20/Nov/24 13:37
Resolved: 20/Nov/24 13:40

CVE-2022-1471 BDSA-2022-3447 9.8 Critical

apache-cassandra-5.0.2/lib/snakeyaml-1.26.jar

 

CVSS score:-9.8 Critical
Bug: jackson-databind2.13.2.2 still exists in apache-cassandra-5.0.2
Status: Resolved
Priority: Urgent
Resolution: Duplicate
Created: 20/Nov/24 13:25
Resolved: 20/Nov/24 13:33

Component name - jackson-databind
Component version name - 2.13.2.2
CVE-2022-42004 - 7.5
CVE-2023-35116 - 4.7

Archive Context and Path
apache-cassandra-5.0.2/lib/jackson-databind-2.13.2.2.jar

 

Above CVE's are still applicable for the jackson-databind2.13.2.2 in apache-cassandra-5.0.2

CVSS score is 7.5 High
Bug: Harry 2.0, Make it easier to reuse generators, make Harry more extensible and accord-proof, refactor Harry's major subsystems
Status: Resolved
Priority: High
Resolution: Fixed
Created: 14/Nov/24 19:07
Resolved: 27/Nov/24 13:27

Make it easier to reuse generators, make Harry more extensible and accord-proof, refactor Harry's major subsystems:

  * SELECT is now a regular Harry operation, not a separate "validate" step
  * Flushes (or any custom operation) are now a part of a model, Harry will run them for you
  * Simplified validation of subset/wildcard queries
  * All Operations and are now SchemaSpec-agnostic, and can use _any_ generators (like QT or whatever)
  * You read it right, Harry now can use _any_ generator and _still_ be able to not to keep generated values in memory
  * There are no concepts of “descriptor selectors”, or “bijection” exposed anymore. You just provide a Generator, Harry figures out the rest.
  * _All_ Cassandra data types are now supported
  * _No_ upper limit on the number of columns (also, clustering columns)
  * Basic (for now) support for Transactions
  * It is now possible to visit _multiple_ partition in the same Visit (essentially, for transactions)
  * Much much simpler to generate synthetic workloads now (basically, you just write a Generator<Operation>
  * Add support for _all_ single-cell types.

Most of it was just a refactoring of Harry, to allow the following (still pending):
  * Easier to add subset SELECTION (important for SAI testing)
  * Easier add support for functions
  * Easier to add support for collections and UDTs
  * Easier to add support for LET bindings