av9ash/GitBugs huggingface dataset:
Bug: Circle move git:// to https://
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Jan/22 09:59
Resolved: 12/Jan/22 02:14

Effective today this https://github.blog/2021-09-01-improving-git-protocol-security-github/ is being enforced which fails all our circle builds on the clone stage:

{noformat}
Cloning into '/home/cassandra/cassandra'...
fatal: remote error: 
  The unauthenticated git protocol on port 9418 is no longer supported.
Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.

Exited with code exit status 128
CircleCI received exit code 128
{noformat}

Tests show that replacing git:// to https:// fixes the problem.

CC [~mck] as it may also affect ci-jenkins
Bug: cleanup cqlshlib unit test failures and warnings
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 19/Feb/22 02:12
Resolved: 09/Mar/22 16:16

The cqlshlib unit tests have some regular failures and warnings.
 # test_copyutil.py fails with too many open files when ulimit is 256
 # cassconnect.py:39: DeprecationWarning: Legacy execution parameters
 # sslhandling.py:42: DeprecationWarning: The SafeConfigParser class has been renamed to ConfigParser 
 # remove unused reference to nosetests.xml in Jenkinsfile

 

The cqlshlib test test_copyutil.py fails on MacOS with 

*E           OSError: [Errno 24] Too many open files*

This happens because the test does not close the file descriptors after use.  The default ulimit on MacOS is small, 256:

     % ulimit -n

     256

The simple fix is to close the pipe after use.  Increasing ulimit nofiles works also, but shouldn't be necessary.
Bug: Add Native Transport Rate Limiter Options to Example cassandra.yaml and Expose Metric for Dispatch Rate
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Mar/22 19:22
Resolved: 21/Mar/22 16:41

In CASSANDRA-16663, we added a new rate limiter that applies to all incoming requests over the native transport. Given it has now had some time to marinate in 4.1 snapshot builds, there are a few things we should do to make sure it hits the bar for usability and observability:

1.) Add the {{native_transport_rate_limiting_enabled}} and {{native_transport_max_requests_per_second}} options to the example {{cassandra.yaml}} file w/ brief documentation.

2.) Add a simple Meter/metric, probably in {{ClientMetrics}} for the rate* of native transport requests dispatched for processing. Given there are a few cases where native transport requests may not translate 1:1 to the number of times we hit StorageProxy, having this metric should make verifying the correct operation of the limiter easier. (Ex. IN queries w/ multiple partition keys, {{GROUP BY}} queries, etc.) It may make sense to look further into adjusting the behavior of the rate limiter to take these kinds of queries into account, but that is a bit more complex than either of these 2 items, and deserves another Jira.

\* We do expose a per-connection counter for the number of requests dispatched, and it is exposed over a virtual table and JMX, but even if it were global, it is not a rate metric.
Bug: Runtime-configurable YAML option to prohibit USE statements
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Jan/22 23:49
Resolved: 04/Feb/22 20:26

The ambiguity around prepared statements without fully qualified table names has led to a number of sub-optimal outcomes for our users. With CASSANDRA-15252 and CASSANDRA-17248 resolved, there aren't any lingering _known_ issues, but operators may still want to cordon {{USE}} statements off completely. This is especially important if we plan to deprecate and remove {{USE}} at some point in the future.

A couple details...

1.) The new option will be opt-in.
2.) It will be possible to change at runtime.
3.) There will be a new metric for {{USE}} that tells us whether it's been used at all. (This should make it easier to disable {{USE}} for an existing cluster where we don't know a-priori that it isn't in play.)
Bug: Test Failure: unit test compression: testTokenGenerator_single_rack_or_single_rf
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 25/Jan/22 16:45
Resolved: 23/May/22 17:03

org.apache.cassandra.dht.tokenallocator.OfflineTokenAllocatorTest

[https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/168/workflows/1d30a113-c14b-4cf4-a631-bedb9eb65762/jobs/1447]

Looks like a pretty simple / straightforward timeout

 {code}
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout.
	at java.util.Vector.forEach(Vector.java:1277)
	at java.util.Vector.forEach(Vector.java:1277)
	at java.util.Vector.forEach(Vector.java:1277)
	at org.apache.cassandra.anttasks.TestHelper.execute(TestHelper.java:53)
	at java.util.Vector.forEach(Vector.java:1277)
{code}
Bug: On CFS.flushLargestMemtable, log candidate at info instead of debug
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Mar/22 16:39
Resolved: 22/Mar/22 19:20

When flushing the largest memtable in {{{}ColumnFamilyStore.flushLargestMemtable{}}}, we should log at info that a memtable was flushed due to this case rather than at debug. We don't expect this to trigger often and this information is useful for operators and shouldn't require elevated logging to see it.
Bug: Timestamp issue with Cassandra 4.0.3 with Timezone value
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Mar/22 19:28
Resolved: 23/Mar/22 14:30

Hello Team ,

 

Facing one issue related to the timestamp type in cassandra 4.0.3 version . This is not repro in 3.11.x and works fine in 3.11.

 

when i have table with timestamp as fields :

 

CREATE TABLE timetest (

    id int PRIMARY KEY,

    enddate timestamp,

    startdate timestamp

)

*Now when try to insert the record like :*

INSERT INTO timetest (id,startdate,enddate) VALUES (1 ,'2022-03-20 12:48:56 +0530','2022-03-20 12:48:56 +0530')

I am receiving error:

 

"{color:#de350b}InvalidRequest{color}: Error from server: code=2200 [Invalid query] message="Unable to parse a date/time from '2022-03-20 12:48:56 +0530'".

 ** 

*This works fine on 3.11 but fails on 4.x.*

 

*same issue if we query the record:*

 

select * from timetest where id = 1 and enddate = '2022-03-20 12:48:56 +0530';

{color:#de350b}InvalidRequest{color}: Error from server: code=2200 [Invalid query] message="Unable to parse a date/time from '2022-03-20 12:48:56 +0530'"

 

 

{color:#ffab00}*If we change the timestamp value and remove the extra space which is present before the timezone value then it works fine in 4.x*{color}

 

if we change timestamp value From {'2022-03-20 12:48:{color:#de350b}56 +0530{color}'}  To    {'2022-03-20 12:48:{color:#ffab00}56+0530{color}’} then 4.0.3 accept this value otherwise not. but 3.11 was accepting both the values.

 

This is causing a challenge for upgrading from 3.11 to 4.x.

 

Seeking help on this issue and any fix / workaround or suitable Cassandra version which is higher than 3.11 but lesser then 4.0.3 which can be used to fix this issue asap.
Bug: Remove global Guardrails Enable flag
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 29/Mar/22 19:15
Resolved: 12/Apr/22 10:54

This ticket removes the global Guardrails enable flag. Currently the flag turns all Guardrails on and off regardless of the individual setting of the guardrail property. This presents a problem for maximum replication factor and minimum replication factor configurations which will soon be moved to guardrails. Those configurations will always need to be enabled so no problems arise as Cassandra users create keyspaces. This ensures all Guardrail properties follow the same enable / disable procedure.
Bug: For rpm packaging replace centos8 docker images with almalinux
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 31/Jan/22 22:59
Resolved: 01/Feb/22 14:48

Centos went EOL, and CI started failing today with
{code}
Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
{code}
ref: https://ci-cassandra.apache.org/job/Cassandra-trunk-artifacts/1010/jdk=jdk_1.8_latest,label=cassandra/consoleFull

Switching to almalinux seems one common path forward…
Bug: Shutdown all ScheduledExecutors as part of node drainage
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Mar/22 19:43
Resolved: 14/Apr/22 07:52

We are currently shutting down only non-periodic executors in StorageService#drain. We should shut down all of them. As of now, there does not seem to be any reason why these executors should be active.
Bug: Hint documentation page has a misaligned table of settings / description / default values
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/Feb/22 21:18
Resolved: 04/Feb/22 13:51

Within the versioned documentation, the page for configuring hints contains a misaligned table of hint settings, caused the the hinted_handoff_disabled_datacenters row, which shifts all columns 1 further, mis-aligning the rest of the content and then removing the final cell. Screenshot attached of the start of the misalignment on the table.
Bug: Fixes for intermittent in-JVM dtest failures
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 12/Jan/22 00:01
Resolved: 03/Feb/22 21:35

Improvements to StreamingTransferTest, MixedModeMessageForwardTest and SchemaDisagreementTest flakes.

For 3.0, adopt the ring settling properties used on trunk. For all releases increase to 15s for ring settling.
For all branches, disable autocompaction during shutdown.
For 4.0 and up, shut down the scheduled executors a little later.
Bug: python unit tests don't cleanup test keyspace
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Mar/22 17:57
Resolved: 20/Apr/22 19:37

The python unit tests in pylib/cqlshlib creates a temporary keyspace cqlshtests_xxxxxxxx, but does not drop it upon completion.

To reproduce:
{code:java}
$ pytest
$ cqlsh
cassandra@cqlsh> describe keyspaces
cqlshtests_oatjjlyyxh ...
{code}
The above keyspace, created with a 10 character random suffix, will remain after the tests are run.  This will then cause subsequent runs of test_cqlsh_completion to fail.

 
{code:java}
cqlshlib/test/test_cqlsh_completion.py:138: in _trycompletions_inner
E   AssertionError: Items in the second set but not the first:
E   'cqlshtests_oatjjlyyxh'
{code}
test/cassconnect.py;'s remove_db() has the code to perform this, but does not seem to be properly called.
Bug: Snapshot true size is miscalculated
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 19/Jan/22 01:38
Resolved: 14/Mar/22 20:26

As far as I understand, the snapshot "size on disk" is the total size of the snapshot, while the "true size" is the (size_on_disk - size_of_live_sstables).

I created a snapshot on a 3.11 node without traffic and I expected the "true size" to be 0KB since the original sstables were still present, but this didn't seem to be the case:
{noformat}
$ nodetool listsnapshots
Snapshot Details:
Snapshot name Keyspace name Column family name True size Size on disk
test          ks1           tbl1               4.86 KiB  5.69 KiB

Total TrueDiskSpaceUsed: 4.86 KiB
{noformat}
Bug: OWASP failures, 3/2022
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Mar/22 11:52
Resolved: 22/Mar/22 12:44

On trunk there are a few failures:

{quote}
Dependency-Check Failure:
One or more dependencies were identified with vulnerabilities that have a CVSS score greater than or equal to '1.0': 
cassandra-driver-core-3.11.0-shaded.jar: CVE-2021-44521
jackson-databind-2.12.5.jar: CVE-2020-36518
See the dependency-check report for more details.
{quote}

Notably, we have bitten ourselves with CVE-2021-44521, which the driver has echoed, which we depend on.
Bug: Fix user_functions_test.py::TestUserFunctions::test_udf_overload
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Feb/22 22:34
Resolved: 17/Feb/22 20:08

user_functions_test.py::TestUserFunctions::test_udf_overload
is failing after CASSANDRA-17190 on all branches except trunk.
Bug: testUnwriteableFlushRecovery is never called
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Mar/22 08:12
Resolved: 31/Mar/22 08:29

`testUnwriteableFlushRecovery` of `CommitLogTest` was implemented 6 years ago and seems to became a dead code immediately. It should be either enabled back or removed. The source of the test is at https://github.com/apache/cassandra/blob/2b2c6decfafc6235ad537e72073fab2fd4467e2f/test/unit/org/apache/cassandra/db/commitlog/CommitLogTest.java#L932
Bug: Remove unused dependencies from pylib/requirements.txt
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Jan/22 16:34
Resolved: 20/Jan/22 16:04

The dependency 'cql' appears to be obsolete in requirements.txt and is not imported by the python code.

The package 'cql' was last updated in 2012 and has been deprecated by its authors

     [https://pypi.org/project/cql|https://pypi.org/project/cql/]    (see release history)

     _A DB-API 2.0 compliant client library for Cassandra/CQL_

     _This driver has been {*}deprecated{*}. Please use python-driver [https://github.com/datastax/python-driver] instead._

    above from homepage at  [https://code.google.com/archive/a/apache-extras.org/p/cassandra-dbapi2|https://code.google.com/archive/a/apache-extras.org/p/cassandra-dbapi2_]
Bug: Flatten guardrails configuration
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 07/Feb/22 17:03
Resolved: 09/Feb/22 18:32

Modify guardrails configuration at {{cassandra.yaml}} to have a flat format, instead of the current nested format. This ticket comes from the discussion around guardrails config on CASSANDRA-17292, CASSANDRA-17188 and CASSANDRA-17212, and doesn't include any of the other nested properties on {{cassandra.yaml}}.
Bug: Test Failures: write_failures_test.TestMultiDCWriteFailures.test_oversized_mutation
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Mar/22 20:07
Resolved: 09/May/22 09:05

https://ci-cassandra.apache.org/job/Cassandra-trunk/1002/testReport/dtest-offheap.write_failures_test/TestMultiDCWriteFailures/test_oversized_mutation/
{code:java}
Error Message
AssertionError: assert 0 == 8  +  where 8 = <bound method JolokiaAgent.read_attribute of <tools.jmxutils.JolokiaAgent object at 0x7f1fca78dac0>>('org.apache.cassandra.metrics:type=Storage,name=TotalHints', 'Count')  +    where <bound method JolokiaAgent.read_attribute of <tools.jmxutils.JolokiaAgent object at 0x7f1fca78dac0>> = <tools.jmxutils.JolokiaAgent object at 0x7f1fca78dac0>.read_attribute  +    and   'org.apache.cassandra.metrics:type=Storage,name=TotalHints' = make_mbean('metrics', type='Storage', name='TotalHints')
Stacktrace
self = <write_failures_test.TestMultiDCWriteFailures object at 0x7f1fca9dac40>

    def test_oversized_mutation(self):
        """
            Test that multi-DC write failures return operation failed rather than a timeout.
            @jira_ticket CASSANDRA-16334.
            """
    
        cluster = self.cluster
        cluster.populate([2, 2])
        cluster.set_configuration_options(values={'max_mutation_size_in_kb': 128})
        cluster.start()
    
        node1 = cluster.nodelist()[0]
        session = self.patient_exclusive_cql_connection(node1)
    
        session.execute("CREATE KEYSPACE k WITH replication = {'class': 'NetworkTopologyStrategy', 'dc1': 2, 'dc2': 2}")
        session.execute("CREATE TABLE k.t (key int PRIMARY KEY, val blob)")
    
        payload = '1' * 1024 * 256
        query = "INSERT INTO k.t (key, val) VALUES (1, textAsBlob('{}'))".format(payload)
    
        assert_write_failure(session, query, ConsistencyLevel.LOCAL_ONE)
        assert_write_failure(session, query, ConsistencyLevel.ONE)
    
        # verify that no hints are created
        with JolokiaAgent(node1) as jmx:
>           assert 0 == jmx.read_attribute(make_mbean('metrics', type='Storage', name='TotalHints'), 'Count')
E           AssertionError: assert 0 == 8
E            +  where 8 = <bound method JolokiaAgent.read_attribute of <tools.jmxutils.JolokiaAgent object at 0x7f1fca78dac0>>('org.apache.cassandra.metrics:type=Storage,name=TotalHints', 'Count')
E            +    where <bound method JolokiaAgent.read_attribute of <tools.jmxutils.JolokiaAgent object at 0x7f1fca78dac0>> = <tools.jmxutils.JolokiaAgent object at 0x7f1fca78dac0>.read_attribute
E            +    and   'org.apache.cassandra.metrics:type=Storage,name=TotalHints' = make_mbean('metrics', type='Storage', name='TotalHints')

write_failures_test.py:277: AssertionError
REST API
CloudBees CI Client Controller 2.319.3.4-rolling



{code}
Bug: DOCS - Update changed config + JVM, guide post CASSANDRA-15234
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Jan/22 01:30
Resolved: 18/Mar/22 18:49

When CASSANDRA-15234 gets committed we need to update everywhere in the docs the changed parameters names and we need to document:
 * new types
 * guide how to add new parameters
 * some information about the backward compatibility framework

Possibly missing some bullets, to be considered while doing the actual work.
Bug: Disallow removal of a DC from system_auth replication settings
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 23/Mar/22 16:46
Resolved: 28/Mar/22 18:04

It's never correct for an operator to remove a DC from system_auth replication settings while there are currently nodes up in that DC. We should disallow the ability to manually make this change, as if you need to decommission a DC you need to remove all the nodes from the cluster before allowing the replication settings change on system_auth.
Bug: Unshared heap size excluding data calculation is broken
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Feb/22 09:57
Resolved: 07/Mar/22 16:36

Unshared heap size excluding data calculation is broken. 

For example, it does not takes into consideration whether the buffers are on-heap or off-heap - empty offheap buffer is 64-bytes while empty heap buffer is 48-bytes. 

It matters especially when we have offheap buffers-based memtables.
Bug: Make vtables accessible via internode messaging
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Jan/22 17:34
Resolved: 09/Feb/22 01:09

As an extension of CASSANDRA-15399 and needed to solve CASSANDRA-15566, we need the ability to perform queries against vtables using internode messaging; currently this is limited to CQL which isn’t exposed in internode
Bug: Performance and Semantic Concerns w/ Metrics for Local vs. Remote Requests in StorageProxy
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Mar/22 22:49
Resolved: 29/Mar/22 16:59

In CASSANDRA-10023, we added two new metrics to both {{ClientRequestMetrics}} and {{ClientWriteRequestMetrics}} to represent requests where the driver either does or does not make a correct token-aware choice of coordinator. (Auditing driver behavior is listed as the primary goal of that Jira.)

There are, however, a few concerns we should address before this releases in 4.1:

1.) With paging enabled and a LIMIT < fetch size, {{IN}} queries can hit {{fetchRows()}} multiple times, so the number of local + remote requests isn’t the same as the number of queries marked in {{ClientRequestMetrics}} in {{readRegular()}}.

2.) {{IN}} queries will potentially mark a bunch of “remote” requests even if one key in the {{IN}} set is “local”.

3.) Something similar happens with mutations. If {{StorageProxy#mutate()}} receives multiple mutations, we’ll mark against one of these new metrics in {{ClientWriteRequestMetrics}} for each mutation, while {{ClientWriteRequestMetrics}} will only register the actual client request once.

For cases 2 and 3, we may mark both local and remote requests for the same overall client request, which introduces ambiguity if these are intended to help audit driver coordinator selection behavior. There are a few options:

a.) We can accept the ambiguity, but then we haven’t really accomplished the goal of CASSANDRA-10023 for some request types.

b.) We can simply not record any of these metrics for requests where multiple partitions/tokens are involved.

c.) We can be lenient, marking requests as “local” if any of the partitions/tokens involved in the client request are, in fact, local.

“c” feels like the option that preserves as much functionality as possible without being ambiguous, but problem #2 above is still tricky, given the way IN and GROUP BY queries behave w/ paging. (Perhaps ambiguity in that case is acceptable?)

In addition to the general ambiguity around the above…

4.) There is excessive object creation involved (on a hot path) in our determination of whether a request is local or remote. We should be able to mitigate this by getting rid of {{AbstractReadExecutor#getContactedReplicas()}} and relying on {{ReplicaPlan#lookup()}} rather than creating strings. (Even for writes, we should be able to push down marking into performWrite(), where the write ReplicaPlan is already available.)
Bug: Replace use of 'six' compatibility library with python 3 equivalents
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 04/Mar/22 20:43
Resolved: 30/Mar/22 17:15

_Six_ is a _Python_ 2 and 3 compatibility library.  It provides simple utilities for wrapping the differences between Python 2 and Python 3. It is intended to support codebases that work on both Python 2 and 3.

Since CQLSH requires python version >= 3.6, its use can be replaced with native python 3 constructs.
Bug: Provide JMX endpoint for transient full logging of blocking read repairs
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Mar/22 15:40
Resolved: 13/Apr/22 15:13

While working with live clusters that have elevated blocking read repairs (a not uncommon scenario), it'd be helpful to have the ability to enable full logging of blocking read repairs for a finite amount of time to get insight into what's going on without having to heavily disrupt a cluster with a bounce or config change.