av9ash/GitBugs huggingface dataset:
Bug: Fix test: dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test.TestDropCompactStorage.test_drop_compact_storage_mixed_cluster
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Oct/21 17:50
Resolved: 02/Dec/21 23:27

!https://ci-cassandra.apache.org/static/a177fe56/images/32x32/health-80plus.png! Failed 28 times in the last 28 runs. Flakiness: 0%, Stability: 0%
  
 Example of failure: [https://ci-cassandra.apache.org/job/Cassandra-trunk/801/testReport/junit/dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test/TestDropCompactStorage/test_drop_compact_storage_mixed_cluster/]
   
{code:java}
upgrade_tests/drop_compact_storage_upgrade_test.py:149: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <upgrade_tests.drop_compact_storage_upgrade_test.TestDropCompactStorage object at 0x7fa0e7f1ceb0>
session = <cassandra.cluster.Session object at 0x7fa0e7c56d30>
assert_msg = 'Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying.'

    def drop_compact_storage(self, session, assert_msg):
        try:
            session.execute("ALTER TABLE drop_compact_storage_test.test DROP COMPACT STORAGE")
            pytest.fail("No exception has been thrown")
        except InvalidRequest as e:
>           assert assert_msg in str(e)
E           assert 'Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying.' in 'Error from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127....1:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying."'
E            +  where 'Error from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127....1:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying."' = str(InvalidRequest('Error from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in the...1:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying."'))

upgrade_tests/drop_compact_storage_upgrade_test.py:45: AssertionError
{code}
Bug: Add a virtual table for exposing batch metrics
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 21/Dec/21 09:50
Resolved: 10/Feb/22 08:55

There are some existing metrics within BatchMetrics that provide metrics about related to the different types of Batches being executed. We should expose those through a virtual table.

+Additional information for newcomers:+

A new class should be created for representing the new Virtual Table in {{org.apache.cassandra.db.virtual}}. You can look at {{ThreadPoolsTable}} for an example of virtual table implementation and to {{TableMetricTables}} for how Histogram metrics can be exposed. The new table will need to be registered in {{SystemViewsKeyspace}}.
Some unit tests should be added. For some example of virtual table tests you can look at {{SystemPropertiesTableTest}}.
Bug: Guardrail to disable read-before-write list operations
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Nov/21 18:42
Resolved: 17/Dec/21 17:02

Add guardrail to disable list operations implying read-before-write. For example:
{code}
# Whether read-before-write operation is allowed, eg. setting list element by index, removing list element.
# Defaults to true to allow read before write operations
read_before_write_list_operations_enabled: true
{code}
Bug: Make nodes more resilient to local unrelated files during startup
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 28/Oct/21 19:35
Resolved: 01/Dec/21 17:31

There's a few things we can protect against being in the data dir on startup that might be around from older activity, tool usage, exports, etc on a 3.x -> 4.x update:
 a) file ending with *-old.json
 b) file ending with *.json or *idx.json

A trivial update to the filter on SSTableHeaderFix.java should protect against hitting these types of files on startup and throwing.
Bug: File leaks will not be be detected and released due to strong self-references in the tidier
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Dec/21 17:16
Resolved: 17/Dec/21 16:24

LogTransaction.SSTableTidier holds a reference to a {{Tracker}} which holds references to both a {{ColumnFamilyStore}} and a {{View}}, both of which hold refs to SSTableReaders. As per the comment at the top of the SSTableTidier:
{quote}// must not retain a reference to the SSTableReader, else leak detection cannot kick in
{quote}
We shouldn't hold a reference to the Tracker here; long running unit tests w/-Dcassandra.debugrefcount=true had this pop up.

{code}ERROR [Strong-Reference-Leak-Detector:1] 2020-10-27T01:10:12,421 NoSpamLogger.java:97 - Strong self-ref loop detected{code}
Bug: Avoid unecessary array allocations and initializations in RequestValidations
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Dec/21 11:29
Resolved: 20/Dec/21 13:08

{{RequestValidations}} methods used extensively varargs. As the array creation is necessary only when the check fail we endup creating most of the time unecessary array. This can easily be avoided by overloading the different methods with a number of message argument from zero to three.
Bug: Use com.google.common.collect.Lists instead of com.google.monitoring.runtime.instrumentation.common.collect.Lists in RowUtil.
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 04/Oct/21 08:06
Resolved: 12/Nov/21 06:25

Use com.google.common.collect.Lists instead of com.google.monitoring.runtime.instrumentation.common.collect.Lists in RowUtil.
Bug: Migrate thresholds for number of keyspaces and tables to guardrails
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/Dec/21 17:39
Resolved: 17/Dec/21 16:36

Migrate the existing thresholds for the number of keyspaces and tables:
{code}
# table_count_warn_threshold: 150
# keyspace_count_warn_threshold: 40
{code}
to a new guardrail under the guardrails section, for example:
{code}
guardrails:
    keyspaces:
        warn_threshold: 40
        abort_threshold: -1
    tables:
        warn_threshold: 150
        abort_threshold: -1
{code}
Please note that CASSANDRA-17147 has already added a guardrail for the number of tables, but the previous not-guardrail threshold for warning about the number of tables still exists.
Bug: WEBSITE - December 2021 blog post titled "Configurable Storage Ports and Why We Need Them"
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Dec/21 03:08
Resolved: 14/Jan/22 07:21

This ticket is to capture the work associated with publishing the December 2021 blog post for "Configurable Storage Ports and Why We Need Them"

If this blog cannot be published by the *December 28, 2021 publish date*, please contact me, suggest changes, or correct the date yourself when possible in the pull request for the appropriate time that the blog will go live (on blog.adoc and in the blog).
Bug: Guardrails for disk usage
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Nov/21 18:15
Resolved: 22/Apr/22 15:47

Add guardrails for disk usage establishing soft/hard limits on the percentage of used disk space. For example:
{code}
# Warning threshold to warn when local disk usage exceeds threshold. Valid values: (1, 100]
# Defaults to -1 to disable.
# disk_usage_percentage_warn_threshold: -1

# Failure threshold to reject write requests if replica disk usage exceeds threshold. Valid values: (1, 100]
# Defaults to -1 to disable.
# disk_usage_percentage_failure_threshold: -1
{code}
Bug: Make startup checks configurable
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 18/Dec/21 09:37
Resolved: 27/Feb/22 23:44

This ticket was created from needs discovered in CASSANDRA-17180. We want to be able to configure a startup check so we figured out that it is necessary to treat all startup checks same - to be able to configure them. This ticket is about making startup checks configurable.

Once this ticket is done, we can continue with the implementation of CASSANDRA-17180 where the implementation of gc grace check will be done.

We have identified that there is one check currently in place which needs to be changed to reflect this configuration implementation and that is FileSystemOwnershipCheck.

Because startup checks were not configurable before via means of a configuration file, they were configurable via system properties. This ticket does not aim to get rid system properties configuration mechanism, system properties will have precedence over settings in configuration file. Then, in the next release, I am aiming to get rid of system properties configuration mechanism.
Bug: Allow to GRANT or REVOKE multiple permissions in a single statement
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Oct/21 17:32
Resolved: 22/Oct/21 12:50

In order to GRANT or REVOKE multiple permissions we are currently forced to execute multiple requests. For example:
{code}
GRANT MODIFY ON KEYSPACE field TO manager;
GRANT SELECT ON KEYSPACE field TO manager;
{code}

We should be able to perform the same operations on a single request
{code}
GRANT MODIFY, SELECT ON KEYSPACE field TO manager;
{code}
Bug: Remove use of deprecated Files in tests
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Nov/21 21:23
Resolved: 31/Jan/22 16:23

From checkstyle:

{noformat}
      5  Illegal import - java.io.File. [IllegalImport]
      3  Illegal import - java.io.FileInputStream. [IllegalImport]
      2  Illegal import - java.io.FileOutputStream. [IllegalImport]
      3  Illegal import - java.io.FileWriter. [IllegalImport]
     16  Illegal import - java.io.RandomAccessFile. [IllegalImport]
{noformat}
Bug: CQLSSTableWriter backwards compatibility fix for Date fields
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 03/Nov/21 13:53
Resolved: 05/Nov/21 08:27

{{cassandra-all}} library consumers of the CQLSSTableWriter class cannot easily create an appropriate instance of the {{LocalDate}} class in order to write SSTables including dates, and the current implementation no longer accepts a {{int}} value as a valid input as previous versions of the class would - we used to use {{((AbstractType)columnSpecification.type).decompose(value);}} in order to serialize the value, but now we use the type codec's .serialize method. Unfortunately, this doesn't work when the consumer can't easily create the type needed (serialize, in the case of dates, takes a {{LocalDate}} instance which is not easy to construct outside of Cassandra internal code).

This can be worked around by catching the resulting {{ClassCastException}} thrown from the {{serialize}} call and falling back to the older {{decompose}} implementation, which would maintain backwards-compatibility with other users of the CQLSSTableWriter in cases where they are passing integers for date fields, which used to work.
Bug: Guardrail for page size
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 06/Dec/21 13:17
Resolved: 16/Dec/21 17:26

Add guardrail limiting the query page size, for example:
{code}
# Guardrail to warn about or reject page sizes greater than threshold.
# The two thresholds default to -1 to disable.
page_size:
    warn_threshold: -1
    abort_threshold: -1
{code}
Initially this can be based on the specified number of rows used as page size, although it would be ideal to also limit the actual size in bytes of the returned pages.

+Additional information for newcomers:+

# Add the configuration for the new guardrail on page size in the guardrails section of cassandra.yaml.
# Add a getPageSize method in GuardrailsConfig returning a Threshold.Config object
# Implement that method in GuardrailsOptions, which is the default yaml-based implementation of GuardrailsConfig
# Add a Threshold guardrail named pageSize in Guardrails, using the previously created config
# Define JMX-friendly getters and setters for the previously created config in GuardrailsMBean
# Implement the JMX-friendly getters and setters in Guardrails
# Now that we have the guardrail ready, it’s time to use it. We should search for a place to invoke the Guardrails.pageSize#guard method with the page size that each query is going to use. The DataLimits#forPaging methods look like good candidates for this.
# Finally, add some tests for the new guardrail. Given that the new guardrail is a Threshold, our new test should probably extend ThresholdTester.
Bug: ViewTests flaky on timeouts
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Nov/21 09:50
Resolved: 26/Nov/21 06:20

Recently, specially on trunk, View*Test have stared timing out. This is bad for CI credibility when submitting a patch so it's worth splitting them yet again.

Despite trunk being the main offender the patch will be also applied to 4.0 to be defensive about it.

As we can see [here|https://ci-cassandra.apache.org/job/Cassandra-trunk/861/testReport/junit/org.apache.cassandra.cql3/] the candidates are: ViewComplexLivenessTest, ViewComplexDeletionsTest, ViewComplexTest, ViewFilteringPKTest, ViewFilteringTest
Bug: Queries performed with NODE_LOCAL consistency level do not update request metrics
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 20/Oct/21 14:24
Resolved: 12/Nov/21 11:44

Currently, queries performed with {{NODE_LOCAL}} consistency level are not reflected in request metrics. The suggested patch addresses that, and also allows modification and batch statements to be used with {{NODE_LOCAL}} consistency level.
Bug: MessagingServiceTest listenOptionalSecureConnection and listenRequiredSecureConnection fail sporadically
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 11/Oct/21 17:34
Resolved: 09/Nov/21 17:02

This popped up in a recent CircleCI run: https://app.circleci.com/pipelines/github/maedhroz/cassandra/351/workflows/f8dbf599-df72-4982-8a12-a72a5b8ddd3b/jobs/2195

I was able to reproduce it locally on trunk as well in a single run, although it happens perhaps once per 3-4 runs of the MessagingServiceTest suite. It looks like we open the sockets in an InboundSockets container, await on the future that returns, and then assert that all those sockets are open. It’s not clear to me yet why this assertion fails sporadically, or what state changes exactly are racing.
Bug: Don't block gossip when clearing snapshots for failing repairs
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Nov/21 08:08
Resolved: 17/Jan/22 08:21

We clear snapshots in the GossipTasks thread when a repair session fails due to a replica shutting down. If there are many tables/repair sessions ongoing this can take a long time. With enough tables being repaired at the same time even checking if the snapshots exists can take long enough to mark nodes down.

We should clear snapshots in a separate thread and add a flag to tell us whether this repair session can have snapshots to avoid checking if the directory exists.
Bug: Fix startup issue with internode_application_timeout_in_ms
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Nov/21 21:16
Resolved: 16/Nov/21 15:46

While testing my patch for CASSANDRA-17131 I found that there is a problem with _internode_application_timeout_in_ms_ in 4.0 and trunk.

Seems to me that we can just safely remove it for now?
Bug: Fix test org.apache.cassandra.distributed.test.SecondaryIndexTest#test_only_coordinator_chooses_index_for_query
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 19/Nov/21 01:34
Resolved: 19/Nov/21 20:39

https://app.circleci.com/pipelines/github/dcapwell/cassandra/1140/workflows/3dff9cae-32e7-4e70-874e-6c672c563275/jobs/8355/tests

This test fails some times on j11

{code}
junit.framework.AssertionFailedError: expected:<[/127.0.0.1]> but was:<[]>
	at org.apache.cassandra.distributed.test.SecondaryIndexTest.test_only_coordinator_chooses_index_for_query(SecondaryIndexTest.java:104)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
Bug: Log read timeouts and unavailables at INFO via a no-spam logger
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Nov/21 18:53
Resolved: 18/Nov/21 21:50

We don’t have great logging visibility into timeouts and unavailables for reads and range reads. If we limit output to something like 1 per minute at INFO, we should be able to avoid spam but also have some record of what exactly failed. This could be especially helpful during node bounces and upgrades.
Bug: Migrate threshold for minimum keyspace replication factor to guardrails
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 15/Dec/21 14:46
Resolved: 29/Apr/22 03:15

The config property [{{minimum_keyspace_rf}}|https://github.com/apache/cassandra/blob/5fdadb25f95099b8945d9d9ee11d3e380d3867f4/conf/cassandra.yaml] that was added by CASSANDRA-14557 can be migrated to guardrails, for example:
{code}
guardrails:
    ...
    replication_factor:
        warn_threshold: 2
        abort_threshold: 3
{code}
Bug: Cannot restart a node when there are other nodes being down in in-jvm dtest framework
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Dec/21 07:04
Resolved: 26/Jan/22 12:56

Such scenario:

{code:java}
    @Test
    public void test() throws Exception
    {
        try (Cluster cluster = init(Cluster.build(2).withDataDirCount(1)).start()))
        {
            FBUtilities.waitOnFuture(cluster.get(2).shutdown());
            FBUtilities.waitOnFuture(cluster.get(1).shutdown());
            cluster.get(1).startup();
            cluster.get(2).startup();
        }
    }
{code}

throws

{noformat}
java.lang.IllegalStateException: Can't use shut down instances, delegate is null

	at org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:210)
	at org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.getMessagingVersion(DelegatingInvokableInstance.java:90)
	at org.apache.cassandra.distributed.action.GossipHelper.unsafeStatusToNormal(GossipHelper.java:95)
	at org.apache.cassandra.distributed.impl.Instance.lambda$null$10(Instance.java:640)
{noformat}

when we do not use {{Gossiper}} feature flag.
Bug: Flaky CompactionsBytemanTest
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 26/Nov/21 09:49
Resolved: 09/Dec/21 06:47

See [here|https://ci-cassandra.apache.org/job/Cassandra-trunk/868/testReport/junit/org.apache.cassandra.db.compaction/CompactionsBytemanTest/testCompactingCFCounting/]

{noformat}
junit.framework.AssertionFailedError: expected:<0> but was:<1>
	at org.apache.cassandra.db.compaction.CompactionsBytemanTest.testCompactingCFCounting(CompactionsBytemanTest.java:130)
	at org.jboss.byteman.contrib.bmunit.BMUnitRunner$10.evaluate(BMUnitRunner.java:393)
	at org.jboss.byteman.contrib.bmunit.BMUnitRunner$6.evaluate(BMUnitRunner.java:263)
	at org.jboss.byteman.contrib.bmunit.BMUnitRunner$1.evaluate(BMUnitRunner.java:97)
{noformat}
Bug: InitialConnectionHandler Configured pipeline: DefaultChannelPipeline spamming the logs
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 13/Oct/21 14:04
Resolved: 20/Oct/21 15:39

incoming client requests are generating DEBUG messages that spam the logs:

DEBUG [epollEventLoopGroup-5-30] 2021-10-12 04:55:01,014 InitialConnectionHandler.java:153 - Configured pipeline: DefaultChannelPipeline\{(envelopeDecoder = org.apache.cassandra.transport.Envelope$Decoder), (envelopeEncoder= org.apache.cassandra.transport.Envelope$Encoder), (decompressor = org.apache.cassandra.transport.Envelope$Decompressor), (compressor = org.apache.cassandra.transport.Envelope$Compressor), (messageDecoder =org.apache.cassandra.transport.PreV5Handlers$ProtocolDecoder), (messageEncoder = org.apache.cassandra.transport.PreV5Handlers$ProtocolEncoder), (legacyCqlProcessor =org.apache.cassandra.transport.PreV5Handlers$LegacyDispatchHandler), (exceptionHandler = org.apache.cassandra.transport.PreV5Handlers$ExceptionHandler)}
Bug: Fix formatting in  cassandra.yaml
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 09/Nov/21 20:32
Resolved: 10/Nov/21 21:48

We have a few parameters with small formatting issues in cassandra.yaml which are not caught because by default they are commented out.

Cassandra 4.0:
 * a few parameters are added with _parameter *=* value_ instead of _parameter *:* value_

trunk:
 * same issue as in 4.0
 * few parameters added with _;_ after the value
Bug: Add unix time conversion functions
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Oct/21 16:06
Resolved: 09/Nov/21 13:19

It will be useful to have some functions to conver unix time values into C* native types, like:

*     {{toDate(bigint)}} 
*     {{toTimestamp(bigint)}}
*     {{mintimeuuid(bigint)}}
*     {{maxtimeuuid(bigint)}}

+Additional info for newcomers:+

Those new functions need to be added to {{org.apache.cassandra.cql3.functions.TimeFcts}} and some unit tests need to be added to {{org.apache.cassandra.cql3.functions..TimeFctsTest}}. The other functions in {{TimeFcts}} provide some good example of how to create new functions.
As it is a new functionality they need to be mentioned in the {{NEWS.txt}} file.
Bug: startup fails if directories do not exist
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 29/Oct/21 15:24
Resolved: 14/Jan/22 17:56

Prior to CASSANDRA-16926, having commitlog and data dirs defined that did not exist would be created on startup, but now we throw:

{noformat}
Exception (org.apache.cassandra.exceptions.ConfigurationException) encountered during startup: Unable check disk space in 'bin/../data/commitlog'. Perhaps the Cassandra user does not have the necessary permissions
org.apache.cassandra.exceptions.ConfigurationException: Unable check disk space in 'bin/../data/commitlog'. Perhaps the Cassandra user does not have the necessary permissions
        at org.apache.cassandra.config.DatabaseDescriptor.lambda$tryGetSpace$3(DatabaseDescriptor.java:1188)
        at org.apache.cassandra.io.util.PathUtils.tryOnFileStore(PathUtils.java:639)
        at org.apache.cassandra.io.util.PathUtils.tryGetSpace(PathUtils.java:665)
        at org.apache.cassandra.config.DatabaseDescriptor.tryGetSpace(DatabaseDescriptor.java:1188)
        at org.apache.cassandra.config.DatabaseDescriptor.applySimpleConfig(DatabaseDescriptor.java:553)
        at org.apache.cassandra.config.DatabaseDescriptor.applyAll(DatabaseDescriptor.java:350)
        at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:178)
        at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:162)
        at org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:800)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:736)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:871)
{noformat}

This was at least convenient for development, but also may be relied upon by some tooling/automation.