av9ash/GitBugs huggingface dataset:
Bug: LEAK DETECTED: org.apache.cassandra.io.util.FileHandle$Cleanup in TestSecondaryIndexes.test_failing_manual_rebuild_index
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 11/May/23 10:32
Resolved: 11/May/23 11:20

A leak was detected in CI run: https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2344/workflows/1f57b9a0-3fc9-49c7-a821-52e24b025056/jobs/22547/parallel-runs/4/steps/4-106{{{}

==================================== ERRORS ===================================={}}}
{{_ ERROR at teardown of TestSecondaryIndexes.test_failing_manual_rebuild_index __}}
{{Unexpected error found in node logs (see stdout for full details). Errors: [[node1] 'ERROR [Reference-Reaper] 2023-05-05 17:57:29,429 Ref.java:237 - LEAK DETECTED: a reference (class org.apache.cassandra.io.util.FileHandle$Cleanup@1392096802:/tmp/dtest-nl3k0_2v/test/node1/data0/k/t-4374ee60eb6e11ed99961b44b0ab6f7e/nc-1-big-Index.db) to class org.apache.cassandra.io.util.FileHandle$Cleanup@1392096802:/tmp/dtest-nl3k0_2v/test/node1/data0/k/t-4374ee60eb6e11ed99961b44b0ab6f7e/nc-1-big-Index.db was not released before the reference was garbage collected', [node1] 'ERROR [Reference-Reaper] 2023-05-05 17:57:29,430 Ref.java:237 - LEAK DETECTED: a reference (class org.apache.cassandra.io.util.FileHandle$Cleanup@1379143890:/tmp/dtest-nl3k0_2v/test/node1/data1/k/t-4374ee60eb6e11ed99961b44b0ab6f7e/nc-2-big-Index.db) to class org.apache.cassandra.io.util.FileHandle$Cleanup@1379143890:/tmp/dtest-nl3k0_2v/test/node1/data1/k/t-4374ee60eb6e11ed99961b44b0ab6f7e/nc-2-big-Index.db was not released before the reference was garbage collected']}}
{{==============}}
Bug: Support GRANT REVOKE for DESCRIBE KEYSPACES / KEYSPACE / TABLE
Status: Triage Needed
Priority: Normal
Created: 10/May/23 18:35

Currently, users cannot REVOKE DESCRIBE KEYSPACE, based on the documentation here[1] and my test here[2]. This means that all users can describe all keyspaces and tables, even if they can't access the data within them.

It should be possible to prevent users from describing certain resources, since that leaks schema information. If a user is not permitted to DESCRIBE a certain resource, then DESCRIBE KEYSPACES should work correctly but exclude the revoked resources.

[1]: https://cassandra.apache.org/doc/latest/cassandra/cql/security.html#data-control
[2]: Example here:
```
cqlsh> GRANT DESCRIBE on KEYSPACE keyspace1 TO user1;                                                                   
SyntaxException: Resource type DataResource does not support any of the requested permissions
```
Bug: Make Cassandra more user-friendly (admin-friendly) by cleaning up logging
Status: Triage Needed
Priority: Normal
Created: 10/May/23 12:27

At a very high level we see two types of users:

1. Early adopters generally want to understand Cassandra's design, how the pieces fit together, and why
2. Mainstream users just want it to work, preferably with as little attention from them as possible
 
Group 1 loves verbose logs because it helps them figure out what's going on under the hood.  Group 2 sees verbose logs as intimidating ("this is going to be hard") if not scary ("this isn't debugged enough yet").
 
Early on, group 1 users predominate.  But there's way more group 2 users out there now and as Cassandra sees more adoption it will primarily come from them.
 
It's time to start optimizing for group 2.  Group 1 is, after all, completely capable of adjusting the log levels themselves when necessary.
 
A good rule of thumb is, "is this necessary information for my day-to-day operation of the system."  If not, it should be at debug (or sometimes trace).

Compare [our startup logging|https://gist.githubusercontent.com/jbellis/a1f324c145c4d46c3409969393b24077/raw/a767e4b92c29a51130940326de619a3c418096b7/gistfile1.txt] with [postgresql's.|https://gist.githubusercontent.com/jbellis/a9585d3f04f49c7e3685c9122975d51b/raw/beea37fd0834d1c8f38e5c1667623aa37598d3dd/gistfile1.txt]
Bug: ParseUtils can fail to handle wrong number
Status: Triage Needed
Priority: Normal
Created: 10/May/23 08:42

ParseUtils  has the following code:

 
{code:java}
public abstract class ParseUtils
{
   ...   
   static long parseTime(String str) throws ParseException
   {
        ...      
       int firstColon = str.indexOf(':');
       int secondColon = str.indexOf(':', firstColon + 1);       
       hour = Integer.parseInt(str.substring(0, firstColon));
       ...
   }
}          {code}
In the above code, Integer.parseInt can throw NumberFormatException, but this exception is not caught and handled. 

 

The calling method can pass invalid strings to this method. For example, TypeCodec calls this method:

 
{code:java}
@Override 
public Long parse(String value)
{
    //The below statement does not check whether the value is valid.
    if (value == null || value.isEmpty() || value.equalsIgnoreCase("NULL")) return null;

     //The below statement does not check whether the value is valid either.
    // enclosing single quotes required, even for long literals
     if (!ParseUtils.isQuoted(value))
         throw new InvalidTypeException("time values must be enclosed by single quotes");
      value = value.substring(1, value.length() - 1);

      if (ParseUtils.isLongLiteral(value))
      {
          try
          {
              return Long.parseLong(value);
           }
           catch (NumberFormatException e)//This check works only when the value already passes the check of isLongLiteral
       {
               throw new InvalidTypeException(
                 String.format("Cannot parse time value from \"%s\"", value), e);
          }
      }

      try
      {
           return ParseUtils.parseTime(value); //If this call throws NumberFormatException, the below catch statement will not catch the exception
     }
      catch (ParseException e)
      {
           throw new InvalidTypeException(
                String.format("Cannot parse time value from \"%s\"", value), e);
       }
 }

   {code}
 

 

Although the above code has several checks, none will work if the input is not an integer. When parseTime throws NumberFormatException, the thrown exception will not be handled.  I added my comments to the above code.
Bug: FunctionCall can throw more specific exceptions
Status: Open
Priority: Normal
Created: 27/Apr/23 08:52

FunctionCall has the following code:
{code:java}
 private static ByteBuffer executeInternal(ProtocolVersion protocolVersion, ScalarFunction fun, List<ByteBuffer> params) throws InvalidRequestException   
{
        ByteBuffer result = fun.execute(protocolVersion, params);
        try
        {
            // Check the method didn't lied on it's declared return type
            if (result != null)
                fun.returnType().validate(result);
            return result;
        }
        catch (MarshalException e)
        {
            throw new RuntimeException(String.format("Return of function %s (%s) is not a valid value for its declared return type %s",
                                                     fun, ByteBufferUtil.bytesToHex(result), fun.returnType().asCQL3Type()), e);
        }
    }

{code}
When validate throws MarshalException, it rethrows  RuntimeException. Other methods throw more specific exceptions. For example, BytesConversionFcts throws 
{color:#000000}InvalidRequestException:{color}
{code:java}
public ByteBuffer execute(ProtocolVersion protocolVersion, List<ByteBuffer> parameters)
        {
            ByteBuffer val = parameters.get(0);

            if (val != null)
            {
                try
                {
                    toType.getType().validate(val);
                }
                catch (MarshalException e)
                {
                    throw new InvalidRequestException(String.format("In call to function %s, value 0x%s is not a " +
                                                                    "valid binary representation for type %s",
                                                                    name, ByteBufferUtil.bytesToHex(val), toType));
                }
            }

            return val;
        }

 {code}
{color:#000000}{color:#000000}As another example, Validation also rethrows this exception:{color}{color}
{code:java}
public static void validateKey(TableMetadata metadata, ByteBuffer key)

{
        ...
        try
        {
            metadata.partitionKeyType.validate(key);
        }
        catch (MarshalException e)
        {
            throw new InvalidRequestException(e.getMessage());
        }
    }

   {code}
Bug: Enable single_sstable_uplevel by default for LCS
Status: Triage Needed
Priority: Normal
Created: 09/May/23 16:22

Single sstable uplevel option (single_sstable_uplevel) added to LCS with CASSANDRA-12526 is disabled by default. Considering the performance improvements is provides, I think it makes sense to enable it by default so that it can be leveraged without configuring explicitly.
Bug: LeveledCompactionStrategy does not check its constructor
Status: Resolved
Priority: Normal
Resolution: Not A Problem
Created: 28/Apr/23 06:05
Resolved: 08/May/23 16:11

LeveledCompactionStrategy  has the following code:

 
{code:java}
public static Map<String, String> validateOptions(Map<String, String> options) throws ConfigurationException
    {
        Map<String, String> uncheckedOptions = AbstractCompactionStrategy.validateOptions(options);

        String size = options.containsKey(SSTABLE_SIZE_OPTION) ? options.get(SSTABLE_SIZE_OPTION) : "1";
        try
        {
            int ssSize = Integer.parseInt(size);
            if (ssSize < 1)
            {
                throw new ConfigurationException(String.format("%s must be larger than 0, but was %s", SSTABLE_SIZE_OPTION, ssSize));
            }
        }
        catch (NumberFormatException ex)
        {
            throw new ConfigurationException(String.format("%s is not a parsable int (base10) for %s", size, SSTABLE_SIZE_OPTION), ex);
        }
...
} {code}
This method throws ConfigurationException when the configuration file is wrong. The thrown exception is easy to understand, and calling code can catch this exception to handle configuration files with wrong formats. However, the constructor of this class does not check configuration files:

 

 

 
{code:java}
 public LeveledCompactionStrategy(ColumnFamilyStore cfs, Map<String, String> options)

{    ... 
     
configuredMaxSSTableSize = Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
 ...

configuredLevelFanoutSize = Integer.parseInt(options.get(LEVEL_FANOUT_SIZE_OPTION));

...
}

{code}
As a result, this class can throw NumberFormatException when configuration files are wrong, and will not throw ConfigurationException. 

It is strange for calling code to call the static validation method before calling its constructor. Can this problem be fixed?
Bug: Connection without username not logged in auditlog 
Status: In Progress
Priority: Normal
Created: 03/Apr/23 11:58

Hi,
If making connection *without username* to cassandra cluster with PasswordAuthenticator enabled, 
Connection will fail but not logged on auditlog.

How to reproduce:
 # Enable "authenticator: PasswordAuthenticator" on cluster
 # Enable audit : "nodetool enableauditlog"
 # Open a new screen and run "auditlogviewer -f <log_location>/audit/"
 # Try to connect, and connection will fail:
{code:java}
[root@c1 ~]# cqlsh
Connection error: ('Unable to connect to any servers', {'127.0.0.1:9042': AuthenticationFailed('Remote end requires authentication',)}){code}
 # *But nothing in auditlogviewer*.

Connection with incorrect usernames or password logged correct on auditlog , the problem only on connection without username. 

How it's affecting:
 # Security reason, hard to find unauthorized connections attempt  .
 # When migrating cluster into PasswordAuthenticator, hard to find applications that didn't add username/password. 

Thank you.
Bug: Update testing for drop compact storage and CASSANDRA-15897
Status: Open
Priority: Normal
Created: 05/May/23 08:45

As observed in CASSANDRA-17869 we need to update drop compact storage tests

{{DropCompactStorageTest}} java distributed test does not verify that compact drop is not possible before upgradesstables. This we do in the python test drop_compact storage.

The python test drop_compact_storage tests also CASSANDRA-15897

Though the test is not multi-step upgrade test. It would make no sense for it be 3.x to 5.x (which we don't support).

But still we need to ensure we still verify things in the newer Cassandra versions as compact storage is still around; tests need to be updated to cover CASSANDRA-15897
Bug: Update the upgrade paths for storage_engine_upgrade_test
Status: Open
Priority: Normal
Created: 05/May/23 08:37

Current upgrade paths:
{code:java}
indev_2_2_x -> indev_3_0_x
indev_2_2_x -> indev_3_11_x
indev_3_0_x -> indev_4_0_x
indev_3_0_x -> indev_4_1_x
indev_4_0_x -> indev_5_0_x{code}
As observed in CASSANDRA-17869, we need to add also:
{code:java}
indev_3_11_x -> indev_4_0_x
indev_3_11_x -> indev_4_1_x
indev_4_1_x -> indev_5_0_x{code}
Bug: Warnings when using the perl driver to connect to Cassandra
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 03/May/23 08:56
Resolved: 03/May/23 15:08

When I use the [perl driver|https://github.com/TvdW/perl-DBD-Cassandra] to connect to Cassandra 4.0.3 and onwards, I get the following error just for initialization of a connection.

The error I get is
{noformat}
`USE <keyspace>` with prepared statements is considered to be an anti-pattern due to ambiguity in non-qualified table names. Please consider removing instances of `Session#setKeyspace(<keyspace>)`, `Session#execute("USE <keyspace>")` and `cluster.newSession(<keyspace>)` from your code, and always use fully qualified table names (e.g. <keyspace>.<table>). Keyspace used: null, statement keyspace: null, statement id: <id> at Cassandra/Client/Connection.pm line 957.
{noformat}
 

This is just from initialization of the perl driver connection while choosing the keyspace. [https://github.com/TvdW/perl-DBD-Cassandra/blob/master/Cassandra-Client/lib/Cassandra/Client/Connection.pm#L562] before running any queries. It is emitted even if I use qualified prepared statement.

The same warning does not pop up in Datastax java driver initialization.

On debugging, I found that this warning does not emit for all unqualified prepared statements. It only emits for unqualified prepared "USE ks" statement. But the "USE ks" can never be qualified. So the warning is a bit vague on what is the recommended approach.

And from the perspective of the driver, it is setting the keyspace of the connection for the first time. The same warning does not happen on the datastax java driver and that is because it uses QUERY to set the keyspace on connection. (I tried to follow the same approach on the perl driver - https://github.com/TvdW/perl-DBD-Cassandra/pull/35 )

The warnings are not very clear on what is deprecated and what is not. Does it deprecate only the use of prepared statement of "USE ks"? or does it deprecate "USE ks" completely? And it is not being emitted for other unqualified prepared statements but only for a USE statement which cannot be qualified at all.
Bug: Review documentation about needing to stop the server for sstable tools
Status: Open
Priority: Normal
Created: 03/May/23 15:14

For sstable tools the docs often mention:

bq. Cassandra must be stopped before this tool is executed, or unexpected results will occur. 

In CASSANDRA-8720 it was noticed this is not required, and we should probably review the instances of it that we have for accuracy.
Bug: Formatting looks broken on the metrics page
Status: Triage Needed
Priority: Normal
Created: 21/Apr/23 23:18

[https://cassandra.apache.org/doc/latest/cassandra/operating/metrics.html]

 

Things like "====[cols=",,",options="header",]" look odd, and I believe is a part of some format setting (row HTML?) and isn't supposed to be displayed.

I'm willing to help if there is a place where I can send a merge request.
Bug: CEP 15: Accord - Add a new Status of PreAcceptedInvalidated
Status: Open
Priority: Normal
Created: 02/May/23 16:44

If an invalidation is started and a replica does not know the transaction, then we get into a state of NotWitnessed(promised=[1, 2, 3, 4]).  The issue with this is that all logic relies on Status checks, so they hit NotWitnessed and do not treat this as Invalidated; this confusion causes bugs in different code paths.
Bug: MappedByteBufferIndexInputProvider can better throw UndeclaredThrowableException
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 28/Apr/23 06:49
Resolved: 28/Apr/23 11:02

MappedByteBufferIndexInputProvider has the following code:
{code:java}
 private static BufferCleaner newBufferCleaner(final MethodHandle unmapper) {

...

try {
        unmapper.invokeExact(buffer);
      } catch (Throwable t) {
        throw new IOException("Unable to unmap the mapped buffer: " + resourceDescription, t);
      }

 }{code}
This method rethrow IOException, when calling invokeExact fails. However, other methods throw UndeclaredThrowableException for the same errors. For example, SnowballProgram.find_among is as follows:
{code:java}
protected int find_among(Among v[]) {

 ..,
 
try {
          res = (boolean) w.method.invokeExact(this);
        } catch (Error | RuntimeException e) {
          throw e;
        } catch (Throwable e) {
          throw new UndeclaredThrowableException(e);
        }
}{code}
Another example is AttributeFactory.{color:#000000}createInstance{color}

:
{code:java}
 
 protected A createInstance() {
        try {
          // be explicit with casting, so javac compiles correct call to polymorphic signature:
          final AttributeImpl impl = (AttributeImpl) constr.invokeExact();
          // now cast to generic type:
          return (A) impl;
        } catch (Error | RuntimeException e) {
          throw e;
        } catch (Throwable e) {
          throw new UndeclaredThrowableException(e);
        }
      }

   {code}
UndeclaredThrowableException looks more reasonable.  Can this be fixed?
Bug: NativeTransportEncryptionOptionsTest fails with ant testsome, passes in intellij
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 24/Apr/23 09:38
Resolved: 27/Apr/23 08:46

environment:

linux, java 11:

 

{{$ set|grep JAVA}}
{{JAVA11_HOME=/home/jakub/.sdkman/candidates/java/11.0.11.hs-adpt}}
{{JAVA_HOME=/home/jakub/.sdkman/candidates/java/current}}
{{$ which java}}
{{/home/jakub/.sdkman/candidates/java/current/bin/java}}
{{$ java -version}}
{{openjdk version "11.0.11" 2021-04-20}}
{{OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)}}
{{OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)}}

repro steps:
 # clone cassandra (trunk HEAD as of writing this report: 8df072e104f6ae5391de12d58e9f973f08cb2c57)
 # {{ant testsome -Dtest.name=org.apache.cassandra.distributed.test.NativeTransportEncryptionOptionsTest -Dtest.method=optionalTlsConnectionAllowedToRegularPortTest}} fails after ~2m30s (see attached ant-logs.txt)
 # {{ant generate-idea-files}}
 # open project in idea, navigate to {{{}NativeTransportEncryptionOptionsTest{}}}, right click on {{optionalTlsConnectionAllowedToRegularPortTest}} and {{{}Run test{}}}; the test passes after ~12s (see attached intellij-logs.txt)
Bug: Do not require allow filtering when all primary keys are specified in SELECT 
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Apr/23 11:19
Resolved: 24/Apr/23 11:28

This was discussed in

https://lists.apache.org/thread/loj6jgv54szdvyt3wmvbtwwrrg1dtlxq

Basically, when I have this table:

{code}
create table ks.tb (p1 int, c1 int, col1 int, col2 int, primary key (p1, c1));
{code}

and I do this

{code}
select * from ks.tb where p1 = 1 and c1 = 2 and col2 = 1;
{code}

this will fail and it will require to use ALLOW FILTERING just because we are also specifying "col2". This is clearly a bug - there is no reason to require it if we are going to fetch one row only as all partition and clustering keys were specified.
Bug: Average of "decimal" values rounds the average if all inputs are integers
Status: Triage Needed
Priority: Normal
Created: 20/Apr/23 12:00

When running the AVG aggregator on "decimal" values, each value is an arbitrary-precision number which may be an integer or fractional, but it is expected that the average would be, in general, fractional. But it turns out that if all the values are integer *without* a ".0", the aggregator sums them up as integers and the final division returns an integer too instead of the fractional response expected from a "decimal" value.

For example:
 # AVG of {{decimal}} values 1.0 and 2.0 returns 1.5, as expected.
 # AVG of 1.0 and 2 or 1 and 2.0 also return 1.5.
 # But AVG of 1 and 2 returns... 1. This is wrong. The user asked for the average to be a "decimal", not a "varint", so there is no reason why it should be rounded up to be an integer.
Bug: BLOG - Announcing Monthly Apache Cassandra Town Halls
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Apr/23 23:07
Resolved: 20/Apr/23 06:40

This ticket is to capture the work associated with publishing the April 2023 blog "Announcing Monthly Apache Cassandra Town Halls"

If this blog cannot be published by the *April 14, 2022 publish date*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file).
Bug: Do not include checkstyle or other checks in "jar" target, have separate target for them
Status: Triage Needed
Priority: Normal
Created: 14/Apr/23 07:08

I'd like to propose that we reduce "jar" target in the build to absolute minimum so that it is fast when you do everyday development and you build frequently. 

Checkstyle, and other checks, like Eclipse-Warnings could be a part of a separate target, like "check" that would run all of them. 

That "check" target would be also a part of "artifacts" target. 

I know that we can always add a property to disable checkstyle but I find it a pain to remember the property name. Also, we only run some of the checks with "jar" which is confusing.

cc [~mck]