Bug: ERROR: Invalid consistency level: NODE_LOCAL when using partition denylist in 4.1.2
Status: Resolved
Priority: Normal
Resolution: Invalid
Created: 29/Jun/23 14:41
Resolved: 29/Jun/23 14:45

When I was testing the upgrade from Cassandra-3.11.15 to Cassandra-4.1.2 (single node), for the debugging purpose, I enabled partition_denylist_enabled and set the partition consistency level to {_}NODE_LOCAL{_}.

I encountered the following error message in the system log
{code:java}
ERROR [main] 2023-06-29 14:12:17,870 PartitionDenylist.java:156 - Failed to load partition denylist
java.lang.UnsupportedOperationException: Invalid consistency level: NODE_LOCAL
        at org.apache.cassandra.db.ConsistencyLevel.blockFor(ConsistencyLevel.java:168)
        at org.apache.cassandra.locator.ReplicaPlans.contactForRead(ReplicaPlans.java:558)
        at org.apache.cassandra.locator.ReplicaPlans.forRangeRead(ReplicaPlans.java:611)
        at org.apache.cassandra.service.reads.range.ReplicaPlanIterator.computeNext(ReplicaPlanIterator.java:76)
        at org.apache.cassandra.service.reads.range.ReplicaPlanIterator.computeNext(ReplicaPlanIterator.java:42)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at java.util.Iterator.forEachRemaining(Iterator.java:115)
        at org.apache.cassandra.service.reads.range.RangeCommands.sufficientLiveNodesForSelectStar(RangeCommands.java:135)
        at org.apache.cassandra.schema.PartitionDenylist.checkDenylistNodeAvailability(PartitionDenylist.java:169)
        at org.apache.cassandra.schema.PartitionDenylist.initialLoad(PartitionDenylist.java:148)
        at org.apache.cassandra.service.StorageProxy.initialLoadPartitionDenylist(StorageProxy.java:2914)
        at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1224)
        at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1145)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:936)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:854)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:424)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:751)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:875)
INFO  [main] 2023-06-29 14:12:17,870 PartitionDenylist.java:163 - Exception while loading partition denylist cache. Scheduled retry in 5 seconds. {code}
h2. How to reproduce

I encountered it after the upgrade process. But I think it can also be reproduced without the upgrade process.

1. Set up Cassandra-4.1.2 single node, add the follow configurations to cassandra.yaml
{code:java}
denylist_consistency_level: NODE_LOCAL
partition_denylist_enabled: true{code}
2. Start up Cassandra, the error message will show in system.log file.
h2. Root Cause

After setting denylist_consistency_level to NODE LEVEL, when loading denylist, it invokes the _blockFor_ method
{code:java}
public int blockFor(AbstractReplicationStrategy replicationStrategy)
{
    switch (this)
    {
        case ONE:
        case LOCAL_ONE:
            return 1;
        case ANY:
            return 1;
        case TWO:
            return 2;
        case THREE:
            return 3;
        case QUORUM:
        case SERIAL:
            return quorumFor(replicationStrategy);
        case ALL:
            return replicationStrategy.getReplicationFactor().allReplicas;
        case LOCAL_QUORUM:
        case LOCAL_SERIAL:
            return localQuorumForOurDc(replicationStrategy);
        case EACH_QUORUM:
            if (replicationStrategy instanceof NetworkTopologyStrategy)
            {
                NetworkTopologyStrategy strategy = (NetworkTopologyStrategy) replicationStrategy;
                int n = 0;
                for (String dc : strategy.getDatacenters())
                    n += localQuorumFor(replicationStrategy, dc);
                return n;
            }
            else
            {
                return quorumFor(replicationStrategy);
            }
        default:
            throw new UnsupportedOperationException("Invalid consistency level: " + toString());
    }
} {code}
However, the _NODE_LOCAL_ enum is not included in those cases and thus results in the exception message.

Should this enum be included, or does Cassandra forbid users from using partition denylist with the _NODE_LOCAL_ consistency level (even for debugging purposes)?