Bug: Flaky TestPaxos.test_replica_availability
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 25/May/21 06:43
Resolved: 26/May/21 05:35

Flaky [TestPaxos.test_replica_availability|https://ci-cassandra.apache.org/job/Cassandra-4.0/55/testReport/junit/dtest.paxos_test/TestPaxos/test_replica_availability/]
{noformat}
Error Message

cassandra.Unavailable: Error from server: code=1000 [Unavailable exception] message="Cannot achieve consistency level SERIAL" info={'consistency': 'SERIAL', 'required_replicas': 2, 'alive_replicas': 1}

Stacktrace

self = <paxos_test.TestPaxos object at 0x7ff6f83299d0>

    def test_replica_availability(self):
        """
            @jira_ticket CASSANDRA-8640
    
            Regression test for a bug (CASSANDRA-8640) that required all nodes to
            be available in order to run LWT queries, even if the query could
            complete correctly with quorum nodes available.
            """
        session = self.prepare(nodes=3, rf=3)
        session.execute("CREATE TABLE test (k int PRIMARY KEY, v int)")
        session.execute("INSERT INTO test (k, v) VALUES (0, 0) IF NOT EXISTS")
    
        self.cluster.nodelist()[2].stop()
        session.execute("INSERT INTO test (k, v) VALUES (1, 1) IF NOT EXISTS")
    
        self.cluster.nodelist()[1].stop()
        assert_unavailable(session.execute, "INSERT INTO test (k, v) VALUES (2, 2) IF NOT EXISTS")
    
        self.cluster.nodelist()[1].start()
>       session.execute("INSERT INTO test (k, v) VALUES (3, 3) IF NOT EXISTS")

paxos_test.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../venv/src/cassandra-driver/cassandra/cluster.py:2618: in execute
    return self.execute_async(query, parameters, trace, custom_payload, timeout, execution_profile, paging_state, host, execute_as).result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ResponseFuture: query='<SimpleStatement query="INSERT INTO test (k, v) VALUES (3, 3) IF NOT EXISTS", consistency=Not ...el SERIAL" info={'consistency': 'SERIAL', 'required_replicas': 2, 'alive_replicas': 1} coordinator_host=127.0.0.1:9042>

    def result(self):
        """
            Return the final result or raise an Exception if errors were
            encountered.  If the final result or error has not been set
            yet, this method will block until it is set, or the timeout
            set for the request expires.
    
            Timeout is specified in the Session request execution functions.
            If the timeout is exceeded, an :exc:`cassandra.OperationTimedOut` will be raised.
            This is a client-side timeout. For more information
            about server-side coordinator timeouts, see :class:`.policies.RetryPolicy`.
    
            Example usage::
    
                >>> future = session.execute_async("SELECT * FROM mycf")
                >>> # do other stuff...
    
                >>> try:
                ...     rows = future.result()
                ...     for row in rows:
                ...         ... # process results
                ... except Exception:
                ...     log.exception("Operation failed:")
    
            """
        self._event.wait()
        if self._final_result is not _NOT_SET:
            return ResultSet(self, self._final_result)
        else:
>           raise self._final_exception
E           cassandra.Unavailable: Error from server: code=1000 [Unavailable exception] message="Cannot achieve consistency level SERIAL" info={'consistency': 'SERIAL', 'required_replicas': 2, 'alive_replicas': 1}

../venv/src/cassandra-driver/cassandra/cluster.py:4894: Unavailable
{noformat}