Bug: Running python dtest with --keep-failed-test-dir causes ERROR for SKIP tests
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 20/Jan/21 16:20
Resolved: 21/Jan/21 15:04

There is a new flag, added in CASSANDRA-16070: "--keep-failed-test-dir".

In some cases (I am not sure if always),

when this flag is set and when test is SKIP-ed it will cause test ERROR.

 

Good run (without --keep-failed-test-dir)
{code:java}
 pytest  --cassandra-dir=/home/tomek/repos/apache/cassandra client_network_stop_start_test.py 
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.6.12, pytest-3.6.4, py-1.10.0, pluggy-0.7.1
rootdir: /home/tomek/repos/tlasica/cassandra-dtest, inifile: pytest.ini
plugins: timeout-1.4.2, flaky-3.7.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 3 items                                                                                                                                                                             client_network_stop_start_test.py .ss                                                                                                                                                   [100%]
===Flaky Test Report===test_defaults passed 1 out of the required 1 times. Success!===End Flaky Test Report=============================================================================== 1 passed, 2 skipped in 11.25 seconds ============================================================================={code}
and bad one (with flag):
{code:java}

pytest --keep-failed-test-dir --cassandra-dir=/home/tomek/repos/apache/cassandra client_network_stop_start_test.py 
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.6.12, pytest-3.6.4, py-1.10.0, pluggy-0.7.1
rootdir: /home/tomek/repos/tlasica/cassandra-dtest, inifile: pytest.ini
plugins: timeout-1.4.2, flaky-3.7.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 3 items                                                                                                                                                                             client_network_stop_start_test.py .sEsE                                                                                                                                                 [100%]=========================================================================================== ERRORS ============================================================================================
_____________________________________________________________ ERROR at teardown of TestClientNetworkStopStart.test_hsha_defaults ______________________________________________________________request = <SubRequest 'fixture_dtest_setup' for <Function 'test_hsha_defaults'>>, dtest_config = <dtest_config.DTestConfig object at 0x7fd313af9908>
fixture_dtest_setup_overrides = <dtest_setup_overrides.DTestSetupOverrides object at 0x7fd313263048>, fixture_logging_setup = None, fixture_dtest_cluster_name = 'test'
fixture_dtest_create_cluster_func = <function DTestSetup.create_ccm_cluster at 0x7fd313b15488>    @pytest.fixture(scope='function', autouse=False)
    def fixture_dtest_setup(request,
                            dtest_config,
                            fixture_dtest_setup_overrides,
                            fixture_logging_setup,
                            fixture_dtest_cluster_name,
                            fixture_dtest_create_cluster_func):
        if running_in_docker():
            cleanup_docker_environment_before_test_execution()
    
        # do all of our setup operations to get the enviornment ready for the actual test
        # to run (e.g. bring up a cluster with the necessary config, populate variables, etc)
        initial_environment = copy.deepcopy(os.environ)
        dtest_setup = DTestSetup(dtest_config=dtest_config,
                                 setup_overrides=fixture_dtest_setup_overrides,
                                 cluster_name=fixture_dtest_cluster_name)
        dtest_setup.initialize_cluster(fixture_dtest_create_cluster_func)
    
        if not dtest_config.disable_active_log_watching:
            dtest_setup.begin_active_log_watch()
    
        # at this point we're done with our setup operations in this fixture
        # yield to allow the actual test to run
        yield dtest_setup
    
        # phew! we're back after executing the test, now we need to do
        # all of our teardown and cleanup operations
    
        reset_environment_vars(initial_environment)
        dtest_setup.jvm_args = []
    
        for con in dtest_setup.connections:
            con.cluster.shutdown()
        dtest_setup.connections = []
    
        failed = False
        try:
            if not dtest_setup.allow_log_errors:
                errors = check_logs_for_errors(dtest_setup)
                if len(errors) > 0:
                    failed = True
                    pytest.fail(msg='Unexpected error found in node logs (see stdout for full details). Errors: [{errors}]'
                                         .format(errors=str.join(", ", errors)), pytrace=False)
        finally:
            try:
                # save the logs for inspection
                if failed or not dtest_config.delete_logs:
                    copy_logs(request, dtest_setup.cluster)
            except Exception as e:
                logger.error("Error saving log:", str(e))
            finally:
>               dtest_setup.cleanup_cluster(request)conftest.py:352: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _self = <dtest_setup.DTestSetup object at 0x7fd313adc898>, request = <SubRequest 'fixture_dtest_setup' for <Function 'test_hsha_defaults'>>    def cleanup_cluster(self, request=None):
        with log_filter('cassandra'):  # quiet noise from driver when nodes start going down
>           if self.dtest_config.keep_test_dir or (self.dtest_config.keep_failed_test_dir and request and request.node.rep_call.failed):
E           AttributeError: 'Function' object has no attribute 'rep_call'dtest_setup.py:351: AttributeError
------------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------------
17:18:59,827 ccm DEBUG Log-watching thread starting.
------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------
17:18:59,735 conftest INFO Starting execution of test_hsha_defaults at 2021-01-20 17:18:59.735191
17:18:59,736 dtest_setup INFO cluster ccm directory: /tmp/dtest-10reduiu
---------------------------------------------------------------------------------- Captured stdout teardown -----------------------------------------------------------------------------------_____________________________________________________________ ERROR at teardown of TestClientNetworkStopStart.test_hsha_with_ssl ______________________________________________________________request = <SubRequest 'fixture_dtest_setup' for <Function 'test_hsha_with_ssl'>>, dtest_config = <dtest_config.DTestConfig object at 0x7fd313af9908>
fixture_dtest_setup_overrides = <dtest_setup_overrides.DTestSetupOverrides object at 0x7fd31301f5c0>, fixture_logging_setup = None, fixture_dtest_cluster_name = 'test'
fixture_dtest_create_cluster_func = <function DTestSetup.create_ccm_cluster at 0x7fd313b15488>    @pytest.fixture(scope='function', autouse=False)
    def fixture_dtest_setup(request,
                            dtest_config,
                            fixture_dtest_setup_overrides,
                            fixture_logging_setup,
                            fixture_dtest_cluster_name,
                            fixture_dtest_create_cluster_func):
        if running_in_docker():
            cleanup_docker_environment_before_test_execution()
    
        # do all of our setup operations to get the enviornment ready for the actual test
        # to run (e.g. bring up a cluster with the necessary config, populate variables, etc)
        initial_environment = copy.deepcopy(os.environ)
        dtest_setup = DTestSetup(dtest_config=dtest_config,
                                 setup_overrides=fixture_dtest_setup_overrides,
                                 cluster_name=fixture_dtest_cluster_name)
        dtest_setup.initialize_cluster(fixture_dtest_create_cluster_func)
    
        if not dtest_config.disable_active_log_watching:
            dtest_setup.begin_active_log_watch()
    
        # at this point we're done with our setup operations in this fixture
        # yield to allow the actual test to run
        yield dtest_setup
    
        # phew! we're back after executing the test, now we need to do
        # all of our teardown and cleanup operations
    
        reset_environment_vars(initial_environment)
        dtest_setup.jvm_args = []
    
        for con in dtest_setup.connections:
            con.cluster.shutdown()
        dtest_setup.connections = []
    
        failed = False
        try:
            if not dtest_setup.allow_log_errors:
                errors = check_logs_for_errors(dtest_setup)
                if len(errors) > 0:
                    failed = True
                    pytest.fail(msg='Unexpected error found in node logs (see stdout for full details). Errors: [{errors}]'
                                         .format(errors=str.join(", ", errors)), pytrace=False)
        finally:
            try:
                # save the logs for inspection
                if failed or not dtest_config.delete_logs:
                    copy_logs(request, dtest_setup.cluster)
            except Exception as e:
                logger.error("Error saving log:", str(e))
            finally:
>               dtest_setup.cleanup_cluster(request)conftest.py:352: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _self = <dtest_setup.DTestSetup object at 0x7fd31301f2b0>, request = <SubRequest 'fixture_dtest_setup' for <Function 'test_hsha_with_ssl'>>    def cleanup_cluster(self, request=None):
        with log_filter('cassandra'):  # quiet noise from driver when nodes start going down
>           if self.dtest_config.keep_test_dir or (self.dtest_config.keep_failed_test_dir and request and request.node.rep_call.failed):
E           AttributeError: 'Function' object has no attribute 'rep_call'dtest_setup.py:351: AttributeError
------------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------------
17:19:00,1 ccm DEBUG Log-watching thread starting.
------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------
17:18:59,922 conftest INFO Starting execution of test_hsha_with_ssl at 2021-01-20 17:18:59.922782
17:18:59,923 dtest_setup INFO cluster ccm directory: /tmp/dtest-zlpb7lfk
===Flaky Test Report===test_defaults passed 1 out of the required 1 times. Success!===End Flaky Test Report===
======================================================================== 1 passed, 2 skipped, 2 error in 12.22 seconds ======================================================================== {code}