Bug: bootstrap_upgrade_test has upgrade_test annotation in wrong place
Status: Open
Priority: Normal
Created: 28/Jul/21 04:46

{code:python}
@pytest.mark.upgrade_test
class TestBootstrapUpgrade(TestBootstrap):

    """
    @jira_ticket CASSANDRA-11841
    Test that bootstrap works with a mixed version cluster
    In particular, we want to test that keep-alive is not sent
    to a node with version < 3.10
    """
    @pytest.mark.no_vnodes
    @since('3.10', max_version='3.99')
    def test_simple_bootstrap_mixed_versions(self):
        # Compatibility flag ensures that bootstrapping gets schema information during
        # upgrades from 3.0.14+ to anything upwards for 3.0.x or 3.x clusters.
        # @jira_ticket CASSANDRA-13004 for detailed context on `force_3_0_protocol_version` flag
        self._test_bootstrap_with_compatibility_flag_on(bootstrap_from_version="3.5")
{code}

This is the only upgrade test defined for bootstrap. However since it inherits from TestBoostrap class and has annotation on the class level, when we do the dry run we will see:

{noformat}
(venv) cassandra@b69a382da7cd:~/cassandra-dtest$ pytest --cassandra-dir=/home/cassandra/cassandra --collect-only -q --execute-upgrade-tests --execute-upgrade-tests-only upgrade_tests/bootstrap_upgrade_test.py
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_simple_bootstrap_with_ssl
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_simple_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_bootstrap_on_write_survey
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_simple_bootstrap_small_keepalive_period
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_simple_bootstrap_nodata
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_schema_removed_nodes
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_read_from_bootstrapped_node
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_bootstrap_waits_for_streaming_to_finish
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_consistent_range_movement_true_with_replica_down_should_fail
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_consistent_range_movement_false_with_replica_down_should_succeed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_consistent_range_movement_true_with_rf1_should_fail
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_consistent_range_movement_false_with_rf1_should_succeed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_rf_gt_nodes_multidc_should_succeed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_resumable_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_bootstrap_with_reset_bootstrap_state
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_manual_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_local_quorum_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_shutdown_wiped_node_cannot_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_killed_wiped_node_cannot_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_decommissioned_wiped_node_can_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_decommissioned_wiped_node_can_gossip_to_single_seed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_failed_bootstrap_wiped_node_can_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_node_cannot_join_as_hibernating_node_without_replace_address
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_simultaneous_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_cleanup
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_bootstrap_binary_disabled
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_invalid_host_id
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrap::test_host_id_override
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simple_bootstrap_with_ssl
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simple_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_bootstrap_on_write_survey
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simple_bootstrap_small_keepalive_period
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simple_bootstrap_nodata
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_schema_removed_nodes
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_read_from_bootstrapped_node
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_bootstrap_waits_for_streaming_to_finish
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_consistent_range_movement_true_with_replica_down_should_fail
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_consistent_range_movement_false_with_replica_down_should_succeed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_consistent_range_movement_true_with_rf1_should_fail
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_consistent_range_movement_false_with_rf1_should_succeed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_rf_gt_nodes_multidc_should_succeed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_resumable_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_bootstrap_with_reset_bootstrap_state
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_manual_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_local_quorum_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_shutdown_wiped_node_cannot_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_killed_wiped_node_cannot_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_decommissioned_wiped_node_can_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_decommissioned_wiped_node_can_gossip_to_single_seed
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_failed_bootstrap_wiped_node_can_join
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_node_cannot_join_as_hibernating_node_without_replace_address
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simultaneous_bootstrap
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_cleanup
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_bootstrap_binary_disabled
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_invalid_host_id
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_host_id_override
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simple_bootstrap_mixed_versions
{noformat}

instead of just one test which is the upgrade test:
{noformat}
upgrade_tests/bootstrap_upgrade_test.py::TestBootstrapUpgrade::test_simple_bootstrap_mixed_versions
{noformat}

All other tests are executed when we execute non-upgrade tests. Additionally when we execute upgrade tests, we execute each test from {{TestBootstrap}} twice - first for the base class, and second for the {{TestBoostrapUpgrade}}. Fixing this, by moving the upgrade_test annotation from the class level to the method level will save around 1h30m ~ 2h on upgrade tests.

[~mck] is my reasoning correct?