Repository: apache/airflow

Issue:
Add a flag to list all Upgrade Check Rules
It would be nice to have a list of all the available uprade-checks rules:

```
❯ airflow upgrade_check --help
usage: airflow upgrade_check [-h] [-s SAVE] [-i IGNORE] [-c CONFIG]

optional arguments:
  -h, --help            show this help message and exit
  -s SAVE, --save SAVE  Saves the result to the indicated file. The file
                        format is determined by the file extension.
  -i IGNORE, --ignore IGNORE
                        Ignore a rule. Can be used multiple times.
  -c CONFIG, --config CONFIG
                        Path to upgrade check config yaml file.
```

example (Ideally it should have 2 columns: Check_ID / Class Name and Summary):

```
❯ airflow upgrade_check --list

- Check for latest versions of apache-airflow and checker
- Remove airflowAirflowMacroPlugin class
- Chain between DAG and operator not allowed
- Connectionconn_id is not unique
- Connectionconn_type is not nullable
- Ensure users are not using custom metaclasses in custom operators
- Fernet is enabled by default
- GCP service account key deprecation
- Unify hostname_callable option in core section
- Changes in import paths of hooks, operators, sensors and others
- Users must delete deprecated configs for KubernetesExecutor
- Legacy UI is deprecated by default
- Logging configuration has been moved to new section
- Removal of Mesos Executor
- No additional argument allowed in BaseOperator
- Users must set a kubernetespod_template_file value
- SendGrid email uses old airflowcontrib module
- Changes in import path of remote task handlers
- Jinja Template Variables cannot be undefined
```

Related Slack conversation: https://apache-airflow.slack.com/archives/CCQB40SQJ/p1609259241092000

This allows users to know the progress of their checks and an easy way to ignore a check beforehand (if we list an id / class name too, example `PodTemplateFileRule`).


Discussion:
If I might add, it would be also useful to list the rule class name so people know which rules correspond to which class if they want to ignore them! I had the darndest time trying to find the name of the class when I was looking for a rule to ignore.
> If I might add, it would be also useful to list the rule class name so people know which rules correspond to which class if they want to ignore them!

Was just adding that too :) literally did it few seconds back :)




> Was just adding that too :) literally did it few seconds back :)

Ha, forgive my impatience 😁 ! I'm working on #13349 and #13350 right now, feel free to assign this to me if you wanted!
> > Was just adding that too :) literally did it few seconds back :)
> 
> Ha, forgive my impatience 😁 ! I'm working on #13349 and #13350 right now, feel free to assign this to me if you wanted!

Awesome, assigned it to you 😄 -- thanks. Looking forward to reviewing your PR

Resolving patch:
diff --git a/airflow/upgrade/checker.py b/airflow/upgrade/checker.py
--- a/airflow/upgrade/checker.py
+++ b/airflow/upgrade/checker.py
@@ -41,6 +41,15 @@ def check_upgrade(formatter, rules):
     return all_rule_statuses
 
 
+def list_checks():
+    print()
+    print("Upgrade Checks:")
+    for rule in ALL_RULES:
+        rule_name = rule.__class__.__name__
+        print("- {}: {}".format(rule_name, rule.title))
+    print()
+
+
 def register_arguments(subparser):
     subparser.add_argument(
         "-s", "--save",
@@ -49,17 +58,21 @@ def register_arguments(subparser):
     subparser.add_argument(
         "-i", "--ignore",
         help="Ignore a rule. Can be used multiple times.",
-        action='append',
+        action="append",
     )
     subparser.add_argument(
         "-c", "--config",
         help="Path to upgrade check config yaml file.",
     )
-    subparser.set_defaults(func=run)
+    subparser.add_argument(
+        "-l", "--list",
+        help="List the upgrade checks and their class names",
+        action="store_true",
+    )
 
 
 def run(args):
-    from airflow.upgrade.formatters import (ConsoleFormatter, JSONFormatter)
+    from airflow.upgrade.formatters import ConsoleFormatter, JSONFormatter
     from airflow.upgrade.config import UpgradeConfig
 
     if args.save:
@@ -94,7 +107,10 @@ def __main__():
     parser = argparse.ArgumentParser()
     register_arguments(parser)
     args = parser.parse_args()
-    args.func(args)
+    if args.list:
+        list_checks()
+    else:
+        run(args)
 
 
 if __name__ == "__main__":
diff --git a/airflow/upgrade/rules/__init__.py b/airflow/upgrade/rules/__init__.py
--- a/airflow/upgrade/rules/__init__.py
+++ b/airflow/upgrade/rules/__init__.py
@@ -31,4 +31,6 @@ def get_rules():
             bases = [b.__name__ for b in cls.__bases__]
             if cls.__name__ != "BaseRule" and "BaseRule" in bases:
                 rule_classes.append(cls)
-    return rule_classes
+    # Sort rules alphabetically by class name, while maintaining that the airflow version
+    # check should remain first
+    return rule_classes[:1] + sorted(rule_classes[1:], key=lambda r: r.__name__)


Test patch:
diff --git a/tests/upgrade/rules/test_base_rule.py b/tests/upgrade/rules/test_base_rule.py
--- a/tests/upgrade/rules/test_base_rule.py
+++ b/tests/upgrade/rules/test_base_rule.py
@@ -25,3 +25,19 @@ def test_rules_are_registered(self):
         rule_classes = get_rules()
         assert BaseRule not in rule_classes
         assert ConnTypeIsNotNullableRule in rule_classes
+
+    def test_rules_are_ordered(self):
+        rule_classes = get_rules()
+        from airflow.upgrade.rules.aaa_airflow_version_check import VersionCheckRule
+        from airflow.upgrade.rules.conn_id_is_unique import UniqueConnIdRule
+        from airflow.upgrade.rules.no_additional_args_in_operators import (
+            NoAdditionalArgsInOperatorsRule,
+        )
+
+        # Version check should still be first
+        assert rule_classes[0] == VersionCheckRule
+        # The former rule is defined in a file alphabetically before the latter rule,
+        # but it should appear later in the list since the classes are sorted alphabetically
+        unique_rule_index = rule_classes.index(UniqueConnIdRule)
+        no_addtl_args_rule_index = rule_classes.index(NoAdditionalArgsInOperatorsRule)
+        assert unique_rule_index > no_addtl_args_rule_index, "Rules are not alphabetical by class name"