Repository: Project-MONAI/MONAI
File: monai/bundle/scripts.py (Python)

Code before:
        # Override config values at runtime by specifying the component id and its new value:
        python -m monai.bundle run training --net#input_chns 1 ...

        # Override config values with another config file `/path/to/another.json`:
        python -m monai.bundle run evaluating --net %/path/to/another.json ...

        # Override config values with part content of another config file:
        python -m monai.bundle run training --net %/data/other.json#net_arg ...

        # Set default args of `run` in a JSON / YAML file, help to record and simplify the command line.
        # Other args still can override the default args at runtime:
        python -m monai.bundle run --args_file "/workspace/data/args.json" --config_file <config path>

    Args:
        runner_id: ID name of the expected config expression to run, can also be a list of IDs to run in order.
        meta_file: filepath of the metadata file, if it is a list of file paths, the content of them will be merged.
            Default to "configs/metadata.json", which is commonly used for bundles in MONAI model zoo.
        config_file: filepath of the config file, if `None`, must be provided in `args_file`.
            if it is a list of file paths, the content of them will be merged.
        logging_file: config file for `logging` module in the program. for more details:
            https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig.
            Default to "configs/logging.conf", which is commonly used for bundles in MONAI model zoo.
        tracking: enable the experiment tracking feature at runtime with optionally configurable and extensible.
            if "mlflow", will add `MLFlowHandler` to the parsed bundle with default logging settings,
            if "clearml_stats", will add `ClearMLStatsHandler` to the parsed bundle with default logging settings,
            if "clearml_image", will add `ClearMLImageHandler` to the parsed bundle with default logging settings,
            if other string, treat it as file path to load the logging settings, if `dict`,
            treat it as logging settings, otherwise, use all the default settings.
            will patch the target config content with `tracking handlers` and the top-level items of `configs`.
            example of customized settings:

            .. code-block:: python

                tracking = {
                    "handlers_id": {
                        "trainer": {"id": "train#trainer", "handlers": "train#handlers"},
                        "validator": {"id": "evaluate#evaluator", "handlers": "evaluate#handlers"},
                        "evaluator": {"id": "evaluator", "handlers": "handlers"},
                    },
                    "configs": {
                        "tracking_uri": "<path>",
                        "experiment_name": "monai_experiment",
                        "run_name": None,
                        "is_not_rank0": (
                            "$torch.distributed.is_available() \
                                and torch.distributed.is_initialized() and torch.distributed.get_rank() > 0"
                        ),
                        "trainer": {
                            "_target_": "MLFlowHandler",
                            "_disabled_": "@is_not_rank0",
                            "tracking_uri": "@tracking_uri",

Review comment (refactor):
Currently, we are testing the `--tracking mlflow` here:
https://github.com/Project-MONAI/MONAI/blob/dev/tests/test_integration_bundle_run.py#L127
If you really want to complete the `--tracking clearml_XXX` feature in this PR, please add some unit tests to your test file according to the above code.
The above bundle test code also tested other logic, you can simplify it with some overrides and run, refer to:
https://github.com/Nic-Ma/MONAI/blob/5821-bundle-interface/tests/test_bundle_infer_workflow.py#L217

Thanks in advance.


Code after:
        # Override config values at runtime by specifying the component id and its new value:
        python -m monai.bundle run training --net#input_chns 1 ...

        # Override config values with another config file `/path/to/another.json`:
        python -m monai.bundle run evaluating --net %/path/to/another.json ...

        # Override config values with part content of another config file:
        python -m monai.bundle run training --net %/data/other.json#net_arg ...

        # Set default args of `run` in a JSON / YAML file, help to record and simplify the command line.
        # Other args still can override the default args at runtime:
        python -m monai.bundle run --args_file "/workspace/data/args.json" --config_file <config path>

    Args:
        runner_id: ID name of the expected config expression to run, can also be a list of IDs to run in order.
        meta_file: filepath of the metadata file, if it is a list of file paths, the content of them will be merged.
            Default to "configs/metadata.json", which is commonly used for bundles in MONAI model zoo.
        config_file: filepath of the config file, if `None`, must be provided in `args_file`.
            if it is a list of file paths, the content of them will be merged.
        logging_file: config file for `logging` module in the program. for more details:
            https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig.
            Default to "configs/logging.conf", which is commonly used for bundles in MONAI model zoo.
        tracking: enable the experiment tracking feature at runtime with optionally configurable and extensible.
            if "mlflow", will add `MLFlowHandler` to the parsed bundle with default logging settings,
            if other string, treat it as file path to load the logging settings, if `dict`,
            treat it as logging settings, otherwise, use all the default settings.
            will patch the target config content with `tracking handlers` and the top-level items of `configs`.
            example of customized settings:

            .. code-block:: python

                tracking = {
                    "handlers_id": {
                        "trainer": {"id": "train#trainer", "handlers": "train#handlers"},
                        "validator": {"id": "evaluate#evaluator", "handlers": "evaluate#handlers"},
                        "evaluator": {"id": "evaluator", "handlers": "handlers"},
                    },
                    "configs": {
                        "tracking_uri": "<path>",
                        "experiment_name": "monai_experiment",
                        "run_name": None,
                        "is_not_rank0": (
                            "$torch.distributed.is_available() \
                                and torch.distributed.is_initialized() and torch.distributed.get_rank() > 0"
                        ),
                        "trainer": {
                            "_target_": "MLFlowHandler",
                            "_disabled_": "@is_not_rank0",
                            "tracking_uri": "@tracking_uri",
                            "experiment_name": "@experiment_name",
                            "run_name": "@run_name",