Repository: Lightning-AI/pytorch-lightning
Pull request: Allow sys.argv and args in LightningCLI
## What does this PR do?

Currently this just changes an error into a warning. An alternative would be to let the user specify a flag that changed the error into a warning, but my thought is that might just be more clutter than what it's worth, raising the warning might be good enough.

Fixes #16807


<details>
  <summary><b>Before submitting</b></summary>

- Was this **discussed/agreed** via a GitHub issue? (not for typos and docs)
- [x] Did you read the [contributor guideline](https://github.com/Lightning-AI/lightning/blob/master/.github/CONTRIBUTING.md), **Pull Request** section?
- [x] Did you make sure your **PR does only one thing**, instead of bundling different changes together?
- Did you make sure to **update the documentation** with your changes? (if necessary)
- Did you write any **new necessary tests**? (not for typos and docs)
- [ ] Did you verify new and **existing tests pass** locally with your changes?
- Did you list all the **breaking changes** introduced by this pull request?
- Did you **update the CHANGELOG**? (not for typos, docs, test updates, or minor internal changes/refactors)

</details>

## PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the [review guidelines](https://github.com/Lightning-AI/lightning/wiki/Review-guidelines). In short, see the following bullet-list:

<details>
  <summary>Reviewer checklist</summary>

- [ ] Is this pull request ready for review? (if not, please submit in draft mode)
- [ ] Check that all items from **Before submitting** are resolved
- [ ] Make sure the title is self-explanatory and the description concisely explains the PR
- [ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

</details>



Problem statement:
Allow sys.argv and args with LightningCLI for testing
### Outline & Motivation

I currently am forced to inherit from `LightningCLI` and overload  `parse_arguments` to disable an error when you call LightningCLI with `args` and `sys.argv` is populated.

```python
class LightningCLI_Extension(LightningCLI):
    ...

    def parse_arguments(self, parser: LightningArgumentParser, args) -> None:
        """Parses command line arguments and stores it in ``self.config``."""
        import sys
        if args is not None and len(sys.argv) > 1:
            # Please let us shoot ourselves in the foot.
            import warnings
            warnings.warn(
                "LightningCLI's args parameter is intended to run from within Python like if it were from the command "
                "line. To prevent mistakes it is not allowed to provide both args and command line arguments, got: "
                f"sys.argv[1:]={sys.argv[1:]}, args={args}."
            )
        if isinstance(args, (dict, Namespace)):
            self.config = parser.parse_object(args)
        else:
            self.config = parser.parse_args(args)
```

The reason I'm doing this is testing. I have automated tests and doctests that ensure our LightningCLI scripts play nice with our models / dataloaders. However, when I run testing software like `pytest` or `xdoctest`, the `sys.argv` list is populated, but in my test I call LightningCLI programatically with a dictionary config.




### Pitch

It would be nice if this was either changed to a warning, or allow the user to pass a flag that makes the error a warning.

If the maintainers think this is a good idea I can write the PR.

### Additional context

_No response_

cc @borda @justusschock @awaelchli @carmocca @mauvilsa

Hints from issue discussion:

Commit message: Allow sys.argv and args

Patch to review:
diff --git a/src/lightning/pytorch/cli.py b/src/lightning/pytorch/cli.py
index 1942ce6d156e5..ac612a87ec528 100644
--- a/src/lightning/pytorch/cli.py
+++ b/src/lightning/pytorch/cli.py
@@ -483,9 +483,9 @@ def link_optimizers_and_lr_schedulers(parser: LightningArgumentParser) -> None:
     def parse_arguments(self, parser: LightningArgumentParser, args: ArgsType) -> None:
         """Parses command line arguments and stores it in ``self.config``."""
         if args is not None and len(sys.argv) > 1:
-            raise ValueError(
+            _warn(
                 "LightningCLI's args parameter is intended to run from within Python like if it were from the command "
-                "line. To prevent mistakes it is not allowed to provide both args and command line arguments, got: "
+                "line. To prevent mistakes it is not recommended to provide both args and command line arguments, got: "
                 f"sys.argv[1:]={sys.argv[1:]}, args={args}."
             )
         if isinstance(args, (dict, Namespace)):


Code review comments:
diff_hunk: @@ -483,9 +483,9 @@ def link_optimizers_and_lr_schedulers(parser: LightningArgumentParser) -> None:
     def parse_arguments(self, parser: LightningArgumentParser, args: ArgsType) -> None:
         """Parses command line arguments and stores it in ``self.config``."""
         if args is not None and len(sys.argv) > 1:
-            raise ValueError(
+            _warn(
path: src/lightning/pytorch/cli.py
text: @user1:
Can you use `rank_zero_warn` instead?

@author:
Updated.