Repository: omni-us/jsonargparse

Issue:
enable all arguments as positional
<!-- If you like this project, please ⭐ star it https://github.com/omni-us/jsonargparse/ -->

## 🚀 Feature request

Following the very first code sample in docs and wondering if it is possible to relax the argument parsing and enable to pass all arguments as positional

### Motivation

Limit some unexpected crashes when passing all arguments about some without its name

```py
from jsonargparse import CLI


def command(name: str, prize: int = 100):
    """Prints the prize won by a person.

    Args:
        name: Name of winner.
        prize: Amount won.
    """
    print(f"{name} won {prize}€!")


if __name__ == "__main__":
    CLI(command)
```

and call it as
```bash
python scripts/trying.py any 200
```

gives following:
```
usage: trying.py [-h] [--config CONFIG] [--print_config[=flags]] [--prize PRIZE] name
error: Unrecognized arguments: 200
```

### Pitch

This prevents us from switching from Fire, and this would be breaking many of our workflows
ref: https://github.com/Lightning-AI/utilities/pull/371

### Alternatives

<!-- Any alternative solutions or features you've considered, if any. -->

Hints from issue comments:
@mauvilsa: I agree it would be nice to support accepting arguments both as positional or by name. I will check how difficult it would be to implement it.@mauvilsa: I have implemented an experimental version, see #692.@mauvilsa: I am not yet sure about many edge cases this could introduce, so it still needs work to be viable for merging. But you could test it out to see if it already covers all of your cases, or you find pending things to fix.

Some comments:

- Only works on non-subcommand parsers. This means only for simple cases like the example in #690 or after providing all subcommands. I implemented it like this because it would be confusing to have optional positionals before subcommands.
- It only applies to arguments that accept exactly one value.
- Not yet implemented, but probably should also not apply subclass/dataclass arguments, since giving these as positionals is cumbersome.
- shtab support would be nice, since positionals are error prone due to lacking the explicit ``--name``, so guidance when typing commands would be advisable. But not sure how support for this could be achieved.

To use, before running the CLI do:

```python
from jsonargparse import set_parsing_settings

set_parsing_settings(parse_optionals_as_positionals=True)
```@Borda: > * Only works on non-subcommand parsers.

just curious: why are subcommand treated differently/separately? I would assume they are using the same parsing mechanism... 

> It only applies to arguments that accept exactly one value.

That is fine :)@mauvilsa: > > * Only works on non-subcommand parsers.
> 
> just curious: why are subcommand treated differently/separately? I would assume they are using the same parsing mechanism...

Subcommands are parsed by parent plus child subparsers.

Though, I am not sure if you understood what I meant. A simple parser with `usage: cli.py [--k1 K1] p1` can be run by giving `k1` as the second positional like `cli.py p1 k1`. Also a parser with subcommands with `usage: cli.py [--g1 G1] subcommand [--c1 C1] p1` can be run by giving `c1` as positional, like `cli.py subcommand p1 c1`. But `g1`, an option before the subcommand, can't be given as positional. The only way t provide `g1` is like `cli.py --g1 G1 subcommand p1 c1`.
@Borda: I think `cli.py --g1 G1 subcommand p1 c1` is too specific case, never thought about this complex patter :)@mauvilsa: @Borda have you tested out what I implemented in #692? Let me know if it is working fine or there are any issues.@mauvilsa: shtab support has now been implemented in #700.

Solution patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 56e71b7c..cc402023 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -19,6 +19,8 @@ Added
 ^^^^^
 - Support ``shtab`` completion of ``Literal`` types (`#693
   <https://github.com/omni-us/jsonargparse/pull/693>`__).
+- Support for parsing optionals as positionals (`#692
+  <https://github.com/omni-us/jsonargparse/pull/692>`__).
 
 Changed
 ^^^^^^^
diff --git a/DOCUMENTATION.rst b/DOCUMENTATION.rst
index e8b15c34..2ee1d306 100644
--- a/DOCUMENTATION.rst
+++ b/DOCUMENTATION.rst
@@ -332,6 +332,53 @@ This can be easily implemented with :func:`.capture_parser` as follows:
     :func:`.auto_cli` is by using :func:`.capture_parser`.
 
 
+Optionals as positionals
+------------------------
+
+It can sometimes be useful to allow optional arguments to be passed both by
+name, such as ``--key=val``, and as positional arguments, such as ``val``. This
+behavior can be enabled by using
+``set_parsing_settings(parse_optionals_as_positionals=True)``. Key points to
+note about this feature are:
+
+- Only optional arguments that accept exactly one value can be passed as
+  positional, i.e., when ``nargs`` is not specified or is set to ``nargs=1``.
+- Optional arguments with subclass types cannot be passed as positional
+  arguments.
+- Optionals are treated as positionals only after the standard positionals and
+  in the order they were added to the parser. The usage section in the help
+  displays the optionals that can be passed as positionals and their order.
+- Optional arguments in parsers with subcommands cannot be passed as
+  positionals. Only the child subparsers, after specifying the subcommand
+  name(s), support this feature.
+
+For instance, consider a parser defined as follows:
+
+.. testcode::
+
+    from jsonargparse import set_parsing_settings
+
+
+    set_parsing_settings(parse_optionals_as_positionals=True)
+
+    parser.add_argument("p1")
+    parser.add_argument("--o2")
+    parser.add_argument("--o3")
+
+The help will display ``p1 [o2 [o3]]`` along with a note indicating that this
+feature is enabled. Name-based parsing, such as ``--o2=val2 --o3=val3 val1``,
+will work as expected. Additionally, the following cases are also valid:
+``--o3=val3 val1 val2`` or ``val1 val2 val3``.
+
+.. note::
+
+    Positional arguments take precedence over optional arguments. This means
+    that if a value is provided both as a positional and as an optional
+    argument, the value from the positional argument will be used, regardless of
+    the order. For example, with the parser above, the command ``val1 val2a
+    --o2=val2b`` would result in ``o2=val1a``.
+
+
 Functions as type
 -----------------
 
diff --git a/jsonargparse/_actions.py b/jsonargparse/_actions.py
index 8c50a421..62ba176e 100644
--- a/jsonargparse/_actions.py
+++ b/jsonargparse/_actions.py
@@ -409,6 +409,7 @@ def print_help(self, call_args):
         if ActionTypeHint.is_callable_typehint(typehint) and hasattr(typehint, "__args__"):
             self.sub_add_kwargs["skip"] = {max(0, len(typehint.__args__) - 1)}
         subparser.add_class_arguments(val_class, dest, **self.sub_add_kwargs)
+        subparser._inner_parser = True
         remove_actions(subparser, (_HelpAction, _ActionPrintConfig, _ActionConfigLoad))
         args = self.get_args_after_opt(parser.args)
         if args:
diff --git a/jsonargparse/_common.py b/jsonargparse/_common.py
index 9af90e53..292f1e6a 100644
--- a/jsonargparse/_common.py
+++ b/jsonargparse/_common.py
@@ -34,6 +34,7 @@
 __all__ = [
     "LoggerProperty",
     "null_logger",
+    "set_parsing_settings",
 ]
 
 ClassType = TypeVar("ClassType")
@@ -88,6 +89,63 @@ def parser_context(**kwargs):
             context_var.reset(token)
 
 
+parsing_settings = dict(
+    parse_optionals_as_positionals=False,
+)
+
+
+def set_parsing_settings(*, parse_optionals_as_positionals: Optional[bool] = None) -> None:
+    """
+    Modify settings that affect the parsing behavior.
+
+    Args:
+        parse_optionals_as_positionals: [EXPERIMENTAL] If True, the parser will
+            take extra positional command line arguments as values for optional
+            arguments. This means that optional arguments can be given by name
+            --key=value as usual, but also as positional. The extra positionals
+            are applied to optionals in the order that they were added to the
+            parser.
+    """
+    if isinstance(parse_optionals_as_positionals, bool):
+        parsing_settings["parse_optionals_as_positionals"] = parse_optionals_as_positionals
+    elif parse_optionals_as_positionals is not None:
+        raise ValueError(f"parse_optionals_as_positionals must be a boolean, but got {parse_optionals_as_positionals}.")
+
+
+def get_parsing_setting(name: str):
+    if name not in parsing_settings:
+        raise ValueError(f"Unknown parsing setting {name}.")
+    return parsing_settings[name]
+
+
+def get_optionals_as_positionals_actions(parser, include_positionals=False):
+    from jsonargparse._actions import ActionConfigFile, _ActionConfigLoad, filter_default_actions
+    from jsonargparse._completions import ShtabAction
+    from jsonargparse._typehints import ActionTypeHint
+
+    actions = []
+    for action in filter_default_actions(parser._actions):
+        if isinstance(action, (_ActionConfigLoad, ActionConfigFile, ShtabAction)):
+            continue
+        if ActionTypeHint.is_subclass_typehint(action, all_subtypes=False):
+            continue
+        if action.nargs not in {1, None}:
+            continue
+        if not include_positionals and action.option_strings == []:
+            continue
+        actions.append(action)
+
+    return actions
+
+
+def supports_optionals_as_positionals(parser):
+    return (
+        get_parsing_setting("parse_optionals_as_positionals")
+        and not parser._subcommands_action
+        and not getattr(parser, "_inner_parser", False)
+    )
+
+
 def is_subclass(cls, class_or_tuple) -> bool:
     """Extension of issubclass that supports non-class arguments."""
     try:
diff --git a/jsonargparse/_core.py b/jsonargparse/_core.py
index dd247d4a..f38bdb62 100644
--- a/jsonargparse/_core.py
+++ b/jsonargparse/_core.py
@@ -41,9 +41,11 @@
     InstantiatorsDictType,
     class_instantiators,
     debug_mode_active,
+    get_optionals_as_positionals_actions,
     is_dataclass_like,
     lenient_check,
     parser_context,
+    supports_optionals_as_positionals,
 )
 from ._completions import (
     argcomplete_namespace,
@@ -307,6 +309,23 @@ def parse_known_args(self, args=None, namespace=None):
 
         return namespace, args
 
+    def _positional_optionals(self, cfg, unk):
+        if len(unk) == 0 or not supports_optionals_as_positionals(self):
+            return cfg, unk
+
+        for action in get_optionals_as_positionals_actions(self, include_positionals=True):
+            if action.option_strings == []:
+                if cfg.get(action.dest) is None:
+                    self._logger.debug(f"Positional argument {action.dest} missing, aborting _positional_optionals")
+                    break
+                continue
+
+            cfg[action.dest] = self._check_value_key(action, unk.pop(0), action.dest, cfg)
+            if len(unk) == 0:
+                break
+
+        return cfg, unk
+
     def _parse_optional(self, arg_string):
         subclass_arg = ActionTypeHint.parse_argv_item(arg_string)
         if subclass_arg:
@@ -434,6 +453,7 @@ def parse_args(  # type: ignore[override]
 
             with _ActionSubCommands.parse_kwargs_context({"env": env, "defaults": defaults}):
                 cfg, unk = self.parse_known_args(args=args, namespace=cfg)
+                cfg, unk = self._positional_optionals(cfg, unk)
             if unk:
                 self.error(f'Unrecognized arguments: {" ".join(unk)}')
 
diff --git a/jsonargparse/_formatters.py b/jsonargparse/_formatters.py
index 109d88a5..a96d9be3 100644
--- a/jsonargparse/_formatters.py
+++ b/jsonargparse/_formatters.py
@@ -23,7 +23,12 @@
     _find_action,
     filter_default_actions,
 )
-from ._common import defaults_cache, parent_parser
+from ._common import (
+    defaults_cache,
+    get_optionals_as_positionals_actions,
+    parent_parser,
+    supports_optionals_as_positionals,
+)
 from ._completions import ShtabAction
 from ._link_arguments import ActionLink
 from ._namespace import Namespace, NSKeyError
@@ -88,8 +93,32 @@ def _get_help_string(self, action: Action) -> str:
 
     def _format_usage(self, *args, **kwargs) -> str:
         usage = super()._format_usage(*args, **kwargs)
+
         parser = parent_parser.get()
-        if parser:
+        if not parser:
+            return usage
+
+        if supports_optionals_as_positionals(parser):
+            actions = get_optionals_as_positionals_actions(parser)
+            if len(actions) > 0:
+                extra_positionals = ""
+                for action in reversed(actions):
+                    extra_positionals = f"{action.dest} {extra_positionals}" if extra_positionals else action.dest
+                    extra_positionals = f"[{extra_positionals}]"
+
+                usage_lines = usage.rstrip().split("\n")
+                last_line = usage_lines[-1] + " " + extra_positionals
+                text_width = self._width - self._current_indent
+                if len(usage_lines) == 1 or len(last_line) <= text_width:
+                    usage_lines[-1] = last_line
+                else:
+                    indent = re.sub(r"^( +)[^ ].*$", r"\1", usage_lines[-1])
+                    usage_lines.append(indent + extra_positionals)
+
+                note = "note: extra positionals are parsed as optionals in the order shown above."
+                usage = "\n".join(usage_lines) + f"\n\n{note}\n\n"
+
+        else:
             for key in parser.required_args:
                 try:
                     default = parser.get_default(key)
@@ -97,6 +126,7 @@ def _format_usage(self, *args, **kwargs) -> str:
                     default = None
                 if default is None and f"[--{key} " in usage:
                     usage = re.sub(f"\\[(--{key} [^\\]]+)]", r"\1", usage, count=1)
+
         return usage
 
     def _format_action_invocation(self, action: Action) -> str:
diff --git a/jsonargparse/_typehints.py b/jsonargparse/_typehints.py
index 3b529912..8ea4fc12 100644
--- a/jsonargparse/_typehints.py
+++ b/jsonargparse/_typehints.py
@@ -658,6 +658,8 @@ def get_class_parser(val_class, sub_add_kwargs=None, skip_args=0):
         for link_kwargs in nested_links.get():
             parser.link_arguments(**link_kwargs)
 
+        parser._inner_parser = True
+
         return parser
 
     def extra_help(self):


Test patch:
diff --git a/jsonargparse_tests/conftest.py b/jsonargparse_tests/conftest.py
index 735d05b9..3610c9ee 100644
--- a/jsonargparse_tests/conftest.py
+++ b/jsonargparse_tests/conftest.py
@@ -32,7 +32,7 @@
     set_docstring_parse_options(style=DocstringStyle.GOOGLE)
 
 
-columns_env = {"COLUMNS": "200"}
+columns = "200"
 
 is_cpython = platform.python_implementation() == "CPython"
 is_posix = os.name == "posix"
@@ -190,9 +190,9 @@ def source_unavailable():
         yield
 
 
-def get_parser_help(parser: ArgumentParser, strip=False) -> str:
+def get_parser_help(parser: ArgumentParser, strip=False, columns=columns) -> str:
     out = StringIO()
-    with patch.dict(os.environ, columns_env):
+    with patch.dict(os.environ, {"COLUMNS": columns}):
         parser.print_help(out)
     if strip:
         return re.sub("  *", " ", out.getvalue())
@@ -201,7 +201,7 @@ def get_parser_help(parser: ArgumentParser, strip=False) -> str:
 
 def get_parse_args_stdout(parser: ArgumentParser, args: List[str]) -> str:
     out = StringIO()
-    with patch.dict(os.environ, columns_env), redirect_stdout(out), pytest.raises(SystemExit):
+    with patch.dict(os.environ, {"COLUMNS": columns}), redirect_stdout(out), pytest.raises(SystemExit):
         parser.parse_args(args)
     return out.getvalue()
 
diff --git a/jsonargparse_tests/test_parsing_settings.py b/jsonargparse_tests/test_parsing_settings.py
new file mode 100644
index 00000000..380c5dde
--- /dev/null
+++ b/jsonargparse_tests/test_parsing_settings.py
@@ -0,0 +1,154 @@
+import re
+from dataclasses import dataclass
+from typing import Literal, Optional
+from unittest.mock import patch
+
+import pytest
+
+from jsonargparse import ActionYesNo, ArgumentError, Namespace, set_parsing_settings
+from jsonargparse._common import get_parsing_setting
+from jsonargparse_tests.conftest import capture_logs, get_parse_args_stdout, get_parser_help
+from jsonargparse_tests.test_typehints import Optimizer
+
+
+@pytest.fixture(autouse=True)
+def patch_parsing_settings():
+    with patch.dict("jsonargparse._common.parsing_settings"):
+        yield
+
+
+def test_get_parsing_setting_failure():
+    with pytest.raises(ValueError, match="Unknown parsing setting"):
+        get_parsing_setting("unknown_setting")
+
+
+# parse_optionals_as_positionals
+
+
+def test_set_parse_optionals_as_positionals_failure():
+    with pytest.raises(ValueError, match="parse_optionals_as_positionals must be a boolean"):
+        set_parsing_settings(parse_optionals_as_positionals="invalid")
+
+
+def test_parse_optionals_as_positionals_simple(parser, logger, subtests):
+    set_parsing_settings(parse_optionals_as_positionals=True)
+
+    parser.add_argument("p1", type=Optional[Literal["p1"]])
+    parser.add_argument("--o1", type=Optional[int])
+    parser.add_argument("--flag", default=False, nargs=0, action=ActionYesNo)
+    parser.add_argument("--o2", type=Optional[Literal["o2"]])
+    parser.add_argument("--o3")
+
+    with subtests.test("help"):
+        help_str = get_parser_help(parser)
+        assert " p1 [o1 [o2 [o3]]]" in help_str
+        assert "extra positionals are parsed as optionals in the order shown above" in help_str
+
+    with subtests.test("no extra positionals"):
+        cfg = parser.parse_args(["--o2=o2", "--o1=1", "p1"])
+        assert cfg == Namespace(p1="p1", o1=1, o2="o2", o3=None, flag=False)
+
+    with subtests.test("one extra positional"):
+        cfg = parser.parse_args(["--o2=o2", "p1", "2"])
+        assert cfg == Namespace(p1="p1", o1=2, o2="o2", o3=None, flag=False)
+
+    with subtests.test("two extra positionals"):
+        cfg = parser.parse_args(["p1", "3", "o2"])
+        assert cfg == Namespace(p1="p1", o1=3, o2="o2", o3=None, flag=False)
+
+    with subtests.test("three extra positionals"):
+        cfg = parser.parse_args(["p1", "3", "o2", "v3"])
+        assert cfg == Namespace(p1="p1", o1=3, o2="o2", o3="v3", flag=False)
+
+    with subtests.test("extra positional has precedence"):
+        cfg = parser.parse_args(["p1", "3", "o2", "--o1=4"])
+        assert cfg == Namespace(p1="p1", o1=3, o2="o2", o3=None, flag=False)
+
+    with subtests.test("extra positionals invalid values"):
+        with pytest.raises(ArgumentError) as ex:
+            parser.parse_args(["p1", "o2", "5"])
+        assert re.match('Parser key "o1".*Given value: o2', ex.value.message, re.DOTALL)
+
+        with pytest.raises(ArgumentError) as ex:
+            parser.parse_args(["p1", "6", "invalid"])
+        assert re.match('Parser key "o2".*Given value: invalid', ex.value.message, re.DOTALL)
+
+    parser.logger = logger
+    with subtests.test("unrecognized arguments"):
+        with capture_logs(logger) as logs:
+            with pytest.raises(ArgumentError, match="Unrecognized arguments: --unk=x"):
+                parser.parse_args(["--unk=x"])
+        assert "Positional argument p1 missing, aborting _positional_optionals" in logs.getvalue()
+
+
+def test_parse_optionals_as_positionals_subcommands(parser, subparser, subtests):
+    set_parsing_settings(parse_optionals_as_positionals=True)
+
+    subparser.add_argument("p1", type=Optional[Literal["p1"]])
+    subparser.add_argument("--o1", type=Optional[int])
+    subparser.add_argument("--o2", type=Optional[Literal["o2"]])
+    parser.add_argument("--g1")
+    subcommands = parser.add_subcommands()
+    subcommands.add_subcommand("subcmd", subparser)
+
+    with subtests.test("help global"):
+        help_str = get_parser_help(parser)
+        assert " [g1]" not in help_str
+        assert "extra positionals are parsed as optionals in the order shown above" not in help_str
+
+    with subtests.test("help subcommand"):
+        help_str = get_parse_args_stdout(parser, ["subcmd", "-h"])
+        assert " p1 [o1 [o2]]" in help_str
+        assert "extra positionals are parsed as optionals in the order shown above" in help_str
+
+    with subtests.test("no extra positionals"):
+        cfg = parser.parse_args(["subcmd", "--o2=o2", "--o1=1", "p1"])
+        assert cfg.subcmd == Namespace(p1="p1", o1=1, o2="o2")
+
+    with subtests.test("one extra positional"):
+        cfg = parser.parse_args(["subcmd", "--o2=o2", "p1", "2"])
+        assert cfg.subcmd == Namespace(p1="p1", o1=2, o2="o2")
+
+    with subtests.test("two extra positionals"):
+        cfg = parser.parse_args(["subcmd", "p1", "3", "o2"])
+        assert cfg.subcmd == Namespace(p1="p1", o1=3, o2="o2")
+
+    with subtests.test("extra positionals invalid values"):
+        with pytest.raises(ArgumentError) as ex:
+            parser.parse_args(["subcmd", "p1", "o2", "5"])
+        assert re.match('Parser key "o1".*Given value: o2', ex.value.message, re.DOTALL)
+
+
+def test_optionals_as_positionals_usage_wrap(parser):
+    set_parsing_settings(parse_optionals_as_positionals=True)
+
+    parser.prog = "long_prog_name"
+    parser.add_argument("relatively_long_positional")
+    parser.add_argument("--first_long_optional")
+    parser.add_argument("--second_long_optional")
+
+    help_str = get_parser_help(parser, columns="80")
+    assert "usage: long_prog_name " in help_str
+    assert "                      relatively_long_positional" in help_str
+    assert "                      [first_long_optional [second_long_optional]]" in help_str
+
+
+@dataclass
+class DataOptions:
+    d1: int = 1
+
+
+def test_optionals_as_positionals_unsupported_arguments(parser):
+    set_parsing_settings(parse_optionals_as_positionals=True)
+
+    parser.add_argument("p1", type=Optional[Literal["p1"]])
+    parser.add_argument("--o1", type=Optimizer)
+    parser.add_argument("--o2", type=Optional[int])
+    parser.add_argument("--o3", type=DataOptions)
+    parser.add_argument("--o4.n1", type=float)
+
+    help_str = get_parser_help(parser)
+    assert " p1 [o2 [o3.d1 [o4.n1]]]" in help_str
+
+    help_str = get_parse_args_stdout(parser, ["--o1.help=Adam"])
+    assert "extra positionals are parsed as optionals in the order shown above" not in help_str