Repository: common-workflow-language/cwltool

Issue:
Calling cwltool.main() with preparsed args and provenance fails unless unparsed args are also provided
## Expected Behavior
Calling `cwltool.main()` with preparsed args only, but including `--provenance`, should execute and create a RO-Crate, the same as when running as a script.

## Actual Behavior
The execution fails, raising `Exception("argsl cannot be None")`

## Workflow Code

Program `prov-no-argsl.py`:
```python
import sys

from cwltool.main import main as cwlmain
from cwltool.argparser import arg_parser

unparsed_args = ["--provenance", "ro-crate", "hello_world.cwl", "--message=Hello"]
parsed_args = arg_parser().parse_args(unparsed_args)

cwlmain(args=parsed_args)
```

Assuming [`hello_world.cwl`](https://github.com/common-workflow-language/user_guide/blob/main/src/_includes/cwl/hello_world.cwl) is in the working directory

```yaml
cwlVersion: v1.2
class: CommandLineTool
baseCommand: echo
inputs:
  message:
    type: string
    default: Hello World
    inputBinding:
      position: 1
outputs: []
```

## Full Traceback
```pytb
INFO prov-no-argsl.py 3.1.20230719185429
Traceback (most recent call last):
  File "/private/tmp/cwl-prov/prov-no-argsl.py", line 9, in <module>
    cwlmain(args=parsed_args)
  File "/opt/homebrew/lib/python3.11/site-packages/cwltool/main.py", line 1052, in main
    raise Exception("argsl cannot be None")
Exception: argsl cannot be None

```
## Your Environment
* cwltool version: 
/opt/homebrew/bin/cwltool 3.1.20230719185429


## Remarks

Note that if the unparsed arguments are also provided, then execution proceeds as expected

Program `prov-with-argsl.py`:
```python
import sys

from cwltool.argparser import arg_parser
from cwltool.main import main as cwlmain

unparsed_args = ["--provenance", "ro-crate", "hello_world.cwl", "--message=Hello"]
parsed_args = arg_parser().parse_args(unparsed_args)

cwlmain(args=parsed_args, argsl=unparsed_args)
```

```
INFO prov-with-argsl.py 3.1.20230719185429
INFO [cwltool] prov-with-argsl.py --provenance ro-crate hello_world.cwl --message=Hello
INFO Resolved 'hello_world.cwl' to 'file:///private/tmp/cwl-prov/hello_world.cwl'
INFO [job hello_world.cwl] /private/tmp/docker_tmpg0yk4mky$ echo \
    Hello
Hello
INFO [job hello_world.cwl] completed success
/opt/homebrew/lib/python3.11/site-packages/rdflib/plugins/serializers/nt.py:40: UserWarning: NTSerializer always uses UTF-8 encoding. Given encoding was: None
  warnings.warn(
{}INFO Final process status is success
INFO [provenance] Finalizing Research Object
INFO [provenance] Deleting existing /private/tmp/cwl-prov/ro-crate
INFO [provenance] Research Object saved to /private/tmp/cwl-prov/ro-crate
```

Note further that it is only when requesting provenance that this error occurs.

Program `no-prov.py`:
```python
import sys

from cwltool.main import main as cwlmain
from cwltool.argparser import arg_parser

unparsed_args = ["hello_world.cwl", "--message=Hello"]
parsed_args = arg_parser().parse_args(unparsed_args)

cwlmain(args=parsed_args)
```

```
INFO no-prov.py 3.1.20230719185429
INFO Resolved 'hello_world.cwl' to 'file:///private/tmp/cwl-prov/hello_world.cwl'
INFO [job hello_world.cwl] /private/tmp/docker_tmpt2b9sgm2$ echo \
    Hello
Hello
INFO [job hello_world.cwl] completed success
{}INFO Final process status is success
```


Discussion:
This blocks requesting provenance when using [Calrissian](https://github.com/Duke-GCB/calrissian), which calls `cwltool.main` directly with preparsed arguments and no unparsed arguments.
This issue has been mentioned on **Common Workflow Language Discourse**. There might be relevant details there:

https://cwl.discourse.group/t/cwltool-main-text-arguments-a-hard-requirement-for-provenance/862/2

Dear  @davidjsherman , thank you for the issue and your interest in supporting CWLProv on Calrissian!

I think this is a typing error and I would accept a PR to fix it by removing the exception and adjusting the types to be `Optional`.

Resolving patch:
diff --git a/cwltool/main.py b/cwltool/main.py
--- a/cwltool/main.py
+++ b/cwltool/main.py
@@ -682,8 +682,8 @@ def formatTime(self, record: logging.LogRecord, datefmt: Optional[str] = None) -
 
 def setup_provenance(
     args: argparse.Namespace,
-    argsl: List[str],
     runtimeContext: RuntimeContext,
+    argsl: Optional[List[str]] = None,
 ) -> Tuple[ProvOut, "logging.StreamHandler[ProvOut]"]:
     if not args.compute_checksum:
         _logger.error("--provenance incompatible with --no-compute-checksum")
@@ -1049,10 +1049,8 @@ def main(
 
         prov_log_stream: Optional[Union[io.TextIOWrapper, WritableBagFile]] = None
         if args.provenance:
-            if argsl is None:
-                raise Exception("argsl cannot be None")
             try:
-                prov_log_stream, prov_log_handler = setup_provenance(args, argsl, runtimeContext)
+                prov_log_stream, prov_log_handler = setup_provenance(args, runtimeContext, argsl)
             except ArgumentException:
                 return 1
 


Test patch:
diff --git a/tests/test_main_parsed_args.py b/tests/test_main_parsed_args.py
new file mode 100644
--- /dev/null
+++ b/tests/test_main_parsed_args.py
@@ -0,0 +1,40 @@
+import io
+from pathlib import Path
+
+from cwltool.argparser import arg_parser
+from cwltool.main import main
+
+from .util import get_data
+
+
+def test_main_parsed_args(tmp_path: Path) -> None:
+    """Affirm that main can be called with parsed args only."""
+    stdout = io.StringIO()
+    stderr = io.StringIO()
+
+    unparsed_args = [get_data("tests/echo.cwl"), "--inp", "Hello"]
+    parsed_args = arg_parser().parse_args(unparsed_args)
+
+    try:
+        assert main(args=parsed_args, stdout=stdout, stderr=stderr) == 0
+    except SystemExit as err:
+        assert err.code == 0
+
+
+def test_main_parsed_args_provenance(tmp_path: Path) -> None:
+    """Affirm that main can be called with parsed args only, requesting provenance."""
+    stdout = io.StringIO()
+    stderr = io.StringIO()
+
+    prov_folder = tmp_path / "provenance"  # will be created if necessary
+
+    unparsed_args = ["--provenance", str(prov_folder), get_data("tests/echo.cwl"), "--inp", "Hello"]
+    parsed_args = arg_parser().parse_args(unparsed_args)
+
+    try:
+        assert main(args=parsed_args, stdout=stdout, stderr=stderr) == 0
+    except SystemExit as err:
+        assert err.code == 0
+
+    manifest_file = prov_folder / "metadata" / "manifest.json"
+    assert manifest_file.is_file(), f"Can't find RO-Crate manifest {manifest_file}"