internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: common-workflow-language/cwltool

Issue:
Information about javascript timeouts not printed to the console
At https://github.com/common-workflow-language/cwltool/blob/f454efb1e980b2a9374afba4a8b793f73f4d891a/cwltool/sandboxjs.py#L297-L301, if `debug` is false, no information about the snippet of javascript that caused the timeout is printed.
Information about javascript timeouts not printed to the console
At https://github.com/common-workflow-language/cwltool/blob/f454efb1e980b2a9374afba4a8b793f73f4d891a/cwltool/sandboxjs.py#L297-L301, if `debug` is false, no information about the snippet of javascript that caused the timeout is printed.


Discussion:

Resolving patch:
diff --git a/cwltool/sandboxjs.py b/cwltool/sandboxjs.py
--- a/cwltool/sandboxjs.py
+++ b/cwltool/sandboxjs.py
@@ -292,7 +292,8 @@ def stdfmt(data):  # type: (Text) -> Text
         info = u"returncode was: %s\nscript was:\n%s\nstdout was: %s\nstderr was: %s\n" %\
                (nodejs.returncode, fn_linenum(), stdfmt(stdoutdata.decode('utf-8')), stdfmt(stderrdata.decode('utf-8')))
     else:
-        info = stdfmt(stderrdata.decode('utf-8'))
+        info = u"Javascript expression was: %s\nstdout was: %s\nstderr was: %s" %\
+               (js, stdfmt(stdoutdata.decode('utf-8')), stdfmt(stderrdata.decode('utf-8')))
 
     if nodejs.poll() not in (None, 0):
         if killed:


Test patch:
Repository: common-workflow-language/cwltool

Issue:
singularity image pull triggered for every step when using beegfs



Discussion:

Resolving patch:
diff --git a/cwltool/singularity.py b/cwltool/singularity.py
--- a/cwltool/singularity.py
+++ b/cwltool/singularity.py
@@ -76,16 +76,18 @@ def get_image(dockerRequirement,  # type: Dict[Text, Text]
             candidates.append(_normalizeImageId(dockerRequirement['dockerImageId']))
 
         # check if Singularity image is available in $SINGULARITY_CACHEDIR
-        for target in ("SINGULARITY_CACHEDIR", "SINGULARITY_PULLFOLDER",
-                os.getcwd()):
-            if target in os.environ:
-                for candidate in candidates:
-                    path = os.path.join(os.environ[target], candidate)
-                    if os.path.isfile(path):
-                        _logger.info("Using local copy of Singularity image "
-                                     "found in {}".format(target))
-                        dockerRequirement["dockerImageId"] = path
-                        found = True
+        targets = [os.getcwd()]
+        for env in ("SINGULARITY_CACHEDIR", "SINGULARITY_PULLFOLDER"):
+            if env in os.environ:
+                targets.append(os.environ[env])
+        for target in targets:
+            for candidate in candidates:
+                path = os.path.join(target, candidate)
+                if os.path.isfile(path):
+                    _logger.info("Using local copy of Singularity image "
+                                 "found in {}".format(target))
+                    dockerRequirement["dockerImageId"] = path
+                    found = True
 
         if (force_pull or not found) and pull_image:
             cmd = []  # type: List[Text]


Test patch:
Repository: common-workflow-language/cwltool

Issue:
Setup cwltool.readthedocs.org
https://docs.readthedocs.io/en/stable/intro/getting-started-with-sphinx.html


Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
new file mode 100644
--- /dev/null
+++ b/docs/conf.py
@@ -0,0 +1,60 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath('..'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'Common Workflow Language reference implementation'
+copyright = '2019, Peter Amstutz and contributors'
+author = 'Peter Amstutz and contributors'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+        "sphinx.ext.autodoc",
+        "sphinx.ext.autosummary",
+        "sphinx_rtd_theme",
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = "sphinx_rtd_theme"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+
+from pkg_resources import get_distribution
+release = get_distribution('cwltool').version
+version = '.'.join(release.split('.')[:2])
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -80,7 +80,8 @@
       extras_require={
           ':os.name=="posix" and python_version<"3.5"': ['subprocess32 >= 3.5.0'],
           ':python_version<"3.6"': ['typing >= 3.5.3'],
-          'deps': ["galaxy-lib >= 17.09.9, <= 18.9.2 "]
+          'deps': ["galaxy-lib >= 17.09.9, <= 18.9.2 "],
+          'docs': ["sphinx >= 2.2", "sphinx-rtd-theme"],
       },
       python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4',
       setup_requires=PYTEST_RUNNER,


Test patch:
Repository: common-workflow-language/cwltool

Issue:
loosen ruaml.yaml version requirement?
Good morning. I am interested in the very specific version requirement for ruaml.yaml. This causes me a bit of dependency conflict. Can I help update something in cwltool that would allow for a version of ruaml.yaml > 0.16.0 to be used?


Discussion:
The author of ruamel.yaml has repeatedly released point releases that have broken our code, so we have had to set conservative version constraints on the highest version that has been tested.  It may be that 0.16 works fine, it just hasn't been tested.

To update it, there's two things you need to do.  First, cwltool depends on schema_salad, so it needs a pull request to update its version pin on ruamel.yaml.   The pull request will run the schema_salad tests.  If all the tests pass with ruamel.yaml 0.16.x then we can merge it and make a new schema salad release.

https://github.com/common-workflow-language/schema_salad

Then you can update the version pin on cwltool.  The pull request will run the cwltool tests.  If all the tests pass with ruamel.yaml 0.16.x then we can merge it and make a new cwltool release.
Great, I'll have a go at it. Thanks!
I submitted a PR over at schema-salad. It looks like everything passes with ruaml.yaml 0.16.5 (the newest). I requested you as a reviewer.
https://github.com/common-workflow-language/schema_salad/pull/313

Resolving patch:
diff --git a/cwltool/argparser.py b/cwltool/argparser.py
--- a/cwltool/argparser.py
+++ b/cwltool/argparser.py
@@ -766,11 +766,11 @@ def add_argument(
     else:
         typekw = {}
 
-    toolparser.add_argument(  # type: ignore
+    toolparser.add_argument(
         flag + name,
         required=required,
         help=ahelp,
-        action=action,
+        action=action,  # type: ignore
         default=default,
         **typekw
     )
diff --git a/cwltool/validate_js.py b/cwltool/validate_js.py
--- a/cwltool/validate_js.py
+++ b/cwltool/validate_js.py
@@ -83,7 +83,7 @@ def get_expressions(
             itertools.chain(
                 *map(
                     lambda x: get_expressions(
-                        x[1], schema.items, SourceLine(tool, x[0])
+                        x[1], schema.items, SourceLine(tool, x[0])  # type: ignore
                     ),
                     enumerate(tool),
                 )
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -58,7 +58,7 @@
         "setuptools",
         "requests >= 2.6.1",  # >= 2.6.1 to workaround
         # https://github.com/ionrock/cachecontrol/issues/137
-        "ruamel.yaml >= 0.12.4, <= 0.16",
+        "ruamel.yaml >= 0.12.4, <= 0.16.5",
         "rdflib >= 4.2.2, < 4.3.0",
         "shellescape >= 3.4.1, < 3.5",
         "schema-salad >= 5, < 6",


Test patch:
Repository: common-workflow-language/cwltool

Issue:
SyntaxWarning when running on python v3.8.2
## Expected Behavior
No warnings present when running `cwltool --version`

## Actual Behavior
The following two lines are written to stderr
```
/home/alexiswl/anaconda3/envs/cwl/lib/python3.8/site-packages/cwltool-3.0.20200324120055-py3.8.egg/cwltool/docker.py:423: SyntaxWarning: "is" with a literal. Did you mean "=="?
/home/alexiswl/anaconda3/envs/cwl/lib/python3.8/site-packages/cwltool-3.0.20200324120055-py3.8.egg/cwltool/docker.py:423: SyntaxWarning: "is" with a literal. Did you mean "=="?
```

## Workflow Code
```
cwltool --version
```

## Full Traceback
N/A

## Your Environment
* cwltool version: 
Check using ``cwltool --version``

Version: 3.0.20200324120055
Python Version: 3.8.2
Conda Version: 4.8.3
OS-release: Ubuntu 18.04.4 LTS (Running via WSL2)

Installation steps:
```
conda create --name cwl python=3.8
conda activate cwl
pip install cwltool
cwltool --version
```


Discussion:

Resolving patch:
diff --git a/cwltool/docker.py b/cwltool/docker.py
--- a/cwltool/docker.py
+++ b/cwltool/docker.py
@@ -420,7 +420,7 @@ def create_runtime(
             runtime.append("--memory=%dm" % self.builder.resources["ram"])
         elif not user_space_docker_cmd:
             res_req, _ = self.builder.get_requirement("ResourceRequirement")
-            if res_req is not None and ("ramMin" in res_req or "ramMax" is res_req):
+            if res_req and ("ramMin" in res_req or "ramMax" in res_req):
                 _logger.warning(
                     "[job %s] Skipping Docker software container '--memory' limit "
                     "despite presence of ResourceRequirement with ramMin "


Test patch:
Repository: common-workflow-language/cwltool

Issue:
drop Python3.5 support on 2020-09-13
Something to look forward to :-)

https://devguide.python.org/#status-of-python-branches


Branch | Schedule | Status | First release | End-of-life
-- | -- | -- | -- | --
3.5 | PEP 478 | security | 2015-09-13 | 2020-09-13









Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -107,11 +107,10 @@
         'pydot >= 1.4.1',
     ],
     extras_require={
-        ':python_version<"3.6"': ["typing >= 3.5.3"],
         "deps": ["galaxy-tool-util"],
         "docs": ["sphinx >= 2.2", "sphinx-rtd-theme"],
     },
-    python_requires=">=3.5, <4",
+    python_requires=">=3.6, <4",
     setup_requires=PYTEST_RUNNER,
     test_suite="tests",
     tests_require=[
@@ -142,7 +141,6 @@
         # 'Operating System :: Microsoft :: Windows :: Windows 8',  # not tested
         # 'Operating System :: Microsoft :: Windows :: Windows 7',  # not tested
         "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.5",
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",


Test patch:
Repository: common-workflow-language/cwltool

Issue:
Add command line autocompletion support using "argcomplete"
https://github.com/kislyuk/argcomplete/blob/develop/README.rst#synopsis

- [ ] Add dependency on `argcomplete`
- [ ] Add `# PYTHON_ARGCOMPLETE_OK` to `cwltool/main.py`
- [ ] Add call to `argcomplete.autocomplete()` just before `arg_parser().parse_args()` in `cwltool/main.py`
- [ ] test manually


Discussion:

Resolving patch:
diff --git a/cwltool/main.py b/cwltool/main.py
--- a/cwltool/main.py
+++ b/cwltool/main.py
@@ -1,6 +1,7 @@
 #!/usr/bin/env python3
 """Entry point for cwltool."""
 
+import argcomplete
 import argparse
 import functools
 import io
@@ -100,7 +101,7 @@
     windows_default_container_id,
 )
 from .workflow import Workflow
-
+# PYTHON_ARGCOMPLETE_OK
 
 def _terminate_processes() -> None:
     """Kill all spawned processes.
@@ -396,7 +397,7 @@ def init_job_order(
                 k: v for k, v in cmd_line.items() if k.startswith(record_name)
             }
             for key, value in record_items.items():
-                record[key[len(record_name) + 1 :]] = value
+                record[key[len(record_name) + 1:]] = value
                 del cmd_line[key]
             cmd_line[str(record_name)] = record
         if "job_order" in cmd_line and cmd_line["job_order"]:
@@ -854,7 +855,9 @@ def main(
             addl = []  # type: List[str]
             if "CWLTOOL_OPTIONS" in os.environ:
                 addl = os.environ["CWLTOOL_OPTIONS"].split(" ")
-            args = arg_parser().parse_args(addl + argsl)
+            parser = arg_parser()
+            argcomplete.autocomplete(parser)
+            args = parser.parse_args(addl + argsl)
             if args.record_container_id:
                 if not args.cidfile_dir:
                     args.cidfile_dir = os.getcwd()
@@ -895,7 +898,7 @@ def main(
                 args.workflow = "CWLFile"
             else:
                 _logger.error("CWL document required, no input file was provided")
-                arg_parser().print_help()
+                parser.print_help()
                 return 1
         if args.relax_path_checks:
             command_line_tool.ACCEPTLIST_RE = command_line_tool.ACCEPTLIST_EN_RELAXED_RE
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -105,6 +105,7 @@
         "typing-extensions",
         "coloredlogs",
         "pydot >= 1.4.1",
+        "argcomplete",
     ],
     extras_require={
         "deps": ["galaxy-tool-util"],


Test patch:
Repository: common-workflow-language/cwltool

Issue:
--print-targets doesn't work with inlined processes
Needed so that the results of `cwlpack` from https://github.com/rabix/sbpack works against https://github.com/common-workflow-language/cwl-v1.2/blob/main/tests/timelimit2-wf.cwl


Discussion:

Resolving patch:
diff --git a/cwltool/main.py b/cwltool/main.py
--- a/cwltool/main.py
+++ b/cwltool/main.py
@@ -929,9 +929,11 @@ def print_targets(
         _logger.info("%s steps targets:", prefix[:-1])
         for t in tool.tool["steps"]:
             stdout.write(f"  {prefix}{shortname(t['id'])}\n")
-            run: Union[str, Process] = t["run"]
+            run: Union[str, Process, Dict[str, Any]] = t["run"]
             if isinstance(run, str):
                 process = make_tool(run, loading_context)
+            elif isinstance(run, dict):
+                process = make_tool(cast(CommentedMap, cmap(run)), loading_context)
             else:
                 process = run
             print_targets(process, stdout, loading_context, shortname(t["id"]) + "/")


Test patch:
diff --git a/tests/subgraph/timelimit2-wf.cwl b/tests/subgraph/timelimit2-wf.cwl
new file mode 100644
--- /dev/null
+++ b/tests/subgraph/timelimit2-wf.cwl
@@ -0,0 +1,49 @@
+#!/usr/bin/env cwl-runner
+class: Workflow
+cwlVersion: v1.2
+
+requirements:
+  ToolTimeLimit:
+    timelimit: 5
+  InlineJavascriptRequirement: {}
+
+inputs:
+  i:
+    type: string?
+
+outputs:
+  o:
+    type: string?
+    outputSource: step2/o
+
+steps:
+  step1:
+    in:
+      i: i
+    out: [o]
+    run:
+      class: CommandLineTool
+      baseCommand: ["sleep", "3"]
+      inputs:
+        i:
+          type: string?
+      outputs:
+        o:
+          type: string?
+          outputBinding:
+            outputEval: $("time passed")
+  step2:
+    in:
+      i: step1/o
+    out: [o]
+    run:
+      class: CommandLineTool
+      baseCommand: ["sleep", "3"]
+      inputs:
+        i:
+          type: string?
+      outputs:
+        o:
+          type: string?
+          outputBinding:
+            outputEval: $("time passed")
diff --git a/tests/test_subgraph.py b/tests/test_subgraph.py
--- a/tests/test_subgraph.py
+++ b/tests/test_subgraph.py
@@ -240,3 +240,14 @@ def test_single_step_packed_subwf_step() -> None:
         json.loads(stdout)["out"]["checksum"]
         == "sha1$7608e5669ba454c61fab01c9b133b52a9a7de68c"
     )
+
+
+def test_print_targets_embedded_step() -> None:
+    """Confirm that --print-targets works when a Workflow has embedded Processes."""
+    err_code, stdout, stderr = get_main_output(
+        [
+            "--print-targets",
+            get_data("tests/subgraph/timelimit2-wf.cwl"),
+        ]
+    )
+    assert err_code == 0
Repository: common-workflow-language/cwltool

Issue:
Add helpful error message when a program is terminated by a (Unix) signal, such as a segfault
Currently one gets a error as the expected outputs are not available, which is technically true but unhelpful.

- [ ] Add helpful error message for [return code 139](https://stackoverflow.com/questions/14599670/what-error-code-does-a-process-that-segfaults-return)
- [ ] Add a test for this behaviour

Thanks to @gijzelaerr  for the idea



Discussion:
In general we should emphasis failing return codes over missing outputs, so there may be additional work to do here
A good start would be adding a `_logging.debug` entry for the value of the return code in https://github.com/common-workflow-language/cwltool/blob/352daf6d53f12df4cd960c0343ef659924aa9712/cwltool/job.py#L346
I think the title should be changed to "Add helpful error message when a program is terminated by a (Unix) signal", to make it more generic than just catching segmentation faults.
Who or what sets `self.successCodes`, `self.temporaryFailCodes`, and `self.permanentFailCodes`? I can see these are set in lines 967-969 of `command_line_tool.py`, using `self.tool.get()`, and I see that `self.tool` is assigned a `toolpath_object` in line 586 of `process.py`, but then I'm lost because `toolpath_object` can be one of many objects.
@gmloose they come from the CWL `CommandLineTool` description: https://www.commonwl.org/v1.0/CommandLineTool.html#CommandLineTool 

Resolving patch:
diff --git a/cwltool/job.py b/cwltool/job.py
--- a/cwltool/job.py
+++ b/cwltool/job.py
@@ -5,6 +5,7 @@
 import os
 import re
 import shutil
+import signal
 import stat
 import subprocess  # nosec
 import sys
@@ -354,6 +355,16 @@ def _execute(
             else:
                 processStatus = "permanentFail"
 
+            if processStatus != "success":
+                if rcode < 0:
+                    _logger.warning(
+                        "[job %s] was terminated by signal: %s",
+                        self.name,
+                        signal.Signals(-rcode).name,
+                    )
+                else:
+                    _logger.warning("[job %s] exited with status: %d", self.name, rcode)
+
             if "listing" in self.generatefiles:
                 if self.generatemapper:
                     relink_initialworkdir(


Test patch:
Repository: common-workflow-language/cwltool

Issue:
Allow interpreted subclasses
I have recently attempted updating my code base (https://github.com/crim-ca/weaver), that uses `cwltool` to run CWL workflows with alternative tools, with latest versions (moving from old https://github.com/common-workflow-language/cwltool/tree/3.0.20200324120055 to recent https://github.com/common-workflow-language/cwltool/tree/3.1.20220913185150)

I encountered the following error during this process: 
```python
TypeError interpreted classes cannot inherit from compiled 
```
when calling this code in order to generate custom tool definitions:
https://github.com/crim-ca/weaver/blob/master/weaver/processes/wps_workflow.py#L81-L87

I have managed to work around this using:

```python
from typing import cast

cast(WpsWorkflow, WpsWorkflow(toolpath_object, loading_context, get_job_process_definition))
```

## Expected Behavior

I was wondering if the following definition applied to `cwltool.process.Process` would avoid this `cast` hack?

https://mypyc.readthedocs.io/en/latest/native_classes.html
```python
@mypyc_attr(allow_interpreted_subclasses=True)
```


## Actual Behavior
Allow directly inheriting from the Process class without typing cast workaround.

## Workflow Code
n/a

## Full Traceback
n/a

## Your Environment
* cwltool version: https://github.com/common-workflow-language/cwltool/tree/3.1.20220913185150




Discussion:
Good catch; sorry I forgot about weaver when checking which classes are sub-classed;

Can you send a PR?

Resolving patch:
diff --git a/cwltool/process.py b/cwltool/process.py
--- a/cwltool/process.py
+++ b/cwltool/process.py
@@ -32,6 +32,7 @@
 )
 
 from cwl_utils import expression
+from mypy_extensions import mypyc_attr
 from pkg_resources import resource_stream
 from rdflib import Graph
 from ruamel.yaml.comments import CommentedMap, CommentedSeq
@@ -535,6 +536,7 @@ def eval_resource(
 FILE_COUNT_WARNING = 5000
 
 
+@mypyc_attr(allow_interpreted_subclasses=True)
 class Process(HasReqsHints, metaclass=abc.ABCMeta):
     def __init__(
         self, toolpath_object: CommentedMap, loadingContext: LoadingContext
diff --git a/cwltool/utils.py b/cwltool/utils.py
--- a/cwltool/utils.py
+++ b/cwltool/utils.py
@@ -39,7 +39,7 @@
 import requests
 from cachecontrol import CacheControl
 from cachecontrol.caches import FileCache
-from mypy_extensions import TypedDict
+from mypy_extensions import TypedDict, mypyc_attr
 from schema_salad.exceptions import ValidationException
 from schema_salad.ref_resolver import Loader
 from typing_extensions import TYPE_CHECKING, Deque
@@ -502,6 +502,7 @@ def create_tmp_dir(tmpdir_prefix: str) -> str:
     return tempfile.mkdtemp(prefix=tmp_prefix, dir=tmp_dir)
 
 
+@mypyc_attr(allow_interpreted_subclasses=True)
 class HasReqsHints:
     """Base class for get_requirement()."""
 


Test patch: