Repository: DataBiosphere/toil

Issue:
regression: CWL conformance test "simple workflow with a dynamic resource requirement"
https://ci.commonwl.org/job/toil-conformance/558/testReport/conformance_test_v1/0/Test_simple_workflow_with_a_dynamic_resource_requirement/

from build https://ci.commonwl.org/job/toil-conformance/558/
Changes
added encryption test (commit: b987d2e) 
Removed fileJobStore encryption test (commit: ea59818) 
Add a more robust deleter to the file job store (commit: ca93b2c)
Avoid eagerly loading all versions on AWS jobstore destruction (commit: ec231d9)


Discussion:
So it could be https://github.com/BD2KGenomics/toil/commit/ca93b2c361c0a69d22ad2c99100af7359676f654 ?
@mr-c , where can I find this test? I updated the cwltest version, but still can't find it.
@ejacox `cwltest` is just a test runner; the conformance tests are listed in https://github.com/common-workflow-language/common-workflow-language/blob/master/v1.0/conformance_test_v1.0.yaml

Here is the test in question: https://github.com/common-workflow-language/common-workflow-language/blob/master/v1.0/conformance_test_v1.0.yaml#L1515

From https://ci.commonwl.org/job/toil-conformance/558/testReport/conformance_test_v1/0/Test_simple_workflow_with_a_dynamic_resource_requirement/ we see the command that was run inside https://github.com/common-workflow-language/common-workflow-language/blob/master/v1.0/

```
cwltoil --outdir=/tmp/tmpEIkClr --quiet v1.0/dynresreq-workflow.cwl v1.0/dynresreq-job.json
```

(~~oops, I need to update our CI to use `toil-cwl-runner` instead of `cwltoil`~~ This has been fixed.)

To reproduce this test you can leave out the `--outdir=/tmp/tmpEIkClr` and `--quiet` parts.

As you know, your running of the CWL conformance tests is controlled by https://github.com/BD2KGenomics/toil/blob/master/src/toil/test/cwl/cwlTest.py#L131 which is pulling in a version of the conformances tests prior to the introduction of this test: https://github.com/common-workflow-language/common-workflow-language/pull/626
Drilling down on https://ci.commonwl.org/job/toil-conformance/test_results_analyzer/ we can see that this test was passing until our build 557 of toil (2018-03-28)

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -23,7 +23,7 @@ def runSetup():
     boto = 'boto==2.38.0'
     boto3 = 'boto3==1.4.7'
     futures = 'futures==3.0.5'
-    pycryptodome = 'pycryptodome==3.5.0'
+    pycryptodome = 'pycryptodome==3.5.1'
     psutil = 'psutil==3.0.1'
     protobuf = 'protobuf==3.5.1'
     azure = 'azure==2.0.0'
@@ -35,10 +35,10 @@ def runSetup():
     gcs = 'google-cloud-storage==1.6.0'
     gcs_oauth2_boto_plugin = 'gcs_oauth2_boto_plugin==1.14'
     apacheLibcloud = 'apache-libcloud==2.2.1'
-    cwltool = 'cwltool==1.0.20180306140409'
+    cwltool = 'cwltool==1.0.20180403145700'
     schemaSalad = 'schema-salad >= 2.6, < 3'
     galaxyLib = 'galaxy-lib==17.9.3'
-    cwltest = 'cwltest>=1.0.20180130081614'
+    cwltest = 'cwltest>=1.0.20180209171722'
     htcondor = 'htcondor>=8.6.0'
 
     mesos_reqs = [
diff --git a/src/toil/cwl/cwltoil.py b/src/toil/cwl/cwltoil.py
--- a/src/toil/cwl/cwltoil.py
+++ b/src/toil/cwl/cwltoil.py
@@ -45,6 +45,7 @@
         DependenciesConfiguration, get_container_from_software_requirements)
 from cwltool.utils import aslist
 import schema_salad.validate as validate
+from ruamel.yaml.comments import CommentedSeq
 import schema_salad.ref_resolver
 import os
 import tempfile
@@ -742,8 +743,13 @@ def run(self, fileStore):
                                         raise validate.ValidationException(
                                             "Unsupported linkMerge '%s'", linkMerge)
                                 else:
-                                    jobobj[key] = (shortname(inp["source"]),
-                                                   promises[inp["source"]].rv())
+                                    inputSource = inp["source"]
+                                    if isinstance(inputSource, CommentedSeq):
+                                        # It seems that an input source with a '#' in the name will be
+                                        # returned as a CommentedSeq list by the yaml parser.
+                                        inputSource = str(inputSource[0])
+                                    jobobj[key] = (shortname(inputSource),
+                                                   promises[inputSource].rv())
                             elif "default" in inp:
                                 d = copy.copy(inp["default"])
                                 jobobj[key] = ("default", {"default": d})


Test patch:
diff --git a/src/toil/test/cwl/cwlTest.py b/src/toil/test/cwl/cwlTest.py
--- a/src/toil/test/cwl/cwlTest.py
+++ b/src/toil/test/cwl/cwlTest.py
@@ -128,7 +128,8 @@ def path_without_rev():
     def test_run_conformance(self, batchSystem=None):
         rootDir = self._projectRootPath()
         cwlSpec = os.path.join(rootDir, 'src/toil/test/cwl/spec')
-        testhash = "91f108df4d4ca567e567fc65f61feb0674467a84"
+        # The latest cwl git hash. Update it to get the latest tests.
+        testhash = "f96bca6911b6688ff614c02dbefe819bed260a13"
         url = "https://github.com/common-workflow-language/common-workflow-language/archive/%s.zip" % testhash
         if not os.path.exists(cwlSpec):
             urlretrieve(url, "spec.zip")