Repository: common-workflow-language/cwltool

Issue:
Caching fails to work properly when using the same dockerised tool in different steps
https://github.com/common-workflow-language/cwltool/blob/9e3e720ffc18ade0a19fdbb2bea1bed25e9662ae/cwltool/command_line_tool.py#L299-L302

This uses self.requirement and self.hints, which are mutated when a tool is run:

https://github.com/common-workflow-language/cwltool/blob/9e3e720ffc18ade0a19fdbb2bea1bed25e9662ae/cwltool/docker.py#L39-L40

If a workflow uses the same tool multiple times, whether a cache hit occurs for a step will depend on whether the same tool has been run before or not.

## Example

### Inputs

<details><summary>test_workflow.cwl</summary>
<p>

    #!/usr/bin/env cwl-runner

    cwlVersion: v1.0
    class: Workflow

    inputs: []
    outputs: []

    steps:
    task1:
        run: test_tool.cwl
        in:
        message:
            default: one
        out: []
    task2:
        run: test_tool.cwl
        in:
        message:
            default: two
        out: []
</p>
</details>

<details><summary>test_tool.cwl</summary>
<p>

    #!/usr/bin/env cwl-runner

    cwlVersion: v1.0
    class: CommandLineTool
    baseCommand: touch
    requirements:
    DockerRequirement:
        dockerPull: alpine
    inputs:
    message:
        type: string
        inputBinding:
        position: 1
    outputs:
    - id: out
        type: File
        outputBinding:
        glob: "*"
</p>
</details>

### Runs

<details><summary>Run 1</summary>
<p>

    $ python -m cwltool --debug --cachedir test_cache test_workflow.cwl
    /Users/th10/checkouts/cwltool/cwltool/__main__.py 1.0.20180324152842
    Resolved 'test_workflow.cwl' to 'file:///Users/th10/checkouts/cwltool/test_workflow.cwl'
    Parsed job order from command line: {
        "id": "test_workflow.cwl"
    }
    [workflow test_workflow.cwl] initialized from file:///Users/th10/checkouts/cwltool/test_workflow.cwl
    [workflow test_workflow.cwl] start
    [workflow test_workflow.cwl] starting step task1
    [job step task1] job input {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task1/message": "one"
    }
    [job step task1] evaluated job input to {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task1/message": "one"
    }
    [step task1] start
    [job task1] keydictstr is {"DockerRequirement":{"class":"DockerRequirement","dockerPull":"alpine"},"cmdline":["docker","run","alpine","touch","one"]} -> 3a88555659d51a9ec2330d04380240f8
    [job task1] Output of job will be cached in /Users/th10/checkouts/cwltool/test_cache/3a88555659d51a9ec2330d04380240f8
    [job task1] initializing from file:///Users/th10/checkouts/cwltool/test_tool.cwl as part of step task1
    [job task1] {
        "message": "one"
    }
    [job task1] path mappings is {}
    [job task1] command line bindings is [
        {
            "position": [
                -1000000,
                0
            ],
            "datum": "touch"
        },
        {
            "position": [
                1,
                "message"
            ],
            "datum": "one"
        }
    ]
    [job task1] /Users/th10/checkouts/cwltool/test_cache/3a88555659d51a9ec2330d04380240f8$ docker \
        run \
        -i \
        --volume=/Users/th10/checkouts/cwltool/test_cache/3a88555659d51a9ec2330d04380240f8:/private/var/spool/cwl:rw \
        --volume=/private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmp2l4eeu6t:/tmp:rw \
        --workdir=/private/var/spool/cwl \
        --read-only=true \
        --user=16442:1312 \
        --rm \
        --env=TMPDIR=/tmp \
        --env=HOME=/private/var/spool/cwl \
        alpine \
        touch \
        one
    [job task1] completed success
    [job task1] {
        "out": {
            "location": "file:///Users/th10/checkouts/cwltool/test_cache/3a88555659d51a9ec2330d04380240f8/one",
            "basename": "one",
            "nameroot": "one",
            "nameext": "",
            "class": "File",
            "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
            "size": 0,
            "http://commonwl.org/cwltool#generation": 0
        }
    }
    [step task1] produced output {}
    [step task1] completed success
    [job task1] Removing input staging directory /private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmpdnakizoe
    [job task1] Removing temporary directory /private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmp2l4eeu6t
    [workflow test_workflow.cwl] starting step task2
    [job step task2] job input {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task2/message": "two"
    }
    [job step task2] evaluated job input to {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task2/message": "two"
    }
    [step task2] start
    [job task2] keydictstr is {"DockerRequirement":{"class":"DockerRequirement","dockerImageId":"alpine","dockerPull":"alpine"},"cmdline":["docker","run","alpine","touch","two"]} -> 6cfa521e843e7c924b71f93e1156b7d5
    [job task2] Output of job will be cached in /Users/th10/checkouts/cwltool/test_cache/6cfa521e843e7c924b71f93e1156b7d5
    [job task2] initializing from file:///Users/th10/checkouts/cwltool/test_tool.cwl as part of step task2
    [job task2] {
        "message": "two"
    }
    [job task2] path mappings is {}
    [job task2] command line bindings is [
        {
            "position": [
                -1000000,
                0
            ],
            "datum": "touch"
        },
        {
            "position": [
                1,
                "message"
            ],
            "datum": "two"
        }
    ]
    [job task2] /Users/th10/checkouts/cwltool/test_cache/6cfa521e843e7c924b71f93e1156b7d5$ docker \
        run \
        -i \
        --volume=/Users/th10/checkouts/cwltool/test_cache/6cfa521e843e7c924b71f93e1156b7d5:/private/var/spool/cwl:rw \
        --volume=/private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmp18ovy8e_:/tmp:rw \
        --workdir=/private/var/spool/cwl \
        --read-only=true \
        --user=16442:1312 \
        --rm \
        --env=TMPDIR=/tmp \
        --env=HOME=/private/var/spool/cwl \
        alpine \
        touch \
        two
    [job task2] completed success
    [job task2] {
        "out": {
            "location": "file:///Users/th10/checkouts/cwltool/test_cache/6cfa521e843e7c924b71f93e1156b7d5/two",
            "basename": "two",
            "nameroot": "two",
            "nameext": "",
            "class": "File",
            "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
            "size": 0,
            "http://commonwl.org/cwltool#generation": 0
        }
    }
    [step task2] produced output {}
    [step task2] completed success
    [workflow test_workflow.cwl] completed success
    [job task2] Removing input staging directory /private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmp3hswvlmo
    [job task2] Removing temporary directory /private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmp18ovy8e_
    Removing intermediate output directory /Users/th10/checkouts/cwltool/test_cache/mmtlw31b
    {}
    Final process status is success

</p>
</details>

<details><summary>Run 2</summary>
<p>

    $ python -m cwltool --debug --cachedir test_cache test_workflow.cwl
    /Users/th10/checkouts/cwltool/cwltool/__main__.py 1.0.20180324152842
    Resolved 'test_workflow.cwl' to 'file:///Users/th10/checkouts/cwltool/test_workflow.cwl'
    Parsed job order from command line: {
        "id": "test_workflow.cwl"
    }
    [workflow test_workflow.cwl] initialized from file:///Users/th10/checkouts/cwltool/test_workflow.cwl
    [workflow test_workflow.cwl] start
    [workflow test_workflow.cwl] starting step task1
    [job step task1] job input {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task1/message": "one"
    }
    [job step task1] evaluated job input to {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task1/message": "one"
    }
    [step task1] start
    [job task1] keydictstr is {"DockerRequirement":{"class":"DockerRequirement","dockerPull":"alpine"},"cmdline":["docker","run","alpine","touch","one"]} -> 3a88555659d51a9ec2330d04380240f8
    [job task1] Using cached output in /Users/th10/checkouts/cwltool/test_cache/3a88555659d51a9ec2330d04380240f8
    [step task1] produced output {}
    [step task1] completed success
    [workflow test_workflow.cwl] starting step task2
    [job step task2] job input {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task2/message": "two"
    }
    [job step task2] evaluated job input to {
        "file:///Users/th10/checkouts/cwltool/test_workflow.cwl#task2/message": "two"
    }
    [step task2] start
    [job task2] keydictstr is {"DockerRequirement":{"class":"DockerRequirement","dockerPull":"alpine"},"cmdline":["docker","run","alpine","touch","two"]} -> 5115cb5a928586f413deadf8090375a9
    [job task2] Output of job will be cached in /Users/th10/checkouts/cwltool/test_cache/5115cb5a928586f413deadf8090375a9
    [job task2] initializing from file:///Users/th10/checkouts/cwltool/test_tool.cwl as part of step task2
    [job task2] {
        "message": "two"
    }
    [job task2] path mappings is {}
    [job task2] command line bindings is [
        {
            "position": [
                -1000000,
                0
            ],
            "datum": "touch"
        },
        {
            "position": [
                1,
                "message"
            ],
            "datum": "two"
        }
    ]
    [job task2] /Users/th10/checkouts/cwltool/test_cache/5115cb5a928586f413deadf8090375a9$ docker \
        run \
        -i \
        --volume=/Users/th10/checkouts/cwltool/test_cache/5115cb5a928586f413deadf8090375a9:/private/var/spool/cwl:rw \
        --volume=/private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmph12do9a5:/tmp:rw \
        --workdir=/private/var/spool/cwl \
        --read-only=true \
        --user=16442:1312 \
        --rm \
        --env=TMPDIR=/tmp \
        --env=HOME=/private/var/spool/cwl \
        alpine \
        touch \
        two
    [job task2] completed success
    [job task2] {
        "out": {
            "location": "file:///Users/th10/checkouts/cwltool/test_cache/5115cb5a928586f413deadf8090375a9/two",
            "basename": "two",
            "nameroot": "two",
            "nameext": "",
            "class": "File",
            "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
            "size": 0,
            "http://commonwl.org/cwltool#generation": 0
        }
    }
    [step task2] produced output {}
    [step task2] completed success
    [workflow test_workflow.cwl] completed success
    [job task2] Removing input staging directory /private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmpkma3bhie
    [job task2] Removing temporary directory /private/var/folders/5p/p6z8hhy174q_ctt3pn7mx5_c000h1t/T/tmph12do9a5
    Removing intermediate output directory /Users/th10/checkouts/cwltool/test_cache/025t0_5x
    {}
    Final process status is success
</p>
</details>

---

From the above output, you can see that task2 doesn't have a cache hit, as task1 didn't run, due to it being cached.

A simple way to fix this is to add a `original_requirements` and `original_hints` property, which doesn't get mutated by cwltool - this can be used for cache decisions. I can have a look at submitting a PR for this.

## Your Environment
* cwltool version: 1.0.20180324152842


Discussion:

Resolving patch:
diff --git a/cwltool/command_line_tool.py b/cwltool/command_line_tool.py
--- a/cwltool/command_line_tool.py
+++ b/cwltool/command_line_tool.py
@@ -296,7 +296,7 @@ def job(self,
                            "EnvVarRequirement",
                            "CreateFileRequirement",
                            "ShellCommandRequirement"}
-            for rh in (self.requirements, self.hints):
+            for rh in (self.original_requirements, self.original_hints):
                 for r in reversed(rh):
                     if r["class"] in interesting and r["class"] not in keydict:
                         keydict[r["class"]] = r
diff --git a/cwltool/process.py b/cwltool/process.py
--- a/cwltool/process.py
+++ b/cwltool/process.py
@@ -17,6 +17,7 @@
 from functools import cmp_to_key
 from typing import (Any, Callable, Dict, Generator, List, Set, Text,
                     Tuple, Union, cast, Optional)
+import copy
 
 import schema_salad.schema as schema
 import schema_salad.validate as validate
@@ -453,6 +454,9 @@ def __init__(self, toolpath_object, **kwargs):
                              self.tool.get("requirements", []) +
                              get_overrides(kwargs.get("overrides", []), self.tool["id"]).get("requirements", []))
         self.hints = kwargs.get("hints", []) + self.tool.get("hints", [])
+        # Versions of requirements and hints which aren't mutated.
+        self.original_requirements = copy.deepcopy(self.requirements)
+        self.original_hints = copy.deepcopy(self.hints)
         self.formatgraph = None  # type: Graph
         if "loader" in kwargs:
             self.formatgraph = kwargs["loader"].graph


Test patch:
diff --git a/tests/test_examples.py b/tests/test_examples.py
--- a/tests/test_examples.py
+++ b/tests/test_examples.py
@@ -6,6 +6,8 @@
 import sys
 import json
 import logging
+import tempfile
+import shutil
 
 from io import StringIO, BytesIO
 
@@ -651,13 +653,30 @@ def test_no_js_console(self):
 
 @needs_docker
 class TestCache(TestCmdLine):
+    def setUp(self):
+        self.cache_dir = tempfile.mkdtemp("cwltool_cache")
+
+    def tearDown(self):
+        shutil.rmtree(self.cache_dir)
+
     def test_wf_without_container(self):
         test_file = "hello-workflow.cwl"
-        error_code, stdout, stderr = self.get_main_output(["--cachedir", "cache",
+        error_code, stdout, stderr = self.get_main_output(["--cachedir", self.cache_dir,
                                                    get_data("tests/wf/" + test_file), "--usermessage", "hello"])
         self.assertIn("completed success", stderr)
         self.assertEquals(error_code, 0)
 
+    def test_issue_740_fixed(self):
+        test_file = "cache_test_workflow.cwl"
+        error_code, stdout, stderr = self.get_main_output(["--cachedir", self.cache_dir, get_data("tests/wf/" + test_file)])
+        self.assertIn("completed success", stderr)
+        self.assertEquals(error_code, 0)
+
+        error_code, stdout, stderr = self.get_main_output(["--cachedir", self.cache_dir, get_data("tests/wf/" + test_file)])
+        self.assertNotIn("Output of job will be cached in", stderr)
+        self.assertEquals(error_code, 0)
+
+
 @needs_docker
 class TestChecksum(TestCmdLine):
 
diff --git a/tests/wf/cache_test_workflow.cwl b/tests/wf/cache_test_workflow.cwl
new file mode 100644
--- /dev/null
+++ b/tests/wf/cache_test_workflow.cwl
@@ -0,0 +1,21 @@
+#!/usr/bin/env cwl-runner
+
+cwlVersion: v1.0
+class: Workflow
+
+inputs: []
+outputs: []
+
+steps:
+  task1:
+    run: touch_tool.cwl
+    in:
+      message:
+         default: one
+    out: []
+  task2:
+    run: touch_tool.cwl
+    in:
+      message:
+         default: two
+    out: []
diff --git a/tests/wf/touch_tool.cwl b/tests/wf/touch_tool.cwl
new file mode 100644
--- /dev/null
+++ b/tests/wf/touch_tool.cwl
@@ -0,0 +1,18 @@
+#!/usr/bin/env cwl-runner
+
+cwlVersion: v1.0
+class: CommandLineTool
+baseCommand: touch
+requirements:
+  DockerRequirement:
+    dockerPull: alpine
+inputs:
+  message:
+    type: string
+    inputBinding:
+      position: 1
+outputs:
+  - id: out
+    type: File
+    outputBinding:
+      glob: "*"