Repository: common-workflow-language/cwltool

Issue:
Input directory corruption and infinite symlink using successive InitialWorkDirRequirement
There appear to be 2 bugs in `cwltool` handling of `InitialWorkDirRequirement`, in the use of symlinks to optimize and avoid making copies of directories:
- Directories get corrupted with incorrect symlinks that point to themselves.
- Symlinks that point to themselves cause an inifinite loop in `cwltool`

The issue is subtle, and seems to manifest when different parts of a single directory structure are referenced.

## Expected Behavior
Input directories should be left pristine, and output directories should be emitted correctly, when using `InitialWorkDirRequirement`. In the example workflow, a subdirectory with one test file should be the final output.

## Actual Behavior
A symlink is installed at `dir/subdir/file` with target being itself (1st bug), causing an infinite loop in `cwltool` as it tries to dereference the symlink within `PathMapper.visit()` (2nd bug).

## Workflow Code
Here is a code example that demonstrates the issue: `test.cwl`
```
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: Workflow

inputs: []

steps:
  # Create a test directory structure; could be done outside CWL and passed in as input.
  # This input directory should be left pristine.
  mkdirs:
    run:
      class: CommandLineTool
      baseCommand: [bash, '-c', 'mkdir dir dir/subdir && touch dir/subdir/file', '-']
      inputs: []
      outputs:
        mkdirs_out:
          type: Directory
          outputBinding:
            glob: dir
    in: []
    out: [mkdirs_out]

  # Given an input directory, emit a subdirectory as output.
  passthrough1:
    run:
      class: CommandLineTool
      requirements:
      - class: InitialWorkDirRequirement
        listing:
        - entry: $(inputs.passthrough1_in)
          writable: false
      baseCommand: ["true"]
      inputs:
        passthrough1_in:
          type: Directory
      outputs:
        passthrough1_subdir:
          type: Directory
          outputBinding:
            glob: $(inputs.passthrough1_in.basename)/subdir
    in:
      passthrough1_in: mkdirs/mkdirs_out
    out: [passthrough1_subdir]

  # Given a (sub-)directory, emit it unchanged.
  passthrough2:
    run:
      class: CommandLineTool
      requirements:
      - class: InitialWorkDirRequirement
        listing:
        - entry: $(inputs.passthrough2_in)
          writable: false
      baseCommand: ["true"]
      inputs:
        passthrough2_in:
          type: Directory
      outputs:
        passthrough2_subdir:
          type: Directory
          outputBinding:
            glob: $(inputs.passthrough2_in.basename)
    in:
      passthrough2_in: passthrough1/passthrough1_subdir
    out: [passthrough2_subdir]

outputs:
  out:
    type: Directory
    outputSource: passthrough2/passthrough2_subdir
```

As input, use this empty `test.yaml`:
```
{}
```
## Full Traceback
Infinite loop in `pathmapper.py` around line 249, in `PathMapper.visit()`; see the comment line `# Dereference symbolic links`.

## Your Environment
* cwltool version: 1.0.20181217162649



Discussion:
Good news, I think. As of version 3.1.20210905131105 cwltool no longer goes into an infinite loop with this test

```
$ cwltool 1029.cwl
INFO /home/michael/cwltool/env/bin/cwltool 3.1.20210905131105
INFO Resolved '1029.cwl' to 'file:///home/michael/cwltool/1029.cwl'
INFO [workflow ] start
INFO [workflow ] starting step mkdirs
INFO [step mkdirs] start
INFO [job mkdirs] /tmp/jka_zrkd$ bash \
    -c \
    'mkdir dir dir/subdir && touch dir/subdir/file' \
    -
INFO [job mkdirs] completed success
INFO [step mkdirs] completed success
INFO [workflow ] starting step passthrough1
INFO [step passthrough1] start
INFO [job passthrough1] /tmp/7o3r93ve$ true
INFO [job passthrough1] completed success
INFO [step passthrough1] completed success
INFO [workflow ] starting step passthrough2
INFO [step passthrough2] start
ERROR Got workflow error
Traceback (most recent call last):
  File "/home/michael/cwltool/cwltool/executors.py", line 250, in run_jobs
    job.run(runtime_context)
  File "/home/michael/cwltool/cwltool/job.py", line 559, in run
    stage_files(
  File "/home/michael/cwltool/cwltool/process.py", line 295, in stage_files
    os.symlink(entry.resolved, entry.target)
FileExistsError: [Errno 17] File exists: '/tmp/jka_zrkd/dir/subdir/file' -> '/tmp/rvsil6z1/subdir/file'
ERROR Workflow error, try again with --debug for more information:
[Errno 17] File exists: '/tmp/jka_zrkd/dir/subdir/file' -> '/tmp/rvsil6z1/subdir/file'
```
Can we reopen this issue? I see the first bug was never addressed. Note that
`java -jar cromwell.jar test.cwl -i test.yaml -t cwl`
prints a warning, but otherwise runs the workflow just fine.

Please note that this minimal example is not contrived. There are CommandLineTools which output a directory tree, and either use files in that directory tree for checkpointing or perform a series of operations which assume the files remain present. cwltool should support this use case.

I'm currently attempting to fix this myself, without much success.

Resolving patch:
diff --git a/cwltool/job.py b/cwltool/job.py
--- a/cwltool/job.py
+++ b/cwltool/job.py
@@ -79,7 +79,7 @@ def relink_initialworkdir(
     container_outdir: str,
     inplace_update: bool = False,
 ) -> None:
-    for _, vol in pathmapper.items():
+    for _, vol in pathmapper.items_exclude_children():
         if not vol.staged:
             continue
 
diff --git a/cwltool/pathmapper.py b/cwltool/pathmapper.py
--- a/cwltool/pathmapper.py
+++ b/cwltool/pathmapper.py
@@ -4,6 +4,7 @@
 import stat
 import urllib
 import uuid
+from pathlib import Path
 from typing import Dict, Iterator, List, Optional, Tuple, cast
 
 from schema_salad.exceptions import ValidationException
@@ -191,6 +192,16 @@ def files(self) -> List[str]:
     def items(self) -> List[Tuple[str, MapperEnt]]:
         return list(self._pathmap.items())
 
+    def items_exclude_children(self) -> List[Tuple[str, MapperEnt]]:
+        newitems = {}
+        keys = [key for key, entry in self.items()]
+        for key, entry in self.items():
+            parents = Path(key).parents
+            if any([Path(key_) in parents for key_ in keys]):
+                continue
+            newitems[key] = entry
+        return list(newitems.items())
+
     def reversemap(
         self,
         target: str,
diff --git a/cwltool/process.py b/cwltool/process.py
--- a/cwltool/process.py
+++ b/cwltool/process.py
@@ -263,8 +263,9 @@ def stage_files(
     fix_conflicts: bool = False,
 ) -> None:
     """Link or copy files to their targets. Create them as needed."""
+    items = pathmapper.items() if not symlink else pathmapper.items_exclude_children()
     targets = {}  # type: Dict[str, MapperEnt]
-    for key, entry in pathmapper.items():
+    for key, entry in items:
         if "File" not in entry.type:
             continue
         if entry.target not in targets:
@@ -287,7 +288,7 @@ def stage_files(
                     % (targets[entry.target].resolved, entry.resolved, entry.target)
                 )
 
-    for key, entry in pathmapper.items():
+    for key, entry in items:
         if not entry.staged:
             continue
         if not os.path.exists(os.path.dirname(entry.target)):


Test patch:
diff --git a/tests/test_iwdr.py b/tests/test_iwdr.py
--- a/tests/test_iwdr.py
+++ b/tests/test_iwdr.py
@@ -25,6 +25,25 @@ def test_empty_file_creation() -> None:
     assert err_code == 0
 
 
+def test_passthrough_successive(tmp_path: Path) -> None:
+    """An empty file can be successively passed through a subdir of InitialWorkingDirectory."""
+    err_code, _, _ = get_main_output(
+        [
+            "--outdir",
+            str(tmp_path),
+            get_data("tests/wf/iwdr-passthrough-successive.cwl"),
+        ]
+    )
+    assert err_code == 0
+    children = sorted(
+        tmp_path.glob("*")
+    )  # This input directory should be left pristine.
+    assert len(children) == 1
+    subdir = tmp_path / children[0]
+    assert len(sorted(subdir.glob("*"))) == 1
+    assert (subdir / "file").exists()
+
+
 @needs_docker
 def test_directory_literal_with_real_inputs_inside(tmp_path: Path) -> None:
     """Cope with unmoveable files in the output directory created by Docker+IWDR."""
diff --git a/tests/wf/iwdr-passthrough-successive.cwl b/tests/wf/iwdr-passthrough-successive.cwl
new file mode 100644
--- /dev/null
+++ b/tests/wf/iwdr-passthrough-successive.cwl
@@ -0,0 +1,71 @@
+#!/usr/bin/env cwl-runner
+
+cwlVersion: v1.0
+class: Workflow
+
+inputs: []
+
+steps:
+  # Create a test directory structure; could be done outside CWL and passed in as input.
+  # This input directory should be left pristine.
+  mkdirs:
+    run:
+      class: CommandLineTool
+      baseCommand: [bash, '-c', 'mkdir dir dir/subdir && touch dir/subdir/file', '-']
+      inputs: []
+      outputs:
+        mkdirs_out:
+          type: Directory
+          outputBinding:
+            glob: dir
+    in: []
+    out: [mkdirs_out]
+
+  # Given an input directory, emit a subdirectory as output.
+  passthrough1:
+    run:
+      class: CommandLineTool
+      requirements:
+      - class: InitialWorkDirRequirement
+        listing:
+        - entry: $(inputs.passthrough1_in)
+          writable: false
+      baseCommand: ["true"]
+      inputs:
+        passthrough1_in:
+          type: Directory
+      outputs:
+        passthrough1_subdir:
+          type: Directory
+          outputBinding:
+            glob: $(inputs.passthrough1_in.basename)/subdir
+    in:
+      passthrough1_in: mkdirs/mkdirs_out
+    out: [passthrough1_subdir]
+
+  # Given a (sub-)directory, emit it unchanged.
+  passthrough2:
+    run:
+      class: CommandLineTool
+      requirements:
+      - class: InitialWorkDirRequirement
+        listing:
+        - entry: $(inputs.passthrough2_in)
+          writable: false
+      baseCommand: ["true"]
+      inputs:
+        passthrough2_in:
+          type: Directory
+      outputs:
+        passthrough2_subdir:
+          type: Directory
+          outputBinding:
+            glob: $(inputs.passthrough2_in.basename)
+    in:
+      passthrough2_in: passthrough1/passthrough1_subdir
+    out: [passthrough2_subdir]
+
+outputs:
+  out:
+    type: Directory
+    outputSource: passthrough2/passthrough2_subdir
\ No newline at end of file