Repository: common-workflow-language/cwltool

Issue:
IWDR+DockerReq: real input files inside Directory literals → PermissionError
> iwdr_dir_literal_real_file.cwl
``` cwl
class: CommandLineTool
cwlVersion: v1.0

requirements:
  DockerRequirement:
      dockerPull: debian
  InlineJavascriptRequirement: {}
  InitialWorkDirRequirement:
    listing: |
      ${
         return [{"class": "Directory",
                  "basename": "subdir",
                  "listing": [ inputs.example ]
                  }]}

inputs:
  example: File

outputs:
  same:
   type: File
   outputBinding:
     glob: subdir/$(inputs.example.basename)
```

result

```
$ cwltool iwdr_dir_literal_real_file.cwl --example README.rst 
/home/michael/cwltool/env3/bin/cwltool 1.0.20190228200117
Resolved 'iwdr_dir_literal_real_file.cwl' to 'file:///home/michael/cwltool/iwdr_dir_literal_real_file.cwl'
[job iwdr_dir_literal_real_file.cwl] /tmp/umhibvch$ docker \
    run \
    -i \
    --volume=/tmp/umhibvch:/fEuOgP:rw \
    --volume=/tmp/n2wem27d:/tmp:rw \
    --volume=/home/michael/cwltool/README.rst:/fEuOgP/subdir/README.rst:ro \
    --workdir=/fEuOgP \
    --read-only=true \
    --user=1000:1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/fEuOgP \
    --cidfile=/tmp/d11wgn0j/20190308144957-649141.cid \
    debian
[job iwdr_dir_literal_real_file.cwl] Max memory used: 0MiB
Exception while running job
Traceback (most recent call last):
  File "/home/michael/cwltool/cwltool/job.py", line 329, in _execute
    inplace_update=self.inplace_update)
  File "/home/michael/cwltool/cwltool/job.py", line 148, in relink_initialworkdir
    os.remove(host_outdir_tgt)
PermissionError: [Errno 13] Permission denied: '/tmp/umhibvch/subdir/README.rst'
[job iwdr_dir_literal_real_file.cwl] completed permanentFail
{}
Final process status is permanentFail
```
Instead of 
```
$ cwltool iwdr_dir_literal_real_file.cwl --example README.rst 
/home/michael/cwltool/env3/bin/cwltool 1.0.20190228200117
Resolved 'iwdr_dir_literal_real_file.cwl' to 'file:///home/michael/cwltool/iwdr_dir_literal_real_file.cwl'
[job iwdr_dir_literal_real_file.cwl] /tmp/mprnbbcc$ docker \
    run \
    -i \
    --volume=/tmp/mprnbbcc:/XteCes:rw \
    --volume=/tmp/jhmr849h:/tmp:rw \
    --workdir=/XteCes \
    --read-only=true \
    --user=1000:1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/XteCes \
    --cidfile=/tmp/zlkkrtdr/20190308145035-998641.cid \
    debian
[job iwdr_dir_literal_real_file.cwl] Max memory used: 0MiB
[job iwdr_dir_literal_real_file.cwl] completed success
{
    "same": {
        "location": "file:///home/michael/cwltool/README.rst",
        "basename": "README.rst",
        "class": "File",
        "checksum": "sha1$a430706cc775c602686593cdf221b4d7ee9b09ea",
        "size": 27810,
        "path": "/home/michael/cwltool/README.rst"
    }
}
Final process status is success
```

Workaround, insert `, "writable": true` after the `"listing": [ inputs.example ]`


Discussion:

Resolving patch:
diff --git a/cwltool/job.py b/cwltool/job.py
--- a/cwltool/job.py
+++ b/cwltool/job.py
@@ -163,7 +163,10 @@ def relink_initialworkdir(
                 host_outdir, vol.target[len(container_outdir) + 1 :]
             )
             if os.path.islink(host_outdir_tgt) or os.path.isfile(host_outdir_tgt):
-                os.remove(host_outdir_tgt)
+                try:
+                    os.remove(host_outdir_tgt)
+                except PermissionError:
+                    pass
             elif os.path.isdir(host_outdir_tgt) and not vol.resolved.startswith("_:"):
                 shutil.rmtree(host_outdir_tgt)
             if onWindows():
@@ -175,7 +178,10 @@ def relink_initialworkdir(
                 elif vol.type in ("Directory", "WritableDirectory"):
                     copytree_with_merge(vol.resolved, host_outdir_tgt)
             elif not vol.resolved.startswith("_:"):
-                os.symlink(vol.resolved, host_outdir_tgt)
+                try:
+                    os.symlink(vol.resolved, host_outdir_tgt)
+                except FileExistsError:
+                    pass
 
 
 class JobBase(HasReqsHints, metaclass=ABCMeta):
diff --git a/cwltool/update.py b/cwltool/update.py
--- a/cwltool/update.py
+++ b/cwltool/update.py
@@ -149,7 +149,9 @@ def v1_2_0dev1todev2(doc, loader, baseuri):  # pylint: disable=unused-argument
     return (doc, "v1.2.0-dev2")
 
 
-def v1_2_0dev2todev3(doc: Any, loader: Loader, baseuri: str) -> Tuple[Any, str]:  # pylint: disable=unused-argument
+def v1_2_0dev2todev3(
+    doc: Any, loader: Loader, baseuri: str
+) -> Tuple[Any, str]:  # pylint: disable=unused-argument
     """Public updater for v1.2.0-dev2 to v1.2.0-dev3"""
     doc = copy.deepcopy(doc)
 


Test patch:
diff --git a/tests/iwdr_dir_literal_real_file.cwl b/tests/iwdr_dir_literal_real_file.cwl
new file mode 100644
--- /dev/null
+++ b/tests/iwdr_dir_literal_real_file.cwl
@@ -0,0 +1,23 @@
+class: CommandLineTool
+cwlVersion: v1.0
+
+requirements:
+  DockerRequirement:
+      dockerPull: debian:stable-slim
+  InlineJavascriptRequirement: {}
+  InitialWorkDirRequirement:
+    listing: |
+      ${
+         return [{"class": "Directory",
+                  "basename": "subdir",
+                  "listing": [ inputs.example ]
+                  }]}
+
+inputs:
+  example: File
+
+outputs:
+  same:
+   type: File
+   outputBinding:
+     glob: subdir/$(inputs.example.basename)
diff --git a/tests/test_iwdr.py b/tests/test_iwdr.py
--- a/tests/test_iwdr.py
+++ b/tests/test_iwdr.py
@@ -30,6 +30,18 @@ def test_empty_file_creation():
     assert err_code == 0
 
 
+@needs_docker
+def test_directory_literal_with_real_inputs_inside():
+    """Cope with unmoveable files in the output directory created by Docker+IWDR."""
+    err_code, _, _ = get_main_output(
+        [
+            get_data("tests/iwdr_dir_literal_real_file.cwl"),
+            "--example={}".format(get_data("tests/__init__.py")),
+        ]
+    )
+    assert err_code == 0
+
+
 @needs_docker
 def test_iwdr_permutations():
     saved_tempdir = tempfile.tempdir