Repository: DataBiosphere/toil

Issue:
Single CWL tools run without a workflow cannot read from S3 when using `loadContents`
As described by @boyangzhao [here](https://github.com/DataBiosphere/toil/issues/4094#issuecomment-1146783257) and in more detail [here](https://github.com/DataBiosphere/toil/issues/4091#issuecomment-1144714219), running a single CWL tool with an S3 `File` input does not work.

If I do:
```
cat >helloworld.cwl <<'EOF'
class: CommandLineTool
cwlVersion: v1.0
id: helloworld
baseCommand:
  - echo
inputs:
  - id: txt_file
    type: File?
    inputBinding:
      position: 0
      loadContents: true
      valueFrom: $(self.contents)
outputs:
  - id: output
    type: File?
    outputBinding:
      glob: helloworld.txt
label: helloworld
requirements:
  - class: ShellCommandRequirement
  - class: DockerRequirement
    dockerPull: 'alpine:3.7'
  - class: InlineJavascriptRequirement
stdout: helloworld.txt
EOF

cat >helloworld.job.yaml <<'EOF'
txt_file:
  class: File
  location: s3://toil-datasets/hello.txt
EOF

toil-cwl-runner --jobStore ./tree --clean=always helloworld.cwl helloworld.job.yaml
```

Then I get:
```
[2022-06-06T07:15:15-0700] [MainThread] [I] [cwltool] Resolved 'helloworld.cwl' to 'file:///public/home/anovak/build/toil/helloworld.cwl'
[2022-06-06T07:15:22-0700] [MainThread] [I] [toil.common] Successfully deleted the job store: FileJobStore(/public/home/anovak/build/toil/tree)
Traceback (most recent call last):
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/builder.py", line 434, in bind_input
    with self.fs_access.open(
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/stdfsaccess.py", line 40, in open
    return open(self._abs(fn), mode)
FileNotFoundError: [Errno 2] No such file or directory: '/public/home/anovak/build/toil/s3://toil-datasets/hello.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/builder.py", line 439, in bind_input
    raise Exception(
Exception: Reading s3://toil-datasets/hello.txt
[Errno 2] No such file or directory: '/public/home/anovak/build/toil/s3://toil-datasets/hello.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/public/home/anovak/build/toil/venv/bin/toil-cwl-runner", line 33, in <module>
    sys.exit(load_entry_point('toil', 'console_scripts', 'toil-cwl-runner')())
  File "/public/home/anovak/build/toil/src/toil/cwl/cwltoil.py", line 3575, in main
    builder = tool._init_job(initialized_job_order, runtime_context)
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/process.py", line 917, in _init_job
    builder.bind_input(
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/builder.py", line 375, in bind_input
    self.bind_input(
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/builder.py", line 306, in bind_input
    self.bind_input(
  File "/public/home/anovak/build/toil/venv/lib/python3.9/site-packages/cwltool/builder.py", line 439, in bind_input
    raise Exception(
  File "schema_salad/sourceline.py", line 256, in __exit__
cwltool.errors.WorkflowException:  Reading s3://toil-datasets/hello.txt
 [Errno 2] No such file or directory: '/public/home/anovak/build/toil/s3://toil-datasets/hello.txt'
```

But if I add workflow wrapper:
```
cat >helloworld.wf.cwl <<'EOF'
cwlVersion: v1.2
class: Workflow
requirements:
  SubworkflowFeatureRequirement: {}
inputs:
  - id: txt_file
    type: File
steps:
  helloworld:
    run: helloworld.cwl
    in:
      txt_file: txt_file
    out: [output]
outputs:
  - id: output
    type: File?
    outputSource: helloworld/output
EOF

toil-cwl-runner --jobStore ./tree --clean=always helloworld.wf.cwl helloworld.job.yaml
```

Then it works:
```
[2022-06-06T07:10:49-0700] [MainThread] [I] [cwltool] Resolved 'helloworld.wf.cwl' to 'file:///public/home/anovak/build/toil/helloworld.wf.cwl'
[2022-06-06T07:11:13-0700] [MainThread] [I] [toil.job] Saving graph of 2 jobs, 2 new
[2022-06-06T07:11:13-0700] [MainThread] [I] [toil.job] Processing job 'ResolveIndirect' kind-ResolveIndirect/instance-euw42xmb v0
[2022-06-06T07:11:13-0700] [MainThread] [I] [toil.job] Processing job 'CWLWorkflow' kind-CWLWorkflow/instance-mlw0xnyq v0
[2022-06-06T07:11:13-0700] [MainThread] [I] [toil] Running Toil version 5.6.0a1-312b6e1f221ee7f7f187dd6dbfce1aecffd00e09 on host courtyard.
[2022-06-06T07:11:15-0700] [MainThread] [I] [toil.leader] 1 jobs are running, 0 jobs are issued and waiting to run
[2022-06-06T07:11:19-0700] [MainThread] [I] [toil.worker] Redirecting logging to /data/tmp/375756c44eb55db3805af3ac81115d77/9456/worker_log.txt
[2022-06-06T07:11:21-0700] [MainThread] [I] [toil.leader] Issued job 'CWLJob' helloworld.wf.cwl.helloworld.helloworld kind-CWLJob/instance-t45hig6d v1 with job batch system ID: 1 and cores: 1, disk: 3.0 
Gi, and memory: 2.0 Gi
[2022-06-06T07:11:26-0700] [MainThread] [I] [toil.worker] Redirecting logging to /data/tmp/375756c44eb55db3805af3ac81115d77/4617/worker_log.txt
[2022-06-06T07:11:53-0700] [MainThread] [I] [toil.worker] Redirecting logging to /data/tmp/375756c44eb55db3805af3ac81115d77/8b77/worker_log.txt
[2022-06-06T07:12:01-0700] [MainThread] [I] [toil.worker] Redirecting logging to /data/tmp/375756c44eb55db3805af3ac81115d77/5c23/worker_log.txt
[2022-06-06T07:12:05-0700] [MainThread] [I] [toil.leader] Finished toil run successfully.

Workflow Progress 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 (0 failures) [00:49<00:00, 0.08 jobs/s]
{
    "output": {
        "location": "file:///public/home/anovak/build/toil/helloworld.txt",
        "basename": "helloworld.txt",
        "nameroot": "helloworld",
        "nameext": ".txt",
        "class": "File",
        "checksum": "sha1$e3efd3aaf726bbf6320bc03fd4b495963f1d9b69",
        "size": 147
    }
}[2022-06-06T07:12:05-0700] [MainThread] [I] [toil.common] Successfully deleted the job store: FileJobStore(/public/home/anovak/build/toil/tree)
```

┆Issue is synchronized with this [Jira Story](https://ucsc-cgl.atlassian.net/browse/TOIL-1178)
┆friendlyId: TOIL-1178



Discussion:
This seems to only fail if `loadContents` is `true` on the input. If I get rid of it, the run manages to start (but fails later because `contents` isn't filled in). If I change this from an `echo` to a `cat` and from passign the contents on the command line to passing the name, it works even without a wrapping workflow.

The problem is that we do the contents loading very early in workflow setup, before we've configured `cwltool` to use Toil's filesystem machinery, so it doesn't know how to read anything but local paths yet.

Resolving patch:
diff --git a/src/toil/common.py b/src/toil/common.py
--- a/src/toil/common.py
+++ b/src/toil/common.py
@@ -43,7 +43,8 @@
     Tuple,
     Type,
     TypeVar,
-    Union
+    Union,
+    overload
 )
 from urllib.parse import urlparse
 
@@ -1140,17 +1141,44 @@ def _setupAutoDeployment(
             logger.debug('Injecting user script %s into batch system.', userScriptResource)
             self._batchSystem.setUserScript(userScriptResource)
 
+    # Importing a file with a shared file name returns None, but without one it
+    # returns a file ID. Explain this to MyPy.
+
+    @overload
+    def importFile(self,
+                   srcUrl: str,
+                   sharedFileName: str,
+                   symlink: bool = False) -> None: ...
+
+    @overload
+    def importFile(self,
+                   srcUrl: str,
+                   sharedFileName: None = None,
+                   symlink: bool = False) -> FileID: ...
+
     @deprecated(new_function_name='import_file')
     def importFile(self,
                    srcUrl: str,
                    sharedFileName: Optional[str] = None,
-                   symlink: bool = False) -> Optional[Union[FileID, str]]:
+                   symlink: bool = False) -> Optional[FileID]:
         return self.import_file(srcUrl, sharedFileName, symlink)
 
+    @overload
+    def import_file(self,
+                    src_uri: str,
+                    shared_file_name: str,
+                    symlink: bool = False) -> None: ...
+
+    @overload
+    def import_file(self,
+                    src_uri: str,
+                    shared_file_name: None = None,
+                    symlink: bool = False) -> FileID: ...
+
     def import_file(self,
                     src_uri: str,
                     shared_file_name: Optional[str] = None,
-                    symlink: bool = False) -> Optional[Union[FileID, str]]:
+                    symlink: bool = False) -> Optional[FileID]:
         """
         Import the file at the given URL into the job store.
 
diff --git a/src/toil/cwl/cwltoil.py b/src/toil/cwl/cwltoil.py
--- a/src/toil/cwl/cwltoil.py
+++ b/src/toil/cwl/cwltoil.py
@@ -48,6 +48,7 @@
     Optional,
     TextIO,
     Tuple,
+    Type,
     TypeVar,
     Union,
     cast,
@@ -63,7 +64,7 @@
 import cwltool.main
 import cwltool.provenance
 import cwltool.resolver
-import cwltool.stdfsaccess
+from cwltool.stdfsaccess import StdFsAccess, abspath
 import schema_salad.ref_resolver
 from cwltool.loghandler import _logger as cwllogger
 from cwltool.loghandler import defaultStreamHandler
@@ -615,7 +616,7 @@ def __init__(
                "streaming_allowed" flag for whether we are running with
                streaming on, and returns a file: URI to where the file or
                directory has been downloaded to. Meant to be a partially-bound
-               version of toil_get_file().   
+               version of toil_get_file().
         """
         self.get_file = get_file
         self.stage_listing = stage_listing
@@ -818,7 +819,7 @@ def visit(
                 )
                 return
 
-            ab = cwltool.stdfsaccess.abspath(path, basedir)
+            ab = abspath(path, basedir)
             if "contents" in obj and path.startswith("_:"):
                 # We are supposed to create this file
                 self._pathmap[path] = MapperEnt(
@@ -1063,7 +1064,7 @@ def encode_directory(
         json.dumps(contents).encode("utf-8")
     ).decode("utf-8")
 
-class ToilFsAccess(cwltool.stdfsaccess.StdFsAccess):
+class ToilFsAccess(StdFsAccess):
     """
     Custom filesystem access class which handles toil filestore references.
 
@@ -1094,15 +1095,15 @@ def _abs(self, path: str) -> str:
         """
         Return a local absolute path for a file (no schema).
 
-        Overwrites cwltool.stdfsaccess.StdFsAccess._abs() to account for toil specific schema.
+        Overwrites StdFsAccess._abs() to account for toil specific schema.
         """
         # TODO: Both we and the ToilPathMapper relate Toil paths to local
         # paths. But we don't share the same mapping, so accesses through
         # different mechanisms will produce different local copies.
 
         # Used to fetch a path to determine if a file exists in the inherited
-        # cwltool.stdfsaccess.StdFsAccess, (among other things) so this should
-        # not error on missing files.
+        # StdFsAccess, (among other things) so this should not error on missing
+        # files.
         # See: https://github.com/common-workflow-language/cwltool/blob/beab66d649dd3ee82a013322a5e830875e8556ba/cwltool/stdfsaccess.py#L43  # noqa B950
 
         parse = urlparse(path)
@@ -1281,7 +1282,7 @@ def listdir(self, fn: str) -> List[str]:
 
             # Now list it (it is probably a directory)
             return [
-                cwltool.stdfsaccess.abspath(quote(entry), fn)
+                abspath(quote(entry), fn)
                 for entry in os.listdir(directory)
             ]
         else:
@@ -1419,7 +1420,7 @@ def write_to_pipe(
         return schema_salad.ref_resolver.file_uri(dest_path)
     elif file_store_id.startswith("file:") or urlsplit(file_store_id).scheme == "":
         # There's a file: scheme or no scheme, and we know this isn't a _: URL.
-        
+
         # We need to support file: URIs and local paths, because we might be
         # involved in moving files around on the local disk when uploading
         # things after a job. We might want to catch cases where a leader
@@ -1484,7 +1485,7 @@ def path_to_loc(obj: CWLObjectType) -> None:
 
 def import_files(
     import_function: Callable[[str], FileID],
-    fs_access: cwltool.stdfsaccess.StdFsAccess,
+    fs_access: StdFsAccess,
     fileindex: Dict[str, str],
     existing: Dict[str, str],
     cwl_object: Optional[CWLObjectType],
@@ -1876,7 +1877,7 @@ def _realpath(
                             f.close()
                             # Import it and pack up the file ID so we can turn around and export it.
                             file_id_or_contents = (
-                                "toilfile:" + toil.importFile(f.name).pack()
+                                "toilfile:" + toil.import_file(f.name).pack()
                             )
 
                     if file_id_or_contents.startswith("toildir:"):
@@ -2021,7 +2022,7 @@ def __init__(
                 resources={},
                 mutation_manager=runtime_context.mutation_manager,
                 formatgraph=tool.formatgraph,
-                make_fs_access=runtime_context.make_fs_access,
+                make_fs_access=cast(Type[StdFsAccess], runtime_context.make_fs_access),
                 fs_access=runtime_context.make_fs_access(""),
                 job_script_provider=runtime_context.job_script_provider,
                 timeout=runtime_context.eval_timeout,
@@ -2190,9 +2191,9 @@ def run(self, file_store: AbstractFileStore) -> Any:
             # stuff in Toil's FileStore. We need to plug in a make_fs_access
             # function and a path_mapper type or factory function.
 
-            runtime_context.make_fs_access = functools.partial(  # type: ignore[assignment]
+            runtime_context.make_fs_access = cast(Type[StdFsAccess], functools.partial(
                 ToilFsAccess, file_store=file_store
-            )
+            ))
 
             runtime_context.path_mapper = functools.partial(  # type: ignore[assignment]
                 ToilPathMapper,
@@ -3412,6 +3413,13 @@ def main(args: Optional[List[str]] = None, stdout: TextIO = sys.stdout) -> int:
         # We use the file store to write to buckets, so we can't do this (yet?)
         logger.error("Cannot export outputs to a bucket when bypassing the file store")
         return 1
+    if not options.bypass_file_store:
+        # If we're using Toil's filesystem wrappers and the ability to access
+        # URLs implemented by Toil, we need to hook up our own StdFsAccess
+        # replacement early, before we try and set up the main CWL document.
+        # Otherwise, if it takes a File with loadContents from a URL, we won't
+        # be able to load the contents when we need to.
+        runtime_context.make_fs_access = ToilFsAccess
 
     loading_context = cwltool.main.setup_loadingContext(None, runtime_context, options)
 
@@ -3572,6 +3580,9 @@ def main(args: Optional[List[str]] = None, stdout: TextIO = sys.stdout) -> int:
             # to resolve them before toil imports them into the filestore.
             # A second builder will be built in the job's run method when toil
             # actually starts the cwl job.
+            # Note that this accesses input files for tools, so the
+            # ToilFsAccess needs to be set up if we want to be able to use
+            # URLs.
             builder = tool._init_job(initialized_job_order, runtime_context)
 
             # make sure this doesn't add listing items; if shallow_listing is
@@ -3588,7 +3599,10 @@ def main(args: Optional[List[str]] = None, stdout: TextIO = sys.stdout) -> int:
 
             # Define something we can call to import a file and get its file
             # ID.
-            file_import_function = functools.partial(toil.importFile, symlink=True)
+            # We cast this because import_file is overloaded depending on if we
+            # pass a shared file name or not, and we know the way we call it we
+            # always get a FileID out.
+            file_import_function = cast(Callable[[str], FileID], functools.partial(toil.import_file, symlink=True))
 
             # Import all the input files, some of which may be missing optional
             # files.
@@ -3702,7 +3716,7 @@ def remove_at_id(doc: Any) -> None:
                 outobj,
                 ("File",),
                 functools.partial(
-                    compute_checksums, cwltool.stdfsaccess.StdFsAccess("")
+                    compute_checksums, StdFsAccess("")
                 ),
             )
 
diff --git a/src/toil/jobStores/abstractJobStore.py b/src/toil/jobStores/abstractJobStore.py
--- a/src/toil/jobStores/abstractJobStore.py
+++ b/src/toil/jobStores/abstractJobStore.py
@@ -343,14 +343,45 @@ def _findJobStoreForUrl(cls, url: ParseResult, export: bool = False) -> 'Abstrac
         raise RuntimeError("No job store implementation supports %sporting for URL '%s'" %
                            ('ex' if export else 'im', url.geturl()))
 
+    # Importing a file with a shared file name returns None, but without one it
+    # returns a file ID. Explain this to MyPy.
+
+    @overload
+    def importFile(self,
+                   srcUrl: str,
+                   sharedFileName: str,
+                   hardlink: bool = False,
+                   symlink: bool = False) -> None: ...
+
+    @overload
+    def importFile(self,
+                   srcUrl: str,
+                   sharedFileName: None = None,
+                   hardlink: bool = False,
+                   symlink: bool = False) -> FileID: ...
+
     @deprecated(new_function_name='import_file')
     def importFile(self,
-                    srcUrl: str,
-                    sharedFileName: Optional[str] = None,
-                    hardlink: bool = False,
-                    symlink: bool = False) -> Optional[FileID]:
+                   srcUrl: str,
+                   sharedFileName: Optional[str] = None,
+                   hardlink: bool = False,
+                   symlink: bool = False) -> Optional[FileID]:
         return self.import_file(srcUrl, sharedFileName, hardlink, symlink)
 
+    @overload
+    def import_file(self,
+                    src_uri: str,
+                    shared_file_name: str,
+                    hardlink: bool = False,
+                    symlink: bool = False) -> None: ...
+
+    @overload
+    def import_file(self,
+                    src_uri: str,
+                    shared_file_name: None = None,
+                    hardlink: bool = False,
+                    symlink: bool = False) -> FileID: ...
+
     def import_file(self,
                     src_uri: str,
                     shared_file_name: Optional[str] = None,
@@ -381,7 +412,7 @@ def import_file(self,
 
         :param str shared_file_name: Optional name to assign to the imported file within the job store
 
-        :return: The jobStoreFileID of the imported file or None if sharedFileName was given
+        :return: The jobStoreFileID of the imported file or None if shared_file_name was given
         :rtype: toil.fileStores.FileID or None
         """
         # Note that the helper method _import_file is used to read from the source and write to
@@ -540,7 +571,7 @@ def _get_is_directory(cls, url: ParseResult) -> bool:
     def read_from_url(cls, src_uri: str, writable: IO[bytes]) -> Tuple[int, bool]:
         """
         Read the given URL and write its content into the given writable stream.
-        
+
         :return: The size of the file in bytes and whether the executable permission bit is set
         :rtype: Tuple[int, bool]
         """
@@ -1620,7 +1651,7 @@ class JobStoreSupport(AbstractJobStore, metaclass=ABCMeta):
     A mostly fake JobStore to access URLs not really associated with real job
     stores.
     """
-    
+
     @classmethod
     def _supports_url(cls, url: ParseResult, export: bool = False) -> bool:
         return url.scheme.lower() in ('http', 'https', 'ftp') and not export
@@ -1667,12 +1698,12 @@ def count(l: int) -> None:
             # Do the download
             shutil.copyfileobj(readable, counter)
             return size[0], False
-    
+
     @classmethod
     def _get_is_directory(cls, url: ParseResult) -> bool:
         # TODO: Implement HTTP index parsing and FTP directory listing
         return False
-        
+
     @classmethod
     def _list_url(cls, url: ParseResult) -> List[str]:
         # TODO: Implement HTTP index parsing and FTP directory listing


Test patch:
diff --git a/src/toil/test/cwl/cwlTest.py b/src/toil/test/cwl/cwlTest.py
--- a/src/toil/test/cwl/cwlTest.py
+++ b/src/toil/test/cwl/cwlTest.py
@@ -47,7 +47,7 @@
 from toil.test.provisioners.aws.awsProvisionerTest import AbstractAWSAutoscaleTest
 from toil.test.provisioners.clusterTest import AbstractClusterTest
 from toil.test import (
-    ToilTest, 
+    ToilTest,
     needs_aws_ec2,
     needs_aws_s3,
     needs_cwl,
@@ -191,7 +191,7 @@ class CWLWorkflowTest(ToilTest):
     test suite. Tests Toil-specific functions like URL types supported for
     inputs.
     """
-    
+
     def setUp(self):
         """Runs anew before each test to create farm fresh temp dirs."""
         self.outDir = f"/tmp/toil-cwl-test-{str(uuid.uuid4())}"
@@ -204,7 +204,7 @@ def tearDown(self):
             shutil.rmtree(self.outDir)
         unittest.TestCase.tearDown(self)
 
-    
+
     def _tester(self, cwlfile, jobfile, expect, main_args=[], out_name="output"):
         from toil.cwl import cwltoil
 
@@ -259,7 +259,15 @@ def download(self, inputs, tester_fn):
             input_location,
             self._expected_download_output(self.outDir),
         )
-    
+
+    def load_contents(self, inputs, tester_fn):
+        input_location = os.path.join("src/toil/test/cwl", inputs)
+        tester_fn(
+            "src/toil/test/cwl/load_contents.cwl",
+            input_location,
+            self._expected_load_contents_output(self.outDir),
+        )
+
     def download_directory(self, inputs, tester_fn):
         input_location = os.path.join("src/toil/test/cwl", inputs)
         tester_fn(
@@ -346,24 +354,39 @@ def test_download_http(self):
 
     def test_download_https(self):
         self.download("download_https.json", self._tester)
-        
+
     def test_download_file(self):
         self.download("download_file.json", self._tester)
-        
+
     @needs_aws_s3
     def test_download_directory_s3(self):
         self.download_directory("download_directory_s3.json", self._tester)
-        
+
     def test_download_directory_file(self):
         self.download_directory("download_directory_file.json", self._tester)
-        
+
     @needs_aws_s3
     def test_download_subdirectory_s3(self):
         self.download_subdirectory("download_subdirectory_s3.json", self._tester)
-        
+
     def test_download_subdirectory_file(self):
         self.download_subdirectory("download_subdirectory_file.json", self._tester)
 
+    # We also want to make sure we can run a bare tool with loadContents on the inputs, which requires accessing the input data early in the leader.
+
+    @needs_aws_s3
+    def test_load_contents_s3(self):
+        self.load_contents("download_s3.json", self._tester)
+
+    def test_load_contents_http(self):
+        self.load_contents("download_http.json", self._tester)
+
+    def test_load_contents_https(self):
+        self.load_contents("download_https.json", self._tester)
+
+    def test_load_contents_file(self):
+        self.load_contents("download_file.json", self._tester)
+
     @slow
     def test_bioconda(self):
         self._tester(
@@ -436,7 +459,7 @@ def path_with_bogus_rev():
             self.fail("Restart with missing directory did not fail")
         except NoSuchJobStoreException:
             pass
-   
+
     @needs_aws_s3
     def test_streamable(self):
         """
@@ -465,7 +488,7 @@ def test_streamable(self):
         self.assertEqual(out, self._expected_streaming_output(self.outDir))
         with open(out[out_name]["location"][len("file://") :]) as f:
             self.assertEqual(f.read().strip(), "When is s4 coming out?")
-   
+
     @staticmethod
     def _expected_seqtk_output(outDir):
         loc = "file://" + os.path.join(outDir, "out")
@@ -505,6 +528,16 @@ def _expected_download_output(outDir):
             }
         }
 
+    @classmethod
+    def _expected_load_contents_output(cls, out_dir):
+        """
+        Generate the putput we expect from load_contents.cwl, when sending
+        output files to the given directory.
+        """
+        expected = cls._expected_download_output(out_dir)
+        expected['length'] = 146
+        return expected
+
     @staticmethod
     def _expected_colon_output(outDir):
         loc = "file://" + os.path.join(outDir, "A%3AGln2Cys_result")
@@ -526,7 +559,7 @@ def _expected_colon_output(outDir):
                 ],
             }
         }
-    
+
     def _expected_streaming_output(self, outDir):
         loc = "file://" + os.path.join(outDir, "output.txt")
         return {
@@ -814,7 +847,7 @@ class CWLOnARMTest(AbstractClusterTest):
     """
     Run the CWL 1.2 conformance tests on ARM specifically.
     """
-    
+
     def __init__(self, methodName):
         super().__init__(methodName=methodName)
         self.clusterName = 'cwl-test-' + str(uuid.uuid4())
diff --git a/src/toil/test/cwl/directory/directory/file.txt b/src/toil/test/cwl/directory/directory/file.txt
--- a/src/toil/test/cwl/directory/directory/file.txt
+++ b/src/toil/test/cwl/directory/directory/file.txt
@@ -0,0 +1,15 @@
+hello
+hello 
+not yellow
+not yellow
+not yellow
+not yellow
+not yellow
+hello
+not yellow
+not yellow
+not yellow
+hello
+not yellow
+not yellow
+not yellow
diff --git a/src/toil/test/cwl/download.cwl b/src/toil/test/cwl/download.cwl
--- a/src/toil/test/cwl/download.cwl
+++ b/src/toil/test/cwl/download.cwl
@@ -1,5 +1,3 @@
-# Example command line program wrapper for the Unix tool "sort"
-# demonstrating command line flags.
 class: CommandLineTool
 doc: "Download a file from different cloud environments"
 cwlVersion: v1.0
diff --git a/src/toil/test/cwl/download_http.json b/src/toil/test/cwl/download_http.json
--- a/src/toil/test/cwl/download_http.json
+++ b/src/toil/test/cwl/download_http.json
@@ -1,6 +1,6 @@
 {
   "input": {
     "class": "File",
-    "location": "http://toil-datasets.s3.amazonaws.com/wdl_templates.zip"
+    "location": "http://toil-datasets.s3.amazonaws.com/hello.txt"
   }
 }
diff --git a/src/toil/test/cwl/download_https.json b/src/toil/test/cwl/download_https.json
--- a/src/toil/test/cwl/download_https.json
+++ b/src/toil/test/cwl/download_https.json
@@ -1,6 +1,6 @@
 {
   "input": {
     "class": "File",
-    "location": "https://toil-datasets.s3.amazonaws.com/wdl_templates.zip"
+    "location": "https://toil-datasets.s3.amazonaws.com/hello.txt"
   }
 }
diff --git a/src/toil/test/cwl/download_s3.json b/src/toil/test/cwl/download_s3.json
--- a/src/toil/test/cwl/download_s3.json
+++ b/src/toil/test/cwl/download_s3.json
@@ -1,6 +1,6 @@
 {
   "input": {
     "class": "File",
-    "location": "s3://toil-datasets/wdl_templates.zip"
+    "location": "s3://toil-datasets/hello.txt"
   }
 }
diff --git a/src/toil/test/cwl/load_contents.cwl b/src/toil/test/cwl/load_contents.cwl
new file mode 100644
--- /dev/null
+++ b/src/toil/test/cwl/load_contents.cwl
@@ -0,0 +1,26 @@
+class: CommandLineTool
+doc: "Download a file from different cloud environments"
+cwlVersion: v1.2
+requirements:
+  InlineJavascriptRequirement: {}
+
+inputs:
+  - id: input
+    type: File
+    inputBinding:
+      position: 1
+      loadContents: true
+
+outputs:
+  output:
+    type: File
+    outputBinding:
+      glob: output.txt
+  length:
+    type: int
+    outputBinding:
+        outputEval: $(inputs.input.contents.length)
+
+
+baseCommand: ["test","-f"]
+stdout: output.txt