Repository: DataBiosphere/toil

Issue:
Toil lets jobs schedule with actual 0 disk requirements, leading to complaints that they used 4kb
anthfm on Gitter was [complaining](https://gitter.im/bd2k-genomics-toil/Lobby?at=613a8fdc5b92082de192adae) about messages like this, when forcing CWL internal jobs to schedule on their cluster:

```
Job 'CWLWorkflow' kind-CWLWorkflow/instance-li66gwjv used 0.00% disk (4.0 KiB [4096B] used, 0 bB [0B] requested)
```

We seem to create at least one empty directory or lock file or something (a 4kb single-sector inode thing) for every job, but when a job asks for 0 bytes of disk, we somehow believe that it will use 0 bytes of disk, and complain when we see our own temp directory.

We should add a little bit of overhead or a Toil-imposed minimum to the job disk requirements (I think this case was on Slurm, but we could do it universally), or at least suppress this message in cases like this where the job's run method doesn't actually use disk.

┆Issue is synchronized with this [Jira Task](https://ucsc-cgl.atlassian.net/browse/TOIL-1014)
┆Issue Number: TOIL-1014



Discussion:

Resolving patch:
diff --git a/src/toil/cwl/cwltoil.py b/src/toil/cwl/cwltoil.py
--- a/src/toil/cwl/cwltoil.py
+++ b/src/toil/cwl/cwltoil.py
@@ -1618,7 +1618,7 @@ class ResolveIndirect(Job):
 
     def __init__(self, cwljob: dict):
         """Store the dictionary of promises for later resolution."""
-        super(ResolveIndirect, self).__init__(cores=1, memory=1024 ^ 2, disk=0)
+        super(ResolveIndirect, self).__init__(cores=1, memory="1GiB", disk="1MiB")
         self.cwljob = cwljob
 
     def run(self, file_store: AbstractFileStore) -> dict:
@@ -1770,7 +1770,7 @@ def __init__(
         conditional: Union[Conditional, None] = None,
     ):
         """Store our context for later evaluation."""
-        super(CWLJobWrapper, self).__init__(cores=1, memory=1024 * 1024, disk=8 * 1024)
+        super(CWLJobWrapper, self).__init__(cores=1, memory="1GiB", disk="1MiB")
         self.cwltool = remove_pickle_problems(tool)
         self.cwljob = cwljob
         self.runtime_context = runtime_context
@@ -2125,7 +2125,7 @@ def __init__(
         conditional: Union[Conditional, None],
     ):
         """Store our context for later execution."""
-        super(CWLScatter, self).__init__(cores=1, memory=100 * 1024 ^ 2, disk=0)
+        super(CWLScatter, self).__init__(cores=1, memory="1GiB", disk="1MiB")
         self.step = step
         self.cwljob = cwljob
         self.runtime_context = runtime_context
@@ -2268,7 +2268,7 @@ def __init__(
         outputs: Union[Mapping, MutableSequence],
     ):
         """Collect our context for later gathering."""
-        super(CWLGather, self).__init__(cores=1, memory=10 * 1024 ^ 2, disk=0)
+        super(CWLGather, self).__init__(cores=1, memory="1GiB", disk="1MiB")
         self.step = step
         self.outputs = outputs
 
@@ -2310,7 +2310,7 @@ class SelfJob(Job):
 
     def __init__(self, j: "CWLWorkflow", v: dict):
         """Record the workflow and dictionary."""
-        super(SelfJob, self).__init__(cores=1, memory=1024 ^ 2, disk=0)
+        super(SelfJob, self).__init__(cores=1, memory="1GiB", disk="1MiB")
         self.j = j
         self.v = v
 
@@ -2364,7 +2364,7 @@ def __init__(
         conditional: Union[Conditional, None] = None,
     ):
         """Gather our context for later execution."""
-        super(CWLWorkflow, self).__init__(cores=1, memory=100 * 1024 ^ 2, disk=0)
+        super(CWLWorkflow, self).__init__(cores=1, memory="1GiB", disk="1MiB")
         self.cwlwf = cwlwf
         self.cwljob = cwljob
         self.runtime_context = runtime_context


Test patch: