Repository: DataBiosphere/toil

Issue:
Conditionals doesn't work for unnecessary inputs
Hello,

I'm running a simple wf of 2 steps: move 2 files if forward and reverse reads are presented OR move one file if single_reads is presented (workflows attached).

```bash
#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.2.0-dev2

requirements:
  SubworkflowFeatureRequirement: {}
  MultipleInputFeatureRequirement: {}
  InlineJavascriptRequirement: {}
  StepInputExpressionRequirement: {}
  ScatterFeatureRequirement: {}


inputs:
    forward_reads: File?
    reverse_reads: File?
    single_reads: File?

outputs: []

steps:

  step_paired:
    run: mv.cwl
    when: $(inputs.single == undefined)
    scatter: initial_file
    in:
      single: single_reads
      initial_file:
        - forward_reads
        - reverse_reads
      out_file_name: { default: 'filename_paired' }
    out: [ renamed_file ]

  step_single:
    run: mv.cwl
    when: $(inputs.single != undefined)
    in:
      single: single_reads
      initial_file: single_reads
      out_file_name: { default: 'filename_single' }
    out: [ renamed_file ]
```

I do `cwltool --no-container --enable-dev wf.cwl wf.yml` and it works fine. Example for forward+reverse input:

```bash
INFO /usr/local/bin/cwltool 3.0.20200530110633
INFO Resolved 'wf.cwl' to 'file:///Users/kates/Desktop/EBI/CWL/pipeline-v5/workflows/test/wf.cwl'
WARNING Workflow checker warning:
wf.cwl:16:5: Source 'single_reads' of type ["null", "File"] may be incompatible
wf.cwl:39:7:   with sink 'initial_file' of type "File"
wf.cwl:14:5: Source 'forward_reads' of type ["null", "File"] may be incompatible
wf.cwl:28:7:   with sink 'initial_file' of type {"type": "array", "items": "File"}
               source has linkMerge method merge_nested
wf.cwl:15:5: Source 'reverse_reads' of type ["null", "File"] may be incompatible
wf.cwl:28:7:   with sink 'initial_file' of type {"type": "array", "items": "File"}
               source has linkMerge method merge_nested
INFO [workflow ] start
INFO [workflow ] starting step step_single
INFO [step step_single] will be skipped
INFO [step step_single] completed skipped
INFO [workflow ] starting step step_paired
INFO [step step_paired] start
INFO [job step_paired] /private/tmp/docker_tmpbu8okwlo$ mv \
    /private/var/folders/ps/xwjn82ms7qz1s2p6qbywmlt4v7qk6n/T/tmp_jaz7j5w/stg4b01e05b-343a-4661-ae88-48f7069dd36a/test.file \
    filename_paired
INFO [job step_paired] completed success
INFO [step step_paired] start
INFO [job step_paired_2] /private/tmp/docker_tmpcd2e7p0m$ mv \
    /private/var/folders/ps/xwjn82ms7qz1s2p6qbywmlt4v7qk6n/T/tmpq0h7oov4/stg04845da8-4f0c-4f28-ae8b-e25d7d5428f1/test.file \
    filename_paired
INFO [job step_paired_2] completed success
INFO [step step_paired] completed success
INFO [workflow ] completed success
{}
INFO Final process status is success
```

Unfortunately, the same wf doesn't work with toil (full log attached):
`toil-cwl-runner --preserve-entire-environment --enable-dev --disableChaining --logFile log --jobStore job-store --outdir output  --no-container --batchSystem LSF --disableCaching --defaultMemory 5G --defaultCores 8 --retryCount 3 wf.cwl wf.yml`
Toil version 4.2.0a1-3f4e790c1fa32a2c2257bd5cd203a3b51d5d5661

```bash
Log from job kind-file_hps_nobackup2_production_metagenomics_pipeline_testing_kate_pipeline-v5_workflows_test_mv.cwl/instance-l3053lvf follows:
=========>
        INFO:toil.worker:---TOIL WORKER OUTPUT LOG---
        INFO:toil:Running Toil version 4.2.0a1-3f4e790c1fa32a2c2257bd5cd203a3b51d5d5661.
        Traceback (most recent call last):
          File "/hps/nobackup/production/metagenomics/software/toil-v4/v3nv/lib/python3.6/site-packages/cwltool/process.py", line 455, in fill_in_defaults
            "Missing required input parameter '%s'" % shortname(inp["id"])
        cwltool.errors.WorkflowException: Missing required input parameter 'initial_file'
        
        The above exception was the direct cause of the following exception:
        
        Traceback (most recent call last):
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/worker.py", line 368, in workerScript
            job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer)
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/job.py", line 1392, in _runner
            returnValues = self._run(jobGraph, fileStore)
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/job.py", line 1329, in _run
            return self.run(fileStore)
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/cwl/cwltoil.py", line 938, in run
            self.runtime_context.make_fs_access(""))
          File "/hps/nobackup/production/metagenomics/software/toil-v4/v3nv/lib/python3.6/site-packages/cwltool/process.py", line 455, in fill_in_defaults
            "Missing required input parameter '%s'" % shortname(inp["id"])
```

[test.zip](https://github.com/DataBiosphere/toil/files/4826329/test.zip)



┆Issue is synchronized with this [Jira Task](https://ucsc-cgl.atlassian.net/browse/TOIL-575)
┆Issue Number: TOIL-575



Discussion:
@DailyDreaming Our conditionals are supposed to be working, right? I know we don't yet have 100% support for CWL 1.2.

I haven't had much luck digging up the work-in-progress 1.2 spec to verify that implementations are in fact supposed to allow steps with missing inputs to exist if conditionals say not to run them. But that would make sense.
Tagging in @kaushik-work who wrote the `toil-cwl-runner` conditionals implementation
@KeteSakharova thanks for bringing this case to attention. @tetron and @mr-c I'd like to discuss what behavior we want in this case. I see how `cwltool` handles it, but I don't agree quite yet.

**My position is that if the user expects that an input might be missing/null, then they have to explicitly mark that input in the step as optional.**

My reasoning is as follows:

1. Marking inputs as required or optional is a way for the developer to make their workflow more robust.
1. The behavior for `cwltool` seems to be that we silently mark all inputs as optional for the purposes of evaluating the conditional expression
1. This silent conversion of all types to optional types will lead to hard to debug errors in workflows: when the expression is written, a developer has to remember that all inputs, regardless of whether they are marked required, are actually optional for the purposes of the expression only.

@KeteSakharova for your use case, what do you think about marking the input as optional in the step?

Thanks
-Kaushik


I think this is a Toil bug with handling optional inputs.

> The behavior for cwltool seems to be that we silently mark all inputs as optional for the purposes of evaluating the conditional expression

That's not what's happening here.  They are optional because at Workflow level (not the step level) they were explicitly declared to be optional.  Missing optional parameters are assigned `null` to make a valid input object.  It is those `null` values that propagate to `step_paired` and `step_single`.  Here's what the spec says:

> When accepting an input object, all input parameters must have a value. If an input parameter is missing from the input object, it must be assigned a value of null (or the value of default for that parameter, if provided) for the purposes of validation and evaluation of expressions.

What's happening here is that it is calling `fill_in_defaults` before evaluating conditional, and `fill_in_defaults` noticed an input was missing.  However that fills in defaults from the  command line tool definition, not the workflow step definition (which can have its own defaults).  The conditional is part of the workflow step definition, so the conditional should be evaluated before `fill_in_defaults` -- the command line tool defaults don't apply yet.

We should add this case to the conformance tests.

Also, the conditionals should be `$(inputs.single === null)`  and `$(inputs.single !== null)`

Resolving patch:
diff --git a/src/toil/cwl/cwltoil.py b/src/toil/cwl/cwltoil.py
--- a/src/toil/cwl/cwltoil.py
+++ b/src/toil/cwl/cwltoil.py
@@ -927,6 +927,10 @@ def run(self, file_store: AbstractFileStore) -> Any:
         
         
         cwljob = resolve_dict_w_promises(self.cwljob)
+
+        if self.conditional.is_false(cwljob):
+            return self.conditional.skipped_outputs()
+
         fill_in_defaults(
             self.step_inputs,
             cwljob,
@@ -934,9 +938,6 @@ def run(self, file_store: AbstractFileStore) -> Any:
 
         required_env_vars = self.populate_env_vars(cwljob)
 
-        if self.conditional.is_false(cwljob):
-            return self.conditional.skipped_outputs()
-
         immobile_cwljob_dict = copy.deepcopy(cwljob)
         for inp_id in immobile_cwljob_dict.keys():
             found = False


Test patch: