Repository: common-workflow-language/cwltool

Issue:
Add nothing to command line if prefix is absent for boolean
Fix https://github.com/common-workflow-language/cwltool/issues/618
Conformance test: https://github.com/common-workflow-language/common-workflow-language/pull/610
Expression-like valid javascript fails
Running the command line tool:

```yaml
cwlVersion: v1.0
class: CommandLineTool
baseCommand: echo

requirements:
  - class: InlineJavascriptRequirement
inputs:
  - id: parameter
    inputBinding:
      valueFrom: $(true)
    type: string?

outputs: []
```

Fails:

```
/usr/local/bin/cwltool 1.0.20180130110340
Resolved '../../workflows/test.cwl' to 'file:///Users/th10/checkouts/arvados-pipeline/arvados-pipelines/cwl/workflows/test.cwl'
Parsed job order from command line: {
    "id": "../../workflows/test.cwl",
    "parameter": null
}
[job test.cwl] initializing from file:///Users/th10/checkouts/arvados-pipeline/arvados-pipelines/cwl/workflows/test.cwl
[job test.cwl] {
    "parameter": null
}
[job test.cwl] path mappings is {}
[job test.cwl] command line bindings is [
    {
        "position": [
            -1000000,
            0
        ],
        "datum": "echo"
    },
    {
        "valueFrom": "$(true)",
        "position": [
            0,
            "parameter"
        ],
        "datum": null
    }
]
Workflow error, try again with --debug for more information:
../../workflows/test.cwl:10:7: Traceback (most recent call last):
                               
                                 File
                                 "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line
                                 163, in evaluator
                                   return next_seg(m.group(0)[m.end(1) - m.start(0):-1],
                                   obj[m.group(1)])
                               
                               KeyError: 'true'
                               
                               
                               During handling of the above exception, another exception occurred:
                               
                               
                               Traceback (most recent call last):
                               
                                 File
                                 "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line
                                 234, in do_eval
                                   js_console=js_console)
                               
                                 File
                                 "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line
                                 187, in interpolate
                                   debug=debug, js_console=js_console)
                               
                                 File
                                 "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line
                                 165, in evaluator
                                   raise WorkflowException("%s%s" % (m.group(1), w))
                               
                               cwltool.errors.WorkflowException: true'true'
                               
                               
                               During handling of the above exception, another exception occurred:
                               
                               
                               Traceback (most recent call last):
                               
                                 File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py",
                                 line 211, in generate_arg
                                   value = self.do_eval(binding["valueFrom"], context=value)
                               
                                 File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py",
                                 line 263, in do_eval
                                   force_docker_pull=self.force_docker_pull)
                               
                                 File
                                 "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line
                                 237, in do_eval
                                   raise WorkflowException("Expression evaluation error:\n%s" % e)
                               
                               cwltool.errors.WorkflowException: Expression evaluation error:
                               true'true'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 163, in evaluator
    return next_seg(m.group(0)[m.end(1) - m.start(0):-1], obj[m.group(1)])
KeyError: 'true'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 234, in do_eval
    js_console=js_console)
  File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 187, in interpolate
    debug=debug, js_console=js_console)
  File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 165, in evaluator
    raise WorkflowException("%s%s" % (m.group(1), w))
cwltool.errors.WorkflowException: true'true'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py", line 211, in generate_arg
    value = self.do_eval(binding["valueFrom"], context=value)
  File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py", line 263, in do_eval
    force_docker_pull=self.force_docker_pull)
  File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 237, in do_eval
    raise WorkflowException("Expression evaluation error:\n%s" % e)
cwltool.errors.WorkflowException: Expression evaluation error:
true'true'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/cwltool/main.py", line 976, in main
    **vars(args))
  File "/usr/local/lib/python3.6/site-packages/cwltool/main.py", line 291, in single_job_executor
    for r in jobiter:
  File "/usr/local/lib/python3.6/site-packages/cwltool/draft2tool.py", line 500, in job
    j.command_line = flatten(list(map(builder.generate_arg, builder.bindings)))
  File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py", line 211, in generate_arg
    value = self.do_eval(binding["valueFrom"], context=value)
  File "/usr/local/lib/python3.6/site-packages/schema_salad/sourceline.py", line 155, in __exit__
    raise self.makeError("\n".join(traceback.format_exception(exc_type, exc_value, tb)))
cwltool.errors.WorkflowException: ../../workflows/test.cwl:10:7: Traceback (most recent call last):
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 163, in evaluator
../../workflows/test.cwl:10:7:     return next_seg(m.group(0)[m.end(1) - m.start(0):-1], obj[m.group(1)])
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: KeyError: 'true'
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: During handling of the above exception, another exception occurred:
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: Traceback (most recent call last):
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 234, in do_eval
../../workflows/test.cwl:10:7:     js_console=js_console)
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 187, in interpolate
../../workflows/test.cwl:10:7:     debug=debug, js_console=js_console)
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 165, in evaluator
../../workflows/test.cwl:10:7:     raise WorkflowException("%s%s" % (m.group(1), w))
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: cwltool.errors.WorkflowException: true'true'
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: During handling of the above exception, another exception occurred:
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: Traceback (most recent call last):
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py", line 211, in generate_arg
../../workflows/test.cwl:10:7:     value = self.do_eval(binding["valueFrom"], context=value)
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/builder.py", line 263, in do_eval
../../workflows/test.cwl:10:7:     force_docker_pull=self.force_docker_pull)
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7:   File "/usr/local/lib/python3.6/site-packages/cwltool/expression.py", line 237, in do_eval
../../workflows/test.cwl:10:7:     raise WorkflowException("Expression evaluation error:\n%s" % e)
../../workflows/test.cwl:10:7: 
../../workflows/test.cwl:10:7: cwltool.errors.WorkflowException: Expression evaluation error:
../../workflows/test.cwl:10:7: true'true'
```

This is because this javascript expression is being evaluated as a CWL expression and failing here: https://github.com/common-workflow-language/cwltool/blob/52c276d1a384bed977f2355513f1c07a66b11971/cwltool/expression.py#L165 A fix for this is to fall back to parsing the expression as JavaScript if it fails to be parsed as a CWL expression - I'll have a look at making a pull request with this in.


Discussion:
Jenkins, test this please
Jenkins, test this please
Thanks! I'm combing this with #630 into a single PR: https://github.com/common-workflow-language/cwltool/pull/632
Hello @ThomasHickman , thank you for the report! This is a bug, yes; also a terrible error message and user experience.

PRs to fix would be very welcome. We should also turn this into a CWL conformance test.

Resolving patch:
diff --git a/cwltool/expression.py b/cwltool/expression.py
--- a/cwltool/expression.py
+++ b/cwltool/expression.py
@@ -120,55 +120,76 @@ def scanner(scan):  # type: (Text) -> List[int]
         return None
 
 
-def next_seg(remain, obj):  # type: (Text, Any) -> Any
-    if remain:
-        m = segment_re.match(remain)
+def next_seg(parsed_string, remaining_string, current_value):  # type: (Text, Text, Any) -> Any
+    if remaining_string:
+        m = segment_re.match(remaining_string)
+        next_segment_str = m.group(0)
+
         key = None  # type: Union[Text, int]
-        if m.group(0)[0] == '.':
-            key = m.group(0)[1:]
-        elif m.group(0)[1] in ("'", '"'):
-            key = m.group(0)[2:-2].replace("\\'", "'").replace('\\"', '"')
+        if next_segment_str[0] == '.':
+            key = next_segment_str[1:]
+        elif next_segment_str[1] in ("'", '"'):
+            key = next_segment_str[2:-2].replace("\\'", "'").replace('\\"', '"')
 
         if key:
-            if isinstance(obj, list) and key == "length" and not remain[m.end(0):]:
-                return len(obj)
-            if not isinstance(obj, dict):
-                raise WorkflowException(" is a %s, cannot index on string '%s'" % (type(obj).__name__, key))
-            if key not in obj:
-                raise WorkflowException(" does not contain key '%s'" % key)
+            if isinstance(current_value, list) and key == "length" and not remaining_string[m.end(0):]:
+                return len(current_value)
+            if not isinstance(current_value, dict):
+                raise WorkflowException("%s is a %s, cannot index on string '%s'" % (parsed_string, type(current_value).__name__, key))
+            if key not in current_value:
+                raise WorkflowException("%s does not contain key '%s'" % (parsed_string, key))
         else:
             try:
-                key = int(m.group(0)[1:-1])
+                key = int(next_segment_str[1:-1])
             except ValueError as v:
                 raise WorkflowException(u(str(v)))
-            if not isinstance(obj, list):
-                raise WorkflowException(" is a %s, cannot index on int '%s'" % (type(obj).__name__, key))
-            if key >= len(obj):
-                raise WorkflowException(" list index %i out of range" % key)
+            if not isinstance(current_value, list):
+                raise WorkflowException("%s is a %s, cannot index on int '%s'" % (parsed_string, type(current_value).__name__, key))
+            if key >= len(current_value):
+                raise WorkflowException("%s list index %i out of range" % (parsed_string, key))
+
         try:
-            return next_seg(remain[m.end(0):], obj[key])
-        except WorkflowException as w:
-            raise WorkflowException("%s%s" % (m.group(0), w))
+            return next_seg(parsed_string + remaining_string, remaining_string[m.end(0):], current_value[key])
+        except KeyError:
+            raise WorkflowException("%s doesn't have property %s" % (parsed_string, key))
     else:
-        return obj
+        return current_value
 
 
 def evaluator(ex, jslib, obj, fullJS=False, timeout=None, force_docker_pull=False, debug=False, js_console=False):
     # type: (Text, Text, Dict[Text, Any], bool, int, bool, bool, bool) -> JSON
     m = param_re.match(ex)
+
+    expression_parse_exception = None
+    expression_parse_succeeded = False
+
     if m:
-        if m.end(1)+1 == len(ex) and m.group(1) == "null":
+        first_symbol = m.group(1)
+        first_symbol_end = m.end(1)
+
+        if first_symbol_end + 1 == len(ex) and first_symbol == "null":
             return None
         try:
-            return next_seg(m.group(0)[m.end(1) - m.start(0):-1], obj[m.group(1)])
-        except Exception as w:
-            raise WorkflowException("%s%s" % (m.group(1), w))
-    elif fullJS:
+            if obj.get(first_symbol) is None:
+                raise WorkflowException("%s is not defined" % first_symbol)
+
+            return next_seg(first_symbol, ex[first_symbol_end:-1], obj[first_symbol])
+        except WorkflowException as w:
+            expression_parse_exception = w
+        else:
+            expression_parse_succeeded = True
+
+    if fullJS and not expression_parse_succeeded:
         return sandboxjs.execjs(ex, jslib, timeout=timeout, force_docker_pull=force_docker_pull, debug=debug, js_console=js_console)
     else:
-        raise sandboxjs.JavascriptException(
-            "Syntax error in parameter reference '%s' or used Javascript code without specifying InlineJavascriptRequirement.",
-            ex)
+        if expression_parse_exception is not None:
+            raise sandboxjs.JavascriptException(
+                "Syntax error in parameter reference '%s': %s. This could be due to using Javascript code without specifying InlineJavascriptRequirement." % \
+                    (ex[1:-1], expression_parse_exception))
+        else:
+            raise sandboxjs.JavascriptException(
+                "Syntax error in parameter reference '%s'. This could be due to using Javascript code without specifying InlineJavascriptRequirement." % \
+                    ex)
 
 
 def interpolate(scan, rootvars,


Test patch: