Question: *Edit for future readers: I assume the problem here was that the tool i was running tried to contact the servers of the manufacturer. It wouldn't continue working unless the connection was successful, but wouldn't indicate what it was trying to do. The `--preserve-entire-environment` flag of cwltool fixed the problem because it enabled to software to communicate through the network.

Original Questions:

I'm new to cwl and not very experienced in programming in general, so please bear with me.

I have created CommandLineTool which is supposed to use Oxford Nanopore's albacore basecaller to generate .fastq-Files from .fast5 current data.

When using this CommandLineTool, it takes about five minutes to generate all output files in the output directory. However, the cwl-runner job does not end for several more hours.
When i invoke the basecaller manually from the command line, using the same parameters i use in the CommandLineTool, it generates the exact same data and then finishes in about 5 minutes.

Could this be caused by a faulty outputs-field? I am still having difficulties understanding how outputs in cwl work. But the fact that the tool takes multiple hours to terminate makes it difficult for me to trace where my mistake lies.

The outputs i am expecting are some report files: *"configuration.cfg", "pipeline.log", "sequencing_summary.txt", "sequencing_telemetry.js"*
As well as a directory called *"workspace"* containing several subdirectories filled with .fastq-files.

This is the code of the CommandLineTool:

cwlVersion: v1.0
class: CommandLineTool
baseCommand: read_fast5_basecaller.py

inputs:
input_directory:
label: |
Folder of current data in .fast5 format.
type: Directory
inputBinding:
prefix: --input
worker_threads:
label: |
Number of CPU-Cores used for computation.
type: int
inputBinding:
prefix: --worker_threads
flowcell:
label: |
Type of flowcell used in experiment.
type: string
inputBinding:
prefix: --flowcell
kit:
label: |
Type of kit used in experiment.
type: string
inputBinding:
prefix: --kit
output_directory:
label: |
Folder where albacore saves results.
type: string
inputBinding:
prefix: --save_path

outputs:
sequences:
type:
type: array
items: File
outputBinding:
glob: $(inputs.output_directory+"/workspace/pass/*.fasta")
config:
type: File
outputBinding:
glob: $(inputs.output_directory+"configuration.cfg")
pipeline:
type: File
outputBinding:
glob: $(inputs.output_directory+"pipeline.log")
summary:
type: File
outputBinding:
glob: $(inputs.output_directory+"sequencing_summary.txt")
telemetry:
type: File
outputBinding:
glob: $(inputs.output_directory+"sequencing_telemetry.txt")

Thanks in advance for any help/advice.

*edited the post to make it shorter & more comprehensible*

Answer: I figured out a workaround for the hanging, at least on my end (I have albacore installed and have done some MinION runs so I can test in house :)_). I will explain the changes I made so you can hopefully understand. The code:

cwlVersion: v1.0
class: CommandLineTool

inputs:
input_directory:
type: Directory
inputBinding:
prefix: --input
worker_threads:
type: int
inputBinding:
prefix: --worker_threads
flowcell:
type: string
inputBinding:
prefix: --flowcell
kit:
type: string
inputBinding:
prefix: --kit
outputs:
outdir:
type: Directory
outputBinding:
glob: $(runtime.outdir)
baseCommand: read_fast5_basecaller.py
arguments:
- valueFrom: $(runtime.outdir)
prefix: --save_path

Some serious differences here. I added an InlineJavascriptRequirement which can be seen at the bottom under arguments. Basically, I got rid of the output_directory input and made it a java argument referencing save_path. So what cwltool actually does behind the scenes is create a temporary directory where it runs all of its operations (all intermediate files and operations are performed in this directory). It is called and can be referenced by $(runtime.tmpdir). An example of this in the portion of code you posted above is:

[job basecallerBioStar.cwl] /tmp/tmpBOQSQ7$ read_fast5_basecaller.py \

You see it is performing your job in this weird tmp/tmpBOQSQ7 dir which cwltool created. When the tool finishes successfully it copies all of the final outputs from runtime.tmpdir to a final directory called $(runtime.outdir) which is what I changed your save_path to. So now, your read_fast5_basecaller.py script is using this value is your final save_path. Then I used another functionality called glob (which is just a unix functionality) to grab everything that is seen in this $(runtime.outdir) directory and place it in your current working directory. Now you may be thinking, "well I don't want all my read_fast5_basecaller.py data to output to the same directory I run the tool from. I also don't want to have to run my cwltool from within the directory I want the output." Fortunately for us, cwltool has a built-in functionality for that called outdir. So the example of my command-line input is the following:

[dkennetz@node albacore]$ cwltool --outdir /path/where/you/want/read_fast5_basecaller/output/ --preserve-entire-environment \
>readFast5.cwl --flowcell FLO-MIN106 --kit SQK-LSK108 --worker_threads 6 --input_directory /path/to/fast5/1/
Resolved 'readFast5.cwl' to 'file:///users/dkennetz/albacore/readFast5.cwl'
[job readFast5.cwl] /tmp/tmpdhpob_gu$ read_fast5_basecaller.py \
--save_path \
/tmp/tmpdhpob_gu \
--flowcell \
FLO-MIN106 \
--input \
/tmp/tmpf7ewqea7/stgc28aa3d7-0f40-4620-9bfc-a895d8f55416/1 \
--kit \
SQK-LSK108 \
--worker_threads \
4
| 4000 of 4000|##############################################|100% Time: 0:09:32

The tool I wrote completed success and then immediately exited (no hanging).

I hope it works the same for you when you try it tomorrow! This was very fun for me so feel free to contact me if you have any more questions. Dennis