Repository: DataBiosphere/toil

Issue:
NameError: name 'jobID' is not defined
I'm trying to update from a previous toil version but am running into issues with CWL-based workflows on Torque. From a fresh run (no --restart):

```
cwltoil --batchSystem torque --disableCaching --stats --writeLogs /projects/ps-yeolab4/software/eclip/0.5.0/examples/204_RBFOX2/.tmp/toillogs --jobStore file:/projects/ps-yeolab4/software/eclip/0.5.0/examples/204_RBFOX2/.tmp/cwltoil_jobstore --workDir /projects/ps-yeolab4/software/eclip/0.5.0/examples/204_RBFOX2/.tmp/workdir --outdir /projects/ps-yeolab4/software/eclip/0.5.0/examples/204_RBFOX2/.tmp/outdir --clean never --cleanWorkDir onSuccess --logInfo --clusterStats --realTimeLogging --stats --defaultMemory 64.0G /projects/ps-yeolab4/software/eclip/0.5.0/cwl/wf_get_peaks_scatter_pe.cwl ./204_RBFOX2.yaml

Version: 4.2.9
Commit: 4ab084cdbe285bb53ed7dc3323dcbbaaf6cf14fc
INFO:toil:Running Toil version 3.20.0-cf34ca3416697f2abc816b2538f20ee29ba16932.
INFO:toil.realtimeLogger:Starting real-time logging.
DEBUG:toil.jobStores.fileJobStore:Path to job store directory is '/projects/ps-yeolab4/software/eclip/0.5.0/examples/204_RBFOX2/.tmp/cwltoil_jobstore'.
INFO:toil.worker:Redirecting logging to /projects/ps-yeolab4/software/eclip/0.5.0/examples/204_RBFOX2/.tmp/workdir/toil-d5f51da8-e1be-4935-a76d-b413af78b420-ae138f116f66f80aa24dcf740000001c/tmpujo6hpjn/worker_log.txt
Exception in thread Thread-251:
Traceback (most recent call last):
  File "/projects/ps-yeolab4/software/eclipconda/envs/toil-3.19.0/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/projects/ps-yeolab4/software/eclipconda/envs/toil-3.19.0/lib/python3.6/site-packages/toil/batchSystems/abstractGridEngineBatchSystem.py", line 234, in run
    activity |= self.createJobs(newJob)
  File "/projects/ps-yeolab4/software/eclipconda/envs/toil-3.19.0/lib/python3.6/site-packages/toil/batchSystems/abstractGridEngineBatchSystem.py", line 136, in createJobs
    subLine = self.prepareSubmission(cpu, memory, jobID, command)
  File "/projects/ps-yeolab4/software/eclipconda/envs/toil-3.19.0/lib/python3.6/site-packages/toil/batchSystems/torque.py", line 126, in prepareSubmission
    return self.prepareQsub(cpu, memory, jobID) + [self.generateTorqueWrapper(command)]
  File "/projects/ps-yeolab4/software/eclipconda/envs/toil-3.19.0/lib/python3.6/site-packages/toil/batchSystems/torque.py", line 212, in generateTorqueWrapper
    stdoutfile = self.boss.formatStdOutErrPath(jobID, 'torque', '$PBS_JOBID', 'std_output')
NameError: name 'jobID' is not defined
WARNING:toil.leader:A result seems to already have been processed for job 0
```

┆Issue is synchronized with this [Jira Task](https://ucsc-cgl.atlassian.net/browse/TOIL-406)
┆Issue Number: TOIL-406



Discussion:
Looks like nobody tested Torque since https://github.com/DataBiosphere/toil/commit/e9911416a8ea2ef96e5e12e17983bdb495da9b8b

We need to add jobID as an argument to generateTorqueWrapper and pass it through from prepareSubmission.

@jeffrey856 You could probably do this.

Resolving patch:
diff --git a/src/toil/batchSystems/torque.py b/src/toil/batchSystems/torque.py
--- a/src/toil/batchSystems/torque.py
+++ b/src/toil/batchSystems/torque.py
@@ -123,7 +123,7 @@ def killJob(self, jobID):
             subprocess.check_call(['qdel', self.getBatchSystemID(jobID)])
 
         def prepareSubmission(self, cpu, memory, jobID, command):
-            return self.prepareQsub(cpu, memory, jobID) + [self.generateTorqueWrapper(command)]
+            return self.prepareQsub(cpu, memory, jobID) + [self.generateTorqueWrapper(command, jobID)]
 
         def submitJob(self, subLine):
             process = subprocess.Popen(subLine, stdout=subprocess.PIPE)
@@ -204,7 +204,7 @@ def prepareQsub(self, cpu, mem, jobID):
 
             return qsubline
 
-        def generateTorqueWrapper(self, command):
+        def generateTorqueWrapper(self, command, jobID):
             """
             A very simple script generator that just wraps the command given; for
             now this goes to default tempdir


Test patch: