fix: showq may not be available on all slurm systems
fix: add option for cuda
fix: enable cuda not required - variable set by asking for gres=gpu
ENH: Large SGE jobs were overloading on qstat

When large jobs were run with SGE, the accumulation of
qstat queries was causing a massive load on the cluster
server, and was affecting overal system performance.

This modification queries based on userid, stores information
about all that users jobs (both running and recently finished),
subsequent queries are then addressed by looking at the cached
values first, then only updating with a system qstat call
periodically.  This change so that qstat is called on a
more reasonable replication time frame.

User can supply externally supplied version of qmake.

Prevent DOS style of attacks on the batch processing
server by preventing continuous queries by many jobs.

This was affecting the performance of the entire server
and the excess load of querying when jobs were done in the
niavie way was affecting the performance of dispatching new
jobs.

The following two scripts can be used as plugin_args to
provide cached versions of qmake suitable for running
huge jobs.

plugin_args=dict(template=JOB_SCRIPT,
            qsub_args="-S /bin/bash -cwd -pe smp 1-12 -l h_vmem=19G,mem_free=9G -o /dev/null -e /dev/null " + CLUSTER_QUEUE,
            qstatProgramPath=qstat_immediate.sh,
            qstatCachedProgramPath=qstat_cached.sh))

=qstat_cached.sh===================================================
\#!/bin/bash
\# \author Hans J. Johnson
\# This file provides a wrapper around qstat to ensure that
\# the qstat server is not overloaded with too many requests

\#debug_file=/dev/null
debug_file=/tmp/TESTINGLOG
qstat_cache=/tmp/qstat.xml

echo "USING EXTERNAL QSTAT: $@" >> ${debug_file}  2>&1

older_than_60_sec=$( find $(dirname ${qstat_cache}) -maxdepth 1 -name $(basename ${qstat_cache}) -mmin $(echo 5/60 |bc -l) )
if [ -z "${older_than_60_sec}" ]; then
  DoQstatNow=$(dirname ${0})/qstat_immediate.sh
  ${DoQstatNow} $@
else
  echo "using cache $(date)" >> ${debug_file} 2>&1
  cat ${qstat_cache}
fi
===================================================================

=qstat_immediate.sh===============================================
\#!/bin/bash
\# \author Hans J. Johnson
\# This file provides a wrapper around qstat to ensure that
\# the qstat server is not overloaded with too many requests

\#debug_file=/dev/null
debug_file=/tmp/TESTINGLOG
qstat_cache=/tmp/qstat.xml

echo "USING EXTERNAL QSTAT: $@" >> ${debug_file}  2>&1

echo "Refreshing $(date)" >> ${debug_file} 2>&1
cacheUpdated=0;
while [ ${cacheUpdated} -eq 0 ]; do
  if [ ! -f  ${qstat_cache}_lock ]; then
    touch ${qstat_cache}_lock
    DoQstatNow=$(which qstat)
    ${DoQstatNow} $@ > ${qstat_cache}_tmp 2>&1
    mv ${qstat_cache}_tmp ${qstat_cache}
    rm ${qstat_cache}_lock
    let cacheUpdated=1
  else
    echo "Waiting for contention lock $(date)"  >> ${debug_file} 2>&1
    sleep 1
  fi
done

cat ${qstat_cache}
===================================================================
fix: update conda
dbg: conda update
dbg: anaconda to miniconda update
fix: parenthesis fixes
fix: update conda in environment
install ordereddict for py26
fix: moving save from numpy to pickle
install ordereddict for py26
fix: moving save from numpy to pickle
Merge pull request #6 from satra/enh/sge

fix: parenthesis fixes
Merge pull request #797 from satra/fix/conda

fix: update conda
Merge pull request #728 from BRAINSia/EnhanceQSubProcessing

ENH: Large SGE jobs were overloading on qstat
Merge pull request #786 from satra/fix/slurm

fix: showq may not be available on all slurm systems
Address large file with dcm2nii _parse_stdout

dcm2nii doesn't print the line _parse_stdout expects when it handles a very large file, which causes large files to be omitted from the output. This fixes that.
remove redundant imports
Added options to FSL UnaryMaths