Revert "Set number of worker cores to 1 to prevent bz2 decompression problems"

I mistakenly thought that "spark.cores.max" means max. number of cores per worker. It's actually max. cores to use in the whole cluster.

This reverts commit 87f1743ebe330700d68f07e6196fc8c449178a74.
Call ExecutorService.shutdown() at finish to gracefully stop the dangling threads

The program was not stopping even after SparkContext was stopped and everything. The ExecutorService needs to be shut down too.
Fix the issue where DistConfig.sparkLogLevel was getting null and not the default Level.WARN

Problem was: getValue() returns null directly instead of passing it through the closure.
Return 0 for completed tasks and failed tasks count respectively, if the maps have no such value.

Example: Often in a stage the first TaskEnd event may tell us that the task has failed - in that case doing this.stageIdToTasksComplete(stageId) would cause a NoSuchElementException. Similarly for stageIdToTasksFailed.
Merge branch 'feature-loglevel-config' into nildev2-final
Merge branch 'output-nothreads' into nildev2-final