Actually, my first patch just did what you said. But a case called DriverSuite failed because of this, as I stated in my code comments. This case new a SparkContext and didn't call SparkContext.stop, so TaskScheduler.stop has no chance to run, and this threadpool has no chance to be shutdown, and the main thread just can't exit. I also know that new a Threadpool in every call is not that good, but currently I can't come up other good idea. Any other advice?