Stop executors even when server was never started