Add multiple writer threads to improve throughput

Refer https://jira.corp.inmobi.com/browse/SCRIBE-37
We can remove the check for subDirectory.empty since we are not using it to build spooling path.