hpcc-systems/HPCC-Platform — Issue #906: DFS physical delete can be slow, consider more threads
Author: jakesmith
Created: 2011-11-04 18:15:07+00:00
State: open

When DFS deletes a logical file, it performs a N way delete of the physicals using an asyncfor, this is currently hard-wired to limit to 10 concurrent threads.
On a slow filing system (ext2 + ext3 ?) with large files, this can be slow.
I think default could/should be higher, and/or configurable and possible an option to fileservice calls.