Isn't there a built-in way to do the merge in hadoop? If I'm reading the code correctly, this would double the size of output data(status files + merged). Shouldn't you delete those after merging them?