bug fix for partitioner so it uses actual data and cleaning up test data so its complete sets