updated to deal with canned data problems after partitioning