prune and dedupe