clean up dedup and trim