full re-sort when removing duplicates
passing in required criteria