Perhaps this should be a Set ? Removing on a List is O(N).