Can't use remove on unmodifiable sets