Remove redundant erase calls

We used IDToValue.clear() already, no need to call erase before it.