Fix looping through attachment ids.

The 'id' hash will be built further down.
Only remove attachments the have been removed from the current object.