AbstractObservedSet: don't mark objects as removed if they weren't actually removed from the underlying set
Tests for observed set & map components.