Bug 936206: Short-circuit contact remove().
Merge pull request #13488 from wanderview/contacts-fast-remove

Bug 936206: Short-circuit contact remove().  r=jmcf