Bug 935636 - Avoid duplicating data on contact merge

It's explained in a comment one line below.