Remove parents belong to when merging to avoid wrong binding