Fix potential segv in pairremove
Better way of doing pairremove