Remove the correct mismatches when retracting a matching