improve remove party member logic