Fix being kicked when removing leader