Fix logic in toggle_active_speakers.