Fix speaker retry bug