Correctly handle bye in conferences.