Fix differentiation between group and one-to-one conversations when leaving conversation