only send source-remove when there is > 1 participant, closes #16