Fix the issue that GroupBy may not call 'unsubscribe'

Wouldn't this be an `if/else`? If we just `unsubscribed` in the previous lines I don't think we should ever go through this flow.