Fix channel deletion on remote clients (Fixes #171)

Refactor some of the channel removal code to make channel removal work again in order to fix #171.