Remove users from registry when they have no common channels with us

Fixes #42.