holoirc: fix another crash related with removing channels