daemon: avoid double close

Fixes #29. I tested by starting then SIGTERMing and I didn't see a crash
without this.
Merge pull request #30 from philips/fix-double-channel-close

daemon: avoid double close