Fix threading issue in ConnectionManager

As reported here: https://github.com/SamSaffron/message_bus/issues/18

Let us (@tskogberg and I) know if you think this is not good enough.
oops