Fix connection manager

Targets #580.
You can merge the `if`s.