Only add listeners for new server connections, not all in
auto-configuration (#192)