Fix a bug when disconnecting a server via config.

- Fix bug where when disconnecting a server via a config change, the
  server's config would be gone, but it would still try to read from it.