Use the log-out icon for the disconnect button.

The current (cross) icon looks like it would close the buffer.