colorg.el (colorg-select-server): Added.  Use it.  Various nits:

Rename colorg-process to colorg-reply-handler.
Leave on the server when getting out of colorg mode.
(colorg-mode): Do not login, this is done in colorg-select-server.