Take care of cleanup procedure.

We should wait for subthreads that uses sockets to exit
before closing a socket. This was fixed.
Also LocalEndPoint state was hidden when EP is going
down.