Fix switching servers for clients

Fix non default clients ability to switch servers