Clean up the client factory + solve some minor bugs