Actually add server to server set.