Note that the server doesn't need to know who it is serving, but the client needs to know who the server is. This is like calling someone on the phone. Note that this is where the objects as people metaphor breaks down. In the real world, when you ask a doctor for a service, the doctor immediately asks you for information. In the object world, however, this doesn't happen automatically.
Frequent role changing is common in both the real world and the object world.
To make this work requires that the doctor object have a reference to the patient object. This has to be specifically arranged in the object world. One way to do so is to have the original client pass a reference to itself to the server in some message. This reference can then be used to send a message back to the original client. One of the things that needs to happen when a program executes, is that the objects need to be created.