Hmm, the man said in the voice of the first speaker.
The man was talking in the first speaker's voice.