A computer-implemented method comprising: receiving, by a first user device, an audio signal encoding an utterance; obtaining, by the first user device, a first speaker model for a first user of the first user device; obtaining, by the first user device for a second user of a corresponding second user device that is co-located in a physical area near a physical location of the first user device, a second score that indicates a respective likelihood that the utterance was spoken by the second user, wherein obtaining the second score comprises: determining, by the first user device, that the second user device is co-located with the first user device; and either: receiving, by the first user device, the second score from the second user device; or: determining, by the first user device, a device identifier for the second user device; providing, by the first user device, the device identifier to the server; and receiving the second score from the server in response to providing the identifier to the server; and determining, by the first user device, whether the utterance was spoken by the first user using the first speaker model and the second score. receiving, by a first user device, an audio signal encoding an utterance; obtaining, by the first user device, a first speaker model for a first user of the first user device; obtaining, by the first user device for a second user of a corresponding second user device that is co-located in a physical area near a physical location of the first user device, a second score that indicates a respective likelihood that the utterance was spoken by the second user, wherein obtaining the second score comprises: determining, by the first user device, that the second user device is co-located with the first user device; and either: receiving, by the first user device, the second score from the second user device; or: determining, by the first user device, a device identifier for the second user device; providing, by the first user device, the device identifier to the server; and receiving the second score from the server in response to providing the identifier to the server; and determining, by the first user device, whether the utterance was spoken by the first user using the first speaker model and the second score. determining, by the first user device, that the second user device is co-located with the first user device; and either: receiving, by the first user device, the second score from the second user device; or: determining, by the first user device, a device identifier for the second user device; providing, by the first user device, the device identifier to the server; and receiving the second score from the server in response to providing the identifier to the server; and receiving, by the first user device, the second score from the second user device; or: determining, by the first user device, a device identifier for the second user device; providing, by the first user device, the device identifier to the server; and receiving the second score from the server in response to providing the identifier to the server; and determining, by the first user device, a device identifier for the second user device; providing, by the first user device, the device identifier to the server; and receiving the second score from the server in response to providing the identifier to the server; and determining, by the first user device, whether the utterance was spoken by the first user using the first speaker model and the second score.