A computer-implemented method comprising: receiving, by a speech-enabled device, an incoming voice call, wherein the speech-enabled device stores user account information for each of a plurality of known users of the speech-enabled device, wherein a first user of the plurality of known users has, in their stored user account information, a contact entry for a particular number, and wherein a second user of the plurality of known users has, in their stored user account information, a different contact entry for the particular number; determining, by the speech-enabled device, a calling number and a called number from the incoming voice call, wherein the calling number is the particular number and the called number corresponds to the first user; identifying, by the speech-enabled device, from the user account information stored on the speech-enabled device, the user account of the first user that corresponds to the called number; determining a contact name for the calling number based on the contact entry for the particular number in the identified user account of the first user, comprising: providing, by the speech-enabled device and to a voice-enabled virtual assistant server, a query that includes the identified user account of the first user and the calling number; and receiving, by the speech-enabled device and from the voice-enabled virtual assistant server, the contact name in response to the query; and providing, by the speech-enabled device, the contact name for output. receiving, by a speech-enabled device, an incoming voice call, wherein the speech-enabled device stores user account information for each of a plurality of known users of the speech-enabled device, wherein a first user of the plurality of known users has, in their stored user account information, a contact entry for a particular number, and wherein a second user of the plurality of known users has, in their stored user account information, a different contact entry for the particular number; determining, by the speech-enabled device, a calling number and a called number from the incoming voice call, wherein the calling number is the particular number and the called number corresponds to the first user; identifying, by the speech-enabled device, from the user account information stored on the speech-enabled device, the user account of the first user that corresponds to the called number; determining a contact name for the calling number based on the contact entry for the particular number in the identified user account of the first user, comprising: providing, by the speech-enabled device and to a voice-enabled virtual assistant server, a query that includes the identified user account of the first user and the calling number; and receiving, by the speech-enabled device and from the voice-enabled virtual assistant server, the contact name in response to the query; and providing, by the speech-enabled device, the contact name for output. providing, by the speech-enabled device and to a voice-enabled virtual assistant server, a query that includes the identified user account of the first user and the calling number; and receiving, by the speech-enabled device and from the voice-enabled virtual assistant server, the contact name in response to the query; and providing, by the speech-enabled device, the contact name for output.