Patent ID: 11966986
Assignee: META PLATFORMS, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 0:
1. A method comprising, by a client system:
receiving, at the client system, a user query comprising an audio input from a user, wherein the audio input comprises a coreference to a target object, and wherein the user is associated with a current context;
accessing, responsive to receiving the audio input from the user, visual data from one or more camera associated with the client system;
analyzing, by a scene understanding engine, the visual data to identify a plurality of objects portrayed in the visual data;
resolving the coreference to the target object from among the plurality of objects by identifying the target object from among the plurality of identified objects portrayed in the visual data;
resolving the target object to a specific entity from a plurality of selected entities corresponding to the plurality of objects, wherein the plurality of selected entities are selected based on a respective recency of each of the selected entities and a respective correlation of each of the selected entities to the current context associated with the user; and
providing, at the client system, a response to the user query, wherein the response comprises information about the specific entity, and wherein the response is in one or more modalities determined based on device capabilities of the client system.