Patent ID: 11948580
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 19:
20. A method implemented by one or more processors, the method comprising:
obtaining a plurality of first-party interpretations of a spoken utterance, wherein each of the first-party interpretations of the spoken utterance are determined by an automated assistant, and wherein the automated assistant determines the first-party interpretations based on:
processing, using an automatic speech recognition (ASR) model, audio data that captures the spoken utterance of a user to generate ASR output, the audio data being generated by one or more microphones of a client device of the user, and the spoken utterance being directed to the automated assistant executed at least in part at the client device;
processing, using a natural language understanding (NLU) model, the ASR output, to generate NLU output; and
determining, based on the NLU output, the plurality of first-party interpretations of the spoken utterance, each of the plurality of first-party interpretations being associated with a corresponding first-party predicted value indicative of a magnitude of confidence that each of the first-party interpretations are predicted to satisfy the spoken utterance;

obtaining a plurality of third-party interpretations of the spoken utterance, wherein each of the third-party interpretations of the spoken utterance are determined by one or more third-party agents, and wherein each of the one or more third-party agents determine one or more of the third-party interpretations based on:
receiving, from the automated assistant over one or more networks, and based on the NLU output, one or more structured requests; and
determining, based on the one or more structured requests, the plurality of third-party interpretations of the spoken utterance, each of the plurality of third-party interpretations being associated with a corresponding third-party predicted value indicative of a magnitude of confidence that each of the third-party interpretations are predicted to satisfy the spoken utterance;

selecting, based on the corresponding first-party predicted values and the corresponding third-party predicted values, a given interpretation of the spoken utterance from among the plurality of first-party interpretations and the plurality of third-party interpretations; and
transmitting the given interpretation of the spoken utterance to the automated assistant or a given third-party agent, of the one or more third-party agents.