Patent ID: 11907674
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 22:
23. A method implemented by one or more processors, the method comprising:
receiving natural language (NL) based input associated with a client device of a user;
generating a multi-modal response that is responsive to the NL based input, wherein generating the multi-modal response that is responsive to the NL based input comprises:
processing, using a large language model (LLM), LLM input to generate LLM output, the LLM input including at least the NL based input; and
determining, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response, wherein the textual content includes a plurality of textual segments, and wherein the multimedia content includes a multimedia content item; and

causing the multi-modal response to be rendered at the client device of the user, wherein causing the multi-modal response to be rendered at the client device of the user comprises:
causing the plurality of textual segments to be visually rendered via a display of the client device; and
causing the multimedia content item to be visually rendered via the display of the client device and/or via one or more speakers of the client device, wherein the multimedia content item is interleaved between a first textual segment, of the plurality of textual segments, and a second textual segment, of the plurality of textual segments.