Patent ID: 11935530
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G

Claim 0:
1. A method implemented by one or more processors, the method comprising:
determining a client device action based on one or more instances of user interface input provided by a user of a multimodal client device;
determining, based at least in part on sensor data from one or more sensors of the multimodal client device, a current client device modality of the multimodal client device,
wherein the current client device modality is one of a plurality of discrete client device modalities available for the multimodal client device, and
wherein the sensor data based on which the current client device modality is determined is in addition to any sensor data generated by the one or more instances of user interface input;

generating client device output for the client device action using a multimodal response,
wherein the multimodal response includes:
first output characteristics for the current client device modality and for the client device action, and
second output characteristics for a second client device modality and for the client device action, and

wherein generating the client device output comprises selecting the first output characteristics in response to determining the current client device modality; and

causing the client device output to be rendered by the multimodal client device;
while at least part of the client device output is being rendered by the one or more user interface output devices of the multimodal client device:
detecting, based at least in part on updated sensor data from the one or more sensors, a switch of the multimodal client device from the current client device modality to the second client device modality;
in response to detecting the switch, generating alternate client device output using the multimodal response, wherein generating the alternate client device output comprises selecting the second output characteristics in response to detecting the switch to the second client device modality; and

causing the alternate client device output to be rendered by the multimodal client device.