Patent Description:
As mentioned above, many automated assistants are configured to be interacted with via spoken utterances. To preserve user privacy and/or to conserve resources, a user must often explicitly invoke an automated assistant before the automated assistant will fully process a spoken utterance. The explicit invocation of an automated assistant typically occurs in response to certain user interface input being received at a client device. The client device includes an assistant interface that provides, to a user of the client device, an interface for interfacing with the automated assistant (e.g., receives spoken and/or typed input from the user, and provides audible and/or graphical responses), and that interfaces with one or more additional components that implement the automated assistant (e.g., remote server device(s) that process user inputs and generate appropriate responses).

Some user interface inputs that can invoke an automated assistant via a client device include a hardware and/or virtual button at the client device for invoking the automated assistant (e.g., a tap of a hardware button, a selection of a graphical interface element displayed by the client device). Such interface inputs invoke the automated assistant and, since they are provided at the client device, explicitly indicate a desire of the user to interact with the automated assistant via the client device. However, many automated assistants can also be invoked in response to one or more spoken invocations, which are also known as "hot words/phrases" or "trigger words/phrases". For example, a spoken invocation such as "Hey, Assistant," can be spoken to a variety of different devices in order to invoke an automated assistant. This allows for uniformity of operation, without requiring a user to learn multiple spoken invocations for their different devices.

Often, a client device that includes an assistant interface includes one or more locally stored models that the client device utilizes to monitor for an occurrence of a spoken invocation phrase. Such a client device can locally process received audio data utilizing the locally stored model, and discard any audio data that does not include the spoken invocation phrase. However, when local processing of received audio data indicates an occurrence of a spoken invocation phrase, the client device will then cause following audio input to be further processed by the automated assistant. For instance, if a spoken invocation phrase is "Hey, Assistant", and a user speaks "Hey, Assistant, what time is it", audio data corresponding to "what time is it" can be processed by an automated assistant based on detection of "Hey, Assistant", and utilized to provide an automated assistant response of the current time. If, on the other hand, the user simply speaks "what time is it" (without first speaking an invocation phrase), no response from the automated assistant will be provided as a result of "what time is it" not being preceded by an invocation phrase.

Many client devices offer an assistant interface for interfacing with a corresponding automated assistant. For example, mobile phones, stand-alone speaker devices, tablets, in-vehicle computing devices, and/or other client devices can each be utilized by a user to interact with an automated assistant. While offering users the ability to interface with an automated assistant via any of a plurality of disparate client devices can be advantageous, it is often the case that multiple client devices in an environment of a user will at least initially invoke an automated assistant in response to a spoken utterance that includes an invocation phrase. For example, if a user is in an environment with four client devices that each include an assistant interface that is "triggered" by "Hey, Assistant", a spoken utterance of "Hey, Assistant, what time is it" can cause all four devices to transmit data packets including audio data corresponding to at least "what time is it" to one or more remote server devices associated with an automated assistant. While some techniques have been proposed for determining which of the four client devices will respond when the four client devices are all associated with the same user account, it is still the case that all four client devices will transmit audio data in response to the spoken utterance. This causes a waste of network resources (by transmitting four separate instances of the same audio data) and can cause a waste of computational resources (e.g., as a result of remote servers processing all four separate instances of audio data). Moreover, where the four client devices are not associated with the same account, it may be the case that multiple client devices will respond to the spoken utterance. In addition to providing a poor user experience, this also wastes client device resources and networking charges for mobile client devices on a metered network, as a single response from a single client device would have sufficed. For example, if four individuals are in a vehicle and all have a personal client device that is responsive to "Hey, Assistant", the spoken utterance of "Hey, Assistant, what time is it" can cause all four devices to provide an audible response of the current time.

<CIT> discloses a first electronic device with a display and a microphone: sampling audio input using the first microphone; in accordance with the sampling of audio input using the first microphone, sending stop instructions to a second electronic device with a second microphone, the second electronic device external to the first electronic device, wherein the second electronic device is configured to respond to audio input received using the second microphone, and wherein the stop instructions instruct the second electronic device to forgo responding to audio input received using the second microphone, wherein responding to audio input received using the second microphone comprises providing perceptible output.

Implementations set forth herein relate to isolating at least one device, from multiple devices in an environment, for being responsive to assistant invocations (e.g., spoken assistant invocations). These implementations utilize one or more user interface inputs of a user to determine which client device(s) (e.g., a single client device) should be responsive to one or more spoken invocations of an automated assistant and/or to determine which client device(s) should at least temporarily limit their responsiveness to one or more of the spoken invocations.

As one example, assume a user is in an environment (e.g., a vehicle) with three client devices that each include an assistant interface that is "triggered" by "Hey, Assistant". Implementations described herein can be utilized to cause only one of those three devices to be responsive to subsequent spoken invocations of "Hey, Assistant" - while causing the other two of those three device to temporarily limit their responsiveness to "Hey, Assistant". For instance, the other two devices can temporarily limit their responsiveness by completely "turning off" their microphones, by suspending monitoring for occurrences of the spoken invocation phrase (e.g., suspending any local processing of received audio data utilizing a locally stored model to recognize an invocation phrase), and/or by suspending transmission of audio data to remote automated assistant server(s).

Accordingly, when the responsiveness of the other two devices is limited, the isolated device will be the only one of the three devices that performs one or more functions in response to various spoken utterances. For example, the isolated device can be the only one of the three devices that: actively monitors for an occurrence of the spoken invocation phrase, transmits audio data to remote automated assistant server(s) (e.g., based on a detection of an instance of the spoken invocation phrase), and/or that renders (e.g., audibly and/or graphically) automated assistant output for presentation to the user. In these and other manners, network resources, resources of client device(s), and/or resources of server device(s) are conserved. For example, network resources and/or resources of server device(s) can be conserved as a result of only the isolated device transmitting data packets for processing by the server device(s) when the responsiveness of the other two devices is limited. Also, for example, resources of the non-isolated client devices can be conserved as a result of not performing, when their responsiveness is limited, certain local processing of spoken utterances, not transmitting data packets in response to certain spoken utterances, and/or not receiving and/or rendering any automated assistant output.

Typically, in an environment where a user is proximate to multiple devices, such as a car where multiple devices are present (e.g., a vehicle client device and multiple smart phone client devices), the user may provide a spoken utterance that incidentally invokes multiple devices in the environment. This can be frustrating for the user and/or other people in the vicinity, as their respective devices may simultaneously respond to the spoken utterance, wasting power and computational resources of the devices. In order to ensure that a desired device is isolated for receiving spoken utterances, a user can request that the devices change their settings such that a single device is responsive to certain spoken utterances, and other devices are not.

In some implementations, a remote server device can assist with coordinating the device(s) such that a single primary device will exclusively be responsive to certain spoken utterances, at least for a finite duration of time. For instance, the remote server device can receive data packets indicating that multiple users are simultaneously providing spoken utterances to their respective devices. The remote server device can specifically identify the data packets corresponding to requests for isolating a client device from other client devices for being responsive to spoken utterances. For example, multiple users can provide a spoken utterance such as, "Setup my devices," which can be embodied as audio data in data packets provided to the remote server device. The remote server device can also identify groups of devices that have similar contexts in which the spoken utterance is being provided. For example, the remote server device can determine a relative location, velocity, route, time stamps (e.g., a time of receipt of a spoken utterance), network name, voice signature, proximity to one or more networks or locations, and/or any other features from which a context of a spoken utterance can be gleaned using contextual data provided in data packets. In some implementations, data packets used to determine a context of a spoken utterance can also be used to generate a hash for identifying a token from which a personal identifier (e.g., an account name or email address name) can be determined. The personal identifier can then be used to specifically identify the client devices that received the spoken utterance.

The remote server device can compile one or more personal identifiers into a group or a subset that is associated with the context. For instance, when the user is driving in a vehicle with multiple client devices, a group of client devices corresponding to a single personal identifier can be identified at least based on a location of the group of client devices, a time that the spoken utterance was received at the group of client devices, and/or a velocity at which the vehicle is traveling (e.g., as determined by the remote server device). The remote server device can then generate a unique identifier for each client device in the group of client devices. A unique identifier can be provided as a visual output (e.g., a display graphic), an audible output (e.g., a name output by a client device), and/or any other unique identifier that can be used to identify a device.

In some implementations, the remote server device can either cause all of the client devices or some of the client devices in the group of client devices to present their respective identifier to the user. For example, the remote server device can designate a primary client device of the group of client devices to audibly provide a query to the user regarding selecting one of the client devices to be responsive to subsequent spoken utterances and invocation phrases. The primary client device can then transmit the selection back to the remote server device in order that the remote server device can modify the settings of each client device of the group of client devices accordingly. Alternatively, each of the client devices in the group of client devices can listen for the selection from the user and adjust their individual settings accordingly. For instance, if the user selects the primary client device, the other client devices can acknowledge that they were not selected and adjust their settings to temporarily not be responsive to subsequent spoken utterances and invocation phrases.

In other implementations, which form part of the claimed invention, a process of isolating a client device of a group or a subset of client devices can be performed without necessarily communicating with a remote server device. For example, a user can provide a spoken utterance (i.e., a device isolation utterance) to a group of client devices in their apartment in order to isolate a primary client device for being responsive to spoken utterances. Initially, each client device can receive the spoken utterance and generate a random listening time period in which each client device will listen for a response from another client device. The client device that generated the shortest listening time period can be the first to provide a response to the user, and the other client devices can listen for the first response. In some implementations, each of client devices can generate unique images (e.g., grocery store items such as coffee or produce) or graphics (e.g., random colors) for presenting to the user, in order that the user can identify one of the client devices from the images. When the other client devices acknowledge the first response or present their unique image, the other client devices can listen for the user to identify a primary client device. If the user provides a response identifying a primary client device (e.g., "Isolate my tablet computer" or "Isolate the device showing green"), the other client devices not identified in the response from the user can adjust their settings to be temporarily unresponsive to certain spoken utterances and invocation phrases.

In some implementations, which form part of the claimed invention, a user can isolate a group of devices, which can include third party devices and/or applications. For example, a user can have a variety of devices that include an assistant application and a third party assistant application. The third party assistant application can be associated with an application programming interface (API) that allows the assistant application to interact with the third party assistant application. In this way, the assistant application can be arranged to temporarily limit a responsiveness of third party assistant applications at the direction of the user, without the user having to directly engage with the third party assistant application.

Turning now to the Figures, <FIG> illustrates a system <NUM> for isolating a client device from multiple client devices to receive commands and/or subsequent dialog. The system <NUM> can include a remote device <NUM>, such as a server device <NUM>, which can host an automated assistant <NUM>. In some implementations, the automated assistant <NUM> can be hosted at the server device <NUM>, and/or one or more computing devices capable of hosting an assistant application. For example, in some implementations, the automated assistant <NUM> can be hosted at a cellular computing device <NUM>, a vehicle computing device <NUM> of a vehicle <NUM>, and/or a personal computing device <NUM> such as a tablet computing device <NUM>. Processes and/or functions performed by the automated assistant <NUM> can be performed by the server device <NUM>, or distributed over separate devices. For instance, a user can provide an input, such as a voice command or a textual input, at an assistant interface <NUM> of the cellular computing device <NUM>. The input from the user can be transmitted from the cellular computing device <NUM>, over a network <NUM>, such as a private network (e.g., peer-to-peer), or a public network (e.g., the internet), to the server device <NUM>. When the input is a voice command, or other audible input from the user, the input can be received at a voice to text engine <NUM>, which can be part of the automated assistant <NUM> or separate from the automated assistant <NUM>. The voice to text engine <NUM> can convert the input into text, which can be parsed by the automated assistant <NUM> to determine how the automated assistant <NUM> can sufficiently respond to the input from the user.

Interactions between one or more users and the automated assistant <NUM> can be stored as assistant interaction data <NUM>. The assistant interaction data <NUM> can, for example, provide correspondence data that identifies one or more inputs received from the one or more users and one or more responses generated from the automated assistant <NUM>. The automated assistant <NUM> can employ one or more machine learning algorithms for learning or otherwise adapting to user requests, preferences, and/or how the user otherwise interacts with the automated assistant <NUM>. In some implementations, the server device <NUM> and/or the automated assistant <NUM> can include a client grouping engine <NUM>. The client grouping engine <NUM> can be an application, script, and/or other executable data capable of processing data packets from one or more devices (e.g., devices <NUM>, <NUM>, and/or <NUM>, other server devices) and generating a list of devices that are providing similar requests. Such requests can include a request from a user to isolate a particular device from multiple devices for receiving subsequent commands and/or dialog.

As an example, a user can be navigating their vehicle <NUM> along a highway and inside of the vehicle <NUM> can be multiple different devices. The multiple different devices can include one or more that are owned by, or otherwise associated with, the user - and/or one or more that are each owned by, or otherwise associated with, another corresponding user (e.g., an occupant of the vehicle). The devices can include an integral device such as a vehicle computing device <NUM>, a portable computing device <NUM> such as a cellular computing device <NUM>, and a personal computing device <NUM> such as a tablet computing device <NUM>. While in transit, the user can provide a spoken utterance to an assistant interface <NUM> of the tablet computing device <NUM>. The spoken utterance can include dialog such as, "Assistant, please setup my devices. " The spoken utterance can correspond to a function that is executable by the automated assistant <NUM> for isolating one device of multiple devices to receive commands and dialog, thereby causing the other devices to be temporarily unresponsive to particular commands and dialog.

In response to receiving the spoken utterance, the tablet computing device <NUM> can communicate with the server device <NUM> regarding the spoken utterance. Furthermore, because the server device <NUM> can be in communication with other client devices associated with other users, the server device <NUM> can employ a client grouping engine <NUM> to correlate particular requests. Specifically, the client grouping engine <NUM> can identify client devices that are receiving similar spoken utterances within a particular time period. From the group of client devices that are receiving similar spoken utterances, the client grouping engine <NUM> can optionally further group client devices according to contexts in which the spoken utterances were provided. For example, the client grouping engine <NUM> can identify a group of client devices because they received similar spoken utterances within a particular time period and are located in similar locations. Contextual data, such as vehicle speed, route information, destination, origin, times stamps, network name, and/or any other data can also be used to identify a group of client devices. Such contextual data can be processed to generate a hash or other data that can be used to identify a token from which to identify a single instance of a spoken utterance. For instance, the spoken utterance provided by the user can be received nearly concurrently by the tablet computing device <NUM>, the vehicle computing device <NUM>, and the cellular computing device <NUM>. The server device <NUM> can determine that the spoken utterance corresponds to the single instance of the spoken utterance based on voice data (e.g. a voice signature, tone, pitch), contextual data (e.g., location, vehicle speed), device identifiers, and/or any other data from which a single instance of a spoken utterance can be identified.

When a group of client devices has been identified by the server device <NUM>, the automated assistant <NUM> can acknowledge the group of client devices and designate at least one client device of the group of client devices as a primary client device. The primary client device can be designated for querying the user regarding which client device they would like to isolate from the other client devices in the group for receiving commands and dialog. The primary client device can be identified by the server device <NUM> based on, for example, the assistant interaction data <NUM>, which can identify the device most recently used by the user to perform a particular function (e.g., place a phone call, send a message, search the internet, access media, etc.). Additionally or alternatively, the primary client device can be identified by the server device <NUM> based on a property of the client device, such as how engaged a user would have to be with the client device to respond to the query given the environment they are interacting (e.g., driving a vehicle). For instance, the automated assistant <NUM> can determine a level of engagement for each client device and select the vehicle computing device <NUM> as the primary client device. As yet another example, the automated assistant <NUM> can randomly or pseudo-randomly select the primary client device.

The automated assistant <NUM> can cause the selected primary client device (e.g., the vehicle computing device <NUM>) to provide a query to the user regarding selecting a client device to be isolated from the other client devices. Specifically, the query can be provided at an assistant interface <NUM>, such as a speaker system, and include dialog such as, "Please identify the device you would like to remain interactive. " In response, the user can identify a particular device by its apparent feature(s) or name, such as "my tablet computer," "my cell phone," or "the vehicle computer. " However, in some implementations the server device <NUM> can cause the primary client device and/or the other client devices in the group of client devices to provide a unique identifier. For instance, the server device <NUM> and/or the automated assistant <NUM> can identify the group of client devices, determine that each of the client devices include display devices, and generate unique graphics to be displayed at each display device. The server device <NUM> and/or the automated assistant <NUM> can then cause each client device of the group of client devices to display a unique graphic from which the user can identify a client device. For instance, each user interface of each client device can be caused to display different colored shapes (e.g., a green circle, a blue circle, a red circle, etc.).

In some implementations, the server device <NUM> and/or the automated assistant <NUM> can identify the group of client devices and determine that one or more of the client devices do not include a display device and/or include an audio interface. The server device <NUM> and/or the automated assistant <NUM> can then generate unique device names, which can be audibly presented through an audio interface (e.g., a speaker) of each device. For example, the unique device names can be "first device," "second device," and "third device. " In some implementations, each client device can audibly provide the unique device name that has been assigned to the client device. Alternatively, the unique device names can be "tablet device," "vehicle device," and "cell device. " Furthermore, the primary client device can be caused to recite each of the unique device names (e.g., "Ok, would you like the 'tablet device,' the 'vehicle device,' or the 'cell device,' to remain interactive?"). In some implementations, the unique device name of a client device can be based on a name previously assigned to the client device (e.g., in a device set up process), a manufacturer of the client device, a model of the client device, and/or other feature(s) of the client device.

The user can respond to the primary client device by reciting the unique name and/or describing the graphic that is displayed (e.g., "The device with the circle. ") at a user interface. The primary client device can then transmit a data packet corresponding to the selection to the server device <NUM>. The server device <NUM> can thereafter cause the client devices of the group of client devices that are not identified by the selection (e.g., the green circle) to be temporarily unresponsive to spoken utterances, dialog, spoken invocation phrases, and/or any other inputs that can be provided from the user. In some implementations, the unselected client devices can be temporarily unresponsive until the contextual data provided by one or more of the client devices to the server device <NUM> indicate a change in context (e.g., the user is no longer driving, in a different location, has participated in a calendar event that is over, etc.). In some implementations, a user can transition an unresponsive client device out of an unresponsive state via certain user interface input, such as interaction with a hardware button of the client device, interaction with a touchscreen of the client device (e.g., to select a particular virtual element), etc..

In some implementations, which form part of the claimed invention, when the primary client device provides the query regarding the client device that will be isolated, the other non-primary client devices can listen for a response from the user. Further, each client device can be aware of their unique name and the unique name of the other client devices. Each client device can listen for a response from the user, and if the user does not identify a respective client device, the respective client device can become temporarily unresponsive without necessarily communicating the selection to the server device <NUM>. Furthermore, the client device that is selected can remain interactive until the user otherwise modifies a setting of the selected client device.

<FIG> illustrates a perspective view <NUM> of a user <NUM> interacting with an automated assistant in order to perform an isolation process to isolate a client device without necessarily communicating with a server device. The user <NUM> can possess multiple devices that are capable of connecting over a common network without necessarily being directly connected to each other. For example, the user <NUM> can own a tablet computing device <NUM>, a television <NUM>, and a cellular computing device <NUM>. Each of these devices can be connected to a local network (e.g., a Wi-Fi network), which can allow them to interface with an automated assistant over the internet. However, the devices can also perform functions associated with the automated assistant without necessarily communicating over the internet. For example, the user <NUM> can provide spoken utterances to the devices to cause them to change their operating modes and/or access particular applications installed in the devices. Problems can occur when the user <NUM> issues a spoken utterance that causes the devices to be simultaneously responsive. This can cause the devices to waste power and computational resources by attempting to respond to a spoken utterance that may have been intended for a single device.

In order to isolate a single device from the multiple devices for receiving subsequent spoken utterances and dialog, the user <NUM> can provide a command <NUM> to one or more of the devices regarding setting up the devices. For example, the user <NUM> can provide a command <NUM> that includes the spoken utterance "Assistant, please setup my devices. " The command <NUM> can be heard by each of the tablet computing device <NUM>, the cellular computing device <NUM>, and the television <NUM>. However, each of the devices can be configured to not simultaneously respond, but rather listen, using an audio interface of each device, to determine whether one of the devices has responded. Specifically, each of the devices can generate a random value corresponding to delay period for responding to the command <NUM> from the user. For example, the tablet computing device <NUM> can generate a <NUM> second delay, the cellular computing device <NUM> can generate a <NUM> second delay, and the television <NUM> can generate a <NUM> second delay. During the delay period, each respective device will not provide a response to the command <NUM>, thereby causing the device having the shortest delay period to provide the first response to the user <NUM>. For instance, because the tablet computing device <NUM> generated the shortest value for the delay period (e.g., <NUM> seconds), the tablet computing device <NUM> will provide a response <NUM> before any other device.

In response to hearing that the tablet computing device <NUM> provided the response <NUM> (e.g., "Ok, which device would you like to isolate?"), each of the devices can present a unique identifier to the user <NUM> and listen for a subsequent command <NUM> from the user <NUM>. For instance, each user interface of each device can generate a unique display graphic and/or audible sound to present to the user <NUM> in order that the user can identify a particular device to isolate. The display graphic can include a color, shape, image, video, and/or any other visual content that can be presented at a display device. The audible sound can include a device name, network name, personal identifier, random word, random phrase, device descriptor, and/or any other sound that can be used to identify a device.

As provided in <FIG>, each client device can present a unique graphical shape in order that the user <NUM> can identify a particular device from a displayed graphical shape. The tablet computing device <NUM> can display a triangle, the cellular computing device <NUM> can display a circle, and the television <NUM> can display a square. In some implementations, each device will not be put on notice of the identifiers being displayed by other devices, whereas in other implementations each device can be put on notice of the identifiers being displayed by other devices. For instance, when each device is not put on notice of the identifiers being presented by other devices, each device can listen for a response from the user <NUM> to determine whether it was identified. When selecting a device, the user <NUM> can provide a command <NUM> such as, "The device displaying a triangle. " Each device can acknowledge the response and determine the identifier (e.g., "triangle") that was selected by the user <NUM>. Each device can compare the user selected identifier to the identifier that is being displayed by the device in order to determine whether it was selected. For instance, the tablet computing device <NUM> can determine that a triangle was described by the user <NUM> and therefore remain in an operating mode that allows the tablet computing device <NUM> to respond to spoken utterances. Furthermore, the cellular computing device <NUM> and the television <NUM> can determine that they were not identified by the user <NUM>, at least based on the user <NUM> not describing their respective identifiers in the command <NUM> within a response threshold period of time. As a result, each of the cellular computing device <NUM> and the television <NUM> can transition into an operating mode where they are temporarily unresponsive to spoken utterances and/or dialog from the user <NUM> for a period of time. In some implementations, during this period of time, the devices not identified by the user <NUM> can still provide notifications to the user <NUM> and/or respond to certain inputs from the user <NUM> (e.g., touch inputs, peripheral device inputs, remote server inputs, and/or any other input that can control an operation of a computing device).

Subsequent to the user <NUM> selecting the tablet computing device <NUM> to be isolated from the other devices for receiving spoken utterances and/or dialog, the cellular computing device <NUM> and the television <NUM> can listen for commands from the user <NUM>. For example, the user <NUM> can decide to change the device that is isolated subsequent to the user <NUM> identifying the tablet computing device <NUM> as the device to be isolated. For instance, this can be performed in response to someone in the same home of the user <NUM> needing to use the tablet computing device <NUM> (e.g., a child needing to do homework with the tablet computing device <NUM>). In order to identify a different device to be isolated, the user <NUM> can provide a command such as, "Assistant, designate the device that displayed the circle for receiving commands. " In response, each of the devices can determine that the user <NUM> has referenced an identifier previously presented when the user <NUM> was setting up the devices to isolate the tablet computing device <NUM>. Furthermore, each device can determine whether they were subsequently selected to be the isolated device. For instance, an automated assistant accessible to the cellular computing device <NUM> can determine that a circle was described by the user <NUM> and transition from a temporarily unresponsive mode to a responsive mode. Optionally, the device that was previously isolated (e.g., the tablet computing device <NUM>) can transition from the responsive mode to a temporarily unresponsive mode, at least with respect to the voice of the user <NUM>. In other words, the tablet computing device <NUM> can be temporarily unresponsive to certain commands from the user <NUM>, and the cellular computing device <NUM> will be responsive to those certain commands.

<FIG> illustrates a method <NUM> for temporarily isolating a client device from other client devices for receiving commands and/or dialog. The method <NUM> can be performed by one or more server devices, computing devices, and/or any other apparatus capable of processing commands associated with a computing device. The method <NUM> includes an operation <NUM> of receiving data packets that were generated based on a spoken utterance that was provided to multiple client devices that are operating in an environment. The environment can refer to a context, a location, a time, and/or any other descriptor that can be used to define an environment of one or more persons. Furthermore, the data packets can include various types of data that can identify the environment, the spoken utterance, the client devices and/or properties of the multiple client devices, and/or any other types of data that can be associated with a command provided to a client device. For example, the data packets can include data that is based on audio received at the multiple client devices, in order that one or more server devices receiving the data can identify a spoken command (e.g., "Please setup my devices. ") being provided to the multiple client devices. For instance, each data packet can include a segment of audio data captured at a corresponding client device.

The method <NUM> also includes an operation <NUM> of identifying a group or a subset of client devices according to one or more common properties of the data packets. At least one of the common properties can be a spoken utterance indicated by the data packets. The spoken utterance can be received by the multiple client devices substantially simultaneously in the environment (e.g., a car driving along the highway). Furthermore, at least one of the common properties can be associated with location data (e.g., a point along a route and/or a velocity of a device). In this way, the group of client devices can correspond to a group of devices that received the same command while in the same location, as indicated by the data packets. It should be noted that the location data (e.g., velocity, Wi-Fi SSID, GPS data, location services data, coordinates) can be generated at the client device and/or the server device. The location data can thereafter be compared with location data from various other client devices to identify a group of client devices in the same or similar location or context. In some implementations, the data packets can include audio data that can be processed according to a fast Fourier transform process or a machine learning classifier process. The resulting audio data (e.g., audio spectra data and/or audio classification data) can be generated at a client device(s) or the server device and be compared to data accessible to the server device in order to identify the user(s) that provided the spoken command. A group of client devices can then be identified based on their receipt of a spoken utterance from the same user. In these and other manners, operation <NUM> enables analysis of a large quantity of data packets (e.g., hundreds or thousands) received over a time period (e.g., received from hundreds or thousands of client devices in disparate environments) to determine a group of the data packets that all relate to receipt of a single spoken utterance of a single user. The client devices that transmitted those data packets can then be identified as the subset of the client devices that are all in the same environment and that all received the single spoken utterance of the single user.

The method <NUM> further includes a block <NUM> of causing at least one client device of the group of client devices to provide an identifier that is unique relative to other identifiers associated with other client devices in the group of client devices. The identifier(s) can be generated by one or more server devices and provided to at least one client device in the group of client devices. Alternatively, the identifiers can be generated by the group of client devices, and a server device can cause at least one client device of the group of client devices to identify itself and/or other client devices in the group of client devices. For instance, if the user is driving along a highway in a vehicle that includes the group of client devices, one of the client devices can be caused to identify itself for the user to select to be isolated from the other client devices. Alternatively, one of the client devices can be caused to provide identifiers for all client devices in the group of client devices (i.e., all devices in the vehicle) for the user to select from. In another implementation, each of the client devices can be caused to provide an identifier that has been assigned to them such that the user will see and/or hear a series of identifiers from different client devices (e.g., "first device. second device. third device.

The method <NUM> also includes an operation <NUM> of determining whether a selection of a client device from the group of client devices was received. Specifically, the determination can be regarding whether the user provided some user interface input that identified one of the identifiers assigned to each client device of the group of client devices (e.g., spoken user interface input of "I'd like to continue using the first device. If the user does not select a client device, the client devices and/or a server device can wait a reasonable amount of time before each of the client devices in the group of client devices return to their responsive operating modes. If the user has selected a client device, the method <NUM> can proceed to operation <NUM>.

Operation <NUM> of method <NUM> includes causing, based on determining that a client device was selected from the group of client devices, the other client devices of the group of client devices to temporarily limit a responsiveness of at least the other client devices. For instance, temporarily limiting a responsiveness of a client device can include limiting the ability of the client device to respond to particular spoken commands, touch inputs, gestures, and/or any other inputs that can be used to control a computing device. The temporary limit can be applied for a period of time that is based on: a predetermined time period, contextual data provided by one or more client devices, application data available to a server device, instructions provided by the user, calendar data accessible to the client device, and/or any other data that can provide a basis for enforcing a temporary limit on a responsiveness of a client device. By limiting a client device in this way, computational resources and power can be preserved by not requiring multiple devices to simultaneously listen for and/or process inputs from a user. This can also preserve network bandwidth caused by multiple devices transmitting data to a network device in response to simultaneously receiving a command from a user.

<FIG> illustrates a method <NUM> for limiting a responsiveness of a client device to speakable voice commands, at least based on a command provided by a user to isolate a particular client device. The method <NUM> can be performed by one or more server devices, one or more client devices, and/or any computing device capable of modifying response settings. The method <NUM> includes an operation <NUM> of receiving a spoken utterance at a first client device that is operating in an environment with a second client device that also received the spoken utterance. The environment can be a location, such as a house, that includes the first client device and the second client device. The first client device can be, for example, a tablet computing device and the second client device can be an assistant device. The spoken utterance can be a speakable command such as, but not limited to, "Assistant, please setup my devices. " Because each of the client devices can include an audio interface (e.g., a microphone and a speaker), the client devices can receive the spoken utterance substantially simultaneously (e.g., within milliseconds of each other).

The method <NUM> further includes an operation <NUM> of providing a query for requesting a user to identify either one of the first client device or the second client device to be responsive to subsequent spoken utterances received at the first client device or the second client device. Optionally, the first client device can provide the query after waiting an amount of time for a delay period to expire. The query can be, for example, "Ok, identify the device that should remain responsive?" The delay period can correspond to a listening period when the first client device listens, using a microphone, to determine whether another client device has responded to the spoken utterance from the user. If no other device responds within the time allotted by the delay period, the first client device can provide the query to the user. However, if another client device does provide the query to the user, the first client device can acknowledge the response and wait for the user to provide a selecting to the query.

The method <NUM> also includes an operation <NUM> of determining, based on a response to the query, that the second client device was selected to be responsive to the subsequent spoken utterances. The response to the query can be received at the first client and/or the second client device. Furthermore, the response to the query can be, for example, "Please keep the assistant device responsive to commands. " The first client device can compare the device identifier (e.g., "assistant device") provided in the response to one or more identifiers that the first client device is configured to respond to or otherwise associate itself with. If the response from the user identifies the first device as the device to response responsive, the first client device can remain in the same operating mode that the first client device was operating in before receiving the initial spoken utterance. However, as provided in operation <NUM> of method <NUM>, if the first client device is not identified in the response, the first client device can temporarily limit a responsiveness of the first client device to subsequent spoken utterances and invocation phrases. By limiting the responsiveness of the first client device to particular spoken utterances, power and computational resources can be conserved. Furthermore, user interactions between the user and a client device can be uninterrupted by other devices that may be incidentally invoked by spoken commands.

Storage subsystem <NUM> stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem <NUM> may include the logic to perform selected aspects of method <NUM>, method <NUM>, and/or to implement one or more of server device <NUM>, automated assistant <NUM>, cellular computing device <NUM>, tablet computing device <NUM>, vehicle computing device <NUM>, tablet computing device <NUM>, cellular computing device <NUM>, television <NUM>, and/or any other apparatus that perform any operations discussed herein.

Claim 1:
A method implemented by one or more processors, the method comprising:
receiving (<NUM>) an instance of a spoken utterance at a client device that is operating in an
environment with one or more additional client devices that also received the instance of the spoken utterance,
wherein each of the client device and the one or more additional client devices is connected to a local network and includes an assistant application that is responsive to the spoken utterance;
providing (<NUM>), by the client device and based on receiving the instance of the spoken
utterance at the client device, user interface output that provides a prompt, to a user, related to whether the client device is to be responsive to invocations of the assistant application,
wherein each of the one or more additional client devices provides a respective prompt in response to receiving the instance of the spoken utterance, and
wherein the prompt provided at the client device is unique relative to each respective prompt provided at each of the one or more additional client devices;
determining (<NUM>), based on a response of the user to the prompt, that the client device is not to be responsive to invocations of the assistant application, and that a device of the one or more additional devices is to be isolated for being responsive to assistant invocations; and
in response to determining that the client device is not to be responsive to invocations of the assistant application:
causing (<NUM>) the assistant application of the client device to temporarily limit responsiveness to one or more spoken invocations of the assistant application.