Outputting audio based on user location

For outputting audio based on a user's location, an apparatus is disclosed. A system, method, and program product also perform the functions of the apparatus. The apparatus for outputting audio based on a user's location includes a processor and a memory that stores code executable by the processor. The processor receives an audible command to output audio and associates the audible command with a particular user. Additionally, the processor monitors a location of the particular user and output audio, based on the command, from a device nearest a location of the particular user.

FIELD

The subject matter disclosed herein relates to audio output and more particularly relates to outputting audio based on a user's location.

BACKGROUND

Electronic devices providing a digital personal assistant can be configured to send audio output to a user at designated times. However, if a user is in a different location at a designated time, they are unable to hear the audio output.

BRIEF SUMMARY

An apparatus for outputting audio based on a user's location is disclosed. A method and computer program product also perform the functions of the apparatus.

One apparatus for outputting audio based on a user's location includes a processor and a memory. The memory stores code executable by the processor. The processor receives an audible command to output audio and associates the audible command with a particular user. The processor also monitors a location of the particular user and outputs audio, based on the command, from a device nearest a location of the particular user.

In certain embodiments, outputting audio from a device nearest a location of the particular user includes the processor identifying the device nearest a location of the particular user and transferring the audible command to the identified device. In some embodiments, the audible command to output audio is a command to output a notification at a certain time. Here, outputting audio based on the command from a device nearest a location of the particular user includes the processor transferring the command to output the notification at the certain time to the device nearest the location of the particular user.

In certain embodiments, associating the audible command with the particular user includes the processor identifying the particular user from voice analysis of the audible command. In further embodiments, monitoring the location of the particular user includes the processor receiving additional audio data from one or more of a plurality of stationary digital personal assistant devices and determining that the user is located in a particular room of a plurality of rooms based on the additional audio data. In such embodiments, each stationary digital personal assistant device is located in a different one of the plurality of rooms.

In some embodiments, the apparatus includes a camera device that captures image data. In such embodiments, associating the audible command with the particular user includes the processor identifying the particular user from image data in response to receiving the audible command to output audio. In certain embodiments, monitoring the location of the particular user includes the processor receiving image data from one or more of a plurality of stationary digital personal assistant devices and determining that the user is located in a particular room of a plurality of rooms based on the additional image data. In such embodiments, each stationary digital personal assistant device is located in a different one of the plurality of rooms.

One method for outputting audio based on a user's location includes receiving an audible command to output audio and associating, by use of a processor, the audible command with a particular user. The method includes monitoring a location of the particular user and outputting audio based on the command from a device nearest a location of the particular user.

In certain embodiments, associating the audible command with the particular user includes identifying the particular user from voice analysis of the audible command. In some embodiments, receiving the audible command to output audio includes receiving a command to output streaming audio content at a first device. In such embodiments, outputting audio based on the command from a device nearest a location of the particular user may include transferring the command to output streaming audio content to a second device nearest the location of the particular user, the second device being located in a different room than the first device.

In some embodiments, the method also includes capturing image data in response to receiving the audible command to output audio. In such embodiments, associating the audible command with the particular user includes identifying the particular user from image data. Additionally, monitoring the location of the particular user may include determining that the user is located in a particular room of a plurality of rooms based on the image data. In certain embodiments, receiving the image data includes receiving image data from one or more of a plurality of stationary digital personal assistant devices. Here, each stationary digital personal assistant device is located in a different one of the plurality of rooms.

In certain embodiments, monitoring the location of the particular user includes receiving additional audio data and determining that the user is located in a particular room of a plurality of rooms based on the additional audio data. In further embodiments, receiving the additional audio data may include receiving audio data from one or more of a plurality of stationary digital personal assistant devices. In such embodiments, each stationary digital personal assistant device is located in a different one of the plurality of rooms.

In certain embodiments, determining that the user is located in a particular room of a plurality of rooms based on the additional audio data includes detecting the users voice in the audio data of a particular one of the plurality of stationary digital personal assistant devices and determining that the user is located in the room associated with the particular one of the plurality of stationary digital personal assistant devices. In further embodiments, outputting audio from a device nearest a location of the particular user may include instructing a mobile digital personal assistant device associated with the particular user to output the audio in response to determining that the user is not located in any of the plurality of rooms containing a stationary digital personal assistant device.

One program product for outputting audio based on a user's location includes a computer readable storage medium that stores code executable by a processor. Here, the executable code includes code to: receive an audible command to output audio, correlate the audible command with a particular user, track a location of the particular user, and output audio based on the command from a device nearest a location of the particular user.

In certain embodiments, the audible command is received at a first device and the audio is output at a second device located in a different room than the first device. In some embodiments, the audible command to output audio is a command to output a notification at a specified time. In such embodiments, outputting audio based on the command from a device nearest a location of the particular user includes monitoring a current time and outputting the notification at the device nearest the location of the particular user in response to the current time matching the specified time.

DETAILED DESCRIPTION

Any combination of one or more computer readable medium may be utilized. The computer readable medium may be a computer readable storage medium that is not a transitory signal. Said computer readable storage medium may be a storage device storing the code. The storage device may be, for example, but not limited to, an electronic storage device, magnetic storage device, optical storage device, electromagnetic storage device, infrared storage device, holographic storage device, micromechanical storage device, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

The present disclosure describes systems, apparatus, and method for outputting audio based on a user's location. Generally, when a user sets an alarm on one digital personal assistant (“DPA”) device of a set DPA devices, and the user moves to another room, location, etc., then the DPA devices alert the user at the designated time and from a DPA device at the new location the user is now located. For example, if a user sets a wake-up alarm from the kitchen, they want the DPA devices to wake them up in the bedroom the next day.

Accordingly, the set of DPAs track the user's movement between the several DPA devices. Tracking the user's location may be done by listening to the user's voices, analyzing camera data, tracking their wearable devices, etc. In one embodiment, the user may announce their current location to a nearby DPA device once they enter the new area. At the designated time, the DPA device nearest the user will output the alarm.

In some embodiments, the user may be actively listening to audio output (streaming radio) from their DPA device. Here, the active audio output may follow the user around between their multiple DPA devices.

FIG. 1depicts a system100for outputting audio based on a user's location, according to embodiments of the disclosure. The system100includes a first digital personal assistant (“DPA”) device105located a first monitored location125, a second DPA device110located in a second monitored location130, a third DPA device115located in a third monitored location135, and a data network120communicatively coupling the DPA devices105-115. In some embodiments, the system100may optionally include a DPA manager145and/or a mobile DPA150. Each of the locations125-135is a monitored location due to the presence of a corresponding DPA device105-115. While a specific number of DPA devices105-115and monitored locations125-135are depicted, in other embodiments the number of DPA devices and locations may be greater or fewer than those shown inFIG. 1.

Each of the DPA devices105-115is an electronic device hosting a digital personal assistant client. Generally, the DPA devices105-115are stationary in that they are not worn or carried on the person of the user140. In contrast, a mobile DPA device150is a mobile device hosting a digital personal assistant client, such as a smart phone or smart wearable device, that is typically worn or carried on the person of the user140. As discussed in greater detail below, a DPA device105-115may transfer an audible command to a mobile DPA device150when the user140is no longer located within any of the monitored locations125-135.

The DPA devices105-115are each configured to receive an audible command to output audio and associate the audible command with a particular user (e.g., the user140). Various means for identifying a user with which to associate the audible command are discussed below with reference toFIG. 2. Here, the audible command may be a location-tracking command. As used herein, a “location-tracking command” refers to command from the user140to perform an action and “follow” the user140, wherein the DPA devices105-115(or the DPA manager145) are to track the location of the user140and perform the action at a DPA device105-115corresponding to the current location of the user.

Accordingly, the DPA devices105-115also monitor a location of the particular user (user140). In certain embodiments, the DPA devices105-115collaboratively monitor the location of the user140using one or more of audio data, image data, and wireless signal data. Here, each DPA device105-115is capable of determining whether the user140was within a monitored location corresponding to that DPA device. Each DPA device105-115may respond to queries from others of the DPA devices105-115and share data indicative of the user's location. Further, the DPA devices105-115are each configured to identify a DPA device nearest the location of the user140, wherein the nearest DPA device105-15outputs audio based on the command.

The data network120, in one embodiment, is a telecommunications network configured to facilitate electronic communications, for example among the DPA devices105-115. The data network120may be comprised of wired data links, wireless data links, and/or a combination of wired and wireless data links. Examples of wireless data networks include, but are not limited to, a wireless cellular network, a local wireless network, such as a Wi-Fi® network, a Bluetooth® network, a near-field communication (“NFC”) network, an ad hoc network, and/or the like.

In certain embodiments, the data network120forms a local area network (“LAN”) or a wireless local area network (“WLAN”). In one embodiment, DPA devices on the same LAN (or WLAN) are associated with one another. In another embodiment, the DPA devices105-115may form a peer-to-peer network (such as a “workgroup” network) were files, network connections (e.g., an Internet connection), and other computing resources are shared among members of the peer-to-peer network. For example, the data network120may be an ad hoc network formed by the DPA devices105-115.

In some embodiments, the data network120may include a wide area network (“WAN”), a storage area network (“SAN”), a LAN, an optical fiber network, the internet, or other digital communication network. In some embodiments, the data network120may include two or more networks. The data network120may include one or more servers, routers, switches, and/or other networking equipment. The data network120may also include computer readable storage media, such as a hard disk drive, an optical drive, non-volatile memory, random access memory (“RAM”), or the like.

The system100may optionally include a digital personal assistant (“DPA”) manager145that receives an audible command to output audio (e.g., from one of the DPA devices105-115), correlates the audible command with a particular user (e.g., the user140), tracks a location of the particular user (e.g., via the DPA devices105-115), and outputs audio based on the command from a device nearest a location of the particular user (e.g., by identifying the nearest DPA device and instructing that device to output the audio).

In such embodiments, the DPA manager145tracks the location of the user140by receiving data (e.g., audio data, image data, wireless signal data, etc.) and/or receiving user location reports from the various DPA devices105-115. For example, a DPA device105-115may send a user location report in response to an event, such as detecting that the user140enters or leaves the location125-135corresponding to the DPA device105-115. As another example, a DPA device105-115may periodically send a user location report indicating where the user is present or absent in the location125-135corresponding to the DPA device105-115.

From the user location reports, the DPA manager145identifies the current location of the user140(e.g., one of the locations125-135) and further identifies a DPA device corresponding to that location (e.g., one of the DPA devices105-115). The DPA manager145then sends an instruction to identify DPA device to output audio based on the received audible command.

In a first example, the user140may issue an audible command to output a notification at a specified time. Here, the first DPA device105may receive this audible command. As described above, the DPA devices105-115may collaboratively track the location of the user or the DPA manager145may track the location of the user using sensor inputs from the DPA devices105-115. At the specified time, the DPA device105-115nearest the current location of the user140outputs the notification. In certain embodiments, the second DPA device110or the third DPA device115may output the notification to the user140being located in the second location130or the third location135at the specified time.

In a second example, the user140may issue an audible command to output streamed audio content (e.g., internet radio). Here, the first DPA device105may receive this audible command. As described above, the DPA devices105-115may collaboratively track the location of the user or the DPA manager145may track the location of the user using sensor inputs from the DPA devices105-115. As the user moves from the first location1252either the second location130or the third location135, the second DPA device110or the third DPA device115may begin outputting the streamed audio content (and the first DPA device105stop outputting the streamed audio content) in response to the user140being located in the second location130or the third location135.

FIG. 2depicts a digital assistant apparatus200for outputting audio based on a user's location, according to embodiments of the disclosure. The digital assistant apparatus200may be one embodiment of the digital personal assistant device105, described above. Alternatively, the digital assistant apparatus200may be one embodiment of the digital personal assistant manager145, described above. In some embodiments, the digital assistant apparatus200is a user terminal device, such as a DPA device, personal computer, terminal station, laptop computer, desktop computer, tablet computer, smart phone, personal digital assistant, and the like. The digital assistant apparatus200includes a processor205, a memory210, an input device215, an output device230, and communication hardware240.

The processor205, in one embodiment, may comprise any known controller capable of executing computer-readable instructions and/or capable of performing logical operations. For example, the processor205may be a microcontroller, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processing unit, a FPGA, an integrated circuit, or similar controller. In certain embodiments, the processor205may include multiple processing units, such as multiple processing cores, multiple CPUs, multiple microcontrollers, or the like. In some embodiments, the processor205executes instructions stored in the memory210to perform the methods and routines described herein. The processor205is communicatively coupled to the memory210, input device215, output device230, and communication hardware240.

In some embodiments, the processor205receives an audible command to output audio and associates the audible command with a particular user. The processor205also monitors (e.g., tracks) the location of the particular user and outputs audio, based on the command, from a device nearest a location of the particular user.

Receiving the audible command may include the processor205receiving data corresponding to the audible command. In one embodiment, the data is audio data received from an audio input device, such as a microphone220communicatively coupled to the processor205. In another embodiment, the data is a digital command containing one or more parameters derived from the audible command. Here, the audible command may be received and processed by another device and then sent to the processor205. Accordingly, receiving the audible command may include the processor205receiving a predefined command from a networked device.

In order to associate the audible command with a particular user, the processor205identifies the user giving the audible command. In certain embodiments, the processor205associates the audible command with a particular user by analyzing the audible command (e.g., performing voice analysis) to identify the user. In one embodiment, the user may identify herself by name in the audible command. Here, the processor205correlates the spoken name with a particular user.

In some embodiments, the processor205analyzes image data in response to receiving the audible command in order to identify the user giving the command. For example, the processor205may control the camera225to capture image data in response to receiving the audible command and may analyze the capture data to identify the user. In further embodiments, processor205may analyze wireless signal data (e.g., RFID signals) to identify the user. Where the audible command is received and processed by another device, the processor205may parse the command received from another device for parameter identifying user. In other embodiments, the command received from the other device may include image data and/or voice data usable by the processor205to identify the user issuing the command.

In order to monitor/track the location of the particular user, the processor205identifies a current location of the user. In various embodiments, the location of the user is defined with respect to one or more rooms. For example, the digital assistant apparatus200may be located in a particular room of a house, office, or other structure. The processor205may periodically verify that the user is in the same room as the digital assistant apparatus200. As discussed in more detail below, the processor205may use audio data, image data, and/or wireless signaling to determine a room in which the users located. Upon determining that the user is no longer located in the same room as the digital assistant apparatus200, the processor205determines a new current location of the user, also as discussed below.

In some embodiments, the processor205determines the current location of the user by analyzing audio data and/or image data. For example, the processor205may receive audio data (e.g., in addition to the audio data corresponding to the audible command) and analyze the audio data to determine whether the user is in the same room as the digital assistant apparatus200. The audio data may be sampled periodically or may be received continuously. Here, voice analysis may be performed on the audio data and the processor205determines the likelihood that the user is in the same room as the digital assistant apparatus200.

Where the user's voice is detected within the audio data, then it is more likely that the user is in the same room as the digital assistant apparatus200. However, if the user's voice is not detected within the audio data (e.g., for a predetermined period of time), then it is less likely that the user is in the same room as the digital assistant apparatus200. Once the likelihood that the user is in the same room as the digital assistant apparatus200drops below a certain value (e.g., for a predetermined period of time), then the processor205determines that the user is no longer in the same room as the digital assistant apparatus200.

As another example, the processor205may receive image data and analyze the image data to determine whether the user is in the same room as the digital assistant apparatus200. The received image data may be in addition to any image data used to identify the user issuing the audible command. In certain embodiments, the image data may be sampled periodically, such as every second, every 5 seconds, every 10 seconds, etc. In other embodiments, the image data may be continuously received and analyzed. Here, image analysis, such as face detection and recognition, may be performed on the image data. Where the user is not detected within the image data (e.g., for a predetermined period of time), then the processor205may determine that the user is no longer in the same room as the digital assistant apparatus200.

In some embodiments, the processor205determines the current location of the user by receiving wireless signals from a device associated with the particular user, such as an electronic device worn or carried by the user. Examples of wireless signals usable to locate the user include, but are not limited to, radiofrequency signals, infrared signals, ultrasonic signals, and the like. As an example, the digital assistant apparatus200may include an RFID reader and a particular user may wear/carry an RFID tag, such that the processor205tracks the location of the particular user using the RFID tag.

As another example, the processor205may track the location of the particular user by detecting the presence of a mobile device worn/carried by the user via the wireless signals. Here, the mobile device may use Bluetooth, Wi-Fi, or other short- or medium-range wireless communication to connect to the digital assistant apparatus200, wherein the processor205detects the presence of the mobile device through the establishment of the wireless connection. In yet another example, the processor205may listen for (e.g., detect) a predetermined wireless signal from the mobile device, such as a response to a poll from the digital assistant apparatus200. Where the user is not detected via the wireless signals (e.g., for a predetermined period of time), then the processor205may determine that the user is no longer in the same room as the digital assistant apparatus200.

The processor205acquires additional data when determined that the user is no longer in the same room as the digital assistant apparatus200(or responsive to the likelihood of the user being in the same room reaching a threshold value), in order to identify a current location of the particular user. In certain embodiments, the processor205acquires additional data from additional sensors of the digital assistant apparatus200(or communicatively coupled to the digital assistant apparatus200). For example, where the processor205determines that the users no longer in the same room based on audio data received from an audio sensor (e.g., the microphone220), then the processor205may acquire image data and/or data from wireless sensors (e.g., monitoring for wireless signals indicating the presence of the user). In this manner, the processor205may verify that the user is not in the same room by using the additional data from the additional sensors of the digital assistant apparatus200.

In some embodiments, the processor205acquires additional data from one or more network devices in response to determining that the user is no longer in the same room. As discussed above, the digital assistant apparatus200may be networked with one or more digital personal assistant (“DPA”) devices located in other rooms. Thus, when the processor205determines users no longer in the same room as the digital assistant apparatus200, the processor205may query other DPA devices located on the same local area network or associated with the same user account for additional data indicative of the location of the particular user.

In one embodiment, the processor205receives audio data from one or more (stationary) DPA devices, with each stationary DPA device is located in a different one of the plurality of rooms. The processor205may then determine that the user is located in a particular room based on the additional audio data. Where audio data from multiple devices indicates the presence of particular user (e.g., where multiple DPA devices detect the user's voice), the processor205may request the type of data (e.g., image data) to determine the user's location or may analyze volume levels (e.g., audio signal strength) and associated the user's location with a DPA device detecting the highest volume levels.

In another embodiment, the processor205receives image data from one or more DPA devices (each associated with a particular room) and determines that the user is located in a particular room based on the additional image data. In still another embodiment, the processor205may receive wireless signal data from one or more DPA devices. Again, the processor205determines a room in which the user is located based on the wireless signal data. Where multiple DPA devices detect wireless signals from the user, the processor205may request another type of data (e.g., image data) or may analyze wireless signal strengths to determine the user's location.

In certain embodiments, the processor205queries each networked device (e.g., DPA device), asking whether the particular user is located in the room associated with that network device. In such embodiments, the queried device will analyze audio data, image data, and/or wireless signal data to determine whether the particular user is located in the room associated with that device. The queried device will then respond to the processor205with an indication (e.g., positive or negative) of whether the particular user is present. This approach distributes the burden of identifying the user's current location to the plurality of network devices.

Still further, the processor205may transmit a user location report to one or more networked DPA devices. In one embodiment, the user location report indicates whether the user is present or absent within an area monitored by the digital assistant apparatus200. In another embodiment, the user location report indicates that the user has entered or left the area monitored by the digital assistant apparatus200.

In some embodiments, the processor205is unable to determine a current location of the user. For example, the user may no longer be within range of any of the networked DPA devices. Where the processor205is unable to determine the current location, it may set the user's current location as “unknown,” with certain behaviors triggered by the “unknown” user location.

As will be apparent to one of skill in the art, the processor205may use a combination of the above approaches to track/monitor the location of the particular user. For example, the processor205may be constantly monitoring audio data in order to receive and perform a command from a user. Here, the processor205may use the audio data to monitor the user's location and may supplement audio data with image data, wireless signal data, and the like. Such an approach may prove the accuracy of the user's current location.

In order to output audio from the nearest device, the processor205determines which device (of a plurality of devices) is closest to the tracked user. In certain embodiments, outputting audio from a device nearest a location of the particular user includes the processor205identifying the device nearest a location of the particular user and transferring the audible command to the identified device. For example, the processor205may be part of a first DPA device associated with the first room and may determine that the particular user is currently in the second room. The processor205may then identify a second DPA device associated with the second room and transfer the audible command to the second DPA device.

In one embodiment, the audible command is command to play audio (e.g., music) streamed from a network location. Here, the streamed audio should be played only in the room where the users located. Thus, when a particular user moves from the first room to the second room, the processor205transfers the command to stream audio such that the streamed audio ceases playing from the first DPA device and begins playing from the second DPA device. In another embodiment, the audible command is command to provide a notification at a certain time or in response to certain condition. Here, the notification should be presented only from advice near the user. Thus, when the particular user moves from the first room to the second room, the processor205transfers the notification command to the second DPA device such that when the time (or the condition) associated with the notification is met, the second DPA device (located in the same room as the user) is the only device presenting the notification.

The memory210, in one embodiment, is a computer readable storage medium. In some embodiments, the memory210includes volatile computer storage media. For example, the memory210may include a random-access memory (RAM), including dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), and/or static RAM (SRAM). In some embodiments, the memory210includes non-volatile computer storage media. For example, the memory210may include a hard disk drive, a flash memory, or any other suitable non-volatile computer storage device. In some embodiments, the memory210includes both volatile and non-volatile computer storage media.

In some embodiments, the memory210stores data relating to outputting audio based on a user's location. For example, the memory210may store audible commands, voice prints, image data, user profiles, the current user location, identities of networked devices, and the like. In some embodiments, the memory210also stores executable code and related data, such as an operating system or other controller algorithms operating on the digital assistant apparatus200.

The input device215, in one embodiment, may comprise any known computer input device including a touch panel, a button, a keyboard, a microphone, a camera, and the like. For example, the input device215includes a microphone220or similar audio input device with which a user inputs audio data (e.g., audible commands). In some embodiments, the input device215may include a camera225, or other imaging device, that captures image data. As described above, the image data may be used to associate an audible command with a particular user and/or to determine whether the particular user is located in a particular room. In some embodiments, the input device215comprises two or more different devices, such as the microphone220and a button.

The output device230, in one embodiment, is configured to output visual, audible, and/or haptic signals. In some embodiments, the output device230includes an electronic display capable of outputting visual data to a user. For example, the output device230may include an LCD display, an LED display, an OLED display, a projector, or similar display device capable of outputting images, text, or the like to a user. In certain embodiments, the output device230includes one or more speakers235for producing sound, such as an audible alert/notification or streamed audio content. In some embodiments, the output device230includes one or more haptic devices for producing vibrations, motion, or other haptic output.

In some embodiments, all or portions of the output device230may be integrated with the input device215. For example, the input device215and output device230may form a touchscreen or similar touch-sensitive display. As another example, the input device215and output device230may form a display that includes haptic response mechanisms. In other embodiments, the output device230may be located near the input device215. For example, a camera225, microphone220, speaker235, and touchscreen may all be located on a common surface of the digital assistant apparatus200. The output device230may receive instructions and/or data for output from the processor205and/or the communication hardware240.

The communication hardware240, in one embodiment, is configured to send and to receive electronic communications. The communication hardware240may communicate using wired and/or wireless communication links. Further, the communication hardware240may be used to receive wireless signal data. In some embodiments, the communication hardware enables the digital assistant apparatus200to communicate via the data network120. The digital assistant apparatus200may also include communication firmware or software, including drivers, protocol stacks, and the like, for sending/receiving the electronic communications.

In certain embodiments, the communication hardware240includes a wireless transceiver capable of exchanging information via electromagnetic radiation (e.g., communication via radio frequencies, infrared, visible light, and the like) or sound (e.g., ultrasonic communication). In one embodiment, the wireless transceiver acts as a wireless sensor for detecting wireless signals. As discussed above, the communication hardware240, in conjunction with the processor205, may be used to identify the particular user and/or detect location of the particular user by receiving wireless signals, such as radio signals, ultrasonic signals, infrared signals, and the like. In certain embodiments, the communication hardware240includes a wired transceiver capable of exchanging information via a transmission line (e.g., wire, data cable, or optical fiber).

FIG. 3depicts a procedure300for outputting audio based on a user's location, according to embodiments of the disclosure. The procedure300involves a first digital personal assistant310at a first location, a second digital personal assistant325at a second location320, and a user315. The first digital personal assistant310and the second digital personal assistant325are digital personal assistant (“DPA”) devices and may be embodiments of the digital assistant apparatus200, described above. Further, the first digital personal assistant310and the second digital personal assistant325are communicatively coupled to each other, for example via a local area network.

As depicted, the user315is initially at the first location305. Here, the user315interacts with first digital personal assistant310, instructing the first digital personal assistant310to stream audio. As the first digital personal assistant310is listening for audible commands, the first digital personal assistant310receives the audible command (see action330). Upon receiving the audible command, the first digital personal assistant310identifies the user issuing the audio command (see action335). Further, the first digital personal assistant310processes the audible command, resulting in the first digital personal assistant310outputting audio pursuant to the command to “stream audio” (see action340).

Additionally, the first digital personal assistant310tracks the location of the user315(see action345). As described above, a digital personal assistant (such as the first digital personal assistant310) may track the location of the user using audio data, image data, wireless signal data, and the like. At some point in time, the user315moves to a new location (see action350). By tracking the location of the user315, the first digital personal assistant310determines that the user315is no longer at the first location305and that the first digital personal assistant310is no longer the closest DPA device to the current location of the user315(see action355).

The first digital personal assistant310identifies a DPA device closest to the users and transfers the audible command to identify device (see action360). In certain embodiments, the first digital personal assistant310stops streaming audio in response to transferring the audible command to the new DPA device. In the depicted embodiment, the first digital personal assistant310determines that the second digital personal assistant325is the closest DPA device.

Accordingly, the second digital personal assistant325receives the transferred command from the first digital personal assistant310. As the audible command is to “stream audio,” the second digital personal assistant325(as the DPA device closest to the user's current location) outputs audio based on audible command (see action365). The second digital personal assistant325also tracks the location of the user315.

Although not shown inFIG. 3, the user315may travel to one or more additional locations. In one embodiment, the user315may travel to an area (e.g., room) associated with a third DPA device. Here, the second digital personal assistant325transfers the audible command to their DPA device. In another embodiment, the user315may travel to an area that is not associated with any DPA device for example, the user315may travel to room where no digital system device is located. In such an embodiment, the second digital personal assistant325may cease streaming audio upon determining that the user315is absent from the second location320or has been absent from the second location320for a predetermined amount of time. In a further embodiment, the second digital personal assistant325may periodically attempt to locate the user315and may either stream audio again or transfer the audible command to another device in response to locating the user315.

WhileFIG. 3is described in terms of the first digital personal assistant310and second digital personal assistant325identifying the user issuing the audible command, tracking the location of the user, and determining a closest device to the user's current location, in other embodiments a digital personal assistant manager (such as the DPA manager145) magnify the user issuing audible command, track the location of the user, and determine a closest device to the user's current location.

FIG. 4depicts a floor plan of a home400for outputting audio based on a user's location, according to embodiments of the disclosure. As depicted, the home400includes a plurality of rooms, including monitored rooms410A-E and unmonitored rooms411. Each of the monitored rooms410A-E (collectively referred to as “monitored rooms410”) includes one of the digital personal assistant (“DPA”) devices405A-E (collectively referred to as “DPA devices405”). Note that some rooms of the home400are not monitored because no DPA device405is located therein. Each DPA device405listens for audible commands and/or monitors the location of the user401. The DPA devices405may be embodiments of the digital assistant apparatus200, described above. Further, the DPA devices405are communicatively coupled to one another, for example via a local area network or peer-to-peer network.

Initially, the user401is located in the monitored room410C. While in the monitored room410C, the user401issues and audible command which is received (and processed) by the DPA device405C located in monitored room410C. As an example, the audible command may be an instruction to output a notification (e.g., alarm) at a specific time. In some embodiments, the DPA device405C issues a copy of the audible command to the other DPA devices405in the home400with an indication to perform the audible command whenever the user401enters a monitored room410associated with that DPA device405. In other embodiments, the DPA devices405collaboratively track the location of the user401and the DPA device405C transfers the audible command to an appropriate DPA device405in response to the user401entering a monitored room410associated with that DPA device405.

For example, the user401may enter a kitchen/dining area corresponding to the monitored room410B where the DPA device405B is also located. In this example, the DPA device405B may detect the presence of the user401and inform the DPA device405C. Here, the DPA device405C stops performing the audible command and (optionally) transfers the audible command to the DPA device405B (e.g., the DPA device405closest to the location of the user401). Accordingly, the DPA device405B begins performing the audible command in response to detecting the presence of the user401and/or in response to the DPA device405C transferring the audible command.

If the user401returns to the monitored room410C, the above steps are reversed with the DPA device405B stopping performance of the audible command and (optionally) transferring the audible command back to the digital system device405B. In one embodiment, the user401may announce their presence in a monitored room410upon entering said room.

As another example, the user401may enter a bedroom area corresponding to the monitored room410D. In certain embodiments, some bedrooms corresponding to monitored rooms410may not be associated with the user401. For example, certain DPA devices405within the home400may be associated exclusively with a particular user. Thus, when the user401enters their own bedroom, a DPA device405in that bedroom may accept transfer of an audible command.

However, should the user401enter another bedroom, a DPA device405in that bedroom may not be associated with the user401and therefore does not accept transfer of the audible command. Still further, a DPA device405in a bedroom not belonging to the user401(and is not associated with the user401) may accept a transfer of certain types audible commands, such as a request to output alarm/notification at a certain time, but not accept a transfer of other types of audible commands, such as a request to stream audio content (e.g., internet radio). In the depicted embodiment, the DPA device405D accepts transfer of audible commands associated with the user401, at the DPA device405E does not accept transfer of auto complete commands associated with the user401.

In certain embodiments, the user401may be associated with a mobile device that can accept transfer of at least some types of audible commands. For example, a mobile phone carried by the user401may be configured to accept transfer of an alarm/notification command. Accordingly, the user401may travel from the monitored room410C to an unmonitored room411of the home400. In response to the networked DPA devices405being unable to locate the user401(e.g., due to the user401not being located in a monitored room410), the DPA device405C may transfer the audible command (e.g., alarm/notification command) to the mobile phone. If the networked DPA devices405later locate the user401(e.g., due to the user401entering a monitored room410), then a DPA device405may send a message the mobile phone revoking the audible command.

FIG. 5depicts a data structure for an audible command data500, according to embodiments of the disclosure. The audible command data500may be generated by a digital personal assistant (“DPA”) device or DPA manager upon receiving an audible command from a user. In certain embodiments, the DPA device/manager generates the audible command data500in response to determining that the auto command is a “location-tracking” command where the command is to “follow” the user as he/she travels to various locations. The audible command data500includes various data fields including, but not limited to, a user field505, a current user location field510, and a command data field515.

The user field505contains a parameter identifying the particular user associated with the audible command data500. As discussed above, a DPA device/manager may use audio data, image data, wireless signal data, and other types of data to identify a particular user issuing the audible command. The current user location field510contains a parameter indicating the present location of the user identified in the user field505. The DPA device/manager updates the current user location field510whenever the identified user travels to a new (monitored) location. In certain embodiments, a first DPA device corresponding to an initial location of the user transfers the audible command data500to a second DPA device corresponding to a new location of the user in response the user traveling to the new location.

The command data field515includes various parameters describing the actions to be performed in response to the audible command. Accordingly, the command data field515may include a command type parameter520and/or one or more command condition parameters525. Examples of values of the command type parameter520include, but are not limited to, an indication to output an alert/notification and an indication stream audio content. The one or more command condition parameters525are used to further define the actions to be performed. For example, where the command type parameter520indicates an alert/notification, the command condition parameters525may indicate a certain time at which the alert/notification is to be output or other conditions to be satisfied before outputting the alert/notification. As another example, where the command type parameter520indicate streaming audio output, the command condition parameters525may indicate a streaming source, a duration (e.g., limit) for streaming the audio content, or other conditions relating to streaming the audio content.

In certain embodiments, the command condition parameters525may include exceptions, such as monitored rooms in which the audio is not to be output. In some embodiments, the command condition parameters525may specify one or more default actions to be performed if the user's location cannot be determined. Additionally, a command condition parameter may be used indicate that the user's location is to be tracked and the audible command data500transferred to a nearest DPA device.

FIG. 6depicts a method600for outputting audio based on a user's location, according to embodiments of the disclosure. In certain embodiments, the method600is performed by one of the DPA devices105-115, by the DPA manager145, by the digital assistant apparatus200, by the first digital personal assistant310, by the second digital personal assistant325, and/or by a DPA device405. Alternatively, the method600may be performed by a processor and a computer readable storage medium that is not a transitory signal. Here, computer readable storage medium stores code that is executed on the processor to perform the functions of the method600.

The method600begins and receives605an audible command to output audio. In one embodiment, the audible command is a command to output a notification at a certain time. In another embodiment, the audible command is a command to output streaming audio content. In certain embodiments, receiving605the audible command to output audio may trigger the capturing of image and/or video data, as discussed below.

The method600includes associating610the audible command with a particular user. In some embodiments, associating610the audible command with the particular user includes identifying the particular user from voice analysis of the audible command. In certain embodiments, associating610the audible command with the particular user includes capturing image data and identifying the particular user from image data. In other embodiments, associating610the audible command with a particular user includes detecting the presence of an electronic device associated with the user, such as a mobile phone, a wearable device, an RFID tag, and the like.

The method600includes monitoring615a location of the particular user. In some embodiments, monitoring615the location of the particular user includes receiving additional audio data and determining whether the user is located in a particular room of a plurality of rooms based on the additional audio data. In such embodiments, receiving the additional audio data may include receiving audio data from one or more of a plurality of stationary digital personal assistant (“DPA”) devices, each stationary DPA device being located in a different one of the plurality of rooms. Here, determining that the user is located in a particular one of the plurality of rooms may include detecting the user's voice in the audio data of a particular one of the plurality of stationary DPA devices and determining that the user is located in the room associated with that stationary DPA device.

In some embodiments, monitoring615the location of the particular user includes capturing image data and determining that the user is located in a particular room of a plurality of rooms based on the image data. In such embodiments, receiving the image data may include receiving image data from one or more of the plurality of stationary DPA devices, each stationary DPA device being located in a different one of the plurality of rooms. Still further, monitoring615the location of the particular user may include receiving wireless signal data (e.g., from onboard sensors and/or one or more DPA devices) and determining the user's location from the wireless signal data.

The method600includes outputting620audio based on the command from a device nearest a location of the particular user and the method600ends. In some embodiments, outputting620audio from a device nearest a location of the particular user includes instructing a mobile DPA device associated with the particular user to output the audio in response to determining that the user is not located in any of the plurality of rooms containing a stationary DPA device. In certain embodiments, outputting620audio from a device nearest a location of the particular user may include transferring the audible command to the DPA device nearest the user.

For example, receiving605the audible command to output audio may include receiving a command to output streaming audio content at a first device. Here, outputting620audio based on the command from a device nearest a location of the particular user includes transferring the streaming command to a second device nearest the location of the particular user, the second device being located in a different room than the first device.

As another example, receiving605the audible command to output audio may include receiving a command to output a notification at a certain time. Here, outputting620audio based on the command from a device nearest a location of the particular user may include transferring the notification command to a second device nearest the location of the particular user, the second device being located in a different room than the first device.

FIG. 7depicts a method700for outputting audio based on a user's location, according to embodiments of the disclosure. In certain embodiments, the method700is performed by one of the DPA devices105-115, by the DPA manager145, by the digital assistant apparatus200, by the first digital personal assistant310, by the second digital personal assistant325, and/or by a DPA device405. Alternatively, the method700may be performed by a processor and a computer readable storage medium that is not a transitory signal. Here, computer readable storage medium stores code that is executed on the processor to perform the functions of the method700.

The method700begins and receives705an audible command to output audio. As described above with reference toFIG. 6, the audible command may be a command to output a notification at a certain time, a command to output streaming audio content, or the like. In certain embodiments, receiving705the audible command to output audio may trigger the capturing of image and/or video data, as discussed below.

The method700includes associating710the audible command with a particular user. As described above with reference toFIG. 6, associating710the audible command with the particular user may include identifying the particular user by analyzing a voice in the auto come command, by analyzing image data capture in response the article command, sensing the presence of an electronic device associated with a particular user, and the like.

The method700optionally includes outputting715audio based on the command. For example, a spoken acknowledgment may be output715in response to receiving the audible command. As another example, streaming audio content to be output715in response to receiving a command to output streaming audio. Accordingly, the method700includes outputting705audio based on the command, when appropriate.

The method700includes identifying720a current location of the particular user. As described above with reference toFIG. 6, identifying720a current location of the particular user may include receiving additional audio data, image data, and/or wireless signal data (e.g., from one or more DPA devices) and determining the current location from the additional data. The method700further includes identifying a device (e.g., a DPA device) nearest the current location of the particular user.

The method700includes determining730whether another device is closer to the user. Here it is assumed that the device receiving the audible command is initially the device closest to the user. In response to determining730that another device is closer to the user, then the method700includes transferring735the audible command to the identified device and the method700ends. In one embodiment, audible command data500is transferred to the identified device.

However, in response to determining730that another device is not closer to the user, then the method700proceeds to output715audio based on the command (e.g., where applicable based on the nature and conditions of audible command), identifies720the current location of the particular user, identify725a device nearest to the current location of the particular user, and again determine730whether another device is closer to the user.