Patent Description:
As users begin to acquire more Internet-of-Things ("IOT") devices, some of them will include functionality to allow a user to engage in audio or video communications. For example, a user may engage in traditional cellular or wired phone calls by connecting an IOT device to the user's smartphone or home phone and communicating via the connected IOT device. In a scenario where multiple IOT devices are suitable for providing such communications, when an incoming phone call is detected at the user's smartphone, for example, the smartphone can notify each of these IOT devices of the incoming call, thereby causing them to ring to notify the user of the incoming call.

<CIT> discloses a method for managing an incoming call includes sending a first predetermined voice message to a caller when a fixed-line telephone receives the incoming call from the caller, capturing an image of a predetermined scene range of the fixed-line telephone when the incoming call is determined to be calling for the owner of the fixed-line telephone, sending prompt information to at least one mobile device when the owner of the fixed-line telephone is not detected, performing a corresponding function according to a selected operation of an owner of the at least one mobile device.

<CIT> discloses that a user may have multiple devices that are associate with a single user address such as an email address, a telephone number, a user name, a user ID, etc. When a communication is addressed to the single user address, each of the devices is queried to determine its location, and any devices that are co-located within a common geographic area are identified. For those devices within the common geographic area, one of the devices is selected and instructed to play an audible alert. The other devices within the common geographic area are not instructed or are specifically instructed not to play the audible alert. All of the user's multiple devices may present a visual alert.

<CIT> discusses routing telecommunications to a user in dependence upon location. <CIT> discloses an apparatus for automatically alerting a user of a handheld communications device of an incoming transmission.

Preferred embodiments of the invention are stipulated in the dependent claims. While several embodiments and/or examples have been disclosed in the description, the subject matter for which protection is sought is limited by the scope of the appended claims. Embodiments and/or examples that do not fall under the scope of the claims are useful for understanding the invention. The systems, methods and devices of this disclosure each have several innovative aspects, no single one of which is solely responsible for the desirable attributes disclosed herein.

Various examples are described for systems and methods for intelligent routing of notifications of incoming voice communication requests. A first innovative aspect of the invention, as defined by independent claim <NUM>, provides for a method performed by an electronic device, the method comprising: receiving an indication of an incoming voice communication request; identifying one or more devices capable of communicating with the electronic device via a connection, each of the one or more devices including a sensor capable of obtaining one or more audio signals or one or more image signals; requesting one or more of the devices to obtain an audio signal or an image signal via its respective sensor; obtaining the audio signal or the image signal obtained by the one or more devices, wherein the audio signal or the image signal is used for recognizing a person, wherein a recognized person is a person known to the electronic device based on maintained information about the person; determining an associated priority for each of the one or more devices based on the recognized persons; based on at least the associated priority, selecting a device of the one or more devices; and providing a signal to the selected device to output a notification indicating the incoming voice communication request.

A second innovative aspect of the invention, as defined by independent claim <NUM>, provides for an electronic device device, comprising: means for receiving an indication of an incoming voice communication request; means for identifying one or more devices capable of communicating with the electronic device via a connection, each of the one or more devices including a sensor capable of obtaining one or more audio signals or one or more image signals; means for requesting, the one or more devices, to obtain an audio signal or an image signal via its respective sensor; means for obtaining the audio signal or the image signal obtained by the one or more devices, wherein the audio signal or the image signal is used for recognizing a person, wherein a recognized person is a person known to the electronic device based on maintained information about the person; means for determining an associated priority for each of the one or more devices based on the recognized persons; means for selecting a device of the one or more devices based on at least the associated priority; and means for providing a signal to the selected device to output a notification indicating the incoming voice communication request.

A third innovative aspect of the invention, as defined by independent claim <NUM>, provides for a computer-program comprising instructions , which , when executed by a possessor, cause the processor to perform the method of the first aspect of the invention. Further embodiments are set out in the dependent claims.

The following description is directed to certain implementations for the purposes of describing the innovative aspects of this disclosure. However, a person having ordinary skill in the art will readily recognize that the teachings herein can be applied in a multitude of different ways, provided that these different ways fall within the scope of the invention which is defined by the appended claims. The described implementations may be implemented in any device, system or network that is capable of transmitting and receiving RF signals according to any of the IEEE <NUM> standards, or any of the IEEE <NUM> standards, the Bluetooth® standard, code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), Global System for Mobile communications (GSM), GSM/General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunked Radio (TETRA), Wideband-CDMA (W-CDMA), Evolution Data Optimized (EV-DO), 1xEV-DO, EV-DO Rev A, EV-DO Rev B, High Speed Packet Access (HSPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Evolved High Speed Packet Access (HSPA+), Long Term Evolution (LTE), AMPS, or other known signals that are used to communicate within a wireless, cellular or internet of things (IOT) network, such as a system utilizing <NUM>, <NUM> or <NUM>, or further implementations thereof, technology.

The present disclosure describes techniques for routing a notification of an incoming voice or video communication request (or "call") to an electronic device. A user may have a communication device, such as smartphone or laptop computer, that can receive voice or video calls from another device. When such a call is received, the communication device itself may output a notification, such as a ring tone or a vibration, or it may request that another device output the notification. The notification may be used to draw the user's attention to that device, which the user may then use to answer and participate in the incoming call, rather than searching for the communication device itself.

For example, a user's smartphone may be connected to multiple different devices that may allow the user to participate in the incoming call, such as a headset or a tablet. But rather than outputting a notification from each of these connected devices, as well as the smartphone itself, the smartphone may instead attempt to determine only one device to output the notification. To do so, the smartphone may request that each of the connected devices (including the smartphone) activate a microphone and begin listening for the user's voice. If the user's voice is recognized near one of the connected devices (or the smartphone itself), the smartphone may request only that device output the notification. Alternatively, the smartphone may request the connected devices to each activate a camera to capture images to try to optically locate the user. The smartphone may then determine whether the user is identified in any of the captured images, and if so, request that the corresponding device output the notification. In some examples, the communication device may also detect distances to each detected person from each of the connected devices, thereby providing an additional metric by which to select a device to output the notification. Further, in some examples, the communication device may be able to discriminate between different persons' voices or faces, and prioritize the detected persons to output a notification using a device in proximity to the most important detected person.

Particular implementations of the subject matter described in this disclosure can be implemented to realize one or more of the following potential advantages. By selecting only one device to output an incoming call notification, the user's smartphone will not inundate the user with notifications from multiple devices at the same time, which are all attempting to notify the user of the same incoming call. Further, by only outputting a notification using one device, systems according to this disclosure may avoid disturbing other individuals at the same location. In addition, by using audio or video sensors to attempt to locate the user, the communication device may select a device that is likely to catch the user's attention when the notification is output. Additionally, by only outputting a notification using one device, excess power consumption to output the redundant notifications by other devices may be avoided.

<FIG> shows a layout of an example environment suitable for intelligent routing of notifications of incoming voice communication requests. In particular, <FIG> shows an example layout of a user's home <NUM>, which has multiple rooms <NUM>-<NUM>. In this example, the user <NUM> left the smartphone <NUM> on a bed <NUM> in the bedroom. The user <NUM> is currently at the desk <NUM> in the study <NUM>, browsing the web on a tablet <NUM>. In the living room <NUM>, there is a smart TV <NUM> (such as a television configured to execute voice communication or video communication applications) and an IOT hub <NUM>, which provides voice-controlled on-demand virtual assistant features, such as shopping features, weather and traffic information, music and video features, etc. Each of these devices is connected to the smartphone <NUM>, such as via a Bluetooth ("BT") connection or a WiFi connection using the WiFi access point ("AP") <NUM>.

When an incoming call is detected by the smartphone <NUM>, it determines whether it should handle the incoming call itself or whether to route the call to another connected device <NUM>-<NUM>. To do so, it determines which of the connected devices <NUM>-<NUM> is capable of handling the audio or video communications. For example, a suitable device may have a speaker, a microphone, an image sensor, such as a camera, etc. In this example, each of the user's tablet <NUM>, smart TV <NUM>, and IOT hub <NUM>, as well as the smartphone <NUM> itself, can interface with the user <NUM> to handle voice calls because each has a microphone and a speaker. For each of these devices <NUM>-<NUM>, the smartphone <NUM> may provide a ring notification; however, this may result in a large number of devices <NUM>-<NUM> ringing throughout the user's home or office, which may be irritating to the user <NUM> or to other people in the vicinity.

To reduce the annoyance or inconvenience to the user <NUM> of multiple devices ringing simultaneously throughout the user's house, the smartphone <NUM> may instead attempt to determine which of the available connected devices <NUM>-<NUM> is nearest to the user <NUM> and only instruct that device to provide a ring notification. In this example, the smartphone <NUM> identifies the device nearest the user <NUM> without ringing any of the connected devices <NUM>-<NUM> or the smartphone <NUM> itself, and before answering the call. To do so, the smartphone <NUM> identifies all the connected devices <NUM>-<NUM> that are connected to the smartphone <NUM> and then commands each to activate its microphone to attempt to detect the user's voice. The smartphone <NUM> also may activate its own microphone to attempt to detect the user's device. Each of the devices <NUM>-<NUM> then activates its respective microphone and begins receiving audio signals. Each device <NUM>-<NUM> then either performs voice recognition on those received audio signals itself, or forwards the received audio signals to the smartphone <NUM> or to another computing device to perform voice recognition. One or more of the devices <NUM>-<NUM> also may obtain other information about the obtained audio signals, such as signal to noise ratios ("SNR"), an amplitude of one or more detected voices, etc., which may be used to determine a relative distance from the respective device to the user <NUM>.

If the user's voice is recognized based on the received audio signals, the connected device or devices that obtained the audio signals is then identified by the smartphone <NUM>. The smartphone <NUM> determines which of these devices is the best one to ring. If only one connected device recognized the user <NUM>, such as the tablet <NUM> in this example, that device may be selected. If more than one connected device recognizes the user <NUM>-for example, if the user <NUM> were in the living room near the smart TV <NUM> and the IOT hub <NUM>-the smartphone <NUM> may select one of those devices <NUM>-<NUM> based on different factors, such as strength of the respective audio signals, the SNR of the respective audio signals, etc. Further, if more than one user is detected, such as the user <NUM>, the user's spouse, the user's child(ren), etc., the smartphone <NUM> may then, based on a priority of the various users, select the user to whom the call should be directed.

For example, if the user <NUM> and one of the user's children are detected, the smartphone <NUM> may select the device that detected the user <NUM>, or is closest to the user <NUM>. Or, if the user's spouse and one of the user's children are detected, but not the user <NUM>, the device that detected the spouse may be selected. After selecting one of the devices <NUM>-<NUM>, the smartphone <NUM> commands the selected device to ring and commands the other devices to disable their microphones. The user <NUM> may then interact with the ringing device to answer the phone call.

<FIG> shows an example system <NUM> for intelligent routing of notifications of incoming voice communication requests. In this example, a smartphone <NUM> is connected to a cellular network and is capable of receiving cellular phone calls. In addition, the smartphone <NUM> is connected to multiple other devices: a laptop <NUM>, a BT headset <NUM>, and a smart TV <NUM>. In this example, the other devices <NUM>-<NUM> are connected to the smartphone <NUM> via a previously-established pairing or other peer-to-peer relationship. For example, the BT headset <NUM> is connected to the smartphone <NUM> based on successfully pairing with the smartphone <NUM> via the BT protocol. Thus, in some examples, a connected device may be one that has established communications with the other smartphone <NUM> rather than one that is simply connected to a common network, such as a local area network. For example, while the smartphone <NUM> and the laptop <NUM> may each be connected to the wireless AP <NUM> shown in <FIG>, if they have not previously established communications with each other to exchange messages or data, they are not "connected.

It should be noted that devices that have previously established a connection with each other, which may not be connected at a given time, but are registered with the smartphone <NUM> and are available for a connection may be dynamically connected upon the occurrence of predetermined events. For example, the smartphone <NUM> and the laptop <NUM> may have previously established a connection with each other, such as to allow the laptop <NUM> to handle user interface functions of telephone calls received by the smartphone <NUM>. In such an arrangement, the smartphone <NUM> handles the signaling and communications with a cellular network, while the laptop <NUM> uses its speakers and microphone to allow a user to engage in a telephone call being handled by the smartphone <NUM> or to dial numbers using the laptop's touchscreen or keyboard. However, if no call is active, the connection between the laptop <NUM> and smartphone <NUM> may be suspended and terminated. However, if a new call were to be detected by the smartphone <NUM>, the smartphone <NUM> may attempt to establish a connection to the laptop <NUM>. In such a scenario, the laptop <NUM> is instead available for a connection. The smartphone <NUM> may then reestablish a connection to enable a user to take an incoming voice call via the laptop <NUM>.

In this example, the smartphone <NUM> is connected to each of the other devices <NUM>-<NUM> and maintains a list or registry of devices that are connected or have previously connected or registered with the smartphone <NUM>. The list may identify each device and may include information about the capabilities of the devices. For example, the list for the connected devices <NUM>-<NUM> in this example may include data such as the following:.

In the example list above, each of devices <NUM>-<NUM> is represented, as well as their respective connection status. In addition, device <NUM> (not shown) is known by the smartphone <NUM> but is not currently available. For example, it is powered off or out of range, and has audio input and output capabilities, but not video input and output capabilities.

It should be appreciated that the smartphone <NUM> may simultaneously employ multiple different wireless technologies to establish connections with various devices. In this example, the smartphone <NUM> can connect to the laptop <NUM> and the smart TV <NUM> using a WiFi connection, while it is connected to the BT headset <NUM> via a BT connection. Still other types of communications protocols or techniques may be employed in different examples, including WiFi, BT, BT low-energy ("BLE"), Zigbee, IEEE <NUM>. <NUM>, etc. Any or all of these different technologies may be employed independently or concurrently according to different examples.

<FIG> shows an example computing device <NUM> for intelligent routing of notifications from incoming voice communication requests. The example computing device <NUM> includes a processor <NUM>, memory <NUM>, a display, user input devices <NUM>, a microphone <NUM>, a speaker <NUM>, and a camera <NUM> in communication with each other via bus <NUM>. In addition, the computing device <NUM> includes three wireless transceivers <NUM>, <NUM>, <NUM> and associated antennas <NUM>, <NUM>, <NUM>. The processor <NUM> is configured to execute processor-executable program code stored in the memory <NUM> to execute one or more methods for intelligent routing of notification of incoming voice communication requests according to this disclosure.

In this example, the computing device <NUM> is a smartphone. However, the computing device may be any computing device configured to receive and handle audio or audio/visual ("video") communications over a communications network with a remote computing device constituting the other end of the voice or video communication. Example computing devices according to this disclosure may be laptop computers, desktop computers, tablets, phablets, satellite phones, cellular phones, dedicated video conferencing equipment, IOT hubs, virtual assistant device (such as Alexa®, Home®, etc.), wearable devices (such as smart watches, earbuds, headphones, Google Glass®, etc.), in-vehicle entertainment or communication systems, home security systems, or any device having an interface to a communications network and suitable input and output devices. It should be appreciated that reference to a voice communication or a voice call includes video communications, while reference to voice-only communication or voice-only call excludes such video communications.

In this example, the smartphone <NUM> is equipped with a wireless transceiver <NUM> and antenna <NUM> configured to communicate with a cellular network using any suitable communications technology as discussed above.

In this example, the smartphone <NUM> may receive incoming voice communication requests from a remote device connected to a cellular network. After the remote device initiates the cellular phone call, the smartphone <NUM> may receive an indication from the cellular network of the incoming phone call. In some examples, the incoming voice communication request may be another form of audio or video communication, such as a video conferencing request (FaceTime®, Skype®, Hangouts®, etc.), a walkie-talkie request (such as via a Sprint/Nextel Direct Connect protocol, etc.), a satellite phone call, etc. Such voice or video communication request may be received over any suitable network. For example, a voice communication request may be received from a local area network ("LAN"); wide area network ("WAN"), such as the Internet; metropolitan area network ("MAN"); point-to-point or peer-to-peer connection; etc. Communication between the computing devices may be accomplished using any suitable networking protocol. For example, one suitable networking protocol may include the Internet Protocol ("IP"), Transmission Control Protocol ("TCP"), User Datagram Protocol ("UDP"), or combinations thereof, such as TCP/IP or UDP/IP.

A voice or video communication generally refers to a real-time or near-real communications between users of different computing devices connected via one or more intervening communications network, which enables the users to converse with each other substantially as they could if they were in a face-to-face conversation. For example, conventional phone calls and video conference calls are voice or video communications according to this disclosure.

In addition to the wireless transceiver <NUM> and antenna <NUM>, the smartphone <NUM> also includes additional wireless transceivers <NUM>, <NUM> and antennas <NUM>, <NUM> configured for BT communications and WiFi communications, respectively. Thus, the smartphone <NUM> is able to connect to one or more BT devices, such as a BT headset or earbud, as well as a WiFi access point ("AP").

The smartphone <NUM> also includes a display <NUM> and user input devices <NUM>. Suitable user input devices include touch-sensitive surfaces, such as touch screens, buttons, knobs, rocker switches, directional pads, mice, keyboards, microphones, etc. Suitable output devices include speakers, display devices, braille output devices, haptic output devices, etc..

<FIG> shows an example computing device <NUM> that may be connected to the computing device <NUM> shown in <FIG>. As discussed above, connected devices may include a wide variety of electronic devices, including any of those discussed above; however, connected devices suitable for providing a user interface to a user to engage in a voice or video communication may include features needed to enable such interactions.

In this example, the computing device <NUM> includes a processor <NUM> in communication with memory <NUM> via bus <NUM>. In addition, the computing device <NUM> includes a wireless transceiver <NUM> and associated antenna <NUM>. In addition, the computing device <NUM> includes input and output devices that may be used to engage in a voice or video communication. In this example, the computing device <NUM> includes a speaker <NUM>, a microphone <NUM>, and a camera <NUM>. Thus, this example computing device <NUM> may be suitable for engaging in both voice and video communications. Some example computing devices may lack a camera <NUM>, while having a microphone <NUM> and a speaker <NUM>.

In this example, the example connected computing device <NUM> provides remote input and output interfaces for the computing device <NUM> of <FIG> that handles the protocol aspects of the electronic voice communication with the calling device. For example, if the computing device <NUM> is a smartphone, while the connected computing device <NUM> is a laptop, the smartphone handles the signaling and other messaging needed to maintain a cellular call, while providing the audio output stream to the laptop to be played via the laptop's speakers. Similarly, the laptop provides an audio output stream to the smartphone to be transmitted via the cellular network to the far end of the cellular phone call. Thus, the connected computing device <NUM> in this example provides the user interface portion of the voice or video communication, while the computing device <NUM> handles the mechanics of establishing and maintaining the voice or video communication with the calling device.

<FIG> shows an example system <NUM> for intelligent routing of notifications of incoming voice communication requests. In this example, a cellular device <NUM> places a video call to smartphone <NUM> using the cellular network <NUM> via cellular transceiver 516a. The smartphone <NUM> is connected to multiple other devices: a laptop <NUM>, a BT headset <NUM>, and a smart TV <NUM>. In this example, the other devices <NUM>-<NUM> are registered with or connected to the smartphone <NUM> via a previously-established pairing or other peer-to-peer relationship generally as discussed above with respect to <FIG>.

In this example, when the smartphone <NUM> receives the incoming voice call from the cellular network <NUM> via cellular transceiver 516b, it identifies the connected devices <NUM>-<NUM> and identifies which are capable of handling a video call. In this example, the smartphone <NUM> identifies the laptop <NUM> and smart TV <NUM> as both having microphones, speakers, and cameras, and thus each is capable of handling a video call. In contrast, the BT headset <NUM> has a microphone and speakers, but lacks a camera. Thus, the smartphone <NUM> identifies it as being capable of handling voice calls, but not video calls.

In this example, the smartphone <NUM> then activates its own microphone and also commands the laptop <NUM> and smart TV <NUM> to activate their respective microphone(s) to detect whether the user is near the smartphone <NUM> or to the identified connected devices <NUM>, <NUM>. As the devices <NUM>, <NUM>, <NUM> receive audio signals from their activated microphones, they perform voice recognition to attempt to recognize one or more voices based on the received audio signals. In addition, each device <NUM>, <NUM>, <NUM> also may obtain additional information, such as SNR or voice signal strength, that may be used to determine a relative distance from the respective device to the person or persons.

In this example, the smartphone <NUM> and the laptop <NUM> each have voice recognition capabilities; however, the smart TV <NUM> does not. Thus, the smart TV <NUM> connects using network <NUM> to a remote computing device <NUM> that provides voice recognition, and then streams received audio signals to the remote computing device <NUM>. The remote computing device <NUM> then performs one or more voice recognition techniques on the received audio signals to attempt to recognize one or more user voices. If the remote computing device <NUM> recognizes one or more user voices, it responds to the smart TV <NUM> with information identifying the recognized voices. For example, the remote computing device <NUM> may provide a name, user ID, user profile, or other information to the smart TV to identify each recognized voice. In addition, the remote computing device <NUM> may provide information related to the voice information, such as a quality or confidence rating of the voice recognition, SNR information, etc. For example, the audio signals streamed to the remote computing device <NUM> may include only faint voice information, such as if the speaker is distant from the microphone, or may include background noise that interferes with the voice recognition techniques. Thus, a SNR value may be low or a confidence rating for the recognized voice(s) may be low. Such information may be provided to the smart TV <NUM> along with information about one or more recognized voices. The smart TV <NUM> may then provide information to the smartphone <NUM> identifying the recognized voices as well as any quality, SNR, or other information received from the remote computing device <NUM>.

Similarly, the laptop <NUM>, after performing its own voice recognition techniques, provides to the smartphone <NUM> any information identifying any recognized voices. Further, in some examples, the laptop <NUM> also may provide additional information, such as SNR information, confidence information, etc..

After receiving information about recognized voices from each of the identified connected devices <NUM>, <NUM>, and performing voice recognition on its own obtained audio signals, the smartphone <NUM> determines whether the user was identified by any of the devices <NUM>, <NUM>, <NUM>. If the user was only identified by one device, the smartphone <NUM> can then transmit a notification to that device to play a voice call notification to alert the user of the incoming voice call. If the user was identified by multiple devices, the smartphone <NUM> can determine which of the devices to select, and transmit the notification to that device, or it can select multiple devices and transmit a notification to each such device. After a device outputs the voice call notification, the user can then answer the call using that device.

While in this example, the smartphone <NUM> only commanded the laptop <NUM> and the smart TV <NUM> to activate their microphones because the incoming communication was identified as a video call, in some examples, the smartphone <NUM> also may command the BT headset <NUM> as well, despite it lacking a camera. For example, the smartphone <NUM> can either accept the video call and then only provide audio signals to the BT headset <NUM>, or convert the voice call to a voice-only call, which may then make the BT headset <NUM> a suitable connected device to provide a user interface for the call.

Further, in some examples, after receiving a notification of an incoming voice communication, the smartphone <NUM> may command the connected devices <NUM>-<NUM> to activate their microphones, but also may, or instead, command the laptop <NUM> and smart TV <NUM> to activate their respective cameras to try to identify the user, or other individuals. For example, the laptop <NUM> may activate a built-in webcam and capture one or more images. The laptop <NUM> may then attempt to identify one or more individuals within the image(s), and if any individuals are identified, attempt to recognize the identified individuals and determine relative distances to one or more of the recognized individuals. For example, the laptop <NUM> may employ one or more face identification and facial recognition techniques to identify and recognize individuals from one or more captured images. In some examples, the laptop <NUM> may instead transmit one or more captured images to a remote computing device <NUM>, which then attempts to identify and recognize one or more individuals in the captured images. The remote computing device <NUM> may then provide to the laptop <NUM> an indication of one or more individuals recognized within the image(s) or an indication that no individuals were recognized. In some examples, the smart TV <NUM> or other connected devices may perform a similar technique to visually identify and recognize one or more individuals in one or more captured images and, in some examples, determine a distance to the one or more recognized individuals based on the captured images.

After attempting to identify and recognize the user or other individuals, the connected devices <NUM>-<NUM> may transmit information to the smartphone <NUM> identifying any recognized individuals, whether recognized visually or audibly, or other information such as a determined distance to the user from the connected device, SNR information, etc. The smartphone <NUM> may then employ such information to select one of the connected devices, or to select itself, as the device to output a call notification. After which, the smartphone <NUM> commands the selected device to output a call notification, such as a ring tone, a flashing light or screen, a vibration, etc..

<FIG> shows an example method <NUM> for intelligent routing of notifications of incoming voice communication requests. The example method <NUM> will be described with respect to the example system <NUM> of <FIG>. However, it should be appreciated that example methods according to this disclosure may be employed by any suitable devices, systems, or methods according to this disclosure.

At block <NUM>, the smartphone <NUM> receives an indication of an incoming voice communication request. In this example, cellular device <NUM> places a voice call to the smartphone <NUM> using the cellular network <NUM> via cellular transceiver 516a. The smartphone <NUM> receives the indication of the incoming voice call from the cellular network <NUM> via cellular transceiver 516b. Any suitable cellular technology may be employed in various examples, such as those discussed above.

In this example, the cellular device <NUM> places a standard cellular voice call. Thus, the cellular network <NUM> provides an indication of the incoming cellular voice call using the cellular protocol employed by the smartphone <NUM>. But in some examples, the cellular device <NUM> may initiate a voice communication using a voice or video call application (or "app") executed by the cellular device <NUM>, which data is carried by the cellular network as a data stream, such by using a networking protocol encapsulated within one or more data transmission protocols provided by the cellular network. However, any suitable application that allows users to initiate a voice or video call to another device, or receive a voice or video call by another device may be employed. Such applications may include features to provide indications to a called device that a voice or video communication is requested, which may then initiate a notification to the user of the called device of the request.

While in this example, the called device is a smartphone <NUM>, any other suitable electronic device capable of receiving a voice or video communication request from a remote device may be employed, such as discussed above with respect to <FIG>.

Further, while this example discusses a voice call placed over a cellular network, any suitable communications network or voice or video communication request may be employed, such as discussed above with respect to <FIG>.

In response to the indication and before requesting a device to output a notification, the method <NUM> proceeds to block <NUM>. In this example, the smartphone proceeds to block <NUM> without requesting any device to output a notification.

At block <NUM>, the smartphone <NUM> identifies one or more connected devices having one or more suitable input devices and one or more suitable output devices. Suitable input and output devices may be identified based on the type of voice communication request. Suitable input and output devices for a voice-only communication request include one or more microphones and one or more speakers. Suitable input and output devices for a video communications request include one or more microphones, one or more speakers, and, optionally, one or more cameras and video display devices. In some examples, an incoming video communication request may be downgraded to a voice-only communication, or may be refused entirely, if no suitable connected devices (or the called device itself) have a camera and a video display device.

In this example, the smartphone <NUM> has three devices connected to it: a laptop <NUM>, a BT headset <NUM>, and a smart TV <NUM>; however, any number of devices may be connected to the smartphone <NUM>. As discussed above, a connected device may be statically or dynamically connected to the smartphone <NUM>. For example, the BT headset <NUM> may be statically connected as long as it remains powered on and in range of the smartphone <NUM>. Thus, the connection is actively maintained, even when no audio data is exchanged between the smartphone <NUM> and the BT headset <NUM>. However, some devices may be dynamically connected to the smartphone <NUM>.

For example, the laptop <NUM> may be in communication with LAN via a WiFi AP to which the smartphone <NUM> is also connected. The smartphone <NUM> may detect the presence of the laptop <NUM> and recognize the laptop <NUM> as a device authorized to connect to the smartphone <NUM>, but may not initiate a connection with the laptop <NUM> unless a triggering event occurs. For example, the user may open a music player application on the smartphone <NUM>, which may access a media library stored on the laptop <NUM>. A connection to the laptop <NUM> may then be dynamically established to provide access to the media library. Similarly, in this example, the laptop <NUM> may provide audio or video interfaces to the smartphone <NUM> for voice or video calls handled by the smartphone <NUM> via the laptop <NUM>. In this example, the smart TV <NUM> is also dynamically connectable by the smartphone <NUM>, such as when an incoming voice communication is received.

Thus, at block <NUM>, the smartphone <NUM> identifies devices that are connected to the smartphone <NUM> or that are available for a dynamic connection to the smartphone <NUM>.

In addition to identifying devices that are connected or available for dynamic connection, the smartphone <NUM> determines the capability of each device to serve as an interface device for the incoming voice communication request. In this example, the incoming voice communication request indicates a voice-only communication, thus the smartphone <NUM> identifies all connected or connectable devices that have a microphone and speakers. In some examples, if a video communication request is received, the smartphone <NUM> identifies all devices having a microphone, speaker, camera, and display device. Such information may be stored in a database or other registry of known devices generally as described above with respect to <FIG>.

For example, when the smartphone <NUM> connects to a new device, it may request device capabilities from the device, such as microphone, speakers, camera, display screen, etc. Information regarding such capabilities received from the new device may be stored by the smartphone <NUM> in a record associated with the device, such as based on a unique identifier for the device (MAC address, IMEI, etc.). At a later time, when the smartphone <NUM> connects to the device again, it may access the device's record to determine which capabilities the device has. In some examples, however, the smartphone <NUM> may determine a device's capabilities each time it connects to the device. For example, device capabilities may change over time, or a user may disable one or more input or output devices, such as by muting speakers or disabling a camera.

If the smartphone <NUM> is unable to identify a connected device with suitable input and output devices, the method may end and the smartphone <NUM> itself may output a notification of the incoming voice communication, such as by ringing. Alternatively, if no suitable input or output devices are identified for a video call, but one or more connected devices is capable of handling a voice-only call, the smartphone <NUM> may downgrade the communication request from a video communication request to a voice-only communication request, or provide a notification to the calling device that video capabilities are not supported. If the smartphone <NUM> downgrades the video call to a voice call, it may then identify one or more connected devices capable of handling voice-only calls.

At block <NUM>, the smartphone <NUM> commands each identified device to obtain sensor signals using one or more suitable input devices. In this example, the smartphone <NUM> transmits a command to the BT headset <NUM> to activate its microphone to obtain audio signals. In addition, the smartphone <NUM> initiates a dynamic connection to each of the laptop <NUM> and the smart TV <NUM>. After initiating each dynamic connection, the smartphone <NUM> commands the respective connected device <NUM>, <NUM> to activate its microphone to obtain audio signals. In some examples, the smartphone <NUM> itself also may activate its microphone or camera to obtain sensor signals.

In some examples, if the incoming voice communication request indicates a video communication request. In one such example, the smartphone <NUM> may command each identified device to obtain audio signals using one or more microphones, video signals using one or more cameras, or both audio and video signals. For example, the smartphone <NUM> may transmit a command to the laptop <NUM> to obtain audio signals from its microphone and to obtain video signals from its camera. Further, in some examples, the smartphone <NUM> may command one or more devices <NUM>-<NUM> to obtain sensor signals both from a microphone and a camera even in the context of a voice-only communication request. Such a technique may provide multiple types of sensors that may be used to recognize one or more persons, which may increase the odds of recognizing a person.

In response to receiving such a command, each of the connected devices <NUM>-<NUM> may perform a method according to this disclosure to obtain sensor signals, and either recognize one or more persons based on the sensor signals or provide the sensor signals to the smartphone <NUM>. One such example method <NUM> is shown in <FIG>, which is described in more detail below.

At block <NUM>, the smartphone <NUM> receives an indication of one or more recognized persons based on one or more sensor signals. In this example, the smartphone <NUM> receives an indication of a recognized person from one or more of the devices <NUM>-<NUM>, which determined a recognized person as described below with respect to <FIG>. In some examples, however, the smartphone <NUM> may receive sensor signals from one or more of the devices <NUM>-<NUM> and perform one or more recognition techniques using the received sensor signals from each device that provided them. After performing the recognition technique(s), the smartphone <NUM> receives the indication of the recognized person as an output of the respective recognition techniques.

Examples of suitable recognition techniques include voice recognition, facial recognition, object recognition, etc. Received audio signals may be provided to a voice recognition technique to attempt to recognize one or more voices recorded within the audio signals, while received video signals may be provided to a facial recognition technique to identify one or more persons faces. Indications of the recognized voices or faces may be received by the smartphone <NUM> from such recognition techniques. For example, the indications may include a name, a user profile ID, or some other identifier recognizable by the smartphone <NUM> of one or more persons. It should be understood that multiple persons may be identified by one or more of the connected devices <NUM>-<NUM>, or by the smartphone <NUM> itself.

In some examples, one or more of the devices <NUM>-<NUM> may attempt to determine a distance from the respective device to one or more recognized persons. A distance determination may be based on audio signals, such as by determining SNR information or voice signal strength, or images and a detected size of the person within the captured images. For example, voice signal strength may decrease the further a person is from a microphone. Thus, a device may employ a correspondence of voice signal strength to distance to determine a distance to the individual. It should be appreciated that distance does not necessarily refer to an objective distance measurement, such as in feet or meters. Rather, distance may be a coarse measurement, such as "near" or "far" based on a predetermined threshold value for voice signal strength, SNR, etc. In some examples, a distance may be inferred by the smartphone <NUM> based on different SNR or voice signal strength measurements from multiple devices. By comparing the respective SNR or voice signal strength measurements, the smartphone <NUM> may determine a ranking of the distance of the respective devices from one or more recognized individuals by ordering the devices according to their respective SNR or voice signal strength measurements. Such an ordering may provide relative distance information, such as "nearest" and "farthest" for one or more devices.

At block <NUM>, the smartphone <NUM> selects a device, whether the smartphone <NUM> itself or one of the connected devices <NUM>-<NUM>, based on the identified person or persons. In this example, the smartphone <NUM> is associated with its owner, thus an incoming voice communication request is most likely directed to the owner of the device. Thus, if the owner of the smartphone <NUM> was recognized, any device <NUM>-<NUM> that provided sensor signals used to recognize the owner may be a candidate for selection. If only one device <NUM>-<NUM> provided such sensor signals, the smartphone <NUM> selects that device. However, if multiple devices <NUM>-<NUM> provided such sensor signals, the smartphone <NUM> may select a device <NUM>-<NUM> based on one or more factors.

Such factors may include a detected distance of a device <NUM>-<NUM> to the owner. For example, the laptop <NUM> may have obtained sensor signals used to recognize the owner, but the sensor signals had a low SNR or were at a low amplitude, indicating the owner was likely distant from the laptop <NUM>. At the same time, the smart TV <NUM> may have obtained sensor signals with an SNR greater than the SNR from the laptop <NUM>, or an amplitude greater than those obtained by the laptop <NUM>, potentially indicating that the owner was closer to the smart TV <NUM> than the laptop <NUM>. In such an example, the smartphone <NUM> may select the smart TV <NUM>. Alternatively, the smartphone <NUM> may receive distance information from one or more devices, such as described above.

In some examples, one or more devices <NUM>-<NUM> may recognize the owner from multiple sensor inputs. For example, the owner may have been recognized based on sensor signals obtained both by the laptop's camera and microphone. In contrast, the owner may have been recognized by sensor signals from the smart TV's microphone, but not from its camera. Thus, the smartphone may <NUM> select the laptop based on more of the laptop's sensor inputs indicating the presence of the owner than the smart TV's.

In some examples, the smartphone <NUM> may select a device based on priority information. Priority information may be associated with one or more recognized persons or one or more devices <NUM>-<NUM>. For example, the smartphone <NUM> may maintain information about known persons, such as the owner of the device, the owner's spouse, the owner's child(ren), etc. One or more of such known persons may have associated priority information that can be used to select a device <NUM>-<NUM>. Such priority information may indicate a preference for persons to notify about incoming voice calls, or may identify one or more persons who should not be notified of incoming voice calls. For example, the smartphone <NUM> may have a priority of <NUM> associated with the owner of the device, a priority value of <NUM> associated with the owner's spouse, and a priority value of <NUM> for each of the owner's children. Thus, when the smartphone <NUM> receives information about recognized persons from one or more devices <NUM>-<NUM>, it may determine a priority for each device based on which persons were recognized by which device. For example, if the laptop <NUM> recognizes the owner, while the smart TV recognizes the owner's spouse, the smartphone <NUM> may select the laptop <NUM> as the priority associated with the owner is greater than the priority associated with the owner's spouse.

In some examples, priority information may be associated with one or more devices. For example, the smartphone's owner may provide preferences relating to which devices should be used to answer voice calls. For example, the owner may establish the smartphone with the highest priority, a tablet device (not shown) with a second highest priority, the laptop <NUM> and smart TV <NUM> with a third highest priority, and the BT headset <NUM> with a fourth highest priority. Thus, if multiple devices recognize the owner, the smartphone <NUM> may then select the device that both recognized the owner and has the highest priority.

In some examples, if priority information associated with multiple persons or multiple devices indicate they have the same priority, the smartphone <NUM> may select a device based on other factors, such as distance to the recognized individuals, etc. In some examples, though, the smartphone <NUM> may select multiple devices indicating the same priority levels or information.

At block <NUM>, the smartphone <NUM> transmits a command to the selected device to output a notification of the voice communication request. A notification may include an audible notification, a visual indication, a tactile indication, or any combination of these. The command may include one or more notification identifiers to identify a type of notification to output or information describing the notification itself, such as an audio signal, image, tactile sensation, etc. In some examples, the command may be a command to the device to output the notification without any information describing the notification to be output. In addition to outputting the notification, the smartphone <NUM> also may transmit one or more commands to the devices that were not selected to discontinue obtaining sensor signals.

In some examples, however, no device may respond to the smartphone <NUM> identifying a recognized person with a predetermined time. Such a predetermined time may be a fixed number of rings, a fixed amount of time, or a relative time based on the amount of time before the incoming voice communication request is re-routed to another service, such as a voice message service.

In such a scenario, the smartphone <NUM> may output a command to all devices <NUM>-<NUM> to output the notification, or the smartphone <NUM> may only output the notification itself. In some examples, the smartphone <NUM> may command devices having a priority above a threshold to output a notification of the incoming voice request.

<FIG> shows an example method <NUM> for intelligent routing of notifications of incoming voice communication requests. In this example, the method <NUM> will be described with respect to the example system <NUM> shown in <FIG> and with respect to the example method <NUM> shown in <FIG>. However, it should be appreciated that example methods according to this disclosure may be employed by any suitable devices, systems, or methods according to this disclosure.

At block <NUM>, a connected device <NUM>-<NUM> receives a command to obtain sensor signals. For example, the connected device <NUM>-<NUM> may receive a command to activate a microphone to obtain audio signals or to activate a camera to obtain video signals. In this example, the command identifies one or more types of sensors to be activated and a duration during which to obtain sensor information. If the connected device <NUM>-<NUM> is configured to perform a recognition technique, discussed in more detail below, the command also may include a command to provide information identifying recognized persons, or a deadline by which such information is to be provided.

If the connected device <NUM>-<NUM> is not configured to perform a recognition technique, the command may specify a technique for providing sensor information to the smartphone <NUM>. Suitable techniques may include streaming sensor signals to the smartphone <NUM> or providing a file containing one or more sensor signals or encoded sensor signals. In some examples, sensor signals may be encoded using one or more lossy or lossless encoding techniques, such as full rate, half rate, adaptive multi-rate, MPEG level-<NUM> (mp3), etc. vocoders for audio sensor signals, or bitmap, JPEG, GIF, etc. for images, or MPEG-<NUM>, H. <NUM>, etc. for video signals.

In response to receiving the command, the connected device <NUM>-<NUM> activates the identified sensor(s), such as the microphone or camera. In some examples, the connected device <NUM>-<NUM> may be unable to activate one or more sensors. For example, the user may have disabled either or both, such as via a configuration setting. In some examples, the activation command may be disregarded based on such a configuration, and the method <NUM> may terminate. However, in some examples, the activation command may temporarily override the configuration and activate the sensor(s) for purposes of performing a method according to this disclosure.

At block <NUM>, the connected device <NUM>-<NUM> receives sensor signals from the activated sensor(s). For example, the laptop <NUM> may receive a stream of audio information from its microphone and a stream of images from its camera. In the case of a camera, in some examples, the camera may provide video at a standard frame rate, such as at <NUM>, <NUM>, or <NUM> frames per second, or it may capture images at a specified rate, such as <NUM> per second, or it may only capture a specified number of images. Such sensor signals, after being received, may be stored in a memory for further processing. For example, the audio information may be recorded to an audio file in memory or on a non-volatile storage medium. Images received from the camera may be stored individually in separate files or stored as a video file.

At block <NUM>, the connected device <NUM>-<NUM> determines whether it will perform a recognition technique or will provide the obtained sensor signals to the smartphone <NUM> for processing. If the connected device <NUM>-<NUM> is configured to perform a recognition technique, the method <NUM> proceeds to block <NUM>; otherwise, the method <NUM> proceeds to block <NUM>.

At block <NUM>, the connected device <NUM>-<NUM> provides its sensor signals to the smartphone <NUM>. For example, the connected device <NUM>-<NUM> may transmit an audio file, an image file, or a video file to the smartphone <NUM>. In some examples, the connected device <NUM>-<NUM> may stream sensor signals, or information extracted from sensor signals, to the smartphone <NUM> in real-time. In some examples, such sensor signals may be encoded according to one or more encoding techniques, such as described above with respect to block <NUM> of <FIG>. After providing such sensor signals, the method <NUM> may then terminate.

In this example, the connected device <NUM>-<NUM> will continue to provide sensor signals until it receives a subsequent command to discontinue obtaining sensor signals. In some examples, however, the connected device <NUM>-<NUM> will provide sensor signals for a predetermined period of time or until a predetermined number of sensor signals have been provided. For example, the connected device may provide sensor signals for up to five seconds or may provide up to ten images.

At block <NUM>, the connected device <NUM>-<NUM> recognizes one or more persons based on the received sensor signals. In some examples, the connected device <NUM>-<NUM> itself may execute one or more recognition techniques, such as a voice recognition technique or a facial recognition technique. Suitable voice or facial recognition techniques may employ neural networks, including deep neural networks; hidden Markov models ("HMM"); spectral or cepstral analysis techniques; dynamic time warping techniques; etc. The connected device <NUM>-<NUM> may provide one or more sensor signals to the voice or facial recognition technique and receive an indication of the identity of the person or persons associated with the inputted sensor signals.

In some examples, the connected device <NUM>-<NUM> itself does not perform the recognition technique, but instead provides the sensor signals to a remote computing device <NUM> other than the smartphone <NUM> that executes one or more recognition techniques. Such a remote computing device <NUM> may then respond to the connected device <NUM>-<NUM> with an indication of an identity of the person or persons associated with the inputted sensor signals.

Because in some examples the connected devices <NUM>-<NUM> may obtain sensor signals over a period of time, the connected devices <NUM>-<NUM> may perform multiple iterations of recognition techniques to attempt to identify one or more persons over the period of time the sensor signals are obtained.

At block <NUM>, the connected devices <NUM>-<NUM> provide an indication of one or more recognized persons, or an indication that no persons were recognized, to the smartphone <NUM>. For example, the connected devices <NUM>-<NUM> may provide a name, a user identification number or code, a user profile, etc., to the smartphone <NUM>. In some examples, the connected devices <NUM>-<NUM> may provide multiple indications of recognized persons. As discussed above, the connected devices <NUM>-<NUM> may obtain sensor signals over a period of time and may iteratively perform recognition techniques on obtained sensor signals. As persons are recognized, the connected devices <NUM>-<NUM> may provide indications of the recognized persons to the smartphone <NUM>.

In this example, the connected devices <NUM>-<NUM> will continue to obtain sensor signals and perform recognition techniques until commanded to stop by the smartphone <NUM> or until a predetermined threshold from the command received at block <NUM> is reached.

The hardware and data processing apparatus used to implement the various illustrative logics, logical blocks, modules and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose single- or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, or, any conventional processor, controller, microcontroller, or state machine. A processor also may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some implementations, particular processes and methods may be performed by circuitry that is specific to a given function.

Various modifications to the implementations described in this disclosure may be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other implementations, provided that these various modifications fall within the scope of the invention which is only defined by the appended claims.

Certain features that are described in this specification in the context of separate implementations also can be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation also can be implemented in multiple implementations separately or in any suitable subcombination.

Claim 1:
A method performed by an electronic device, the method comprising:
receiving (<NUM>) an indication of an incoming voice communication request;
identifying (<NUM>) one or more devices capable of communicating with the electronic device via a connection, each of the one or more devices including a sensor capable of obtaining one or more audio signals or one or more image signals;
requesting (<NUM>) one or more of the devices to obtain an audio signal or an image signal via its respective sensor;
obtaining (<NUM>) the audio signal or the image signal obtained by the one or more devices, wherein the audio signal or the image signal is used for recognizing a person, wherein a recognized person is a person known to the electronic device based on maintained information about the person;
determining an associated priority for each of the one or more devices based on the recognized persons;
based on (<NUM>) at least the associated priority, selecting a device of the one or more devices; and
providing (<NUM>) a signal to the selected device to output a notification indicating the incoming voice communication request.