METHODS AND SYSTEMS FOR COMMUNICATIONS MANAGEMENT

A media device, such as a set-top-box, may receive an indication that a video doorbell on the premises has been activated. The indication may activate an application associated with the video doorbell within the set-top-box. While the application is in use, a communication pathway may be created between the set-top-box and the video doorbell. For example, audio and/or video detected by the video doorbell may be sent, directly or indirectly, via the application, to the set-top-box for display on a television or other display. Furthermore, while the application is in use, user voice data for a user associated with the media device may be captured by a voice-enabled remote control and routed to the set-top-box or smart television and then to the video doorbell for output at the video doorbell.

BACKGROUND

Certain conventional media devices that are capable of providing audio/visual content to its users lack microphone inputs for receiving user audio data from users positioned near the media device. This can prevent the nearby user from providing audio data (e.g., voice input) and the media devices from receiving the user audio data from users located in proximity to the media device. This can also prevent users from providing voice communication to other users in another location (e.g., in another room, outside a residence, at another building or residence) via the media device.

SUMMARY

It is to be understood that both the following general description and the following detailed description are exemplary and explanatory only and are not restrictive. Methods and systems for communications management are described.

A user may receive an indication that a video doorbell or other similar security device on the premises has been activated. The indication may be received by a set-top-box, smart television or other media device. The indication may include a notification that may be displayed on the television or other display associated with the media device that the doorbell or other security device has been activated and may provide an opportunity for the user to select to engage with the video doorbell. For example, the indication may activate a module or application associated with the video doorbell within the set-top-box or smart television. While the module or application associated with the video doorbell is in use, a communication pathway may be created between the set-top-box or smart television and the video doorbell or other security device. For example, audio and/or video detected by the video doorbell may be sent, directly or indirectly, via the module or application, to the set-top-box or smart television. Furthermore, while the application associated with the video doorbell is in use, user voice data for a user associated with the media device or display may be captured by a user device, such as a voice-enabled remote control, and routed to the set-top-box or smart television and then to the video doorbell for output at the video doorbell.

This summary is not intended to identify critical or essential features of the disclosure, but merely to summarize certain features and variations thereof. Other details and features will be described in the sections that follow.

DETAILED DESCRIPTION

“Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes cases where said event or circumstance occurs and cases where it does not.

As will be appreciated by one skilled in the art, hardware, software, or a combination of software and hardware may be implemented. Furthermore, a computer program product on a computer-readable storage medium (e.g., non-transitory) having processor-executable instructions (e.g., computer software) embodied in the storage medium. Any suitable computer-readable storage medium may be utilized including hard disks, CD-ROMs, optical storage devices, magnetic storage devices, memresistors, Non-Volatile Random Access Memory (NVRAM), flash memory, or a combination thereof.

These processor-executable instructions may also be stored in a computer-readable memory that may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the processor-executable instructions stored in the computer-readable memory produce an article of manufacture including processor-executable instructions for implementing the function specified in the flowchart block or blocks. The processor-executable instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the processor-executable instructions that execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

“Content items,” as the phrase is used herein, may also be referred to as “content,” “content data,” “content information,” “content asset,” “multimedia asset data file,” or simply “data” or “information”. Content items may be any information or data that may be licensed to one or more individuals (or other entities, such as business or group). Content may be electronic representations of video, audio, text and/or graphics, which may be but is not limited to electronic representations of videos, movies, or other multimedia, which may be but is not limited to data files adhering to MPEG-2, MPEG, MPEG-4 UHD, HDR, 4k, Adobe® Flash® Video (.FLV) format or some other video file format whether such format is presently known or developed in the future. The content items described herein may be electronic representations of music, spoken words, or other audio, which may be but is not limited to data files adhering to the MPEG-1 Audio Layer 3 (.MP3) format, Adobe®, CableLabs 1.0, 1.1, 3.0, AVC, HEVC, H.264, Nielsen watermarks, V-chip data and Secondary Audio Programs (SAP). Sound Document (.ASND) format or some other format configured to store electronic audio whether such format is presently known or developed in the future. In some cases, content may be data files adhering to the following formats: Portable Document Format (.PDF), Electronic Publication (.EPUB) format created by the International Digital Publishing Forum (IDPF), JPEG (0.7PG) format, Portable Network Graphics (.PNG) format, dynamic ad insertion data (.csv), Adobe® Photoshop® (.PSD) format or some other format for electronically storing text, graphics and/or other information whether such format is presently known or developed in the future. Content items may be any combination of the above-described formats.

“Consuming content” or the “consumption of content,” as those phrases are used herein, may also be referred to as “accessing” content, “providing” content, “viewing” content, “listening” to content, “rendering” content, or “playing” content, among other things. In some cases, the particular term utilized may be dependent on the context in which it is used. Consuming video may also be referred to as viewing or playing the video. Consuming audio may also be referred to as listening to or playing the audio.

This detailed description may refer to a given entity performing some action. It should be understood that this language may in some cases mean that a system (e.g., a computer) owned and/or controlled by the given entity is actually performing the action.

Methods and systems are described herein for managing communications between devices.FIG.1shows an example system100. The example system100may be configured for managing communications between devices (e.g., media devices, security devices, network devices, user devices, display devices, etc.). Although only certain devices and/or components are shown, the system100may comprise a variety of other devices and/or components that support a wide variety of network and/or communication functions, operations, protocols, content, services, and/or the like. The system100may comprise a plurality of computing devices/entities in communication via a network114. The network114may be an optical fiber network, a coaxial cable network, a hybrid fiber-coaxial network, a wireless network, a satellite system, a direct broadcast system, an Ethernet network, a high-definition multimedia interface network, a Universal Serial Bus (USB) network, or any combination thereof. Data may be sent on the network114via a variety of transmission paths, including wireless paths (e.g., satellite paths, Wi-Fi paths, cellular paths, etc.) and terrestrial paths (e.g., wired paths, a direct feed source via a direct line, etc.). The network114may comprise public networks, private networks, wide area networks (e.g., Internet), local area networks, and/or the like.

The network114may comprise a content access network, content distribution network, security network, and/or the like. For example, the network114may be configured to provide content from a variety of sources using a variety of network paths, protocols, devices, and/or the like. The network114may also be configured to analyze audio data associated with one or more requests and provide command information or signals based on that audio data. The network114may also be configured to receive security data and provide the security data to one or more devices. For example, the network114may represent multiple separate networks, each providing one or more of the services described above. The network114may be managed (e.g., deployed, serviced) by, for example, a content provider, a service provider, and/or the like. The network114may deliver content (e.g., content items) from a content source104to one or more other computing devices (e.g., media devices120,140A-C). For example, each media device120,140A-C may comprise one of a set top box, a television, a smart television, or a monitor

The network114may comprise or be in communication with a network component116. The network component116may be any device, module, combinations thereof, and the like communicatively coupled to the network114. The network component116may be a router, a switch, a splitter, a packager, a gateway, an encoder, a storage device, a multiplexer, a network access location (e.g., tap), physical link, combinations thereof, and the like.

The system100may comprise a computing device102. The computing device102may be a server. For example, the computing device102may be a cloud-based server. The computing device102may be configured to receive audio data (e.g., user voice data) from media devices (e.g., media devices120,140A-C) and/or user devices (e.g., a voice-enabled remote control122, mobile device123, a voice assistant device, a smart speaker, a Bluetooth headset, etc.). For example, the computing device102may comprise an input/output interface112. The input/output interface112may be configured to connect, cause to connect, launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication session between the computing device102and one or more of the media devices (e.g., media devices120,140A-C) and/or user devices (e.g., a voice-enabled remote control122, mobile device123, a voice assistant device, a smart speaker, a Bluetooth headset, etc.) via the network114or another network.

The computing device102may be configured to convert the received audio data to text data. For example, the computing device102may comprise a speech-to-text engine108. The speech-to-text engine108may be configured to receive the audio data. Further, the speech-to-text engine108may be configured to convert the received audio data to text or text data.

The computing device102may also comprise a command engine110. The command engine110may be configured to analyze the text or text data converted from the audio data to determine one or more commands or actions being requested in the received audio data. For example, the command engine110may receive text that states “change to channel38”. The command engine110may compare one or more of the words or phrases in the text to a stored database of commands or actions to determine the command or action being requested. For example, the command engine110may parse the text into one or more words or phrases and compare the parsed portions of the text to the stored database of commands or actions. The command engine110may determine that a user associated with media device and/or user device wishes to change the channel of their television to channel38. The command engine110may determine the code or instructions to provide to the media device in order to change the channel from the current channel to channel38. The command engine110or the computing device102may send the code or instructions, using the input/output interface112to the media device (e.g., media devices120,140A-C) or user device (e.g., a voice-enabled remote control122, mobile device123, a voice assistant device, a smart speaker, a Bluetooth headset, etc.) that initially sent the audio data to the computing device102. Accordingly, the computing device102may cause the media device (e.g., media devices120,140A-C) and/or display121associated with the media device to change the channel from the current channel to channel38based on the instructions or code provided.

The system100may comprise the one or more content sources104, each of which may be a server or other computing device. The content source104may receive a plurality of content which may comprises a plurality of content items. The content may be video content, audio content, web-based content, audio/video content, games, applications, data, etc. The content may be live content (e.g., a linear content stream) audio-on-demand content, video-on-demand (VOD) content, web data, application data, or the like. The content source104may receive source content from an external server or device (e.g., a stream capture source, a data storage device, a media server, etc.). The content source104may receive the source content via a wired or a wireless network connection, such as the network114or another network (not shown). For example, the content source104may be a multicast or a unicast content source configured to send or otherwise transmit content via a multicast or unicast transmission.

The content source104may be managed by a third-party, such as content providers, service providers, online content providers, over-the-top content providers, and/or the like. The content source104may be configured to provide content (e.g., content items) via the network114or another network. Content may be accessed by one or more media devices120,140A-C via applications, such as mobile applications, television applications, set-top box applications, gaming device applications, web-based applications, and/or the like. An application may be a custom application (e.g., by a content provider, for a specific device), a general content browser (e.g., a web browser), an electronic program guide, and/or the like.

Although one content source104is shown inFIG.1, this is not to be considered limiting. In accordance with the described techniques, the system100may comprise any number of content sources, each of which may receive and provide any number of content items.

The system100may comprise a computing device106. The computing device106may be a server. For example, the computing device106may be a cloud-based server. For example, the computing device106may be an application-based server associated with a security application and/or video doorbell application. The computing device106may be configured to receive at least one of audio data, video data, and/or audio/video data (e.g., security data) from one or more security devices (e.g., video doorbells132and/or security cameras138A-B). For example, the computing device106may comprise an input/output interface107. The input/output interface107may be configured to connect, cause to connect, launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication session between the computing device106and one or more of the security devices (e.g., the video doorbell132and/or one or more of the security cameras138A-B) via the network114or another network. For example, the computing device106, via the input/output interface107, may be configured to receive at least one of video data and/or audio data (e.g., collectively “security data”) from the one or more security devices (e.g., video doorbells132and/or security cameras138A-B). For example, the security data may include or be associated with metadata. For example, the metadata may include user information. The user information may identify the user or users associated with the one or more security devices, contact information for the user, a location or address for the user, a URL or network address associated with the user, a MAC address associated with the user, an email address associated with the user, and/or URLs or network addresses associated with one or more media devices associated with the user.

The computing device106may comprise a device data database109. For example, the computing device106may store all or a portion of the received security data in the device data database109. For example, the computing device106may determine, based on the metadata or another portion of the security data, one or more users or identifiers to associate the received security data with. The computing device106may then store the received security data in the device data database109based on the association. For example, the security data may be associated with a user name, user phone number, user address, the one or more security devices associated with a user, a MAC address associated with the user, an email address associated with the user, or in any other manner.

The computing device106, input/output interface107, may be configured to connect, cause to connect, launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication session between the computing device106and one or more of the media devices (e.g., media devices120,140A-C) and/or user devices (e.g., a voice-enabled remote control122, mobile device123, a voice assistant device, a smart speaker, Bluetooth headset, etc.) via the network114or another network. For example, the computing device106may be configured to send all or a portion of the received security data (e.g., at least one of video data, audio data, or audio/video data) to the one or more of the media devices (e.g., media devices120,140A-C) and/or user devices (e.g., a voice-enabled remote control122, mobile device123, a voice assistant device, a smart speaker, a Bluetooth headset, etc.) via the network114or another network.

The system100may comprise a premises130. The premises130may be a building, portion of a building, house, townhouse, condominium, apartment, duplex, office, or the like. The premises130may be occupied by one or more users. The premises130may comprise a point of entry139(e.g., a doorway). Access may be restricted through the point of entry139by one or more doors137, which may be adjustable from an open position to a closed position. The premises130may also include one or more rooms and/or one or more floors.

The system100may also include one or more security devices, such as the video doorbell132and/or the video cameras138A-B. The security devices132,138A-B may be located at any location along the premises130. For example, the video doorbell132may be located adjacent to or near the point of entry139and the door137along an exterior of the premises130. For example, the one or more video cameras138A-B may be located along or near the exterior of the premises130.

The video doorbell132may comprise a camera134, a microphone135, an activation device136, a speaker, a transmitter, a receiver (or transceiver), and one or more processors. The activation device136may be a doorbell button, a doorbell switch, a doorbell touchpad, a capacitive sensor, a motion sensor, an audio sensor, an infrared sensor, or other similar device. For example, the video doorbell may be activated based on the use or triggering of the activation device136. For example, a visitor may depress the activation device136to activate a doorbell chime, the camera134, and the microphone135to begin recording the security data. The video doorbell132may be configured to record security data such as at least one of video data, via the camera134, and audio data, via the microphone135. The video doorbell132may be configured to launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication session with the computing device106and send the security data, via the transmitter or transceiver, to the computing device106via the network114or another network. The video doorbell132may be configured to receive audio data, such as from the media device120, via the receiver or transceiver, and output the audio data via the speaker of the video doorbell132. For example, the audio data may comprise user voice data. For example the user voice data may be received by the media device120from one of the user devices (e.g., a voice-enabled remote control122, mobile device123, a voice assistant device, a smart speaker, a Bluetooth headset, etc.).

Each video camera138A-B may comprise a camera, a microphone, an activation device, a speaker, a transmitter, a receiver (or transceiver), and one or more processors. The activation device may be a motion sensor, an audio sensor, an infrared sensor, or other similar device. For example, the video camera138A-B may be activated based on the triggering of the activation device. For example, a person may pass within a viewing window of the video camera138A-B and/or the activation device of the video camera138A-B to activate the camera and the microphone to begin recording the security data. The video camera138A-B may be configured to record security data, such as at least one of video data, via the camera, and audio data, via the microphone. The video camera138A-B may be configured to launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication session with the computing device106and send the security data, via the transmitter or transceiver, to the computing device106via the network114or another network. The video camera138A-B may be configured to receive audio data, such as from the media device120, via the receiver or transceiver, and output the audio data via the speaker of the video camera138A-B. For example, the audio data may comprise user voice data. For example the audio data (e.g., user voice data) may be received by the media device120from one of the user devices (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, a smart speaker, a Bluetooth headset, etc.).

The system100may comprise one or more media device120,140A-C. Each media device120,140A-C may be a computing device and may be configured to receive one or more content items from the one or more content sources104. Each media device120,140A-C may be configured to receive audio data from one or more user devices (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, a smart speaker, a Bluetooth headset, etc.). Each media device120,140A-C may be configured to receive at least a portion of the security data from the computing device106. Each media device120,140A-C may be configured to send audio data to the computing device102and, based on the audio data, receive command instructions from the computing device102for executing a command related to display121/television. Each media device120,140A-C may be configured to send audio data to one or more of the security devices (e.g., the video doorbell132or the one or more video cameras138A-B) and/or the computing device106.

Each media device120,140A-C may comprise a device configured to enable a display device121(e.g., a display, a television, a computer or other similar device) to output media (e.g., video data and/or audio data). For example, each media device120,140A-C may be configured to receive, decode, transcode, encode, send, and or otherwise process data and send data to, for example, the display device121. Each media device120,140A-C may comprise one or more processors, transmitter, receiver, transceiver, a demodulator, decoder, frequency tuner, combinations thereof, and the like. Each media device120,140A-C may be directly connected to the network114or another network (e.g., for communications via in-band and/or out-of-band signals of a content delivery network) and/or connected to the network114via the gateway device124(e.g., for communications via a packet switched network). Each media device120,140A-C may implement one or more applications, such as content viewers, security applications, chat applications, telehealth applications, social media applications, news applications, gaming applications, content stores, electronic program guides, combinations thereof, and the like. Those skilled in the art will appreciate that the signal may be demodulated and/or decoded in a variety of equipment, including the gateway device124, a computer, a TV, a monitor, or a satellite dish.

The system100may comprise the gateway device124. The gateway device124may comprise a local gateway (e.g., router, modem, switch, hub, combinations thereof, and the like) configured to connect (or facilitate a connection (e.g., a communication session) between) a local area network (e.g., a LAN) to a wide area network (e.g., a WAN) such as the network114. The gateway device124may be associated with the premises130. The gateway device124may configured to receive incoming data (e.g., data packets or other signals) from the network114and route the data to one or more other devices associated with the premises130(e.g., the media device120, the display121, the user devices (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), combinations thereof, and the like. The gateway device124may be configured to communicate with the network114. The gateway device124may be configured for communication with the network114via a variety of protocols, such as IP, transmission control protocol, file transfer protocol, session initiation protocol, voice over IP (e.g., VoIP), combinations thereof, and the like. The gateway device124, for a cable network, may be configured to facilitate network access via a variety of communication protocols and standards, such as Data Over Cable Service Interface Specification (DOCSIS).

The gateway device124may be configured to receive the audio data (e.g., user voice data) from the media device120and send the audio data to the computing device102and/or the computing device106via the network114or another network. The gateway device124may be configured to receive commands or code associated with one or more commands from the computing device102and/or security data from the computing device106, via the network114or another network, and may send the received data to the media device120. The gateway device124may be configured to receive audio data from the media device120, via the network114or another network, and may send the received audio data to the one or more media devices140A-C. The gateway device124may be configured to receive audio data from one or more of the media devices140A-C, via the network114or another network, and may send the received audio data to the media device120.

The system100may also comprise one or more user devices. For example, a user may receive content (e.g., audio data, video data, or audio/video data) from the network114on the user device. For example, the user device may send the audio data (e.g., user voice data) to the media device120. The user device may be configured to communicate with the media device120via wireless communications. For example, the wireless communications may be one or more of Bluetooth, Bluetooth Low Energy, WiFi, radio frequency (RF), Zigbee, or another wireless communication protocol. The user device may be the voice-enabled remote control122or a mobile device123(e.g., a smart phone, a voice assistant device, a Bluetooth headset, a laptop computer, a tablet device, a computer station, a personal data assistant (PDA), a smart watch, smart glasses), a smart speaker, a portable media player, a combination thereof, or the like). The voice-enabled remote control may include a microphone within the body of the remote control122and may be configured to receive audio data (e.g., user voice data) from a user operating the remote control122. The audio data may take the form of one or more of commands, requests, search queries, or general discussion. For example, the remote control122may include a button or switch to activate the microphone on the remote control. Once activated, the user may speak into the microphone to provide the audio data (e.g., the user voice data). Each of the mobile devices123may also be voice-enabled and may be configured to receive the user voice data in a similar or different manner than described above with regard to the remote control122. The user device may communicate with a variety of gateways, such as the gateway124.

FIG.2is an example device architecture for the media device120for the management of communications between a user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), the media device120, and optionally other media devices140A-C, the computing device102, the computing device106, and/or one or more of the security devices (e.g., video doorbell132and/or security cameras138A-B).

The media device120may comprise a communications module202. For example, the communications module202may comprise a hardware abstraction for receiving communications from one or more of the user devices (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) may be configured to communicate with the media device120via wireless communications. For example, the wireless communications may be one or more of Bluetooth, Bluetooth Low Energy, WiFi, radio frequency (RF), Zigbee, or another wireless communication protocol. For example, the communications module202may be configured to receive non-verbal commands for adjusting or changing the content on a display device121(e.g., a display, a television, a computer or other similar device) associated with the media device120. For example, the communications module202may be configured to receive audio data (e.g., user voice data) from the user device. The audio data may comprise one or more of a command, a request, a query or other instruction associated the user and with the content or the display device121outputting the content. For example, the audio data may comprise an instruction to raise the volume, lower the volume, change the channel, search for a channel, search for a content item (e.g., a movie, television show, or sporting event) or other similar commands or instructions. The audio data may comprise discussion data received from the user device. For example, the discussion data may be responsive to voice and/or video data received from one or more of the security devices (e.g., video doorbell132and/or security cameras138A-B) or one or more of the other media devices140A-C.

The media device120may comprise one or more device drivers204. The device drivers204may comprise software and instructions for communicating with one or more of the user devices (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the communications module202may access one of the one or more device drivers204based on the type of user device that is currently trying to communication with the media device.

The media device120may comprise a controller206. The controller206may be configured to receive the audio data (e.g., the user voice data) from the user device via the communications module202. The controller206may be communicably coupled to one or more processors of the media device120. The controller206may be configured to evaluate the received audio data (e.g., from the user device) and determine if the received audio data should be evaluated to determine a command/instruction associated with the media device120or display device121or if the audio data is associated with a communications application (e.g., a security application, a telehealth application, a chat application, a watch party application). The controller206may make the determination on how to route the audio data based on information (or the lack of information) provided by the resident application220.

For example, the controller206may have a default instruction to determine that the received audio data is associated with a command for one of the media device120or display device121and forward the received audio data to the computing device102. The controller206may, based on the default instruction, forward the audio data to the computing device102so that the audio data (e.g., user voice data) may be converted to text and the command or instruction in the audio data may be determined by the computing device102. For example, the resident application220may notify or signal to the controller206that no communications applications are active. Based on the notification that not communications applications are active, the controller206may send the audio data to the computing device102so that the audio data may be converted to text and the command or instruction in the audio data may be determined by the computing device102. For example, the resident application220may notify or signal to the controller206that a communications application is active. Based on the indication that the communications application is active, the controller206may route the received audio data to the communications application as discussed below.

The media device120may comprise a resident application220. The resident application220may be configured to provide a user interface for the media device and optionally the display device121. For example, the resident application220may be configured to generate and cause to be displayed a content guide, content search functions, and/or available applications (e.g., media applications, communications applications (e.g., security applications, telehealth applications, chat applications, watch party applications), streaming applications, etc.) for selection. The resident application220may be configured to communicate with the available applications and determine if one or more of the available applications has been selected and/or is in use by the media device120. The resident application220may be configured to signal or notify the controller206that the one or more available application has been selected and/or is in use by the media device120.

The media device120may comprise a platform configuration application programming interface (API)222. The platform configuration API222may be configured to allow the resident application220to communicate with the controller206. For example, the resident application220may use the platform configuration API222to communicate with the controller206.

As discussed above, a user, via the user device, may launch, initiate, execute, or run a communications application or respond to a notification to connect, cause to connect, launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication pathway via the communications application. For example, a user, via a user device, may select to activate, execute, run, initiate, or launch one of the available communications applications, such as the doorbell/security application218, telehealth application, chat application, watch party application, etc. For example, the user may provide audio data or a non-verbal instruction to the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) to run, execute, activate, launch, or initiate one of the available communications applications. The resident application220may send a signal to the selected application to launch or start the selected application (e.g., the doorbell/security application218). Similarly, launching or initiating other communications applications for chatting with one or more other users associated with the other media device140A-C for telehealth, audio chatting, watch parties etc. may be similarly accomplished. Once the communications application (e.g., the doorbell/security application218) is launched, the resident application220may notify the controller206to route audio data (e.g., user audio data) received from the user devices to the open communications application.

For example, the media device120, via the doorbell/security application218or another communications application (e.g., telehealth, chat, watch party) may receive a notification of an activation associated with one or more of the security devices. For example, the activation may comprises a visitor activating or depressing the doorbell button, switch, or touchpad, or the security device detecting motion or sound. For other communications applications, the notification may be a request to initiate, run, begin, launch, or execute a chat session with the user via the user device and the media device120. The doorbell/security application218or the other communications application to be opened may send a signal or notification to the resident application220indicating the received notification or activation and requesting instructions to open, initiate, or launch a communications pathway between the user device and the security device via the media device120. For example, the user may accept the request (or provide an indication of acceptance of the request) based on an input at the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the input may comprise a button depressed by the user at the user device or the user providing a signal via audio data into the user device. For example, the signal may comprise audio data of the user stating “yes,” “open,” “accept,” or the like.

The media device120may comprise a web browser212. The web browser212may be associated with a WebRTC platform port216for receiving input data from the media device and outputting data received from a communications application to the media device120. The web browser212may comprise a WebRTC stack214. The WebRTC stack214may be configured to enable the web browser212to process and output voice, text, and/or video data received from the security devices (or for other communications applications, from one or more of the media devices140A-C) via the web browser212to, for example, the display device121.

For example, the doorbell/security application218, or another communications application, may be configured to connect to, cause to connect to, launch, cause to launch, run, cause to be run, execute, cause to be executed, initiate, or cause to be initiated a communication session between one or more of the security devices (e.g., video doorbell132and/or security cameras138A-B) and the media device120, via the computing device106and the network114or another network. For example, the open or launched communications application (e.g., the doorbell/security application218may provide a user interface via the web browser212and send audio data and/or video data received from the communications application to the web browser to cause the web browser to output the audio data and/or video data at the display device121. For example, the doorbell/security application218may be configured to receive security data (e.g., at least one of audio data and/or video data) from one or more of the security devices either directly from the security devices via the gateway124or via the computing device106, the network114(or another network) and the gateway124. The doorbell/security application218may be configured to output the received audio data and/or security data at the display device121via the web browser212.

The media device120may comprise a virtual microphone audio sink208and a sound card210. The virtual microphone audio sink208and the sound card210may be configured to provide audio data received from the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) to the open communications application (e.g., the doorbell/security application218) via the web browser212. For example, the audio sink208may allow audio data (e.g., user audio data) to be output through the sound card210to the doorbell/security application218(or another communications application) and output at the associated security device (or other media devices140A-C for other communications applications). For example, when the controller206has been notified by the resident application220to route audio data (e.g., user audio data) received from the user devices to the open communications application, the controller206, upon receiving the audio data from the user device and through the communications module202, may send the audio data to the audio sink208for processing through the sound card210.

For example, the sound card210may be a virtual sound card (e.g., a sound card presenting via software in the media device120. For example, the sound card210simulates a microphone virtually in the software of the media device by converting the received audio data from the user device (e.g., the user voice data) so that communications applications operating on the media device120can access the audio data as if the media device itself had a physical microphone. For example, the sound card210may be a virtual loop-back sound card. As shown inFIG.3, the sound card210may form a full-duplex loopback virtual sound card. At a first portion of the sound card210the received audio data from the user device may be received at an audio input port302as if it is coming in from a physical microphone of the media device. At another portion of the sound card210, the received audio data may be output through an audio output port304to the open communications application (e.g., the doorbell/security application218, the telehealth application310, the chat application315, the watch party application320, or the like). The loopback within the virtual sound card210may create, for example, two devices within the virtual sound card210. Each of the devices can be used as either an audio playback or audio capture device. The portion of the sound card210associated with the audio input port302, recognizes the audio data being received as being received from a microphone, even though the media device120may not include the microphone. The portion of the sound card210associated with the audio output port304, recognizes the audio data going to a speaker (when it is initially being sent to the communications application operating on the web browser212.

FIG.4shows a flowchart of an example method400for managing communications. The methods described inFIG.4may be completed by a computing device (e.g., the media device120or any other computing device described herein). While the method400ofFIG.4will describe as being completed by the media device120, this is for example purposes only, as all or any other portion of the method400may be completed by any other computing device described herein.

At410, first audio data may be received via a first communication session. The first audio data may be received by the media device120. For example, a user may speak into a microphone of a user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). The user device may convert the spoken language into the first audio data. The user device may send the first audio data to the media device120. For example, the first audio data may be sent to the media device120via a wireless communication, such as Bluetooth, BLE, WiFi, RF, Zigbee or any other wireless communication. For example, a controller206of the media device120may receive the first audio data from the user device via the communications module202.

At420, the first audio data may be determined to be associated with a command for one of the media device120or a display device121. For example, the determination may be made by the media device120. For example, the first audio data may be determined to be associated with a command for one of the media device120or the display device121based on a default instruction to determine that the first audio data is associated with a command for one of the media device120or display device121. For example, the first audio data may be determined to be associated with a command for one of the media device120or the display device121based on an indication or notification that a communications application for the media device120is not open or launched. For example, the resident application220of the media device120may determine the status of the one or more communications applications for the media device. For example, the resident application may determine that the communications applications have not been opened or launched and may send a notification to the controller206indicating the status of the communications applications.

At430, the first audio data may be sent for analysis. For example, the first audio data may be sent to a remote computing device. For example, the first audio data may be to the computing device102, such as a cloud computing device. The computing device102may receive the first audio data. The computing device102may use a speech-to-text engine108to convert the first audio data to text data.

The computing device102may use a command engine110to analyze the converted text data to determine one or more commands or actions being requested in the received audio data. For example, the command engine110may receive text that states “change to channel38”. The command engine110may compare one or more of the words or phrases in the text to a stored database of commands or actions to determine the command or action being requested. For example, the command engine110may parse the text into one or more words or phrases and compare the parsed portions of the text to the stored database of commands or actions. The command engine110may determine that a user associated with media device120and/or user device wishes to change the channel shown on the display device121to channel38. The computing device102(e.g., the command engine110) may determine the code or instructions to provide to the media device120in order to change the channel output on the display device121from the current channel to channel38. The command engine110or the computing device102may send the code or instructions to the media device120.

For example, the media device120may receive the code or instructions associated with the requested command in the first audio data and may execute the command on the media device120and/or cause the command to be executed on the display device121. Examples of commands may include, but are not limited to, instruction to raise the volume, lower the volume, turn on display device121, turn off display device121, change the channel, search for a channel, search for a content item (e.g., a movie, television show, or sporting event) or other similar commands or instructions. The first communication session may then be closed or otherwise terminated.

A visitor (e.g., another user) may approach one of the security devices (e.g., video doorbell132and/or security cameras138A-B) on the premises130. For example, the visitor may activate one or more of the security devices. For example activating the security device may comprise waking the security device or a portion of the security device from a sleep state. For example, if the security device is a video doorbell132, the visitor may activate the video doorbell132by pressing or engaging the activation device136(e.g., a doorbell button, a doorbell switch, a doorbell touchpad, a capacitive sensor, etc.) or triggering a motion sensor or audio sensor associated with the video doorbell132. For example, if the security device is a security camera138A-B, the visitor may activate the video camera138A-B by triggering a motion sensor or audio sensor associated with the particular video camera138A-B. For example, a visitor activating the security device may, in the case where the security device is a video doorbell132, activate a doorbell chime, the camera134, and the microphone135to begin recording the security data and in the case where the security device is a security camera138A-B, activate the camera and microphone associated with the particular security camera138A-B. Each security device may be configured to record security data such as at least one of video data, via the camera, and audio data, via the microphone.

The media device120may receive a notification of an activation associated with one or more of the security devices. For example, the security device may send the activation notification to the computing device106via the network114or another network. For example, the activation notification may include security data (e.g., one or more of audio data or video data) recorded at the particular security device. The computing device106may evaluate the metadata associated with the notification and/or the security data and determine a user or location associated with the notification or security data. The computing device may compare the user and/or location information to device data109in the computing device106, to determine the media device120, network address, or location of the media device120. The computing device106may send the notification and, if included, the security data to the media device120associated with the same user and or associated with the security devices.

The media device120may receive the notification of activation of the security device. The doorbell/security application218may send a signal or notification to the resident application220indicating the received notification of activation and requesting instructions to open, launch, execute, or initiate a communications pathway between the user device and the security device via the media device120. The resident application220may display a message associated with the notification of activation on the display device121and request instructions for initiating the communication pathway between the user device and the security device. For example, the user may accept the request (or provide an indication of acceptance of the request) based on an input at the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the input may comprise a button depressed by the user at the user device or the user providing a signal via audio data into the user device. For example, the signal may comprise audio data of the user stating “yes,” “open,” “accept,” or the like.

The security device (e.g., video doorbell132and/or security cameras138A-B) may record security data (e.g., security audio data and/or security video data). For example, the security audio data may comprise voice data from the visitor received via the microphone135. Once the communications pathway between the user device and the security device, via the media device120, is opened, the security data may be sent by the security device to the computing device106via the network114and then to the media device120via the network114or the security data may be sent by the security device to the media device120via the gateway124. The security data may be received by the media device120(e.g., by the doorbell/security application218). The doorbell/security application218may output or cause to be output the security data at the media device and/or the associated display device121. For example, the doorbell/security application218may cause the security data to be output via the web browser212. For example, the display device121may output both security audio data and security video data of the security data.

At440, second audio data may be received via the second communication session. The second audio data may be received by the media device120. For example, a user may speak into the microphone of the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example the second audio data may be received prior to or in response to the all or a portion of the security data caused to be displayed at the display device121.

The user device may convert the spoken language into the second audio data. The user device may send the second audio data to the media device120. For example, the second audio data may be sent to the media device120via wireless communication, such as Bluetooth, BLE, WiFi, RF, Zigbee or any other wireless communication. For example, the controller206of the media device120may receive the second audio data from the user device via the communications module202.

At450, the second audio data may be determined to be associated with a security device. For example, the determination may be made by the media device120(e.g., the controller206). For example, when the doorbell/security application218is launched and the user accepts the request (or provides an indication of acceptance of the request) to open a communication pathway between the user device and the security device, the resident application220of the media device120can determine that one of the communications applications (e.g., the doorbell security application218) has launched and is active. The resident application220may send a signal or notification to the controller206of the media device120to route audio data (e.g., user audio data) received from the user devices to the open communications application (e.g., the doorbell/security application218). For example, based on the open communications application being the doorbell/security application218, the media device120(e.g., the controller206) may determine that the second audio data is associated with one or more of the security devices.

At460, the second audio data may be sent to the security device. For example, the second audio data may be sent by the media device120and received by one or more of the security devices (e.g., the video doorbell132and/or security cameras138A-B). For example, while the doorbell/security application218is in use, the second audio data may be sent directly from the media device120to the one or more security devices via the doorbell/security application218. For example, while the doorbell/security application218is in use, the second audio data may be sent to the one or more security devices via the gateway124, the network114, and the second computing device106via the doorbell/security application218. For example, the second audio data may be sent to the one or more security devices via the gateway124and then to the particular security device. For example, the second audio data may be sent to the one or more security devices based on a determination that the second audio data is associated with at least one of the security devices.

For example, upon receiving the second audio data from the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), the controller206may send the second audio data to the audio sink208for processing through the virtual sound card210. The virtual sound card210may receive the second audio data and act as a virtual microphone for the media device120. The virtual sound card210may send the second audio data to the doorbell/security application218via the web browser212. The doorbell/security application may send the received second audio data to the one or more security devices either directly, via the gateway124or via the gateway124, the network114, and the computing device106.

The one or more security devices (e.g., the video doorbell132and/or security cameras138A-B) may receive the second audio data (e.g., via the transceiver of the security device). The one or more security devices may output the second audio data at the security device (e.g., via a speaker associated with the particular security device of the one or more security devices).

While the second communication session and the communication pathway between the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) and the one or more security devices (e.g., video doorbell132and/or security cameras138A-B) remain open and active, security data can continue to be sent by the security devices to the media device120for output on the associated display device121and additional audio data may be received by the media device from the user device and routed to the one or more security devices to be output at the one or more security devices.

The second communication session, and thus the communication pathway between the user device and the one or more security devices, may be terminated. For example the communication session may be terminated by one or more of the media device120, the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), or the security device (e.g., the video doorbell132and/or security cameras138A-B). For example, the second communication session may terminate upon the expiration of a timeout timer. The timeout timer may be a threshold amount of time during which no audio data or security data is being passed from one device to another during the second communication session. For example, the second communication session may terminate based on a verbal or non-verbal command received by the media device120from the user device. For example, the user may press a button on the user device or submit a voice command to close the doorbell/security application218.

The resident application220, based on the second communication session or the communication pathway being terminated, may send a notification or instruction to the controller206of the media device that there are no communication applications open. Based on the notification from the resident application220, the controller206may revert back to sending all received audio data from the user device to the computing device102to determine the command or instruction requested in the received audio data.

FIG.5shows a flowchart of an example method500for managing communications. The methods described inFIG.5may be completed by a computing device (e.g., the media device120or any other computing device described herein). While the method500ofFIG.5will describe as being completed by the media device120, this is for example purposes only, as all or any other portion of the method500may be completed by any other computing device described herein.

A visitor (e.g., another user) may approach the video doorbell132on the premises130. For example, the visitor may activate the video doorbell132. For example activating the video doorbell132may comprise waking the video doorbell132or a portion of the video doorbell132from a sleep state. For example, the visitor may activate the video doorbell132by pressing or engaging the activation device136(e.g., a doorbell button, a doorbell switch, a doorbell touchpad, a capacitive sensor, etc.) or triggering a motion sensor or audio sensor associated with the video doorbell132. For example, activating the video doorbell132may cause the video doorbell132to activate a doorbell chime, the camera134, and the microphone135to begin recording the security data. For example, the video doorbell132may be configured to record security data such as video data, via the camera, and audio data, via the microphone.

At510, an activation signal may be received. For example, the activation signal (e.g., a notification that a visitor is at the video doorbell) may be associated with the video doorbell132. For example, the activation signal may be received by the media device120. For example, the video doorbell132may send the activation signal to the computing device106via the network114or another network. For example, the activation signal may include security data (e.g., one or more of audio data or video data) recorded at the video doorbell132. The computing device106may evaluate the metadata associated with the notification and/or the security data and determine a user or location associated with the notification or security data. The computing device106may compare the user and/or location information to device data109at the computing device106, to determine the media device120, network address, or location of the media device120. The computing device106may send the activation signal and, if included, the security data to the media device120associated with the same user and or associated with the video doorbell132via the network114. The media device120may receive the activation signal of the video doorbell132from the computing device106.

At520, a communications application may be initiated. For example, initiating the communications application may comprise launching the communications application or causing the communications application to be launched. For example, initiating the communications application may comprise running the communications application or causing the communications application to be run. For example, initiating the communications application may comprise executing the communications application or causing the communications application to be executed.

For example, the communications application may be initiated, run, executed, or launched by the media device120. For example, the media device120may determine which communications application to initiate, run, execute, or launch. For example, determining which communications application to initiate, run, execute, or launch may be based on the activation signal received by the media device120. For example, the media device120may determine that the activation signal received is for (or ultimately from) the video doorbell132. Based on this determination, the media device120may determine that the communications application to initiate, run, execute, or launch is the doorbell/security application218. The resident application220, or another portion of the media device120may notify or send an instruction to the doorbell/security application218to open or launch.

At530, a communication session between the user device and the video doorbell may be established. For example, establishing the communications session may comprise launching the communications session or causing the communications session to be launched. For example, establishing the communications session may comprise running the communications session or causing the communications session to be run. For example, establishing the communications session may comprise executing the communications session or causing the communications session to be executed.

For example, the communication session may be established by the media device120. For example, the media device120(e.g., the resident application220) may determine the communications application (e.g., the doorbell/security application218) has initiated, run, been executed, or launched and is open; is launching, running, executing; or has a request to be initiated, executed, run, or launched. The determination may be based on the received activation signal. The resident application220of the media device120may retrieve or generate a notification of the received activation signal, a notification of a communication request associated with the activation signal or audio data from the video doorbell132, and/or a request for instructions to open or establish a communication session between the user device and the video doorbell132via the media device120. The resident application220may display or cause to be displayed a message associated with the activation signal on the display device121and request instructions for establishing the communication session between the user device and the video doorbell132. For example, the user may accept the request (or provide an indication of acceptance of the request) based on an input at the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the input may comprise a button depressed by the user at the user device or the user providing a signal via audio data into the user device. For example, the signal may comprise audio data of the user stating “yes,” “open,” “accept,” or the like.

The video doorbell132may record security data (e.g., security audio data and/or security video data). For example, the security audio data may comprise voice data from the visitor received via the microphone135. For example, the security video data may comprise video data received from the camera134. For example, the security video data may comprise one or more of video and still pictures. Once the communications session between the user device and the security device, via the media device120, is opened, the security data may be sent by the video doorbell132to the computing device106via the network114and then to the media device120via the network114or the security data may be sent by the video doorbell132to the media device120via the gateway124. The security data may be received by the media device120(e.g., by the doorbell/security application218). The doorbell/security application218may output or cause to be output the security data at the media device120and/or the associated display device121. For example, the doorbell/security application218may cause the security data to be output via the web browser212. For example, the display device121may output both security audio data and security video data of the security data.

At540, user audio data may be received. The user audio data may be received by the media device120from a user device. For example, a user may speak into the microphone of the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example the audio data may be received prior to or in response to the all or a portion of the security data caused to be displayed at the display device121.

The user device may convert the spoken language into the audio data. The user device may send the audio data to the media device120. For example, the audio data may be sent to the media device120via wireless communication, such as Bluetooth, BLE, WiFi, RF, Zigbee or any other wireless communication. For example, the controller206of the media device120may receive the audio data from the user device via the communications module202.

For example, the audio data may be determined to be associated with the video doorbell132. For example, the determination may be made by the media device120(e.g., the controller206). For example, when the doorbell/security application218is launched and the user accepts the request to open a communication session between the user device and the security device, the resident application220of the media device120can determine that one of the communications applications (e.g., the doorbell security application218) has launched and is active. The resident application220may send a signal or notification to the controller206of the media device120to route the audio data (e.g., user audio data) received from the user devices to the open communications application (e.g., the doorbell/security application218). For example, based on the open communications application being the doorbell/security application218, the media device120(e.g., the controller206) may determine that the audio data is associated with the video doorbell132.

At550, the user audio data may be sent to the video doorbell132. For example, the user audio data may be sent by the media device120and received by the video doorbell132. For example, while the doorbell/security application218is in use, the user audio data may be sent directly from the media device120to the video doorbell132via the doorbell/security application218. For example, while the doorbell/security application218is in use, the user audio data may be sent to the video doorbell132via the gateway124, the network114, and the second computing device106. For example, while the doorbell/security application218is in use, the user audio data may be sent to the video doorbell132via the gateway124. For example, the user audio data may be sent to the video doorbell132based on a determination that the user audio data is associated with the video doorbell132(which may be based on the fact that the doorbell/security application218is currently in use).

For example, upon receiving the user audio data from the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), the controller206may send the user audio data to the audio sink208for processing through the virtual sound card210. The virtual sound card210may receive the user audio data and act as a virtual microphone for the media device120. The virtual sound card210may send the user audio data to the doorbell/security application218via the web browser212. The doorbell/security application218may send the received user audio data to the video doorbell132either directly, via the gateway124or via the gateway124, the network114, and the computing device106.

The video doorbell132may receive the user audio data (e.g., via the transceiver of the security device). The video doorbell132may output the user audio data at the video doorbell132(e.g., via a speaker associated with the video doorbell132).

While the communication session and the communication pathway between the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) and the video doorbell132remain open and active, security data can continue to be sent by the video doorbell132to the media device120for output on the associated display device121and additional audio data may be received by the media device from the user device and routed to the video doorbell132to be output at the video doorbell132.

The communication session, and thus the communication pathway between the user device and the video doorbell132, may be terminated. For example the communication session may be terminated by one or more of the media device120, the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), or the video doorbell132. For example, the communication session may terminate upon the expiration of a timeout timer. The timeout timer may be a threshold amount of time during which no audio data or security data is being passed from one device to another during the communication session. For example, the communication session may terminate based on a verbal or non-verbal command received by the media device120from the user device. For example, the user may press a button on the user device or submit a voice command to close the doorbell/security application218.

The resident application220, based on the communication session or the communication pathway being terminated, may send a notification or instruction to the controller206of the media device that there are no communication applications open. Based on the notification from the resident application220, the controller206may revert back to sending all received audio data from the user device to the computing device102to determine the command or instruction requested in the received audio data.

FIG.6shows a flowchart of an example method600for managing communications. The methods described inFIG.6may be completed by a computing device (e.g., the media device120or any other computing device described herein). While the method600ofFIG.6will describe as being completed by the media device120, this is for example purposes only, as all or any other portion of the method600may be completed by any other computing device described herein.

A visitor (e.g., another user) may approach or be within a viewing window of a security camera138A-B on the premises130. For example, the visitor may activate the security camera138A-B. For example activating the security camera138A-B may comprise waking the security camera138A-B or a portion of the security camera138A-B from a sleep state. For example, the visitor may activate the security camera138A-B by triggering a motion sensor or audio sensor associated with the security camera138A-B. For example, activating the security camera138A-B may cause the security camera138A-B to activate a camera and a microphone associated with the security camera138A-B to begin recording security data. For example, the security camera138A-B may be configured to record security data such as video data, via the camera, and audio data, via the microphone.

At610, an activation signal may be received. For example, the activation signal (e.g., a notification that a visitor has triggered or awaken the security camera138A-B) may be associated with the security camera138A-B. For example, the activation signal may be received by the media device120. For example, the security camera138A-B may send the activation signal to the computing device106via the network114or another network. For example, the activation signal may include security data (e.g., one or more of audio data or video data) recorded at the security camera138A-B. The computing device106may evaluate the metadata associated with the notification and/or the security data and determine a user or location associated with the notification or security data. The computing device106may compare the user and/or location information to device data109at the computing device106, to determine the media device120, network address, or location of the media device120. The computing device106may send the activation signal and, if included, the security data to the media device120associated with the same user and or associated with the security camera138A-B via the network114. The media device120may receive the activation signal of the security camera138A-B from the computing device106.

At620, a communications application may be initiated. For example, initiating the communications application may comprise launching the communications application or causing the communications application to be launched. For example, initiating the communications application may comprise running the communications application or causing the communications application to be run. For example, initiating the communications application may comprise executing the communications application or causing the communications application to be executed.

For example, the communications application may be initiated, run, executed, or launched by the media device120. For example, the media device120may determine which communications application to initiate, run, execute, or launch. For example, determining which communications application to initiate, run, execute, or launch may be based on the activation signal received by the media device120. For example, the media device120may determine that the activation signal received is for (or ultimately from) the security camera138A-B. Based on this determination, the media device120may determine that the communications application to initiate, run, execute, or launch is the doorbell/security application218. The resident application220, or another portion of the media device120may notify or send an instruction to the doorbell/security application218to initiate, run, execute, or launch.

At630, a communication session between the user device and the security camera138A-B may be established. For example, establishing the communications session may comprise launching the communications session or causing the communications session to be launched. For example, establishing the communications session may comprise running the communications session or causing the communications session to be run. For example, establishing the communications session may comprise executing the communications session or causing the communications session to be executed.

For example, the communication session may be established by the media device120. For example, the media device120(e.g., the resident application220) may determine the communications application (e.g., the doorbell/security application218) has initiated, run, executed, or launched and is open; is executing, running, initiating, or launching; or has a request to be initiated, executed, run, or launched. The determination may be based on the received activation signal. The resident application220of the media device120may retrieve or generate a notification of the received activation signal, a notification of a communication request associated with the activation signal or audio data from the security camera138A-B, and/or a request for instructions to open or establish a communications session between the user device and the security camera138A-B via the media device120. The resident application220may display or cause to be displayed a message associated with the activation signal on the display device121and request instructions for initiating the communication session between the user device and the security camera138A-B. For example, the user may accept the request (or provide an indication of acceptance of the request) based on an input at the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the input may comprise a button depressed by the user at the user device or the user providing a signal via audio data into the user device. For example, the signal may comprise audio data of the user stating “yes,” “open,” “accept,” or the like.

The security device138A-B may record security data (e.g., security audio data and/or security video data). For example, the security audio data may comprise voice data from the visitor received via the microphone associated with the security camera138A-B. For example, the security video data may comprise video data received from the camera associated with the security camera138A-B. For example, the security video data may comprise one or more of video and still pictures. Once the communications session between the user device and the security camera138A-B, via the media device120, is opened, the security data may be sent by the security camera138A-B to the computing device106via the network114and then to the media device120via the network114or the security data may be sent by the security camera138A-B to the media device120via the gateway124. The security data may be received by the media device120(e.g., by the doorbell/security application218). The doorbell/security application218may output or cause to be output the security data at the media device120and/or the associated display device121. For example, the doorbell/security application218may cause the security data to be output via the web browser212. For example, the display device121may output both security audio data and security video data of the security data.

At640, user audio data may be received. The user audio data may be received by the media device120from a user device. For example, a user may speak into the microphone of the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example the audio data may be received prior to or in response to the all or a portion of the security data caused to be displayed at the display device121.

The user device may convert the spoken language into the audio data. The user device may send the audio data to the media device120. For example, the audio data may be sent to the media device120via wireless communication, such as Bluetooth, BLE, WiFi, RF, Zigbee or any other wireless communication. For example, the controller206of the media device120may receive the audio data from the user device via the communications module202.

For example, the audio data may be determined to be associated with the security camera138A-B. For example, the determination may be made by the media device120(e.g., the controller206). For example, when the doorbell/security application218is launched and the user accepts the request to open a communication session between the user device and the security camera138A-B, the resident application220of the media device120can determine that one of the communications applications (e.g., the doorbell security application218) has launched and is active. The resident application220may send a signal or notification to the controller206of the media device120to route the audio data (e.g., user audio data) received from the user devices to the open communications application (e.g., the doorbell/security application218). For example, based on the open communications application being the doorbell/security application218, the media device120(e.g., the controller206) may determine that the audio data is associated with the security camera138A-B.

At650, the user audio data may be sent to the triggered security camera138A-B. For example, the user audio data may be sent by the media device120and received by the security camera138A-B. For example, while the doorbell/security application218is in use, the user audio data may be sent directly from the media device120to the triggered security camera138A-B via the doorbell/security application218. For example, while the doorbell/security application218is in use, the user audio data may be sent to the security camera138A-B via the gateway124, the network114, and the second computing device106. For example, while the doorbell/security application218is in use, the user audio data may be sent to the security camera138A-B via the gateway124. For example, the user audio data may be sent to the security camera138A-B based on a determination that the user audio data is associated with the security camera138A-B (which can be determined based on the doorbell/security application218being in use by the media device120).

For example, upon receiving the user audio data from the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), the controller206may send the user audio data to the audio sink208for processing through the virtual sound card210. The virtual sound card210may receive the user audio data and act as a virtual microphone for the media device120. The virtual sound card210may send the user audio data to the doorbell/security application218via the web browser212. The doorbell/security application218may send the received user audio data to the security camera138A-B either directly, via the gateway124or via the gateway124, the network114, and the computing device106.

The security camera138A-B may receive the user audio data (e.g., via the transceiver of the security camera138A-B). The security camera138A-B may output the user audio data at the security camera138A-B (e.g., via a speaker associated with the security camera138A-B).

While the communication session and the communication pathway between the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) and the security camera138A-B remain open and active, security data can continue to be sent by the security camera138A-B to the media device120for output on the associated display device121and additional audio data may be received by the media device from the user device and routed to the security camera138A-B to be output at the security camera138A-B.

The communication session, and thus the communication pathway between the user device and the security camera138A-B, may be terminated. For example the communication session may be terminated by one or more of the media device120, the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), or the security camera138A-B. For example, the communication session may terminate upon the expiration of a timeout timer. The timeout timer may be a threshold amount of time during which no audio data or security data is being passed from one device to another during the communication session. For example, the communication session may terminate based on a verbal or non-verbal command received by the media device120from the user device. For example, the user may press a button on the user device or submit a voice command to close the doorbell/security application218.

The resident application220, based on the communication session or the communication pathway being terminated, may send a notification or instruction to the controller206of the media device that there are no communication applications open. Based on the notification from the resident application220, the controller206may revert back to sending all received audio data from the user device to the computing device102to determine the command or instruction requested in the received audio data.

FIG.7shows a flowchart of an example method700for managing communications. The methods described inFIG.7may be completed by a computing device (e.g., the media device120or any other computing device described herein). While the method700ofFIG.7will describe as being completed by the media device120, this is for example purposes only, as all or any other portion of the method700may be completed by any other computing device described herein.

A remote user (e.g., a user located at a different premises than the premises130of the media device120) may access one of the remote media devices140A-C. For example, the remote user may access the remote media device140A using another user device (e.g., another voice-enabled remote control, another mobile device, another voice assistant device, another smart speaker, another Bluetooth headset, etc.). The remote user may request to establish (e.g., initiate, cause to initiate, run, cause to be run, execute, cause to be executed, launch, or cause to be launched, etc.) a communication session with another media device (e.g., the media device120). For example, the communication session may be one of a chat session, a watch party, or a telehealth review. For example, the remote user may launch a communications application that corresponds to the communication session desired at the remote media device140A. For example, if the remote user wants to establish a telehealth review, the remote user may use a user device to select the telehealth application310to be launched at the remote media device140A. For example, if the remote user wants to establish a chat session, the remote user may use a user device to select the chat application315to be launched at the remote media device140A. For example, if the remote user wants to establish a watch party with one or more other users, the remote user may use a user device to select the watch party application320to be launched at the remote media device140A.

The remote user, via the user device, may select the other users to request or invite to be a part of the communication session. The number of other users invited or requested to be a part of the communication session can be one or greater than one. For example, the launched communication application on the remote media device140A may request an indication of which other users to invite to the communication session. The remote user may input user identifying information via the user device to the remote media device140A for the opened communication application. For example, the user identifying information can be any information that identifies another user and/or the media device, URL, or MAC address associated with that other user. The remote user may selected the number of other users to request or invite to the communication session and indicate to the remote media device140A to send out an indication of the request or invitation. The remote media device140A may send a notification of request or invitation to join the communication session to the identified other users via, for example, the communications application on the other media devices (e.g., the media devices140B-C,120.

At710, a notification to begin a communication session may be received. For example, the notification (e.g., a notification of invitation or request to join a communication session from another user) may be associated with the user sending the request and the communications application the user is employing to send the request. For example, the notification may be received by the media device120. For example, the remote media device140A may send the notification to the media device120via the network114or another network. For example, the notification may include remote user media data (e.g., one or more of remote user audio data or remote user video data) recorded at a remote of the remote user and or at the remote media device140A or display associated with the remote media device140A. The media device120may receive the notification of the request or invitation to join the communication session from the remote media device140A via the network114or another network.

At720, the notification is determined to be associated with a communications application. For example, the notification may be determined to be associated with the particular communications application by the media device120(e.g., the resident application220). For example, the media device120may parse the notification and identify from within the notification the remote user sending the request and the communications application to be used as part of the requested communication session. For example, the notification may identify the communications application and may be associated with particular communications application. Examples of the communications application include, but are not limited to, a telehealth application310, a chat application315, and a watch party application320. Each of these communications applications may be configured to send and receive one or both of video data and audio data, which may be output by the media device120or a display121associated with the media device120. For example, each of these communications applications may receive the audio data from a user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) and may receive the video data from a camera associated with the media device and/or display associated with the media device.

At730, the communications application associated with the notification may be initiated or opened. For example, the communications application may be initiated or opened by the media device120. For example, initiating the communications application may comprise launching the communications application or causing the communications application to be launched. For example, initiating the communications application may comprise running the communications application or causing the communications application to be run. For example, initiating the communications application may comprise executing the communications application or causing the communications application to be executed.

For example, the media device120may determine which communications application to initiate or open. For example, determining which communications application to initiate or open may be based on the notification received by the media device120. For example, the media device120(e.g., the resident application220) may determine that the notification includes a request to use or is sent by a particular communications application. Based on this determination, the media device120may determine which communications application to open or initiate within the media device120. The resident application220, or another portion of the media device120may notify or send an instruction to the particular communications application (e.g., the telehealth application310, the chat application315, the watch party application320, or another similar communications application) to open or initiate.

A communication session between the user device and media device120and the remote media device140A (and optionally an associated user device of the remote media device140A) associated with the remote user may be established. For example, establishing the communications session may comprise launching the communications session or causing the communications session to be launched. For example, establishing the communications session may comprise running the communications session or causing the communications session to be run. For example, establishing the communications session may comprise executing the communications session or causing the communications session to be executed. For example, establishing the communications may comprise opening the communications session or causing the communications session to be opened.

For example, the communication session may be established by the media device120and/or the remote media device140A. For example, the media device120(e.g., the resident application220) may determine the communications application associated with the notification has launched and is open, is launching, or has a request to be launched. The determination may be based on the received notification. The resident application220of the media device120may retrieve or generate a notification of the received notification and request or invitation to open or establish a communications session. The resident application220may display a message associated with the notification on the display device121associated with the media device120and request instructions for establishing the communication session between the user device and media device120and the media device140A. For example, the user may accept the request (or provide an indication of acceptance of the request) based on an input at the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example, the input may comprise a button depressed by the user at the user device or the user providing a signal via audio data into the user device. For example, the signal may comprise audio data of the user stating “yes,” “open,” “accept,” or the like.

At740, remote media data (e.g., remote audio data and/or remote video data) may be received. For example, the remote media data may be received by the media device120from the remote media device140A via the network114or another network. For example, the remote user may record the remote media data using a user device associated with the remote media device140A and/or a camera associated with the user device or display associated with the remote media device140A. For example, the remote audio data may comprise voice data from the remote user received via the user device associated with the remote media device140A. For example, the remote media device140A may include a virtual sound card substantially the same as described with the media device120and may similarly act as a virtual microphone with regard to providing the received remote audio data to the communications application for sending from the remote media device140A to the media device120. Once the communications session between the user device and media device120and the remote media device140A is opened, the remote media data may be sent by the remote media device140A to the media device120. The remote media data may be received by the media device120(e.g., by the opened communications application).

At750, the remote media data may be caused to be output. For example, the remote media data may be output or caused to be output by the media device120. For example, the remote media data may be output or caused to be output by the particular communications application associated with the opened communications session between the media device120and the remote media device140A. The particular communications application may output or cause to be output the remote media data at the media device120and/or the associated display device121. For example, the particular communications application may cause the remote media data to be output via the web browser212. For example, the display device121may output both the remote audio data and the remote video data of the remote media data.

At760, local user audio data may be received. The local user audio data may be received by the media device120from a user device. For example, a user may speak into the microphone of the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.). For example the audio data may be received prior to or in response to the all or a portion of the remote media data caused to be output at the display device121.

The user device may convert the spoken language into the local user audio data. The user device may send the local user audio data to the media device120. For example, the local user audio data may be sent to the media device120via wireless communication, such as Bluetooth, BLE, WiFi, RF, Zigbee or any other wireless communication. For example, the controller206of the media device120may receive the local user audio data from the user device via the communications module202.

For example, the local user audio data may be determined to be associated with the open/active communications application. For example, the determination may be made by the media device120(e.g., the controller206). For example, when the particular communications application is launched and the user accepts the request to open a communication session between the media device120and the remote media device140A, the resident application220of the media device120can determine that one of the communications applications has launched and is active. The resident application220may send a signal or notification to the controller206of the media device120to route the audio data (e.g., local user audio data) received from the user devices to the open communications application. For example, based on the open communications application, the media device120(e.g., the controller206) may determine that the local audio data is associated with the particular open communications application.

At770, the local audio data may be sent to a remote media device (e.g., the remote media device140A). For example, the local user audio data may be sent by the media device120and received by the remote media device140A. For example, the local user audio data may be sent to the remote media device140A via the gateway124and the network114. For example, the local user audio data may be sent to the remote media device140A based on a determination that the local user audio data is associated with the communications session.

For example, upon receiving the local user audio data from the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), the controller206may send the local user audio data to the audio sink208for processing through the virtual sound card210. The virtual sound card210may receive the local user audio data and act as a virtual microphone for the media device120. The virtual sound card210may send the local user audio data to the open communications application via the web browser212. The particular communications application may send the received local user audio data to the remote media device140A via the gateway124and the network114.

The remote media device140A may receive the local user audio data (and optionally local user video data). The remote media device140A may output the local user audio data (and optionally the local user video data) at the remote media device140A or a display associated with the local media device140A). While the example describes receiving remote media data from a single remote media device and sending local audio data to a single remote media device, this is for example purposes only. In operation, the chat, watch party, and or telehealth communications applications (or other communications application) may involve a plurality of users associated with a plurality of remote media devices. As such, within a single communications session, the media device120may receive remote media data from a plurality of remote media devices and may send the local audio data to the plurality of remote media devices associated with a particular communications session.

While the communication session between the media device120and the remote media device140A remain open and active, remote media data can continue to be sent by the remote media device140A to the media device120for output on the associated display device121and additional local audio data may be received by the media device120from the user device and routed to the remote media device140A to be output at the remote media device140A or an associated display device.

The communication session may be terminated. For example the communication session may be terminated by one or more of the media device120, the user device (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.), or the remote media device140A. For example, the communication session may terminate upon the expiration of a timeout timer. The timeout timer may be a threshold amount of time during which no local audio data or remote media data is being passed from one device to another during the communication session. For example, the communication session may terminate based on a verbal or non-verbal command received by the media device120from the user device. For example, the user may press a button on the user device or submit a voice command to close the opened communications application.

The resident application220, based on the communication session being terminated, may send a notification or instruction to the controller206of the media device that there are no communication applications open. Based on the notification from the resident application220, the controller206may revert back to sending all received audio data from the user device to the computing device102to determine the command or instruction requested in the received audio data.

FIG.8shows a system800for managing communications. Any one or more of the content source104, computing device102, computing device106, media devices120,140A-C, gateway124, user devices (e.g., the voice-enabled remote control122, mobile device123, the voice assistant device, the smart speaker, the Bluetooth headset, etc.) or security device (e.g., video doorbell132and/or security cameras138A-B) ofFIG.1may be a computer801as shown inFIG.8.

The computer801may comprise one or more processors803, a system memory813, and a bus814that couples various components of the computer801including the one or more processors803to the system memory813. In the case of multiple processors803, the computer801may utilize parallel computing.

The bus814may comprise one or more of several possible types of bus structures, such as a memory bus, memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures.

The computer801may operate on and/or comprise a variety of computer-readable media (e.g., non-transitory). Computer-readable media may be any available media that is accessible by the computer801and includes, non-transitory, volatile and/or non-volatile media, and removable and non-removable media. The system memory813has computer-readable media in the form of volatile memory, such as random access memory (RAM), and/or non-volatile memory, such as read-only memory (ROM). The system memory813may store data and/or program modules such as an operating system805, a commands database806, and device data807that are accessible to and/or are operated on by the one or more processors803.

The computer801may also comprise other removable/non-removable, volatile/non-volatile computer storage media. The mass storage device804may provide non-volatile storage of computer code, computer-readable instructions, data structures, program modules, and other data for the computer801. The mass storage device804may be a hard disk, a removable magnetic disk, a removable optical disk, magnetic cassettes or other magnetic storage devices, flash memory cards, CD-ROM, digital versatile disks (DVD) or other optical storage, RAM, ROM, electrically erasable programmable read-only memory (EEPROM), and the like.

Any number of program modules may be stored on the mass storage device804. An operating system805, commands database806, and device data807may be stored on the mass storage device804. One or more of the operating system805, commands database806, and device data807(or some combination thereof) may comprise one or more program modules.

A user may enter commands and information into the computer801via an input device (not shown). Such input devices include, but are not limited to, a keyboard, pointing device (e.g., a computer mouse or remote control), a microphone, a joystick, a scanner, tactile input devices such as gloves, and other body coverings, a motion sensor, and the like These and other input devices may be connected to the one or more processors803via a human-machine interface802that is coupled to the bus814, but may be connected by other interface and bus structures, such as a parallel port, game port, an IEEE 1394 Port (also known as a Firewire port), a serial port, network adapter809, and/or a universal serial bus (USB).

A display device812may also be connected to the bus814via an interface, such as a display adapter810. It is contemplated that the computer801may have more than one display adapter810and the computer801may have more than one display device812. A display device812may be a monitor, an LCD (Liquid Crystal Display), a light-emitting diode (LED) display, a television, smart lens, smart glass, and/or a projector. In addition to the display device812, other output peripheral devices may comprise components such as speakers (not shown) and a printer (not shown) which may be connected to the computer801via Input/Output Interface811. Any step and/or result of the methods may be output (or caused to be output) in any form to an output device. Such output may be any form of visual representation, including, but not limited to, textual, graphical, animation, audio, tactile, and the like. The display812and computer801may be part of one device, or separate devices.

The computer801may operate in a networked environment using logical connections to one or more computing devices, such as the one or more remote computing devices816, the one or more security devices817, and the one or more user devices818. The one or more remote computing devices816may comprise a personal computer, computing station, workstation, portable computer, laptop computer, a network computer, a server or cloud computing device, and so on. The one or more security devices817may comprise a video doorbell or security camera. The one or more user devices818may comprise one or more of a voice-enabled remote control, a mobile device, a voice assistant device, a smart speaker, a Bluetooth headset, a client device, a personal computer, computing station, workstation, portable computer, laptop computer, mobile phone, tablet device, smartphone, smartwatch, activity tracker, smart apparel, smart accessory, or the like. Logical connections between the computer801, the one or more remote computing devices816, the one or more security devices817, and the one or more user devices818may be made via a network815, such as a local area network (LAN) and/or a general wide area network (WAN) and one or more network devices (e.g., a router, an edge device, an access point or other common network nodes, such as a gateway). Such network connections may be through a network adapter809. The network adapter809may be implemented in both wired and wireless environments. Such networking environments are conventional and commonplace in dwellings, offices, enterprise-wide computer networks, intranets, and the Internet.

Application programs and other executable program components such as the operating system805, the commands database806, and the device data807are shown herein as discrete blocks, although it is recognized that such programs and components may reside at various times in different storage components of the computing device801, and are executed by the one or more processors803of the computer801. Any of the disclosed methods may be performed by processor-executable instructions embodied on computer-readable media.

While specific configurations have been described, it is not intended that the scope be limited to the particular configurations set forth, as the configurations herein are intended in all respects to be possible configurations rather than restrictive.

It will be apparent to those skilled in the art that various modifications and variations may be made without departing from the scope or spirit. Other configurations will be apparent to those skilled in the art from consideration of the specification and practice described herein. It is intended that the specification and described configurations be considered as exemplary only, with a true scope and spirit being indicated by the following claims.