Patent Publication Number: US-11659229-B2

Title: System and method for management and presentation of alternate media

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims the benefit of U.S. Provisional Patent Application No. 63/017,272, filed Apr. 29, 2020, which is incorporated by reference herein in its entirety. 
    
    
     BACKGROUND OF THE INVENTION 
     The availability of an ever-increasing number and type of video and visual media sources, the increasing size and definition of digital televisions (“DTVs”), and the almost ubiquitous availability of residential high-speed bidirectional data/video communication are combining to change the consumer media viewing experience. In particular, it has become more feasible than ever for consumers to simultaneously access multiple media presentations. Audio and video can be sourced from cable, optical fiber, smartphones, tablets, security devices, video doorbells, computers and digital assistants. Video can be streamed to one or more large format displays (such as DTVs or video monitors), as well as smaller format displays (tablets, smartphones, etc.). Audio can be streamed to headsets and smart speakers. For example, it is not uncommon for a consumer to be able to have independent audio streaming from a media controller to Bluetooth®-enabled headsets, while a different audio and/or video presentation is sent by that controller to an HDMI port. Consumers are also able to view multiple video windows on DTVs using picture-in-picture (“PIP) technology, split-screen or mosaic video presentation formats. All these allow for more than one media presentation to be rendered for simultaneous viewing. 
     The types of visual media available to consumers is also not limited to broadcast or video-on-demand services that are received at or streamed into a residence. Consumers have a wide variety of alternate video and audio sources available to them, including the aforementioned smartphones, tablets, security devices, video doorbells, computers and digital assistants. Consequently, when a consumer is viewing a presentation from a given video source, there may be a host of reasons why that consumer would want, or need to temporarily access an alternate media (audio and/or video) presentation. It would not be unusual for a consumer viewing broadcast content on in a primary window on a DTV, to temporarily want to view video from a video doorbell via a secondary PIP window upon the DTV, and temporarily stream only audio from the video doorbell to the headset of a the consumer viewing the DTV (while still rendering audio associated with the broadcast content via HDMI to the DTV). Once the consumer video doorbell interaction terminated (the individual(s) at the door either entered the or left the vicinity), the consumer would close the PIP window and switch back to the broadcast audio in their headphones. 
     It is also a common practice for consumers to switch between two media viewing experiences over a limited time period. For example, when a consumer is watching a given broadcast or streaming program and a commercial break occurs, they often switch to another channel or video source so as to watch something of interest and then switch back to the initial broadcast/streaming program at a time at which they estimate or guess the commercial break has terminated. This usually results in a consumer switching several times back and forth until the commercial is actually over. 
     There exists a need for system and method that provides a consumer with a convenient and automatic means of controlling automated rendering of multiple sources of video and audio streams and sources based upon the type of video and/or audio as well as predetermined consumer preferences, the disclosure of which is hereby incorporated herein by reference. 
     BRIEF SUMMARY OF THE INVENTION 
     A system and method for the automatic management of the presentation of information from two or more media sources. This automatic management includes the selective viewing of video information on a prescribed screen, screen window or screen configuration, as well as the selective provision of audio information to a particular port or appliance. This management is performed in response to and a s a function of consumer preferences, as well as the source, type and content of the video and audio information. The management system may be entirely located within the consumer&#39;s residence, or reside in whole or in part in a connected network or cloud. The system can initiate video/audio management in an entirely autonomous manner, or initiate only in response to user input (keypad, graphical user interface, voice, etc.). 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings in which: 
         FIG.  1    is a functional diagram of a system supporting a first preferred embodiment of an alternate media management and presentation. 
         FIG.  2    provides a table representative of media stored in the system of  FIG.  1   . 
         FIG.  3    provides a table representative of available vectors within the system of  FIG.  1   . 
         FIG.  4    provides a table representative of user-specific provider and account information associated with the system of  FIG.  1   . 
     
    
    
     DETAILED DESCRIPTION 
       FIG.  1    provides a functional diagram of a system ( 100 ) for the management and presentation of alternate media. As shown, the system includes digital media controller  102  which serves as the nexus for the system. Digital media controller  102  includes processor  104  (which includes at least one digital processor), and memory  106 . This digital media controller may be implemented via a general-purpose personal computer, a dedicated appliance (such as a set-top box or other consumer premises equipment), or via an off-site device connected to the wireless interface and other peripherals via private or public networks. 
     Digital media controller  102  is shown to be interfaced with digital televisions  108   a  and  108   b , Wi-Fi interface  110 , cable/optical media provider  112 , laptop computer  114 , Internet providers  116   a  and  116   b , Bluetooth® transceiver  118  and telecom provider  120 . In addition, mobile devices (smartphones)  122   a  and  122   b , and security video camera ( 124 ) are adapted to interface with digital media controller  102  via a bidirectional Wi-Fi connection supported by Wi-Fi interface  110 , and wireless headphones  126   a  and  126   b  interface with digital media controller  102  via Bluetooth transceiver  118 . Remote control  128  is shown to be communicatively linked to digital media controller  102  via a wireless connection. This wireless connection can be optical (infrared) or radio frequency (“RF”). If the digital media controller is located off-site it will be understood that a network connection to an optical or RF transceiver could be utilized to relay remote control commands to the off-site digital media controller. 
     Processor  104  serves to control and direct incoming and outgoing media to and from digital media controller  102 , including video, image and audio information stored in memory  106 . In addition, processor  104  is responsive to user-specific information stored in memory  106 , as well as user commands received by digital media controller  102 . Such commands may be issued by users via laptop computer  114 , mobile devices  122   a  and  122   b , remote control  128 , or digital assistant  130 . As shown, remote control  128  includes a keypad that may be actuated by a user to generate a command, or a user may issue a voice command ( 132 ) which the remote-control forwards to digital media controller  102  for voice recognition and processing. This received voice command can also be forwarded by processor  104 , via media controller  102  and Internet provider  116   a , to an off-site server ( 134 ) for recognition and processing. A voice command could also be received and forwarded in a similar manner by digital assistant  130 . Memory  106  is capable of storing digital video and audio which may viewed or listened to via the various peripherals ( 108   a ,  108   b ,  114 ,  122   a ,  122   b ,  126   a ,  126   b ) interfaced with digital media controller  102 . 
     As mentioned above, memory  106  stores video, image and audio information. The video stored information can consist of recorded programs received via cable/optical media provider  112 , or Internet providers  116   a  and  116   b  (memory  106  functioning as a DVR), downloaded video and images from computers, mobile devices, residential video cameras, etc., as well as downloaded music files. A user can also identify or tag these stored files within memory  106  so as to designate a particular name, genre, event, or an association with a particular user. Processor  104  is adapted to receive and process user commands for such tagging via the graphical user interface (“GUI”) provided by laptop computer  114 , or mobile devices  122   a  and  122   b , as well as via remote control  128 . 
     For example, as shown in table  200  of  FIG.  2   , there are four identified users (Andrew, Beth, Charles and Donna). Particular photo, video, voice message and music files are associated with each of these users. A video file labeled “Band Gigs” is shown to be associated with the user named Andrew, as is a music file labeled “Classical Mix”. To make such an association, a user could have utilized a GUI to a) enter the name “Andrew” in a field associated with the video and music files at issue, or b) used a mouse, stylus or touch-screen associated with the GUI to designate the files as associated with Andrew. A user (via remote control  128 , digital assistant  130 , or mobile devices  122   a  and  122   b ) may also tag files using a set of voice commands that system  100  is adapted to recognize and respond to. For, example, a user could recite a command such as “Associate the music file Ramones with Beth” or “The image file Charles&#39; School Photos belongs to Charles”. 
     Memory  106  is also adapted to store information identifying the various peripherals within system  100  that are available as vectors for the presentation of streaming, live or stored media, as well as any association between particular users and those peripherals. For example, DTV  108   a  is designated as the primary video display for all users. Laptop computer  114  has been designated as Andrew&#39;s secondary video display and headset  126   a  as his Bluetooth headset. DTV  108   b  is designated as the secondary video display for users Beth and Donna. Information associating an additional headset ( 126   b ) as well as two mobile devices ( 122   a  and  122   b ) is also stored in memory  106  (as shown in Table  300 ). Bluetooth pairing of devices with the system is done in the normal manner via user interface such as that provided by laptop computer  114 , DTVs  108   a  and  108   b , or mobile devices (smartphones)  122   a  and  122   b . The association between various Bluetooth peripherals and users can be made using GUIs or voice commands in a manner similar to that described with respect to tagging media files. 
     In addition, memory  106  also stores information that associates various media providers and account information with users. Such stored information is represented in Table  400  of  FIG.  4   . As shown, when applicable, the various users are associated with a cable/optical media provider, an Internet provider, a social media account, a video conferencing account, as well as streaming media accounts. Memory  106  also stores any required passwords associated with the providers and accounts, so as to enable system  100  to gain access. 
     The ability of the system to associate provider, account and password information with a user permits the execution of pre-configured routines that enable users to easily access personal media from a host of sources. For example, the command “Show Andrew&#39;s Internet Media Account A Holiday Photos”. As evidenced by Table There is nothing stored in memory  106  that has been indexed or labeled as “Holiday Photos” associated with Andrew. Rather as the command stated, these particular photos are associated with Andrew&#39;s Internet Media Account A. Rather, information would be stored in Memory  106  so as to pre-configure the systems response when the “Andrew&#39;s Internet Media Account A” was recognized. This pre-configuration routine, entered and stored by a user via a GUI, would instruct the system to access and retrieve the requisite ID and password from memory  106 , and utilize Internet Provider  116   a  to access the requested photos (memory  106  contains information associating this provider with Andrew—see table  400 ). 
     The voice command responsiveness of the system is enabled by processor  104  and media controller  102  function to recognize, process and execute a predetermined set of commands based upon user input and the information stored in memory  106 . These commands can be entered by a user via GUI or as voice commands. Examples of syntax for such commands are shown below:
         Show [sporting event] with commercials on [user name]&#39;s mobile device.   Show [broadcast program] and [user name]&#39;s [stored image files].   Show [user name]&#39;s [social media account] and [Internet media].   Show [sporting event] and [security camera] and commercials.   Show [stored video file] on [user name]&#39;s mobile device and route audio to [user name]’ headphones.   Show [event] on all displays.   Show [social event] on all displays.   Play [user name]&#39;s [music file] on [user name]&#39;s headphones.   Play [music file] on [peripheral].   Play [video file] on [peripheral].   Switch to [user image file] for [interval].   Switch to [music file].   Switch to [alternate provider or media] during commercials.   Switch to [sporting event] during commercials.   Switch to [user name]&#39;s text messages during commercials.   Switch to [user name]&#39;s video conference call with [name].   Switch to [stored image files] from [start time] to [end time].   Switch to [user name]&#39;s primary display.   Also show [event] with audio routed to [user name]&#39;s headphones for the next [interval].   Also show Eagles Game on [vector].   Also show [device].       

     The syntax has the basic format of “action” (show, play, switch, also), “content” (video, image, account, Internet) and “vector” (primary/secondary display, headphones, mobile device). The initiating words of the command phrases (“Show”, “Play”, “Switch”, “Also”) serve to instruct system  100  as to base action being requested. 
     “Show” is indicative of the rendering of visual content; For example, the command “Show the Eagles Game on Beth&#39;s TV” would instruct the system to route a video stream of that sporting event to the DTV identified in the system as Beth&#39;s primary video display. Whereas the command “Show the Eagles Game” would instruct the system to route a video stream of that sporting event to whatever display was active, as no particular vector was specified. 
     “Play” is indicative of audio content. For example, the command “Play the Eagles game on Andrew&#39;s Headphones” would cause system  100  to play the audio portion of the video stream of that sporting event on the headphones associated with Andrew in memory  106 . 
     “Switch” indicates that the command is requesting a change in content or vector. For example, “Switch to the Eagles game during commercials” would instruct system  100  to show the video stream of that sporting event whenever a commercial break occurred in the program that was presently being displayed. The command “Switch to Andrew&#39;s mobile device” would instruct system  100  to cease showing/playing content on presently active vector and show/play that content on Andrew&#39;s mobile device. 
     As shown above, the syntax also permits user names to be employed as modifiers within the commands. A user name can modify content (“Show Charles&#39; video; Play Beth&#39;s voice messages”) or a vector (Beth&#39;s mobile device). 
     Time constraints, either specific (start time, stop time, fixed interval), or tied to a particular condition (“during commercials”) are also permitted within the syntax. 
     The conjunctions “also” and “with” serve to permit a user to command that more than a single content be simultaneously presented on a given vector. For example, in response to the command “Also show Eagles Game on DTV  108   a ” processor  104  can be pre-programmed to respond by causing media controller  102  to display the video stream of that sporting event in a picture-in-picture (“PIP”) window overlaid upon whatever content was already being displayed on DTV  108   a . Whereas processor  104  can be pre-programmed to respond to the command “Show Eagles game and security camera on Andrews mobile device” by instructing media controller  102  to display, upon Andrew&#39;s mobile device, the video stream of that sporting event on a split-screen side-by-side with a live feed from the security camera. 
     The above commands and syntax can be utilized to create tailored media experiences that can incorporate broadcast video, live streaming video, as well as stored media. For example, a user could recite the following commands:
         “Show the Eagles game on DTV  108   a  and switch to Andrew&#39;s Vacation Photos during commercials.”   “Also show live feed from Andrew&#39;s social media account”
 
The first command would result in the system routing a live feed of the Eagles game, sourced from cable/optical media provider  112 , to DTV  108   a  and switching to a slideshow of the images stored in the file “Andrew&#39;s Vacation Photo&#39;s”—sourced from memory  106 , when a commercial breaks occurs during that sporting event. The second command would cause the system to open a PIP window within the display of the Eagles game on DTV  108   a , and display the video stream currently associated with Andrew&#39;s social media account therein. All of the requisite information and connectivity to establish this live feed is available to the system. As shown in  FIG.  1   , digital media controller interfaces with Internet provider  116   b  and Table  400  of  FIG.  4    indicates that memory  106  contains information associating provider  116   b  with Andrew. In addition, table  400  shows that memory  106  also contains information indicates that social media account “@Andrew” is also associated with Andrew and provides the required password (“1112”) for accessing that account.
 
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. For example, in addition to the various interfaces specifically mentioned in as providing GUIs in the above described system (laptop computer, smartphone), a dedicated appliance, a personal computer or a tablet could also serve this function. Similarly, although DTVs, a laptop computers and smartphones were described as vectors for displaying or playing media, any type of visual or audio device capable of reasonably reproducing the particular type of media being accessed by a user would be a suitable vector (tablet, analog TV, projector, audio system, etc.). In addition, the particular syntax of the voice commands disclosed above is not intended to be limiting. Technology supporting the recognition of and response to such commands is well-known in the art and continually advancing—It will be understood that the principles of the disclosed embodiments can be applied to this advancing technology without departing from the scope of the invention.