Patent ID: 12219200

DETAILED DESCRIPTION

With reference toFIG.1, the methods described herein may be used in conjunction with an exemplary system comprised of one or more of a cable/satellite/digital media streaming set top box, (individually and collectively a “STB”104), an appliance having digital video recorder (“DVR”) capabilities, and a display device, such as a television (“TV”). One or more individual smart devices100,102such as an “iPhone”, “iPad”, “Android”, “Google Assistant”, “Amazon Echo”, etc. may be provisioned with a remote-control app or “skill” for use in causing commands to be issued to appliances such as STB104, TV106, AV receiver108, etc., particularly in response to voice input being provided by one or more users. Examples of smart device remote-control apps/skills may be found in co-pending U.S. patent application Ser. Nos. 12/406,601 and 13/329,940 and U.S. Publication 2019/0020493. In some circumstances, it will be appreciated that an appliance may itself be adapted to receive and act upon voice input according to the description which follows, e.g., the described smart device would be a component part of the appliance.

Communication114from a smart device100,102to another device, such as STB104, may take the form of any convenient infrared (IR), radio frequency (RF), hardwired, point-to-point, or networked protocol, as necessary to cause the respective target appliances to perform the desired operational functions. By way of example, an IR signal may be transmitted to an appliance, such as STB104, directly as described in co-pending U.S. patent application Ser. No. 13/043,915 or via an intermediate relay device (not illustrated) such as described in co-pending U.S. patent application Ser. No. 13/071,661. Similarly, an RF communication, such as any of RF4CE, Bluetooth, WiFi, etc., may be utilized as appropriate for a given intended target appliance.

In keeping with the teachings set forth herein, a media access device, such as STB104, television106, etc., may be capable of adapting content listings, such as program guide displays, DVR recording listings, favorite channels listings, etc., in accordance with which one (or more) of multiple users is currently interacting with the media access device, particularly via use of a voice communication that is provided to a smart device and/or an appliance having the described smart device capabilities. When a device is not itself capable of displaying a listing or guide, the device, such as STB104, may be connected to TV106using an HDMI interface. The HDMI connection may be via an intermediate switching device such as an AV receiver108(HDMI cables110,112) or may be direct.

It will be appreciated that a voice recognition, user identification technique may be used alone or with other techniques, such as with a camera based, user identification technique, to determine which one or more of multiple users are interacting with the system as desired.

Turning now toFIG.2, a media access device, such as STB104, may include, as needed for a particular application, a processor400coupled to a memory402which may comprise RAM, ROM, and/or Flash memory; a power supply404; a hard disk (HDD) interface406and associated disk drive408, for example to support DVR functionality; a smart card and/or PCMIA interface410where required to accommodate service enablement/decryption cards412; other input/output interfaces414such as USB, Ethernet, etc.; a front panel interface418with associated front panel keypad420and display means422; a remote control receiver or transceiver424for reception of signals from a controlling device200; a digital audio/video output interface426such as an HDMI port; a digital audio output432such as an SPDIF connection; analog video and audio outputs430,432; and one or more tuners416for receiving and demodulating a cable or satellite signal. As will be appreciated, in general tuner416may be capable of receiving both one or more digital media streams comprising program content and a so called out-of-band data stream comprising, for example, program and/or program guide information, software updates, etc.

The controlling device200may be a smart device having an app/skill installed thereon, may be a conventional remote control provisioned with a microphone for receiving voice input and for providing data indicative of the voice input to the media access device (either directly or via a voice processing service), or may be a conventional remote control with the media access device having a microphone and being used to receive and process voice input.

The media access device, such as STB104, may also include a wireless LAN and/or PAN interface434supporting communication over Bluetooth, WiFi and/or WiFi Direct links. As will be appreciated, when so equipped, in certain embodiments STB operational commands may be received via wireless interface434, for example from a smart device such as100, in place of or in conjunction with command transmissions received via remote control transceiver424. Furthermore, in certain embodiments, Ethernet and/or WiFi interfaces414,434(or any other appropriate method) may be used to network together a group of media access devices, for example to allow multiple TVs in a household to access a centralized DVR capability. Still further, in some embodiments an input peripheral, such as a camera and/or a microphone, and/or a peripheral interface436may be provided for use in receiving commands and for use in user recognition purposes as described herein, etc. It will be appreciated that in alternative embodiments a peripheral may be interfaced via wireless network interface434.

As is known in the art, programming instructions may be stored in a non-transitory readable memory402which, when executed by processor400, may enable the media access device, e.g., STB104, to do all or some of receive and decode digital media streams and data streams, including program and/or program guide information, from tuner416, from the Internet, etc.; to facilitate the storage of received media content and data on HDD408or in memory402; to output received (from tuner416, the Internet, etc.) or retrieved (from HDD408) media content via one or more of the output ports426,428; to monitor other I/O ports414for activity; to receive, decode, and act upon user commands received from a controlling device200or input via front panel420; to display content listings, such as one or more of program guide information, listings of stored content, listings of favorite channels, etc.; to display setup and configuration menus at user request and act upon any resulting user input; etc. As will described hereafter, the media access device (or smart device usable with the media access device104) may also include programming for monitoring the viewing habits and, optionally, recording requests of multiple users and utilizing the information gathered to prioritize, filter, and/or otherwise arrange content information (which may include DVR content) for display.

FIG.3illustrates in flowchart form a series of actions which may be performed by the programming of a media access device, such as STB104, upon executing a command. Note that in this context “executing a command” constitutes an action to be performed by the media access device. For example, an action may comprise a response to one or more commands received as a result of button presses on an associated remote control, a response to a voice input received from a smart device, a response to an internally generated action, a command received via a local area or wide area network, etc. The response may include causing a display of a content listing.

Upon receipt of a command at step500(whether received from an external source or generated internally), at step502the device programming may first determine if the current user has been identified or is identifiable. For purposes of this disclose, user identification will be determined at least using a received voice input and the discerning of a speaking user may be performed using any convenient technique or combination of techniques as appropriate for a particular embodiment. An example technique for using voice recognition is described in U.S. Pat. No. 6,256,109.

If the current user is identifiable, at step504an ID for the user is stored for use in subsequent steps. If not, the system may request that the speaker speak again for purposes of again attempting to identify the user and/or may assign a new ID for the new user506for tracking purposes as described below. As will be appreciated a voice print for the voice input provided by the user as captured in step504may be mapped to the new ID for subsequent use in subsequent command processing. In other instances, the actions of an unknown user may simply be ignored for tracking purposes.

If at step508it is determined that the received command is a request to initiate a content guide display, at step510the information that is presented in the content listing, e.g., broadcast channels and/or other available content items (e.g., streaming or VOD programs), may prioritized as described hereafter in conjunction withFIG.4considering the current user's viewing history, gathered as described hereafter in conjunction withFIGS.5through7. Similarly, if at step512it is determined that the request is to initiate display of previously recorded content, at step514entries which correspond to recording requests which were initiated by the current user may be given priority in the resulting listings display.

As will be appreciated, in some embodiments recorded content may also be incorporated into the display of program guide listings, either automatically or by user request. In some embodiments, the recorded content may be placed at the top of a content listing over content guide information when the system is operating in a DVR control mode, e.g., the remote control is in a DVR control mode, and may be placed at the bottom of a content listing following program guide information when the system is operating in a TV, Satellite, Cable, Internet Media, or the like mode. Additionally, a recorded item portion of content listing may be filtered, e.g., comprise only recorded content attributable to the current user and/or comprise only recorded content suitable for the current user (e.g., considering content ratings or the like versus the actual or perceived age of the current user); and/or may be prioritized, e.g., encompass all recorded material, but with recordings attributable to or frequently viewed by the current user appearing first in the displayed listing or being otherwise made more visually prominent. Depending on the particular embodiment, the style of recorded item listing presentation, filtered and/or prioritized, may be predetermined by design or may user configurable, either globally for all users or on a per-user basis.

If at step516it is determined that the received command is a request to change to a new channel, to commence a different media stream, etc., the identity of the new channel, media stream, etc. (and, optionally, the source thereof) may be stored at step518for reference during the viewing history accumulation process as will be described hereafter in conjunction withFIG.5. Thereafter, at step520the requested action may be performed by the device programming and command execution is complete.

FIG.5illustrates in flowchart form a series of actions which may be performed by the device programming to accumulate user viewing history data, stored for example as illustrated inFIG.6. Referring first toFIG.6, the device programming may cause a set of viewing history data files700corresponding to each separately identifiable user of the system to be maintained in storage, e.g., in local memory and/or remote memory (such as memory associated with a cloud server). In certain embodiments, an additional data file for an “unknown,” or default, user may also be maintained. As illustrated, each individual user history file702may comprise a series of records, each record704comprising a series of fields, for example:

Content source identity706. By way of example possible content sources may include cable or satellite service providers providing channels accessible via tuner416, Internet content sources, e.g., media streaming services, accessible via Ethernet414or WiFi434interfaces, previously recorded DVR material accessible from HDD406, etc.

Channel ID708. By way of example, possible channel IDs may comprise a cable system channel number, a URL, a DVR directory entry, a media streaming title, etc. as appropriate for the content source706associated with that record.

Cumulative viewing time710for this user, which may be accumulated as described hereafter.

Last viewed time712, which may comprise the absolute time and date of the last viewing of this particular content source and channel by this particular user, maintained in some embodiments to allow for removal of and/or modification of stale entries by the device programming.

In some embodiments, additional information714regarding viewing habits may be captured and used to further refine selection or prioritization of items for content listing display. By way of example and without limitation, field714may contain several subfields corresponding to time of day, day of week, and/or month of year; genre; etc. which may be used to accumulate data for use in further optimizing content guide displays based on daily, weekly, seasonal, or other factors.

In order to track and capture user viewing history, the STB programming of STB104may utilize a timer function to periodically inspect and capture the state of the system. This timer period may be any convenient value suitable for providing an appropriate level of granularity to the captured data while not imposing an overly large processing burden on the STB. In one exemplary embodiment, the timer period may be of the order of several minutes. It will also be appreciated that in some embodiments the timer interval may be variable: for example, the timer interval may be reduced (or lengthened) for several cycles immediately following execution of a channel change command in order to better detect and accurately capture (or disregard) channel surfing activity in which a user may only dwell on a particular content stream for a short period of time.

Returning now toFIG.5, upon timer expiry600, at step602the STB programming may first determine content is being rendered. To this end, the STB programming may determine if TV106(or another device by which media content can be consumed) is in fact powered on and, when determinable, rendering STB-supplied content. Such a determination may be made using any convenient method, for example TV status as reported over HDMI interface426; TV power monitoring by the STB itself or by an external device such as described for example in U.S. Pat. No. 6,870,463; inspection of simulated state of TV106maintained as described for example in U.S. Pat. Nos. 7,126,468 or 6,784,805; etc. Additionally, in embodiments where viewer presence detection is available, for example via a camera interface436on STB104; by detection of the presence of a user's smart device on a local network; via passive infrared detection; etc., the user presence status may also be ascertained at this step. If the TV is not on or no viewer is present, no further action is required, the timer is reset at step614, and processing is concluded.

If TV106is powered on and viewer(s) are present (if presence is utilized), at step604the STB programming may next determine if the current user is known to the system (i.e., a valid user ID was stored at step504ofFIG.3when executing the last user command received by STB104). If not, processing concludes as before. If so, at step606the STB programming may next determine if a record704corresponding to the current channel (i.e., the source and channel ID value last stored at step518ofFIG.3when executing a channel or media stream change command) already exists in the individual viewing history file702of the current user. If not, at step608a new record may be created. Thereafter, at step610the cumulative viewing time710for that record may be incremented, for example by adding the elapsed timer value to any current value in field710and storing that new value, and the last viewed field712may be updated by storing the current date/time into field712. In those embodiments where they exist, additional data field(s)714may be updated as well. Thereafter, at step614the timer is reset, and processing is concluded.

As will be appreciated, to ensure optimal performance, the STB programming of STB104may periodically engage in data maintenance or “housekeeping” activity with respect to the user viewing history files. Scheduling of such activity may be on a calendar basis, e.g., weekly or monthly; may be on an elapsed time basis, e.g., after a predetermined number of active viewing hours have been logged; may be by explicit user request; may be based on a combination thereof; or may be on any other basis appropriate for an embodiment. Maintenance activity may comprise for example removal of stale records704, e.g., those with a “last viewed” date older than a certain date, which date may be a predetermined value or may be dynamically determined based on system resources such as available memory, those that have a date, are related to content genre, etc. that are no longer in season, and the like. Alternatively, in some embodiments stale records may not be removed immediately, but may rather be subjected to a gradual downgrading process, for example by a step-wise reduction in accumulated time value710, which, as will become apparent, may cause the corresponding channel ID to appear increasingly lower in the program guide listing display.

It will be understood that, while the above processing is being described as being performed by the STB, some or all of the processing may be performed by a smart device that receives voice commands (and other input) from a user, by a cloud server to which voice commands (and other input) is provided for processing, etc.

FIG.7illustrates an example, recorded content directory800which may be used in conjunction with the methods described herein. Within this content directory, each entry802corresponds to a single recorded or to-be-recorded item and comprises a series of fields as follows:

Item ID field804may comprise a unique item identifier, which may be used for example to populate a channel ID field708in a user viewing history record704when source device field706is “DVR”.

Requesting user field806may be the user ID of the user requesting the recording, for example the user ID captured at step504ofFIG.3during input of the recording request command.

Status field808may reflect the current status of this request within the DVR system, for example “Successfully recorded”, “Yet to be recorded”, “Recording in progress”, “Recording error”, etc. In some embodiments, the status field808may also reflect if a user has watched a recorded program.

Title field810may contain the title of the content, which, for example, may be derived from the program guide information stored in STB memory at the time the recording request was initiated.

Source/channel field812may contain the content source and channel ID from which the desired content was or is to be recorded.

Date field814, start time field816, and end time field818may define the recording instance, which as before may be derived from the program guide information stored in STB memory at the time the recording request was initiated. It will be appreciated that the contents of date field814may also include an indication of that the request is for “series recording,” i.e., all episodes of a recurring program are to be recorded. By way of example, one method for implementing such a feature may comprise automatically inserting a further entry802into the DVR content directory800each time an existing series recording request is completed (i.e., when updating status field808to “Successfully recorded”).

Other information field820may contain, as appropriate for an embodiment, additional information regarding the content (e.g., release date, cast members, etc.) and/or DVR housekeeping information (e.g., HDD directory pointers, retention period settings, etc.). Additionally, in certain embodiments other information may include subfield(s) for storing the user IDs of secondary users, e.g., users who, although not the originators of the recording request, may have viewed or otherwise expressed interest in this item of content. As will be appreciated, in the case of series recording requests, such additional user information may also be propagated forward as future recording request entries are generated.

For filtering purposes, it is also contemplated that the system may capture genre and/or ratings information for a recorded program. In this manner, for example, the genre and/or ratings information can be utilized to filter information within a displayed content listing based upon a viewing permission level for that user thus providing a form of parental control.

With reference toFIG.4, processing of a user request for display of a program guide listing by the STB programming of STB104, for example as may occur at step510ofFIG.3, may comprise the following steps:

At step902, STB programming may first determine—using the voice command provided by the user—if the current user is known or unknown. If unknown, at step912the default action established for that condition may be taken. This may comprise, for example, simply arranging the information in a displayed content listing, e.g., a program guide, in ascending or descending channel number sequence; prioritizing the information if a displayed content listing as described hereafter using a default viewing history that has been maintained for “unknown user;” using a content provider supplied preferred order; or any other arrangement as appropriate for a particular embodiment. In some instances, if the voice input from the unknown speaker appears to indicate that the speaker is a minor, the content can be filtered in accordance with any parental control established for the system

If the current user is known, at step904the STB programming may retrieve the user history file702corresponding to the current user. At step906, the individual entries704of the retrieved history file may be arranged in descending order of accumulated viewing time710. At step908any additional filtering factors may be applied to further adjust the order of the sorted entries, for example without limitation, a factor derived from the “last viewed” information field712may be applied to promote more recently viewed items over older entries; time/date information from additional data714(where available) may be compared to the current time/date to adjust for daily, weekly or seasonal factors; etc

Thereafter, at step910the source and channel ID fields706,708from the newly-ordered viewing history data may be used to prioritize an order in which the information within a content listing is displayed, i.e., to thereby provide a user-specific order to the listing of the content of the program guide that is caused to be displayed, for example, during the execution of step520ofFIG.3.

With reference toFIG.8, processing of a user request for display of a listing of available, recorded content by the STB programming of STB104, for example as may occur at step514ofFIG.3, may comprise the following steps:

At step1002, STB programming may first determine if the current user is known or unknown. If unknown, at step1012the default action for that condition may be taken, for example preparing to list all DVR content in chronological or alphabetical order; preparing to list only content corresponding to DVR content directory entries802in which the requesting user field806is set to “unknown;” or any other arrangement as appropriate for a particular embodiment. As before, voice processing can be utilized to determine if a speaker is a minor whereupon further filtering of content may be required depending upon set parental control parameters.

If the current user is known, at step1004the STB programming may initially build a set of pointers to a subset of DVR content directory entries corresponding to only those items for which the requesting user field806matches the current user and in which status field808contains the status “Successfully recorded” or “Recording in progress.” Thereafter, at step1006the STB programming may add to the set further pointers to DVR content directory entries corresponding to those items for which the current user ID appears as a secondary user in the other information field820and status field808contains the status “Successfully recorded” or “Recording in progress.” Next, at step1008, the STB programming may determine if the DVR content display feature is operating in filtering or prioritizing mode. As described previously, in filtering mode only the DVR content attributable to the current user (and/or accessible by the current user) may be displayed. Accordingly, if the STB programming is operating in filtering mode, generation of DVR content directory pointers for use by a display routine, for example during the execution of step520ofFIG.3, is complete and processing ends.

If operating in prioritizing mode, at step1010the STB programming may next append additional DVR content directory pointers to the set created during steps1004and1006above, e.g., to display such additional DVR content at the end of the displayed DVR content directory, in a supplemental directory, etc. Such additional pointers may comprise the balance of the DVR content directory entries in chronological or alphabetical order, or any other ordering as appropriate for a particular embodiment. Thereafter, generation of an ordered list of DVR content pointers is complete and ready for use by a display routine, for example during the execution of step520ofFIG.5.

As also described above, a combination of prioritizing and filter may be employed whereupon a known user would be provided with a content listing in which the information in the content listing is prioritized considering recording requests made by that user, that user's content watching history, etc. while removing from the displaying listing of content those recorded programs that the user is not authorized to access, e.g., the rating is not within a ratings limit set for that user.

As noted above, voice commands provided to a voice assistant can be used to generate data sets or logs and these data sets can be analyzed to identify a user's behavior with respect to content being accessed, the timing (Day, weekday etc.) of such access and/or the location (geo, IP location, etc.) of such access for the purpose of creating a dynamic favorite list for a user based on voice recognition. It will therefore be appreciated that favorite channel lists can vary by user in home and each user in a home can have their own favorite channel list.

More particularly, for use in creating a dynamic favorite list, the system may store user activity with respect to a selection of content in a database. The stored information can include an identifier for the content and one or more of a timestamp, activity text, intent, the user command (voice file), responsive action taken by the voice assistant, user location, etc. Preferably, user activity with respect to content selection is monitored for a predetermined period of time that will be sufficient to identify one or more content selection patterns, such as frequency for selecting a content, channel, or content source, weekend patterns, weekday patterns, holiday patterns, seasonal patterns, time of day patterns, etc. As described above, the collection of data and the revision of the one or more identified patterns (and the creation and/or deletion of patterns) may be continued after the initial, seeding period of time.

Once one or more patterns have been identified from the data that has been collected, when a give user commands the system to access a favorites list, the system should respond with a dynamically created favorites list. For example, at the specific time when a recognized user provides a command such as “Watch TV” or “Show my Favorite List,” the most accessed channel by that user may get selected for viewing or included in a listing that is caused to be displayed. Thus, if user1in a home is watching BBC channel at 8:00 PM on most weekdays, a “Watch TV” or “Favorite Channel” command given by the user1at approximately 8:00 PM will be used to cause the system to automatically access the BBC channel while, if user2is regularly watching CNN at 8:00 PM in same house, then “Watch TV” or “Favorite Channel” command given by the user2at approximately 8:00 PM will cause the system to automatically access the CNN channel. It will be appreciated that the system can be adapted to allow the user to access channels—via repeated verbal commands—in an order that is determined by the favorites list and/or the system can allow the user to simply view the listing whereby the user can interact with the displayed listing as desired.

As also described above, it a user is an unknown user or the favorite channel listing for the user is empty, the system may take a default action and/or execute the above steps to create one or more favorites lists for the user.

While various concepts have been described in detail, it will be appreciated by those skilled in the art that various modifications and alternatives to those concepts could be developed in light of the overall teachings of the disclosure. For example, while the illustrative embodiment presented above is implemented in an STB device, it will be appreciated that the inventive methods may be equally applied to any system in which available media content is to be listed or displayed to a user, including without limitation TV devices, personal computers, media servers, cloud-based media aggregation services, digital media receivers, etc. Furthermore, it will be appreciated that while the exemplary data structures are presented in tabular form for ease of reference herein, in practice these may be implemented in various forms using any convenient data representation, for example a structured database, XML file, cloud-based service, etc., as appropriate for a particular embodiment.

Additionally, it will be appreciate that the methods of the instant disclosure may be combined with other program guide display techniques—for example, a user may define an explicit list of favorite channels to which the techniques described herein are to applied to the exclusion of other content, or conversely may define a list of preferred channels which are exempt for the prioritization process and will always be displayed; and/or the prioritization process may be applied to multiple genres of content, e.g., separate viewing histories may be maintained for individual categories such as sports, news, drama, comedy, etc.

Further, while described in the context of functional modules and illustrated using block diagram format, it is to be understood that, unless otherwise stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or a software module, or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary for an enabling understanding of the invention. Rather, the actual implementation of such modules would be well within the routine skill of an engineer, given the disclosure herein of the attributes, functionality, and inter-relationship of the various functional modules in the system. Therefore, a person skilled in the art, applying ordinary skill, will be able to practice the invention set forth in the claims without undue experimentation. It will be additionally appreciated that the particular concepts disclosed are meant to be illustrative only and not limiting as to the scope of the invention which is to be given the full breadth of the appended claims and any equivalents thereof.

All patents, patent applications, and other publications cited within this document are hereby incorporated by reference in their entirety.