Patent ID: 12206944

DETAILED DESCRIPTION

Methods and systems are provided herein for recommending media assets during a user browsing session based on user input that occurs during that browsing session. For example, a media recommendation application may provide a first set of media asset recommendations for display to a user. The user may then scroll, select a title, or provide some other user input. The media recommendation application may then update the recommendations and provide a second set of media asset recommendations to the user, while the user continues to browse or interact with the application.

The amount of content available to users in any given content delivery system can be substantial. Consequently, many users desire a form of media recommendation through an interface that allows users to efficiently navigate content selections and easily identify content that they may desire. An application that provides such guidance is referred to herein as a media recommendation application, recommendation application, interactive media recommendation application or, sometimes, a media guidance application or a guidance application.

Media recommendation applications may take various forms, depending on the content for which they provide recommendations. One typical type of media recommendation application is an interactive television program guide. Interactive television program guides (sometimes referred to as electronic program guides) are well-known recommendation applications that, among other things, allow users to navigate among and locate many types of content or media assets. Interactive media recommendation applications may generate graphical user interface screens that enable a user to navigate among, locate and select content. As referred to herein, the terms “media asset,” “title” and “content” should be understood to mean an electronically consumable user asset, such as television programming, as well as pay-per-view programs, on-demand programs (as in video-on-demand (VOD) systems), Internet content (e.g., streaming content, downloadable content, Webcasts, etc.), video clips, audio, content information, pictures, rotating images, documents, playlists, websites, articles, books, electronic books, blogs, chat sessions, social media, applications, games, and/or any other media or multimedia and/or combination of the same. As referred to herein, the term “multimedia” should be understood to mean content that utilizes at least two different content forms described above, for example, text, audio, images, video, or interactivity content forms. Content may be recorded, played, displayed or accessed by user equipment devices, but can also be part of a live performance. Furthermore, as referred to herein, the terms “state” and “genre” should be understood to mean a category of media assets, titles, or content, and may be used interchangeably.

The media recommendation application and/or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable media. Computer-readable media includes any media capable of storing data. The computer-readable media may be transitory, including, but not limited to, propagating electrical or electromagnetic signals, or may be non-transitory, including, but not limited to, volatile and non-volatile computer memory or storage devices such as a hard disk, floppy disk, USB drive, DVD, CD, media cards, register memory, processor caches, random access memory (RAM), etc.

With the advent of the Internet, mobile computing, and high-speed wireless networks, users are accessing media on user equipment devices on which they traditionally did not. As referred to herein, the phrases “user equipment device,” “user equipment,” “user device,” “electronic device,” “electronic equipment,” “media equipment device,” or “media device” should be understood to mean any device for accessing the content described above, such as a television, a Smart TV, a set-top box, an integrated receiver decoder (IRD) for handling satellite television, a digital storage device, a digital media receiver (DMR), a digital media adapter (DMA), a streaming media device, a DVD player, a DVD recorder, a connected DVD, a local media server, a BLU-RAY player, a BLU-RAY recorder, a personal computer (PC), a laptop computer, a tablet computer, a WebTV box, a personal computer television (PC/TV), a PC media server, a PC media center, a hand-held computer, a stationary telephone, a personal digital assistant (PDA), a mobile telephone, a portable video player, a portable music player, a portable gaming machine, a smart phone, or any other television equipment, computing equipment, or wireless device, and/or combination of the same. In some embodiments, the user equipment device may have a front-facing screen and a rear-facing screen, multiple front screens, or multiple angled screens. In some embodiments, the user equipment device may have a front-facing camera and/or a rear-facing camera. On these user equipment devices, users may be able to navigate among and locate the same content available through a television. Consequently, media guidance and/or recommendations may be available on these devices, as well. The guidance and/or recommendations provided may be for content available only through a television, for content available only through one or more of other types of user equipment devices, or for content available through both a television and one or more of the other types of user equipment devices. The media recommendation applications may be provided as online applications (i.e., provided on a website), or as stand-alone applications or clients on user equipment devices. Various devices and platforms that may implement media recommendation applications are described in more detail below.

One of the functions of the media recommendation application is to provide media recommendation data to users. As referred to herein, the phrase “media recommendation data” or “recommendation data” should be understood to mean any data related to content or data used in operating the recommendation application. For example, the recommendation data may include program information, recommendation application settings, user preferences, user profile information, media listings, media-related information (e.g., broadcast times, broadcast channels, titles, descriptions, ratings information (e.g., parental control ratings, critic's ratings, etc.), genre or category information, actor information, logo data for broadcasters' or providers' logos, etc.), media format (e.g., standard definition, high definition, 3D, etc.), on-demand information, blogs, websites, and any other type of data that is helpful for a user to navigate among and locate desired content selections.

FIG.1shows a simplified state transition diagram of a media asset recommendation model100that is used to provide media asset recommendations to a user, along with an example display150. The simplified state transition diagram ofFIG.1illustrates a current state in which the model operates (i.e., the drama state110), as well as a possible next state that the model can transition to (i.e., the romance state120). The diagram100also illustrates an exit state130that the model can transition to under certain circumstances. As shown inFIG.1, when a user is browsing media assets within the drama genre, the current state of the model is the drama state110. The example display150ofFIG.1illustrates the currently browsed state (i.e., the drama state110), as well as a cursor152which identifies a selected media asset of the currently browsed state.

Each state of the model100has a corresponding set of media asset recommendations. These media asset recommendations may be displayed, such as on example display150, wherein a set of media assets1-5is displayed for the drama state110, and a set of media assets10-14is displayed for the romance state120. These media asset recommendations can be determined using one or more existing systems, models, or algorithms. For example, one set of media asset recommendations may include a list, score, ranking, or ordering of titles within a state (or genre) based on popularity, relevance to the user, user preferences, user information, system information, trending assets, length of the assets, type of assets, and more.

In some examples, the media asset recommendation model100may determine the initial set of media asset recommendations based on system-wide data, user data, similar user data, and any other suitable information. The media asset recommendation model100may determine a first set of media asset recommendations for each state of the model prior to or in response to the beginning of a browsing session (i.e., when a user logs in).

In some examples, the media asset recommendation model100operates in conjunction with one or more machine learning algorithms that can prioritize media asset selection and ordering, including traditional machine learning and deep learning models. For example, media assets associated with a given genre can be ranked in the order of user importance. As an extension, the rewards within a given genre can vary based on the importance to a user. For example, the first media asset displayed for a genre is expected to have high reward, while the 10thmedia asset is expected to have a lower reward. In this scenario, the media recommendation application can consider the reduced reward from the 10thmedia asset and decide to transition from the current state to a next state within the media asset recommendation model as the user scrolls to see additional movies within a state. The media asset recommendation model (otherwise known as a reinforcement learning model) can also learn the average reward correlated to the media asset position and adjust the actions, accordingly, as described in further detail below.

Each state of the media asset recommendation model also includes a plurality of user input options that are available for a user to select. These user input options are browsing actions available to a user, and can include (a) scrolling to next media asset, (b) playing the currently selected media asset, (c) changing to a different state (i.e., genre), and (d) exiting the system. The media asset recommendation model can also detect various other user actions or user interactions such as (e) searching for a title, (f) tracking how long the user spends viewing a given media asset, (g) whether the user accesses additional information about a given asset, (h) whether the user watches a trailer for the asset, and more.

Each user input option has a corresponding user input option value that is used by the media asset recommendation model for various purposes. The user input option value comprises a combination of the reward value associated with the user input option and the selection probability associated with the user input option. When a user exercises an input option, the user may stay in the same state or transition to a new state as shown inFIG.1. The expectation of rewards for each of these states is the product of the reward associated with the new (or same) state if that user input option is exercised and the probability of achieving that state. In some embodiments, the probability of transitioning to a state may factor in information such as the device used, time of day, location, user profile, and more.

The reward values associated with each user input option are used to optimize the media asset recommendation model, to improve the user experience and to increase user engagement. To that end, the model may generally associate positive rewards with actions that increase engagement, such as selecting a movie or watching a movie for a certain amount of time, and associate negative rewards with actions that decrease engagement, such as scrolling, changing states or genres, and exiting the system.

A first user input option is scrolling. When a plurality of media assets are displayed (e.g., in a carousel), a first asset may be selected, and the user may have the option to scroll left or right to highlight a second media asset. The media recommendation application associates a small negative reward with a scrolling user input option (e.g., −1), because the first asset recommended to the user was not acted upon, and thus the model was not effective in recommending that desired media asset.

A second user input option is watching a media asset. This includes the user selecting the highlighted media asset (e.g., a movie), and causing the media asset to play. The media recommendation application associates a large positive reward with this user input option (e.g., +100), because the model was effective in recommending that media asset to the user. In some examples, the reward may be proportional to the amount of time the user spends watching the movie. For example, the reward may be +50 for starting the movie, and then an additional reward amount up to +50 depending on how far into the movie the user watched, for a total of +100 if the user watched the entire movie. These values are for example only, and one of ordinary skill in the art would understand that the values, weights, and distribution of rewards can change from what is described herein while remaining within the scope of this disclosure.

A third user input option is changing states or changing genres. The media recommendation application may associate this action with a small negative reward (e.g., −1), because the model failed to recommend the right genre to the user.

A fourth user input option is exiting the system entirely. The media recommendation application may associate a large negative reward (e.g., −50) with the exit option. In some examples, the reward associated with the exit option may be a large negative reward (e.g., −50) if a total watching time for the user is below a threshold time during the current browsing session. Alternatively, the reward may be a small negative reward (e.g., −5) when the user has spent a significant amount of time already browsing and/or watching media assets (e.g., when the total watching time is above the threshold).

In some examples, the rewards and selection probabilities associated with each user input option may be determined based on or using a model that was trained using a large dataset. For example, the rewards and/or selection probabilities may be set to default or initial values determined based on all other users, all similar users, all local users, local similar users, or any other set of data. The reward values and/or selection probabilities may be set and/or modified in order to optimize the model to achieve increased user engagement. In some examples, the reward values and/or selection probabilities may be set or modified with other goals in mind, such as for exploration of the media asset recommendation model, exploitation of the media asset recommendation model, and more.

The reward values and selection probabilities may be updated over time as the user interacts with the media recommendation application. For example, a first scrolling event may be associated with a reward of −1, while a subsequent scrolling event is associated with a reward of −0.9, and each subsequent scrolling event has a decreasing small negative reward. This may be referred to as “discounting” rewards or using a “discount factor,” to enable the media asset recommendation model to converge, and to prioritize early actions compared to later actions. The media asset recommendation model may put greater reward weight (either positive or negative) on the earlier user input options taken by a user, and put less reward weight on later user input options, particularly where the user takes the same user input option multiple times in a row. Additionally, the selection probabilities associated with each action may change in response to interaction of the user with the media recommendation application. For example, when the user scrolls through the media assets associated with a given state (i.e., movies within a displayed genre), the selection probability associated with the “scroll” user input option for that state may increase each time the user selects the scroll option, while the selection probabilities associated with all other options for that state decrease, such that sum of the selection probabilities across all possible transition states remains constant (e.g., a value of one).

The state value associated with a state may be based on the user input option values associated with that state. For instance, the state value may be a simple addition of all the user input option values associated with the state. In another example, the state value may be an average, weighted average, or some other mathematical combination of the user input option values (and/or other information) associated with the state.

In the examples of this disclosure, the media asset recommendation model comprises a Markov model. However, it should be appreciated that other models may be use as well. Additionally,FIG.1shows the model having three states, but it should be understood that there may be many other states and state transitions. In some examples, each state may have the same set of associated user input options (e.g., scrolling, exiting, etc.). In other examples, one or more states may have a different set or subset of associated user input options from one or more other states. The illustrated and described media asset recommendation model can be solved using any traditional reinforcement learning model, including value iteration, policy iteration, Q-learning and more. As the user makes additional choices (e.g., scrolling right to see additional media assets), the model can track the changes by updating the value of each state and the policy at each state. Techniques like deep Q-learning can also be applied to the system to learn the optimal policy given a particular user state.

FIG.2shows a simplified flowchart of steps involved in operating the media recommendation application, and in particular the steps involved with respect to taking an exploit action or an explore action. While the application operates within a given model state (e.g., the drama state110shown inFIG.1), the application can perform two actions when the user makes certain inputs (e.g., when the user scrolls right). These actions can be used to optimize the model behavior to either exploit the information already available (and thereby provide a more immediate reward), or to explore the states of the model to develop higher confidence in values associated with one or more states.

As a general concept, the media recommendation application makes use of the media asset recommendation model to optimize the long-term engagement and long-term rewards for each user. Deciding whether to exploit the model or explore the model involves a tradeoff between exploitation of data already known or collected by providing recommended media assets resulting in rewards to the system with high confidence, and exploration of states or genres less selected by the user, but that may provide greater long-term rewards if recommended to the user. Without exploration of the model, the model may converge to a local maximum, such that some states or genres are not recommended due to associated low confidence levels, even if in the long run these states or genres would result in greater long-term rewards and user engagement.

In an ideal (but largely impractical) scenario, the media asset recommendation model would include data about a user's preference for every possible media asset. That is, in order to provide the best possible recommendation, the model would need to know everything about the user, so as to determine which state and/or media asset is the perfect recommendation at any given time. The model would have a high confidence level associated with every possible state, and would need to exploit only the information available to make the perfect recommendation. However, since in practice the model must operate using limited information, the model must be explored to improve the confidence level associated with various states. Exploration enables the system to present assets that may or may not be preferred by the user, because these assets are different from or are associated with a different state than the known preferences of the user.

In the media asset recommendation model for a user, some states may be more explored and used than other states. For example, the user may enjoy and routinely browse and watch drama TV shows. Based on constantly watching drama TV shows, the model may only recommend drama TV shows and not recommend action movies or comedy movies. The model for this user has a high confidence value associated with the drama state, and low confidence levels associated with the action state and the comedy state. However, the user may in the long term actually prefer comedy movies, or may have complex tastes that change over time. If the model is only exploited by recommending drama TV shows, the user may be turned off or disengaged from the system because they are not being recommended the ideal set of media assets for their tastes.

The media recommendation application exploits the model by selecting the state (i.e., genre) having a highest or relatively high state value, and with a relatively high associated confidence value. In the scenario above, the media recommendation application would present media assets associated with the drama TV state.

Referring back toFIG.1, the current state is a drama state110and the user is browsing in the drama genre. When the media recommendation application detects a scroll right user input, there may be two possible actions in response: (1) showing an additional drama movie from the drama state110or showing a romance movie from the romance state120. Each of these actions is associated with a discounted value based on the action and the history of the previous rewards. In the exploit scenario, the application chooses the action of recommending a drama movie since the long-term reward for the action of recommending a drama movie is higher than for recommending a romance movie.

Exploration, on the other hand, may include the application providing a recommendation from a different state from the current state. This enables the application to, over time, develop a higher confidence level that the user does or does not enjoy particular states or genres. Each state within the model has a state value (e.g., the average reward for all user input options corresponding to the state). The state value may have a standard deviation associated with it, which may be correlated with a confidence level for the state. For instance, the drama state may be relatively well explored and used by the user, and may thus have a relatively high state reward, a low standard deviation associated with the state reward, and thus a high confidence level. In contrast, the comedy state may be relatively unexplored, but may have a relatively high state reward (based on initial values from the average user). Since the comedy state is relatively unexplored for the target user, the comedy state has a high associated standard deviation, and thus a low confidence level. Thus, while it may appear that the drama state provides the best long-term reward based on current knowledge of the user's preferences, the reality is less clear. As the user interacts with and explores the comedy state more, the standard deviation for the comedy state will decrease, and the confidence level will increase. The comedy state reward value itself will change as the user input option rewards and selection probabilities are updated. As a result, exploration of the comedy state creates a clearer picture of the potential long-term rewards. After performing sufficient exploration, the media asset recommendation model may determine that the comedy state has a higher long-term reward than the drama state, even though the opposite was true based on the limited initial information about the user.

In some examples, the media recommendation application decides to present the state that results in the highest long-term rewards, meaning some of the potential states a user may be interested in will go unexplored. In other examples, the media recommendation application decides, based on selecting an explore option, to present a state that has a high variability in the predicted long-term rewards. This may occur where a user has chosen a movie from a particular genre, but in the long term does not often intend to choose from this genre. The media recommendation application can select the explore option with a high probability during the initial stages of a user (i.e., when there is limited information known about the user and their preferences), and may decay as higher confidence is achieved on the probability and rewards for each state.

Referring toFIG.2, a simplified flowchart illustrates the media recommendation application operation with respect to exploration and exploitation of the media asset recommendation model. The process starts at step202.

At step204, the process200includes determining whether the user is browsing for a media asset (e.g., a movie, TV show, or other title). If the user is not browsing for a media asset, then the process remains at step204to wait for a positive determination that the user is receptive to a media asset recommendation.

If the user is browsing for a media asset at step204, at step206the process200includes determining whether to exploit or explore the information in the media asset recommendation model associated with the user. The concepts of exploring and exploiting the model are described in detail above. As noted above, the media recommendation application may incentivize maximizing the long-term rewards gained through user interaction with the application, which can mean performing a mixture of both (a) exploitation of the model to achieve rewards with a high confidence level, and (b) exploration of the model to improve the confidence level associated with less-explored states. The overall goal of the media recommendation application may be to maximize rewards, and to have high confidence that the model recommendations result in the best possible long-term rewards (e.g., high engagement, user satisfaction, etc.).

Step206includes determining whether the application should exploit the model or explore the model based on the confidence levels associated with various states, the standard deviation of rewards associated with various states, and the overall goals of the media recommendation application. To that end, the media recommendation application may change its goals over time, such that the application may select exploration at step206with a higher probability early on in the lifetime of a user's account when less is known about the user's preferences, and reduce the probability of selecting exploration over time (e.g., the application may select exploration with a 50% probability to start, and then reduce the probability to 20% or less over time as the model for the user is filled in with known preferences of the user). Additionally, the media recommendation application may set or modify the probability that exploration is selected based on user profile information, such as whether the user is receptive to new genres and recommendations, or whether the user wishes to be presented with new recommendations in order to improve the model's recommendations to the user.

If the media recommendation application decides to exploit the information in the model at step206, step208of the process200includes selecting a state with the best state reward, and selecting an action with the best predicted long-term reward. The selected state with the best state reward may be the state with the highest state value, wherein the state values are determined by combining the expected rewards for each user input option multiplied by the selection probability of the user input option). For example, if a user's most-watched genre is comedy movies, the application may select the comedy state at step208. The application can consider various factors in addition to the most-watched genre, such as the time of day, device used, device location, and trending or popular selections for (a) other similar users, (b) all other users, (c) close by users, (d) close by similar users, and more.

Selecting the action with the best long-term reward can include selecting a set of media asset recommendations corresponding to the selected state that result in the best long-term reward. This can include selecting both the media assets included in the set of recommendations, as well as the order of presentation (i.e., ranking) of those selected media assets. For example, the application may select a set of movies with the highest likelihood of being watched (i.e., the movies that include the highest probability of a user action being to play the selected media asset). Additionally, the application may rank the set of movies according to the likelihood of being played by the user.

In other words, in response to determining that the next media recommendation application action is an exploit action, the media recommendation application may be configured to switch from the current state to an exploit state having a highest state value, determine a third set of media asset recommendations for the exploit state based on a predicted long-term system reward associated with the exploit state, and provide the third set of media asset recommendations to the user.

Instead, if the application selects the explore option at step206, step210of the process200includes the application selecting a state that is relatively unexplored, and selecting an action with an associated high variability. For example, the application may select a relatively unexplored state having an associated low confidence level and high standard deviation of expected rewards. The application may also select a set of media asset recommendations that have a high variability, and/or a low confidence level associated with whether the user is likely to select the assets or not. In some examples, the application may select a set of media assets (and order of presentation of the set of media assets) that include a standard set of assets. The particular selected media assets may act as a barometer or bellwether that provides the best improvement in confidence associated with the states of the model. For instance, by analyzing other users' preferences, the application may determine that a user's ratings with respect to media assets A, B, and C provide the best indication of whether that user prefers assets from a state, and thus provides the best indication of what the predicted long-term rewards for the state will be. The media recommendation application may present media assets A, B, and C during exploration of the state, in order to best improve the model's associated confidence level for that state. Put another way, certain media assets for a given state may be guideposts or indicators that, when interacted with by the user, provide a high confidence level in how the user will interact with other media assets of that state. By presenting only these guidepost media assets, the application can improve its understanding of the user's preferences with the least amount of additional presentation of media assets from less-explored states.

In other words, in response to determining that the next media recommendation application action is an explore action, the media recommendation application is configured to switch from the current state to an explore state having a low confidence level associated with a state value of the explore state, determine a fourth set of media asset recommendations based on the explore state, and provide the fourth set of media asset recommendations to the user.

Although not shown, process200may also include the application presenting the set of media asset recommendations to the user. The user may then interact by selecting a user input option, such as scrolling, changing genres, watching a selected media asset, exiting, or taking some other action.

At step212, the process200includes determining a reward based on the received user input option. For example, if the user watches the recommended media asset, a large positive reward is granted. If the user scrolls, changes genres, or exits, a negative reward is granted. This is discussed in greater detail above.

At step214the process200includes updating the media asset recommendation model and rewards for each state, as well as the selection probability for each user input option associated with the current state. The selection probabilities may be updated based on the user input option that was selected, in order to reflect the new selection probabilities. The reward values may be updated based on the media asset that was selected, for instance by increasing the reward value associated with the state.

Process200may then proceed back to step204, to continue in a loop as long as the user browses for a media asset to view. In some examples, the process200may continue until there is no user input for some threshold time (e.g., five minutes), the user exits the application, or some other ending condition is met.

FIGS.3-5illustrate example sets of displays that show various media asset recommendations, and illustrate how the display of media asset recommendations changes in response to various user inputs.

In some examples, a media recommendation application is configured to access a media asset recommendation model comprising a plurality of states (e.g., simplified model100described with respect toFIG.1). The media recommendation application is configured to determine a current state corresponding to the user. The current state may be the state or genre the user is currently browsing (e.g., drama). The current state may be the initial state upon startup of the media recommendation application, the initial state upon the user logging in or beginning the browsing session, or the state that the user selects (e.g., when the user changes from one genre to another).

The media recommendation application is then configured to provide a first set of media asset recommendations to the user based on the current state. For example, where the current state is the drama state, the media recommendation is configured to provide a set of media assets that correspond to the drama state. The current state may refer to the genre that is currently selected by the user. A set of media asset recommendations can include a list of titles of movies or other media assets that correspond to the current state. A given media asset may fall into several different genres, and thus may be associated with several different states. The set of media asset recommendations can also include the order or ranking of media assets that are associated with the current state. For example, the media assets may include a number of assets that fall into the drama category, and these assets may be ranked or ordered based on popularity, number of plays by similar users or all users, the expected reward granted if the user plays the asset, the expected time or percentage of the asset that will be viewed by the user, etc. In some examples, the media recommendation application may employ existing algorithms, models, and/or decision-making processes to determine the first set of media asset recommendations to provide to the user.

In some examples, a set of media asset recommendations may include one or more subsets or groupings of media assets. The first subset of media assets may be provided to the user for immediate display, while the second subset of the media assets is provided for storage in a cache, and only displayed after the user has scrolled or provided some additional input. In one example, the first subset of media assets provided for immediate display are shown on the screen to the user. Display screen310inFIG.3illustrates the first subset of media assets1,2,3,4, and5displayed to the user in a carousel on the screen. As described herein, the media recommendation application may display media assets in a carousel format, such that a limited number of media assets are displayed at any given time, and additional media assets can be accessed by scrolling to the right or the left along the carousel. In some examples, the carousel may include placeholders that can be populated with media assets when the user scrolls.

After the media recommendation application provides the first set of media asset recommendations for display to the user, the application may be configured to receive a user input option from the user. As described above, the user input options can include actions taken by the user during the browsing session, such as (a) scrolling to a next media asset recommendation within the current genre, (b) playing a currently selected media asset, (c) changing from the current genre or state to a next genre or state, and (d) exiting the application. Other user input options may include (e) searching for a title, (f) spending a tracked amount of time viewing a given media asset, (g) accessing additional information about a given asset such as the credits, (h), watching a trailer for the media asset, and more. The user may input the user input option to the media recommendation application using a remote control, keyboard, touch-screen, voice input, gesture input, or any other suitable user interface device.

In response to receiving the user input option, the media recommendation application is configured to determine a second set of media asset recommendations for the current state based on the received user input option. The second set of media asset recommendations may include the same or different media assets from the first set of media asset recommendations. The second set of media asset recommendations may also include a reordering or re-ranking of the media assets. For example, in determining the second set of media asset recommendations, the application may replace one or more media assets with different media assets that are more likely to be watched based on the received user input option. In another example, the second set of media asset recommendations may include the same media assets as the first set of recommendations, but with a different ranking or order of display of the media assets. A combination of new media assets and new ranking of media assets is possible as well.

In some examples, the media recommendation application may determine the second set of media asset recommendations in real time in response to the user input option, before a second user input option is received. The application may update the media asset recommendation model and recommendations based on the model on-the-fly, such that the user is continually presented with updated recommendations.

In other examples, the application may update the model and media asset recommendations based on the model in response to the user exiting the application. When the user exits the application, data about the user's browsing session may be transmitted to a server for processing, and the model may be updated at this time.

In one example, the media recommendation application is configured to determine an updated media asset recommendation model in response to receiving the first user input option. Updating the model can include modifying one or more selection probabilities, reward values, and more, as discussed above. The application can then determine the second set of media asset recommendations for the current state based on the received first user input option and the updated media asset recommendation model. For example, a first user input option may be to change from an initial state to a next state. The application will update the model to reflect a lower reward for the initial state and a higher reward for next state, based on the user's selection of the next state. The user switching states may also indicate a greater propensity of the user to watch media assets from a wide variety of genres. This information can be used along with the updated model values to determine the second set of media asset recommendations the application provides.

After determining the second set of media asset recommendations, the application is configured to provide the second set of media asset recommendations to the user. The second set of media asset recommendations can be provided in real time during the browsing session, after the first user input option is received and before a second user input option is received. As noted above, a first subset can be provided for immediate display to the user, and a second subset can be provided to be stored in a cache (such as on a user device) for later display to the user.

FIG.3illustrates how the display seen by a user may change over time as the user scrolls through the media assets of a genre. Display310includes a first subset of media assets1,2,3,4, and5displayed in a carousel. The selection icon is placed on media asset1. Media assets6,7,8,9, and10(e.g., the second subset) are stored in a cache for later display to the user when the user scrolls far enough from the initial selection icon position. In response to a user input, the media recommendation application may determine a modified first subset, a modified second subset, or a combination of both.

Display320ofFIG.3illustrates the display after the user has scrolled to the right. The rightward scroll user input option moves the selection icon to the right, placed on media asset2. In response to the rightward scroll user input option, the media recommendation application determines that media asset5is more likely to be viewed by the user than media asset4. The media recommendation application changes the ranking of media assets4and5and swaps their position on the display, as shown inFIG.3. Display320shows that the second set of media asset recommendations provided to the user includes the same media assets, but with a different ranking. The first set of media asset recommendations (media assets1-10shown with respect to display310) comprises a first subset available for immediate display (media assets1-5) and a second subset available for display after additional user input (media assets6-10). After the rightward scroll user input is received, the media recommendation application determines the second set of media asset recommendations by determining a modified first subset (e.g., reranking media assets1-5) based on the received user input option, but leaving the second subset as originally determined (i.e., media assets6-10). The media recommendation application then provides the second set of recommended media assets to the user by providing for display the modified first subset (i.e., reranked media assets1-5) and the second subset (i.e., originally ranked media assets6-10).

In another example, determining the second set of media asset recommendations for the current state based on the received user input option comprises determining a modified second subset based on the received user input option, and providing the second set of recommended media assets to the user comprises providing for display the first subset and the modified second subset. When presented with the display320inFIG.3, the user may make a further input such as a second rightward scroll. In response, the media recommendation application may determine a new set of media asset recommendations to provide to the user. InFIG.3, the new set of media asset recommendations includes a modified first subset (e.g., media assets1,2,3,14, and15), and a new second subset (e.g., media assets16-20).FIG.3illustrates the third display screen330after the user has input the second rightward scroll, showing the modified first subset displayed on the carousel and the new second subset stored in a cache for later display.

In some examples, the media recommendation application is configured to determine, for each state, initial values for (a) the set of media asset recommendations, (b) the reward values for each user input option available in the state, and (c) the selection probabilities for the user input options available in the state. These initial values may be based on user profile information associated with the user, default values, averages for all users of the application, averages for similar users, location-based user averages, location-based similar user averages, the type of device used, user interests, user age, user demographic information, and more.

In some examples, the media recommendation application is configured to switch states based on an expected or predicted long-term reward. Increased user engagement is desirable, and the expected or predicted long-term rewards associated with a given genre may correlate with user engagement. If a long-term reward for the current state decreases (e.g., in response to continuous user scrolling or other user input options with negative rewards), the application may switch the current state or genre so as to present a state with an expected better reward, better fit for the user interests, and better probability of increasing user engagement. To that end, in some examples, the media recommendation application is configured to determine a first predicted long-term system reward corresponding to the current state and determine a second predicted long-term system reward corresponding to a next state. In response to determining that the second predicted long-term system reward is greater than the first predicted long-term system reward, the media recommendation application switches from the current state to the next state. The predicted long-term reward for a state can be a combination of the state value and the confidence value associated with the state, or any other suitable function.

In response to switching from the current state to the next state, the media recommendation application may cause a user interface associated with the user to change one of a text, image, color, highlight, or shading.

FIGS.4and5illustrate example techniques for making changes to the display.

Using the media asset recommendation model, the media recommendation application may determine it is beneficial to change states or genres and present media assets from a different genre than is currently selected. The application may determine this based on a predicted reward, or a likelihood that the user will select one or more media assets for viewing. During a browsing session as the user scrolls right, the likelihood of an asset being a match for the user decreases. As such, if the user continues to scroll right within a genre, the application may determine to change genres rather than continuing to present media assets that have less and less likelihood of being viewed. The new genre presented may enable the presentation of media assets that are more likely to be selected and viewed by the user.

FIG.4illustrates a first example wherein the display presented to the user is changed. A first display410presents two example genres and two sets of corresponding recommended media assets (media assets1-5for genre1and media assets10-14for genre2).FIG.4illustrates that upon detecting a user input option to scroll right, the media recommendation application replaces genre1with genre3, and correspondingly replaces media assets1-5with media assets21-25. This is shown in display420. The media recommendation application determined that genre3and media assets21-25are more likely to result in a positive reward, and thus an increase in user engagement and satisfaction. Even though the user selected a rightward scroll user input, the selection icon remains in the leftmost position, centered on media asset21. The user may not be immediately aware that the genre has changed, and as such the display of genre3may also include a change of the text (i.e., displaying genre3instead of genre1), a change of the color or color scheme, a change of the highlighting or shading of one or more aspects of the display, and more. The change from genre1to genre3(and the corresponding media assets) can occur based on the user input option being a scroll to the right, or some other user input, or without any further input from the user. To the user, it may appear that the application has automatically updated and changed the display on its own in response to the scroll input.

FIG.5illustrates another example where the media recommendation application changes the displayed genre and corresponding media assets. Display510includes genre1and genre2displayed along with their corresponding recommended media assets in respective carousels (1-5and10-14respectively). In response to a user scrolling right within genre1, the media recommendation application may replace genre2with genre3(and replace media assets10-14with21-25). This is shown inFIG.5as display520. In this way, the user can continue to scroll through the media assets of genre1, while genre3is moved into view just below. Similar toFIG.4, the media recommendation application may determine that the media assets from genre3are more likely to be watched by the user than those of genre2(and/or genre1), and thus determine to present genre3and its media assets to the user. But rather than replacing the genre currently being browsed (genre1) with the new genre (genre3), the new genre (genre3) is simply inserted or moved into place below the currently browsed genre (genre1).

AlthoughFIGS.4and5illustrate example changes to the display that can occur in response to user input, other changes are possible as well.FIG.4illustrates the replacement of the genre currently being browsed with a more relevant genre.FIG.5illustrates the addition or insertion of a more relevant genre below the genre currently being browsed. In other examples, the more relevant genre may replace or be inserted above the genre currently being browsed, the more relevant genre may replace or be inserted in a non-adjacent row to the genre currently being browsed, or any other suitable replacement or insertion. Additionally, the more relevant genre being added or replacing a displayed genre can be displayed with a different scaling or sizing, different color, shading, highlighting, or any other visual change. In some examples, the display change may include a change based on the device or display size used by the user. For instance, where the display only includes two genres (as inFIGS.4and5), the more relevant genre or replacement genre may replace or be inserted as shown inFIGS.4and5. Alternatively, where the display is large enough to include three or more genres, the more relevant genre or replacement genre may replace any of the three or more rows, and/or be inserted between any of the rows.

As noted above, the media recommendation application may change the display in response to scrolling input from the user. In some examples, the media recommendation application may change the display in response to determining that the state reward for the currently browsed state is below the state reward for another state. In another example, the media recommendation application may change the display in response to determining that the average reward for the media assets currently displayed with respect to the currently browsed state (i.e., the N currently displayed media assets) is (a) below a threshold amount, and/or (b) below the average reward for the first N media assets recommended to the user for a different state. For instance, if the five currently displayed movies for genre1have an average expected reward of 10, while the first five recommended movies in genre2have an average expected reward of 15, the media recommendation application may use this determination as a trigger to change the display to display genre2.

In some examples, the media recommendation application may change the display in response to receiving a threshold number of inputs (e.g., 10 or more scrolling inputs). The threshold number of inputs may change depending on the genre. For instance, the user may scroll more often within genre1than in genre2, and the threshold number of scrolling inputs needed to trigger a change to the display may differ based on which genre is currently being browsed.

In some examples, the media recommendation application may determine a more relevant genre, replacement genre, or next genre based on one or more factors. In one example, the media recommendation application may select the state or genre with the highest state value. In another example, the media recommendation application may select a state or genre that has the highest average reward for the first N media assets recommended for the state (i.e., the average reward for the top N media assets as opposed to the average reward for all media assets in the genre). The number N of media assets may change depending on the user device, display size, and more, and may reflect the number of media assets displayed at any given time for the genre. In still other examples, the media recommendation application may replace or insert two or more genres.

In some examples, the media recommendation application may mix genres, or display media assets from two or more different genres within the same carousel.

In some examples, the media recommendation application may carry out the display change practically by receiving a request and transmitting various data. For instance, the configuration data to seamlessly integrate media assets from genre2into a carousel that represents genre1can be dynamically generated and sent to the user device during the initial set-up of the browsing session. For example, configuration data may describe the event or events (e.g., the number of scroll inputs) that will trigger the user device to transmit a request (e.g., an XMLHttpRequest) to update a portion of the user interface of the user device (e.g., update a portion of the carousel for a genre). Updating in this context may include replacing existing content being displayed, or even replacing originally slotted content items that were supposed to be sent by the server (e.g., changing media asset20to media asset30if the user requests such content by scrolling past media asset19). The update to the content of the display is based on real-time navigation and user action and may include real-time update technologies (e.g., AJAX, etc.) to update specific portions of the user interface (e.g., carousel).

In one example, placeholder media asset positions (e.g., tiles) that represent media assets that would be displayed when the user scrolls right are transmitted to the user device as part of a static webpage, while the actual content (e.g., media assets themselves) are dynamically changed based on user action. In some cases, the media recommendation application may also prefetch the media assets for when the user scrolls right, since the alternate genre to display to the user is already known.

In another example, “filler” tiles are pre-inserted in at least one carousel at a predetermined location (e.g., at a particular index) based on the carousel's category or genre. These tiles may be automatically updated (e.g., via an XMLHttpRequest) when each tile comes into view on the display, or as the user scrolls towards it. The criteria to fill or request content for such filler tiles is dependent on scrolling events with the carousel that it is associated with.

In some examples, the media recommendation application is configured to make a prediction about what the next user input option will be. In one example, the media recommendation application predicts that the next user input option is to exit the application. This prediction may be made based on historic user inputs over time, as well as various other pieces of information (time of day, user device, last title viewed, etc.). In response to predicting that the user input option will be an exit action, the media recommendation application is configured to, before receiving the user input option, (a) switch from the current state to an exploit state having a highest state value, (b) determine a third set of media asset recommendations for the exploit state, and (c) provide the third set of media asset recommendations to the user. For step (b), the media recommendation application may attempt to keep the user engaged with the application by recommending media assets that have the highest likelihood of being watched by the user. The application may present these media asset(s) to the user by inserting or replacing one or more media assets on the display.

In another example, the media recommendation application is configured to predict that a next user input option will be a genre change. This prediction may be made based on historic user inputs over time, as well as various other information (time of day, user device, last title viewed, etc.). In anticipation of receiving a genre change user input, the media recommendation application may be configured to display media assets from a different genre from the currently viewed genre, and/or to reorder or change the titles displayed for one or more genres. By automatically modifying the displayed titles before the user decides to make a genre change input, the application may anticipate the user's preferences and provide more relevant titles for display to the user. This can increase user satisfaction and engagement.

FIGS.6and7are now described. Users may access content and the media recommendation application (and its display screens described above and below) from one or more of their user equipment devices.FIG.6shows a generalized embodiment of illustrative user equipment device600. More specific implementations of user equipment devices are discussed below in connection withFIG.7. User equipment device600may receive content and data via input/output (“I/O”) path602and corresponding I/O circuitry. I/O path602may provide content (e.g., broadcast programming, on-demand programming, Internet content, content available over a local area network (LAN) or wide area network (WAN), and/or other content) and data to control circuitry604, which includes processing circuitry606and storage608. Control circuitry604may be used to send and receive commands, requests, and other suitable data using I/O path602and corresponding I/O circuitry. I/O path602may connect control circuitry604(and specifically processing circuitry606) to one or more communications paths (described below). I/O functions may be provided by one or more of these communications paths, but are shown as a single path inFIG.6to avoid overcomplicating the drawing.

Control circuitry604may be based on any suitable processing circuitry such as processing circuitry606. As referred to herein, processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, processing circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry604executes instructions for a media recommendation application stored in memory (i.e., storage608). Specifically, control circuitry604may be instructed by the media recommendation application to perform the functions discussed above and below. For example, the media recommendation application may provide instructions to control circuitry604to generate the media recommendation displays. In some implementations, any action performed by control circuitry604may be based on instructions received from the media recommendation application.

In client-server based embodiments, control circuitry604may include communications circuitry suitable for communicating with an application server or other networks or servers. The instructions for carrying out the above mentioned functionality may be stored on the application server. Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, Ethernet card, or a wireless modem for communications with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communications networks or paths (which is described in more detail in connection withFIG.7). In addition, communications circuitry may include circuitry that enables peer-to-peer communication of user equipment devices, or communication of user equipment devices in locations remote from each other (described in more detail below).

Memory may be an electronic storage device provided as storage608that is part of control circuitry604. As referred to herein, the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVR, sometimes called a personal video recorder, or PVR), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and/or any combination of the same. Storage608may be used to store various types of content described herein as well as media recommendation data described above. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions). Cloud-based storage, described in relation toFIG.7, may be used to supplement storage608or instead of storage608.

Control circuitry604may include video generating circuitry and tuning circuitry, such as one or more analog tuners, one or more MPEG-2 decoders or other digital decoding circuitry, high-definition tuners, or any other suitable tuning or video circuits or combinations of such circuits. Encoding circuitry (e.g., for converting over-the-air, analog, or digital signals to MPEG signals for storage) may also be provided. Control circuitry604may also include scaler circuitry for upconverting and downconverting content into the preferred output format of the user equipment600. Circuitry604may also include digital-to-analog converter circuitry and analog-to-digital converter circuitry for converting between digital and analog signals. The tuning and encoding circuitry may be used by the user equipment device to receive and to display, to play, or to record content. The tuning and encoding circuitry may also be used to receive recommendation data. The circuitry described herein, including for example, the tuning, video generating, encoding, decoding, encrypting, decrypting, scaler, and analog/digital circuitry, may be implemented using software running on one or more general purpose or specialized processors. Multiple tuners may be provided to handle simultaneous tuning functions (e.g., watch and record functions, picture-in-picture (PIP) functions, multiple-tuner recording, etc.). If storage608is provided as a separate device from user equipment600, the tuning and encoding circuitry (including multiple tuners) may be associated with storage608.

A user may send instructions to control circuitry604using user input interface610. User input interface610may be any suitable user interface, such as a remote control, mouse, trackball, keypad, keyboard, touch screen, touchpad, stylus input, joystick, voice recognition interface, or other user input interfaces. Display612may be provided as a stand-alone device or integrated with other elements of user equipment device600. For example, display612may be a touchscreen or touch-sensitive display. In such circumstances, user input interface610may be integrated with or combined with display612. Display612may be one or more of a monitor, a television, a liquid crystal display (LCD) for a mobile device, amorphous silicon display, low temperature poly silicon display, electronic ink display, electrophoretic display, active matrix display, electro-wetting display, electrofluidic display, cathode ray tube display, light-emitting diode display, electroluminescent display, plasma display panel, high-performance addressing display, thin-film transistor display, organic light-emitting diode display, surface-conduction electron-emitter display (SED), laser television, carbon nanotubes, quantum dot display, interferometric modulator display, or any other suitable equipment for displaying visual images. In some embodiments, display612may be HDTV-capable. In some embodiments, display612may be a 3D display, and the interactive media recommendation application and any suitable content may be displayed in 3D. A video card or graphics card may generate the output to the display612. The video card may offer various functions such as accelerated rendering of 3D scenes and 2D graphics, MPEG-2/MPEG-4 decoding, TV output, or the ability to connect multiple monitors. The video card may be any processing circuitry described above in relation to control circuitry604. The video card may be integrated with the control circuitry604. Speakers614may be provided as integrated with other elements of user equipment device600or may be stand-alone units. The audio component of videos and other content displayed on display612may be played through speakers614. In some embodiments, the audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers614.

The recommendation application may be implemented using any suitable architecture. For example, it may be a stand-alone application wholly-implemented on user equipment device600. In such an approach, instructions of the application are stored locally (e.g., in storage608), and data for use by the application is downloaded on a periodic basis (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach). Control circuitry604may retrieve instructions of the application from storage608and process the instructions to generate any of the displays discussed herein. Based on the processed instructions, control circuitry604may determine what action to perform when input is received from input interface610. For example, movement of a cursor on a display up/down may be indicated by the processed instructions when input interface610indicates that an up/down button was selected.

In some embodiments, the media recommendation application is a client-server based application. Data for use by a thick or thin client implemented on user equipment device600is retrieved on-demand by issuing requests to a server remote to the user equipment device600. In one example of a client-server based recommendation application, control circuitry604runs a web browser that interprets web pages provided by a remote server. For example, the remote server may store the instructions for the application in a storage device. The remote server may process the stored instructions using circuitry (e.g., control circuitry604) and generate the displays discussed above and below. The client device may receive the displays generated by the remote server and may display the content of the displays locally on equipment device600. This way, the processing of the instructions is performed remotely by the server while the resulting displays are provided locally on equipment device600. Equipment device600may receive inputs from the user via input interface610and transmit those inputs to the remote server for processing and generating the corresponding displays. For example, equipment device600may transmit a communication to the remote server indicating that an up/down button was selected via input interface610. The remote server may process instructions in accordance with that input and generate a display of the application corresponding to the input (e.g., a display that moves a cursor up/down). The generated display is then transmitted to equipment device600for presentation to the user.

In some embodiments, the media recommendation application is downloaded and interpreted or otherwise run by an interpreter or virtual machine (run by control circuitry604). In some embodiments, the recommendation application may be encoded in the ETV Binary Interchange Format (EBIF), received by control circuitry604as part of a suitable feed, and interpreted by a user agent running on control circuitry604. For example, the recommendation application may be an EBIF application. In some embodiments, the recommendation application may be defined by a series of JAVA-based files that are received and run by a local virtual machine or other suitable middleware executed by control circuitry604. In some of such embodiments (e.g., those employing MPEG-2 or other digital media encoding schemes), the recommendation application may be, for example, encoded and transmitted in an MPEG-2 object carousel with the MPEG audio and video packets of a program.

User equipment device600ofFIG.6can be implemented in system700ofFIG.7as user television equipment702, user computer equipment704, wireless user communications device706, or any other type of user equipment suitable for accessing content, such as a portable or non-portable gaming machine. For simplicity, these devices may be referred to herein collectively as user equipment or user equipment devices, and may be substantially similar to user equipment devices described above. User equipment devices, on which a media recommendation application may be implemented, may function as a standalone device or may be part of a network of devices. Various network configurations of devices may be implemented and are discussed in more detail below.

A user equipment device utilizing at least some of the system features described above in connection withFIG.6may not be classified solely as user television equipment702, user computer equipment704, or a wireless user communications device706. For example, user television equipment702may, like some user computer equipment704, be Internet-enabled allowing for access to Internet content, while user computer equipment704may, like some television equipment702, include a tuner allowing for access to television programming. The media recommendation application may have the same layout on various different types of user equipment or may be tailored to the display capabilities of the user equipment. For example, on user computer equipment704, the recommendation application may be provided as a web site accessed by a web browser. In another example, the recommendation application may be scaled down for wireless user communications devices706.

In system700, there is typically more than one of each type of user equipment device but only one of each is shown inFIG.7to avoid overcomplicating the drawing. In addition, each user may utilize more than one type of user equipment device and also more than one of each type of user equipment device.

In some embodiments, a user equipment device (e.g., user television equipment702, user computer equipment704, wireless user communications device706) may be referred to as a “second screen device.” For example, a second screen device may supplement content presented on a first user equipment device. The content presented on the second screen device may be any suitable content that supplements the content presented on the first device. In some embodiments, the second screen device provides an interface for adjusting settings and display preferences of the first device. In some embodiments, the second screen device is configured for interacting with other second screen devices or for interacting with a social network. The second screen device can be located in the same room as the first device, a different room from the first device but in the same house or building, or in a different building from the first device.

The user may also set various settings to maintain consistent media recommendation application settings across in-home devices and remote devices. Settings include those described herein, as well as channel and program favorites, programming preferences that the recommendation application utilizes to make programming recommendations, display preferences, and other desirable recommendation settings. For example, if a user sets a channel as a favorite on, for example, the web site www.allrovi.com on their personal computer at their office, the same channel will appear as a favorite on the user's in-home devices (e.g., user television equipment and user computer equipment) as well as the user's mobile devices, if desired. Therefore, changes made on one user equipment device can change the experience on another user equipment device, regardless of whether they are the same or a different type of user equipment device. In addition, the changes made may be based on settings input by a user, as well as user activity monitored by the recommendation application.

The user equipment devices may be coupled to communications network714. Namely, user television equipment702, user computer equipment704, and wireless user communications device706are coupled to communications network714via communications paths708,710, and712, respectively. Communications network714may be one or more networks including the Internet, a mobile phone network, mobile voice or data network (e.g., a 4G or LTE network), cable network, public switched telephone network, or other types of communications network or combinations of communications networks. Paths708,710, and712may separately or together include one or more communications paths, such as, a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IPTV), free-space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths. Path712is drawn with dotted lines to indicate that in the exemplary embodiment shown inFIG.7it is a wireless path and paths708and710are drawn as solid lines to indicate they are wired paths (although these paths may be wireless paths, if desired). Communications with the user equipment devices may be provided by one or more of these communications paths, but are shown as a single path inFIG.7to avoid overcomplicating the drawing.

Although communications paths are not drawn between user equipment devices, these devices may communicate directly with each other via communication paths, such as those described above in connection with paths708,710, and712, as well as other short-range point-to-point communication paths, such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or other short-range communication via wired or wireless paths. BLUETOOTH is a certification mark owned by Bluetooth SIG, INC. The user equipment devices may also communicate with each other directly through an indirect path via communications network714.

System700includes content source716and media recommendation data source718coupled to communications network714via communication paths720and722, respectively. Paths720and722may include any of the communication paths described above in connection with paths708,710, and712. Communications with the content source716and media recommendation data source718may be exchanged over one or more communications paths, but are shown as a single path inFIG.7to avoid overcomplicating the drawing. In addition, there may be more than one of each of content source716and media recommendation data source718, but only one of each is shown inFIG.7to avoid overcomplicating the drawing. (The different types of each of these sources are discussed below.) If desired, content source716and media recommendation data source718may be integrated as one source device. Although communications between sources716and718with user equipment devices702,704, and706are shown as through communications network714, in some embodiments, sources716and718may communicate directly with user equipment devices702,704, and706via communication paths (not shown) such as those described above in connection with paths708,710, and712.

Content source716may include one or more types of content distribution equipment including a television distribution facility, cable system headend, satellite distribution facility, programming sources (e.g., television broadcasters, such as NBC, ABC, HBO, etc.), intermediate distribution facilities and/or servers, Internet providers, on-demand media servers, and other content providers. NBC is a trademark owned by the National Broadcasting Company, Inc., ABC is a trademark owned by the American Broadcasting Company, Inc., and HBO is a trademark owned by the Home Box Office, Inc. Content source716may be the originator of content (e.g., a television broadcaster, a Webcast provider, etc.) or may not be the originator of content (e.g., an on-demand content provider, an Internet provider of content of broadcast programs for downloading, etc.). Content source716may include cable sources, satellite providers, on-demand providers, Internet providers, over-the-top content providers, or other providers of content. Content source716may also include a remote media server used to store different types of content (including video content selected by a user), in a location remote from any of the user equipment devices. Systems and methods for remote storage of content, and providing remotely stored content to user equipment are discussed in greater detail in connection with Ellis et al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, which is hereby incorporated by reference herein in its entirety.

Media recommendation data source718may provide media recommendation data, such as the media recommendation data described above. Media recommendation data may be provided to the user equipment devices using any suitable approach. In some embodiments, the recommendation application may be a stand-alone interactive television program guide that receives program guide data via a data feed (e.g., a continuous feed or trickle feed). Program schedule data and other recommendation data may be provided to the user equipment on a television channel sideband, using an in-band digital signal, using an out-of-band digital signal, or by any other suitable data transmission technique. Program schedule data and other media recommendation data may be provided to user equipment on multiple analog or digital television channels.

In some embodiments, recommendation data from media recommendation data source718may be provided to users' equipment using a client-server approach. For example, a user equipment device may pull media recommendation data from a server, or a server may push media recommendation data to a user equipment device. In some embodiments, a recommendation application client residing on the user's equipment may initiate sessions with source718to obtain recommendation data when needed, e.g., when the recommendation data is out of date or when the user equipment device receives a request from the user to receive data. Media recommendations may be provided to the user equipment with any suitable frequency (e.g., continuously, daily, a user-specified period of time, a system-specified period of time, in response to a request from user equipment, etc.). Media recommendation data source718may provide user equipment devices702,704, and706the media recommendation application itself or software updates for the media recommendation application.

In some embodiments, the media recommendation data may include viewer data. For example, the viewer data may include current and/or historical user activity information (e.g., what content the user typically watches, what times of day the user watches content, whether the user interacts with a social network, at what times the user interacts with a social network to post information, what types of content the user typically watches (e.g., pay TV or free TV), mood, brain activity information, etc.). The media recommendation data may also include subscription data. For example, the subscription data may identify to which sources or services a given user subscribes and/or to which sources or services the given user has previously subscribed but later terminated access (e.g., whether the user subscribes to premium channels, whether the user has added a premium level of services, whether the user has increased Internet speed). In some embodiments, the viewer data and/or the subscription data may identify patterns of a given user for a period of more than one year. The media recommendation data may include a model (e.g., a survivor model) used for generating a score that indicates a likelihood a given user will terminate access to a service/source. For example, the media recommendation application may process the viewer data with the subscription data using the model to generate a value or score that indicates a likelihood of whether the given user will terminate access to a particular service or source. In particular, a higher score may indicate a higher level of confidence that the user will terminate access to a particular service or source. Based on the score, the media recommendation application may generate promotions that entice the user to keep the particular service or source indicated by the score as one to which the user will likely terminate access.

Media recommendation applications may be, for example, stand-alone applications implemented on user equipment devices. For example, the media recommendation application may be implemented as software or a set of executable instructions which may be stored in storage608, and executed by control circuitry604of a user equipment device600. In some embodiments, media recommendation applications may be client-server applications where only a client application resides on the user equipment device, and server application resides on a remote server. For example, media recommendation applications may be implemented partially as a client application on control circuitry604of user equipment device600and partially on a remote server as a server application (e.g., media recommendation data source718) running on control circuitry of the remote server. When executed by control circuitry of the remote server (such as media recommendation data source718), the media recommendation application may instruct the control circuitry to generate the recommendation application displays and transmit the generated displays to the user equipment devices. The server application may instruct the control circuitry of the media recommendation data source718to transmit data for storage on the user equipment. The client application may instruct control circuitry of the receiving user equipment to generate the recommendation application displays.

Content and/or media recommendation data delivered to user equipment devices702,704, and706may be over-the-top (OTT) content. OTT content delivery allows Internet-enabled user devices, including any user equipment device described above, to receive content that is transferred over the Internet, including any content described above, in addition to content received over cable or satellite connections. OTT content is delivered via an Internet connection provided by an Internet service provider (ISP), but a third party distributes the content. The ISP may not be responsible for the viewing abilities, copyrights, or redistribution of the content, and may only transfer IP packets provided by the OTT content provider. Examples of OTT content providers include YOUTUBE, NETFLIX, and HULU, which provide audio and video via IP packets. YouTube is a trademark owned by Google Inc., Netflix is a trademark owned by Netflix Inc., and Hulu is a trademark owned by Hulu, LLC. OTT content providers may additionally or alternatively provide media recommendation data described above. In addition to content and/or media recommendation data, providers of OTT content can distribute media recommendation applications (e.g., web-based applications or cloud-based applications), or the content can be displayed by media recommendation applications stored on the user equipment device.

Media recommendation system700is intended to illustrate a number of approaches, or network configurations, by which user equipment devices and sources of content and recommendation data may communicate with each other for the purpose of accessing content and providing media recommendations. The embodiments described herein may be applied in any one or a subset of these approaches, or in a system employing other approaches for delivering content and providing media recommendations. The following four approaches provide specific illustrations of the generalized example ofFIG.7.

In one approach, user equipment devices may communicate with each other within a home network. User equipment devices can communicate with each other directly via short-range point-to-point communication schemes described above, via indirect paths through a hub or other similar device provided on a home network, or via communications network714. Each of the multiple individuals in a single home may operate different user equipment devices on the home network. As a result, it may be desirable for various media recommendation information or settings to be communicated between the different user equipment devices. For example, it may be desirable for users to maintain consistent media recommendation application settings on different user equipment devices within a home network, as described in greater detail in Ellis et al., U.S. Patent Publication No. 2005/0251827, filed Jul. 11, 2005. Different types of user equipment devices in a home network may also communicate with each other to transmit content. For example, a user may transmit content from user computer equipment to a portable video player or portable music player.

In a second approach, users may have multiple types of user equipment by which they access content and obtain media recommendations. For example, some users may have home networks that are accessed by in-home and mobile devices. Users may control in-home devices via a media recommendation application implemented on a remote device. For example, users may access an online media recommendation application on a website via a personal computer at their office, or a mobile device such as a PDA or web-enabled mobile telephone. The user may set various settings (e.g., recordings, reminders, or other settings) on the online recommendation application to control the user's in-home equipment. The online guide may control the user's equipment directly, or by communicating with a media recommendation application on the user's in-home equipment. Various systems and methods for user equipment devices communicating, where the user equipment devices are in locations remote from each other, is discussed in, for example, Ellis et al., U.S. Pat. No. 8,046,801, issued Oct. 25, 2011, which is hereby incorporated by reference herein in its entirety.

In a third approach, users of user equipment devices inside and outside a home can use their media recommendation application to communicate directly with content source716to access content. Specifically, within a home, users of user television equipment702and user computer equipment704may access the media recommendation application to navigate among and locate desirable content. Users may also access the media recommendation application outside of the home using wireless user communications devices706to navigate among and locate desirable content.

In a fourth approach, user equipment devices may operate in a cloud computing environment to access cloud services. In a cloud computing environment, various types of computing services for content sharing, storage or distribution (e.g., video sharing sites or social networking sites) are provided by a collection of network-accessible computing and storage resources, referred to as “the cloud.” For example, the cloud can include a collection of server computing devices, which may be located centrally or at distributed locations, that provide cloud-based services to various types of users and devices connected via a network such as the Internet via communications network714. These cloud resources may include one or more content sources716and one or more media recommendation data sources718. In addition or in the alternative, the remote computing sites may include other user equipment devices, such as user television equipment702, user computer equipment704, and wireless user communications device706. For example, the other user equipment devices may provide access to a stored copy of a video or a streamed video. In such embodiments, user equipment devices may operate in a peer-to-peer manner without communicating with a central server.

The cloud provides access to services, such as content storage, content sharing, or social networking services, among other examples, as well as access to any content described above, for user equipment devices. Services can be provided in the cloud through cloud computing service providers, or through other providers of online services. For example, the cloud-based services can include a content storage service, a content sharing site, a social networking site, or other services via which user-sourced content is distributed for viewing by others on connected devices. These cloud-based services may allow a user equipment device to store content to the cloud and to receive content from the cloud rather than storing content locally and accessing locally-stored content.

A user may use various content capture devices, such as camcorders, digital cameras with video mode, audio recorders, mobile phones, and handheld computing devices, to record content. The user can upload content to a content storage service on the cloud either directly, for example, from user computer equipment704or wireless user communications device706having content capture feature. Alternatively, the user can first transfer the content to a user equipment device, such as user computer equipment704. The user equipment device storing the content uploads the content to the cloud using a data transmission service on communications network714. In some embodiments, the user equipment device itself is a cloud resource, and other user equipment devices can access the content directly from the user equipment device on which the user stored the content.

Cloud resources may be accessed by a user equipment device using, for example, a web browser, a media recommendation application, a desktop application, a mobile application, and/or any combination of access applications of the same. The user equipment device may be a cloud client that relies on cloud computing for application delivery, or the user equipment device may have some functionality without access to cloud resources. For example, some applications running on the user equipment device may be cloud applications, i.e., applications delivered as a service over the Internet, while other applications may be stored and run on the user equipment device. In some embodiments, a user device may receive content from multiple cloud resources simultaneously. For example, a user device can stream audio from one cloud resource while downloading content from a second cloud resource. Or a user device can download content from multiple cloud resources for more efficient downloading. In some embodiments, user equipment devices can use cloud resources for processing operations such as the processing operations performed by processing circuitry described in relation toFIG.6.

As referred herein, the term “in response to” refers to initiated as a result of. For example, a first action being performed in response to a second action may include interstitial steps between the first action and the second action. As referred herein, the term “directly in response to” refers to “caused by.” For example, a first action being performed directly in response to a second action may not include interstitial steps between the first action and the second action.

FIG.8is a flowchart of illustrative steps involved in recommending media assets to a user. As shown inFIG.8, in accordance with some embodiments, a process800may be executed by processing circuitry606(FIG.6). It should be noted that process800or any step thereof could be performed on, or provided by, the system and/or devices ofFIGS.6and7. In addition, one or more steps of process800may be incorporated into or combined with one or more other steps described herein. For example, process800may be executed by control circuitry604ofFIG.6as instructed by a media recommendation application implemented on a user device in order to provide media asset recommendations to users. Also, one or more steps of process800may be incorporated into or combined with one or more steps of any other process or embodiment.

Process800begins at step810. At step820, process800includes the media recommendation application accessing a media asset recommendation model (i.e., the model). The model may correspond to a particular user, and may include a plurality of states that correspond to different genres or categories of media assets. The states each also have a corresponding plurality of possible user input options available to the user, and a state value that corresponds to a reward value and selection probability of the user input options available in each state. Each state also includes a set of media asset recommendations, wherein these recommendations comprise a list or ranking of media assets based on how relevant they are to the user, and/or how likely the user is to view the media asset.

At step830, the process800includes the media recommendation application determining a current state corresponding to the user. This can include control circuitry determining the initial genre to present to the user when the user begins the browsing session, the current genre that is being browsed by the user, or some other determination of the genre or state being presented to the user.

At step840, the process800includes the media recommendation application providing a first set of media asset recommendations to the user based on the current state. This can include input/output circuitry presenting a first subset of media assets on a display, and transmitting a second subset of media assets for storage and later display (e.g., after the user scrolls to the right).

At step850, the process800includes the media recommendation application determining whether a user input option has been received. This may be done using input/output circuitry. The user input options can include a scroll input, genre change input, selection of a media asset, exiting the application, and more. If no input is received, the process may include the media recommendation application maintaining the display of the first set of media asset recommendations, and remaining at step850until a user input option is received.

When a user input option is received at step850, the process800includes control circuitry of the media recommendation application determining a second set of media asset recommendations based on the received user input at step860. The second set of media asset recommendations can include one or more of the same media asset recommendations as the first set. The second set can include a replacement or reordering of one or more media assets. The second set may be determined based on the received user input. For example, if the user input includes scrolling to the right, the second set of media asset recommendations can include one or more media assets from a different genre than the currently browsed genre (i.e., because the likelihood of the user selecting a title from the currently browsed genre has decreased).

At step870, the process800then includes the input/output circuitry of the media recommendation application providing the second set of media asset recommendations to the user. This can include replacing or inserting the second set of media asset recommendations onto the display viewed by the user. This can also include replacing one or more of the currently visible media asset recommendations, and/or replacing one or more of the stored media asset recommendations (e.g., the second subset that is stored for later display).

After the media recommendation application provides the second set of media asset recommendations, the process800may continue back to step850to await further user input. Alternatively, the process800may end in response to a user input exiting the application, a time-out, or a threshold duration of time passing without user input.

It is contemplated that the steps or descriptions of FIG. * may be used with any other embodiment of this disclosure. In addition, the steps and descriptions described in relation toFIG.8may be done in alternative orders or in parallel to further the purposes of this disclosure. For example, each of these steps may be performed in any order or in parallel or substantially simultaneously to reduce lag or increase the speed of the system or method. Furthermore, it should be noted that any of the devices or equipment discussed in relation toFIGS.6-7could be used to perform one or more of the steps inFIG.8.

The above-described embodiments of the present disclosure are presented for purposes of illustration and not of limitation, and the present disclosure is limited only by the claims that follow. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and flowcharts or examples relating to one embodiment may be combined with any other embodiment in a suitable manner, done in different orders, or done in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and/or methods described above may be applied to, or used in accordance with, other systems and/or methods.