Generating digital event sequences utilizing a dynamic user preference interface to modify recommendation model reward functions

The present disclosure relates to generating and modifying recommended event sequences utilizing a dynamic user preference interface. For example, in one or more embodiments, the system generates a recommended event sequence using a recommendation model trained based on a plurality of historical event sequences. The system then provides, for display via a client device, the recommendation, a plurality of interactive elements for entry of user preferences, and a visual representation of historical event sequences. Upon detecting input of user preferences, the system can modify a reward function of the recommendation model and provide a modified recommended event sequence together with the plurality of interactive elements. In one or more embodiments, as a user enters user preferences, the system additionally modifies the visual representation to display subsets of the plurality of historical event sequences corresponding to the preferences.

BACKGROUND

Recent years have seen significant improvement in hardware and software platforms for generating digital sequence recommendations. For example, digital recommendation systems can automatically analyze events (e.g., points of interest, objects, or actions) that are otherwise too numerous to explore and generate digital recommendations for sequentially performing selected events. For example, a digital recommendation system may generate a digital recommendation for particular points of interest for an individual visiting a new location and may further recommend the order for visiting the points of interest, providing the information necessary to make efficient use of available time.

Despite these advantages, conventional digital recommendation systems have several technological shortcomings that result in inaccurate, inflexible, and inefficient operation. For example, conventional digital recommendation systems often struggle to generate accurate digital recommendations. To illustrate, conventional digital recommendation systems often utilize a black-box approach that generates a digital recommendation without regard to dynamic, individual circumstances or preferences. Consequently, conventional systems often generate digital sequence recommendations to users who have no interest in pursuing one or more of the suggested events. Furthermore, this black-box approach also fails to convince users regarding accuracy of the generated recommendations, resulting in decreased utilization of such conventional systems.

In addition to accuracy concerns, conventional recommender systems are also inflexible. In particular, because conventional digital recommendation systems typically generate recommended event sequences utilizing a “black-box” approach, such systems generally recommend the same event sequences. In other words, conventional systems rigidly recommend a default event sequence even if a user is unsatisfied with the digital recommendation or even if different users have different preferences.

In addition to problems with accuracy and flexibility, conventional digital recommendation systems are also inefficient. Indeed, conventional systems often employ inefficient models to generate recommendations. For example, conventional systems often employ models that require a significant amount of time for training. This not only leads to slow operation of the systems, but it also requires a significant amount of computing resources (e.g., processing power). Further, in making model adjustments, conventional systems typically require retraining of these models, requiring further use of time and resources.

SUMMARY

One or more embodiments described herein provide benefits and/or solve one or more of the foregoing or other problems in the art with systems, methods, and non-transitory computer readable storage media that generate and modify digital recommended event sequences utilizing a dynamic user preference interface. For instance, the disclosed systems can provide a user interface with interactive elements for entry of user preferences. The disclosed systems can dynamically generate recommended event sequences by modifying a reward function of a recommendation model based on various user preferences identified via the interactive elements. By modifying a reward function and providing digital sequence recommendations in relation to different user preferences, the disclosed systems can provide controls and guidance to more accurately and flexibly personalize digital recommendation sequences. Moreover, the disclosed systems can provide interactive sequence recommendations that utilize digital visualizations to accurately and efficiently explain and justify the digital sequence recommendations.

For example, in one or more embodiments, the disclosed systems generate a recommended event sequence and provide the recommendation in a user interface displayed via a client device. In particular, the disclosed systems generate the recommended event sequence using a recommendation model that has been trained based on a plurality of historical event sequences. Additionally, the recommendation model includes a reward function that facilitates generation of recommendations. In one or more embodiments, the disclosed systems provide a plurality of interactive elements in the user interface that allow for entry of user preferences. Upon detecting input of one or more user preferences via the interactive elements, the system can modify the reward function of the recommendation model and generate a modified recommended event sequence. In one or more embodiments, the system further provides a visual representation of historical event sequences in the user interface. As a user enters user preferences, the system can modify the visual representation to display subsets of the plurality of historical event sequences corresponding to the preferences. In this manner, the disclosed systems can accurately, flexibly, and efficiently provide event sequence recommendations that are personalized to the user's preferences while informing the user of the justification for the recommendations.

The following description sets forth additional features and advantages of one or more embodiments of the disclosed systems, computer readable storage media, and methods. In some cases, such features and advantages will be obvious to a skilled artisan from the description or may be learned by the practice of the disclosed embodiments.

DETAILED DESCRIPTION

One or more embodiments described herein include an event sequence recommender system that generates and modifies digital recommendation event sequences utilizing a dynamic user preference interface. In particular, the event sequence recommender system can utilize machine learning to provide interactive sequence recommendations together with digital visualizations of user preferences and historical data utilized to generate the sequence recommendations. For example, the event sequence recommender system can generate a recommended event sequence using a recommendation model that includes a reward function. In one or more embodiments, the event sequence recommender system provides a user interface that displays both the generated recommendation and a plurality of interactive elements for entry of user preferences. The event sequence recommender system can dynamically detect different user preferences via the interactive elements, modify the reward function of the recommendation model, and provide a modified recommended event sequence with the interactive elements. Thus, the event sequence recommend system can provide accurate sequence recommendations with visualizations that explain the recommendation generation process while efficiently and flexibly personalizing recommended action plans.

To illustrate, in one or more embodiments, the event sequence recommender system generates a recommended event sequence by utilizing a recommendation model (with a reward function) trained based on a plurality of historical event sequences. The event sequence recommender system then provides, for display via a client device, a user interface that displays the recommended event sequence. The user interface further displays a plurality of interactive elements through which a user can enter user preferences. Upon detecting successive entries of user preferences via the interactive elements, the event sequence recommender system can iteratively modify the reward function of the recommendation model to generate modified recommended event sequences.

As just mentioned, the event sequence recommender system can generate recommended event sequences by utilizing a recommendation model trained based on a plurality of historic event sequences (i.e., sequences of events engaged in by previous users). The event sequence recommender system can utilize a variety of trained recommendation models. In one or more embodiments, the event sequence recommender system trains the recommendation model by using historical event sequences to build a probabilistic suffix tree. Using the probabilistic suffix tree, the event sequence recommender system can construct Markov Decision Process models and use the Markov Decision Process models in conjunction with Thompson sampling to determine an optimal recommended event sequence.

As discussed, in one or more embodiments, the recommendation model includes a reward function for generating recommendations. For instance, a reward function can indicate a frequency or desirability of one or more events (e.g., as determined by historical event sequences or some other source). The event sequence recommender system can utilize the reward function to determine the expected value of a recommendation. For example, the event sequence recommender system can perform Thompson sampling according to a reward function that reflects the value of events within an event sequence.

Further, as mentioned above, the event sequence recommender system can provide a plurality of interactive elements through which a user can input user preferences (e.g., particular events of interest). Upon detecting input of user preferences, the event sequence recommender system modifies the reward function of the recommendation model. In one or more embodiments, the event sequence recommender system modifies the reward function by providing a weighting factor to parameters of the reward function corresponding to the user preferences. For example, if a user inputs a preference indicating a desire to engage in a particular event, the event sequence recommender system can modify parameters of the reward function using a weighting factor to indicate that the event would provide a higher expected value for the user. The event sequence recommender system can then generate a modified recommended event sequence using the recommendation model and based on the modified reward function.

In one or more embodiments, the plurality of interactive elements includes a set of interactive elements for entry of user constraints (i.e., limitations on eligible event sequences). The event sequence recommender system can recommend an event sequence further based on the submitted user constraints. For example, where a user submits a limit to the number of events that can be included within an event sequence, the event sequence recommender system can recommend an event sequence that includes a number of events equal to or less than the limit.

As further mentioned above, the event sequence recommender system provides a user interface that displays the recommended event sequence via a client device. Additionally, the user interface can display the plurality of interactive elements for entry of user preferences. Where a user has entered user preferences and the event sequence recommender system accordingly generates a modified recommended event sequence, the user interface can display the modified recommended event sequence and the original (i.e., unmodified) recommended event sequence simultaneously.

In one or more embodiments, the user interface further includes a visual representation of historical event sequences. For example, the visual representation can display all of the historical event sequences upon which the recommended event sequences are based. In some embodiments, the visual representation displays only a subset of the historical event sequences. Moreover, where the event sequence recommender system generates a modified recommended event sequence due to entry of user preferences, the system can modify the visual representation to display one or more subsets of the historical event sequences corresponding to the user preferences. For example, where a user submits a preference to engage in a particular event, the event sequence recommender system can modify the visual representation to display only the subset of historical event sequences that include that event.

The event sequence recommender system provides several advantages over conventional systems. For example, the event sequence recommender system can improve the accuracy of implementing computer systems. In particular, the event sequence recommender system recommends event sequences that more accurately satisfy user preferences. To illustrate, by providing interactive elements that allow for dynamic, iterative entry of user preferences, the event sequence recommender system can determine the events that are most valuable to the user. By generating modified recommendations based on the submitted preferences, the event sequence recommender system can then accurately recommend valuable events to the user.

In addition, by providing interactive visualizations of user preferences together with recommended event sequences, the event sequence recommender system can increase user engagement and accuracy of resulting recommendations. Indeed, as outlined below, researchers have conducted experiments with regard to interactions by users with the event sequence recommender system. The results indicate that users are more likely to engage with implementations of the event sequence recommender system and refine the results to more accurately generate recommended event sequences that align to user preferences.

Further, the event sequence recommender system improves flexibility. In particular, by modifying the reward function of the recommendation model to then generate a modified recommended event sequence, the event sequence recommender system can flexibly adjust its recommendations based on the needs of a user. For example, upon detecting a user preference indicating that a user strongly desires to engage in a particular event, the event sequence recommender system can modify the recommended event sequence to include that event. In addition, the event sequence recommender system can flexibly apply to any variety of different events or event sequences. For example, the event sequence recommender system can generate recommended event sequences with regard to a sequence of points of interest for a user to visit, a sequence of digital content transmissions across client devices, or a sequence of other events.

The event sequence recommender system also improves the efficiency of implementing computing devices. For example, the event sequence recommender system can more efficiently provide recommendations in response to entry of user preferences. Specifically, by modifying the reward function in accordance with the user preferences, the event sequence recommender system can avoid retraining the recommendation model (and avoid a significant amount of time and processing power). Consequently, the event sequence recommendation system can use the previously trained recommendation model in conjunction with the modified reward function to quickly generate modified recommendations. The reduction in time and computing resources only increases as users repeatedly adjust user preferences before accepting a recommendation.

As illustrated by the foregoing discussion, the present disclosure utilizes a variety of terms to describe features and benefits of the image relighting system. Additional detail is now provided regarding the meaning of these terms. As used herein, the term “event” refers to an occurrence. In particular, an event can refer to the occurrence of an action performed by a user or computing device. For example, an event can include the occurrence of a user visiting a point of interest (e.g., a tourist attraction) or the occurrence of a computing device transmitting a digital communication (e.g., a transmission of digital content). As used herein, the term “event sequence” refers to a sequentially ordered combination of events. Similarly, as used herein, the term “historical event sequence” refers to a previous sequence of events (performed by one or more previous users or computing devices).

Additionally, as used herein, the term “user preference” refers to a user partiality (e.g., like or dislike), inclination, selection, or desire with respect to an event or an event sequence. In particular, a user preference includes a user desire to engage in or avoid a particular event, category of event, or other action or characteristic related to an event. A user preference can be represented using a binary value (e.g., a value indicating that the user either wants to engage in or avoid an event) or using a more granular scale (e.g., a number scale where a higher number indicates a stronger preference to engage in an event and a lower number indicates a stronger preference to avoid the event).

Further, as used herein, the term “user constraint” refers to a limitation. In particular, a user constraint includes a limitation, selected by a user, on an event sequence. For example, a user constraint can limit the number of events to be included within a recommended event sequence, a time required to complete the recommended event sequence, or the methods or resources used to complete the recommended event sequence.

Additionally, as used herein, the term “recommendation model” refers to a computer algorithm or model that generates event sequence recommendations. In particular, a recommendation model includes a computer algorithm that recommends a sequence of events based on the highest expected reward provided by the available event sequences as determined by historical data. For example, the recommendation model can include a machine learning model. As used herein, a “machine learning model” refers to a computer representation that can be tuned (e.g., trained) based on inputs to approximate unknown functions. In particular, the term “machine-learning model” can include a model that utilizes algorithms to learn from, and make predictions on, known data by analyzing the known data to learn to generate outputs that reflect patterns and attributes of the known data. For instance, a machine-learning model can include but is not limited to a neural network (e.g., a convolutional neural network or deep learning), a decision tree (e.g., a gradient boosted decision tree), association rule learning, inductive logic programming, support vector learning, Bayesian network, regression-based model, principal component analysis, or a combination thereof.

Additional detail regarding the event sequence recommender system will now be provided with reference to the figures. For example,FIG. 1illustrates a schematic diagram of an exemplary system environment (“environment”)100in which an event sequence recommender system106can be implemented. As illustrated inFIG. 1, the environment100can include a server(s)102, a network108, client devices110a-110n, and users114a-114n.

AlthoughFIG. 1illustrates a particular number of client devices, it will be appreciated that the environment100can include any number of computing devices (fewer or greater than shown). Similarly, althoughFIG. 1illustrates a particular arrangement of the server(s)102, the network108, the client devices110a-110n, and the users114a-114n, various additional arrangements are possible.

The server(s)102, the network108, and the client devices110a-110nmay be communicatively coupled with each other either directly or indirectly (e.g., through the network108, discussed in greater detail below in relation toFIG. 11). Moreover, the server(s)102and the client devices110a-110nmay include any type of computing device (including one or more computing devices as discussed in greater detail below in relation toFIG. 11).

As mentioned above, the environment100includes the server(s)102. The server(s)102can generate, store, receive, and/or transmit any type of data, including the plurality of historical event sequences and the user preferences. For example, the server(s)102can receive user preferences from the client device110a. In one or more embodiments, the server(s)102comprises a data server. The server(s)102can also comprise a communication server or a web-hosting server.

As shown inFIG. 1, the server(s)102can include the analytics system104. In particular, the analytics system104can collect analytics data. For example, the analytics system104can collect analytics data related to a user's history of engagement with one or more events, such as the events with which the user has engaged and the sequence of events engaged. The analytics system104can collect the analytics data in a variety of ways. For example, in one or more embodiments, the analytics system104causes the server(s)102and/or a third-party server to track user data (e.g., event sequence data) and report the tracked user data for storage on a database. In one or more embodiments, the analytics system104receives user data directly from the client devices110a-110nvia data stored thereon.

Additionally, the server(s)102can include the event sequence recommender system106. In particular, in one or more embodiments, the event sequence recommender system106uses the server(s)102to generate modified recommended event sequences based on user preferences. For example, the event sequence recommender system106can use the server(s)102to receive one or more user preferences and then generate a modified recommended event sequence.

For example, in one or more embodiments, the server(s)102can receive a plurality of historical event sequences and train a recommendation model that includes a reward function based on the plurality of historical event sequences. The server(s)102can then provide, for display via a client device (e.g., one of the client devices110a-110n) a user interface that includes the recommended event sequence as well as a visual representation of historical event sequences and a plurality of interactive elements that allow for entry of user preferences. Upon detecting submission of one or more user preferences via the interactive elements, the server(s)102can modify the reward function based on the preferences submitted and generate a modified recommended event sequence. The server(s)102can then provide the modified recommended event sequence for display via the client device and continue to provide the interactive elements for further submission of user preferences.

As illustrated by the previous example embodiments, the event sequence recommender system106can be implemented in whole, or in part, by the individual elements of the environment100. AlthoughFIG. 1illustrates the event sequence recommender system106implemented with regards to the server(s)102, it will be appreciated that components of the event sequence recommender system106can be implemented in any of the components of the environment100.

In one or more embodiments, the client devices110a-110ninclude computer devices that allow users of the devices (e.g., the users114a-114n) to access and interact with digital content. For example, the client devices110a-110ncan include smartphones, tablets, desktop computers, laptop computers, or other electronic devices. The client devices110a-110ncan include one or more applications (e.g., the recommendation application112) that allows users114a-114nto access and interact with digital content. For example, the recommendation application112can include a software application installed on the client devices110a-110n. Additionally, or alternatively, the recommendation application112can include a software application hosted on the server(s)102, which may be accessed by the client devices110a-110nthrough another application, such as a web browser.

As just mentioned, the event sequence recommender system106can generate recommendations based on historical event sequences and user preferences.FIGS. 2A-2Billustrate block diagrams for generating event sequence recommendations in accordance with one or more embodiments. In particular,FIG. 2Ashows a block diagram for generating a recommended event sequence based on a plurality of historical event sequences. Moreover,FIG. 2Billustrates a block diagram of generating a modified recommended event sequence based on both the plurality of historical event sequences and submitted user preferences.

As can be seen inFIG. 2A, the event sequence recommender system106uses a plurality of historical event sequences204to generate a recommended event sequence222. In one or more embodiments, the event sequence recommender system106uses the plurality of historical event sequences204to train a recommendation model, which then generates the recommended event sequence222as will be discussed in more detail below with reference toFIGS. 3-4. For simplicity,FIG. 2Aillustrates a plurality of historical event sequences including three historical event sequences; however, it should be noted that the event sequence recommender system106can use any number of historical event sequences.

As shown inFIG. 2A, the plurality of historical event sequences204includes a first historical event sequence206, a second historical event sequence208, and a third historical event sequence210. Each of the historical event sequences206-210shown inFIG. 2Arepresents a sequence of points of interest (referred to here as “POIs”) visited by a corresponding user while at a location. For example, the first historical event sequence206represents a sequence where the corresponding user visited POI_A212followed by POI_B214and then POI_C216. Similarly, the second historical event sequence208represents a sequence where the corresponding user visited POI_A212followed by POI_D218and POI_E220. And the third historical event sequence210represents a sequence where the corresponding user visited POI_A212followed by POI_D218and POI_C216. In one or more embodiments, one or more of the plurality of historical event sequences204represents a sequence of more or less points of interest.

The event sequence recommender system106generates the recommended event sequence222based on the plurality of historical event sequences204. For example, the recommended event sequence222shown inFIG. 2Arecommends a sequence of several points of interest for a user to visit. As can be seen inFIG. 2A, the recommended event sequence222recommends for the user to visit POI_A212followed by POI_D218and then POI_C216.

FIG. 2Billustrates the event sequence recommender system106using the plurality of historical event sequences204as well as user preferences224to generate a modified recommended event sequence226. As shown inFIG. 2B, the user preferences224indicate that the user has submitted a preference for visiting POI_B214and a preference for avoiding POI_C216. Accordingly, the event sequence recommender system106modifies the recommended event sequence222ofFIG. 2A. The modified recommended event sequence226recommends for the user to visit POI_A212followed by POI_B214and POI_E220.FIG. 2Bshows that, by generating the modified recommended event sequence226based on the user preferences224, the event sequence recommender system106provides a recommendation that is personalized for the user that submitted the user preferences224. Therefore, by incorporating user preferences, the event sequence recommender system106provides recommendations that users are more likely to follow.

As mentioned above, the event sequence recommender system106uses a plurality of historical event sequences to generate recommended event sequences. In particular, the event sequence recommender system106generates recommended event sequences by utilizing a recommendation model trained based on the plurality of historical event sequences.FIG. 3illustrates a block diagram of training the recommendation model in accordance with one or more embodiments. In one or more embodiments, the event sequence recommender system106trains the recommendation model in accordance with the methods disclosed in G. Theocharous, N. Vlassis, and Z. Wen, 2017 An Interactive Points of Interest Guidance System inProceedings of the International Conference on Intelligent User Interfaces Companion,49-52, which is incorporated herein by reference in its entirety and summarized below.

As can be seen inFIG. 3, the event sequence recommender system106uses the historical event sequences302to build the recommendation model304. In particular, the event sequence recommender system106constructs a probabilistic suffix tree306based on the historical event sequences302. The probabilistic suffix tree306includes a plurality of nodes, where each node represents a suffix (i.e., a sequential history of events leading up to that node). Accordingly, the suffix of each leaf node (i.e., final node) of the probabilistic suffix tree306represents a historical event sequence from the historical event sequences302taken by one or more previous users. In one or more embodiments, the event sequence recommender system106uses the probabilistic suffix tree306to represent every historical event sequence from the historical event sequences302.

In addition to representing histories of events, each node is associated with a probability distribution that provides a probability of user engagement for each of the available events in light of the history of events with which the user has already engaged as represented by the suffix associated with that node. To illustrate, where four events are available for engagement and the suffix of a particular node represents that a user has already engaged in the first event followed by the second event, the probability distribution provides, in light of the user having already engaged in the first event followed by the second event, a first probability that the user will subsequently engage in the first event (again), a second probability that the user will subsequently engage in the second event (again), a third probability that the user will subsequently engage in the third event, and a fourth probability that the user will engage in the fourth event. The event sequence recommender system106determines these probabilities from the historical event sequences302.

Thus, depending on what a user has already done, the event sequence recommender system106can use the probabilistic suffix tree306to predict what the user will do next. For example, based on the probabilistic suffix tree306, the event sequence recommender system106can represent the probability that a user will engage in a subsequent event s=st+1given the sequence of events with which the user has already engaged X=(s1, s2. . . st) as P(s|X) and can represent the probability of an entire sequence of events as P(X). The event sequence recommender system106determines P(X) as the product of the probability of each event in the sequence given the maximum suffix available in the tree (i.e., the leaf node representing the event sequence).

In one or more embodiments, the event sequence recommender system106constructs the probabilistic suffix tree306from the historical event sequences302using the pstree algorithm. In particular, the pstree algorithm can take multiple parameters (e.g., depth of the tree, the number of minimum occurrences of a suffix, and parametrized tree pruning methods) as input. To select the best set of parameters, the event sequence recommender system106uses the Modified Akaike Information Criterion (AICc) as follows:

In the equation 1, log() represents the log likelihood of a set of sequences and is determined by the following where S represents all of the events appearing in the historical event sequences302and X maps the longest suffix available in the tree for each event:
log()=Σs∈Slog(P(s|X))  (2)

After constructing the probabilistic suffix tree306, the event sequence recommender system106constructs action models308for representing various personas (i.e., user types) based on the probabilistic suffix tree306. In particular, the event sequence recommender system106constructs the action models308by parameterizing the probabilistic suffix tree306into multiple models where each model corresponds to a different persona. In one or more embodiments, the event sequence recommender system106creates the action models by perturbing the passive dynamics of the probabilistic suffix tree306so that each of the action models308increases the probability that a user having a persona corresponding to the action model listens to the resulting recommendation. In one or more embodiments, the event sequence recommender system106generates the action models308using the following equation where θ represents the user type and z is a normalizing factor:

From the action models308, the event sequence recommender system106constructs the Markov Decision Process models310. The states/contexts of each of the Markov Decision Process models310correspond to the nodes of the probabilistic suffix tree306. Using x to represent a suffix available in the probabilistic suffix tree306, the event sequence recommender system106determines the probability of transitioning from every node to every other node by finding resulting longest suffixes in the tree for every additional event that an action can produce as provided by the action models308. The probability of transitioning can be represented as P(x′|x, a, θ). The event sequence recommender system106further represents the reward of the Markov Decision Process models310using the following reward function where xnis the last event of the suffix x:
r(x)=r(xn)  (5)

In some embodiments, a reward for a particular event indicates a frequency or desirability of that event as determined by the historical event sequences. In other embodiments, the reward is determined using a different dataset that provides the value of each event (e.g., a dataset that includes responses to survey questions that asked about the value of particular events). The event sequence recommender system106utilizes the reward function to compute the expected value of a recommendation. A higher reward indicates a higher expected value.

Using policy iteration, the event sequence recommender system106can determine the optimal policies and value function Vθ*(x) corresponding to each of the Markov Decision Process models310. In particular, a policy includes a function that specifies the action a user will take when in a particular state of the model.

After constructing the Markov Decision Process models310, the event sequence recommender system106uses Thompson Sampling, which chooses actions in real time to maximize the expected experience as calculated by the reward on each state. In particular, through Thompson Sampling, the event sequence recommender system106recommends actions based on their probability of maximizing the expected reward as shown below:
∫[(r|X,a,θ)=maxa′(|X,a′,θ)]P(θ|)dθ(6)

In the equation above, X represents the current context and={(X; a; r)} represents past observations of contexts, actions, and rewards. The event sequence recommender system106implements Thompson Sampling by sampling, in each round, a parameter θ* from the posterior P(θ|), and choosing the action a* that maximizes[r|X a*, θ*] (i.e., the expected reward given the parameter, the action, and the current context). Algorithm 1 presented below describes the procedure in detail in accordance with one or more embodiments.

As can be seen in Algorithm 1, in one or more embodiments, the event sequence recommender system106uses the recommendation model304to provide the entire recommended event sequence314to the user before the user has engaged in any events. For example, in one or more embodiments, the recommendation model304determines the first recommended action for the user (i.e., the first event of the recommended event sequence314), assumes that the user will follow the recommendation, determines the second recommended action for the user (i.e., the second event of the recommended event sequence314) based on that assumption, and continuously iterates through the process until the recommended event sequence314has been determined.

Once the recommendation model has been trained, the event sequence recommender system106can accept user preferences and generate modified recommended event sequences.FIG. 4provides a block diagram representing one or more embodiments in which the event sequence recommender system106uses the recommendation model to generate a modified recommended event sequence based on user preferences. In one or more embodiments, the recommendation model418shown inFIG. 4is the same as the recommendation model304illustrated inFIG. 3.

As shown inFIG. 4, the event sequence recommender system106provides a set of user preferences402and the historical event sequences416to the recommendation model418. In particular, the user preferences402include preferred POI categories404, preferred POIs406. The preferred POI categories404indicate one or more point of interest types that the submitting user prefers to visit or avoid. For example, the preferred POI categories404can indicate that the user prefers to visit or avoid public parks, museums, or places of entertainment. The preferred POIs406indicate one or more specific points of interest that the user prefers to visit or avoid. For example, the preferred POIs406can indicate that the user prefers to visit or avoid a specific public park (e.g., Central Park), a specific museum (e.g., the Metropolitan Museum), or a specific place of entertainment (e.g., Madison Square Garden).

In one or more embodiments, the preferred POI categories404or the preferred POIs406can indicate that one or more point of interest categories or particular points of interest, respectively, must be included in or excluded from the recommended event sequence. For example, the preferred POIs406can indicate that the resulting recommendation must include a visit to Central Park. Accordingly, the event sequence recommender system106will generate a recommendation that includes Central Park.

In providing the user preferences402, the event sequence recommender system106modifies the reward function420of the recommendation model418in order to generate the modified recommended event sequence422. In particular, the event sequence recommender system106modifies the reward function420without modifying the entire recommendation model418, allowing the event sequence recommender system106to generate the modified recommended event sequence422without having to retrain the recommendation model418. The event sequence recommender system106modifies the reward function420as presented by equation 5 by introducing a weighting factor w to represent the user's preferences as follows:
r(x)=w(xn)·r(xn)  (7)

In one or more embodiments, the weighting factor w includes parameters associated with each of the points of interest available for the user to visit. For example, the weighting factor w includes a first parameter associated with a first point of interest and a second parameter associated with a second point of interest. The value of the weighting factor (i.e., the preference weight associated with each parameter) corresponds to the degree of preference indicated by the user and thereby affects the expected reward associated with having a user visit a particular point of interest and the likelihood that the point of interest will be included in the resulting event sequence. For example, if a user visiting New York City indicates a strong preference for visiting Central Park, the event sequence recommender system106modifies the reward function420by providing a corresponding preference weight to the parameter of the weighting function w associated with Central Park so that the resulting reward for having the user visit Central Park increases. Consequently, the event sequence recommender system106is more likely to include Central Park in the modified recommended event sequence422.

In one or more embodiments, upon providing the preferred POI category404, the event sequence recommender system106applies a corresponding preference weight to the parameters associated with each point of interest that falls under that category. For example, if a user visiting New York City indicates a strong preference for visiting public parks, the event sequence recommender system106modifies the reward function420by providing a corresponding preference weight to the parameters associated with each point of interest that is categorized as a public park (e.g., Central Park, Thomas Jefferson Park, Isham Park, etc.).

As shown inFIG. 4, the user preferences402further include the user constraints408. In particular, the user constraints408include a maximum time410, a maximum distance412, and a maximum POIs414. The maximum time410indicates a limit on the time required to visit the points of interest in the recommended event sequence. For example, the maximum time410can indicate that a recommended event sequence should require no more than three hours for the user to visit all of the included points of interest. The maximum distance412indicates a limit on the distance required for a user to travel in order to visit all of the points of interest in a recommended event sequence. For example, the maximum distance412can indicate that a recommended event sequence should require the user to travel no more than fifteen miles to visit all of the included points of interest. The maximum POIs414indicates a limit on the number of points of interest included within a recommended event sequence. For example, the maximum POIs414can indicate that the recommended event sequence should include no more than four points of interest.

In one or more embodiments, the event sequence recommender system106modifies the reward function420when providing some of the user preferences402—such as the preferred POI category404and the preferred POI406—but does not modify the reward function420when providing the user constraints408. Rather, the event sequence recommender system106utilizes the user constraints408to filter out (i.e., avoid selecting) eligible event sequences when generating the modified recommended event sequence422. For example, where the maximum time410indicates that a recommended event sequence should require no more than three hours for the user to visit all of the points of interest included within the event sequence, the event sequence recommender system106filters out all event sequences that would require the user to spend more than three hours visiting the included points of interest-even if those event sequences provide a higher total reward as provided by the reward function420.

After generating the modified recommended event sequence422, the event sequence recommender system106can receive further input of user preferences via the interactive elements, such as a modification of the user preferences402. Subsequently, the event sequence recommender system106can further modify the reward function420of the recommendation model418(if appropriate) and generate another modified recommended event sequence based on the recommendation model and the modified reward function. Therefore, the algorithms and acts described in reference toFIGS. 3-4can comprise the corresponding structure for performing a step for iteratively generating modified recommended event sequences by modifying the reward function of the recommendation model based on detecting user preferences via the plurality of interactive elements.

Turning now toFIG. 5A, this figure illustrates a user interface500that the event sequence recommender system106provides for display via a client device in accordance with one or more embodiments. In particular, the user interface500displays recommended event sequences and interactive elements for identifying and illustrating user preferences utilized to generate recommended event sequences. More specifically, as illustrated inFIG. 5A, the user interface500includes a recommendation view502, a user preferences view504, and a historical event sequences view506.

In some embodiments, the user interface500provides only the recommendation view502initially (i.e., when the event sequence recommender system106initially provides the user interface500—such as when the application running the event sequence recommender system106activates) but provides an option to expand the user interface500to further display the user preferences view504and the historical event sequences view506. In further embodiments, the user interface500always displays the views502-506at all times.

As illustrated inFIG. 5A, the recommendation view502includes a map display508, a general recommendation510, a most popular sequence512, and a personalized recommendation514. In particular, the map display508displays the event sequence corresponding to one of the general recommendation510, the most popular sequence512, or the personalized recommendation514. More specifically, the map displays the points of interest included in the event sequence and the recommended trajectory for visiting the points of interest superimposed over a map of the location to which the user is visiting. For example, if the event sequence provided points of interest to visit within New York City, the map display508would superimpose the event sequence over a map of the area of New York City containing the points of interest. In one or more embodiments, the user interface500can change which event sequence is displayed in the map display508in response to detecting a user selection of one of the general recommendation510, the most popular sequence512, or the personalized recommendation514.

The general recommendation510provides a recommended event sequence based on a plurality of historical event sequences. In particular, the general recommendation510provides a recommended event sequence generated by the event sequence recommender system106before a user has submitted user preferences (e.g., the recommended event sequence314ofFIG. 3). Because the recommended event sequence can be based only on the plurality of historical event sequences, the general recommendation510can be the same event sequence for all users.

The most popular sequence512provides the most popular event sequence as determined by the plurality of historical event sequences. In particular, the most popular sequence512provides the event sequence that was most frequently used by prior visitors to the location (e.g., regardless of the expected reward of that event sequence). Because the most frequently used event sequence is independent of any user preferences, the most popular sequence512can be the same event sequence for all users.

The personalized recommendation514provides a modified recommended event sequence generated based on the plurality of historical event sequences and one or more submitted user preferences. In particular, the personalized recommendation514provides a modified recommended event sequence (e.g., the modified recommended event sequence422ofFIG. 4) generated by the event sequence recommender system106after a user has submitted one or more user preferences via the user preferences view504. In other words, the personalized recommendation514provides an event sequence tailored to the needs and/or desires of a particular user.

As illustrated inFIG. 5A, the map display508and each of the views510-514display the events within the event sequences with visual indicators, such as the visual indicators516a-516fcorresponding to the points of interest provided in the personalized recommendation514. In particular,FIG. 5Aillustrates that the visual indicators516a-516fcan vary in size and color. In one or more embodiments, the size of a visual indicator encodes the popularity of the corresponding event. For example, the visual indicator516bis larger than the visual indicator516a, indicating that the point of interest corresponding to the visual indicator516bis more popular (i.e., more frequently visited by prior users) than the point of interest corresponding to the visual indicator516a. Additionally, as shown inFIG. 5A, the color of the visual indicator encodes the event category that corresponds to the visual indicator. For example, the color of the visual indicator516ashows that the corresponding points of interest is a place of transportation while the color of the visual indicator516bshows that the corresponding point of interest is a park.

As shown inFIG. 5A, the user preferences view504includes a plurality of interactive elements for entry of user preferences. For example, the user preferences view504includes the user constraint element518for input of a user constraint, the POI category preference element526for input of a preferred point of interest category, and the POI preference element536for input of a preferred point of interest. Each of the interactive elements include one or more components that facilitate input of user preferences. The user preferences view504further includes a submission button546that enables a user to submit user preferences after engaging with the interactive elements to adjust the desired preferences.

As illustrated inFIG. 5A, the user constraint element518includes an exclusion option520, an inclusion option522, a bar chart524and constraint indicators525a-525b. In particular, when a user selects the exclusion option520, the event sequence recommender system106ignores the corresponding user constraint when generating the modified recommended event sequence. When the user selects the inclusion option522, on the other hand, the event sequence recommender system106generates the modified recommended event sequence based, at least partly, on the corresponding user constraint. Moreover, bar charts (e.g., the bar chart524) provide additional information to the user regarding characteristics of the historical event sequences in relation to the constraints. As shown inFIG. 5A, bar charts can provide detail regarding the frequency (e.g., the number of historical event sequences) that satisfy a particular duration, distance, or number of points of interest reflected by the historical event sequences. For example, the bar chart524indicates that the majority of previous users represented by the historical event sequences only traveled to three points of interest during their visit (i.e., the majority of historical event sequences only include three different points of interest). The bar charts provide live feedback to a user regarding the number of historical event sequences (previous users) that satisfy particular selected constraints.

The constraint indicators525a-525ballow a user to adjust the constraint that will be submitted to the event sequence recommender system106. In particular, the positioning of the constraint indicators525a-525bprovides the constraint range. For example, the current positioning of the constraint indicators525a-525bshown inFIG. 5Aindicates that the current constraint range allows the event sequence recommender system106to generate a modified recommended event sequence having any number of points of interest between three and twelve, inclusive. A user can adjust the position of either or both of the constraint indicators525a-525bto modify the constraint range.

As shown inFIG. 5A, the POI category preference element526includes an exclusion option528, a preference select option530, an inclusion option532, and a slide bar534. In particular, when a user selects the exclusion option528, the event sequence recommender system106ignores the corresponding user preference. In particular, the event sequence recommender system106does not apply a preference weight to the parameter of the weighting function corresponding to the particular user preference.

When a user selects the preference select option530, the user can then interact with the slide bar534to adjust the user preference. As shown inFIG. 5A, the positioning of the marker535along the slide bar534corresponds to a numerical value indicating the user's desire to visit or avoid points of interest falling under the corresponding point of interest category. For example, in one or more embodiments, a numerical value greater than one indicates that the user prefers to visit points of interest from the corresponding point of interest category while a numerical value less than one indicates that the user would prefer to avoid such points of interest. In one or more embodiments, the event sequence recommender system106uses this numerical value directly as the preference weight applied to the parameter of the weighting function corresponding to the particular user preference (i.e., applied to the parameters associated with each of the points of interest from the point of interest category). In some embodiments, the event sequence recommender system106converts the numerical value to a corresponding preference weight.

The inclusion option532enables the user to indicate that one or more points of interest from the corresponding point of interest category must be included within the modified recommended event sequence. In one or more embodiments, when a user selects the inclusion option, the event sequence recommender system106maximizes the preference weight applied to the parameter of the weighting function that corresponds to the particular user preference. Consequently, an event sequence that includes one or more points of interest from the corresponding point of interest category is likely to be the event sequence having the highest associated reward and, therefore, likely to be selected as the modified recommended event sequence. In some embodiments, selection of the inclusion option532operates as an additional constraint. In particular, when generating the modified recommended event sequence, the event sequence recommender system106filters out any event sequence that does not include one or more points of interest from the corresponding point of interest category even if those event sequences provide the highest overall reward.

Further, as shown inFIG. 5A, the POI preference element536includes an exclusion option538, a preference select option540, an inclusion option542, and a slide bar544. In one or more embodiments, the POI preference element536operates in the same manner as the POI category preference element526. In particular, the exclusion option538indicates that the corresponding point of interest preference is to be ignored, the preference select option540allows the user to position a marker545along the slide bar544to indicate the preference, and the inclusion option542can maximize the preference weight applied to the parameter of the weighting function that corresponds to the particular user preference (and/or operate as an additional constraint).

The historical event sequences view506provides users with context regarding the recommended event sequences and the modified recommended event sequences. As illustrated inFIG. 5A, the historical event sequences view506displays a visual representation of historical event sequences from the plurality of historical event sequences upon which the recommendations are based. In one or more embodiments, the historical event sequences view506displays all of the plurality of historical event sequences. In some embodiments, the historical event sequences view506displays only a subset of the plurality of historical event sequences.

For example, the plurality of historical event sequences may be too numerous to display within a single frame within an understandable and/or visually appealing manner. Accordingly, the event sequences view506can display a number of historical event sequences sufficient to provide the user with useful context. In one or more embodiments, the event sequence recommender system106selects the subset to display by selecting a representative sample of the plurality of historical event sequences. More detail will be provided regarding display of historical event sequences below with reference toFIG. 5B.

In one or more embodiments, the historical event sequences view506further includes tools (not shown) enabling a user to interact with the display of historical event sequences. For example, the historical event sequences view506can include a zoom tool allowing a user to zoom into or out of the displayed historical event sequences. Additionally, the historical event sequences view506can include a dragging tool allowing a user to drag or scroll through the current display of historical event sequences to view historical event sequences or portions of historical event sequences that are included within the historical event sequences view506but not currently displayed within the frame. Further, the historical event sequences view506can include a highlighting tool, allowing a user to highlight one or more of the displayed historical event sequences or portions of historical event sequences.

In one or more embodiments, the event sequence recommender system106changes the user interface500depending on the user preferences input by the user. For example, in one or more embodiments, when the events sequence recommender system106initially displays the user interface500(e.g., before the user has input any user preferences), the recommendation view502displays the map display508, the general recommendation510, and the most popular sequence512without displaying the personalized recommendation514. After the user inputs one or more user preferences using the user preferences504view, the recommendation view502changes to include the personalized recommendation514. In some embodiments, the map display508itself changes to display the modified recommended sequence represented by the personalized recommendation514. As the user iteratively submits modified user preferences, the event sequence recommender system106updates the modified recommended event sequence, and the personalized recommendation514(and the map display508) iteratively changes to provide the updated modified recommended event sequence.

Additionally, in one or more embodiments, the event sequence recommender system106changes the historical event sequences view506to reflect user preferences. For example, in one or more embodiments, the historical event sequences view506displays a visual representation of the plurality of historical event sequences or a subset of the plurality of historical event sequences (e.g., a subset containing a representative sample) before the user has submitted any user preferences. After the user inputs one or more user preferences, the event sequence recommender system106changes the visual representation provided by the historical event sequences view506to display a subset of the plurality of historical event sequences corresponding to the user preferences. For example, if a user submits a user constraint requiring that the modified recommended event sequence has no more than three points of interest, the event sequence recommender system106changes the visual representation to display the subset of historical event sequences that include no more than three points of interest.

It should be noted that in one or more embodiments, because the recommendation model has already been trained, it still generates the modified recommended event sequences based on all of the plurality of historical event sequences (as well as the submitted user preferences); however, its display of the subset corresponding to the submitted preferences provides the user with the most relevant historical event sequences. As the user iteratively submits modified user preferences, the visual representation iteratively changes to display subsets of the plurality of historical events sequences corresponding to the user preferences.

FIG. 5Billustrates an enlarged view of a portion of the visual representation of historical event sequences provided by the historical event sequences view506ofFIG. 5A. As shown inFIG. 5B, the visual representation includes a plurality of rows550and a plurality of columns552. In particular, each row from the plurality of rows550represents a historical event sequence. Each column from the plurality of columns552represents an event (e.g., a point of interest) within the historical event sequence (or, more specifically, the event with which the user was engaged during a particular time step). In one or more embodiments, each column reflects a predetermined length of the time; however, in some embodiments, the length of the time step is adjustable by the user.

Further, as shown inFIG. 5B, each rectangle within a particular historical event sequence is associated with a color. In one or more embodiments, the color represents the category of the point of interest corresponding to the rectangle. For example, a rectangle having a first color can represent that the corresponding point of interest was a public park. In some embodiments, the color represents a specific point of interest (e.g., the first color represents that the corresponding point of interest was Central Park). Considering the color and number of columns spanned, the rectangles of a historical event sequence indicate (approximately) where the previous user visited, in what sequence, and/or how long the previous user visited each point of interest. For example, the row554includes two rectangles with each rectangle spanning one column. Therefore, the previous user corresponding to the historical event sequence represented by the row554visited two points of interest and remained at each point of interest for approximately one time step. As an additional example, the row556includes three rectangles with each rectangle spanning one column. Therefore, the previous user corresponding to the historical event sequence represented by the row556visited three points of interest and remained at each point of interest for approximately one time step.

In one or more embodiments, the event sequence recommender system106organizes the plurality of rows550by grouping similar historical event sequences together. For example, as shown inFIG. 5B, the event sequence recommender system106can group all historical event sequences that start at a first point of interest, allowing a user to see how the previous users that started at that point of interest progressed through a sequence of events (e.g., which subsequent points of interest the previous users visited, how long the previous users remained at each point of interest, etc.). To illustrate, the historical event sequences represented by the plurality of rows550can include historical event sequences where the corresponding previous users first visited the Empire State Building, enabling a user to see where the previous users visited after visiting the Empire State Building. The event sequence recommender system106can additionally organize each group of historical event sequences into subgroups so that each historical event sequence in the subgroup includes the same second point of interest (and further, the same third point of interest, the same fourth point of interest, etc.)

Providing the user interface500ofFIG. 5Aand modifying the different views included therein, the event sequence recommender system106more efficiently engages users by providing more informative and interactive experiences. As discussed above, researchers conducted a study to determine effectiveness of one or more embodiments of the event sequence recommender system106.FIGS. 6A-8illustrate graphs reflecting results of the study. In particular, the graphs ofFIGS. 6A-8compare the performance, as provided by the user feedback, of the event sequence reminder system (e.g., implementations of the user interface500, referred to as “ESRM” and a more simplified, conventional “black-box” user interface (referred to as “conventional”).

The user feedback was gathered through a study, which asked questions to twelve participants who were visiting Melbourne, Australia for the first time. All of the participants were male university students, ten of whom were between the ages of 25 and 34, one of whom was between the ages of 18 and 24, and one of whom was between the ages of 35 and 44. None of the participants had previous experience engaging with the event sequence recommender system106. The study was conducted on a lap-top computer.

The historical event sequences used to build the recommendation model of the event sequence recommender system106extracted from the YFCC100M dataset disclosed in B. Thomee, D. A. Shamma, G. Friedland, B. Elizalde, K. Ni, D. Poland, D. Borth, and L. Li, 2016. YFCC100M: The new data in multimedia research. Commun. ACM 59, 2 (2016), 64-73. In particular, the YFCC100M dataset contains photos and videos, including meta information such as the time and location of the media. The location sequences were extracted from the dataset, which was then narrowed to the ten most popular points of interest. After preprocessing and removing loops, the resulting dataset included 1,399 historical event sequences and ten points of interest. Each historical event sequence included an average of five points of interest.

Each participant session lasted about sixty minutes, which included five minutes for general training and the study task overview, which provided instruction to the participant. Each participant was shown a brief tutorial covering the components and operations of each interface. Each participant then planned a trip using each interface and, when satisfied with the recommendation, clicked a “finish” button. Subsequently, each participant then completed a questionnaire using a seven-point Likert scale.FIGS. 6A-6Cillustrate graphs providing the feedback collected from the questionnaires.FIGS. 7-8illustrate graphs providing passive feedback (i.e., completion time and number of refinements, respectively). In particular, the study used the Wilcoxon test to compare questionnaire ratings and T-tests for the passive feedback, with a significance level of 0.01

FIG. 6Aprovides a graph reflecting feedback to a question asking, “Do you agree that the interface informed you about how the recommendations were made?” As seen from the graph, the ESRS interface (receiving an average score of 6.17) was rated as more informative than the conventional user interface (receiving an average score of 1.58).FIG. 6Bprovides a graph reflecting feedback to a question asking, “How confident are you that you will follow the recommended plan in your trip?” The graph shows that the participants were generally more confident that they would follow the recommendation provided by the ESRS user interface (receiving an average score of 5.42) than they were for the recommendation provided by the conventional user interface (receiving an average score of 3.75). The graph ofFIG. 6Creflects feedback to the question asking, “How confident are you that the recommended plan will provide a good experience?” The graph shows that the participants were generally more confident that the recommendation provided by the ESRS user interface (receiving an average score of 5.50) would provide a good experience than they were for recommendation provided by the conventional user interface (receiving an average score of 3.50).

FIG. 7illustrates a graph providing the average engagement time the participants took in planning their trip using each user interface. In particular, the graph shows that the participants spent an average time of 2.80 minutes (with a standard deviation of 1.61 minutes) engaging with the conventional user interface to plan their trip. By contrast, the graph shows that the participants spent an average of 10.38 minutes (with a standard deviation of 3.86 minutes) engaging with the ESRS user interface. As discussed above, this increased engagement increases the confidence and likelihood that users will utilize recommendations from the event sequence recommender system106.

FIG. 8illustrates the average number of refinements the participants made while using each user interface. The graph shows that the participants made no results refinements when using the conventional user interface. This corresponds to the lack of user interaction provided by the conventional user interface. By contrast, the graph further shows that the participants made an average of 20 result refinements (with a standard deviation of 9.22 refinements) when using the ESRS user interface, showing that the participants found that the ability provided by the event sequence recommender system106to submit user preferences was beneficial, leading to increased engagement, confidence, and accuracy of results.

Although many of the foregoing examples discuss event sequences in relation to physical points of interest, the event sequence recommender system106can also generate other recommended event sequences. As mentioned above, in one or more embodiments, the event sequence recommender system106recommends a sequence of digital content transmissions across client devices. For example, an event can include the occurrence of a transmission of digital content to one or more client devices and the event sequence recommender system106can utilize a trained recommendation model to generate a recommended sequence of digital content transmissions. In particular, the event sequence recommender system106can train the recommendation model based on a plurality of historical event sequences where each historical event sequence includes a sequence of digital content transmissions carried out by one or more previous users.

Subsequently, the event sequence recommender system106can provide, for display via a client device, a user interface that displays the recommended sequence of digital content transmissions, the plurality of historical digital content transmissions, and a plurality of interactive elements for entry of user preferences. In one or more embodiments, a user preference can include a distribution channel through which to transmit the digital content (e.g., email, multimedia messaging, social media post, etc.), a preferred digital content category (e.g., video advertisement, digital image, informative literature etc.), or a preferred digital content item to transmit (e.g., a particular advertisement or piece of informative literature). In some embodiments, the plurality of interactive elements can include interactive elements for entry of user constraints. In particular, a user constraint can include a constraint on the number of digital content transmissions (or a period of time over which digital content transmissions will occur).

The event sequence recommender system106can modify the reward function of the recommendation model based on the input of the user preferences. As mentioned above, the event sequence recommender system106can utilize the reward function to determine the expected value of a recommendation. In particular, the reward function can indicate the likelihood of a recommendation producing a reward. For example, in embodiments in which an event includes the occurrence of a digital content transmission, a reward can include an action taken by a recipient of the digital content transmission (e.g., a conversion, a click, or a view) or the amount of revenue obtained as a result of the digital content transmission.

After modifying the reward function, the event sequence recommender system106can generate a modified recommended event sequence of digital content transmissions based on the recommendation model and the modified reward function. Subsequently, the event sequence recommender system106can provide the modified recommended sequence for display via the client device along with the plurality of interactive elements for further entry of user preferences. Further, upon generating the modified recommended event sequence of digital content transmissions, the event sequence recommender system106can modify the visual representation of historical event sequences to display those historical event sequences corresponding to the selected user preferences.

Furthermore, the event sequence recommender system106can also execute digital content transmissions (e.g., execute a digital content campaign) in accordance with a recommended event sequence. For example, upon providing a recommended event sequence for display, the event sequence recommender system106can distribute digital content across a computer network to one or more client devices as outlined in the recommended event sequence.

Turning now toFIG. 9, additional detail will be provided regarding various components and capabilities of the event sequence recommender system106. In particular,FIG. 9illustrates the event sequence recommender system106implemented by the computing device902(e.g., the server(s)802and/or the client devices810a-810nas discussed above with reference toFIG. 8). Additionally, the event sequence recommender system106is also part of the analytics system804. As shown, the event sequence recommender system106can include, but is not limited to, a recommendation model training engine904, a recommendation model application manager906, a user preference manager908, a user interface manager910, and data storage912(which includes recommendation model914and historical event sequences916).

As just mentioned, and as illustrated inFIG. 9, the event sequence recommender system106includes the recommendation model training engine904. In particular, the recommendation model training engine904trains a recommendation model to generate recommended event sequences based on a plurality of historical event sequences. For example, the recommendation model training engine904utilizes the plurality of historical event sequences to build a probabilistic suffix tree, parameterize the probabilistic suffix tree into several action models, and construct Markov Decision Process models based on those action models. Subsequently, the recommendation model training engine904trains the recommendation to employ Thompson Sampling in conjunction with the Markov Decision Process models to generate recommendations.

As shown inFIG. 9, the event sequence recommender system106also includes the recommendation model application manager906. In particular, the recommendation model application manager906uses the recommendation model trained by the recommendation model training engine904. For example, the recommendation model application manager906generates a recommended event sequence from the trained recommendation model based on the plurality of historical event sequences. Further, when the user preferences have been submitted, the recommendation model application manager906modifies a reward function of the recommendation model and then uses the recommendation model with the modified reward function to generate a modified recommended event sequence.

Additionally, as shown inFIG. 9, the event sequence recommender system106includes the user preferences manager908. In particular, the user preferences manager908detects the input of one or more user preferences (including user constraints) and passes the submitted user preferences to the recommendation model application manager906. In one or more embodiments, the user preferences manager908passes the numerical values associated with the submitted user preferences directly to the recommendation model application manager906. In some embodiments, the user preferences manager user preferences manager908determines a preference weight from the numerical value and sends the preference weight to the recommendation model application manager906.

Further, as shown inFIG. 9, the event sequence recommender system106includes the user interface manager910. In particular, the user interface manager910provides, for display via a client device, a user interface (e.g., the user interface500ofFIG. 5A) that displays various interface elements. For example, the user interface manager910can provide a recommended event sequence, a modified recommended event sequence, a plurality of interactive elements for entry of user preferences, and a visual representation of historical event sequences. Further, the user interface manager910can modify the user interface as the inputs user preferences. For example, after receiving one or more user preferences, the user interface manager910can modify the visual representation of historical event sequences to display a subset of the plurality of historical event sequences that corresponds to the received user preferences.

Additionally, as shown inFIG. 9, the event sequence recommender system106includes data storage912. In particular, data storage912includes recommendation model914and historical event sequences916. Recommendation model914stores the recommendation model trained by the recommendation model training engine904and applied by the recommendation model application manager906to generate recommended event sequences and/or modified recommended event sequences. Historical event sequences916stores the plurality of historical event sequences (i.e., the event sequences with which prior users have engaged). The recommendation model training engine904can obtain the plurality of historical event sequences from historical event sequences916when training the recommendation model.

The components904-916and their corresponding elements can comprise software, hardware, or both. For example, the components904-916and their corresponding elements can comprise one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices. The components904-916and their corresponding elements can comprise hardware, such as a special purpose processing device to perform a certain function or group of functions. Additionally, or alternatively, the components904-916and their corresponding elements can comprise a combination of computer-executable instructions and hardware.

Furthermore, the components904-916of the event sequence recommender system106may, for example, be implemented as one or more stand-alone applications, as one or more modules of an application, as one or more plug-ins, as one or more library functions or functions that may be called by other applications, and/or as a cloud-computing model. Thus, the components904-916of the event sequence recommender system106may be implemented as a stand-alone application, such as a desktop or mobile application. Furthermore, the components904-916of the event sequence recommender system106may be implemented as one or more web-based applications hosted on a remote server. Alternatively, or additionally, the components904-916of the event sequence recommender system106may be implemented in a suite of mobile device applications or “apps.” For example, in one or more embodiments, the event sequence recommender system106can comprise or operate in connection with digital software applications such as ADOBE® ANALYTICS CLOUD® or ADOBE® MARKETING CLOUD®. “ADOBE,” “ANALYTICS CLOUD,” and “MARKETING CLOUD” are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries.

Turning now toFIG. 10, this figure illustrates a series of acts1000performed by the event sequence recommender system106to modify a recommended event sequence based on user preferences. WhileFIG. 10illustrates acts according to one embodiment, alternative embodiments may omit, add to, reorder, and/or modify any of the acts shown inFIG. 10. The acts ofFIG. 10can be performed as part of a method. In one or more embodiments, a non-transitory computer readable medium can comprise instructions that, when executed by one or more processors, cause a computing device to perform the acts ofFIG. 10. In still further embodiments, a system can perform the acts ofFIG. 10.

The series of acts1000includes an act1002of generating a recommended event sequence using a recommendation model comprising a reward function. For example, the act1002involves generating a recommended event sequence by utilizing a recommendation model trained based on a plurality of historical event sequences, wherein the recommendation model comprises a reward function. In one or more embodiments, utilizing the recommendation model includes iteratively utilizing a Markov Decision Process model in conjunction with Thompson sampling to generate a sequence of recommended events based on the plurality of historical event sequences. In some embodiments, the recommended event sequence includes a sequence of points of interest. In further embodiments, the recommended event sequence includes a sequence of digital content transmissions across client devices.

The series of acts1000also includes an act1004of providing, for display via a client device, the recommended event sequence. For example, the act1004involves providing, for display via a client device, a user interface comprising the recommended event sequence, a visual representation of historical event sequences, and a plurality of interactive elements for entry of user preferences. In one or more embodiments, the visual representation of historical event sequences includes a subset of the plurality of historical event sequences. In some embodiments, the visual representation of historical event sequences includes a plurality of rows—wherein each row represents a historical event sequence of the plurality of historical event sequences taken by a user—and a plurality of a columns—wherein each column represents an event within the historical event sequence.

In one or more embodiments (e.g., where the recommended event sequence includes a sequence of points of interest), a user preference of the user preferences comprises one of a preferred point of interest category or a preferred point of interest. In some embodiments (e.g., where the recommended event sequence includes a sequence of digital content transmissions across client devices), a user preference of the user preferences comprises one of a distribution channel, a digital content category, or a digital content item for transmission.

In some embodiments, the plurality of interactive elements further includes a set of interactive elements for entry of user constraints. In one or more embodiments (e.g., where the recommended event sequence includes a sequence of points of interest), a user constraint of the user constraints comprises one of a maximum time, a maximum distance, or a maximum number of points of interest to visit. In some embodiments (e.g., where the recommended event sequence includes a sequence of digital content transmissions across client devices), a user constraint of the user constraints comprises a maximum number of digital content transmissions.

The series of acts1000further includes an act1006of detecting input of user preferences. For example, the act1006involves detecting input of user preferences via the plurality of interactive elements. In one or more embodiments, detecting input of user preferences includes generating preference weights based on the detected input of user preferences via the plurality of interactive elements.

The series of acts1000also includes an act1008of modifying the reward function based on the user input. For example, the act1008involves modifying the reward function of the recommendation model based on the input of the user preferences. In one or more embodiments, modifying the reward function includes modifying the reward function of the recommendation model by applying the preference weights to the reward function.

Additionally, the series of acts1000includes an act1010of generating a modified recommended event sequence. For example, the act1010involves generating a modified recommended event sequence based on the recommendation model and the modified reward function. In one or more embodiments, the act1010further involves generating the modified recommended event sequence further based on input of one or more user constraints via the set of interactive elements

Further, the series of acts1000includes an act1012of providing the modified recommended event sequence for display via the client device. For example, the act1012involves providing the modified recommended event sequence for display via the client device with the plurality of interactive elements for additional entry of user preferences. In one or more embodiments, the event sequence recommender system106provides the modified recommended event sequence for display by providing a first visual indicator—representing historical user responses corresponding to an event of the modified recommended event sequence—and providing a second visual indicator—representing a type of the event. In one or more embodiments, the event sequence recommender system106provides the recommended event sequence for display with the modified recommended event sequence via the client device. In other words, the event sequence recommender system106provides the unmodified recommendation and the modified recommendation for simultaneous display.

In one or more embodiments, the series of acts1000further includes modifying the visual representation of historical event sequences to display a subset of the plurality of historical event sequences corresponding to the user preferences (not shown). In some embodiments, the series of acts1000further includes iteratively generating modified recommended event sequences by modifying the reward function of the recommendation model based on detecting user preferences via the plurality of interactive elements (not shown). In particular, in response to detecting additional entry of user preferences (e.g., a user submits the user preferences in successive iterations), the event sequence recommender system106can further modify the reward function and generate a new modified recommended event sequence. Consequently, the series of acts1000can further include iteratively modifying the visual representation of historical event sequences to display subsets of the plurality of historical event sequences corresponding to the user preferences.

FIG. 11illustrates a block diagram of exemplary computing device1100that may be configured to perform one or more of the processes described above. One will appreciate that server(s)802and/or client devices810a-810nmay comprise one or more computing devices such as computing device1100. As shown byFIG. 11, computing device1100can comprise processor1102, memory1104, storage device1106, I/O interface1108, and communication interface1110, which may be communicatively coupled by way of communication infrastructure1112. While an exemplary computing device1100is shown inFIG. 11, the components illustrated inFIG. 11are not intended to be limiting. Additional or alternative components may be used in other embodiments. Furthermore, in certain embodiments, computing device1100can include fewer components than those shown inFIG. 11. Components of computing device1100shown inFIG. 11will now be described in additional detail.

In particular embodiments, processor1102includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor1102may retrieve (or fetch) the instructions from an internal register, an internal cache, memory1104, or storage device1106and decode and execute them. In particular embodiments, processor1102may include one or more internal caches for data, instructions, or addresses. As an example, and not by way of limitation, processor1102may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory1104or storage device1106.

Memory1104may be used for storing data, metadata, and programs for execution by the processor(s). Memory1104may include one or more of volatile and non-volatile memories, such as Random Access Memory (“RAM”), Read Only Memory (“ROM”), a solid state disk (“SSD”), Flash, Phase Change Memory (“PCM”), or other types of data storage. Memory1104may be internal or distributed memory.

Storage device1106includes storage for storing data or instructions. As an example, and not by way of limitation, storage device1106can comprise a non-transitory storage medium described above. Storage device1106may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage device1106may include removable or non-removable (or fixed) media, where appropriate. Storage device1106may be internal or external to computing device1100. In particular embodiments, storage device1106is non-volatile, solid-state memory. In other embodiments, Storage device1106includes read-only memory (ROM). Where appropriate, this ROM may be mask programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these.

Communication interface1110can include hardware, software, or both. In any event, communication interface1110can provide one or more interfaces for communication (such as, for example, packet-based communication) between computing device1100and one or more other computing devices or networks. As an example, and not by way of limitation, communication interface1110may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI.