Facilitating changes to online computing environment by assessing impacts of actions using a knowledge base representation

A method includes accessing a subject entity and a subject relation of a focal platform and accessing a knowledge graph representative of control performance data. Further, the method includes computing a set of ranked target entities that cause the subject entity based on the subject relation or are an effect of the subject entity based on the subject relation. Computing the set of ranked target entities is performed using relational hops from the subject entity within the knowledge graph performed using the subject relation and reward functions. The method also includes transmitting the set of ranked target entities to the focal platform. The set of ranked target entities is usable for modifying a user interface of an interactive computing environment provided by the focal platform.

TECHNICAL FIELD

This disclosure relates generally to facilitating modifications to interactive computing environments based on an evaluation of factors that impact performance of the interactive computing environments. More specifically, but not by way of limitation, this disclosure relates to evaluating impacts of actions in an interactive computing environment with an assessment model that assesses how actions impact the interactive computing environment, and, in some cases, performing modifications to the evaluated interactive computing environment.

BACKGROUND

Interactive computing environments, such as web-based applications or other online software platforms, allow users to perform various computer-implemented functions through graphical interfaces. For instance, an interactive computing environment can provide functionalities such as allowing users to complete transactions in the computing environment, or post content, such as text, images, or videos, to graphical interfaces provided by the computing environment. The performance of the interactive computing environment can thus be measured by performance metrics such as the number of transactions completed using the computing environment or the number of items added to the graphical interfaces.

An intervention to the interactive computing environment, such as a change to the content of the graphical interfaces in the interactive computing environment, can impact users' behaviors in interacting with the computing environment and thus impact the performance of the computing environment. For example, new content added to the graphical interfaces might increase the number of transactions completed in the computing environment, whereas removing some important content from the computing environment might reduce the number of the transactions completed in the computing environment.

Understanding the impact of the interventions can help to determine the proper actions that can be taken to modify the interactive computing environment so that the performance of the interactive computing environment is improved. However, determining the impact of the intervention is often difficult for any person or system without significant experience in a particular field. Existing methods are insufficient to reliably evaluate effects of interventions due to reliance on a knowledge base developed and grown experientially.

SUMMARY

Certain embodiments involve evaluating an interactive computing environment with an assessment model that assesses how various causes and effects impact the interactive computing environment, and, in some cases, performing modifications to the evaluated interactive computing environment. For example, a method includes accessing a subject entity and a subject relation of a focal platform and accessing a knowledge graph representative of control performance data. Further, the method includes computing a set of ranked target entities that cause the subject entity based on the subject relation or are an effect of the subject entity based on the subject relation. Computing the set of ranked target entities is performed using relational hops from the subject entity within the knowledge graph performed using the subject relation and reward functions. The method also includes transmitting the set of ranked target entities to the focal platform. The set of ranked target entities is usable for modifying a user interface of an interactive computing environment provided by the focal platform.

DETAILED DESCRIPTION

Certain embodiments involve evaluating an interactive computing environment with an assessment model that assesses how various causes and effects impact the interactive computing environment, and, in some cases, performing modifications to the evaluated interactive computing environment. For instance, a performance evaluation system accesses focal performance data of a focal online platform, which tracks performance data of the evaluated interactive computing environment, and a knowledge graph representative of control performance data. The performance evaluation system determines, based on the performance data and the knowledge graph, one or more resulting target causes or effects (i.e., target entities) that either lead to or result from the performance data (i.e., subject entities) of the focal online platform. Further, the performance evaluation system also determines one or more paths associated with the resulting target causes or effects. A path can indicate relations and interim entities that also affect the performance data. The performance evaluation system transmits the target causes or effects and the paths to the focal platform for use in modifying how the focal platform provides an interactive computing environment to user devices (e.g., modifying a functionality or layout of a graphical interface of the interactive computing environment).

The following non-limiting example is provided to introduce certain embodiments. In this example, a performance evaluation system is included in or in communication with a focal online platform, which provides an interactive computing environment that is being evaluated. The performance evaluation system executes one or more algorithms that allow for enhancing the performance of the focal online platform based on the impact of the user interactions with the interactive computing environment. To accomplish this enhancement, the performance evaluation system accesses performance data from the focal online platform, where the performance data measures performance metrics of the focal online platform (e.g., a number of transactions performed on the focal online platform, a click-through rate on the focal online platform, total visits by the user to the focal online platform, etc.). The performance evaluation system records various features associated with the focal online platform (e.g., the number of searches performed on the focal online platform, user click-through history, etc.).

Continuing with this example, the performance evaluation system services queries involving subject entities using a knowledge graph. The subject entities include an effect indicated by the performance data of the focal online platform. For instance, the performance evaluation system receives a query identifying a subject entity (e.g., an effect indicated by the performance data of the focal online platform) and a subject relationship (e.g., a question regarding a cause of the subject entity). To service the query, the performance evaluation system selects a knowledge graph having control performance data, which provides qualitative insights that can be based on quantitative information, where interlinked paths between the causes and effects are detailed in the control performance data. This control performance data is extracted from resources (e.g., websites, textbooks, or other data sources) providing indications of known, but incomplete, data change causes and effects. The interlinked paths in the knowledge graph are used to infer a target entity, which represents a cause or effect of the queried subject entity.

For instance, upon receiving a query including a subject entity (e.g., “product_positioning”) and a subject relationship (e.g., “canAchieve”), the knowledge graph uses the subject entity and the subject relationship to generate a target entity of “more_payment” and a target entity path to “more_payment” of “hasprerequisite/importanto/capableof” (i.e., the target entity is reachable through a three-hop path within the knowledge graph of (i) hasprerequisite, (ii) importanto, and (iii) capableof). That is, beginning with the subject entity of “product_positioning,” the knowledge graph is trained to answer the subject relationship of “canAchieve.” The subject entity is represented in the knowledge graph as a node, and a path is taken in the knowledge graph from the subject entity node along a relational link of “hasprerequisite” to an interim node. The path continues along relational links of “importantto” and “capableof” until the target entity of “more_payment” is reached in the knowledge graph. Further, a reward policy may influence the selection of knowledge graph paths from the subject entity to the target entity by maximizing reward potential at every hop between interim entities along the target entity path. Other target entities are generated in a similar manner using the knowledge graph, and the set of target entities and the target entity paths are scored using the reward policy such as providing a higher score if an end entity of a path is a correct target entity, when a correct sequence of relations are followed to achieve a target entity, or a combination thereof.

The performance evaluation system can provide the ranked target entities and target entity paths to the focal online platform, which allows for modifying a presentation of the interactive computing environment based on predicted actions of the subject entity in an interactive computing environment, estimated causes of actions by the subject entity in the interacting computing environment, or both. In one example, the content or layout of a user interface of the focal online platform can be modified to change a performance metric associated with predicted actions or causes. For example, the content or layout of a user interface may be changed to enhance product positioning because the target entity indicated that enhanced product positioning can lead to increases in payments (e.g., by increasing a number of transactions completed in the focal online platform). In another example, where the performance evaluation system receives a subject entity (e.g., low sales) with an antecedent looking relationship (e.g., caused by), the focal online platform can be adjusted to address the target entity or entities that resulted in the low sales subject entity.

As described herein, certain embodiments provide improvements to interactive computing environments by solving problems that are specific to online platforms. These improvements include more effectively configuring the functionality of an interactive computing environment based on accurate and precise evaluations of online platform performance that can impact the ability of user devices to interact with the online platforms in a desired manner. Evaluating the impact of such online platform performance within an interactive computing environment is uniquely difficult because online platform performance could significantly impact a user's online experience but would be difficult to assess in real time or through surveys upon completion of the user's online experience.

Because these subject-entity evaluation problems are specific to computing environments, embodiments described herein utilize automated models that are uniquely suited for assessing computing environments. For instance, a computing system automatically applies various rules of a particular type (e.g., various relationships between the knowledge graph and the focal online platform captured in one or more models) to clickstream data, transaction data, or other interaction data and thereby computes objective measurements of online experiences, sometimes in a real-time manner. The measurements are objective at least because a computing system applies these rules in an automated manner, rather than relying on subject memories and judgments. The objective measurements are usable for enhancing a computing environment by, for example, modifying interface elements or other interactive aspects of the environment. Accurate and precise evaluations of the impact of performance of an online platform can enable effective configuration of the interactive computing environment, thereby reducing future negative impacts on the online platform. Consequently, certain embodiments more effectively facilitate modifications to a computing environment that facilitates desired functionality, as compared to existing systems.

As used herein, the term “online platform” is used to refer to a software program, which when executed, provides an interactive computing environment having various interface elements with which user devices interact to achieve various goals. In this disclosure, the term “online platform” may also be used to refer to the interactive computing environment that the online platform provides. For example, clicking or otherwise interacting with one or more interface elements during a session causes the online platform to manipulate electronic content, perform transactions, or otherwise interact with electronic content that is accessible via the online platform.

As used herein, the term “performance metric” is used to refer to a metric used to quantitatively measure the performance of a focal online platform. Examples of performance metrics include, but are not limited to, a number of user visits to the focal online platform, a click-through rate for an interface element of a graphical interface, a number of transactions completed on the focal online platform, and so on.

As used herein, the term “performance data” is used to refer to data generated or recorded by an online platform that describes the performance of the online platform and the associated features. An example of performance data includes the performance metric measuring the number of transactions completed on the online platform, and data associated with the transactions, such as the number of visits to the online platform, the number of queries submitted to the online platform, the content shown in the user interface of the online platform that are related to the transactions, and so on.

As used herein, the term “focal online platform” is used to refer to an online platform that is analyzed and for which an online platform performance is determined.

As used herein, the term “intervention” is used to refer to a change to one or more aspects of the focal online platform, such as the appearance, the functionality, and/or the availability of the platform. For example, the intervention includes one or more changes to the content, layout, settings, or other aspects of a user interface provided by the focal online platform.

As used herein, the term “control performance data” is used to refer to a corpus of data that analyzes similar functionalities as the focal online platform but is not subject to the intervention. For example, the control performance data includes known entities and their relations with other entities.

As used herein, the term “knowledge graph” is used to refer to an interlinking graph of entities and relations generated from the control performance data. For example, the knowledge graph links entities and relationships provided from a data corpus of the control performance data.

As used herein, the term “assessment model” is used to refer to a model, or the program code that implements the model, for assessing performance data of a focal online platform subject to an intervention based on the relationship between the focal online platform and the control performance data. The assessment model also provides a diagnosis for poor performance (e.g., what causes the poor performance) of the focal online platform and provides an ability to suggest interventions that improve performance of the focal online platform.

Referring now to the drawings,FIG.1is an example of a computing environment100in which a performance evaluation system102can be used to analyze performance of a focal online platform114and, in some cases, facilitate modifications to the focal online platform114. In various embodiments, the computing environment100includes the performance evaluation system102, a focal host system112hosting the focal online platform114, and control host systems122hosting control performance data124.

Multiple user devices118A-118C interact with the focal online platform114, such as through a user interface116over a network108. The network108may be a local-area network (“LAN”), a wide-area network (“WAN”), the Internet, or any other networking topology capable of connecting the user devices118to the focal host system112. The interactions can include users clicking on interface elements in a user interface116of the focal online platform114, users initiating transactions through the user interface116, such as a purchase transaction or a registration transaction, or users creating contents on the user interface116, such as posting a comment, an image, a blog, or any other content.

The interactions between the users or their associated user devices118and the focal online platform114are saved as focal performance data134. The focal performance data134includes a performance metric measuring the performance of the focal online platform114, such as the number of clicks on an interface element for a given period of time (e.g., in a minute, an hour, or a day), the number of transactions conducted within a given period of time, the number of new content items created by the users on the focal online platform114, or other kinds of metrics. Other features associated with the focal online platform114during the interactions, such as the content displayed in the user interface116, the number of user visits to the focal online platform114, and the number of queries submitted to the focal online platform114when the transactions are conducted, can also be included in the focal performance data134. While the focal performance data134is described herein resulting from performance of the focal online platform114with which a user is able to interact, the focal performance data134may also be any number of offline systems. For example, the focal performance data134can include performance data from a print or video marketing campaign, performance data associated with employee efficiency, or any other trackable data.

An intervention132might change the user interface116of the focal online platform114to alter interactions between the user devices118and the focal online platform114. In an example, the intervention132is a proactive intervention that is introduced intentionally by an administrator or other individuals associated with the focal online platform114, such as a change to the layout of the user interface116, for example, to launch a campaign on the focal online platform114. Changing the user interface layout might enhance performance of the focal online platform114, thereby enhancing the focal performance data134. The intervention132may also include changing the content of the user interface116, such as adding or changing the content to provide an incentive for users to complete a transaction. For example, a displayed price associated with an item on the user interface116might be reduced. Offerings such as coupons, temporary price change, extra benefits, or special informational emails might be provided on or through the user interface116. A focal online platform hosting a media site might change its user interface to offer a subscription at a lower price than usual or to provide access to its archive. In an example, these types of interventions132lead to an increase in the number of transactions completed through the focal online platform114.

To determine the intervention132to apply to the focal online platform114, the performance evaluation system102is employed to analyze the focal performance data134before application of the intervention132on the focal online platform114. In addition, the performance evaluation system102examines the performance data hosted by one or more control host systems122. The control performance data124is selected to include information with similar functionalities as the focal online platform114but not affected by the intervention132. For example, the control performance data124may be data that is mined or otherwise extracted from marketing textbooks. The data extracted from the marketing textbooks may provide information about how different marketing entities are related. For example, an entity of an “increase in cost” has a relationship of “causes” with an additional entity of a “decrease in sales.” The control performance data124may be generated from a number of sources such as textbooks, websites, or any other resource providing indications of relationships between entities. Because the control performance data124is not subject to the interventions132, the data provided by the control performance data124can be used to adjust the interventions132of the focal host system112to provide enhancement to the functionality of the focal online platform114. Further, in an example, the performance or other data obtained from the focal online platform114is used over time to inform the control performance data124. For example, data in text books or other knowledge sources can be updated or supplemented by the performance or other data obtain from the focal online platform114.

To perform the analysis, the focal performance data134is transmitted to the performance evaluation system102through a network120and stored at a datastore138associated with the performance evaluation system102. The network120may be the same as, or different from, the network108and may also be a LAN, a WAN, the Internet, or any other networking topology that connects the focal host system112to the performance evaluation system102. Similarly, the entities and the relationships identified in the control performance data124may be compiled and saved as a knowledge graph136, which is transmitted over the network120and stored in the datastore138. The knowledge graph136may be compiled and obtained directly from the control host systems122or from a third party data service. In an example, the knowledge graph136provides indications of how various entities interlink with one another through the relationships identified in the control performance data124.

The performance evaluation system102employs a reinforcement learning agent110to build an assessment model128based on the relationship between the focal online platform114and the control performance data124. In the assessment model128, the reinforcement learning agent110provides reward functions104to paths and target entities of the knowledge graph136and models entity relationship paths from a subject entity140and a subject relation142originating from the focal performance data134of the focal online platform114. The performance evaluation system102provides the reward functions104to the assessment model128based on, for example, a determined terminal entity (e.g., a target entity130) of the knowledge graph136, a relation path to the determined terminal entity of the knowledge graph136, or a combination thereof. The performance evaluation system102, through the assessment model128, is able to provide a diagnosis for poor performance (e.g., what causes the poor performance) of the focal online platform and to provide suggested interventions that improve performance of the focal online platform114.

Based on the reward functions104, the subject entity140, and the subject relation142, the reinforcement learning agent110uses the assessment model128to generate a ranked set of target entities130for the focal online platform114. The ranked set of target entities130may result in identification of interventions132that are most likely to have an effect on the focal performance data134based on the subject entity140and the subject relation142provided to the performance evaluation system102.

The performance evaluation system102may transmit the ranked target entities130to the focal host system112. In some embodiments, doing so causes the host system112to modify the focal online platform114to improve its performance. The focal host system112receives the ranked target entities130and controls one or more interventions132based on the ranked target entities130to impact the focal online platform114. Based on the interventions132, the user interface116is modified to enhance a desired positive impact on the focal online platform114.

One or more computing devices are used to implement the performance evaluation system102and the focal host system112. For instance, the performance evaluation system102, the host system112, or both could include a single computing device, a group of servers or other computing devices arranged in a distributed computing architecture, etc.

The online platform114can be any suitable online service for interactions with the user devices118. Examples of an online platform include a content creation service, an electronic service for entering into transactions (e.g., searching for and purchasing products for sale, or registering for certain services), a query system, etc. In some embodiments, one or more host systems112are third-party systems that operate independently of the performance evaluation system102(e.g., being operated by different entities, accessible via different network domains, etc.). In additional or alternative embodiments, one or more host systems112include a performance evaluation system102as part of a common computing system. The user devices118may be any devices that are capable of accessing an online service. For non-limiting examples, the user devices118may be smartphones, smart wearables, laptop computers, desktop computers, or other types of user devices.

FIG.2depicts an example of a process200for generating a ranked answer to a query using the assessment model128. One or more computing devices (e.g., the computing environment100) implement operations depicted inFIG.2by executing suitable program code. For illustrative purposes, the process200is described with reference to certain examples depicted in the figures. Other implementations, however, are possible.

At block202, the process200involves accessing a query, which represents a portion of the focal performance data134. For instance, the query accessed at block202includes an indication of a subject entity and a relation. The subject entity can reflect either a performance metric of the focal online platform134(e.g., low_sales), or the subject entity can reflect a cause of a performance metric of the focal online platform134(e.g., product_positioning). The relation included in the query is a relation to the subject entity that leads to the target entity130. For example, a subject entity of “product_positioning” and a relation of “canAchieve” leads to one or more target entities identified by the assessment model128. In an example, the query is automatically generated by the performance evaluation system102when the performance evaluation system102detects performance data134outside of a desired performance envelope (e.g., when sales for a particular item are too low). In another example, the query is input manually into the performance evaluation system102when a user would like to improve functionality of the focal online platform114or other sales or marketing platform.

At block204, the process200involves performing the assessment model128. As discussed above with respect toFIG.1, the query accessed at block202is analyzed by the assessment model128using the knowledge graph136. The knowledge graph136represents a series of interlinked entities and relations. Using the knowledge graph136and the reward functions104, the assessment model128is able to generate and rank target entities130that provide an accurate answer the query accessed at block202.

At block206, the process200involves outputting ranked answers to the query from the knowledge graph136. In an example, the assessment model128, relying on the control performance data124of the knowledge graph136, analyzes the query and outputs all of the target entities130that correctly answer the query and are within a defined number of relational hops within the knowledge graph136from the subject entity140. For example, if the assessment model128is looking for answers within three relational hops from the subject entity140, all of the target entities130that answer the query within three relational hops (i.e., three (entity, relation, entity) linked data triplets within the knowledge graph136) may be ranked and output at block206. Additionally, in some examples, the ranked answers include paths that achieve the target entities. For example, when the subject entity of the query is “product_positioning” and the relation of the query is “canAchieve,” one of the ranked answers includes a target entity of “more_payment” and a three-hop path of “hasprerequisite/importantto/capableof.” The paths that achieve the target entities are provided as a reasoning for how the target entity is reached from the subject entity and the relation. During a reinforced learning operation, the paths are used to train the assessment model128to reach the correct target entities.

FIG.3is an example of a process300for facilitating the modification of a focal online platform based on computing a set of ranked target entities and target entity paths. One or more computing devices (e.g., the computing environment100) implement operations depicted inFIG.3by executing suitable program code. For illustrative purposes, the process300is described with reference to certain examples depicted in the figures. Other implementations, however, are possible.

At block302, the process300involves accessing the subject entity140and the subject relation142(i.e., the query collectively) of the focal online platform114. Interactions between user devices118and the focal online platform114generate the focal performance data134. In an example, the focal performance data134may be analyzed to generate the subject entity140that is representative of one or more elements indicated in the focal performance data134. The performance evaluation system102accesses the subject entity140from a suitable non-transitory computer-readable medium or other memory device, such as the datastore138associated with the performance evaluation system102. In some embodiments, the focal performance data134is stored on one or more non-transitory computer-readable media within host system112. The performance evaluation system102accesses the focal performance data134via suitable communications with a host system112. In one or more examples, the subject entity140and the subject relation142are manually provided by a user to the performance evaluation system102.

At block304, the process300involves accessing the knowledge graph136that represents the control performance data124. Similar to the focal performance data134, the performance evaluation system102may access the knowledge graph136from the control performance data124via suitable communications with the control host systems122. Alternatively, or additionally, the performance evaluation system102may access the knowledge graph136from the datastore138where the knowledge graph136is stored.

At block306, the process300involves computing a set of ranked target entities130. In an example, the target entity paths associated with the ranked target entities130are also computed at block306. In some embodiments, the assessment model128locates the target entities130within the knowledge graph136that answer the query within the desired number of relational hops. The target entities130and the associated target paths are ranked based on the reward functions104applied by the performance evaluation system102. In an example, the ranking indicates which of the set of target entities130impact the subject entity140the most when an intervention132directed at the target entity130is implemented at the focal online platform114. The target entities130are ranked based on a probability of reaching the specific target entities130from the subject entity140and the subject relation142.

At block308, the process300involves transmitting the set of ranked target entities130to the focal host system112so that modification can be made to the focal online platform114. In some embodiments, the ranked target entities130may be transmitted to a management system configured to manage and configure the focal online platform114. Based on the one or more of the ranked target entities130, the focal host system112or other systems control application of an intervention132associated with the ranked target entities130. Based on effects provided by the intervention132, the focal host system112or other systems associated with the focal online platform114is able to modify the focal online platform114to improve its performance.

For instance, the focal host system112includes one or more computing devices that can implement the intervention132to modify interface elements of the user interface116provided by the focal online platform114, such as by changing the content, layout, or settings of the user interface116. The focal online platform114may also be modified by the intervention132in any suitable manner including, but not limited to, the examples discussed above with respect toFIG.1.

FIG.4depicts an example of a process400for generating the knowledge graph136, according to certain embodiments of the present disclosure. The process400can be used to implement block204of the process200. One or more computing devices (e.g., the computing environment100) implement operations depicted inFIG.4by executing suitable program code (e.g., the performance evaluation system102). For illustrative purposes, the process400is described with reference to certain examples depicted in the figures. Other implementations, however, are possible.

At block402, the process400involves accessing the control performance data124. The control performance data124includes a knowledge base obtained from a data source, such as from textbooks, websites, or any other resource providing indications of relations between entities. The knowledge base may be generated from the data source in the format of a triplet. In an example, a triplet refers to two entities that are connected by a relation. In such an example, the triplet is stored in the knowledge base of the control performance data124in the form (e1, r, e2), where e1and e2are elements of a set of entities of the knowledge base and r is a relation element (i.e., a linking relation) of a set of relations of the knowledge base that relates the two elements.

As described with respect toFIG.1, the triplets pertain to marketing topics like product life cycle, product positioning, and market segmentation. These specific marketing topics are provided as examples, but are in no way limiting on what marketing topics can be associated with the triplets. In general, the triplets are extracted from a complex sentence within the data source of the control performance data124relevant to a marketing topic, and the triplets are distilled down to a semantically simpler format that maintains the meaning of the original complex sentence. As an example, the sentence, “Segmentation is a useful marketing decision when the consumer preferences in a geographical area or region are heterogeneous,” is distilled into the triplet (heterogeneous preference, CausesDesire, segmentation). Similar triplets may be distilled from a corpus of data provided by the control performance data124. In some examples, the relation component of the triplet may be selected from a set of available relationships that is most similar to the relationship described in the source sentence.

At block404, the process400involves performing entity and relation vectorization on the entities and relations of the triplets. Instead of performing one-hot vectorization on the entities and relations of the triplets, the entities and relations are vectorized using a long short-term memory (LSTM) embedding tool. For example, the words representing an entity (e.g., heterogeneous preference) is fed into the LSTM embedding tool where the entities are assigned a vector (e.g., heterogeneous_preference) that includes a value that is similar to vector values of other similarly defined entities. The same process is performed with the relations of the triplets. The embeddings may be based on a global vectors for word representation model (e.g., a GloVe model), a trained GloVe model based on the data source of the control performance data124, or a combination thereof.

At block406, the process400involves determining inverses of the relations between the entities. For example, the performance evaluation system102generates an inverse triplet for each of the triplets identified in the control performance data124. The inverse triplet is represented as (e2, r−1, e1). Using the example triplet from block402, the inverse triplet may be represented as (segmentation, ResultsFrom, heterogeneous preference). The inverse triplets provide paths within the knowledge graph136for antecedent relationship paths to be analyzed. That is, the inverse triplets enable answering a query looking for why the subject entity140occurred rather than what will result from the subject entity140when provided to the assessment model128. In an example, the inverse relations are also vectorized using the LSTM embedding tool.

At block408, the process400involves learning paths between entities using the subject entity140and the subject relation142. For example, each entity from the triplets identified in the control performance data124generates a node of the knowledge graph136, and each relation from the triplets identified in the control performance data124generates an edge that links two of the entities. Additionally, the knowledge graph136interlinks triplets based on common entities located within the triplets identified in the control performance data124. Because the knowledge base from the control performance data124is incomplete (e.g., the answer to every query is not directly represented), the paths between entities may provide the assessment model128with the ability to infer answers (e.g., the target entities130) from the knowledge graph136. For example, the path from the subject entity140to the target entity130may travers multiple relational hops (e.g., triplets) before the path reaches the target entity130.

At block410, the process400involves scoring the paths between entities using the reinforcement learning agent110of the performance evaluation system102. In an example, scoring the paths between entities provides an indication of path efficiency for the path from the subject entity140to the target entity130, and also provides an indication of relevancy of the target entity130to the subject entity140. That is, it provides an indication of how much effect a change to the target entity130will have on the subject entity140or how much effect a change to the subject entity140will have on the target entity130.

The query being assessed over the knowledge graph136by the assessment model128is a finite horizon sequential decision making problem. The reinforcement learning agent110is used at block410to help solve the decision making problem. Starting with a query in the form (e1q, rq, ?), the reinforcement learning agent110learns to traverse the knowledge graph136to retrieve the answer, e2q. For example, during implementation, the knowledge graph136is unrolled up to a fixed number of time steps. At each step from a current location (e.g., a current entity) within the knowledge graph136, the reinforcement learning agent110is able to take an action in an attempt to advance toward the answer to the query, e2q. In an example, the action taken by the reinforcement learning agent110is a relation that extends as an edge from the current entity and links the current entity with a subsequent entity (e.g., either the target entity130or an intermediate entity along the path to the target entity130). Selection of the action taken by the reinforcement learning agent110is based on the query and the relations traversed up to the current entity (i.e., the history).

The reinforcement learning agent110defines an environment of the knowledge graph136. This environment is defined as a 5-tuple (S, O, A, δ, R), where S is the state Space, O is an observation function, A is a set of possible actions from the state Space S, δ is a transition function, and R is the reward functions104. The state Space S encodes the query (e1q, rq), the target entity (e2q), and a current location of exploration (et) within the knowledge graph136(i.e., a hop location within the knowledge graph136between the subject entity140and the target entity130). The state is represented by the following equation:
s=(et,e1q,rq,e2q)  (1)
A complete state of the environment is not observable, but the current location of exploration may be observed without the answer. Formally, the observation function O is represented by the following equation:
O(s=(et,e1q,rq,e2q))=(et,e1q,rq)  (2)
The set of possible actions A from the state Space S includes all outgoing edges (i.e., relations) in the knowledge graph136from a vertex v (i.e., the current location of exploration et). Formally, the set of possible actions A is represented by the following equation:
As={(et,r,v)∈E:S=(et,e1q,rq,e2q),r∈R,v∈V}∪{(S,Ø,s)}  (3)
where Ø indicates that an available action is to remain at a current location etby taking no action (i.e., NO OP). The set of possible actions A indicates that the reinforcement learning agent110has an option to select which outgoing edge from the current location etshould be taken on a path to the target entity130. During implementation, the knowledge graph136is unrolled up to a fixed number of time steps T (e.g., relational hops). The NO OP element of the set of possible actions A provides the ability for no hop to be taken to another entity from the current location etwithin the knowledge graph136. This NO OP element may be beneficial when the reinforcement learning agent110reaches a correct answer at a time step t that is less than the fixed number of time steps T. Additionally, the addition of the inverse relations (e.g., at block406) equips the reinforcement learning agent110with the ability to undo a wrong decision and return to a prior node at the next time step t.

In an example, the environment defined by the reinforcement learning agent110evolves deterministically by updating the state S to a new vertex (e.g., a new current location et) by traversing a selected edge (e.g., relation r) from the previous location et-1. The transition function δ associated with such an update is defined by the following equation:
δ(S,A)=(v,e1q,rq,e2q)  (4)
where S=(et, e1q, rq, e2q) and A=(et, r, v). Further, the reward functions104may be provided when the target entity130is equal to etin the final state, when a correct path is taken to the target entity130, or both.

A policy network is employed to make decisions that choose an action from all of the available actions Asfrom a specific state S. In an example, the policy network is a randomized history-dependent policy network π equal to (d1, d2, . . . , dT-1) where dtis a probability distribution over all of the available actions Asat a time step t taking into account history ht. The history htrepresents a sequence of observations and actions taken up to time step t from the subject entity140. In an example, the history htis encoded as a continuous vector using a long short-term memory (LSTM) network. For example, the embedding vectors of the current entity et, as determined using the LSTM embedding tool at block404, the relation rt-1chosen to reach the current entity et, and the history embedding ht-1from the previous step are all encoded by the LSTM network to generate the current history ht. For example, the current history htis represented by the following equation:
ht=LSTM(ht-1,[rt-1;et])  (5)

In an example, the policy network is a two-layer feedforward network with rectified linear unit (ReLU) nonlinearity. The policy network receives the current history htand the embedding for the subject relation142(e.g., the query relation rq) and outputs a probability distribution over the available actions Asfrom the current entity et. The distribution is represented by the following equation:
dt=softmax(At(W2ReLU(W1[ht;et;rq])))  (6)
where weights W1and W2are biases corresponding to a regression constant or intercept.

In an example, parameters of the policy network are updated during training. In such an example, the parameters are adjusted using a policy gradient based on a cross-entropy loss function and the reward functions104. The changes to the parameters of the policy network are intended to maximize the expected rewards for a path originating from the current entity et. As discussed above, the expected rewards are based on achieving a correct target entity, following a correct path to the target entity, or a combination thereof. In an example, the parameters of the policy network include embedding size, hidden size, weight of the batch loss, a learning rate, a maximum number of actions, a combination thereof, or any other parameters capable of contributing to an increase in the expected rewards.

Examples of Knowledge Graph Operation

FIG.5is a visual representation of the knowledge graph136in a forward relation orientation. That is, a query (e.g., the subject entity140and the subject relation142) provided to or otherwise accessed by the assessment model128is requesting information about what target entities130will be affected by changes to the subject entity140based on the subject relation142(e.g., consequences of the changes to the subject entity140). As illustrated, the subject entity140is “psychographics” and the subject relation142is “enables_effectiveness.” Thus, a query of (psychographics, enables_effectiveness, ?) requests target entities130of which psychographics can enable effectiveness.

The results of the query are provided in the knowledge graph136as the target entities130of “sell_effort” and “high_advertising_expenditure.” That is, psychographics can be leveraged to enable the effectiveness of selling effort and high advertising expenditures. In contrast, the knowledge graph136may also identify entities502where psychographics are not leverageable to enable the effectiveness of the entities502. For example, the entities502of “perceptual_map,” “opportunity_for_new_product,” and “competition” may all be accessible from the subject entity140along a relation path within a set number of actions (e.g., 3 relational hops in the illustrated figure), but none of these entities502have effectiveness enabled by leveraging psychographics.

In an example, the target entities130are output by the assessment model128as the ranked set of target entities130. The ranking of the target entities130are determined based on how significantly changes to the subject entity140change the target entities130. In one or more examples, the target entity130with a highest likelihood of being reached is included at the top of the ranked list. In one or more examples, the assessment model128also outputs a path504from the subject entity140to the target entity130to help understand how the subject entity140affects the target entity130. Using the ranked set of target entities130output by the assessment model128, the focal host system112is able to update or apply the interventions132to the focal online platform114to enhance a desired positive impact on the focal online platform114.

FIG.6is a visual representation of the knowledge graph136in an antecedent relation orientation. That is, a query (e.g., the subject entity140and the subject relation142) provided to the assessment model128is requesting information about what target entities130have changes that affected the subject entity140based on the subject relation142. The relations between the nodes of the knowledge graph136are represented as inverse relations because of the antecedent relation orientation. As illustrated, the subject entity140is “identify_future_opportunity” and the subject relation142is “what_kind_of_analysis.” Thus, a query of (identify_future_opportunity, what_kind_of_analysis, ?) is requesting target entities130that provide analysis capable of identifying future opportunities.

The results of the query are provided in the knowledge graph136as the target entities130of “selling_effort,” “product_life_cycle,” “segmentation,” and “measured_differences_between_consumers.” That is, selling effort, a product life cycle, segmentation, and measured differences between consumers can all provide an analysis capable of identifying future opportunities. In contrast, the knowledge graph136may also identify entities602the do not provide an analysis capable of identifying future opportunities. For example, the entity602of “identify_market_changes” may be accessible from the subject entity140along a relation path within a set number of actions (e.g., 3 relational hops in the illustrated figure), but the entity602does not provide an analysis capable of identifying future opportunities. This determination is made through training of the assessment model128. For example, ground truths are provided during training that have the “correct answers.” Once trained, the assessment model128can incorporate these “correct answers” into the test data to determine that specific entities are incorrect (e.g., the entity602).

In an example, the target entities130are output by the assessment model128as the ranked set of target entities130. The ranking of the target entities130are determined based on how significantly changes to the target entities130change the subject entities140. In one or more examples, the target entity130with a highest likelihood of changing the subject entities140is included at the top of the ranked list. In an example, the assessment model128also outputs a path604from the subject entity140to the target entity130to help understand how the subject entity140is affected by the target entity130. Using the ranked set of target entities130output by the assessment model128, the focal host system112is able to update or apply the interventions132to the focal online platform114to enhance a desired positive impact on the focal online platform114.

While the knowledge graphs136depicted inFIGS.5and6depict examples of 3-hop answers to the query, other action numbers may be used to determine answers to the query. For example, the performance evaluation system102with sufficient memory and computational power may include answers resulting from 4 or more hops from the subject entity140. In another example, the performance evaluation system102with limited memory and computational power may include answers resulting from 2 or fewer hops from the subject entity140.

Example of a Computing System for Implementing Certain Embodiments

Any suitable computing system or group of computing systems can be used for performing the operations described herein. For example,FIG.7depicts an example of the computing system700. The implementation of computing system700could be used for one or more of a performance evaluation system102, a focal host system112, and a control host system122. In other embodiments, a single computing system700having devices similar to those depicted inFIG.7(e.g., a processor, a memory, etc.) combines the one or more operations and data stores depicted as separate systems inFIG.1.

The depicted example of a computing system700includes a processor702communicatively coupled to one or more memory devices704. The processor702executes computer-executable program code stored in a memory device704, accesses information stored in the memory device704, or both. Examples of the processor702include a microprocessor, an application-specific integrated circuit (“ASIC”), a field-programmable gate array (“FPGA”), or any other suitable processing device. The processor702can include any number of processing devices, including a single processing device.

The computing system700executes program code705that configures the processor702to perform one or more of the operations described herein. Examples of the program code705include, in various embodiments, the reward functions104and the reinforcement learning agent110by the performance evaluation system102, the focal online platform114, the control performance data124or other suitable applications that perform one or more operations described herein (e.g., one or more development applications for configuring the online platforms114or122). The program code may be resident in the memory device704or any suitable computer-readable medium and may be executed by the processor702or any other suitable processor.

In some embodiments, one or more memory devices704stores program data707that includes one or more datasets and models described herein. Examples of these datasets include interaction data, performance data, etc. In some embodiments, one or more of data sets, models, and functions are stored in the same memory device (e.g., one of the memory devices704). In additional or alternative embodiments, one or more of the programs, data sets, models, and functions described herein are stored in different memory devices704accessible via a data network. One or more buses706are also included in the computing system700. The buses706communicatively couples one or more components of a respective one of the computing system700.

In some embodiments, the computing system700also includes a network interface device710. The network interface device710includes any device or group of devices suitable for establishing a wired or wireless data connection to one or more data networks. Non-limiting examples of the network interface device710include an Ethernet network adapter, a modem, and/or the like. The computing system700is able to communicate with one or more other computing devices (e.g., a computing device executing a performance evaluation system102) via a data network using the network interface device710.

The computing system700may also include a number of external or internal devices, an input device720, a presentation device718, or other input or output devices. For example, the computing system700is shown with one or more input/output (“I/O”) interfaces708. An I/O interface708can receive input from input devices or provide output to output devices. An input device720can include any device or group of devices suitable for receiving visual, auditory, or other suitable input that controls or affects the operations of the processor702. Non-limiting examples of the input device720include a touchscreen, a mouse, a keyboard, a microphone, a separate mobile computing device, etc. A presentation device718can include any device or group of devices suitable for providing visual, auditory, or other suitable sensory output. Non-limiting examples of the presentation device718include a touchscreen, a monitor, a speaker, a separate mobile computing device, etc.

AlthoughFIG.7depicts the input device720and the presentation device718as being local to the computing device that executes the performance evaluation system102, other implementations are possible. For instance, in some embodiments, one or more of the input device720and the presentation device718can include a remote client-computing device that communicates with the computing system700via the network interface device710using one or more data networks described herein.

General Considerations