Patent ID: 12217190

DETAILED DESCRIPTION

In the following description of various illustrative embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown, by way of illustration, various embodiments in which aspects of the disclosure may be practiced. In some instances, other embodiments may be utilized, and structural and functional modifications may be made, without departing from the scope of the present disclosure.

It is noted that various connections between elements are discussed in the following description. It is noted that these connections are general and, unless specified otherwise, may be direct or indirect, wired or wireless, and that the specification is not intended to be limiting in this respect.

As a brief introduction to the concepts described further herein, one or more aspects of the disclosure describe a hybrid solution for leveraging both machine learning and knowledge graphs in decision making. For example, machine learning and artificial intelligence may be divided into two broad methods—statistical machine learning and syntactic machine learning. Statistical machine learning may be concerned with automatic discovery of regularities in data through the use of computer algorithms and with the use of these regularities to take actions such as classifying data into different categories. Machine learning systems may be trained from labelled training data (e.g., in supervised learning) or unlabeled data (e.g., in unsupervised learning).

Structural and syntactic machine learning is based on symbolic data structures such as strings, trees, graphs, arrays for pattern representation, and/or other structures. Such data structures may allow description of relations between elementary pattern components and may provide means for hierarchical models showing how complex patterns are built up from simpler parts. The recognition of an unknown pattern may be accomplished by comparing its symbolic representation with a number of predefined object models. In the structural approach, the comparison may be made by a symbolic match that computes a measure of similarity between the unknown input and a number of prototype models. In syntactic pattern recognition, a parser or error correcting parser may check an unknown input to identify whether or not it is in accordance with the rules of a grammar that describes all members of a pattern class.

A knowledge graph is a graph representation of knowledge and information. For example, a knowledge graph may be a graph representation of an abstraction of relationships among related entities. The entities in a knowledge graph may be represented by nodes and all the attributes of the entities are stored along with the nodes. Related nodes may be connected by directed or undirected edges. These edges may describe the relationships among those nodes or entities.

For example, the information “Person #1 lives in New York” may be represented by a knowledge graph having two nodes “Person #1” and “New York” with a directed edge connection node from “Person #1” to “New York” where the edge has the associated property describing the relationship “lives in.”

In this regard, knowledge graphs may be used for inferencing. For example, the above described knowledge graph may be expanded to include another node “USA” with a directed edge connection between “New York” and “USA” that has the property “lives in.” Accordingly, the knowledge graph may be used to infer that “Person #1 lives in USA.”

Historically, statistical machine learning and syntactic machine learning have been pursued separately. However, purely statistical machine learning may demand a significant amount of data, which may be difficult to obtain. Even if the necessary amount of data is obtained, data may be inaccurate or mislabeled, which may lead to an unstable model and/or deteriorated prediction rate.

On the other hand, purely knowledge graph based syntactic methods may need to store so much symbolic information that the data load may become extremely large and computationally expensive.

In a hybrid approach both statistical and syntactic methods may be used together to reach a decision. For example, both statistical as well as syntactic machine learning methods may be used independently to make an inference. However, only one of the inferences from one of these methods may be returned as the final inference and the other result may be thrown away. In some instances, several statistical and syntactic methods might be used, each one independent from another, and only one of the inferences may ultimately be used.

There does not exist a systematic method to choose one method over the other. For example, confidence levels may be used to select a more accurate model, however, if the confidence levels of two different inferences are similar, there is no obvious solution to choose one over the other.

Choosing one inference over others may also reduce the value of information of hybrid inferencing methods since it utilizes the results of merely one computation, and disregards the other. Accordingly, the systems and methods described herein describe a solution to these technical problems, in which the inferences of both syntactic and statistical machine learning methods may be combined together so as to achieve the benefits of both methods. In doing so, the accuracy of such inferencing methods may be increased by incorporating the information and benefits of both syntactic and statistical machine learning methods.

FIGS.1A-1Bdepict an illustrative computing environment for correlating machine learning models and knowledge graphs for improved decision making in accordance with one or more example embodiments. Referring toFIG.1A, computing environment100may include one or more computer systems. For example, computing environment100may include machine learning and knowledge graph host platform102, enterprise data source103, and enterprise computing device104.

As described further below, machine learning and knowledge graph host platform102may be a computer system that includes one or more computing devices (e.g., servers, server blades, or the like) and/or other computer components (e.g., processors, memories, communication interfaces) that may be used to train, maintain, and implement machine learning models and/or knowledge graphs that may be applied and correlated for improved decision making.

Enterprise data source103may include one or more computing devices (e.g., servers, server blades, or the like) and/or other computer components (e.g., processors, memories, communication interfaces) that may be used to store data (e.g., labelled or unlabeled data) that may be used to train a machine learning model and/or generate a knowledge graph. In some instances, the enterprise data source103may be configured to communicate with the machine learning and knowledge graph host platform102for the purpose of sending the training data.

Enterprise computing device104may be a laptop computer, desktop computer, mobile device, tablet, smartphone, or the like that may be used by an employee or administrator of an enterprise organization (e.g., a financial institution, or the like). For example, the enterprise computing device104may be used by one or more individuals for event processing and/or otherwise accessing results produced by the machine learning and knowledge graph host platform102. In some instances, enterprise computing device104may be configured to display one or more user interfaces (e.g., event processing, and/or other interfaces).

Computing environment100also may include one or more networks, which may interconnect machine learning and knowledge graph host platform102, enterprise data source103, and/or enterprise computing device104. For example, computing environment100may include a network101(which may interconnect, e.g., machine learning and knowledge graph host platform102, enterprise data source103, and/or enterprise computing device104).

In one or more arrangements, machine learning and knowledge graph host platform102, enterprise data source103, and/or enterprise computing device104may be any type of computing device capable of sending and/or receiving requests and processing the requests accordingly. For example, machine learning and knowledge graph host platform102, enterprise data source103, enterprise computing device104, and/or the other systems included in computing environment100may, in some instances, be and/or include server computers, desktop computers, laptop computers, tablet computers, smart phones, or the like that may include one or more processors, memories, communication interfaces, storage devices, and/or other components. As noted above, and as illustrated in greater detail below, any and/or all of machine learning and knowledge graph host platform102, enterprise data source103, and/or enterprise computing device104, may, in some instances, be special-purpose computing devices configured to perform specific functions.

Referring toFIG.1B, machine learning and knowledge graph host platform102may include one or more processors111, memory112, and communication interface113. A data bus may interconnect processor111, memory112, and communication interface113. Communication interface113may be a network interface configured to support communication between machine learning and knowledge graph host platform102and one or more networks (e.g., network101, or the like). Memory112may include one or more program modules having instructions that when executed by processor111cause machine learning and knowledge graph host platform102to perform one or more functions described herein and/or one or more databases that may store and/or otherwise maintain information which may be used by such program modules and/or processor111. In some instances, the one or more program modules and/or databases may be stored by and/or maintained in different memory units of machine learning and knowledge graph host platform102and/or by different computing devices that may form and/or otherwise make up machine learning and knowledge graph host platform102. For example, memory112may have, host, store, and/or include machine learning and knowledge graph module112a, machine learning and knowledge graph database112b, and machine learning engine112c.

Machine learning and knowledge graph module112amay have instructions that direct and/or cause machine learning and knowledge graph host platform102to execute advanced techniques to correlate machine learning models and knowledge graphs for improved decision making. Machine learning and knowledge graph database112bmay store information used by machine learning and knowledge graph module112aand/or machine learning and knowledge graph host platform102in application of advanced machine learning techniques to correlate machine learning models and knowledge graphs for improved decision making, and/or in performing other functions. Machine learning engine112cmay have instructions that direct and/or cause the machine learning and knowledge graph host platform102to set, define, and/or iteratively refine optimization rules and/or other parameters used by the machine learning and knowledge graph host platform102and/or other systems in computing environment100.

FIGS.2A-2Edepict an illustrative event sequence for correlating machine learning models and knowledge graphs for improved decision making in accordance with one or more example embodiments. Referring toFIG.2A, at step201, enterprise data source103may establish a wireless data connection with the machine learning and knowledge graph host platform102. For example, the enterprise data source103may establish a first wireless data connection with the machine learning and knowledge graph host platform102to link the enterprise data source103to the machine learning and knowledge graph host platform102(e.g., in preparation for sending event processing data). In some instances, the enterprise data source103may identify whether or not a connection is already established with the machine learning and knowledge graph host platform102. If a connection is already established with the machine learning and knowledge graph host platform102, the enterprise data source103might not re-establish the connection. If a connection is not yet established with the machine learning and knowledge graph host platform102, the enterprise data source103may establish the first wireless data connection as described herein.

At step202, the enterprise data source103may send event processing data to the machine learning and knowledge graph host platform102. For example, the enterprise data source103may send event processing data to the machine learning and knowledge graph host platform102while the first wireless data connection is established. In some instances, in sending the event processing data, the enterprise data source103may send data corresponding to historical event processing (e.g., transactions amounts, account information, and/or other historical processing information).

At step203, the machine learning and knowledge graph host platform102may receive the event processing data sent at step202. For example, the machine learning and knowledge graph host platform102may receive the event processing data via the communication interface113and while the first wireless data connection is established.

At step204, the machine learning and knowledge graph host platform102may train a machine learning model (e.g., a supervised machine learning model, unsupervised learning model, and/or combination of models). For example, the machine learning and knowledge graph host platform102may train the machine learning model, using the historical event processing data received at step203, to solve various event processing requests (e.g., distinguish between questionable, maybe questionable, and valid transactions, and/or identify a corresponding account, transaction number, payor, payee, payment amount, date, enterprise, and/or other information corresponding to previously processed events). In doing so, the machine learning and knowledge graph host platform102may generate various data clusters, each corresponding to various event processing data (e.g., a cluster of transactions corresponding a particular account, and/or otherwise that share a particular property). As another example, the machine learning and knowledge graph host platform102may distinguish between a questionable, maybe questionable, or not questionable (e.g., valid) transaction. This may enable the machine learning and knowledge graph host platform102to group similar data together, which may enable labeling future data and/or otherwise outputting a data response based on future data by associating it with a particular cluster.

At step205, the model generation and knowledge graph host platform102may generate a knowledge graph using the historical event processing data. For example, the model generation and knowledge graph host platform102may generate nodes and relationships between the generated nodes to represent the historical event processing data. As a particular example, if a particular data point of the historical event processing data indicates that $50 were transmitted from “Account #1” to “Account #2,” nodes may be generated to represent each account, and an edge may connect the two nodes that is stored with the property “transmitted $50 to,” flowing from the “Account #1” node to the “Account #2” node. In some instances, in generating the knowledge graph, the model generation and knowledge graph host platform102may generate a knowledge graph that includes the same information that is stored in the machine learning model that was trained at step204.

In some instances, the machine learning and knowledge graph host platform102may enhance the knowledge graph by creating clusters (e.g., using a depth first search method that checks off all nodes reached within a threshold number of hops). By creating such clusters, the machine learning and knowledge graph host platform102may enable correlation of information stored in the machine learning model and the knowledge graph.

Referring toFIG.2B, at step206, the machine learning and knowledge graph host platform102may generate a correlation matrix to relate data nodes on the knowledge graph (e.g., generated at step205) with data points in the machine learning model (e.g., generated at step204). For example, the machine learning and knowledge graph host platform102may generate a correlation matrix that indicates which data points (in the machine learning model) correspond to which data nodes (in the knowledge graph). For example, the machine learning and knowledge graph host platform102may store the same data in the machine learning model and the knowledge graph, and may generate the correlation matrix to maintain correlations between the two different representations of the same data (e.g., the machine learning representation and the knowledge graph representation of a particular data point).

More specifically, the machine learning and knowledge graph host platform102may perform a statistical calculation based on the data points in the machine learning model and the data nodes in the machine learning model to express a relationship between the historical event processing data as represented in these two different forms (e.g., machine learning and knowledge graph). In doing so, the machine learning and knowledge graph host platform102may identify a value between −1 and 1, where 1 indicates a perfect positive relationship between two clusters (a machine learning cluster and a knowledge graph cluster) while −1 indicates a perfect negative relationship between the two clusters (a machine learning cluster and a knowledge graph cluster) (0 indicates that two data sets are not correlated).

As a particular example, the machine learning and knowledge graph host platform102may apply Pearson's formula to compute the correlation coefficients

(e.g.,correlation⁢⁢coefficient=Covariance⁡(A,B)(S⁢t⁢d⁢D⁢e⁢v⁡(A)*S⁢t⁢d⁢D⁢e⁢v⁡(B),
where A and B are corresponding clusters of the machine learning model and the knowledge graph respectively). After computing these correlation coefficients between clusters of the machine learning model and clusters of the knowledge graph, the machine learning and knowledge graph host platform102may store the correlation coefficients in the correlation matrix. For example, the machine learning data points may represent the y axis and the knowledge graph nodes may represent the x axis, with the correlation coefficients filling in the intersections of corresponding machine learning data points and knowledge graph nodes.

At step207, the machine learning and knowledge graph host platform102may identify compactness values for the machine learning model and the knowledge graph. For example, the machine learning and knowledge graph host platform102may use compactness to define how well clustered and/or compacted clusters in both the machine learning model and the knowledge graph are. For example, if the machine learning and knowledge graph host platform102identifies that the clusters do not have well defined boundaries and are scattered, the machine learning and knowledge graph host platform102may identify the corresponding knowledge graph and/or machine learning model as an inaccurate model.

For the machine learning model, the machine learning and knowledge graph host platform102may compute a compactness value by taking a sum of the area of the convex hull of each cluster, and divide this sum by the total area of all the clusters. In these instances, the machine learning and knowledge graph host platform102may then assign a compactness value equivalent to the result. Additionally or alternatively, the machine learning and knowledge graph host platform102may identify whether or not entropy of the machine learning clusters exceeds a predetermined entropy threshold, and may assign a compactness value accordingly (e.g., if not exceeded, value=0.3; if equal to threshold, value=0.5; if exceeded, value=1). Additionally or alternatively, the machine learning and knowledge graph host platform102may identify an average Euclidian distance between data points and the center of their corresponding data cluster, compare the average distance to a plurality of threshold distances, and assign a corresponding compactness value based on the comparison.

With regard to the knowledge graph, the machine learning and knowledge graph host platform102may identify the maximum topological distances of each cluster (e.g., measured in hops or nodes), and may divide these distances by the largest path of the entire knowledge graph to identify compactness of each cluster. Additionally or alternatively, the machine learning and knowledge graph host platform102may identify an average number of hops or edges between data nodes of the data nodes to a center of a corresponding cluster, compare the distance to a plurality of threshold distances, and assign a corresponding compactness value based on the comparison.

At step208, the enterprise data source103may send new event processing data to the machine learning and knowledge graph host platform102. For example, the enterprise data source103may send new event processing data to the machine learning and knowledge graph host platform102while the first wireless data connection is established. In some instances, in sending the new event processing data, the enterprise data source103may send data/information similar to the historical event processing data sent at step202.

At step209, the machine learning and knowledge graph host platform102may receive the new event processing data sent at step208. For example, the machine learning and knowledge graph host platform102may receive the new event processing data via the communication interface113and while the first wireless data connection is established.

At step210, the machine learning and knowledge graph host platform102may input the new event processing data (received at step209) into the machine learning model. In some instances, in inputting the new event processing data into the machine learning model, the machine learning and knowledge graph host platform102may identify k nearest neighbor data points (e.g., of the historical event processing data stored in the machine learning model) closest to the data point corresponding to the new event processing data. For example, the machine learning and knowledge graph host platform102may identify a Euclidian distance between the data point corresponding to the new event processing data and the data points corresponding to the historical event processing data. The machine learning and knowledge graph host platform102may then rank the Euclidian distances from smallest to largest, select the k smallest distances, and identify the data points corresponding to these k smallest distances. Although a Euclidian distance is described herein, other metrics such as the Manhattan metric, L-Infinity metric, and/or other metrics may be applied without departing from the scope of the disclosure.

Referring toFIG.2C, at step211, the machine learning and knowledge graph host platform102may input the new event processing data (e.g., that was input into the machine learning model at step210) into the knowledge graph. In some instances, in inputting the new event processing data into the knowledge graph, the machine learning and knowledge graph host platform102may identify k nearest neighbor data nodes (e.g., of the historical event processing data stored in the knowledge graph) closest to the data node corresponding to the new event processing data. For example, the machine learning and knowledge graph host platform102may identify a number of hops and/or edges between the data node corresponding to the new event processing data and the data nodes corresponding to the historical event processing data (e.g., a number of data nodes between the data node corresponding to the new event processing data and a particular cluster corresponding to the historical event processing data). The machine learning and knowledge graph host platform102may then rank the hop distances from smallest to largest, select the k smallest distances, and identify the data nodes corresponding to these k smallest distances.

At step212, the machine learning and knowledge graph host platform102may use the correlation matrix (generated at step206) to identify correlation values corresponding to the k nearest data points and/or data nodes. At step213, the machine learning and knowledge graph host platform102may compute relative weighted distances between the new event processing data/node and the k nearest data points and/or data nodes. To do so for the machine learning model, the machine learning and knowledge graph host platform102may define the relative distances as Euclidian distances between the new event processing data and the k nearest data points divided by the corresponding confidence values. With regard to the knowledge graph clusters, the machine learning and knowledge graph host platform102may define the relative distances as the hop distances between the new event processing node and the k nearest data nodes divided by the corresponding confidence values. This may result in ratios that may be comparable as relative distances (e.g., distance ratios based on corresponding cluster sizes).

At step214, the machine learning and knowledge graph host platform102may divide the relative weighted distances, computed at step213, by the corresponding compactness values identified at step207. In doing so, the machine learning and knowledge graph host platform102may generate comparable distance ratios between the machine learning model and the knowledge graph. In some instances, the machine learning and knowledge graph host platform102may rank these adjusted relative weighted distances from smallest to largest and identify the smallest adjusted relative weighted distance.

Referring toFIG.2D, at step215, the machine learning and knowledge graph host platform102may select a data point and/or data node with the smallest compacted relative weighted distance (e.g., based on the computations from step214), and may identify that the new event processing data should be labelled based on a cluster corresponding to the selected data point and/or data node. Additionally or alternatively, the machine learning and knowledge graph host platform102may poll the k nearest data points and/or data nodes to identify a cluster corresponding to a majority (or some other threshold number) of the k nearest data points and/or data nodes. In these instances, the machine learning and knowledge graph host platform102may identify that the new event processing data should be labelled based on the identified cluster.

In doing so, the machine learning and knowledge graph host platform102may increase accuracy of the automated prediction making described above. For example, rather than sending decisions from each of the machine learning model and the knowledge graph to a decision engine, and having the decision engine select one of the two decisions based on confidence values, the machine learning and knowledge graph host platform102effectively applies a unique resolution process that makes a predictive decision based on correlations between the machine learning model and the knowledge graph, as well as the distance of a given point from each of the clusters that exist in both the machine learning model and the knowledge graph.

At step216, the machine learning and knowledge graph host platform102may label the new event processing data based on the selected data point/data node and its corresponding cluster (e.g., label a corresponding account, payor, payee, transaction amount, and/or other event processing information). Additionally or alternatively, the machine learning and knowledge graph host platform102may use the method described to label a particular event as a questionable, maybe questionable, or not questionable (e.g., valid) transaction.

At step217, the machine learning and knowledge graph host platform102may update the machine learning model and/or knowledge graph based on the labelled new event processing data. At step218, machine learning and knowledge graph host platform102may send event processing information, indicating the labelled new event processing data, to the enterprise computing device104. For example the machine learning and knowledge graph host platform102may establish a second wireless data connection with the enterprise computing device104to link the machine learning and knowledge graph host platform102to the enterprise computing device104(e.g., in preparation for sending the event processing information). In some instances, the machine learning and knowledge graph host platform102may identify whether or not a connection is already established with the enterprise computing device104. If a connection is already established with the enterprise computing device104, the machine learning and knowledge graph host platform102might not re-establish the connection. If a connection is not yet established with the enterprise computing device104, the machine learning and knowledge graph host platform102may establish the second wireless data connection as described herein.

Accordingly, the machine learning and knowledge graph host platform102may send event processing information to the enterprise computing device104via the communication interface113and while the second wireless data connection is established. In some instances, along with the event processing information, the machine learning and knowledge graph host platform102may send one or more commands directing the enterprise computing device104to display the event processing information.

At step219, the enterprise computing device104may receive the event processing information sent at step218. For example, the enterprise computing device104may receive the event processing information while the second wireless data connection is established. In some instances, the enterprise computing device104may also receive the one or more commands directing the enterprise computing device104to display the event processing information.

Referring toFIG.2E, at step220, based on or in response to the one or more commands directing the enterprise computing device104to display the event processing information, the enterprise computing device104may display the event processing information. For example, the enterprise computing device104may display a graphical user interface similar to graphical user interface405, which is illustrated inFIG.4.

FIG.3depicts an illustrative method for correlating machine learning models and knowledge graphs for improved decision making in accordance with one or more example embodiments. Referring toFIG.3, at step305, a computing platform having at least one processor, a communication interface, and memory may receive historical event processing data. At step310, the computing platform may train the machine learning model using the historical event processing data. At step315, the computing platform may generate a knowledge graph using the historical event processing data. At step320, the computing platform may generate a correlation matrix to relate the knowledge graph and the machine learning model. At step325, the computing platform may identify compactness values for the machine learning model and the knowledge graph. At step330, the computing platform may receive new event processing data. At step335, the computing platform may input the new event processing data into the machine learning model. At step340, the computing platform may input the new event processing data into the knowledge graph. At step345, the computing platform may use the correlation matrix to identify correlation values between results of inputting the new event processing data into the machine learning model and the knowledge graph. At step350, the computing platform may compute relative weighted distances corresponding to the results. At step355, the computing platform may divide the relative weighted distances by the compactness values. At step360, the computing platform may select a corresponding data point for the event processing data. At step365, the computing platform may label the new event processing data based on the corresponding data point. At step370, the computing platform may update the machine learning model and the knowledge graph based on the labelled event processing data. At step375, the computing platform may send event processing information, based on the labelled event processing data, to an enterprise computing device for display.

One or more aspects of the disclosure may be embodied in computer-usable data or computer-executable instructions, such as in one or more program modules, executed by one or more computers or other devices to perform the operations described herein. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types when executed by one or more processors in a computer or other data processing device. The computer-executable instructions may be stored as computer-readable instructions on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, RAM, and the like. The functionality of the program modules may be combined or distributed as desired in various embodiments. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGA), and the like. Particular data structures may be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated to be within the scope of computer executable instructions and computer-usable data described herein.

Various aspects described herein may be embodied as a method, an apparatus, or as one or more computer-readable media storing computer-executable instructions. Accordingly, those aspects may take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, or an embodiment combining software, hardware, and firmware aspects in any combination. In addition, various signals representing data or events as described herein may be transferred between a source and a destination in the form of light or electromagnetic waves traveling through signal-conducting media such as metal wires, optical fibers, or wireless transmission media (e.g., air or space). In general, the one or more computer-readable media may be and/or include one or more non-transitory computer-readable media.

As described herein, the various methods and acts may be operative across one or more computing servers and one or more networks. The functionality may be distributed in any manner, or may be located in a single computing device (e.g., a server, a client computer, and the like). For example, in alternative embodiments, one or more of the computing platforms discussed above may be combined into a single computing platform, and the various functions of each computing platform may be performed by the single computing platform. In such arrangements, any and/or all of the above-discussed communications between computing platforms may correspond to data being accessed, moved, modified, updated, and/or otherwise used by the single computing platform. Additionally or alternatively, one or more of the computing platforms discussed above may be implemented in one or more virtual machines that are provided by one or more physical computing devices. In such arrangements, the various functions of each computing platform may be performed by the one or more virtual machines, and any and/or all of the above-discussed communications between computing platforms may correspond to data being accessed, moved, modified, updated, and/or otherwise used by the one or more virtual machines.

Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and variations within the scope and spirit of the appended claims will occur to persons of ordinary skill in the art from a review of this disclosure. For example, one or more of the steps depicted in the illustrative figures may be performed in other than the recited order, and one or more depicted steps may be optional in accordance with aspects of the disclosure.