Linear late-fusion semantic structural retrieval

Systems and methods for generating a fusion score between electronic documents. The method includes receiving a first electronic document by a document management system. The method further includes extracting a first set of features from the first electronic document including at least one feature type indicating the hierarchical structure of the first electronic document. The method also includes receiving a second electronic document by the document management server. The method further includes extracting a second set of features from the second electronic document including at least one feature type indicating the hierarchical structure of the second electronic document. The method further includes generating a fusion score based on a comparison of the first set of features and the second set of features.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present disclosure generally relates to systems and methods for retrieving relevant information using multiple aspects of the structure.

2. Description of the Related Art

Electronic documents have been widely adopted as a means for facilitating collection, storage, or maintenance of information. The wide adoption of electronic documents has generated voluminous quantities across every organization and searching to find a document relevant to a particular query has presented an increasing challenge.

SUMMARY OF THE INVENTION

Systems and methods for generating a fusion score based on comparing sets of features extracted from electronic documents are disclosed. Certain embodiments involve extracting a set of features including a hierarchical structure of an electronic document and using the set of features to generate fusion scores between electronic documents. A document management system may extract features from a first electronic document. The features may include structural and semantic elements that may be used by the document management system to determine the type or content of the electronic document.

In one embodiment, the system and method include a document management server that receives a first electronic document. The method further includes extracting a first set of features from the first electronic document including at least one feature type indicating the hierarchical structure of the first electronic document. The method also includes receiving a second electronic document by the document management server. The method further includes extracting a second set of features from the second electronic document including at least one feature type indicating the hierarchical structure of the second electronic document. The method further includes generating a fusion score based on a comparison of the first set of features and the second set of features.

In some aspects, the method of extracting the first set of features may include extracting a tree path structure having a plurality of nodes from the first electronic document. For instance, each tree path structure may be a subset of the plurality of nodes that includes a particular arrangement of nodes (e.g., tree 1=node1_node2_node4_node3). The first electronic document may include a plurality of tree path structures.

In some aspects, the tree path structure includes semantic data for each node of the plurality of nodes. For instance, the semantic data may identify a position of each node within the tree path structure such as by including semantic data that is unique to the node's position within the tree path structure (e.g., a semantic meaning may be different for a different position of the node).

In some aspects, the method of generating the fusion score includes generating a weighted feature, by a machine learning model, for each feature of the first electronic document and the second electronic document. The method also may include computing a first similarity score between a first feature of the first electronic document and a second feature of the second electronic document. The method may also include computing the fusion score based on the first similarity score and the weighted feature.

In some aspects, computing the first similarity score may include computing a positive pointwise mutual information between the first feature and the second feature.

In some aspects, computing the first similarity score may include computing a term frequency-inverse document frequency between the first feature and the second feature. For example, the first similarity score may include computing how many times a word appears in the document (e.g., a first feature), and the inverse document frequency of the word across a set of documents (e.g., a second feature).

DETAILED DESCRIPTION

Embodiments are directed to systems and methods for generating a semantic fusion score of electronic documents, according to aspects of the present disclosure. For instance, an electronic document may include multiple types of information including information, such as textual fields in the document, file name, metadata, document structure, and server responses, that may be associated with the type of electronic document. The document management server may extract the various types of information and represent each type of information using a feature representation. By representing multiple types of information as features of the electronic document, a more relevant comparison between documents may be achieved.

The following non-limiting example is provided to introduce certain embodiments. In this example, an electronic document management system generates a fusion score between features of a first electronic document and a second electronic document. The document management may server may receive a first electronic document. The document management server may receive the first electronic document from a client device, another document management server, or from a similar computing device. The document management server may extract a first set of features from the first electronic document using a machine learning model or another featurization application. The first set of features may include a feature that indicates the hierarchical structure of the first electronic document. The hierarchical structure may be indicated by file name, text with semantic meanings that describe a file structure (e.g., “table”, “form”, etc.) or content of the electronic document, or the like. The document management system may also receive a second electronic document. The document management system may extract a second set of features from the second electronic document using a machine learning model or another featurization application. The second set of features may include a feature that indicates the hierarchical structure of the second electronic document. The document management system may generate a fusion score based on a comparison of the first set of features and the second set of features. The document management system may generate the fusion score by using a feature fusion engine that determines a similarity by applying a machine learning model to the first electronic document and the second electronic document using the first set of features and the second set of features.

Continuing with the present example, the document management system extracts the first set of features that includes extracting a tree path structure including multiple nodes from the first electronic document. The tree path structure may include semantic data that identifies a position of each node within the tree path structure (e.g., each node may have semantic data). Each node may have semantic data that is unique to the node's position within the tree path structure (e.g., a semantic meaning may be different (e.g., node 2 in tree path 1) for a different position of the node (node 2 in tree path 2), such as tree path 2 being a portion of tree path 1 that starts at node 2. The document management system computes a fusion score by generating a weighted feature, by a machine learning model, for each feature of the first electronic document and the second electronic document. The document management system further computes the fusion score by computing a first similarity score between a first feature of the first electronic document and a second feature of the second electronic document. The document management system computes the fusion score based on a combination of the first similarity score and the weighted feature.

In one aspect, the document management system computes the first similarity score using a positive pointwise mutual information between the first feature and the second feature.

In another aspect, the document management system computes the first similarity score including computing a term frequency-inverse document frequency for each document.

Referring now to the Figures,FIG.1depicts a system for generating a semantic fusion score of electronic documents, according to aspects of the present disclosure. System100may include a document management system101, one or more client devices130, and a network124. It should be noted that these entities are exemplary only; additional, fewer, and/or different entities may be provided as is necessary and/or desired.

For instance, the document management system101may receive an electronic document140from client device130. In one example, the electronic document140is an electronic file that includes any combination of text, images, or other data that stores information. The document management system101may include a feature extraction module110, a machine learning model116, a feature fusion engine120, and a data store103.

The client device(s)130may include a graphical user interface (GUI)132that provides an interface for selecting an electronic document and transmitting the electronic document140from the client device(s)130to the document management system101via the network124. The client device(s)130may also generate the electronic document140as a response to input received by the GUI132from the user of the client device(s)130.

The feature extraction module110extracts one or more features from the electronic document140. In one example, the feature extraction module110may extract features from the electronic document140including a feature that indicates the hierarchical structure of the electronic document140, one or more features that represent text or image content of the electronic document140, and one or more features that represent a previous response of the document management server in response to the electronic document140. An example of the features that represent the hierarchical structure of the electronic document140include feature vectors that represent tree-based features (e.g. the paths of nodes in an XML/JSON data source). The feature extraction module110may also generate a feature representation of a server response (e.g., a response to an HTTP request) associated with the document. In other aspects, the feature extraction module110may generate feature representations of application programming interface (API) corresponding to the electronic document140(e.g., a header, a version, a programming language) or file type such as PDF, XML, HTML, or the like. Additional detail with regard to feature extraction is discussed at least with respect toFIG.3.

In one aspect, the feature extraction module110may extract a tree structure from the electronic document. The feature extraction module110may extract one or more tree paths associated with the electronic document. The feature extraction module110may vectorize the tree paths to represent the one or more tree paths of the electronic document.

In another aspect, the feature extraction module110may extract text content from the electronic document. The feature extraction module110may extract one or more fields of text, such as flat text, associated with (e.g., metadata) or contained in the electronic document. The feature extraction module110may vectorize the text fields to represent the one or more text fields of the electronic document. Additionally, or alternatively, the feature extraction module110may extract the text and generate semantic relationships based on the text.

The document management system101may store the electronic document140in the data store103, such as in electronic document files104. The document management system101may also store the extracted features in document feature store105. In some embodiments, the document management system may store one of the electronic document140, the extracted features, or any combination thereof.

The client device(s)130may submit a second electronic document and request from the document management system101to return a set of relevant documents similar to the second electronic document. The document management system101may extract a second set of features from the second electronic document using a machine learning model or another featurization application. The second set of features may include a feature that indicates the hierarchical structure of the second electronic document. The document management system may compare the extracted features of the second electronic document with previously extracted features of one or more stored electronic documents. The document management system may generate a fusion score based on a comparison of the first set of features from the first electronic document (e.g., the stored electronic document features) and the second set of features. The document management system may generate the fusion score by using a feature fusion engine that determines a similarity by applying a machine learning model to the first electronic document and the second electronic document using the first set of features and the second set of features. In one example, the features can represent semantic of a tree path structure, semantics of text included in various fields, server responses associated with the electronic document, or the like.

The machine learning model116may be trained to generate a set of weights that correspond to the set of features from the electronic documents. In one example, the machine learning model116may determine that a subset of features from the set of features may be more relevant in determining the content, purpose, or type of documents. The machine learning model116may adjust the weights iteratively before, during, or after processing an electronic document. The machine learning model may additionally receive feedback from an expert system or a user (e.g., a user or another trained machine learning model) that provides an additional parameter, such as a quality score, to determine whether the weights generated by the machine learning model produced a fusion score above a threshold. In other words, the feedback may be used by the machine learning model to determine the weights that most accurately produce a maximum fusion score between a set of a first electronic document and the second electronic document.

The feature fusion engine120may generate a fusion score by comparing the sets of features between the first electronic document and the second electronic document. In one example, the feature fusion engine computes a similarity score between corresponding features of the first electronic document and the second electronic document.

In one aspect, the feature fusion engine computes a similarity score for a first tree structure of the first electronic document and a second tree structure of the second electronic document. An example of the tree structure is likely best understood with reference toFIG.3.

FIG.2illustrates an example of a method for generating a semantic fusion score of electronic documents, according to aspects of the present disclosure.

In step202, the process200involves receiving a first electronic document by the document management server. For example, the document management system101may receive a first electronic document, such as the electronic document, from a client device via a network. The document management server may be communicatively coupled to the network via a network interface device, such as an ethernet connection, wireless internet or cellular connection, Bluetooth, or any device or group of devices suitable for establishing a data connection to one or more data networks.

In step204, the process200involves extracting a first set of features from the first electronic document. For example, the document management system may employ a feature extraction module to extract one or more features of the first electronic document.

In step206, the process200involves generating a featurized representation of the first set of features from the first electronic document. The feature extraction module generates a feature representation of the first electronic document such as a feature vector that may include multiple features, each feature representing an individual measurable property of the first electronic document.

In step208, the process200involves receiving a second electronic document by the document management server. The operations and processes of step208may be similar to the operations or processes of step202.

In step210, the process200involves extracting a second set of features from the second electronic document. The operations and processes of step210may be similar to the operations or processes of step204.

In step212, the process200involves generating a featurized representation of the second set of features from the second electronic document. The operations and processes of step212may be similar to the operations or processes of step206.

In step214, the process200involves determining a fusion score based on a comparison of the first set of features and the second set of features. For instance, the feature fusion engine may use positive pointwise mutual information to determine the relevance between electronic documents. Positive pointwise mutual information determines a difference between a probability of a coincidence between a first feature and a second feature and the individual probabilities of the first feature and the second feature assuming that the features are independent. An example of the computations of the similarity score is provided below. The feature fusion engine may compute a similarity score using the positive pointwise mutual information (ppmi) between features of the first electronic document features of the second electronic document. In this example, the frequency of a feature may be used to weight a ppmi value to compute the occurrence of a feature (e.g., featurei) in an electronic document. In some cases, using the frequency weighting improves performance as it reduces noise for infrequent features and amplifies the impact of frequent features.

In another example, the feature fusion engine may compute a feature using a Term-Frequency Inverse-Document-Frequency (TF-IDF) on the features of the first electronic document. An example of the computation of the fusion score using TF-IDF is provided below.

In yet another example, the feature fusion engine may compute a similarity score using a cosine similarity score to compute the similarities of the tree structure and text fields and an edit distance measurement to determine a similarity of the API endpoints of the electronic document. An example of the cosine similarity score is a measure of similarity between two non-zero vectors of an inner product space. The edit distance may be used to compare text or other strings to determine a number of operations required to transform a first text or string to a second text or string. The similarity score computation may be represented by the equation below. In the example below, api may represent an electronic document or other electronic file, tree may represent a vectorized and transformed textual description (e.g., a ppmi vector or a TF-IDF vector), text may represent a tokenized, vectorized, and transformed textual description, and endpoint may represent dictionary keys or xml tags as a string representation. Further in the example below, a first cosine similarity may compute the similarity of the tree path structure on a per-node basis, while a second cosine similarly may compute the similarity of the tree path structure at a level of the complete tree path structure.

The feature fusion engine may aggregate the similarity scores into the fusion score. In one example, the feature fusion engine performs a weighted linear combination of the similarity scores by weighting the similarity scores with the feature weight determined by the machine learning model. An example calculation of the fusion score is provided below.
sionScore(apii,apij)=Σ(λxsimilarity(apii,apij))+(λymean(validatorScoreapij))

FIG.3depicts an example of a process of extracting features from an electronic document, according to aspects of the present disclosure. For instance, the feature extraction module may extract tree paths300of the electronic documents. The tree paths300may include various nodes based on the structural hierarchy of the electronic document. An example of the tree paths300may be JSON, XML, HTML, or the like. The feature extraction module may extract the node relationships of the tree structure, the types of nodes, the number of nodes, the depth of the tree, or details of the nodes.

FIG.4depicts an example of a process for generating a fusion score between electronic documents, according to aspects of the present disclosure. For instance, the feature fusion engine may generate a fusion score that represents a similarity between electronic documents by combining multiple similarity scores of various features of the electronic documents. An example of a process for computation of the fusion score is described with one or more blocks ofFIG.2. The feature fusion engine may compute the fusion score between electronic documents, such as between stored documents, or a first electronic document that has been stored and a second electronic document that is received or selected subsequent to the first electronic document.

FIG.5depicts an example of a computing system for implementing certain aspects of the present disclosure. For example,FIG.5depicts an example of computing system500. The implementation of computing system500could be used for one or more of document management system101. The depicted example of the computing system500includes a processor503coupled to a memory506. The processor503executes computer-executable program code stored in memory506, such as software programs515. The processor503and the memory506may be coupled by a bus509. In some examples, the bus509may also be coupled to one or more network interface devices (not shown).

It should be recognized that the embodiments and examples disclosed herein are not exclusive to each other; features and elements from one embodiment may be used with others as is necessary and/or desired.

The processing machine used to implement the invention may utilize a suitable operating system. Thus, embodiments of the invention may include a processing machine running the iOS operating system, the OS X operating system, the Android operating system, the Microsoft Windows™ operating systems, the Unix operating system, the Linux operating system, the Xenix operating system, the IBM AIX™ operating system, the Hewlett-Packard UX™ operating system, the Novell Netware™ operating system, the Sun Microsystems Solaris™ operating system, the OS/2™ operating system, the BeOS™ operating system, the Macintosh operating system, the Apache operating system, an OpenStep™ operating system or another operating system or platform.