Source: http://www.google.com/patents/US20060041597?dq=4484186
Timestamp: 2017-05-24 06:54:24
Document Index: 702411738

Matched Legal Cases: ['art 210', 'art 210', 'arts 210', 'art 250', 'art 250', 'art 250', 'art 250', 'art 250', 'art 210']

Patent US20060041597 - Information retrieval systems with duplicate document detection and ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsMany companies provide online search facilities that enable users to conduct computerized searches for documents. Unfortunately, these searches frequently provide results that include duplicate documents—that is, documents that are completely or substantially identical to each other. This problem is...http://www.google.com/patents/US20060041597?utm_source=gb-gplus-sharePatent US20060041597 - Information retrieval systems with duplicate document detection and presentation functionsAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS20060041597 A1Publication typeApplicationApplication numberUS 11/122,577Publication dateFeb 23, 2006Filing dateMay 5, 2005Priority dateAug 23, 2004Also published asCA2578157A1, CA2578157C, CN101076800A, CN101076800B, EP1805661A1, US7809695, WO2006023941A1Publication number11122577, 122577, US 2006/0041597 A1, US 2006/041597 A1, US 20060041597 A1, US 20060041597A1, US 2006041597 A1, US 2006041597A1, US-A1-20060041597, US-A1-2006041597, US2006/0041597A1, US2006/041597A1, US20060041597 A1, US20060041597A1, US2006041597 A1, US2006041597A1InventorsJack Conrad, Joanne Claussen, Jie LinOriginal AssigneeWest Services, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (17), Referenced by (91), Classifications (9), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetInformation retrieval systems with duplicate document detection and presentation functions
1. An information-retrieval system comprising: a plurality of databases; and one or more servers for facilitating client access to the plurality of databases over a network, with each of the servers including at least one of: signature-generation means for generating a plurality of document signatures, with each document signature based on a plurality of features from a corresponding document in one or more of the databases; query-definition means for defining a query and selecting an option related to identification of search-result documents that include content duplicative of one or more other search-result documents; duplicate-determination means for determining, based on a subset of the document signatures, whether one or more documents within results of the query include content duplicative of content in one or more other documents within the results; means for controlling display of results of the query based on the selected option, with at least one of the displayed results indicated as including content duplicative of content in one or more other documents within the results; and means for controlling output of results of the query to a printer or email transmission device, based on user selected options related to output of documents that include content duplicative of content of one or more other documents within the results. 2. The system of claim 1, wherein the signature-generation means comprises means for determining at least two of a temporal, a length, and a content component for each document. 3. The system of claim 1, wherein each means comprises one or more sets of machine-executable instructions. 4. The system of claim 1, wherein the query-definition means for defining a query provides an option to define the query using Boolean or natural language. 5. An information-retrieval system comprising: a plurality of databases; and a server for providing users access to one or more of the databases, the server including: query-definition means for defining a query and selecting an option related to identification of documents within results of the query that include content duplicative of content within one or more other documents within the results; duplicate-determination means for determining whether one or more documents within the results of the query include content duplicative of content within one or more other documents within the results; means for controlling display of results of the query based on the selected option, with at least one of the displayed results indicated as including content duplicative of content in one or more other documents within the results; and means for controlling output of results of the query to a printer or email transmission device, based on user selected options related to output of documents that include content duplicative of content of one or more other documents within the results. 6. The system of claim 5, wherein the duplicate-determination means comprises means for comparing on a real-time basis a temporal, a length, and a content component for at least one document within the results of the query to a temporal, length, and a content component for at least one other document with the results. 7. The system of claim 5, wherein each means comprises one or more sets of machine-executable instructions. 8. The system of claim 5, wherein the query-definition means for defining a query provides an option to define the query using Boolean or natural language. 9. A method comprising: comparing first and second lengths of respective first and second documents; comparing first and second content sets for the respective first and second documents; and determining whether the first and second documents are duplicates based on results of comparing the first and second lengths or results of comparing the first and second content sets. 10. The method of claim 9, wherein determining whether the first and second documents are duplicates is based on results of comparing the first and second lengths and results of comparing the first and second content sets. 11. The method of claim 9, further comprising retrieving first and second document signatures corresponding to the respective first and second documents, with the signatures having respective first and second length components and respective first and second content components. 12. The method of claim 9, further comprising comparing first and second document signatures to each other, with the first and second document signatures associated respectively with the first and second documents, wherein the determination of whether the first and second documents are duplicates is contingent on comparison of the first and second document signatures. 13. The method of claim 12, wherein each document signature comprises a hash value and wherein the comparison of the first and second signature is affirmative if and only if the first and second hash values are equal. 14. The method of claim 13, further comprising: identifying a set of features from each document, based on their corresponding inverse-document-frequency (idf) values; determining positions of the features in each document; rounding the determined positions of each feature; concatenating the set of features and their rounded determined positions to define a string; and hashing the string to define the hash value. 15. The method of claim 12, wherein each document signature comprises a feature vector and wherein the comparison of the first and second signatures is affirmative if and only if the feature vector of the first signature has at least a threshold number of features in common with the feature vector of the second signature. 16. The method of claim 15, wherein the threshold number is at least as great as 80 percent of the number of features in each term vector. 17. The method of claim 15, wherein each feature vector comprises a set of two or more top inverse-document-frequency terms from its corresponding document. 18. The method of claim 15, wherein the threshold number is variable. 19. The method of claim 9, wherein the first and second documents are identified in response to a user defining a query for automatic future execution and in response to an automatic execution of the user defined query. 20. A method of identifying whether first and second documents are likely to contain duplicate content, the method comprising: determining whether the first and second documents have corresponding temporal traits that are within a first range of each other; determining whether the first and second documents have corresponding length traits that are within a second range of each other; determining whether the first and second documents have a significant number of features in common with each other; and identifying the first and second documents as duplicates in response to determining that the temporal traits of the first and second documents are within the first range of each other, that the lengths of the first and second documents are within the second range of each other, and that the first and second documents have at least a significant number of features in common with each other. 21. The method of claim 20, wherein determining whether the first and second documents have a significant number of features in common with each other occurs in response to an affirmative determination that the temporal traits of the first and second documents are within the first range of each other and an affirmative determination that the length traits of the first and second documents are within the second range of each other. 22. The method of claim 20, wherein the first range is predetermined to be 30 days; the second range is predetermined to be ±20%; and the significant number of features is predetermined to be at least 80 percent of a number of representative terms in each document, with the terms selected based on corresponding inverse-document-frequency values. 23. The method of claim 20, wherein the first and second documents have respective first and second document signature data structures, with each document signature data structure including: a temporal component based on a date of publication associated with the document; a length component based on a word count associated with the document; and a term vector based on a fixed number of (top-ranked) inverse-document frequency terms for the document. 24. The method of claim 23, wherein the term vector includes at least 10 terms. 25. A method comprising: determining whether first and second documents have at least a threshold number of selected features in common with each other, with each feature selected based on a corresponding inverse-document-frequency (idf) value; and identifying the first and second documents as duplicates in response to the determination being affirmative. 26. The method of claim 25, wherein determining whether the first and second documents have at least a significant number of features in common with each other, comprises: defining respective first and second term vectors for the first and second documents, with each term vector including a plurality of equal-length binary representations of idf terms for its respective document; and comparing a number of the binary representations of the first term vector to a number of binary representations of the second term vector. 27. The method of claim 25, further comprising determining whether the first and second documents are sufficiently similar in length, wherein the first and second documents are identified as duplicates in response to determining that the first and second documents have at least the threshold number of features in common with each other and determining that the first and second documents are sufficiently similar in length. 28. A method comprising: receiving a user query; identifying at least first and second documents from a database in response to the user query, with the first and second documents associated with respective first and second feature vectors, each feature vector having a plurality of equal-length binary representations of terms or features within its respective document, with the terms or features selected based on relative magnitude of corresponding inverse-document-frequency (idf) values within a table of inverse-document-frequency values; and in response to identifying the first and second documents, comparing the first and second feature vectors. 29. The method of claim 28, further comprising: presenting search results identifying at least one of the first and second documents in response to the user query, with the presented listing including an indication, based on the comparison of the first and second feature vectors, of whether the first and second documents contain content that is duplicative of each other. 30. The method of claim 28, wherein presenting search results identifying at least one of the first and second documents comprises presenting a listing of a title of the first document and wherein the indication of whether the first and second documents contain content that is duplicative of each other, includes: presenting a listing of a title of the second document that is below and indented relative to the title of the first document; or presenting a listing of a title of the second document in a font that differs from that of the title of the first document; or presenting a folder or other container icon which is selectable to display a listing of one or more documents that identifies the second document. 31. The method of claim 29, wherein comparing the first and second feature vectors occurs in response to an affirmative determination that temporal traits of the first and second documents are within a first range of each other and an affirmative determination that length traits of the first and second documents are within a second range of each other. 32. A graphical user interface comprising: one or more interactive control features for facilitating user definition of a query; and at least one interactive control feature for indicating whether search results provided in response to the query are to identify documents determined to have substantial duplicate content. 33. The graphical user interface of claim 32, wherein the one interactive control feature comprises a check box. 34. The graphical user interface of claim 32, further comprising at least one interactive control feature for controlling output of documents determined to have substantial duplicate content with other documents in the search results. 35. The graphical user interface of claim 32, further comprising a query-results region including: one or more interactive control features for invoking display of corresponding search-result documents; and at least one interactive control feature for identifying and invoking display of a search-result document deemed to be a duplicate of at least another of the search-result documents. 36. A graphical user interface comprising: one or more interactive control features for submitting a query; and a query results region for displaying search results based on the query, the region including at least one interactive control feature for identifying and invoking display or retrieval of a corresponding search-result document and at least one duplicate-indication feature for indicating whether the search results include any documents that are deemed to be duplicative of the corresponding search-result document. 37. The graphical user interface of claim 36, wherein the one duplicate-indication feature comprises a selectable link that is positioned below and indented relative to the one interactive control feature for the corresponding search-result document. 38. The graphical interface of claim 36, further comprising at least one interactive control feature for defining a default setting of whether search results presented in response to a query are to identify documents determined to have substantial duplicate content. 39. The graphical user interface of claim 38, wherein the one interactive control feature comprises a check box.
COPYRIGHT NOTICE AND PERMISSION [0002] A portion of this patent document contains material subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyrights whatsoever. The following notice applies to this document: Copyright©2004, West Services, Inc. TECHNICAL FIELD [0003] Various embodiments of the present invention concern information-retrieval systems, such as those that provide news documents or other related content. BACKGROUND [0004] Companies, such as Thomson Legal & Regulatory, Inc. of St. Paul, Minn. (doing business as Thomson West), collect and store a vast spectrum of documents, including news, from all over the world, for online access in a system of databases and research tools, known as the Westlaw™ system. The Westlaw system empowers users to search over 100 million documents. [0005] One problem recognized by the present inventors is that searches conducted against news or other databases frequently provide results that include duplicate documents—that is, documents that are completely or substantially identical to each other. The problem stems from news providers, such as Associated Press (AP), selling their news stories for re-publication to multiple publishers around the world. This in turn means that systems, such as the Westlaw system, that provide users searchable access to collections of news stories from a wide array of publishers typically present users with many duplicate copies of news stories in their search results. Unfortunately, the duplicate stories are mixed generally according to relevance with other distinct stories, leaving users to manually manage the complexities of identifying and/or filtering them. [0006] Accordingly, the present inventors recognized a need to effectively address how information-retrieval systems, such as the Westlaw system, handle the existence of duplicate documents in their document collections, and more importantly within the search results of their users. SUMMARY [0007] To address this and other needs, the present inventors devised systems, methods, and software that facilitate the identification and/or grouping of duplicate documents in search results. One exemplary system includes three major components: 1) a signature generation module which generates document signatures based on length, temporal, and/or content components; 2) a duplicate detection module which uses the document signatures to identify “exact” or “fuzzy” duplicate documents; and 3) a user-interface (or presentation) module which allows users to control how duplicate documents are presented in their search results. For instance, users can select whether to exclude duplicates from their search results or to group duplicates together in the results presentation. In some embodiments, the identification and grouping ultimately streamline the process of users interpreting and accessing search results that contain duplicate documents. BRIEF DESCRIPTION OF DRAWINGS [0008] FIG. 1 is a diagram of an exemplary information-retrieval system 100 corresponding to one or more embodiments of the invention; [0009] FIG. 2 is a flowchart corresponding to one or more exemplary methods of operating system 100 and one or more embodiments of the invention; [0010] FIG. 3 is a diagram of an exemplary document signature and IDF table corresponding to one or more embodiments of the invention; [0011] FIG. 4 is a facsimile of an exemplary query window 400 corresponding to one or more embodiments of the invention; [0012] FIG. 5 is a facsimile of an exemplary user interface 500 corresponding to one or more embodiments of the invention; [0013] FIG. 6 is a facsimile of an exemplary user interface 600 corresponding to one or more embodiments of the invention; [0014] FIG. 7 is a a facsimile of an exemplary HTML-formatted electronic message corresponding to one or more embodiments of the invention; [0015] FIG. 8 is a facsimile of an exemplary user interface 800 corresponding to one or more embodiments of the invention; and [0016] FIG. 9 is a diagram of an exemplary binning scheme that corresponds to one or more embodiments of the invention.
wherein the terms are ranked and presented in descending order of idf value and the position, shown in brackets, is measured in tokens relative to a first token in the document. [0049] Some embodiments measure position of each idf term relative to the position of the preceding highest-ranked idf term. And, some embodiments, which provide a relaxed or more tolerant definition of duplicate documents, round the relative positional offsets into bins. For instance, one embodiment rounds the positional offset for each corresponding top idf term into the closest of a series of ten-token “bins,” with the number of bins determined by the size of the document in tokens divided by ten. This positional binning is effective in allowing this embodiment to handle cases where a document has been subjected to insertions or short substitutions and would still generally be regarded as a duplicate of another different, but otherwise identical document. The table below shows a sample set of six idf terms presented in descending rank order along with their original (or absolute) position within the document in the second column, their relative positions in the third column, and their binned (or rounded) relative positions in the fourth column. idf term Original position relative (descending document to preceding “binned” rank order) position ranked idf term relative position irate 11 11 − 0 = 11 20 flabbergasted 35 35 − 11 = 24 30 dishonorable 62 62 − 35 = 27 30 disgraceful 67 67 − 62 = 5 10 outrageous 80 80 − 67 = 13 20 ignoble 120 120 − 80 = 40 40 Note that all rounding used in the positional binning is upward, and that differences ending in 0 are left untouched (for example, see binned relative position for ignoble in the table.) Also note that in this embodiment the terms present in the title of the document (and in any of the accompanying headings and subheadings) do not participate (that is, are not counted in) the generation of the offsets within the document. After defining the fingerprint, execution advances to block 214A. [0050] Block 214A entails determining a hash value (or other unique value) based on the fingerprint. Specifically, the exemplary embodiment concatenates the top idf-terms and positional information into a single string, such as “irate20flabbergasted30dishonorable30disgraceful10outrageous20ignoble40,” and then hashes the resulting string according to the algorithm to determine the hash value. In the exemplary embodiment, this entails hashing the vector into a 20-byte key using National Institute of Standards and Technology's SHA1 hashing algorithm. Some embodiments may use other methods of determining a hash value. [0051] Block 215A entails forming or defining a document signature (that is, data structure) based on one or more of the length values and the determined fingerprint number (such as hash value based on content.) In the exemplary embodiment, this entails aggregating the scalar length value and the fingerprint number into a data structure that is logically associated with the corresponding document, for example using a document identifier or pointer. [0052] Block 216A entails storing the document signature data structure in a memory device. To this end, the exemplary embodiment stores the data structure in an index or metadata database, such as document signature database 125 in FIG. 1. Fuzzy Signature Generation [0053] Flow chart 210B—which shows generation of a document signature data structure (or characteristic feature set) based on document temporal, length, and content components—includes process blocks 211B-215B. [0054] Specifically, block 211B entails determining one or more temporal components or values for the document. In the exemplary embodiment, this determination entails extracting a publication date or time stamp from the dateline of a document, and then converting the date or time stamp to a simple integer representing the number of hours, days, weeks, or months relative to a reference date, such as Jan. 1, 1950. Other documents may use other dates associated with or contained in the document, such as a first or last occurring date in the document or a portion of the first or last occurring date. For example, some embodiments may extract and use the first occurring year in the document as a basis for the temporal component. Still others may determine a temporal component based on multiple dates within the document, for example an average or aggregate of two or more dates. Execution continues at block 212B. [0055] Block 212B determines one or more length components or values for the document. In the exemplary embodiment the length value is based on length of the document, and is determined by extracting a document-length indicator from a predefined word-count field associated with the document. However, other embodiments independently determine a word count and use this as the length value. Execution proceeds to block 213B. [0056] Block 213B entails determining one or more content values or features for the document. In the exemplary embodiment, this entails identifying one or more lexical features for the document and forming a ‘fingerprint’ or term vector. The fingerprint generally includes the top Y, for example 60, highest-ranking idf terms for the document (excluding title, author, and other header information or metadata). [0057] More specifically, the exemplary embodiment tokenizes and parses a document into terms, and then sorts these terms by their associated idfs. Any terms that are not in the idf table plus any terms included on a list of stop words are excluded from the term (or feature) vector. If the number of idf terms y in a document is less than 10, no signature is created for the document. However, if y is between 10 and Y-1 inclusive, the exemplary embodiment pads the term vector with additional terms to ensure inclusion of Y terms. [0058] Exemplary padding proceeds as follows. If the number of idf terms y for a document falls between 30 and 59 inclusive, the exemplary embodiment pads by adding up to 30 different alphanumeric terms to the vector, such as ‘pad1’, ‘pad2’, ‘pad3’, . . . , ‘padn’, where n=Y−y. If y is between 10 and 29 inclusive, the exemplary embodiment pads the term vector with some combination of n predetermined non-idf terms and m randomly selected non-idf terms, such that y+n+m=Y. The n predetermined non-idf-tabled terms, in some embodiments, are (as above) alphanumeric terms, such as ‘pad1’, ‘pad2’, . . . , ‘padn’, which have a common text portion and a sequenced numeric portion. The m randomly generated non-idf-tabled terms, in some embodiments, are alphanumeric terms, R1, R2, Rm, which represent terms that are not matchable with existing idf terms. The table below illustrates the padding scheme used in the exemplary embodiment. # of non- matchable # of idf terms, y # of non-random pad terms, n terms, m y >= 60 0 0 30 <= y <= 59 pad1, pad2, . . . , padn, n = Y − y 0 28 or 29 pad1, pad2, . . . , padn, n = Y − y − 1 1 26 or 27 pad1, pad2, . . . , padn, n = Y − y − 2 2 24 or 25 pad1, pad2, . . . , padn, n = Y − y − 3 3 22 or 23 pad1, pad2, . . . , padn, n = Y − y − 4 4 20 or 21 pad1, pad2, . . . , padn, n = Y − y − 5 5 18 or 19 pad1, pad2, . . . , padn, n = Y − y − 6 6 16 or 17 pad1, pad2, . . . , padn, n = Y − y − 7 7 14 or 15 pad1, pad2, . . . , padn, n = Y − y − 8 8 12 or 13 pad1, pad2, . . . , padn, n = Y − y − 9 9 10 or 11 pad1, pad2, . . . , padn, n = Y − y − 10 10 1 <= y <= 9 no signature created no signature [0059] Next, block 214B encodes the term vector. To this end, the exemplary embodiment encodes each term separately, with the encoding based on position of the term within the ranked idf table. Specifically, the exemplary embodiment encodes each term vector token as a unique three-byte (24 bit) serial number or index that not only uniquely corresponds with the token in the idf table (which includes about one million entries), but also (in the exemplary embodiment) indicates the rank of the term in the idf table. (The exemplary embodiment organizes the terms in the vector from highest idf value to lowest to accelerate mismatch count and thus reduce computation time.) After encoding the term vector, execution advances to block 215B. [0060] Block 215B entails storing the document signature data structure in a memory device. To this end, the exemplary embodiment stores the signature in a metadata database, such as signature database 125 (in FIG. 1). (Some embodiments append the document signature data to the document.) [0061] Some embodiments address issues surrounding the maintenance of collection statistics and idf table updates. For example, one embodiment, which creates signatures according to flow chart 210A and/or 210B, recognizes the sensitivity of these document signatures to updates of the idf table that typically occur with addition, deletion, or correction of documents in a given collection or database. This embodiment provides a pair of signatures for those documents published close to an idf-table update date (for example, within a two-month window centered on the update date.) One signature, a pre-update signature, is based on the idf table prior to update, and the other, a post-update signature, is based on the idf table after update. (In use, for example at block 250, if at least one of the two signatures matches at least one of the signatures for another document, the two documents are regarded as duplicates.) [0062] FIG. 3 shows how a document signature 300 is related to an idf table 340 via its content component 310. Specifically, document signature 300 includes a length component 310, a temporal component 320, and a content component 330. Content component 330 takes the exemplary form of a 60-term vector 330′, which includes terms T0-T59. As shown in FIG. 3 one or more of the terms, for example all of the terms, map to terms in idf table 340, which has one million terms and corresponding idf values. [0063] After generating and storing document signatures for all the documents of the select databases according to one or both of the methods shown in flow charts 210A and 210B, exemplary execution eventually proceeds to block 220. [0064] Block 220 entails presenting a search interface to a user. In the exemplary embodiment, this entails a user directing a browser in a client access device to an internet-protocol (IP) address for an online information-retrieval system, such as the Westlaw system, and then logging onto the system. Successful login results in a web-based search interface, such as interface 138 in FIG. 1 or interface 300 in FIG. 4 (or one or more portions thereof) being output from server 120, stored in memory 132, and displayed by client access device 130. [0065] As FIG. 4 shows, interface 400 includes a number of interactive control features, including a query input region 410, a query field restriction region 420, a duplicate directive region 430, and a query submit command 440. Query input region 410 receives textual input defining a query. Query targeting region 420 allows the user to target the query to a specific subsection of documents, the headlines and lead paragraphs, in a fielded database. (Other embodiments may have one or more other selectable subsections.) Duplicate-directive region 430 allows the user to specifically enable identification of duplicate documents within search results for the query being defined. The initial state of this directive region is determined by a default user preference value stored in a subscriber database, such as database 124. Changing the state of the directive region, in the exemplary embodiment, changes the directive for the current query; the default preference value is unaffected unless changed at a higher control level. [0066] Using interface 138 or 400, the user can define or submit a query and cause it to be output to a server, such as server 120. In other embodiments, a query may have been defined or selected by a user to automatically execute on a scheduled or event-driven basis. In these cases, the query may already reside in the memory of a server for the information-retrieval system (such as done for clipping services), and thus need not be communicated to the server repeatedly. Execution then advances to block 230 (in FIG. 2.) [0067] Block 230 entails receiving a query. In the exemplary embodiment, the query includes a query string and/or a set of target databases, which includes one or more of the select databases. In some embodiments, the query string includes a set of terms and/or connectors, and in other embodiment includes a natural-language string. Also, in some embodiments, the set of target databases is defined automatically or by default based on the form of the system or search interface. Also in some embodiments, the received query may be accompanied by other information, such as information defining whether to check for duplicate documents as discussed above. In any case, execution continues at block 240. [0068] Block 240 entails identifying a set of documents or search results based on or in response to the received query. In the exemplary embodiment, this entails the server or components under server control or command, executing the query against the targeted set of databases and identifying documents that satisfy the query criteria. Execution proceeds to block 250. [0069] Block 250 entails identifying sets of duplicate documents in the search results. (In some embodiments, execution of the duplicate identification block is contingent on a default or selected user option that specifies whether to identify duplicate documents. Some embodiments allow users or administrators to select which of two or more duplicate-detection techniques or algorithms to use.) In the exemplary embodiment, duplicate identification generally entails comparing one or more aspects of one or more document signatures to corresponding aspects of other document signatures and determining whether documents are duplicates based on “exact” or “fuzzy” (less exact) standard of what constitutes a duplicate document. [0070] More specifically, the exemplary embodiment follows the method shown in flow chart 250A, which includes process blocks 251A-254A for the exact or more exact detection algorithm, or the method shown in flow chart 250B, which includes process blocks 251B-255B for a “fuzzy” detection algorithm. Some embodiments which use both types of signatures described above use both corresponding methods. Exact Duplicate Detection [0071] In flow chart 250A, the exemplary method begins at block 251A, which entails selecting two or more documents of the search results for comparison. In the exemplary embodiment, this entails retrieving document signature data structures for each document in the search results based on their document identifiers and defining a number of document pairs for real-time duplicate detection or comparison. Defining the document pairs entails selecting a primary document and pairing the primary document (or more precisely its document signature) with each of the other documents in the search results, then selecting a second primary document and pairing it with all other documents with which it has not already been paired. Similarly, each document can be selected as a primary document and paired with all other documents with which it has not already been paired, ultimately defining a complete set of unique document pairings for comparison. (In some embodiments, the primary documents are selected in order of their relevance ranking within the search results. Also, some embodiments restrict application of the duplicate detection process to documents that exceed a certain relevance threshold or that have a certain minimum ranking.) Execution then advances to block 252A. [0072] Block 252A entails determining whether the length criteria for the set of documents selected for comparison are satisfied. In the exemplary embodiment, this entails determining whether the length scalars of a selected pair of document signature data structures are within a predetermined range, such as ±40 tokens or ±10% of each other. The fixed or relative range allows for potential differences in documents that are near the header material, for example, Dateline: Amsterdam. If the determination is that the length criteria (or condition(s)) are not satisfied, execution returns to block 251A for selection of another set of documents for comparison. However, if the length criteria are satisfied, execution advances to block 253A. [0073] Block 253A determines whether content criteria for the selected documents are satisfied. In the exemplary embodiment, this entails comparing the document fingerprints of the selected documents to each other. If the two fingerprints are not identical, execution returns to block 251A for selection of another set of documents for comparison. If the fingerprints are identical, execution advances to block 254A. [0074] Block 254A entails marking the selected documents as duplicates of each other. In the exemplary embodiment, this marking entails storing the document identifiers for the documents deemed to be duplicates in a duplicate-set buffer. (In other embodiments, the marking includes adding the document identifiers for the selected documents to a master duplicate document database or to the respective document signature data structures for the selected documents, which may be reused for special duplicate-document queries.) [0075] Execution then returns to block 251A for the selection of the next set of documents for comparison. In the exemplary embodiment, block 251A includes logic for terminating the comparison process after all selected sets of documents are processed. Fuzzy Duplicate Detection [0076] Flow chart 250B depicts an alternative detection process, which generally entails a real-time multilevel processing of the signature data structures for the documents identified in the search results. (Some embodiments may perform duplicate detection prior to, rather than in response to a user query.) Flow chart 250B includes process blocks 251B-255B. [0077] At block 251B, the process ensues with retrieval of at least two document signature data structures for documents identified in the search results. In the exemplary embodiment, this entails retrieving signature data structures from signature database 125 (shown in FIG. 1) for each document in the search results in a manner similar to that described for block 251A. Once the signatures are retrieved, a set of two or more document signatures are selected for use in determining whether their corresponding documents are duplicates. Execution then continues at block 252B. [0078] Block 252B determines whether the temporal criteria for components associated with the two or more document signatures are within a certain time period of each other. In the exemplary embodiment, this entails determining whether the temporal components of the selected document signatures are within 30 days of each other. (Some embodiments use smaller or larger temporal windows.) A negative determination results in the documents being deemed as non-duplicates and execution returning to block 251B to get another set of document signatures for comparison, whereas a positive determination extends processing to block 253B. [0079] Block 253B determines whether the length components of the signature data structures for one or more corresponding documents are within some range of each other. In the exemplary embodiment, this entails determining in real-time whether the length components are within ±20% of each other. If the length determination is negative, the documents are considered as non-duplicates and execution returns to block 251B to select another set of document signatures for comparison. However, if the determination is positive, indicating that the document lengths are sufficiently close, execution continues at block 254B. [0080] Block 254B entails determining whether the set of document signatures satisfy the fingerprint or content criteria for duplicate documents. In the exemplary embodiment, this entails determining whether at least 80% of the terms in the term vector for one document are contained within the term vector for the other document. Some embodiments may use alternative thresholds, for example, 60, 65, 70, 75, 80, 85, 90, or 95%. Other embodiments may use dynamic thresholds that vary based on type of documents or database; indeed some embodiments allow users to select the threshold. [0081] More particularly, in determining whether the content criteria are met, the exemplary embodiment performs a real-time term-by-term comparison, incrementing a mismatch counter for every term mismatch and a match counter for every match. The term-by-term comparison is terminated when the mismatch counter exceeds a non-duplicate threshold, such as 12, which indicates that the content criteria cannot be satisfied (since, for example, more than 20% of the terms have not matched). Alternatively, the term-by-term comparison could terminate when the match counter meets a duplicate threshold, such as 49, which indicates that more than 80% of the terms have matched. In any event, if the content criteria are not satisfied, execution returns to block 251B. And, if the content criteria is satisfied, execution proceeds to block 255B. [0082] Block 255B entails marking the documents in the document sets as duplicates. In the exemplary embodiment, this marking entails storing the document identifiers for the documents deemed to be duplicates in a duplicate-set buffer. After identifying duplicates within the search results, execution proceeds to block 260. [0083] Block 260 entails presenting search results to a user. In the exemplary embodiment, this entails outputting the search results in the form of a ranked list to a client access device, such as client access device 130 in FIG. 1. Specifically, the exemplary embodiment provides the list within the context of a graphical user-interface, as exemplified by interface 138, specifically region 1382 in FIG. 1 or alternatively by interface 500 in FIG. 5. [0084] Interface 500 shows a results list 510 of selectable document citations or hyperlinks and window 520 for displaying text of at least one of the documents in results list 510. Results list 510 includes one or more sets or groupings of duplicate document identifiers or citations, of which a duplicate set 512 is representative. In the exemplary embodiment, the position of set 512 within results list 510 is determined based on reverse chronological listing, relevance rank, or score of the highest ranked document in or associated with the set. For example, if the original set includes documents 3 and 5 (in rank order) that are duplicates and document 5 is the primary document, the entire duplicate set is displayed in the number 3 position in the cite list. Set 512 includes a primary document identifier 5121, duplicate count indicator 5122, and a duplicate identifier 5123. [0085] Primary document identifier 5121 identifies a primary document, which in the exemplary embodiment is defined as the longest of the documents in the duplicate set. If two or more duplicate documents are the same length (that is, have the same word count), then the more recent document is listed as the primary document. In some embodiments, the primary document identifier corresponds to the document having the highest relevance score or the most recent publication date. [0086] Duplicate count indicator 5122 indicates the number of documents in the search results that are considered to be duplicates of the primary document—that is, to include substantial amounts of content that are duplicative of content of the primary document. In the example shown, the primary document is indicated to have one duplicate in the results list. [0087] Duplicate listing 5123 lists one or more selectable document citations or identifiers, such as document citation or identifier 5123A, each associated with a document (or document URL) deemed relevant to the search query and considered to include a substantial amount of content that is duplicative of content in the primary document. (In the exemplary embodiment, duplicates are determined in accord with the techniques and principles presented herein; however, other embodiments may use alternative techniques and/or principles.) In the duplicate listing, the order of listing multiple duplicates can be determined by length, publication date, document relevance, or access frequency. In some embodiments, the labels for these links contain the date of the document, publication, and word count. [0088] Rather than presenting the duplicate listing in the results lists, some embodiments display a container icon, such as a duplicates folder, in association with the primary document identifier. The duplicates folder is a user-selectable icon that when selected opens a window to display a listing of the duplicate documents (or corresponding URLs) along with bibliographic information. [0089] From presentation of search results at block 260, execution advances to block 270, which entails outputting one or more select documents from the search results. In the exemplary embodiment, this output entails printing, emailing, or saving one or more of the identified documents to memory associated with a client access device, such as access device 130, in response to a user selection. To facilitate user control or direction of this output, the exemplary embodiment presents the user a graphical user interface, such as interface 600 in FIG. 6, which is incorporated and accessible via interface 138 in FIG. 1. [0090] Interface 600 includes an output-destination region 610, a range-definition region 620, a content-definition region 630, and a request-submission region 640. Output-destination region 610 allows a user to specify printer, email, or memory destinations for one or more portions of the search results. (See FIG. 7 and supporting text for sample email message.) Range-definition region 620 allows a user to identify the documents within the search results that are to be output to the destination(s) identified in region 610. [0091] Content-definition region 630, which allows the user to identify specific portions of documents selected in range-definition region 620 portion for output, includes, among other things, a duplicate-output control feature 631. This feature allows a user to indicate and control whether to include any documents that are duplicates of the identified documents as part of the output. If the user has not modified her corresponding preference, such as preference setting 1241D in subscriber database 124 of FIG. 1, the default is to exclude duplicate documents from the output, meaning that the duplicate-output control region initially indicates that duplicates are to be excluded from the output. [0092] In the exemplary embodiment, selection or invocation of the include-duplicates feature not only causes output of duplicate documents for all options in the range definition region except for the selected documents or citations option, but also affects displays within interface 600. The “all documents” range option in region 620 includes a label indicating the number of primary documents in the search result, with the set of primary documents being the entire set of search results minus any duplicates. If the search results include 30 total documents and documents 1, 5 and 7 are duplicates, the label next to the “all documents” option in region 620 ordinarily would indicate “28 documents.” However, if the user invokes the duplicate-inclusion feature (manually or by default) the label would indicate both primary and duplicate documents, or pursuant to the example, “30 documents.”
[0093] FIG. 7 shows an exemplary HTML-formatted electronic message 700 which includes among other things a results list that identifies duplicate documents. Specifically, message 700, which can also be sent in response to automatic running of a query defined for repeated, periodic or event-driven execution, includes a header 710, a query region 720, and a result listing region 730. [0094] Header 710 includes from, sent, to, and subject regions, which respectively identify the sender, time-sent, recipient, and subject of the email. Query region 720 includes a database-identification field 722 and a query field 724. Database identification field 722 identifies the database(s) that were searched, and query field 724 lists text of a query. In the exemplary embodiment, the particular query takes the form of a Boolean or natural-language query. [0095] Results listing region 730 includes one or more document citations or hyperlinks, such as identifiers 732 and 734, which are selectable to invoke retrieval and display of all or a portion of each corresponding document. In some embodiments, selection of one of the hyperlinks immediately causes retrieval of the corresponding document in a browser window. In others the selection results in display of a sign-on screen in the browser window, prompting the user to enter appropriate login and/or client-matter-identification data prior to document display.) Associated with document identifier 732 is a set 7322 of one or more selectable duplicate-document identifiers or links. In the exemplary embodiment, labels for these links contain respective publication date and word count. Exemplary Options-Control Interface [0096] FIG. 8 shows an exemplary options-control interface 800, which functions as a portion of interface 138 in FIG. 1 and allows users to set values for preferences in subscriber database 123, such as those related to duplicate processing and/or presentation. In the exemplary embodiment, interface 800 includes an identify-duplicates control feature 810, duplicate-inclusion-or-exclusion control features 820, primary-duplicate-selection features 830, and a save-command feature 840. [0097] Identify-duplicates control feature 810, a check box in this embodiment, enables users to set the default for whether duplicate processing is performed on eligible search results. [0098] Duplicate-inclusion-or-exclusion control elements 820 include control features 821 and 822. In this embodiment, features 821 and 822 take the form of radio buttons and respectively allow users to select whether duplicates are to be included in the displayed results list or excluded from the displayed results list. [0099] Primary-duplicate-selection features 830 include features 831 and 832, which also take the form of radio buttons and which allow users to specify which document in a set of duplicate documents to display as a primary document in a results list. In this embodiment, the user can select to have the longest document or the most recent or most relevant document as the primary document. If duplicates are to be excluded from the search results, this option governs which document of a set of duplicates is displayed in the results list, whereas if documents are to be included within the list, this option governs which document is to be displayed first in the list. [0100] Save command feature 840 enables a user to save changes made via control features 810, 820, 830 to subscriber database 124 for use during the remainder of a current search session and during future search sessions. Use of Temporal and Length Bins [0101] Some embodiments integrate the temporal and length comparison of blocks 252B and 253B (in FIG. 2) by defining sets or bins of potentially duplicate documents. For example, some embodiments retrieve a set of corresponding signature data structures as defined in flow chart 210B from signature database 124 and sort them in reverse chronological order based on their respective temporal components. [0102] After this temporal sorting, these embodiments define one or more temporal sets or bins, by “moving” a fixed-temporal window down the sorted document list. The first temporal bin includes the first sorted document and all documents having a temporal value within a time period, such as 30 days, of the first sorted document. The second temporal bin includes the second sorted document and all documents having a temporal value within 30 days of the temporal value for the second sorted document. Additional bins are defined similarly, moving down the list of sorted documents. (Some embodiments define mutually exclusive sets or bins of documents.) [0103] Once these temporal sets or bins are defined, these embodiments define one or more length-based bins or sets of signatures within each of the defined temporal bins. This entails sorting the signatures in each of the temporal bins in descending order of the length components in their corresponding signatures and then “moving” a length window down the sorted list to define one or more length bins or sets. The first length bin within the first temporal bin includes the first signature in the length-ranked list in that bin (that is, the longest document in the temporal bin) and all documents within the first temporal bin that are no more than 20% shorter than the length of the longest document. [0104] In other words, each subsequent signature has a length value at least 80% of that of each other signature in the first length bin. The second length bin includes the second longest document in the first temporal bin and all documents that are no more than 20% shorter than it. Subsequent length bins are defined similarly until all the documents in each temporal bin are assigned to a length bin. Thus, two documents are members of the same length bin if their temporal features (for example, publication dates) are within the same 30-day window and the length of the shorter of the documents is no less than 80% the length of the longer document. Further length bins for the first temporal bin and any other temporal bin are defined similarly. [0105] To illustrate this binning, FIG. 9 shows a diagram 900. Diagram 900 includes a reverse chronologically sorted list 910 of document identifiers or document signatures D1-D20, which has been organized as three temporal bins or sets TB1, TB2, and TB3, with each bin identifying or corresponding to a set of documents that are published within 30 days of each other. Temporal bin TB1 includes document identifiers or document signatures D1-D7. Sorting the contents of temporal bin TB1 based on the length components of its document signatures yields a length sorted list 920. List 920 is shown organized as two length bins LB1 and LB2, with each bin identifying or corresponding to a set of documents that are no more than 20% shorter than the length of the longest document in the bin. Once all the length bins for each temporal bin are defined, all unique pairs of documents in each length bin are compared pursuant to the content-comparison process outlined for block 254B (in FIG. 2.) [0106] Some embodiments omit actual defining of the length bins, instead comparing the length of each document to each other document in the current temporal bin and performing the content comparison only for those pairs of documents that have lengths within ±20% of each other. In effect, these embodiments define length bins in a virtual manner. Conclusion [0107] In furtherance of the art, the present inventors have not only recognized a need to effectively address how information-retrieval systems handle the existence of duplicate documents in their document collections, but also presented herein systems, methods, and software that facilitate the identification and/or grouping of duplicate documents in search results, in accordance with user preferences. This identification and grouping ultimately streamline the process of users accessing and reviewing search results that contain duplicate documents. [0108] The embodiments described above are intended only to illustrate and teach one or more ways of making and using the present invention, not to restrict its breadth or scope. The actual scope of the invention, which embraces all ways of practicing or implementing the teachings of the invention, is defined only by one or more issued patent claims and their equivalents. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5488725 *Mar 30, 1993Jan 30, 1996West Publishing CompanySystem of document representation retrieval by successive iterated probability samplingUS5826260 *Dec 11, 1995Oct 20, 1998International Business Machines CorporationInformation retrieval system and method for displaying and ordering information based on query element contributionUS5913208 *Jul 9, 1996Jun 15, 1999International Business Machines CorporationIdentifying duplicate documents from search results without comparing document contentUS6138113 *Aug 10, 1998Oct 24, 2000Altavista CompanyMethod for identifying near duplicate pages in a hyperlinked databaseUS6654739 *Jan 31, 2000Nov 25, 2003International Business Machines CorporationLightweight document clusteringUS6658423 *Jan 24, 2001Dec 2, 2003Google, Inc.Detecting duplicate and near-duplicate filesUS6757675 *Feb 12, 2003Jun 29, 2004The Regents Of The University Of CaliforniaMethod and apparatus for indexing document content and content comparison with World Wide Web search serviceUS6785669 *Mar 8, 2000Aug 31, 2004International Business Machines CorporationMethods and apparatus for flexible indexing of text for use in similarity searchesUS6978419 *Nov 15, 2000Dec 20, 2005Justsystem CorporationMethod and apparatus for efficient identification of duplicate and near-duplicate documents and text spans using high-discriminability text fragmentsUS7013310 *Jan 3, 2002Mar 14, 2006Cashedge, Inc.Method and apparatus for retrieving and processing dataUS7139756 *Jan 22, 2002Nov 21, 2006International Business Machines CorporationSystem and method for detecting duplicate and similar documentsUS7264274 *Jun 22, 2005Sep 4, 2007Delphi Technologies, Inc.Tuneable energy absorbing mounting structure for steering columnUS20020161788 *Mar 19, 2001Oct 31, 2002Mcdonald David T.System and method for efficiently processing messages stored in multiple message storesUS20020174101 *Jul 9, 2001Nov 21, 2002Fernley Helen Elaine PenelopeDocument retrieval systemUS20030172063 *Mar 7, 2002Sep 11, 2003Koninklijke Philips Electronics N.V.Method and apparatus for providing search results in response to an information search requestUS20040093323 *Nov 7, 2002May 13, 2004Mark BluhmElectronic document repository management and access systemUS20050060643 *Aug 12, 2004Mar 17, 2005Miavia, Inc.Document similarity detection and classification system* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7516130 *Feb 24, 2006Apr 7, 2009Trend Micro, Inc.Matching engine with signature generationUS7617231 *Dec 6, 2006Nov 10, 2009Electronics And Telecommunications Research InstituteData hashing method, data processing method, and data processing system using similarity-based hashing algorithmUS7672971 *Feb 17, 2006Mar 2, 2010Google Inc.Modular architecture for entity normalizationUS7698317 *Apr 20, 2007Apr 13, 2010Yahoo! Inc.Techniques for detecting duplicate web pagesUS7720835 *May 7, 2007May 18, 2010Visible Technologies LlcSystems and methods for consumer-generated media reputation managementUS7756798 *Mar 6, 2007Jul 13, 2010Oracle International CorporationExtensible mechanism for detecting duplicate search itemsUS7818278 *Jun 14, 2007Oct 19, 2010Microsoft CorporationLarge scale item representation matchingUS7966291 *Jun 26, 2007Jun 21, 2011Google Inc.Fact-based object mergingUS7970766Jul 23, 2007Jun 28, 2011Google Inc.Entity type assignmentUS7991797Feb 17, 2006Aug 2, 2011Google Inc.ID persistence through normalizationUS8122026Oct 20, 2006Feb 21, 2012Google Inc.Finding and disambiguating references to entities on web pagesUS8180740 *Aug 12, 2009May 15, 2012Netapp, Inc.System and method for eliminating duplicate data by generating data fingerprints using adaptive fixed-length windowsUS8239350May 8, 2007Aug 7, 2012Google Inc.Date ambiguity resolutionUS8240554Mar 28, 2008Aug 14, 2012KeycorpSystem and method of financial instrument processing with duplicate item detectionUS8244689Feb 17, 2006Aug 14, 2012Google Inc.Attribute entropy as a signal in object normalizationUS8260785Feb 17, 2006Sep 4, 2012Google Inc.Automatic object reference identification and linking in a browseable fact repositoryUS8266115 *Jan 14, 2011Sep 11, 2012Google Inc.Identifying duplicate electronic content based on metadataUS8266228Dec 8, 2009Sep 11, 2012International Business Machines CorporationTagging communication files based on historical association of tagsUS8271493Oct 11, 2007Sep 18, 2012Oracle International CorporationExtensible mechanism for grouping search resultsUS8280861 *Sep 30, 2011Oct 2, 2012Google Inc.Identifying duplicate electronic content based on metadataUS8301637 *Jul 25, 2008Oct 30, 2012Seiko Epson CorporationFile search system, file search device and file search methodUS8347202Mar 14, 2007Jan 1, 2013Google Inc.Determining geographic locations for place names in a fact repositoryUS8442969 *Aug 14, 2008May 14, 2013John Nicholas GrossLocation based news and search engineUS8566369Oct 25, 2007Oct 22, 2013Inria Institut National De Recherche En Informatique Et En AutomatiqueComputer-based tool for managing digital documentsUS8589497Dec 8, 2009Nov 19, 2013International Business Machines CorporationApplying tags from communication files to usersUS8650175Jul 13, 2012Feb 11, 2014Google Inc.User interface for facts query engine with snippets from information sources that include query terms and answer termsUS8682891Sep 4, 2012Mar 25, 2014Google Inc.Automatic object reference identification and linking in a browseable fact repositoryUS8682913Mar 31, 2005Mar 25, 2014Google Inc.Corroborating facts extracted from multiple sourcesUS8700568Mar 31, 2006Apr 15, 2014Google Inc.Entity normalization via name normalizationUS8725830 *Jun 22, 2006May 13, 2014Linkedin CorporationAccepting third party content contributionsUS8738643Aug 2, 2007May 27, 2014Google Inc.Learning synonymous object names from anchor textsUS8751498Feb 1, 2012Jun 10, 2014Google Inc.Finding and disambiguating references to entities on web pagesUS8799236 *Jun 15, 2012Aug 5, 2014Amazon Technologies, Inc.Detecting duplicated content among digital itemsUS8812435Nov 16, 2007Aug 19, 2014Google Inc.Learning objects and facts from documentsUS8825471Mar 31, 2006Sep 2, 2014Google Inc.Unsupervised extraction of factsUS8825641Nov 9, 2010Sep 2, 2014Microsoft CorporationMeasuring duplication in search resultsUS8955120Sep 26, 2013Feb 10, 2015Kaspersky Lab ZaoFlexible fingerprint for detection of malwareUS8996470May 31, 2005Mar 31, 2015Google Inc.System for ensuring the internal consistency of a fact repositoryUS9092495Feb 28, 2014Jul 28, 2015Google Inc.Automatic object reference identification and linking in a browseable fact repositoryUS9208229Mar 31, 2006Dec 8, 2015Google Inc.Anchor text summarization for corroborationUS9269068Oct 16, 2009Feb 23, 2016Visible Technologies LlcSystems and methods for consumer-generated media reputation managementUS9298836 *Jul 7, 2015Mar 29, 2016Yext, Inc.Suppressing duplicate listings on multiple search engine web sites from a single source system given a synchronized listing is unknownUS9317180Jun 8, 2012Apr 19, 2016Vocus, Inc.Systems and methods for consumer-generated media reputation managementUS9342623Oct 8, 2013May 17, 2016Facebook, Inc.Automatically generating nodes and edges in an integrated social graphUS9348934 *Nov 25, 2014May 24, 2016Lockheed Martin CorporationSystems and methods for facilitating open source intelligence gatheringUS9367625 *May 3, 2013Jun 14, 2016Facebook, Inc.Search query interactions on online social networksUS9405792 *Aug 14, 2008Aug 2, 2016John Nicholas and Kristin Gross TrustNews aggregator and search engine using temporal decodingUS9430526 *Sep 29, 2008Aug 30, 2016Teradata Us, Inc.Method and system for temporal aggregationUS9465848Apr 11, 2014Oct 11, 2016Facebook, Inc.Detecting social graph elements for structured search queriesUS9514218Sep 18, 2014Dec 6, 2016Facebook, Inc.Ambiguous structured search queries on online social networksUS9542409 *Nov 26, 2008Jan 10, 2017Red Hat, Inc.Deduplicated file systemUS9558186Aug 14, 2014Jan 31, 2017Google Inc.Unsupervised extraction of factsUS9594852Oct 28, 2015Mar 14, 2017Facebook, Inc.Filtering suggested structured queries on online social networksUS9602965Nov 6, 2015Mar 21, 2017Facebook, Inc.Location-based place determination using online social networksUS20060253438 *Feb 24, 2006Nov 9, 2006Liwei RenMatching engine with signature generationUS20070050423 *Aug 30, 2006Mar 1, 2007Scentric, Inc.Intelligent general duplicate management systemUS20070130188 *Dec 6, 2006Jun 7, 2007Moon Hwa SData hashing method, data processing method, and data processing system using similarity-based hashing algorithmUS20070143282 *Mar 31, 2006Jun 21, 2007Betz Jonathan TAnchor text summarization for corroborationUS20070150800 *Mar 31, 2006Jun 28, 2007Betz Jonathan TUnsupervised extraction of factsUS20070198597 *Feb 17, 2006Aug 23, 2007Betz Jonathan TAttribute entropy as a signal in object normalizationUS20070198598 *Feb 17, 2006Aug 23, 2007Betz Jonathan TModular architecture for entity normalizationUS20070198600 *Mar 31, 2006Aug 23, 2007Betz Jonathan TEntity normalization via name normalizationUS20070294281 *May 7, 2007Dec 20, 2007Miles WardSystems and methods for consumer-generated media reputation managementUS20080222063 *Mar 6, 2007Sep 11, 2008Oracle International CorporationExtensible mechanism for detecting duplicate search itemsUS20080263026 *Apr 20, 2007Oct 23, 2008Amit SasturkarTechniques for detecting duplicate web pagesUS20080313111 *Jun 14, 2007Dec 18, 2008Microsoft CorporationLarge scale item representation matchingUS20090030898 *Jul 25, 2008Jan 29, 2009Seiko Epson CorporationFile search system, file search device and file search methodUS20090043767 *Aug 7, 2007Feb 12, 2009Ashutosh JoshiApproach For Application-Specific Duplicate DetectionUS20090048928 *Aug 14, 2008Feb 19, 2009John Nicholas GrossTemporal Based Online Search and AdvertisingUS20090049018 *Aug 14, 2008Feb 19, 2009John Nicholas GrossTemporal Document Sorter and Method Using Semantic Decoding and PredictionUS20090049037 *Aug 14, 2008Feb 19, 2009John Nicholas GrossTemporal Document Sorter and MethodUS20090049038 *Aug 14, 2008Feb 19, 2009John Nicholas GrossLocation Based News and Search EngineUS20090055359 *Aug 14, 2008Feb 26, 2009John Nicholas GrossNews Aggregator and Search Engine Using Temporal DecodingUS20090070683 *Aug 15, 2008Mar 12, 2009Miles WardConsumer-generated media influence and sentiment determinationUS20090100039 *Oct 11, 2007Apr 16, 2009Oracle International CorpExtensible mechanism for grouping search resultsUS20090106697 *Oct 14, 2008Apr 23, 2009Miles WardSystems and methods for consumer-generated media reputation managementUS20100049760 *Oct 25, 2007Feb 25, 2010Guillaume RousseauComputer tool for managing digital docementsUS20100082705 *Sep 29, 2008Apr 1, 2010Bhashyam RameshMethod and system for temporal aggregationUS20100131480 *Nov 26, 2008May 27, 2010James Paul SchneiderDeduplicated file systemUS20100145952 *Dec 10, 2009Jun 10, 2010Yeo Chan YoonElectronic document processing apparatus and methodUS20100174698 *Jan 6, 2009Jul 8, 2010Global Patent Solutions, LlcMethod for a customized and automated forward and backward patent citation searchUS20110137999 *Dec 8, 2009Jun 9, 2011International Business Machines CorporationTagging communication files based on historical association of tagsUS20110138000 *Dec 8, 2009Jun 9, 2011International Business Machines CorporationApplying tags from communication files to usersUS20140330819 *May 3, 2013Nov 6, 2014Rajat RainaSearch Query Interactions on Online Social NetworksUS20150081573 *Nov 20, 2014Mar 19, 2015International Business Machines CorporationSystem and method for avoiding duplication of effort in drafting documentsUS20150121265 *Nov 25, 2014Apr 30, 2015Lockheed Martin CorporationSystems and methods for facilitating open source intelligence gatheringUS20150134632 *Jul 30, 2012May 14, 2015Shahar GolanSearch methodUS20150161267 *Sep 12, 2012Jun 11, 2015Google Inc.Deduplication in Search ResultsWO2007143314A3 *May 7, 2007Oct 30, 2008Dean GrazianoSystems and methods for consumer-generated media reputation managementWO2008059124A3 *Oct 25, 2007Nov 6, 2008Inst Nat Rech Inf AutomatComputer tool for managing digital documentsWO2017007686A1 *Jun 30, 2016Jan 12, 2017Yext, Inc.Suppressing duplicate listings on multiple search engine web sites from a single source system* Cited by examinerClassifications U.S. Classification1/1, 707/E17.108, 707/E17.058, 707/999.2International ClassificationG06F17/30Cooperative ClassificationG06F17/3061, G06F17/30864European ClassificationG06F17/30W1, G06F17/30TLegal EventsDateCodeEventDescriptionMay 5, 2005ASAssignmentOwner name: WEST SERVICES, INC., MINNESOTAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CONRAD, JACK G.;CLAUSSEN, JOANNE R.S.;LIN, JIE;REEL/FRAME:016540/0747Effective date: 20050504Feb 8, 2007ASAssignmentOwner name: THOMSON GLOBAL RESOURCES, SWITZERLANDFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WEST SERVICES INC;REEL/FRAME:018878/0682Effective date: 20061009Oct 6, 2008ASAssignmentOwner name: THOMSON REUTERS GLOBAL RESOURCES, SWITZERLANDFree format text: CHANGE OF NAME;ASSIGNOR:THOMSON GLOBAL RESOURCES;REEL/FRAME:021630/0917Effective date: 20080603Mar 26, 2014FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services