Patent EA-002016-B1 (EA)
Title: A method of searching for fragments with similar text and/or semantic contents in electronic documents stored on a data storage devices
IPC: G06F17/30
Filed: 2001-04-06
Abstract: 1. A method of searching for fragments with similar text and/or semantic content in electronic documents stored on data storage devices comprising: indexing each document which is saved in the archive, dividing said documents into fragments and forming subject groups containing one and more fragments, defining search parameters, conducting a search, ranking the list of document fragments received as a result of the search, wherein a set of unique information blocks appeared in the selected document fragment is defined as search parameters and said set is extended by preprocessing each of said unique information blocks, where a unique information block is an information block that appeared in the selected document fragment one or more times, where the preprocessing is an operation of obtaining from at least one unique information block one or more information blocks connected with said unique information block by a specified relation. 2. The method of claim 1, wherein an information block is a character string appeared in a document and limited by certain characters. 3. The method of claim 1, wherein a document fragment is any selected information block sequence in a document. 4. The method of claim 3, wherein a document fragment is a complete copy of a document. 5. The method of claim 3, wherein rules of dividing documents into fragments are specified. 6. The method of claim 5, wherein a document is divided into fragments using at least one rule. 7. The method of claim 5, wherein documents are automatically divided into fragments. 8. The method of claim 1, wherein a set of document fragments which are being searched for fragments with similar text and/or semantic content to the selected one is limited by indicating at least one rule which was used to divide documents into fragments and/or indicating a subject group. 9. The method of claim 1, wherein users form a subject group. 10. The method of claim 1, wherein a program module for processing documents is used to divide documents and their fragments into a set of information blocks. 11. The method of claim 1, wherein at least one operation of preprocessing is selected and users specify the order of preprocessing operations. 12. The method of claim 11, wherein the preprocessing consists of one logical operations of identity. 13. The method of claim 1, wherein after performing the operation of preprocessing, at least one initial block is removed or left in the query. 14. The method of claim 1, wherein all unique information blocks of the selected document fragment are included in a set of unique information blocks defined as search parameters. 15. The method of claim 1, a set of unique information blocks, which is defined as search parameters, is formed one or more functions. 16. The method of claim 15, a function of making a list of unique information blocks is used for the selected fragment. 17. The method of claim 15, wherein it is used a function of determining the number of occurrences of information blocks obtained after preprocessing unique information blocks in the text content of fragments of the selected subject group. 18. The method of claim 15, wherein it is used a function of determining the frequency of occurrences of unique information blocks in the text content of fragments of the selected subject group, where the frequency of occurrences is expressed in percents from the number of occurrences of the most frequently appeared unique information block in the text content of the selected subject group. 19. The method of claim 1, wherein a set of unique information blocks which is defined as search parameters includes a specified number of unique information blocks from the set which has been created according to user-defined rules. 20. The method of claim 1, wherein the search for document fragments is carried out on local and/or remote data storage devices. 21. The method of claim 20, wherein a remote device is any information resource or information retrieval system that operates in a computer network and provides in response to a search query a list of document fragments relevant to search parameters. 22. The method of claim 1, wherein the list of document fragments received as a result of search is ranked according to the number of information block groups appeared in found document fragments which combine information blocks obtained after preprocessing the unique information block. 23. The method of claim 22, wherein while ranking, in addition for document fragments with the coincident number of information block groups the number of occurrences of said information block groups in the text content of fragments of the selected subject group is determined. 24. The method of claim 1, wherein the fragments from the list of document fragments received as a result of search are displayed with highlighting differences between their text content and the text content of the selected fragment. 25. The method of claim 0, wherein a document which is being added to the archive is saved as a new version of previously saved document which has a specified similarity measure by text and/or semantic content with the document which is being added to the archive, and said new version is saved as a complete copy of the document which is being added to the archive or in the form of differences between the text content of the document which is added to the archive and the text content of said previously saved document. 26. The method of claim 0, wherein fragments which are saved on a data storage device are automatically classified. 27. The method of searching for electronic documents stored on data storage devices comprising indexing each document which is saved in the archive, defining parameters of searching for electronic documents which include forming  the initial query containing two or more information blocks, specifying the maximal number of information blocks which may appear between said two or more information blocks in the desired document (the interval) and the order of alternation of said two or more information blocks in the desired document in the specified interval ranking the list of documents received as a result of search, said method further comprises: extending the initial query containing two or more information blocks by preprocessing one or more information blocks from the initial query, where the preprocessing of the initial query  is an operation of obtaining from at least one unique information block one or more information blocks connected with said unique information block by a specified relation, searching for documents using any specified number of information blocks defined as search parameters while forming the initial query. 28. The method of claim 27, wherein an information block is a character string appeared in a document and limited by certain characters. 29. The method of claim 27, wherein a program module for processing documents is used to divide documents and their fragments into a set of information blocks. 30. The method of claim 27, wherein at least one operation of initial query preprocessing is selected and users specify the order of initial query preprocessing operations. 31. The method of claim 30, wherein the preprocessing of the initial query consists of one logical operations of identity. 32. The method of claim 27, wherein after performing the operation of preprocessing, at least one information block is removed or left in the query 33. The method of claim 27, wherein the search for documents is carried out on local and/or remote data storage devices. 34. The method of claim 33, wherein a remote device is any information resource or information retrieval system that operates in a computer network and provides in response to a search query a list of document fragments relevant to search parameters. 35. The method of claim 27, wherein the list of document fragments received as a result of search is ranked in accordance with the number of occurrences of relevant information block sequences in documents, the length of the interval in which said sequences fall and weighting coefficient assigned to information blocks. 36. The method of claim 35, wherein documents are displayed with highlighting the information block sequences which correspond to search parameters and the preprocessed query, and providing the navigation through the text content of documents using information block sequences within a whole list of documents received after search.