Abstract:
A system and method for fast concept search in multiple documents where the concept is expressed by plurality of words, all of which have to be in the same sentence and within specified range. The system automatically finds equivalent expressions of the same concept, and returns as search results all documents in which the concept is contained.

Description:
FIELD OF THE INVENTION 
       [0001]    The invention generally relates to the field of information retrieval and more particularly to retrieving answers to the concepts expressed in the search queries 
       BACKGROUND OF THE INVENTION 
       [0002]    In recent years there has been a massive movement towards computerizing medical data for various health service e organizations. However, making doctors write down their examination documents and their diagnostics using specific codes and sentences to write down the prognosis of each patient, will inevitably lower their productivity. Thus, most modern systems designed for computerizing medical data today go the path of natural language processing (NLP), allowing the doctors to write down their prognosis the way they are used to, and using computer analysis to extract vital information such as information about a patient, about illnesses, treatments etc. through the use of natural language processing (NLP). 
         [0003]    Naturally, this process presents many problems. One of them is the need to analyze and normalize sentences—for example “there is no sign of a hernia”; This prognosis can be written in many forms in natural language—for example “hernia has been ruled out”, or “no apparent sign of a hernia” and so on. These variations appear in different documents, and they all express the same concept. 
         [0004]    Most algorithms, such as the ones described in the public Stanford NLP pages and in many patents, refer to web searches. In these cases users fail to choose effective query terms. Often documents that satisfy user&#39;s information need may use different words than the query terms. We are interested in professional information retrieval system aimed to be used by professional community, such as health data retrieval system. In this case the query is expressed with the exact terms, but the meaning of the query depends on the whole phrase. In many cases the query defines allowed distances between words, but they do not require that that words in the phrase are in the same sentence. Thus wrong results can be retrieved. 
       SUMMARY OF THE INVENTION 
       [0005]    The disclosed invention assumes that an meaningful information that is searched by a user is expressed in a sentence, thus when a set of keywords are searched for, they are all expected to be in the same sentence. Usually, search engines define maximum distance between the words in the query regardless of the sentence limits. Hence, in the first phase of the processing, each new document that is added to the corpus is analyzed and broken into sentences so that for every word information as to its position in the document and to the sentence in which it appears is kept. In addition to the indexing information, normalized version and phonetic representation of the word are saved. 
         [0006]    From the phrase query entered by the user, many search phrases are derived. These search phrases are generated by finding dictionary synonyms to all query words, and retrieving semantic synonyms from an ontology. Phonetic representation is prepared for each word in the derived search phrases. From this data a comprehensive set of logical queries is prepared. 
         [0007]    It is an object of the disclosed invention to retrieve maximal set of relevant documents that relates to the query phrase 
         [0008]    It is another objective of the disclosed invention is to enable a professional user, who is not familiar with complex query structure, to retrieve information he is interested in. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]      FIG. 1  is a top level flow chart of the search preparation process. 
           [0010]      FIG. 2  presents a flow chart of the processing of the query. 
       
    
    
     DETAILED DESCRIPTION 
       [0011]    The invention will be described more fully hereinafter, with reference to the accompanying drawings, in which a preferred embodiment of the invention is shown. The embodiment refers to a corpus containing medical documents. 
         [0012]    The invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein; rather this embodiment is provided so that the disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. 
         [0013]    Before describing the processing that each document goes through, it is important to explain the corpus of the system. The corpus of the system is a database that stores information on each document ever entered the system, documents that constitute the search domain. Among the information on each word the system corpus keeps a list of all words and their locations within the sentences as well as the sentence number within the document where that word is located, referred to as the search indexes. It also contains a phonetic representation for each word as well as statistical information on the word. 
         [0014]    The top level flow chart of the preparation process is shown in  FIG. 1 . Each new medical document, which is part of the search domain, goes through the preparation process. The system reads a new document in step  100 . The document is split into sentences—step  102 . The sentences are temporarily saved in a List of Sentences  170 . Each sentence is processed by steps  104  to  122 . 
         [0015]    A sentence is retrieved from the List of Sentences  170  and an index to the sentence is added—step  104 . The retrieved sentence is tokenized in step  106  and a temporary List of Words and symbols in the sentence  180  is prepared. Each word in the sentence is processed in steps  108  to  120  as described hereafter. 
         [0016]    A new word is retrieved—step  108  from the List of words in a sentence  180 . In step  110  the system corpus  190  is searched to find out if the word is already known. If the word is new, as checked in step  112 , than the new word is processed in step  114 , where it is normalized and goes through phonetic conversion and is added to the corpus— 190  and the processing proceeds with step  116 . If the retrieved word is not new, processing continues in step  116 . In step  116  the statistics related to the word is updated, and the search index of the word in the sentence is updated in step  118 . 
         [0017]    If the retrieved word was not the last word in the sentence, as tested in step  120 , then the processing returns to step  108 , where processing of a new word begins. If the retrieved word was the last word in the sentence, then step  122  is executed. If the retrieved sentence was not the last one, then the processing of new sentence is executed, starting with step  104 . Otherwise the processing of the new document terminates. 
         [0018]      FIG. 2  describes the processing of the phrase query. For the purpose of explanation we assume that there are  3  documents in the corpus that contain the following sentences respectively, “there is no sign of Carcinoma”, “Carzinoma has been ruled out”, and “no apparent sign of cancer”. These three sentences clearly express the same idea. The user wants to find out the cases where cancer was suspected but was not found. The professional user enters the query “no carcinoma”. These words all have to be in the same sentence, but they do not have to be consecutive. The expression “ruled out” is synonym for “no” and it may appear after the subject “carcinoma” in the sentence and it gives the sentence the same meaning. Skin cancer, carcinoma, SCC are all semantic synonyms, and carcinoma is frequently misspelled as carzinoma, carsinome etc. The process as described hereafter can find all wording combinations that have the same meaning. 
         [0019]    The incoming search query is tokenized in step  202 . For each word in the query, syntactic synonyms based on phonetic similarity and normalization are generated in step  204  and are temporarily saved in a List of Synonyms  260 . The synonyms are looked for in the corpus  190 . Referring to the above give example, in this step the words carcinoma, carzinoma, are found because they are similar from phonetic point of view. This similarity is determined by the distance between these words measured by Jaro-Winkler algorithm. 
         [0020]    Semantic synonyms for each word in the query are derived in step  206  from an ontology  290 , and are added to the List of Synonyms  260 . Again, referring to the above given example, in this step the words cancer, SCC are semantic synonyms for carcinoma, and the words ruled-out, without, not and negative are semantic synonyms for “no”. 
         [0021]    Using the stored list of synonyms  260 , in step  208  a set of logical queries is prepared. The query set is comprised of all combinations of search phrases that express the same concept of the query. A search query within the set can include, in addition to the words, also logical constrains such as distance between the words in a sentence, or define that a specific word has to precede another one etc. For example, the query can include multiple phrases with logical operators that determine the relationship between them, e.g. hypertension OR [edema extremities]. Note that every query in the set includes the constraint that the words have to be in the same sentence. In step  210 , the set of queries are applied to the documents in the system corpus  190 , and a list of all sentences that contain the required words is prepared and these sentences are temporarily saved in a list  270 . 
         [0022]    A candidate search result sentence saved in the list  270  is popped from the list  270  in step  212 . The logical constraints and the distance between words are evaluated in step  216 . The maximum distance is checked against predefined threshold. If the logical constraints are met and the distance between the words in the sentence is below the query defined threshold, then the sentence with its relevant data, such as its document number is added—step  218  to the result set  280 . When all searched sentences in the list  270  have been processed, the test in step  214  indicates that there is no new sentence, and the search results are displayed to the user—step  222 . 
         [0023]    what has been described above is just one embodiment of the disclosed innovation. It is of course, not possible to describe every conceivable combination of components and/or methodologies, but one of ordinary skill in the art may recognize that many further combinations and permutations are possible. Accordingly, the innovation is intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims. 
         [0024]    Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.