Patent ID: 7062485

Claim:
A method facilitated by a human annotator and performed in a computer environment for normalizing a score associated with a document, the method comprising the steps of: (a) establishing (1) through the computer environment a set of training documents most of which are believed not to be relevant to a topic (off-topic) and (2) through the human annotator a query relevant to the topic (on-topic); (b) assigning, through the computer environment, a training document relevance score to each one of the training documents, each training document relevance score representing a measure of relevance of its respective document to the topic; (c) determining, through the computer environment, statistics relating to all training document relevance scores and thereby obtaining determined statistics; (d) receiving a testing document; (e) calculating, through the computer environment, a score of relevance of the testing document to the topic to obtain a testing document relevance score; (f) normalizing, through the computer environment and based on the statistics, the testing document relevance score to obtain a normalized score wherein: normalizing adjusts the testing document relevance score based on the statistics to be comparable to other scores from which the statistics were determined, and the normalized score is a better predictor of probability of the testing document being relevant than the testing document relevant score; (g) establishing, through the computer environment, a threshold score representing a relevance threshold for the topic; (h) comparing the normalized score to the threshold score to obtain a comparison; and (i) designating the testing document as relevant or not relevant to the topic based on the comparison.