Patent ID: 8027832

Claim:
A method of storing information to be used in identifying the natural language of text, the method comprising: receiving text documents, each text document being written in a known natural language; counting occurrences of each unique feature, for a plurality of unique features, in each text document for each language of a plurality of candidate languages to generate expected feature probability values for each unique feature; storing said expected feature probability values for each unique feature for each of the plurality of candidate languages; receiving a text sample written in an unidentified natural language; counting a total number of features in the text sample written in the unidentified natural language; determining actual occurrences of one or more of the plurality of unique features in the text sample written in the unidentified natural language; determining actual feature counts for the one or more unique features based on the actual occurrences; calculating probability values for the one or more unique features based on the total number of features of the text sample written in the unidentified natural language, the stored expected feature probability values for each of the one or more unique features for each of the plurality of candidate languages, and the actual feature counts for the one or more unique features; calculating language scores for the plurality of candidate languages using a computer, said calculating comprising estimating a joint probability of seeing the one or more unique features in the text sample with the actual feature counts for the one or more unique features by multiplying the probability values calculated for the one or more unique features in the text sample; and identifying at least one language for the text sample from among the plurality of candidate languages based on the language scores.