A method performed by data processing apparatus, the method comprising: accessing training data indicating queries submitted by one or more users; determining, for each of the queries, a count of a number of times the training data indicates the query was submitted; selecting, based on the counts, a first proper subset of the queries that were most frequently submitted; training a first component of a language model based on the counts, the first component including first probability data indicating, for each respective query among the first proper subset of the queries that were most frequently submitted, a relative frequency of the respective query among the training data; training a second component of the language model based on the training data, the second component including second probability data for assigning scores to any language sequence; determining adjustment data that normalizes the second probability data with respect to the first probability data; storing the first component, the second component, and the adjustment data; determining a first score for a particular query, among the first proper subset of the queries, using the first component of the language model; determining a second score for the particular query using the second component of the language model; determining that the first score and the second score do not satisfy a predetermined relationship; in response to determining that the first score and the second score do not satisfy the predetermined relationship, removing the particular query from the first proper subset of the queries to generate an altered first proper subset of the queries. accessing training data indicating queries submitted by one or more users; determining, for each of the queries, a count of a number of times the training data indicates the query was submitted; selecting, based on the counts, a first proper subset of the queries that were most frequently submitted; training a first component of a language model based on the counts, the first component including first probability data indicating, for each respective query among the first proper subset of the queries that were most frequently submitted, a relative frequency of the respective query among the training data; training a second component of the language model based on the training data, the second component including second probability data for assigning scores to any language sequence; determining adjustment data that normalizes the second probability data with respect to the first probability data; storing the first component, the second component, and the adjustment data; determining a first score for a particular query, among the first proper subset of the queries, using the first component of the language model; determining a second score for the particular query using the second component of the language model; determining that the first score and the second score do not satisfy a predetermined relationship; in response to determining that the first score and the second score do not satisfy the predetermined relationship, removing the particular query from the first proper subset of the queries to generate an altered first proper subset of the queries.