Patent ID: 8311997

Claim:
A computer-implemented method comprising: annotating, by one or more processors, each keyword of a plurality of keywords with one or more labels of a plurality of labels, wherein: each keyword comprises one or more words, describes one or more characteristics or features of a specific advertising subject matter, and has a corresponding keyword document, each label comprises one or more words, describes one or more aspects of or one or more categories or concepts represented by a specific keyword of the plurality of keywords, and has a corresponding label document, and annotating each keyword comprises: for each label, computing a score for the keyword document corresponding to the keyword and the label using an annotation model; and annotating the keyword with a specific label of the plurality of labels where the keyword document corresponding to the keyword and the specific label have the highest or the lowest score among a plurality of scores; constructing a classifier based on a plurality of training keywords, wherein: each training keyword comprises one or more words and has a corresponding training keyword document, and constructing the classifier comprises: for each training keyword of the plurality of training keywords, annotating the training keyword with one or more labels of the plurality of labels; and for each label annotating the training keyword, determining a correctness of the label; calculating a first index-wise product between a word count vector of the training keyword document corresponding to the training keyword and a word count vector of the label document corresponding to the label; and forming a pair of the correctness and the first index-wise product; and training the classifier using one or more pairs of the correctness and the first index-wise product; and for each keyword of the plurality of keywords, for each label annotating the keyword, calculating a second index-wise product between a word count vector of the keyword document corresponding to the keyword and a word count vector of the label document corresponding to the label; and predicting whether the label annotating the keyword is correct using the classifier with the second index-wise product as an input to the classifier.