Patent Document ID: 8103650
Application ID: 12493997

Base Claim:
1. A computer implemented method comprising: accessing by one or more computer systems a plurality of keywords, each of the keywords comprising one or more words and describing one or more characteristics or features of particular advertising subject matter; annotating by a processor each of the keywords with one or more labels selected from a plurality of labels, each of the labels comprising one or more words and describing one or more aspects of or one or more categories or concepts represented by the keyword, wherein annotating each of the keywords with one or more labels comprises: for each of the keywords, when the keyword is able to be directly annotated, then directly annotating the keyword; and when the keyword is unable to be directly annotated, then: constructing a keyword document for the keyword; for each of the labels, constructing a label document for the label; and calculating a similarity between the keyword document corresponding to the keyword and the label document corresponding to the label as ∑ k = 1 k = n w ⁢ ⁢ wc i , k ⁢ wc j , k  wc i  ⁢  wc j  , where: (1) n W denotes a total number of unique words in the keyword document and the label document; (2) wc i,k denotes a number of times a specific word, word k , occurs in the keyword document; (3) wc j,k denotes a number of times the specific word, word k , occurs in the label document; (4) wc i denotes a n W -vector consisting of the wc i,k ; and (5) wc j denotes a n W -vector consisting of the wc j,k ; and annotating the keyword based on the similarity between the keyword document corresponding to the keyword and the label document corresponding to each of the labels; grouping the keywords into one or more keyword groups based on similarities between the labels of the keywords, each of the keyword groups comprising one or more of the keywords; and forming one or more advertising groups from the keyword groups, comprising: from each of the keyword groups, forming one or more of the advertising groups, each of the advertising groups comprising one or more of the keywords in the keyword group.

---

Claim 9:
9. The method of claim 1 , further comprising: constructing a classifier based on a first subset of the keywords, comprising: for each of the first subset of keywords, for each of the labels annotating the keyword, determining a correctness of the label; calculating a first index-wise product between a word count vector of the keyword document corresponding to the keyword and a word count vector of the label document corresponding to the label; and forming a pair of the correctness and the first index-wise product; and training the classifier using the pairs of the correctness and the first index-wise product; and for each of a second subset of the keywords, for each of the labels annotating the keyword, calculating a second index-wise product between a word count vector of the keyword document corresponding to the keyword and a word count vector of the label document corresponding to the label; and predicting whether the label is correct using the classifier with the second index-wise product as an input to the classifier.