Patent Document ID: 20110276577
Application ID: 12842921
Patent Status: 0

Claim One:
1. A content modelling system generating feature vectors of documents in different languages, the feature vectors providing scores associated with keywords defined in a base language for use by a profiler for generating or updating a user profile defining user preferences, the content modelling system comprising: a. a memory unit for storing instructions and data; and b. a processing unit for executing the instructions to provide: i. a plurality of keyword sets comprising: 1. a base language keyword set comprising a plurality of base language keywords each associated with a respective identifier (ID); and 2. a second language keyword set comprising a plurality of second language keywords each corresponding in meaning to a respective one of the base language keywords and associated with the ID of the corresponding base language keyword; ii. a plurality of tokenizers, each tokenizer associated with a language and a respective keyword set of the plurality of keyword sets, each tokenizer for: 1. receiving a text based document in the associated language of the tokenizer; 2. parsing the received document to identify keywords from the associated keyword set occurring in the received document; and 3. generating a plurality of ID:score pairs, each pair associating a score with an ID of a keyword in the associated keyword set occurring in the document, the score based on the frequency of occurrence of the ID corresponding to the keyword in the document; iii. a language identifier for receiving the document and identifying the language of the document as one of the base language or the second language; and iv. a tokenizer selector for receiving the identified language and selecting a corresponding tokenizer to parse the received document and produce a feature vector of the received document from the generated ID:score pairs.