Patent Document ID: 7814040
Application ID: 11626835

Base Claim:
1. A computer-implemented method of retrieving media, comprising: (a) creating a probabilistic framework relating media types within a mixed media work to implicit concepts; (b) creating an index of a set of media based on implicit concepts within the probabilistic framework; (c) receiving a query expressed in the form of a media exemplar; (d) determining a set of concepts expressed in the media exemplar; (e) searching the index of the set of media for elements representing similar implicit concepts to those expressed in the media exemplar; and (f) outputting at least one representation or identifier of the elements representing similar implicit concepts to those expressed in the media exemplar; wherein said outputting comprises outputting a representation or identifiers of a plurality of elements, further comprising: ranking the plurality of elements based on at least a similarity of the respective element to implicit concepts expressed in the media exemplar; wherein said determining comprises probabilistically determining a set of semantic concepts inherent in the media exemplar, based on correlations of features in respective multimedia works having predetermined semantic concepts associated therewith; further comprising determining a concept vector for the media exemplar; wherein the probabilistic framework comprises a Bayesian model for associating words with an image having visual features, comprising a hidden concept layer which connects a visual feature layer and a word layer which is discovered by fitting a generative model to a training set comprising images having the visual features and annotation words, wherein the conditional probabilities of the visual features and the annotation words given a hidden concept class are determined based on an Expectation-Maximization (EM) based iterative learning procedure.

---

Claim 2:
2. The method according to claim 1 , wherein f 1 ,iε[1,N] denotes a visual feature vector of images in a training database, where N is the size of the database, w i ,jε[1,M] denotes the distinct textual words in a training annotation word set, where M is the size of an annotation vocabulary in the training database, the visual features of images in the database, f i =[f i 1 , f i 2 , . . . , f i L ],iε[1,N] are known i.i.d. samples from an unknown distribution, having a visual feature dimension L, the specific visual feature annotation word pairs (f i ,w j ), iε[1,N],jε[1,M] are known i.i.d. samples from an unknown distribution, associated with an unobserved semantic concept variable zεZ={z 1 . . . z k }, in which each observation of one visual feature fεF={f i , f 2 , . . . , f N } belongs to one or more concept classes z k and each observation of one word wεV={w 1 , w 2 , . . . , w M } in one image f belongs to one concept class, in which the observation pairs (f i ,w j ) are assumed to be generated independently, and the pairs of random variables (f i ,w j ) are assumed to be conditionally independent given the respective hidden concept z k , such that 
 P ( f i ,w j |z k )= p ℑ ( f i |z k ) P V ( w j |z k ).