Method and system for evaluating and ranking images with content based on similarity scores in response to a search query

According to one embodiment, a request is received for evaluating matching between a content item and an image that were identified and selected from a plurality of content items and images in response to a search query. First metadata associated with the content item, second metadata associated with the image, and third metadata associated with the search query are obtained. A first similarity score is determined based on the first metadata associated with the content item and the third metadata associated with the search query. A second similarity score is determined based on the second metadata associated with the image and the third metadata associated with the search query. An evaluation score is calculated for matching of the content item and the image in view of the search query. The evaluation score is utilized to identify subsequent content items and images in response to a subsequent search query.

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to searching content. More particularly, embodiments of the invention relate to searching content with matching images based on similarity scores in response to a search query.

BACKGROUND

Most search engines typically perform searching of Web pages during their operation from a browser running on a client device. A search engine receives a search term entered by a user and retrieves a search result list of Web pages associated with the search term. The search engine displays the search results as a series of subsets of a search list based on certain criteria. General criteria that is used during a search operation is whether the search term appears fully or partly on a given webpage, the number of times the search string appears in the search result, alphabetical order, etc. Further, the user can decide to open a link by clicking on the mouse button to open and browse. Some of the user interactions with the search results and/or user information may be monitored and collected by the search engine to provide better searches subsequently.

Typically, in response to a search query, a search is performed to identify and retrieve a list of content items. The content items are then returned to a search requester. A conventional search engine would return most of the content items as is without modifying. Some of the content items in the search result are just plain text or description, which may deem to be unattractive or boring. Sometimes content in a search result would be more presentable or attractive if the content items are prepared with certain images that are related to the content items. However, it is quite challenge to match appropriate images with content items. There has been a lack of efficient ways to match content with images and to evaluate the matching of content and images.

DETAILED DESCRIPTION

According to some embodiments, a content item and an image are received, where the content item and the image were identified in response to a search query. Similarity scores amongst the content item, the image, and the search query are calculated. The similarity scores include a similarity score between the search query and the image, a similarity score between the search query and the content item, and a similarity score between the content item and the image. The similarity scores represent relationships amongst the search query, the content item, and the image (e.g., how similar or compatible amongst the parties). An evaluation score or ranking score is determined for matching the content item with the image based on the similarity scores amongst the content item, the image, and the search query. The evaluation score may be utilized to evaluate a result of matching the content item and the image, which have been performed in the past to determine (e.g., offline) whether a matching method or model needs to be adjusted or improved. The same mechanism can be utilized to dynamically rank the paring between a content item and an image in real-time in response to a search query, e.g., online image selection or ranking.

According to one aspect of the invention, in response to a request for evaluating matching of a content item and an image that were identified in response to a search query, metadata associated with the content item, the image, and the search query is obtained. A first similarity score is determined based on the metadata associated with the content item and the search query. A second similarity score is determined based on the metadata associated with the image and the search query. An evaluation score for evaluating the matching between the content item and the image is calculated based on the first and second similarity scores. In one embodiment, the evaluation score may be determined further based on a third similarity score that is determined based on the metadata associated with the content item and the image. The evaluation score may be utilized to adjust the selection methods or algorithms in subsequent matching of content items and images in response to subsequent search queries.

According to another aspect of the invention, a content item and a list of images that have been identified in response to a search query are received. For each of the images, a first similarity score is determined based on metadata associated with the content item and the search query. A second similarity score is determined based on metadata associated with the image and the search query. A ranking score is calculated for the image based on the first similarity score and the second similarity score. The images are then ranked based on their respective ranking scores to select one of the images to be associated with the content item. In one embodiment, a third similarity score is determined based on metadata associated with the content item and the image. The ranking score for the image is calculated further based on the third similarity score.

FIGS. 1A and 1Bare block diagram illustrating an example of system configuration for matching images with content items according to some embodiments of the invention. Referring toFIG. 1A, system100includes, but is not limited to, one or more client devices101-102communicatively coupled to server104over network103. Client devices101-102may be any type of client devices such as a personal computer (e.g., desktops, laptops, and tablets), a “thin” client, a personal digital assistant (PDA), a Web enabled appliance, a Smartwatch, or a mobile phone (e.g., Smartphone), etc. Network103may be any type of networks such as a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof, wired or wireless.

Server104may be any kind of servers or a cluster of servers, such as Web or cloud servers, application servers, backend servers, or a combination thereof. In one embodiment, server104includes, but is not limited to, search engine120, image selection module110, and image selection method(s) or model(s)115. Server104further includes an interface (not shown) to allow a client such as client devices101-102to access resources or services provided by server104. The interface may include a Web interface, an application programming interface (API), and/or a command line interface (CLI).

For example, a client, in this example, a user application of client device101(e.g., Web browser, mobile application), may send a search query to server104and the search query is received by search engine120via the interface over network103. In response to the search query, search engine120extracts one or more keywords (also referred to as search terms) from the search query. Search engine120performs a search in content database133, which may include primary content database130and/or auxiliary content database131, to identify a list of content items that are related to the keywords. Primary content database130(also referred to as a master content database) may be a general content database, while auxiliary content database131(also referred to as a secondary or accessory content database) may be a special content database. Search engine120returns a search result page having at least some of the content items in the list to client device101to be presented therein. Search engine120may be a Baidu® search engine available from Baidu, Inc. or alternatively, search engine120may represent a Google® search engine, a Microsoft Bing™ search engine, a Yahoo® search engine, or some other search engines.

A search engine, such as a Web search engine, is a software system that is designed to search for information on the World Wide Web. The search results are generally presented in a line of results often referred to as search engine results pages. The information may be a mix of Web pages, images, and other types of files. Some search engines also mine data available in databases or open directories. Unlike web directories, which are maintained only by human editors, search engines also maintain real-time information by running an algorithm on a web crawler.

Web search engines work by storing information about many web pages, which they retrieve from the hypertext markup language (HTML) markup of the pages. These pages are retrieved by a Web crawler, which is an automated Web crawler which follows every link on the site. The search engine then analyzes the contents of each page to determine how it should be indexed (for example, words can be extracted from the titles, page content, headings, or special fields called meta tags). Data about web pages are stored in an index database for use in later queries. The index helps find information relating to the query as quickly as possible.

When a user enters a query into a search engine (typically by using keywords), the engine examines its index and provides a listing of best-matching web pages according to its criteria, usually with a short summary containing the document's title and sometimes parts of the text. The index is built from the information stored with the data and the method by which the information is indexed. The search engine looks for the words or phrases exactly as entered. Some search engines provide an advanced feature called proximity search, which allows users to define the distance between keywords. There is also concept-based searching where the research involves using statistical analysis on pages containing the words or phrases you search for. As well, natural language queries allow the user to type a question in the same form one would ask it to a human.

The usefulness of a search engine depends on the relevance of the result set it gives back. While there may be millions of web pages that include a particular word or phrase, some pages may be more relevant, popular, or authoritative than others. Most search engines employ methods to rank the results to provide the “best” results first. How a search engine decides which pages are the best matches, and what order the results should be shown in, varies widely from one engine to another.

Referring back toFIG. 1A, according to one embodiment, in response to a search query received at server104from a client device, in this example, client device101, search engine120performs a search in content database133, such as primary content database130and/or auxiliary content database131, to generate a list of content items. Each of the content items may be associated with a particular Web page of a particular Web site of a particular content provider via a uniform resource link (URL) and/or a uniform resource identifier (URI). In one embodiment, primary content database130stores general content items that have been collected by network crawlers (e.g., unsponsored content). Auxiliary content database135stores specific or special content items that are associated with specific, known, or predetermined content providers (e.g., sponsored content). Alternatively, content database133may be implemented as a single database without distinguishing primary content database131from auxiliary content database132.

Network crawlers or Web crawlers are programs that automatically traverse the network's hypertext structure. In practice, the network crawlers may run on separate computers or servers, each of which is configured to execute one or more processes or threads that download documents from URLs. The network crawlers receive the assigned URLs and download the documents at those URLs. The network crawlers may also retrieve documents that are referenced by the retrieved documents to be processed by a content processing system (not shown) and/or search engine120. Network crawlers can use various protocols to download pages associated with URLs, such as hypertext transport protocol (HTTP) and file transfer protocol (FTP).

In addition, according to one embodiment, image selection module or system110identifies based on the keywords associated with the search query, using image selection method(s) or model(s)115, a list of image IDs identifying images that are related to the keywords associated with a search query. Image selection methods/models115may include a keyword-to-image (keyword/image) mapping table (not shown), which may be implemented in a variety of data structures such as a table or database. Based on the image IDs, a list of image candidates can be identified and retrieved from images123stored in image store125, where image store125may also store image metadata describing images122(not shown).

In one embodiment, images122and their respective metadata may be obtained by one or more image or network crawlers that are designed to craw the networks to collect images as well as their surrounding metadata. Images122may be non-privileged, non-copyrightable, properly licensed images, or any other authorized images. The image candidates may be ranked based on a ranking score between the keywords and the image candidates, as well as metadata of the content items. For each of the content items found in content database133, an image is selected from the list of image candidates to be associated with the content item. The selected image may be incorporated with the content item to generate an incorporated or composite content item. For example, the selected image may serve as a background image to the content item. The list of incorporated content items and images is returned to client device101as part of a search result.

According to one embodiment, image selection methods/models115may be previously configured and generated prior to receiving the search query (e.g., offline), which may include a keyword-to-image (keyword/image) mapping table. The keyword/image mapping table includes a number of mapping entries, each mapping entry mapping a keyword to one or more image IDs, or vice versa, as shown inFIGS. 3A-3Bfor example. The keywords may be identified as the keywords that are more likely used in search queries and/or keywords associated with certain content items (e.g., bidwords of sponsored content). Such keywords may be identified based on an analysis or tracking of user searching activities or search history, which may be compiled for a period of time.

Subsequently when a search query is received at run-time by search engine120from a client device for searching content, a search is performed in content database133to retrieve a list of content items. In addition, an analysis is performed, for example, by image selection module110, on the search query to determine one or more keywords associated with the search query. The determined keywords may be the keywords included in the search query. The determined keywords may further include certain keywords that are semantically similar or have the same meaning of the keywords originally in the search query (e.g., synonymous words or phrases). Based on the keywords, a list of one or more images are identified from image store125using the keyword/image mapping table as part of image selection methods/models115.

According to one embodiment, the identified images may be ranked by image selection module110, using a variety of ranking algorithms or ranking models as part of image selection methods/models115, which have been generated and configured by data analytics system150(also referred to as a query/image mapping system). For each of the images that are identified as image candidates to be matched with a content item, similarity scores between each pair of the search query, the content item, and the image are determined, for example, using a specific scoring algorithm or method. A final ranking score is determined based on the similarity scores representing the relationships amongst the content item, the search query, and the image. The image candidates are then ranked based on their respective ranking scores. One of the images is then selected to be matched with the content item.

According to another embodiment, the same or similar scoring methods or models can be utilized to evaluate the results of prior matching of content items and images. The evaluation may be performed offline by data analytics system or server150based on the data of prior content/image matching processes. The evaluation may be utilized to further adjust the image selection methods/models115for future image selections.

Note that the configuration of server104has been described for the purpose of illustration only. Server104may be a Web server to provide a frontend search service to a variety of end user devices. Alternatively server104may be an application server or backend server that provides specific or special content search services to a frontend server (e.g., Web server or a general content server), and to match and/or integrate images with content items of a content database or server. Data analytics system150may also be implemented as a separate server, which is responsible for creating or training image selection methods/models115based on the content items and their respective associated keywords of content database or server133.

Other architectures or configurations may also be applicable. For example, as shown inFIG. 1B, content database133may be maintained and hosted in a separate server as a content server over a network. Similarly, image store125may be maintained and hosted in a separate server as an image server having an image searching engine therein. Servers133and125may be Web servers, application servers, or backend servers. Content server133and/or image server125may be organized and provided by the same entity or organization as of server104. Alternatively, content server133and/or image server125may be maintained or hosted by separate entities or organizations (e.g., third-party providers), which are responsible for collecting contents in content databases130-131and images122and their metadata.

Also note that content database/server133may include primary content database130and auxiliary content database131. Primary content database130may also be implemented or maintained in a separate content server, referred to as a primary content server. Similarly, auxiliary content database131may be implemented or maintained in a separate content sever, referred to as an auxiliary or accessory content server. The content items obtained from both content databases130-131may be matched with images obtained from image store/server125using a variety of matching formulas. Alternatively, only content items obtained from one of primary content database130and auxiliary content database131will be matched with images obtained from image store/server125. For example, content items obtained from auxiliary content database131(e.g., sponsored content) will be matched with the images obtained from image store/server125, while content items obtained from primary content database130(e.g., general content) will be returned to the client device without modification as part of the search result.

FIG. 2is a block diagram illustrating a system for matching images with content items according to one embodiment of the invention. System200may be implemented as part of system100ofFIGS. 1A-1B. Referring toFIG. 2, when search query201is received from a client device (e.g., client device101ofFIG. 1), search engine120performs a first search in content database or content server133to identify and retrieve a first list of content items based on one or more keywords or search terms associated with search query201. In addition, search engine120communicates with image selection module110to identify a list of images from image store or image sever125based on the keywords associated with search query201using a set of image selection methods/models115. Search engine120and/or image selection module110may perform an analysis on the query to derive a list of keywords that are included in search query201and/or similar to the keywords in search query201(e.g., semantically similar terms, synonymous terms). The searches performed in content database/server133and/or image store/server125may be performed on the list of expanded keywords.

In one embodiment, image selection module110and/or image selection methods/models115may be integrated with search engine120. Image selection methods/models115may be previously configured or compiled, for example, by query/image mapping system150. Query/image mapping system150may be hosted in a separate system or server communicatively coupled to system200via an API or over a network. Query/image mapping system200may include a user interface to allow a user or an administrator to configure a set of query/image matching rules, which may then be expanded and ranked using a predetermined algorithm by processing logic. Further details of query/image mapping system200will be described further below.

Similarly, images stored in image store/server125may be collected by image collection system230, which may be a separate system or server communicatively coupled to system200over a network. Query/image mapping system150and/or image collection system230may be operated by the same or different entity or organization as of system200. In this example, images may be cached and stored in a local image store with respect to system200(e.g., local to server104). Alternatively, images may be maintained by a designated server associated with image collection system230, with which image selection module110communicates to identify and retrieve the list of images via API.

Based on the list of images retrieved from image store/server125, image selection module110ranks the images according to a ranking algorithm, for example, based on similarity scores amongst the content items, the images, and the search query. Some of the images are then matched with some of the content items identified and retrieved from content database/server133. The matched content items and images are then integrated into integrated content items. In one embodiment, an image may be selected as a background image or a complementary image to a content item. For example, content item211may be a description or text and image212may be selected to be a background image for content item211. Image212is selected based on the matching or ranking techniques described throughout this application to complement or describe content item211, or vice versa, in an appropriate manner. For example, the content of content item211is related to the content as shown in image212, or vice versa. The integrated images212and content items211may be returned to the client device as a part of search result215.

According to one embodiment, in response to search query201received from a client device, one or more keywords are determined based on search query201, where the keywords may include those in search query201or those expanded based on an analysis on search query201. Based on the keywords, a lookup operation or search is performed in image selection methods/models115, which may be implemented in a variety of data structures, such as, for example, a database or a table.

Image selection methods/models115includes a keyword/image mapping table having a number of mapping entries. Each mapping entry maps one or more keywords to one or more image IDs that identify one or more images stored in image store/server125. Based on the keywords obtained based on search query201, a list of one or more image IDs may be obtained based on image selection methods/models115. Based on the image IDs, the corresponding images are obtained from image store/server125as image candidates. The image candidates are then ranked and matched using one or more predetermined ranking and/or matching algorithms, which will be described in details further below. The top ranked images may then be selected to be associated with the content items for integration as part of search result215. Note that some or all of the components or modules as shown inFIG. 2may be implemented in software, hardware, or a combination thereof.

FIGS. 3A-3Bare examples of query-image matching tables according to certain embodiments of the invention. Referring toFIG. 3A, query/image matching table300may represent a query/image mapping table as part of image selection methods/models115ofFIGS. 1A-1B and 2as described above. In one embodiment, query/image matching table300includes a number of matching entries. Each of the matching entry maps one or more keywords301to one or more image IDs302, where image IDs302identify the corresponding images in an image store or image server, such as image store/sever125. Matching table300is indexed based on the keywords. In this example, a first entry maps a term “flower” to images1-5. A second entry maps a term “Beijing flower” to image1only. A third entry maps a term “Shanghai flow” to image2. A fourth entry maps a term “flower delivery” to images1-2and4. Thus, if a search query contains “Beijing flower,” images1-5may be identified. However, image1may have a higher ranking.

Referring now toFIG. 3B, which is an example of an alternative embodiment of a matching table, this is an image-keyword (image/keyword) matching table that may be utilized as part of image selection methods/models115. In this example, image/keyword matching table350includes a number of matching entries. Each matching entry maps an image ID351to one or more keywords352. Matching table350is indexed based on image IDs. Both tables300and350may be utilized interchangeably. For example, table300may be used to identify all of the images that are related to one or more keywords.

FIG. 4is a block diagram illustrating an example of image selection system according one embodiment of the invention. System400may be implemented as part of system100or system200ofFIGS. 1A-1B and 2. Referring toFIG. 4, image selection module or system110includes, but is not limited to, keyword extraction module401, image search module402, query-content (QC) scoring module403, query-image (QI) scoring module404, content-image (CI) scoring module405, and image ranking module406. Scoring modules403-405are configured to determine similarity scores representing the relationships amongst the content items, images, and a corresponding search query for ranking purposes. Scoring modules401-406may be implemented in software, hardware, or a combination thereof. For example, scoring modules401-406may be loaded in a memory and executed by one or more processors.

Referring now toFIGS. 4-5, in response to search query501or search terms of search query501, keyword extraction module401(also referred to as a query analysis module) performs an analysis on the search terms to derive a set of keywords502that are associated with the search terms of search query501. Keywords502may be those included in the search terms of search query501. Alternatively, keywords502may further include additional keywords that are semantically related to the search terms of search query501, which may be determined based on the analysis. For example, keyword extraction module401may perform a latent semantic analysis on the search terms to derive additional keywords that are semantically related to the search terms of search query501. The analysis may also remove some of the keywords from search query501that deem to be unrelated to contradict to an intent of a search requester.

A latent semantic analysis (LSA) is a technique in natural language processing, in particular distributional semantics, of analyzing relationships between set of documents and the terms they contain by producing a set of concepts related to the documents and terms. LSA assumes that words that are close in meaning will occur in similar pieces of text. A matrix containing word counts per paragraph (rows represent unique words and columns represent each paragraph) is constructed from a large piece of text and a mathematical technique called singular value decomposition (SVD) is used to reduce the number of rows while preserving the similarity structure among columns. Words are then compared by taking the cosine of the angle between the two vectors (or the dot product between the normalizations of the two vectors) formed by any two rows. Values close to 1 represent very similar words while values close to 0 represent very dissimilar words.

Based on keywords502, image search module402searches in image store125to obtain a list of images504as image candidates, as well as their associated metadata. Image search module402may look up in a keyword/image mapping table based on keywords502to identify a list of image IDs that identify images504. For each of the images504, image selection/ranking module406invokes the scoring modules403-405to determine the similarity scores amongst the content item identified by content ID503, the image, and search query501, using the proper scoring algorithms or models obtained from selection methods/models115as described above. A final ranking score for each image is calculated based on the associated similarity scores. The images are then ranked by their respective ranking scores to generated ranked image candidates505.

FIG. 6is a block diagram illustrating a processing flow of ranking images according to another embodiment of the invention. Referring toFIG. 6, in one embodiment, in response to content item602and image603that have been identified in response to search query601, keyword extraction module401extracts or determines keywords611-613associated with search query601, content item602, and image603, respectively. The keywords may be included in search query601, content item602, and image603. Alternatively, the keywords may further include similar keywords or semantically related keywords that are determined based on an analysis (e.g., latent semantic analysis), for example on metadata604associated with search query601, content item602, and image603.

Keywords612associated with content item602may be obtained based on the title, description, a landing page, as well as other metadata associated with content item602. Keywords613associated image603may be obtained from its title and context associated with image603, including a source from which image603was obtained and/or content recognized from image603(e.g., object, scene, theme represented by the image). A source from which an image is obtained may be a Web page or a document in which the image is attached. An address such as a universal resource locator (URL) of the source page may be collected. In addition, an analysis may be performed on the content of the source page to determine the content possibly represented by the image. An image recognition may also be performed on the image to determine the content of the image (e.g., whether the image is about a person, an object, landscape, texts, or a combination thereof).

Keyword extraction module401may further process the keywords to remove certain keywords that are not relevant. In one embodiment, keyword extraction module401segments the keywords, determines the word frequency, and/or performs a term frequency-inverse document frequency (TF-IDF) operation on the segmented keywords. As a result, certain unrelated words may be removed. For example, words that occur more often (e.g., “a,” “the,” “of,” etc.) may be removed. Certain words that are related to time or a geographic location (e.g., “San Jose”) may also be removed.

Based on keywords611-613, QC scoring module403determines QC similarity score621based on the keywords611and/or other metadata associated with the search query601and the content item602, using QC scoring algorithm or model412. QI scoring module404determines QI similarity score622based on the keywords612and/or other metadata associated with the search query601and the image603, using QI scoring algorithm or model413. CI scoring module405determines CI similarity score623based on the keywords613and/or other metadata associated with the content item602and the image603, using CI scoring algorithm or model414. Based on the QC, QI, and CI similarity scores, image ranking module406determines ranking score625for image603to be paired with content item602, for example, using ranking algorithm or model415. All of the image candidates are then ranked based on their respective ranking scores. These similarity scores are also referred to as two-dimensional (2D) scores, since they are determined based on two set of keywords associated with two parties.

In one embodiment, in determining a similarity score, a probabilistic latent semantic analysis (PLSA), as part of scoring algorithms412-414, is performed on the corresponding sets of keywords. PLSA, also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles), is a statistical technique for the analysis of two-mode and co-occurrence data. In effect, one can derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables, just as in latent semantic analysis, from which PLSA evolved. Probabilistic latent semantic analysis is based on a mixture decomposition derived from a latent class model.

In one embodiment, referring back toFIG. 6, an individual ranking score (e.g., a 3D score) for each of search query601, content item602, and image603is determined based on the 2D similarity scores. For example, an individual ranking score can be calculated based on a product of two 2D similarity scores. The ranking score for search query601is determined based on QC sore621and QI score622. In one embodiment, the ranking score for search query601is calculated by:
Squery=QCδ1*QIδ2
where QC and QI represent QC and QI similarity scores. δ1and δ2are coefficients or weight factors associated with search query601and content item602, which may be greater than 1.

In one embodiment, the ranking score for content item602is determined based on QC score621and CI score623. In one embodiment, the ranking score for content item602is calculated by:
Scontent=QIδ1*CIδ3
where QC and CI represent QC and CI similarity scores. δ1and δ3are coefficients or weight factors associated with search query601and image603, which may be greater than 1.

In one embodiment, the ranking score for image603is determined based on QI score622and CI score623. In one embodiment, the ranking score for image603is calculated by:
Simage=IQδ2*CIδ3
where QI and CI represent QI and CI similarity scores. δ2and δ3are coefficients or weight factors associated with content item602and image603, which may be greater than 1.

In one embodiment, the final ranking score for matching content item602with image603in view of search query601is determined based on the individual ranking scores for search query601, content item602, and image603. In a particular embodiment, the final ranking score is calculated based on the followings:
Sfinal=(Squery+Scontent+Simage)/3

According to another embodiment, the algorithms and ranking mechanism described above can also be utilized to evaluate the prior matchings amongst the content items and images in response to certain search queries to determine whether the matchings are performed properly. Such evaluation can be utilized to adjust the matching methods for subsequent matchings.

FIG. 7is a block diagram illustrating an example of a data analytics system for evaluating the matchings between content and images according to one embodiment of the invention. Referring toFIG. 7, data analytics system150includes keyword extraction module401, QC scoring module403, QI scoring module404, CI scoring module405, and evaluation module406. Modules401and403-406may have the same or similar functionalities as the corresponding modules as shown inFIG. 4. For the purpose of illustration, the same reference numbers are maintained. Modules401and403-406perform the same or similar operations as those described with respect toFIG. 6. For example, modules401and403-406perform the same or similar operations associated withFIG. 6for evaluating matchings of content items and images based on matching information701of known matched content items and images in response to known search queries, using proper scoring algorithms412-415as part of evaluation methods702. The evaluation score is utilized determine the quality of prior matching of content items and images. If a particular individual ranking score is below a predetermined threshold, it indicates that there may be an issue with respect to the corresponding party. For example, if the ranking score for the image is too low, that image needs to be improved for subsequent matching.

FIG. 8is a flow diagram illustrating a process for ranking images for matching with content items according one embodiment of the invention. Process800may be performed by processing logic which may include software, hardware, or a combination thereof. For example, process800may be performed by system400ofFIG. 4. Referring toFIG. 8, at block801, processing logic identifies a content item a list of one or more images as image candidates based on one or more keywords of a search query. For each of the images, at block802, processing logic calculates a first similarity score based on metadata associated with the search query and the content item. At block803, processing logic calculates a second similarity score based on metadata associated with the search query and the image. At block804, the processing logic calculates a third similarity score based on metadata associated with the content item and the image. At block805, processing logic determines a ranking score for the image to be matched with the content item based on the first similarity score, the second similarity score, and the third similarity score. At block806, one of the images is selected based on the ranking scores to be matched with the content item.

FIG. 9is a flow diagram illustrating a process for evaluating matchings of content items and images according one embodiment of the invention. Process900may be performed by processing logic which may include software, hardware, or a combination thereof. For example, process900may be performed by system700ofFIG. 7. Referring toFIG. 9, at block901, processing logic receives a request for evaluating matching of a content item and an image that are identified in response to a search query. At block902, processing logic calculates a first similarity score based on metadata associated with the search query and the content item. At block903, processing logic calculates a second similarity score based on metadata associated with the search query and the image. At block904, the processing logic calculates a third similarity score based on metadata associated with the content item and the image. At block905, processing logic determines an evaluation score for the image matched with the content item based on the first similarity score, the second similarity score, and the third similarity score.

The techniques described above can be applied to matching images with sponsored content. One type of the sponsored content is advertisement (Ads). For example, referring back toFIGS. 1A-1B, content database (DB) or server133may be an Ads database or Ads server. Each of the content items, e.g., Ads, is associated with a list of predefined keywords, terms, phrases, or sentences. These predefined keywords, terms, phrases, or sentences may be bidwords purchased, defined or specified by an Ads provider. In another embodiment, primary content DB130may store the general content generally available in a public network. Auxiliary content DB131may be an Ads DB. Some of the Ads may be just plain texts. By matching and integrating an image with the Ads, for example, as a background image, the Ads may be more appealing or attractive to users. Server104may be a Web server for searching content or alternatively, server104may be an Ads server.

FIG. 10is a block diagram illustrating an example of a data processing system which may be used with one embodiment of the invention. For example, system1500may represents any of data processing systems described above performing any of the processes or methods described above, such as, for example, a client device or a server described above, such as, for example, client devices101-102, server104, content server133, analytics system/server150, as described above.

System1500can include many different components. These components can be implemented as integrated circuits (ICs), portions thereof, discrete electronic devices, or other modules adapted to a circuit board such as a motherboard or add-in card of the computer system, or as components otherwise incorporated within a chassis of the computer system.

Note also that system1500is intended to show a high level view of many components of the computer system. However, it is to be understood that additional components may be present in certain implementations and furthermore, different arrangement of the components shown may occur in other implementations. System1500may represent a desktop, a laptop, a tablet, a server, a mobile phone, a media player, a personal digital assistant (PDA), a Smartwatch, a personal communicator, a gaming device, a network router or hub, a wireless access point (AP) or repeater, a set-top box, or a combination thereof. Further, while only a single machine or system is illustrated, the term “machine” or “system” shall also be taken to include any collection of machines or systems that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

Module/unit/logic1528, components and other features described herein can be implemented as discrete hardware components or integrated in the functionality of hardware components such as ASICS, FPGAs, DSPs or similar devices. In addition, module/unit/logic1528can be implemented as firmware or functional circuitry within hardware devices. Further, module/unit/logic1528can be implemented in any combination hardware devices and software components.

The techniques shown in the figures can be implemented using code and data stored and executed on one or more electronic devices. Such electronic devices store and communicate (internally and/or with other electronic devices over a network) code and data using computer-readable media, such as non-transitory computer-readable storage media (e.g., magnetic disks; optical disks; random access memory; read only memory; flash memory devices; phase-change memory) and transitory computer-readable transmission media (e.g., electrical, optical, acoustical or other form of propagated signals—such as carrier waves, infrared signals, digital signals).