Method and system for searching and identifying content items in response to a search query using a matched keyword whitelist

According to one embodiment, content keyword elements (CKEs) are determined based on content keywords (CKs) associated with content items. Query keyword elements (QKEs) are determined based on query keywords (QKs) associated with search queries. A first search query having one or more query keywords represented by a first QKE. A first content keyword represented by a first CKE that matches the first QKE is identified. A new entry in a query-to-CK (query/CK) whitelist table to map the first search query to the first content keyword. The query/CK whitelist table includes a number of entries, each entry mapping a search query to one or more content keywords. The query/CK whitelist table is utilized to identify content items in response to a subsequent search query.

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to searching content. More particularly, embodiments of the invention relate to searching content using a matched keyword whitelist in response to a search query.

BACKGROUND

Most search engines typically perform searching of Web pages during their operation from a browser running on a client device. A search engine receives a search term entered by a user and retrieves a search result list of Web pages associated with the search term. The search engine displays the search results as a series of subsets of a search list based on certain criteria. General criteria that is used during a search operation is whether the search term appears fully or partly on a given webpage, the number of times the search string appears in the search result, alphabetical order, etc. Further, the user can decide to open a link by clicking on the mouse button to open and browse. Some of the user interactions with the search results and/or user information may be monitored and collected by the search engine to provide better searches subsequently.

Typically, in response to a search query, a search is performed to identify and retrieve a list of content items. The content items are then returned to a search requester. A conventional search engine would return most of the content items as is without modifying. Some of the content items in the search result are just plain text or description, which may deem to be unattractive or boring. Sometimes content in a search result would be more presentable or attractive if the content items are prepared with certain images that are related to the content items. However, it is quite challenge to match appropriate images with content items.

DETAILED DESCRIPTION

According to some embodiments, a query-to-content keyword whitelist table is created to map certain search queries to certain content keywords associated with certain content items, based on an analysis of a set of known queries and known content items. Such tables can be created based on historic known search queries and content items offline. Thus, in addition to a normal query keyword (QK) to content keyword (CK) mapping or index table (QK/CK mapping table), the query/CK whitelist table provides additional whitelist mapping utilized to identify a list of content items in response to a search query in real-time or online.

In one embodiment, content keyword elements (CKEs), also referred to as content keyword features, are extracted from one or more CKs associated with each of content items in a known set. Query keyword elements (QKEs), also referred to as query keyword features, are extracted from one or more QKs associated with each of search queries in a known set. A first search query is identified that has one or more QKs represented by a first QKE of the QKEs. A first search query is identified that includes one or more QKs represented by a first QKE. A first CK represented by a first CKE is identified, where the first CKE matches the first QKE. A new mapping entry is created in the query/CK whitelist table to map the first search query to the first CK. The query/CK whitelist table is utilized, in addition to the QK/CK mapping table, to map a specific search query to one or more content keywords. The content keywords are utilized to identify one or more content items, which will be returned as part of a search result in response to the search query online.

In one embodiment, in determining CKEs, the content keywords are segmented into content sub-keywords, each content sub-keyword represents a CKE. An analysis such as a term frequency-inverse document frequency (TF-IDF) is performed on the content sub-keywords to remove one or more unrelated or the ones that appear more frequently. A latent semantic analysis may also be performed on the content sub-keywords to identify a set of similar content sub-keywords. A CKE index table is created to map a content keyword element to one or more content keywords, where the CKE index table is utilized to identify a content keyword based on a content keyword element. A QKE index table is created similarly. The CKE index table and the QKE index table are utilized to identify the common matched keyword elements to compile the query/CK whitelist table.

FIGS. 1A and 1Bare block diagram illustrating an example of system configuration for matching images with content items according to some embodiments of the invention. Referring toFIG. 1A, system100includes, but is not limited to, one or more client devices101-102communicatively coupled to server104over network103. Client devices101-102may be any type of client devices such as a personal computer (e.g., desktops, laptops, and tablets), a “thin” client, a personal digital assistant (PDA), a Web enabled appliance, a Smartwatch, or a mobile phone (e.g., Smartphone), etc. Network103may be any type of networks such as a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof, wired or wireless.

Server104may be any kind of servers or clusters of servers, such as Web or cloud servers, application servers, backend servers, or a combination thereof. In one embodiment, server104includes, but is not limited to, search engine120, image selection module110, and query/image matching rules115. Server104further includes an interface (not shown) to allow a client such as client devices101-102to access resources or services provided by server104. The interface may include a Web interface, an application programming interface (API), and/or a command line interface (CLI).

For example, a client, in this example, a user application of client device101(e.g., Web browser, mobile application), may send a search query to server104and the search query is received by search engine120via the interface over network103. In response to the search query, search engine120extracts one or more keywords (also referred to as search terms) from the search query. Search engine120performs a search in content database133, which may include primary content database130and/or auxiliary content database131, to identify a list of content items that are related to the keywords. Primary content database130(also referred to as a master content database) may be a general content database, while auxiliary content database131(also referred to as a secondary content database) may be a special content database. Search engine120returns a search result page having at least some of the content items in the list to client device101to be presented therein. Search engine120may be a Baidu® search engine available from Baidu, Inc. or alternatively, search engine120may represent a Google® search engine, a Microsoft Bing™ search engine, a Yahoo® search engine, or some other search engines.

A search engine, such as a Web search engine, is a software system that is designed to search for information on the World Wide Web. The search results are generally presented in a line of results often referred to as search engine results pages. The information may be a mix of Web pages, images, and other types of files. Some search engines also mine data available in databases or open directories. Unlike web directories, which are maintained only by human editors, search engines also maintain real-time information by running an algorithm on a web crawler.

Web search engines work by storing information about many web pages, which they retrieve from the hypertext markup language (HTML) markup of the pages. These pages are retrieved by a Web crawler, which is an automated Web crawler which follows every link on the site. The search engine then analyzes the contents of each page to determine how it should be indexed (for example, words can be extracted from the titles, page content, headings, or special fields called meta tags). Data about web pages are stored in an index database for use in later queries. The index helps find information relating to the query as quickly as possible.

When a user enters a query into a search engine (typically by using keywords), the engine examines its index and provides a listing of best-matching web pages according to its criteria, usually with a short summary containing the document's title and sometimes parts of the text. The index is built from the information stored with the data and the method by which the information is indexed. The search engine looks for the words or phrases exactly as entered. Some search engines provide an advanced feature called proximity search, which allows users to define the distance between keywords. There is also concept-based searching where the research involves using statistical analysis on pages containing the words or phrases you search for. As well, natural language queries allow the user to type a question in the same form one would ask it to a human.

The usefulness of a search engine depends on the relevance of the result set it gives back. While there may be millions of web pages that include a particular word or phrase, some pages may be more relevant, popular, or authoritative than others. Most search engines employ methods to rank the results to provide the “best” results first. How a search engine decides which pages are the best matches, and what order the results should be shown in, varies widely from one engine to another.

Referring back toFIG. 1A, according to one embodiment, in response to a search query received at server104from a client device, in this example, client device101, search engine120performs a search in content database133, such as primary content database130and/or auxiliary content database131, to generate a list of content items. Each of the content items may be associated with a particular Web page of a particular Web site of a particular content provider via a uniform resource link (URL) and/or a uniform resource identifier (URI). In one embodiment, primary content database130stores general content items that have been collected by network crawlers (e.g., unsponsored content). Auxiliary content database135stores specific or special content items that are associated with specific, known, or predetermined content providers (e.g., sponsored content). Alternatively, content database133may be implemented as a single database without distinguishing primary content database131from auxiliary content database132.

Network crawlers or Web crawlers are programs that automatically traverse the network's hypertext structure. In practice, the network crawlers may run on separate computers or servers, each of which is configured to execute one or more processes or threads that download documents from URLs. The network crawlers receive the assigned URLs and download the documents at those URLs. The network crawlers may also retrieve documents that are referenced by the retrieved documents to be processed by a content processing system (not shown) and/or search engine120. Network crawlers can use various protocols to download pages associated with URLs, such as hypertext transport protocol (HTTP) and file transfer protocol (FTP).

In one embodiment, server104maintains query/CK whitelist table161and keyword to content (keyword/content) mapping table162. Tables161-162may be created by data analytics system or server150, which will be described in details further below. Query/CK whitelist table161includes a number of mapping entries, each mapping entry mapping at least a portion of a search query or query keywords to one or more content keywords. Keyword/content table162includes a number of mapping entries, each mapping entry mapping one or more keywords (e.g., content keywords) to one or more content items or content identifies (IDs) identifying the content items.

In response to a search query, for example, received from any of clients101-102over network103, a search term or keyword is extracted from the search query. In addition, an analysis is performed on the query to determine one or more keywords associated with the query. The determined keywords may be the keywords included in the search query. The determined keywords may further include certain keywords that are semantically similar or have the same meaning of the keywords originally in the search query (e.g., synonymous words or phrases).

The search term or keywords are then used to look up in query/CK whitelist table to locate a mapping entry that matches the search term and to obtain one or more content keywords from the matching entry. The content keywords are then utilized to look up in keyword/content mapping table162(also referred to as a content index table) to obtain one or more content IDs. Based on the content IDs, a search is then performed in content database133to retrieve the related content items (e.g., sponsored content). Thereafter, a search result having at least some of the content items is then compiled and transmitted to the client.

In one embodiment, at least query/CK whitelist table161is created by data analytics system150offline based on a set of known queries and content items, for example, using a variety of machine-learning algorithms or models. Data analytics system150includes, but is not limited to, keyword extraction module181, keyword element processing module182, and query/CK whitelist processing module183. Keyword extraction module181is to extract one or more keywords from a set of known search queries and a set of content items (e.g., Ads). Keyword element processing module182is to process the extract keywords to generate keyword elements. Query/CK processing module183is to compile a query/CK whitelist table, which becomes query/CK whitelist table161utilized online. Modules181-183will be described in details further below.

In addition, according to one embodiment, server104further includes an image selection module or system (not shown) to search and identify one or more mages in an image store based on the keywords associated with the search query that are related to the search query and/or the content items. In one embodiment, the images and their respective metadata stored in the image store may be obtained by one or more image crawlers that are designed to craw the networks to collect images as well as their surrounding metadata. The images may be non-privileged, non-copyrightable, properly licensed images, or any other authorized images. For each of the content items found in content database133, an image is selected from the list of image candidates to be associated with the content item. The selected image may be incorporated with the content item to generate an incorporated content item. For example, the selected image may serve as a background image to the content item. The list of incorporated content items and images is returned to the client device as part of a search result.

In one embodiment, each of content items of a content database or content server is associated with a set of one or more keywords (or phrases, sentences). Such associated keywords, phrases, or sentences may be configured or specified by a content provider of the corresponding content item. The keywords, phrases, or sentences may be used for a searching purpose based on a search term for the purpose of identifying the corresponding content item in the content database or content server.

Note that the configuration of server104has been described for the purpose of illustration only. Server104may be a Web server to provide a frontend search service to a variety of end user devices. Alternatively server104may be an application server or backend server that provides specific or special content search services to a frontend server (e.g., Web server or a general content server), and to match and/or integrate images with content items of a content database or server. Data analytics system150may also be implemented as a separate server.

Other architectures or configurations may also be applicable. For example, as shown inFIG. 1B, content database133may be maintained and hosted in a separate server as a content server over a network. Content database/server133may include primary content database130and auxiliary content database131. Primary content database130may also be implemented or maintained in a separate content server, referred to as a primary content server. Similarly, auxiliary content database131may be implemented or maintained in a separate content sever, referred to as an auxiliary content server. Server133may be a Web server, an application server, or a backend server. Content server133and/or an image server may be organized and provided by the same entity or organization as of server104. Alternatively, content server133and/or the image server may be maintained or hosted by separate entities or organizations (e.g., third-party providers).

FIG. 2is a block diagram illustrating an example of a content searching system according one embodiment of the invention. System200may be implemented as part of system100as shown inFIGS. 1A-1B. Referring toFIG. 2, search engine120includes, but is not limited to, keyword extraction module201, CK lookup module202, and search module203. In response to search query211, keyword extraction module201extracts one or more keywords or phrases from search query211. In addition, keyword extraction module201may perform an analysis, such as a latent semantic analysis, on the keywords to derive one or more similar or semantically related keywords or phrases (e.g., synonyms) to be added to the set of extracted keywords. Based on the keywords or phrases, CK lookup module202performs a lookup operation in query/CK whitelist table161to identify one or more content keywords.

FIG. 3is a block diagram illustrating an example of query/CK whitelist table according to one embodiment. Query/CK whitelist table300includes a number of mapping entries. Table300may represent query/CK whitelist table161. Referring toFIG. 3, each mapping entry maps one or more keywords or phrase301of a search query (e.g., previously known query) to one or more content keywords302. Based on one or more keywords of the search query (e.g., search terms), CK lookup module202searches and find a mapping entry having field301matching the keywords of the search query. Content keywords302of the matching entry are obtained.

Based on the content keywords, for example, obtained via query/CK whitelist table161, CK lookup module202searches in keyword/content mapping table162to identify one or more content IDs.FIG. 4is a block diagram illustrating an example of a query/content mapping table according to one embodiment of the invention. Query/content table400may represent keyword/content mapping table162. Query/content table400includes a number of mapping entries, each entry mapping one or more keywords401to one or more content IDs402that identify one or more content items. The content items may be the sponsored content items such as Ads and the keywords may be bidwords purchased by an advertiser as a content provider. Based on the content IDs, search module203searches in content database133to obtain a list of one or more content items (e.g., sponsored content items such as advertisements or Ads). Search result212is then generated including at least some of the content items and the search result is returned to the client.

FIG. 5is a block diagram illustrating an example of a data analytics system according to one embodiment of the invention. System500may be implemented as part of data analytics system or server150. Referring toFIG. 5, system500includes keyword extraction module181, keyword element (KE) processing module182, and query/CK processing module183. These module can be implemented in software, hardware, or a combination thereof. For example, modules181-183can be loaded into a memory and executed by a processor.

In one embodiment, for each of content items501(e.g., sponsored content such as Ads) provided by certain content providers such as advertisers, keyword extraction module181performs an analysis on the content item and/or its associated keywords selected by the corresponding content provider (e.g., bidwords) to determine one or more content keywords or phrases502. Keyword extraction module181may perform a latent semantic analysis on the content keywords to derive additional similar keywords that are related or semantically similar to the original keywords associated with the content item.

For each of the content keywords503, KE processing module182segments content keywords502into sub-keywords. Optionally, KE processing module182calculates the word frequency of each sub-keyword. KE processing module182may also determines the function or meaning of each sub-keyword and orders them accordingly. For example, the sub-keywords may be ordered from high apparent frequency words to low apparent frequency words, verbs and nouns to adjectives and adverbs. The position of the order may be associated with a different weight factor in determining the importance and/or relevancy of the sub-keywords. KE processing module182may further perform a TF-IDF process on the sub-keywords to remove the unrelated or general words (e.g., “a,” “an,” “the,” “and,” etc.) A content keyword element (CKE) index table503(also referred to as a CKE/CK mapping table) is created. CKE index table503includes a number of mapping entries, each mapping entry mapping a CKE to one or more content keywords or phrases (e.g., bidwords). An example of CKE index table503is shown inFIG. 6A.

Similarly, the above processes are performed on a set of queries511to generate query keywords512by keyword extraction module181, which are utilized to generate query keyword element (QKE) index table513(also referred to as a QKE/QK mapping table) by KE processing module182. QKE index table513includes a number of mapping entries, each mapping entry mapping a QKE to one or more query keywords or phrases. An example of QKE index table is shown inFIG. 6B. Based on CKE index table503and QKE index table513, query/CK whitelist processing module183merges the entries of CKE index table503and QKE table513to create query/CK whitelist table161. The merged entries are the entries that have at least one common keyword element from their respective QKEs and CKEs. The query/CK whitelist table161includes a number of mapping entries, each mapping entry mapping a query keyword or phrase to a content keyword or phrase. An example of query/CK whitelist table161is shown inFIG. 3.

Referring now toFIG. 6A, in this example, content keywords601includes “flower delivery,” which may be extracted from a particular one of the known search queries. Content keywords601is analyzed by keyword extraction module181, including performing a latent semantic analysis to derive additional similar or related keywords. The keywords are then processed by CK processing module182to generate content keyword elements602. In this example, content keyword elements602include “one-day flower deliver,” “gift delivery,” and “gift card.” CKE index table603is created to map each of the CKEs611to one or more keywords or phrases612. In this example, keyword element “flower” is mapped to “flower deliver” and “Beijing flower deliver.” Keyword element “gift delivery” is mapped to “flower delivery and “best gift.” Keyword element “gift card” is mapped to “flower delivery” and “Macy's gift card.” CKE index table603may be implemented as part of CKE index table503ofFIG. 5.

Referring now toFIG. 6B, the similar process is performed on query keywords651of a set of known queries. In this example, query keywords include “fast online flower ordering.” Query keywords651is processed to generate query keyword elements652, in this example, including “fast food,” “flower,” and “flower delivery.” CKE index table653is created to map each of the query keyword element661to one or more query keywords or phrases662. CKE index table653may be implemented as part of CKE index table513ofFIG. 5.

QKE index table603and CKE index table653are then merged based on the common keyword elements listed in field611of QKE index table603and field661of CKE index table653. In this example, the common keyword element is “flower.” As a result, each of the corresponding query keywords or phrases662is mapped to each of the corresponding content keywords or phrases612, generating a query/CK whitelist table similar to the one as shown inFIG. 3. The query/CK whitelist table can be utilized online to map a particular search query to one or more content keywords or phrases (e.g., bidwords), which can be utilized to identify and retrieve one or more content items (e.g., sponsored content items such as Ads).

FIG. 7is a flow diagram illustrating a process of generating a query to content keyword whitelist table according to one embodiment of the invention. Process700may be performed by processing logic which may include software, hardware, or a combination thereof. For example, process700may be performed by data analytics system150ofFIGS. 1A and 1B. Referring toFIG. 7, at block701, processing logic determines a set of content keywords (CKs) associated with a set of content items (e.g., sponsored content). At block702, processing logic determines CK elements from the content keywords and creates a CK element index table to map each CK element to one or more content keywords. At block703, processing logic determines a set of query keywords (QKs) associated with a set of known queries. At block704, processing logic determines QK elements from the query keywords and create a QK element index table to map each query element to one or more query keywords. At block705, a query/CK whitelist table is created to map each of the query keywords to one or more content keywords that have a common keyword element.

FIG. 8is a flow diagram illustrating a process of content searching using a query to content keyword whitelist table according to one embodiment of the invention. Process800may be performed by processing logic which may include software, hardware, or a combination thereof. For example, process800may be performed by server104ofFIGS. 1A and 1B. Referring toFIG. 8, at block801, in response to a search query received from a client, processing logic extracts one or more query keywords (QKs) from the search query. At block802, processing logic optionally performs a latent semantic analysis to include additional related or similar keywords. At block703, processing logic looks up in a query/CK whitelist table to identify one or more content keywords based on the query keywords. At block704, processing logic looks up in a CK/content mapping table to identify one or more content IDs based on the content keywords. At block705, processing logic searches in a content database to identify and retrieve one or more content items based on the content IDs. At block706, at least one of the content items is returned to the client.

The techniques described above can be applied to matching images with sponsored content. One type of the sponsored content is advertisement (Ads). For example, referring back toFIGS. 1A-1B, content database (DB) or server133may be an Ads database or Ads server. Each of the content items, e.g., Ads, is associated with a list of predefined keywords, terms, phrases, or sentences. These predefined keywords, terms, phrases, or sentences may be bidwords purchased, defined or specified by an Ads provider. In another embodiment, primary content DB130may store the general content generally available in a public network. Auxiliary content DB131may be an Ads DB. Server104may be a Web server for searching content or alternatively, server104may be an Ads server.

FIG. 9is a block diagram illustrating an example of a data processing system which may be used with one embodiment of the invention. For example, system1500may represents any of data processing systems described above performing any of the processes or methods described above, such as, for example, a client device or a server described above, such as, for example, server104, content server133, content/image mapping system/server150, or image search engine/system/server605, as described above.

System1500can include many different components. These components can be implemented as integrated circuits (ICs), portions thereof, discrete electronic devices, or other modules adapted to a circuit board such as a motherboard or add-in card of the computer system, or as components otherwise incorporated within a chassis of the computer system.

Note also that system1500is intended to show a high level view of many components of the computer system. However, it is to be understood that additional components may be present in certain implementations and furthermore, different arrangement of the components shown may occur in other implementations. System1500may represent a desktop, a laptop, a tablet, a server, a mobile phone, a media player, a personal digital assistant (PDA), a Smartwatch, a personal communicator, a gaming device, a network router or hub, a wireless access point (AP) or repeater, a set-top box, or a combination thereof. Further, while only a single machine or system is illustrated, the term “machine” or “system” shall also be taken to include any collection of machines or systems that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

Module/unit/logic1528, components and other features described herein can be implemented as discrete hardware components or integrated in the functionality of hardware components such as ASICS, FPGAs, DSPs or similar devices. In addition, module/unit/logic1528can be implemented as firmware or functional circuitry within hardware devices. Further, module/unit/logic1528can be implemented in any combination hardware devices and software components.

The techniques shown in the figures can be implemented using code and data stored and executed on one or more electronic devices. Such electronic devices store and communicate (internally and/or with other electronic devices over a network) code and data using computer-readable media, such as non-transitory computer-readable storage media (e.g., magnetic disks; optical disks; random access memory; read only memory; flash memory devices; phase-change memory) and transitory computer-readable transmission media (e.g., electrical, optical, acoustical or other form of propagated signals—such as carrier waves, infrared signals, digital signals).