Content keyword identification

In general, in one aspect, a method includes compiling user interaction statistics for a set of content items displayed in association with a first target media document having a non-textual portion, at least some of the content items associated with one or more keywords, based on the interaction statistics, associating the first target media document with at least some of the keywords associated with the content items, and based on a common attribute of the first target media document and a second target media document having a non-textual portion, associating the second target media document with at least some of the keywords assigned to the first target media document. Other aspects include corresponding systems, apparatus, and computer programs stored on computer storage devices.

BACKGROUND

This specification relates to information presentation and identification of keywords associated with content items.

Information can be transmitted over a network and displayed on a user device. Sometimes the information is textual, for example, a web page containing text. The text of the web page can be analyzed to determine information about the web page such as topics that relate to the web page. For example, a web page containing several instances of particular words could be described as relating to those words. Sometimes the transmitted information does not contain much text, for example, a web page containing non-textual media such as audio and video and may include little or any associated text. In these examples, other cues could be used to determine topics related to the information.

SUMMARY

In general, in one aspect, a method includes compiling user interaction statistics for a set of content items displayed in association with a first target media document having a non-textual portion, at least some of the content items associated with one or more keywords, based on the interaction statistics, associating the first target media document with at least some of the keywords associated with the content items, and based on a common attribute of the first target media document and a second target media document having a non-textual portion, associating the second target media document with at least some of the keywords assigned to the first target media document. Other embodiments of this aspect include corresponding systems, apparatus, and computer programs stored on computer storage devices.

These and other embodiments can each optionally include one or more of the following features. The aspect includes selecting a content item to display in association with at least one of the first target media document and the second target media document based on the keywords assigned to the respective target media document. The user interaction statistics include at least one of a click-through rate of the content item when the content item is displayed in association with the first target media document and a conversion rate of the content item when the content item is displayed in association with the first target media document. The common attribute includes at least one of a source of the target media documents, a content channel of the target media documents, and a single search query associated in a search system with the target media documents. The common attribute includes a media document topic common to the target media documents. The media document topic is identified based on non-textual content of the first target media document and non-textual content of the second target media document. The media document topic is received from a database associating media document topics with media documents. The media document topic includes at least one of a location at which the target media documents were created and an entity portrayed in the target media documents.

Particular embodiments of the subject matter described in this specification can be implemented so as to realize none, one or more of the following advantages. Keywords can be associated with non-textual content. Multiple items of non-textual content can be related to each other according to keywords.

DETAILED DESCRIPTION

Content items (e.g., advertisements or “ads”) can be chosen for display alongside online content based in part on keywords that have been associated with the content items and are also related to the online content. For example, a content delivery service can select content to display alongside the web page by identifying keywords that appear in the text of the web page. This technique is less effective for online content that is primarily non-textual, for example, online content that includes audio and/or video. To improve the effectiveness of the selection of content for delivery along with non-textual online content, the content management service can compile statistics about the way in which users interact with various content items displayed with the non-textual online content. For example, content items that have a high click-through rate or a high conversion rate can be analyzed to determine which keywords those content items have in common, indicating that the keywords are likely to be good matches for the non-textual online content. Those keywords can then be associated with the non-textual online content so that other content items associated with the keywords can be displayed alongside the non-textual content. Further, multiple items of online content having attributes in common can be associated with the same keywords based on this technique.

FIG. 1is a block diagram of an online environment100for presenting and analyzing information. The online environment100facilitates the delivery of media documents102to user devices106using a network108. For example, a user device106can display media documents102using software such as a web browser122. The media documents102may include non-textual content, for example, video, images, audio, or other non-textual content.

Media documents102can be provided by an online resource such as a media service110. A media service110includes functionality for using the network108to facilitate the transfer of data. For example, the media service110can include a web server that makes media documents102available on the network108. The media documents102can be prepared by the media service110from stored media documents114available to the media service110(e.g., residing in data storage available to the media service110). In some examples, the media documents102can be provided by the media service110in the form of a web page suitable for display on the web browser122of a user device106. The media documents102may be viewed on the web browser122by a user120of the user device106.

Content items104can be provided by another online resource such as a content delivery service112. In some implementations, the content items104are advertisements. A content delivery service112includes functionality for using the network108to facilitate the transfer of data. For example, the content delivery service112can include a web server that provides content items104using the network108. The content items104can be prepared or identified by the content delivery service112from stored content items118available to the content delivery service112(e.g., residing in data storage available to the content delivery service112). In some examples, the content items104can be provided by the content delivery service112in a form suitable for display on the web browser122of a user device106. The content items104may be viewed on the web browser122by a user120of the user device106.

A content item104can be provided in association with a media document102. In some examples, one or more content items104can be associated with a media document102, and some or all of the associated content items104can be provided for display by the content delivery service112when the media document102is provided for display by the media service110. For example, if the media document102is a video file provided to a user device106for display on a web browser122, then one or more of the content items104can be provided to the user device106for display in the web browser122either coincident with, before, or after the video file is displayed to a user120. The content items104could be advertisements that relate to the video file, for example. In some examples, the content items104could be provided by the content delivery service112to the media service110, and the media service110provides the content items104to a user device106at the same time as the media service110provides the media document102to the user device106. In some examples, the content delivery service112provides the content items104to a user device106(for example, at the same time as the media service110provides the media document102).

In some examples, media documents102are provided to user devices106in response to a direct request for a particular media document102. For example, a user120of the user device106may click on a hyperlink directed to the media document102, and the media service110provides the media document102in response. In some examples, media documents102are provided to user devices106in response to other actions. For example, the media service110may include search functionality. The user120of the user device106may submit a search query125to the media service110, and in turn the media service can provide the media document102(or a hyperlink to the media document102) in response to the search query125.

The content delivery service112can receive usage data126from the media service110. In some examples, the usage data126indicates information about the way in which users (e.g., the user120) have interacted with content items104provided for display in association with media documents102displayed on user devices106. In some examples, the usage data126can indicate how many times the content items104have been displayed and which media documents102they have been displayed with or alongside. In some examples, if the content items104include clickable hyperlinks (e.g., hyperlinks that can be clicked or otherwise selected on a user interface), the usage data126can indicate how many times the content items104have been clicked/selected. In some examples, the usage data126includes information about search queries (e.g., the search query125) submitted by user devices106to which one or more of the media documents102provided to user devices106were responsive.

In some examples, the content delivery service112can determine some usage data126independently of other entities such as the media service110. In some examples, the content delivery service112can receive some or all of the usage data126directly from user device106. In some examples, usage data126can be received from other entities. For example, if a content item104is an advertisement provided on behalf of a third party advertiser, the content item104may include a link (e.g., a hyperlink displayed on a user device106) to a resource of the third party advertiser such as a web site of the third party advertiser. The web site of the third party advertiser may provide usage data126to the content delivery service112. For example, the web site may provide information about conversions achieved by the advertisement. In some examples, usage data126is provided to the media service110by a third party entity, and the usage data126is in turn provided to the content delivery service112.

The content delivery service112can use usage data126received from one or more sources to compile usage statistics128summarizing the usage data126. For example, the usage statistics128can include how frequently a content item104is interacted with (e.g., clicked) relative to how often it is displayed, e.g., a click-through rate. The usage statistics128can include how frequently a content item104achieves a conversion, e.g., if the content item is an advertisement. The usage statistics128can include which media documents102a content item104is displayed with. The usage statistics128can include sources of media documents102displayed with content items104, e.g., who created the media document102or who uploaded a media document102to the media service110. The usage statistics128can include content channels of media documents102associated with a content item104, e.g., if the media documents102are videos and organized by the media service110as video channels. The usage statistics128can include search queries (e.g., the search query125) used to access media documents102displayed with content items104.

In some implementations, the content delivery service112is operated by the same entity as the media service110. For example, the content delivery service112can be operated by a company or organization that also operates the media service110. In some implementations, one entity operates the content delivery service112and another different entity operates the media service110.

The content delivery service112stores association data124that specifies relationships between types of data. One type of association specified by the association data124is relationships between stored content items118and keywords130. The keywords130are individual words or groups of words that indicate topics that are associated with or related to (e.g., semantically) the content items118. Words or phrases can be semantically related to a content item if they describe a topic of the content item or otherwise relate to content of the content item. For example, a content item118that is an advertisement for a tropical travel destination may be semantically related to keywords such as “vacation,” “beachfront hotel,” “ocean,” and other keywords that may describe the tropical travel destination. The association data124specifies which content items118are related to which keywords130. In some implementations, a keyword130is determined to be related to a content item118if the keyword130appears in textual content of the content item118, or if a synonym for the keyword130(or a portion of the keyword13) appears in textual content of the content item118.

In some examples, the association data124specifying relationships between the content items118and keywords130is data that defines an explicit association between individual content items and individual keywords. For example, the association data124can include a database of keywords130which are linked to particular content items118. In some examples, the association data124specifying relationships between the content items118and keywords130is data that can be used to compute keywords that are associated with content items and content items that are associated with content items. For example, the association data124may specify an algorithm that can accept selected content items118as input and identify keywords130as output (e.g., based on textual or non-textural content of the content items118).

Generally, the association between content items118and keywords130can be used by the content delivery service112to choose a content item118for display. For example, if a content item118is an advertisement, then the advertisement may be chosen for display in a slot on a web page. Keywords relevant to the web page can be identified and advertisements associated with the keywords can be chosen for display. One way to identify keywords relevant to a web page is to identify words that appear in the web page (for example, words which appear frequently in the web page, or proper names that appear in the web page), and use those words as keywords. If the content delivery service112is choosing content items118for display alongside non-textual content, then the non-textual content (e.g., media documents containing little or no text) may be associated with keywords130in other ways.

In some examples, the association data also specifies associations between keywords130and media documents114. In some examples, the association data124includes an association of media document references132with the keywords130. In some implementations, the media document references132each describe one of the media documents114stored by the media service110. For example, the media document references132could be uniform resource locators specifying a location of media documents114available on the network108, or the media document references132could be document names used by the media service110to identify individual media documents114, or the media document references132could be another kind of reference to the media documents114.

The media document references132are associated with keywords130that are related to the underlying media documents114. For example, a media document reference132could reference a video file, and the keywords130associated with the media document reference132by the association data124could describe the content of the video file. If the video file is a video of a vacationer's day at the beach, then keywords130that might describe the content of the video file could be “vacation,” “beach,” “ocean,” and other words or phrases that may relate to places, events, and other topics depicted in the video file.

If the media document references132identify media documents114which contain little or no text (for example, if the media documents114are videos), usage statistics128can be used to determine keywords130that are semantically related to the media document references132(and thus the underlying media documents114).

FIG. 2shows the content delivery service112that can be used to determine keywords130to associate with a media document102based on usage statistics128. In some examples, a content item104is displayed alongside a media document102in a web browser122to one or more users120on many occasions over time. These occasions are reflected in the usage statistics128. For example, the content item104can be an advertisement and the media document102can be a video file. The content item104is associated with one or more keywords130. If the content item104is interacted with frequently when displayed with the media document102(e.g., the advertisement has a high click-through rate), then the keywords130associated with the content item104may be relevant to the media document102, because the user120viewing the media document102appears to be interested in the content item104as well. Thus, the content delivery service112can infer that the keywords130may be relevant to the media document102. The keywords130may be more likely to be relevant to the media document102if other content items104asharing some or all of the keywords130are displayed with the same media document102and also have high click-through rates. The content delivery service112can use this information to associate keywords130with media documents102. In addition to click-through rate, the content delivery service112can also determine that keywords130of a content item104may be relevant to a media document102if the content item104has a high conversion rate when displayed with the media document102.

The content delivery service112can use an inference made about one media document102to infer similar information about a related media document102a. In some examples, if the media documents102,102awere created or uploaded by a particular entity, such as a particular one of the users120, then the media documents102,102amay be related in content. The content delivery service112can identify keywords inferred as relevant to one media document102and associate the keywords with another (e.g., the other) media document102a. For example, a user who uploads a video related to his summer vacation on the beach may upload another video from the same vacation. Thus, keywords associated with one video, such as “vacation” and “beach,” are may be relevant to the other video. In some examples, if the media documents102,102abelong to the same content channel (e.g., as designated by the media service110shown inFIG. 1), then the media documents102,102amay be related in content. For example, if one media document102belongs to a content channel for vacation videos, then keywords describing the media document102may be relevant to other media documents in the vacation videos channel. In some examples, if the media documents102,102awere retrieved by the same search query (e.g., the search query125shown inFIG. 1), then the media documents102,102amay be related in content. For example, if a user120submitted a search query of “vacation videos” and was provided the first media document102in response, and then the user120submitted the same search query of “vacation videos” and was provided the second media document102ain response, then the media documents102,102amay be related in content. Any common attribute of media documents102,102acould be used to identify keywords associated with one media documents102and infer that the keywords should be associated with the second media document102a.

In some examples, deducing that keywords are relevant to a media document includes associating a score with the keywords. In some examples, referring toFIG. 1, for each keyword130associated with a media document reference132, the association can be assigned or characterized by a score. For example, if the content delivery service112has determined that a keyword130is very likely (e.g., above a likeliness threshold) to be related to a media document referenced by a media document reference132, then the content delivery service112may give a high score to the association between the keyword130and the media document reference132. If the content delivery service112has determined that a keyword130is marginally likely (e.g., below one likeliness threshold and above a lower likeliness threshold) to be related to a media document referenced by a media document reference132, then the content delivery service112may give a low score to the association between the keyword130and the media document reference132. The score can be used, for example, to set a cost for displaying a content item118with the underlying media document114. For example, a content item118chosen for display with a media document114based on a high-scoring association between the media document reference132and the keyword130used to choose the content item118can be charged a high price, and a content item118chosen for display with a media document114based on a low-scoring association between the media document reference132and the keyword130can be charged a low price.

Referring toFIG. 1, in some implementations, the content delivery service112can consult a media document topic engine134to identify a media document topic of a media document114referenced by a media document reference132. In some examples, a media document topic engine134is a database or other facility that processes media documents (e.g., media document114) and identifies topics (e.g., topics136) associated with the media document. For example, if the media document114is a video of a race car driving along a waterfront road, then the media document topic engine134may process the media document114and return topics136such as “automotive” and “beach.” The media document topic engine134may determine topics136from attributes of the media document114. For example, the media document topic engine134may identify subjects depicted in the media document114by analyzing the data of the media document114. In the example where the media document114is a video depicting a race car driving along a waterfront road, the media document topic engine134may use computer vision techniques to identify an automobile in the video data and return a topic136of “automotive.” Further, the media document topic engine134may examine metadata associated with the video identifying a geographic location at which the video was recorded to determine that the geographic location is along a waterfront and return a topic136of “beach.” In some examples, the media document topic engine134may be a component of the media service110(e.g., operated by the media service110), a component of the content delivery service112, or the media document topic engine134may be operated by another entity in communication with the content delivery service112.

In some examples, the content delivery service112can use topics136returned by the media document topic engine134to identify media documents114that are related. For example, one media document (e.g., the media document102shown inFIG. 2) that is related to a topic may be related to another media document (e.g., the media document102ashown inFIG. 2). The content delivery service112can infer that keywords130associated with the first media document102can also be associated with the second media document102abecause the media documents102,102ashare a topic.

In some examples, the content delivery service112can use topics136returned by the media document topic engine134to identify keywords associated with the media document114. In some examples, the content delivery service112may categorize keywords130under topics. For example, keywords such as “race car,” “driving,” and “car dealerships” may be categorized under the topic of “automotive.” The content delivery service112can identify keywords130associated with a topic136and associate those identified keywords130with a media document114identified as associated with the topic136by the topic engine134.

FIG. 3shows a flowchart of an example process300for associating keywords with media documents. The process300can be performed, for example, by the content delivery service112shown inFIG. 1, or could be performed by other systems or apparatus.

User interaction statistics are compiled (302) based on usage data received from one or more sources. For example, the content delivery service could compile the user interaction statistics. In some examples, the usage data indicates information about content items being displayed in association with media documents. The media documents can each have a non-textual portion, and the content items can be associated with keywords. In some examples, the user interaction statistics include information such as a click-through rate of content items when the content items are displayed in association with media documents, or information such as a conversion of content items when the content items are displayed in association with media documents.

A target media document is associated (304) with keywords. For example, the content delivery service could associate the target media document with keywords. In some examples, the keywords can be associated with a content item, and the keywords of the content item can be determined to be relevant to the target media document based on the user interaction statistics. For example, keywords relevant to the target media document can be inferred from the user interaction statistics.

A second target media document is associated (306) with keywords. For example, the content delivery service could associate the second target media document with keywords. In some examples, the keywords can be associated with the second target media document based on a common attribute of the first target media document and a second target media document. For example, the common attribute could be a source of the target media documents, or a content channel of the target media documents, or a single search query associated in a search system with the target media documents, or a media document topic of the target media documents. In some implementations, the media document topic can be identified based on non-textual content of the first target media document and non-textual content of the second target media document. The media document topic could be a location at which the target media documents were created or an entity portrayed in the target media documents. Also, the media document topic could be received from a database associating media document topics with media documents.

A content item is selected (308) for display. For example, the content delivery service could select the content item for display. In some examples, the content item is selected for display in association with a first target media document or a second target media document based on keywords assigned to the respective target media document. For example, if one of the target media documents is associated with a particular keyword, then a content item also associated with that keyword can be selected for display with the target media document.

FIG. 4is block diagram of an example computer system400that can be used to facilitate information presentation and identification of keywords associated with content items. The system400includes a processor410, a memory420, a storage device430, and an input/output device440. Each of the components410,420,430, and440can be interconnected, for example, using a system bus450. The processor410is capable of processing instructions for execution within the system400. In one implementation, the processor410is a single-threaded processor. In another implementation, the processor410is a multi-threaded processor. The processor410is capable of processing instructions stored in the memory420or on the storage device430.

The storage device430is capable of providing mass storage for the system400. In one implementation, the storage device430is a computer-readable medium. In various different implementations, the storage device430can include, for example, a hard disk device, an optical disk device, or some other large capacity storage device.

The input/output device440provides input/output operations for the system400. In one implementation, the input/output device440can include one or more of a network interface devices, e.g., an Ethernet card, a serial communication device, e.g., an RS-232 port, and/or a wireless interface device, e.g., and 802.11 card. In another implementation, the input/output device can include driver devices configured to receive input data and send output data to other input/output devices, e.g., keyboard, printer and display devices460. Other implementations, however, can also be used, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, although examples of non-textual content are used, the techniques described could be used to associate keywords with textual content, for example, a word or a sequence of words. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.