Patent ID: 12212791

DETAILED DESCRIPTION

Some examples include a system and method for providing supplemental content with airings of programming media. In such examples, a computer system (e.g., such as provided for a network service) operates to determine an inventory of time slots for a given airing. The computer system assigns a supplemental content item to a time slot of the given airing when the given airing is rendered on at least some of a plurality of playback devices. The computer system may structure a tag element to include data that is specific to a third-party, and then communicate a plurality of tags that include the tag elements to playback devices that are (or will) render the primary content of the airing.

Some examples include a computer system and method for operating a network service to provide content. The computer system operates to determine a media resource that a playback device has requested for playback. The playback device may be provided with a set of time-based metadata that is associated with the media resource, where the set of time-based metadata are correlative to a timeline of events in a playback of the media resource, including start time when the playback initiates, the set of time-based metadata identifying a predetermined available time slot during the playback of the media resource. The playback device may be provided with a programmatic resource for retrieving a supplemental content resource and rendering a corresponding supplemental content during the predetermined available time slot.

One or more aspects described herein provide that methods, techniques and actions performed by a computing device are performed programmatically, or as a computer-implemented method. Programmatically means through the use of code, or computer-executable instructions. A programmatically performed step may or may not be automatic.

One or more aspects described herein may be implemented using programmatic modules or components. A programmatic module or component may include a program, a subroutine, a portion of a program, a software component, or a hardware component capable of performing one or more stated tasks or functions. In addition, a module or component can exist on a hardware component independently of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs or machines.

Furthermore, one or more aspects described herein may be implemented through the use of instructions that are executable by one or more processors. These instructions may be carried on a computer-readable medium. Machines shown or described with figures below provide examples of processing resources and computer-readable media on which instructions for implementing some aspects can be carried and/or executed. In particular, the numerous machines shown in some examples include processor(s) and various forms of memory for holding data and instructions. Examples of computer-readable media include permanent memory storage devices, such as hard drives on personal computers or servers. Other examples of computer storage media include portable storage units, such as CD or DVD units, flash or solid state memory (such as carried on many cell phones and consumer electronic devices) and magnetic memory. Computers, terminals, network enabled devices (e.g., mobile devices such as cell phones) are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable media.

The term “media resource” may include any data source from which a primary content can be generated as audio and visual content. According to some examples, a media resource can be accessed from a corresponding media file, media stream or media transmission (e.g., broadcast).

A “primary content” includes, for example, a work (e.g., a television program, a movie, etc.) or a live broadcast. The primary content can be identified by a content identifier, which can identify the content of the work independent of a channel or distribution medium. Other identifiers may also be associated with the primary content, such as a channel or distribution identifier, one or multiple types of group identifiers and one or more multiple types of publication identifiers (e.g., time of publication, publication source, geographic region, distribution channel, etc.). For example, a work may be associated with a series identifier, a season identifier, and an episode identifier. When a work is broadcast, the work may also be associated with an airing identifier, identifying a time, location and organization of the distribution.

System Overview

FIG.1illustrates an example system for providing time-based metadata, according to one or more embodiments. According to an example ofFIG.1, a metadata delivery system (“MDDS100”) is provided as a network service that can communicate with any one of a plurality of playback devices, represented by a playback device40, in order to provide time-based metadata that is synchronized to the playback of primary media content on the device. The time-based metadata can include content and/or programmatic resources for triggering a playback device to retrieve and include supplemental content at preselected times in a duration in which the primary content is played back.

In an implementation ofFIG.1, the MDDS100is implemented as a network service. For example, the MDDS100may be implemented on one or more servers that communicate with playback devices utilized by a population of users. According to some examples, the MDDS100operates to provide individual playback devices with a time-based metadata set105that is specific to a particular airing of a media resource. The time-based metadata set105can include metadata that is renderable on a given playback device to display supplemental or ancillary content during or concurrently with playback of a primary content. Additionally, in some implementations, the metadata set105includes data to enable the playback device40to retrieve metadata content that can be rendered as supplemental or ancillary content. In some variations, the metadata set105may also include executable data, such as instructions or parametric data, for triggering playback devices to perform operations that include retrieval of supplemental content from network services. The metadata set105may be provided from the MDDS100in connection with corresponding airings of media resources originating from different media sources, including media sources that are transmitted to the MDDS100and/or playback device40over different transmission mediums (e.g., broadcast television, cable television via set-top box, streaming subscription service via web-based application) and/or different content delivery platforms (e.g., broadcasts transmitted through head-end services for set-top boxes, broadcasts streamed over Internet connections, on-demand selections, transmissions of requested content over browser or applications operated by users, etc.).

According to some examples, the MDDS100includes one or multiple system APIs110, a metadata manager130, and one or multiple content source interfaces140. The MDDS100can provide one or more multiple system APIs110, so that specific computing platforms and environments are compatible with the MDDS100. Thus, in an example ofFIG.1, the playback device40can operate under any one of multiple possible platforms, in order to receive data from the MDDS100. Additionally, the system API110can be specific to a particular transmission medium and/or content delivery platform. Thus, for example, the MDDS100may construct and make available system API(s)110to provide time-based metadata in connection with an airing transmitted through any of (i) a head-end to a user's set-top box (e.g., television programs), (ii) an over-the-air broadcast, (iii) a content provider (e.g., streaming service) via a network connection to a user's web-based media player, or (iv) a remote source or network site (e.g., video service site accessible via browser). In some variations, the MDDS100provides a metadata set105in connection with stored media files which the user elects to playback on the playback device40. The playback device40may correspond to one of multiple possible types, distinguished by, for example, a transmission medium, content delivery system and/or platform used to receive and render media resources.

In some examples, the constituent data sets of the metadata set105may include a set of time slot parameters119, content parameters121, and one or more payload data set125. The time slot parameters119can reference specific moments during the playback of the corresponding media resource (also termed the “primary content”) which provide a point for synchronization as between a timer for the metadata set105and a timeline of the primary content. In some variations, the time slot parameters119can also define time slots (sometimes referred to as “avails”) which are predetermined durations in the timeline of the primary content during which supplemental content can be displayed.

Accordingly, in some examples, the time slot parameters119identify pre-determined moments relative to a timeline of the corresponding primary content (“primary content timeline” or “PCT119B”). The time slot parameters119may also be used to define pre-determined time slots. Still further, the time slot parameters119can define time slots which are individual associated with a corresponding identifier129(e.g., unique identifier or UID). In some variations, the time slot parameters119can also reference predetermined segments of the primary content where breaks for commercials or other supplemental content may occur. In this way, the timing parameters 1PCT119B) can be used to identify predefined slots and/or events of interest in the playback of primary content (e.g., “avails”). For example, an airing of the record99may be pre-divided into work segments to define specific locations where supplemental content (e.g., advertisements) can be inserted. The work segments can define, for example, a duration of time between designated break points (e.g., where commercial breaks are shown). In this context, the time slot parameters119can be used to identify for example, a start and end time of a particular time slot, as well as a payload data set125for use in the time slot.

The content parameters121can reference events within the rendering of the primary content which are associated with the primary content. By way of example, the content parameters121can reference events such as the appearance of an individual or object in the airing of the primary content. Accordingly, the content parameters121can also reference markers in time, as defined by one or more time slot parameters119.

The payload data set125can include pre-selected content items which are to be displayed concurrently (e.g., as an overlay) and/or sequentially (e.g., as a commercial between segments of an airing). In some examples, the payload data set125can be linked to the time slot parameters119, which dictate relative moments or durations in the timeline119B of the primary content during which the payload data set125is to be rendered or used. In some examples, the payload data set125can be retrieved during playback of a given media resource, so that the payload data set125is selected and rendered at a playback time that is defined by the time slot parameters119. Still further, in some examples, the payload data set125can be linked with the content parameters121of the primary content, such as individuals, locations, objects or other events which occur during the playback of the primary content. The payload data set125may be selective, so as to appear when certain conditions occur (e.g., user input).

The payload data set125can be provided in a variety of forms or data structures. For example, the payload data set125can be structured as an overlay that appears concurrently with the playback of the airing. Alternatively, the payload data set125can appear in the form of a commercial which appear sequentially once a segment of the primary content is rendered.

In some examples, the metadata set105may also include synchronization data127. The synchronization data127can define or otherwise provide a time reference or resource for enabling the playback device40to playback the playback the primary content while synchronizing the rendering of supplemental content provided with or through the metadata set105. In an example ofFIG.1, the MDDS100utilizes the one or more content source interfaces140to receive scheduling data141from one or more content sources. By way of example, the content sources can include content providers (e.g., television stations) who broadcast television programming to set-top boxes or other types of receivers. As an addition or alternative, the content providers can correspond to streaming services, which can broadcast or otherwise make various types of programming available over a network connection (e.g., using an IP protocol). Such content sources can publish schedules when certain airings are being played back.

In some examples, the metadata set105provided by the MDDS100may include programmatic elements, shown as tags111, which trigger the playback device40to perform operations for retrieving and utilizing supplemental content with a corresponding primary content. In one implementation, the playback device40makes the request44to receive the metadata set105, from which the playback device40can extract identifiers (e.g., time slot identifiers119), and the tags111. Thus, the metadata set105may include or package tags111as part of the metadata set105.

In variations, the playback device40makes multiple requests from the system API110during playback of the primary content, and at least some of the responses54may include tags111. In some examples, the service API110can include, or may be coupled with, tag interface logic114. The tag interface logic114may manage inclusion of tags111in responses54to the playback device40. The tag interface logic114may be responsive to requests44received through the system API110in providing the playback device40with one or more tags111, either before or during the playback of the primary content on the playback device40. As described in greater detail, the playback device40may utilize each tag111to retrieve supplemental content from a network source.

In variations, the tag interface logic114may operate as a separate component, rather than as an integrated component of the system API110. Still further, the tag interface logic114, tag selection logic120and tag store database555may reside as a separate and/or independent entity from the MDDS100. In some variations, the MDDS100may provide identifiers (e.g., time slot identifiers119) which the playback device40can use to retrieve tags111from a separate or independent service.

The tag selection logic120provides logic to select tags111for the playback device40. The tag selection logic120can select tags from a tag data store135. The tags111can be individually structured to trigger the playback device40to perform operations to retrieve supplemental content data137from a network service or other third-party content service82. By way of example, individual tags111can be structured as a link or Uniform Resource Locator (URL) to cause a playback component to access a network site. According to some examples, the MDDS100can leverage existing ad networks and services by including, with the metadata set105, tags111that trigger the playback device40to request and receive the supplemental content data137from an ad network or service. As described with some examples, the use of tags111enable the MDDS100to customize or personalize the selection of supplemental content137for users or playback devices, based on a variety of factors such as geographic location, type of device or platform in use, preference of the user and/or other user profile information. The selection of tags111can also be based on other considerations and/or objectives of the MDDS100, such as attributes or characteristics of the media resource or airing, as well as contextual information about the user, playback device or airing.

The metadata manager130maintains a meta-database metadata store116to store metadata for numerous media resources and/or respective airings of media resources. In one implementation, the metadata store116maintains individual media records99, and each media record links metadata set105with a media resource and/or airing of the media resource. According to some examples, the individual media records99each link an airing identifier101and/or media resource identifier103to constituents data sets of the metadata set105.

In an embodiment, the playback device40can be operated by a user to playback a particular airing of a media resource of the user selection. While some examples may provide for the media resource of the airing to be transmitted from the MDDS100to the playback device40, other examples provide for metadata set105to be transmitted from a network service (e.g., the MDDS100operating as an independent network service) that is separate from the source that provides the playback device40with the airing. In one implementation, the playback device40can interface with a primary content source in order to request an airing of primary content85for playback on the playback device40. By way of example, the playback device40can select the particular primary content source80, and signal a channel selection or on-demand selection to a set-top box. As an addition or variation, the playback device40can make a media file selection from a network streaming source.

In some examples, the metadata manager130includes an editor interface132which receives input133from a publisher class user or customer. The input133can specify an object or event that is to be marked by a content parameter121for the record99of that airing. The content parameter121can be specific to, for example, a frame of the airing and/or a particular visible event (e.g., person appearing in primary content, object). The content parameter121may also be linked with a timing parameter119that identifies when the event specified by the user occurs with respect to the timeline119B of the primary content. In some variations, the input133can also enable, for example, a content publisher (e.g., producer for program) or publisher class user to specify control over the type of supplemental content which can be provided in connection with the primary content.

In some examples, the playback device40can signal a request44to the MDDS100for the metadata set105that is to accompany a particular airing. In one example, the playback device40can signal an airing identifier101. In some examples, the airing identifier101can be in the form of an airing key103. The system API110can receive the request44from the playback device40, with the specified airing ID101. The request44can be signaled contemporaneously or concurrently with the playback device40requesting and/or receiving data for playing back a primary content from a primary content source82.

The metadata manager130may respond to the request44by identifying the particular media record99that matches the airing identifier101. In variations, the request44may specify a media resource identifier103, along with other contextual information about content source, time of request, transmission medium and/or content delivery system. Based on the information provided with the request44, the metadata manager130returns a metadata set105, from which the metadata manager130is able to identify a corresponding media record99. Depending on implementation, the metadata data set105may have alternative forms, structures and/or content type, while including or providing time-based metadata that is specific to the particular media resource (as provided by the media resource identifier103) or airing (as provided by the airing identifier101).

The system API110may return a response54that includes metadata set105for the airing identifier101. The metadata set105may include information from the media record99associated with, for example, the airing identifier101. In particular, the metadata set105can include the time slot parameters119, content parameters121and/or payload data set125. In some implementations, the response54may also include tags111, which can be retrieved by the service API110via the tag interface logic114.

In some examples, the time slot parameters119include the time slot identifiers129. In one implementation, each time slot identifier129is unique, so that a corresponding time slot is uniquely identified from all other pre-determined time slots of the airings and media resources managed through the metadata manager130. The playback device40may receive the metadata set105, extract the time slot identifiers129and communicate the time slot identifiers129back to the API interface110and/or tag interface114. In some variations, the playback device40communicates additional data, including the airing identifier101and/or media resource identifier103. As an addition or alternative, the playback device40may communicate a user or device identifier.

The tag interface114, when operating either as part of the service API or as a separate component (or as part of a separate system) may access or use tag selection logic120to select the set of tags111for playback device40. Thus, the tags111may be selected as part of the response54, in response to the request44. Alternatively, the tags111may be selected a part of a supplemental response56that is communicated to the playback device40, via the service API110or through the tag interface114. As described with other examples, the tags111may be used by the playback component40to obtain a supplemental data set137from one or more third-party sources82(e.g., ad networks). In some examples the tags111may differ from the metadata set105because individual tags may reference a third-party source from which the playback device40is to retrieve a corresponding supplemental data set137when a corresponding airing is viewed. Still further, in some examples, the tags111may exist independent of a metadata set or media resource. For example, the tags111may include parameters that are not specific to the media resource or airing, but may be more descriptive of, for example, a classification of the user and/or type of supplemental content that is to be rendered with the primary content.

The tag selection logic120can select tags111for the playback device40using one or multiple selection criteria145. By way of example, the selection criteria145may include (i) the media resource identifier103and/or airing identifier101being requested by the playback device40, and/or (ii) user or device specific information, such as user profile information109and/or contextual information. Depending on implementation, the selection criteria145may originate from the playback device40at the time of request44, or the information may be stored as part of the MDDS100(e.g., profile information109).

As an alternative or variation, the playback device40may include a component that stores a data set that identifies profile or contextual information about the playback device40and/or user. The profile information109can indicate information about preferences of the user. Contextual information can identify information about a setting or condition that was present during the playback. The profile information109and the contextual information may be used by the tag selection logic120to determine tag selection criteria for the user. By way of example, the playback device40may record and/or store profile information109about prior activities of the user, and the playback device40may communicate parameters that are indicative of the profile and/or contextual information as part of the request44, or as part of a sequence of communications from which playback the tags111are retrieved.

In some variations, the MDDS100may include a user profile store115to maintain profiles for the playback device40and/or its users, including prior activities of a user. The prior activities can, for example, specify a genre of preference, a favorite television series, or other attributes. In other examples, the prior activities of the user can include the user interaction with, for example, event data121rendered by the playback device40during playback. For example, the supplemental content that is rendered on the user screen can be made selectable, through user interaction with the content rendering on the playback device40. Still further, as another example, the profile information can include a user profile151with information obtained from the user or from a third party source, such as the user's age group, gender and/or interests. The profile information can also include a device profile153for playback device40, which can store information such as the type of device, supported features, software versions, etc. The profile information can further include a session profile155for the user's current session with the MDDS100, which can be used for state-based information, security authentication, etc.

According to some examples, each tag111can cause the playback device40to retrieve a supplemental content data set137from a remote source82. An individual tag111can, for example, identify to the playback device40(e.g., to the retrieval component of the playback device40) an advertisement source from multiple possible sources. In some variations, the tag111can also identify a selection criteria which can be communicated from the playback device40to the identified advertisement source. In this way, the playback device40can use the tags111to retrieve supplemental content data sets137(e.g., media data for advertisements and promotions) from an advertising network for display with the content of the airing. By way of example, the selection of tags111for the playback device40may specify an advertisement network, and one or more parameters that are specific to the advertisement network. As an addition or alternative, the tags111can identify a campaign or content source which the playback device40is to use when obtaining supplemental content.

With further reference to an example ofFIG.1, each tag111may provide or enable a trigger mechanism that causes the playback device40to retrieve a supplemental content item for inclusion in the playback of the media resource identified with the airing identifier101. In some implementations, the tag111includes a time marker112and a programmatic trigger element113. The time marker112may synchronize to the time of the primary content, so as to correspond to a time in the playback of the primary content when a trigger event is to occur. The trigger element113can identify a location and/or resource for the trigger event. In some examples, the trigger event can correspond to one or more of (i) the playback device40retrieving a supplemental content data set137, and/or (ii) the playback device40rendering and/or providing content and/or functionality using the supplemental content data set137. Accordingly, the location identified by the trigger element113can include a remote source (e.g., located with a network address), local source (e.g., local memory address) or local network source (e.g., peripheral device). In some examples, the location identified by the trigger element is for an ad network or service. The resource can specify data that enables selection of the supplemental data set137. For example, the resource can identify information associated with a corresponding tag111, or otherwise provided from the MDDS100, to enable a remote service to select an appropriate supplemental data set137for the playback device40.

In some variations, the playback device40may render content from the supplemental content data set137that includes a structure of an overlay, or other content structure that can be concurrently displayed with playback of the primary content. In variations, the supplemental content data set137can identify commercials or content that inserts between, for example, the segments of the airing.

In order to ensure the supplemental content data set137is rendered at the correct playback time, the playback device40synchronizes a timeline of the metadata set105with a timeline119B of the selected media resource (for primary content). For example, the playback device40may initiate a timer that is synchronized with a particular event detected from the playback of the media resource, and/or with an external timing reference such as provided by the broadcast schedule. When the timeline of the metadata set105is synchronized with the playback time, the playback device40can use the metadata set105to perform operations for rendering the supplemental content at the appropriate times of the playback. Depending on the implementation, the tags111can be received when the media resource or airing is requested by the playback device40, or when the media resource or airing is being played back by the playback device40. In some variations, the playback device40can make just-in-time requests for supplemental content using the tags111. Still further, in some variations, the tags111can be retrieved and stored during a prior session, then read from memory to cause the playback device40to perform a corresponding retrieval action.

Some variations may include functionality to render time-sensitive supplemental content in connection with, for example, “live television” (e.g., sporting event, political event, etc.). In such context, the supplemental content may be commercial (e.g., advertisement) or an enhancement to the primary content. Additionally, as with other examples, the supplemental content may be interactive. For example, during a sporting event, the supplemental content may be displayed as an overlay of the primary content, and in the form of interactive content in which viewers are invited to vote on an event that just occurred (e.g., “Which player was the best performer?”). As described with other examples, the MDDS may deliver tags111for enabling the playback device40to retrieve supplemental content that is selected for delivery to the playback device40in real-time (or near real time). For example, tags111may be delivered to the playback device40for purpose of enabling the playback device40to render supplemental content that is selected or responsive to events of the live broadcast.

In some examples, the tag interface114includes or communicates with a tag creation tool162and a delivery trigger164. An operator, such as a provider of the primary content, can interact with the tag creation tool162to create one or more fast-delivery tags111B. In one implementation, the tag creation tool162and the delivery trigger164may be included with, or coupled with the editor interface132, so that editors whom specify metadata content105may also specify creation of fast tags111B.

According to some examples, the operator specifies input for the fast-delivery tag111B that corresponds to a network location (e.g., URL). In some variations, the input for the tag creation tool162includes supplemental content (e.g., overlay content), which can be packaged with the tag. The operator can signal the delivery trigger164, via the tool162, to deliver the tag111B to playback devices40which are connected to the MDDS100. In variations, the tag creation tool162can interact with a programmatic entity that is able to provide input and/or trigger the delivery trigger164.

The delivery trigger164may, for example, initiate a communication to each playback device40that is deemed to be rendering the desired primary content. In one implementation, the delivery tool164may cause each of the identified playback devices40to make a new request54of the system API110for the tag111B. Alternatively, the delivery tool164may cause the system API110to push the tag111B to each of the playback devices40. The MDDS100may maintain, for example, a list of connected playback devices and the airing or media resource identifier101,103most recently communicated by each playback device40. In this way, the tag111B may be instantly communicated, in near-real time from when the delivery tool164is invoked, to those devices which are viewing the desired live content. In some variation, the system API110can exchange communications with connected playback devices before transmitting the fast tag111B.

In some examples, an advertiser interface660may be integrated with the MDDS100in order to populate and distribute tags111in accordance with advertisement campaigns that are purchased and managed through an advertisement system600(seeFIG.6). As described with other examples, the advertisement system600can provide weights629for tag selection logic, and tag data641to enable distribution of content specified by campaigns.

FIG.2illustrates an example client system for use with a playback device, according to an embodiment. According to some examples, the client system200, as shown with an example ofFIG.2, represents software and/or logic to enable a computing device to operate as a playback device40, and further to receive and utilize metadata and services provided through the MDDS100.

In some examples, the playback device40can correspond to a multifunctional computing device capable of a variety of tasks, such as Internet browsing, messaging, voice or telephony operations, or web-based applications and functionality. For example, the playback device40can correspond to a tablet, smartphone, laptop, or desktop computer which can utilize the service application to connect to a streaming content provider (e.g., Netflix, who, HBO etc.). In variations, the playback device40can correspond to a set-top box, smart television, peripheral connected streamer (e.g., ROKU device manufactured by ROKU, INC., APPLE TV manufactured by APPLE INC.), or other device capable of receiving broadcast programming, such as provided through cable systems, satellite broadcast, over the air broadcast or other broadcasting application meetings.

Still further, some examples provide that the client system200can be implemented in part by a service application, such as an “app” downloaded from an “app store.” As a service application, the client system200may connect to a network service over the Internet in order to exchange data and receive services. The client system200may also perform other tasks, such as communicating a user or account identifier to a network service, maintaining and communicating profile or activity date to the network service, and/or integrating or communicating with a media player in order to render content. In variations, the client system200can be implemented through multiple applications or application processes. For example, the playback device40can be implemented as an application and plug-in, or by a browser and in-browser application.

In an example ofFIG.2, the client system200includes a user interface210, a media library interface220, a media player230, a system interface240, a metadata manager250and a network content interface260. The user interface210represents an abstraction of user-facing features which can receive user input. The user interface210can enable the user to interact with a media library or media source82and/or metadata based content delivery system100such as shown with an example ofFIG.1. In an example ofFIG.2, a user interacts with a user interface210to make a selection201of a media resource, and the selection201can trigger the client system200to retrieve metadata content from the MDDS100. In one implementation, the media library interface220can interface with a media source80to receive media library information203, and the media library information203can be displayed or rendered to the user via the user interface210. A user can view the media library information203and provide input to make a selection201of a media resource and/or navigate through a media library or selection interface. Depending on the implementation, the interaction may be through, for example, a keyboard or peripherally connected device, a dedicated input mechanism (e.g., remote control device), a touchscreen and/or a gesture/motion detection mechanism integrated with the playback device40.

In an example ofFIG.2, the client system200implements a primary process to playback primary content synchronously with an ancillary process that renders supplemental content from one or multiple sources. In the primary process, the media resource of the selection201may be played back on the client system200in accordance with a user input, user-specified schedule or broadcast schedule. In some examples, the media library interface220may interface with a remote media source80in order to receive the media resource of a particular airing. When the media resource is selected for playback, an ancillary process is initiated by the client system200to retrieve metadata from the MDDS100and to display supplementary content with the playback of the media resource. In one implementation, MDDS interface240generates the request44, which can identify the media resource of the selection201. In examples such as described withFIG.1, the request44can specify a media resource identifier101, airing identifier103or other identifier that is correlative to the selection201. As described with an example ofFIG.1, the MDDS100may match an identifier of the request44to a media record99, in order to identify the metadata set105that is to be returned to the client system200.

In one implementation, the media library interface220and/or media player230may receive playback data233from the content source82. The media player230can receive the playback data235, and then initiate playback of the media resource, so as to generate output content for user consumption (e.g., via the user interface210).

In the ancillary process, the metadata manager250may process the metadata set105that is received from the MDDS100via the system interface240. Among other functionality, the metadata manager250may buffer the metadata set105, process the metadata set105to identify time-based metadata content that is to be provided with playback of the primary content, extract time slot identifiers119from the metadata set105, and extract or retrieve tags111. Additionally, the metadata manager250can use the tags111to retrieve supplemental content items225from one or more remote or independent network services. The retrieved supplemental content items225may be rendered during predetermined time slots of the primary content, as specified by time slot parameters119of the metadata set105.

The metadata manager250may implement processes to retrieve and render the supplemental content items225in a manner that is synchronized to a predetermined timeline of the primary content. In some examples, the metadata manager250is integrated, or in communication with the media player230(i) to obtain or receive playback data235from the media player230, and (ii) to transmit or otherwise provide the supplemental content data237to the media player230. The playback data235can include individual frames that are used to render the primary content. As an addition or alternative, the playback data235can include information about the playback time of the primary content. Still further, the primary content can include metadata to identify events or aspects of the primary content during playback.

In some examples, the metadata manager250includes a synchronization component252and a content retrieval component254. The synchronization component252may implement logic to synchronize the insertion of supplemental content data237by the media player230, including overlays (e.g., images or video displayed concurrently with primary content) and sequential content (e.g., video such as commercials which temporarily replace the primary content).

In one implementation, the synchronization component252initiates a timer that is synchronized to a timeline of a primary content that is being (or will be) rendered through the media player230. The synchronization component252may synchronize the timer to one or multiple synchronization markers233of the primary content. In some examples, the markers233are determined by the synchronization component232processing an output239of the media player230. The output239may include, for example, an audio output, video output, or metadata generated from the media player230. By way of example, the metadata generated can include a playback clock which the media player displays as runtime of a given playback. In variations, the media player230and/or the metadata manager250implements logic to process the output239for reference moments, which are known to map to a particular point in the timeline of the primary content. For example, the output239can correspond to the audio output of the primary content, and the synchronization component can include audio recognition logic to process the audio signal to detect a particular sequence (e.g., opening score, specific name or event that is descriptive of audio event or signature). Similarly, the output239can correspond to video from the primary content, and the synchronization component252can perform image recognition to identify a particular event, such as the frame for an opening credit. In this way, the synchronization component252can process the primary content to determine markers233, which can reference the points of insertion for supplemental content data237. The markers233determined from the output239may also determine when the metadata manager250implements operations for retrieving supplemental content for the media player230to render. In this way, the output239of the primary content can provide a synchronization reference or marker by which the supplemental content data237can be inserted into the primary content timeline in accordance with a desired time slot119, and independent of delivery platform.

In some examples, the synchronization component252extracts one or more reference content markers from the metadata set105. Each content marker may correspond to an image pattern and/or audio pattern for an event that occurs during the playback of the primary content (e.g., rendering of a hash in a corner of a frame, appearance of a character or object, occurrence of an audible, etc.), and each content marker can be paired with a reference moment in the timeline of the primary content, so as to a relative time during the playback of the primary content. The synchronization component252can implement content analysis logic to recognize or otherwise detect markers in the primary content. In some examples, the synchronization component252can detect markers to initiate a timer for a timeline during which supplemental content of one or multiple types may be rendered. In variations, the synchronization component252uses the markers to detect a window of time during which a supplemental content item is to be rendered.

As an alternative or variation, the markers233can be determined from external sources, such as scheduling data141(e.g., a broadcast schedule, seeFIG.1), provided by, for example, the broadcaster (or publisher or distributer) of the media resource that is being played back. For example, a publisher or distributor can publish a schedule that identifies a time when an airing is be transmitted, and the synchronization component252can synchronize the rendering of the supplemental content based on the reference time. As an addition or alternative, the reference time may identify “commercial breaks” such as instances when the primary content breaks to permit playback of a commercial, or when playback of the primary content is to resume after completion of a commercial break.

The content retrieval component254can implement processes to retrieve supplemental content data225from one or more remote sources88. In some examples, the content retrieval component254retrieves supplemental content data225from remote sources88, such as third-party advertisement networks. In some examples, the content retrieval component254may utilize one or more multiple connectors to advertisement networks in order to retrieve supplemental content data225for a particular airing. According to some examples, the retrieval component254retrieves the supplemental content data225using individual tags111, which may be provided from the MDDS100as part of the ancillary process. In some examples, the tags111specify the remote source (e.g., the particular advertisement network) what is to be used in order to retrieve the supplemental content data225.

In some variations, the content retrieval component254generates a request255for supplemental content, using the tags111. The request255may include one or more criterion211for enabling the remote source88to select supplemental content data225. The criterion211can correspond to, for example, any one or more of a keyword, classification or identifier, from which the remote source88can make selection of supplemental content data225. In some examples, the criterion211can be provided by or determined from the metadata set105. In variations, criterion can be determined from profile or contextual data stored on the playback device40.

FIG.3provides an example method for providing time-based metadata to a playback device, in connection with the playback device playing back a primary content.FIG.4illustrates an example method for selecting and providing supplemental content for live television in near-real time. In describing examples ofFIG.3andFIG.4, reference may be made to elements ofFIG.1orFIG.2for purpose of illustrating a suitable component or element for performing a step or sub-step being described.

With reference to an example ofFIG.3, the MDDS100determines a media resource that the playback device40has requested to playback (310). In some variations, the MDDS100may identify a media resource that is being played back on the playback device40. In some examples, the MDDS100operates as a service, separate from a source of the media resource, from which primary content is rendered on the playback device40. The playback device40may, for example, make a request44over the Internet to a service which implements the MDDS100. The request44may include one or more identifiers for the media resource, such as, for example, a media resource identifier103and/or an airing identifier101. The identifier(s) of the request can, for example, identify the particular work, the medium or device used to render the work (e.g., cable television or streaming network service) and/or context (e.g., broadcast or on-demand). The work may be episodic, so as to include, for example, each of a series identifier, a season identifier and an episode identifier.

In some examples, the MDDS100operates separately or independently from a source of the media resource, and the identifier(s) of the media resource are communicated to the MDDS100programmatically. For example, the playback device40can include logic to determine one or more applicable identifiers for an airing that is requested by a user. Alternatively, the playback device40can include logic to determine one or more applicable identifiers of the media resource based on a channel or media source selection of the user.

In response to receiving the request, the MDDS100can provide the playback device40with a set of time-based metadata that is associated with the media resource (320). As described with other examples, the set of time-based metadata may be correlative to a primary timeline of events in a playback of the media resource. In some examples, the MDDS100provides time-based metadata that includes timing parameters (or time slots)119. The time slot parameters119may, for example, identify one or more available time slots (or durations of time), relative to a timeline of the primary content.

The MDDS100may provide the playback device40with one or more tags111that enable the playback device40to retrieve supplemental content from a remote source (330). In some examples, the playback device40receives time slot parameters119with the metadata set105, and then uses the time slot parameters119to request the tags111. In a variation, the MDDS100provides time slot identifiers119with the metadata set105, and the playback device40uses the time slot identifiers119to request the tags111.

In some examples, the MDDS100includes tag selection logic120to select tags111individually for a given request44of a playback device40. The tag selection logic120may use, for example, profile information that is maintained or received from the playback device40, as well as contextual information (e.g., location or IP address of the playback device) and information about the primary content to select tags111. The tags may identify a remote source for supplemental content data137. As an addition or variation, the tags111may specify terms and identifier for enabling selection of supplemental content for the user or playback device by the remote source. For example, the MDDS100may return individual tags111which are in the form of a link, structured to include keywords or terms as selected by logic of the MDDS100.

With reference to an example ofFIG.4, an operator or programmatic entity interacts with the MDDS100to create a fast-delivery tag111B (410) that can cause playback devices of a given population to instantly display a content item. The fast-delivery tag111B may correspond to a data structure that includes or links to a content item. In one implementation, the data structure integrated a text or image item with a structure that can be pushed to individual media playback devices.

The MDDS100can maintain a list of playback devices which are viewing a particular media resource (e.g., live broadcast) (420). An operator may generate a fast link for the media resource using the tag creation tool162.

Once the fast-delivery tag111B is created, the delivery trigger164can send the link instantly to the playback devices (430). The tags111B may be encoded to trigger the corresponding playback devices40to output the content associated with the tag111B. In one implementation, the media playback devices40access the content item from a network location specified in the tag111B. For example, the tag111B can reference the network location corresponding to the operator website. In a variation, the tag111B can link the playback component to a memory or network location provided by the MDDS100, where the content item is stored. Still further, the111B may integrate the content item with the data structure, so that the playback device40is triggered to render a supplemental content item using the tag111B as the common transport vehicle.

FIG.5is a block diagram that illustrates a computer system upon which embodiments described herein may be implemented. For example, in the context ofFIG.1, MDDS100, or portions thereof, may be implemented using a computer system such as described byFIG.5. The MDDS may communicate with playback devices, paired devices and/or distribution sources of media content and/or associated metadata using a network such as the Internet.

In an embodiment, computer system500includes processor504, main memory506, ROM508, storage device510, and communication interface516. Computer system500includes at least one processor504for processing information. Computer system500also includes a main memory506, such as a random access memory (RAM) or other dynamic storage device, for storing information and instructions to be executed by processor504. Main memory506also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor504. Computer system500may also include a read only memory (ROM)508or other static storage device for storing static information and instructions for processor504. A storage device510, such as a magnetic disk or optical disk, is provided for storing information and instructions. The communication interface516may enable the computer system500to communicate with one or more networks through use of the network link520.

Computer system500can include display512, such as a cathode ray tube (CRT), a LCD monitor, and a television set, for displaying information to a user. An input device514, including alphanumeric and other keys, is coupled to computer system500for communicating information and command selections to processor504. Other non-limiting, illustrative examples of input device514include a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor504and for controlling cursor movement on display512. While only one input device514is depicted inFIG.5, embodiments may include any number of input devices514coupled to computer system500.

Embodiments described herein are related to the use of computer system500for implementing the techniques described herein. According to one embodiment, those techniques are performed by computer system500in response to processor504executing one or more sequences of one or more instructions contained in main memory506. Such instructions may be read into main memory506from another machine-readable medium, such as storage device510. Execution of the sequences of instructions contained in main memory506causes processor504to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement embodiments described herein. Thus, embodiments described are not limited to any specific combination of hardware circuitry and software.

FIG.6illustrates an example content selection system, according to one or more embodiments. A content selection system600such as described with an example ofFIG.6may be implemented as a network service. In some examples, the content selection system600can be provided with or as part of MDDS100. In variations, the content selection system600is provided as a separate network service from the MDDS100.

With reference to an example ofFIG.6, the content selection system600includes inventory manager610, advertiser interface620, distribution logic630and tag distribution logic640. The inventory manager610may maintain a data store of inventory602. The inventory602may maintain records of available opportunities during which an advertiser can insert advertisement in connection with the rendering of an airing on playback devices in a given audience. In some implementations, the advertisement can be supplementary content that is of a form that renders concurrently with the primary content.

In context of MDDS100, an inventory602can be generated and/or integrated with the metadata store116. In some implementations, an aggregation component604can extract relevant information from the metadata store116for specific airings (or for the respective record99) in order to determine inventory attributes for individual airings. By way of example, the inventory attributes can include time slot identifiers601, airing identifiers603, category (or classification) tags605, and/or keywords607. The inventory attributes can be determined from, for example, a content publisher, content creator or other party that has rights to control the rendering of supplemental content in connection with the particular airing. Some inventory attributes, such as category tags605or keywords607, can be based on content-based characteristics of the airing, such as a theme, ambience, or event. The content attributes can be determined programmatically, or provided by manual input. For example, a theme or ambience of the hearing can be identified by performing an analysis of (i) the musical soundtrack accompanying the airing, (ii) the lighting of the scene accompany the airing, and/or (iii) other audiovisual signatures of a theme, setting or ambience of the primary content.

In some examples, the time slot identifiers601and/or airings603can be associated with whitelist or blacklists611A,611B. The respective lists can identify supplemental content items by a type, genre, or source (e.g., retailer manufacturer, Brand, etc.), to permit (whitelist) or deny (blacklist) in connection with the rendering of a corresponding primary content. For example, if the time slot identifier601pertains to a portion of an airing depicting a placed brand product in a scene of a storyline, the blacklist associated with the time slot identifier may exclude competing products to that of things to be the placed brand. Likewise, if a theme of the particular portion of the airing is for a desired demographic of viewers, the blacklist associated with the time slots of the particular airing may exclude advertisements that are by type, genre or source, deemed to be offensive and/or ineffectual to that desired demographic. The use of the time slot identifiers601allows the content distributor or publisher to specify, for example, a preference or designation for certain types of supplemental content (e.g., by theme, source, brand, etc.), as well as preference or designations against certain types of supplemental content. The preference or designation for or against certain types of supplemental content can be made specific to specific slices of the primary content.

The inventory manager610may include an inventory estimator612to determine how much inventory is available for particular airing. The inventory estimator612may be based on the number of time slots which the particular airing can carry. Each time slot identifier601can represent an inventory that corresponds to the number of viewings the corresponding portion of primary content is expected to receive. Additionally, each airing may have a set number of time slot identifiers601, which collectively represent the inventory for the airing. For example, a 20 minute program may carry four time slots which are spaced five minutes apart, while a one hour program may double the number of time slots. The inventory estimator612may also determine and correlate inventory to profile characteristics of the playback devices40(e.g., computing platform of playback device40, transmission medium in which the airing is received by respective playback devices40, etc.). In particular, the available inventory determination can segment the available inventory to profile characteristics of the viewing population (e.g., age and gender of viewer). Therefore, the availability estimator612can estimate inventory for (i) individual time slots of an airing, (ii) collectively for all time slots of an airing, and (iii) further with respect to particular segments of the viewing population.

The inventory valuation614can implement an algorithm, logic or component to determine a valuation615of a particular time slot. In some examples, a bidding process may be used to determine the valuation615. In some variations, the valuation615can be made specific to profile characteristics of the viewing audience. For example, popular shows may have higher valuation for use of the respective time slots, but smaller or less desirable demographics for the particular popular program can also be made subject of inventory and offered for less.

The inventory manager610may generate inventory information613for the advertiser interface. The inventory information613may include, for example, quantitative representations of available inventory for specific time slots of different airings. Additionally, the inventory information613may include the valuations615of the time slots for the respective airings. The advertiser interface620may receive advertiser requests611, and match criteria specified in the respective advertiser requests to inventory information613. Thus, the inventory information613can include data sets associated with the time slot identifiers601.

In some implementations, the advertiser interface620includes matching logic622that matches advertiser criteria, specified with advertiser requests621, to time slots601. The matching may be based on, for example, attributes of categories and/or keywords. In matching advertiser requests, the matching logic622may also determine whether the valuation615, as determined for specific time slots601, conforms to valuation parameters specified by the advertiser requests. The matching logic622may also implement matching rules623, which can include implementation rules or weights to implement one or both of the whitelists611A and/or blacklists611B.

When the advertiser requests621are matched to time slot identifiers601, an advertiser data set625can be linked to the time slot identifier. The advertiser data set625can include, for example, a network location (e.g., URL) where a supplemental content item is provided. In variations, the advertiser data set625includes a content element (e.g., image). In other variations the advertiser data set625identifies an advertisement service and an identifier or criteria to enable programmatic identification of a content item provided by the advertiser.

According to some examples, the advertiser interface620communicates the advertiser data set625to the MDDS100, and distribution data627to the distribution logic630. The distribution data627may, for example, specify a parameter that determines a quantity or frequency of the number of instances in which the supplemental content of the advertiser data set625is to be rendered with the matched portion of an airing corresponding to a particular time slot identifier601. In some variations, the advertiser's requests can specify a campaign and campaign value (how much an advertiser will pay for the particular campaign). The distribution data627may also specify profile parameters that are specific to the matched advertiser request621. For example, the distribution data may specify a weight or quantity for each gender.

In some examples, the advertiser data set625can be communicated to a tag population logic630, and the tag population logic630can populate a tag element641or structure with the advertiser data set625. The tag element641can be assimilated, or otherwise structured into tags111which the MDDS100communicates to playback devices40. In this way, the tag population logic630enables the MDDS100to communicate, as the response54to a requesting playback device40, individual tags111which include identifiers for enabling the playback device to retrieve or receive a specific supplemental content item when the primary content is being viewed.

In some implementations, the tag elements641include identifiers that can identify corresponding supplemental content items by network location (e.g., URL or network address). In variations, the tag elements641include identifiers that can identify a third-party advertiser (e.g., ad network), as well as a campaign or content identifier as provided by the advertiser network. Still further, the tag elements641can include a content element, such as a word or wording that can be displayed as the supplemental content. The tag element641can also provide data that identifies one or more profile parameters for which the tag or tag selection is to be provided. In some variations, the tag element641can be assimilated into tags111that carry one or more distribution parameters (e.g., weights, frequency in which a tag is to select a particular campaign or supplemental content item), and the distribution parameters can be made specific to a particular profile parameter or set of profile parameters.

The distribution logic640can determine weights629for implementing each campaign. The weights629can be communicated to the tag selection logic120via the interface650, and the tag selection logic120may use the weights to select tags (or tag elements) to communicate in responses54to requesting playback devices40. In some variations, the weights629are specific to profile parameters of playback devices40, where the profile parameters may be determined by profile information109of individual devices. The weights629may determine a proportionate count in the number of times which the supplemental content for a given advertiser is rendered relative to a total number of viewing devices. As an alternative or variation, the weights629may identify a frequency in which the supplemental content of a particular advertiser is displayed to individual playback devices40through use of metadata provided by the MSSD100.

In some implementations, the weights can be stored as part of the tag or tag elements in the tag store135. The tag selection logic120can select tags based on the recorded and stored weights. In variations, the tag selection logic120receives and uses the weights629.

In some examples, the advertisement system600can include an interface650to interface with MDDS100. Thus, for example, the tag store135can be programmatically updated and maintained in accordance with campaigns which are purchased and managed through the advertisement system600.

FIG.7illustrates an example method for providing supplemental content with airings of programming media. An example method such as described withFIG.7may be implemented using an example system such as described withFIG.6andFIG.1. In some examples, content selection system600operates to identify supplemental content items, and a metadata delivery platform operates to generate and transmit tags to select playback devices of a population of playback devices. Accordingly, reference may be made to elements ofFIG.1and/orFIG.6for purpose of illustrating functionality for performing a step or sub-step being described.

With reference toFIG.7, a determination is made as to an inventory of time slots for a given airing (710). In some examples, the media record99for the airing may identify one or more multiple time slots during which supplemental content items may be concurrently rendered on media playback devices along with the primary content of the airing. The supplemental content items can include images, text images and/or video. In some examples, the supplemental content items can be interactive to detect selection input or other interaction from playback devices that receive user input during the rendering of the respective airing.

The inventory can include quantitative aspects which include a number of time slots that define segments of an airing. In some variations, the quantitative aspects include an expected number of playback devices that may receive a supplemental content item for each of the defined time slots of the airing. Still further, in some variations, the number of playback devices may be further categorized in accordance with profile parameters which may be associated or determine to be relevant for individual playback devices.

The content selection system600may operate to assign a supplemental content item of a third-party to a time slot of the given airing when the given airing is rendered by individual playback devices in a group of playback devices (720). According to some variations, the assignment of supplemental content is specific to selection parameters that include one or more of profile characteristics of playback devices that render the primary content. By way of example, the profile characteristics include one or more of a geographic region where the playback device is likely located, a computing platform of individual playback devices, and/or a transmission medium of individual playback devices to receive and render the media resource of the airing. In some variations, the profile characteristics include characteristics of users of individual playback devices (e.g., gender, age and/or viewing preferences).

In some variations, the content selection system600determines a weight or priority value that can control the number of times and/or frequency in which a particular supplemental content item is to be rendered in connection with an airing. In variations, the tag selection logic120associates a weight or priority value with profile selection parameter, corresponding to profile attributes of playback devices and/or user profiles.

The content selection system600may structure tag elements to incorporate data for enabling individual playback devices40to retrieve or otherwise receive the assigned supplemental content item during the corresponding pre-defined time slot (730). In some examples, tag population logic640structures a tag element to include an identifier and/or content element for a selected supplemental content item. The identifiers of respective supplemental content items may each correspond to one of (i) a network location of an advertiser or advertiser network, or (ii) an identifier to enable a third-party advertiser network to identify the particular supplemental content items.

The MDDS100or content selection system600may operate to communicate tags111that include the structured tag element to each of multiple client devices that are used to view the corresponding airing (740). As described with other examples, the playback devices that receive the tags111are triggered to access remote sources where the supplemental content item is provided. For example, the playback devices40may be triggered to access third-party advertiser networks, or network locations where supplemental content items are stored.

Although illustrative embodiments have been described in detail herein with reference to the accompanying drawings, variations to specific embodiments and details are encompassed by this disclosure. It is intended that the scope of embodiments described herein be defined by claims and their equivalents. Furthermore, it is contemplated that a particular feature described, either individually or as part of an embodiment, can be combined with other individually described features, or parts of other embodiments. Thus, absence of describing combinations should not preclude the inventor(s) from claiming rights to such combinations.