USER INTERFACE DISPLAYING SCENE DEPENDENT ATTRIBUTES

A method and apparatus for interacting with content data is provided. A current time point for a frame of the content data is determined and at least one user selectable media object included in the frame at the current time point and that occurs at least one further time point of the content data is identified. At least one media object identifier associated with the identified at least one user selectable media object is acquired a user interface display image including the frame of content and the at least one media object identifier is generated. The user interface display image enables selection, by a user, of the at least one media object identifier and automatically initiating playback of the content at the at least one further time point in response to selection of the at least one media object identifier.

TECHNICAL FIELD OF THE INVENTION

The present disclosure generally relates to digital content systems and, more particularly, to a system, method and graphic user interface enabling user access to at least a portion of digital content based on objects within the digital content.

BACKGROUND OF THE INVENTION

Home entertainment systems, including television and media centers, are converging with the Internet and providing access to a large number of available sources of content, such as video, movies, TV programs, music, etc. This expansion in the number of available sources necessitates a new strategy for navigating a media interface associated with such systems and enabling access to certain portions of media content being consumed by the user.

Each piece of content includes different actors, scenes, music etc. and, should a viewer enjoy a particular one of these actors, scenes, music, etc., the viewer is left to engage a trick mode or use some other manual scan of the content to try to locate a portion of the content that is desirable to the user. One drawback associated with this manual scan is that it is time consuming for the user. Another drawback associated with this manual scan is that engaging in such a scan may be resource intensive for the system that is receiving, decoding or performing other processing on the data file or stream including the content. The desire to quickly navigate within media content creates an interface challenge that has not yet been successfully solved in the field of home media entertainment. This challenge involves presenting users with a streamlined user interface that facilitates quick navigation within media content without requiring a user to engage in manual seeking.

Additionally, with the increased interconnection between devices that receive and play media with other sources of information about the media content, it is desirable to provide that information to the user while the user is consuming the content. Thus, a further interface challenge is presented when seeking to provide the user with the ability to link media content being viewed with information about the media content. In prior systems, a user needed pre-existing knowledge of items within the content and knowledge about the sources of additional information. To acquire this information, the user had to actively search and/or browse to these sources, navigate through them and determine what information at the sources is relevant to the media content being consumed. The drawback associated with this manner of acquiring additional information about media content is the time consuming distraction it creates when viewing the media content. To obtain this information about some aspect of the media content being consumed, the user's attention is diverted from the actual media content for an extended period of time without a guarantee of actually obtaining the additional information that may be found useful.

The present disclosure is directed towards overcoming these drawbacks.

SUMMARY

In one embodiment, a method of interacting with multimedia content data is provided. A current time point for a frame of the multimedia content data is determined and at least one user selectable media object included in the frame at the current time point and that occurs at least one further time point of the multimedia content data is identified. At least one media object identifier associated with the identified at least one user selectable media object is acquired in a user interface display image including the frame of multimedia content and the at least one media object identifier is generated. The user interface display image enables selection, by a user, of the at least one media object identifier and automatically initiates playback of the multimedia content at the at least one further time point in response to selection of the at least one media object identifier.

In another embodiment, an apparatus for interacting with multimedia content data is provided. A controller determines a current time point for a frame of the multimedia content data, identifies at least one user selectable media object included in the frame at the current time point and that occurs at least one further time point of the multimedia content data, and acquires at least one media object identifier associated with the identified at least one user selectable media object. A user interface generator coupled to the controller for generating a user interface display image including the frame of multimedia content and the at least one media object identifier, the generated user interface enables selection, by a user, of the at least one media object identifier and the controller automatically initiates playback of the multimedia content at the at least one further time point in response to selection of the at least one media object identifier.

It should be understood that the drawing(s) is for purposes of illustrating the concepts of the disclosure and is not necessarily the only possible configuration for illustrating the disclosure.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input/output interfaces. Herein, the phrase “coupled” is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software based components.

Turning now toFIG. 1, a block diagram of an embodiment of a system100for delivering content to a home or end user is shown. The content originates from a content source102, such as a movie studio or production house. The content may be supplied in at least one of two forms. One form may be a broadcast form of content. The broadcast content is provided to the broadcast affiliate manager104, which is typically a national broadcast service, such as the American Broadcasting Company (ABC), National Broadcasting Company (NBC), Columbia Broadcasting System (CBS), etc. The broadcast affiliate manager may collect and store the content, and may schedule delivery of the content over a delivery network, shown as delivery network1(106). Delivery network1(106) may include satellite link transmission from a national center to one or more regional or local centers. Delivery network1(106) may also include local content delivery using local delivery systems such as over the air broadcast, satellite broadcast, or cable broadcast. The locally delivered content is provided to a receiving device108in a user's home, where the content will subsequently be searched by the user. It is to be appreciated that the receiving device108can take many forms and may be embodied as a set top box/digital video recorder (DVR), a gateway, a modem, etc. Further, the receiving device108may act as entry point, or gateway, for a home network system that includes additional devices configured as either client or peer devices in the home network.

A second form of content is referred to as special content. Special content may include content delivered as premium viewing, pay-per-view, or other content otherwise not provided to the broadcast affiliate manager, e.g., movies, video games or other video elements. In many cases, the special content may be content requested by the user. The special content may be delivered to a content manager110. The content manager110may be a service provider, such as an Internet website, affiliated, for instance, with a content provider, broadcast service, or delivery network service. The content manager110may also incorporate Internet content into the delivery system. The content manager110may deliver the content to the user's receiving device108over a separate delivery network, delivery network2(112). Delivery network2(112) may include high-speed broadband Internet type communications systems. It is important to note that the content from the broadcast affiliate manager104may also be delivered using all or parts of delivery network2(112) and content from the content manager110may be delivered using all or parts of delivery network1(106). In addition, the user may also obtain content directly from the Internet via delivery network2(112) without necessarily having the content managed by the content manager110.

Several adaptations for utilizing the separately delivered content may be possible. In one possible approach, the special content is provided as an augmentation to the broadcast content, providing alternative displays, purchase and merchandising options, enhancement material, etc. In another embodiment, the special content may completely replace some programming content provided as broadcast content. Finally, the special content may be completely separate from the broadcast content, and may simply be a media alternative that the user may choose to utilize. For instance, the special content may be a library of movies that are not yet available as broadcast content.

The receiving device108may receive different types of content from one or both of delivery network1and delivery network2. The receiving device108processes the content, and provides a separation of the content based on user preferences and commands. The receiving device108may also include a storage device, such as a hard drive or optical disk drive, for recording and playing back audio and video content. Further details of the operation of the receiving device108and features associated with playing back stored content will be described below in relation toFIG. 2. The processed content is provided to a display device114. The display device114may be a conventional 2-D type display or may alternatively be an advanced 3-D display.

The receiving device108may also be interfaced to a second screen such as a touch screen control device116. The touch screen control device116may be adapted to provide user control for the receiving device108and/or the display device114. The touch screen device116may also be capable of displaying video content. The video content may be graphics entries, such as user interface entries (as discussed below), or may be a portion of the video content that is delivered to the display device114. The touch screen control device116may interface to receiving device108using any well known signal transmission system, such as infra-red (IR) or radio frequency (RF) communications and may include standard protocols such as infra-red data association (IRDA) standard, Wi-Fi, Bluetooth and the like, or any other proprietary protocols. Operations of touch screen control device116will be described in further detail below.

In the example ofFIG. 1, system100also includes a back end server118and a usage database120. The back end server118includes a personalization engine that analyzes the usage habits of a user and makes recommendations based on those usage habits. The usage database120is where the usage habits for a user are stored. In some cases, the usage database120may be part of the back end server118. In the present example, the back end server118(as well as the usage database120) is connected to the system100and accessed through the delivery network2(112). In an alternate embodiment, the usage database120and backend server118may be embodied in the receiving device108. In a further alternate embodiment, the usage database120and back end server118may be embodied on a local area network to which the receiving device108is connected.

Turning now toFIG. 2, a block diagram of an embodiment of a receiving device200is shown. Receiving device200may operate similar to the receiving device described inFIG. 1and may be included as part of a gateway device, modem, set-top box, or other similar communications device. The device200shown may also be incorporated into other systems including an audio device or a display device. In either case, several components necessary for complete operation of the system are not shown in the interest of conciseness, as they are well known to those skilled in the art. In one exemplary embodiment, the receiving device200may be a set top box coupled to a display device (e.g. television).

In the device200shown inFIG. 2, the content is received by an input signal receiver202. The input signal receiver202may be one of several known receiver circuits used for receiving, demodulation, and decoding signals provided over one of the several possible networks including over the air, cable, satellite, Ethernet, fiber and phone line networks. The desired input signal may be selected and retrieved by the input signal receiver202based on user input provided through a control interface or touch panel interface222. Touch panel interface222may include an interface for a touch screen device. Touch panel interface222may also be adapted to interface to a cellular phone, a tablet, a mouse, a high end remote or the like.

The decoded output signal is provided to an input stream processor204. The input stream processor204performs the final signal selection and processing, and includes separation of video content from audio content for the content stream. The audio content is provided to an audio processor206for conversion from the received format, such as compressed digital signal, to an analog waveform signal. The analog waveform signal is provided to an audio interface208and further to the display device or audio amplifier. Alternatively, the audio interface208may provide a digital signal to an audio output device or display device using a High-Definition Multimedia Interface (HDMI) cable or alternate audio interface such as via a Sony/Philips Digital Interconnect Format (SPDIF). The audio interface may also include amplifiers for driving one more sets of speakers. The audio processor206also performs any necessary conversion for the storage of the audio signals.

The video output from the input stream processor204is provided to a video processor210. The video signal may be one of several formats. The video processor210provides, as necessary, a conversion of the video content, based on the input signal format. The video processor210also performs any necessary conversion for the storage of the video signals.

A storage device212stores audio and video content received at the input. The storage device212allows later retrieval and playback of the content under the control of a controller214and also based on commands, e.g., navigation instructions such as fast-forward (FF) and rewind (RW), received from a user interface216and/or touch panel interface222. The storage device212may be a hard disk drive, one or more large capacity integrated electronic memories, such as static RAM (SRAM), or dynamic RAM (DRAM), or may be an interchangeable optical disk storage system such as a compact disk (CD) drive or digital video disk (DVD) drive.

The converted video signal, from the video processor210, either originating from the input or from the storage device212, is provided to the display interface218. The display interface218further provides the display signal to a display device of the type described above. The display interface218may be an analog signal interface such as red-green-blue (RGB) or may be a digital interface such as HDMI. It is to be appreciated that the display interface218will generate the various screens for presenting the search results (e.g., in a three dimensional grid, two dimensional array, and/or a shelf as will be described in more detail below).

The controller214is interconnected via a bus to several of the components of the device200, including the input stream processor204, audio processor206, video processor210, storage device212, and a user interface216. The controller214manages the conversion process for converting the input stream signal into a signal for storage on the storage device or for display. The controller214also manages the retrieval and playback of stored content. Furthermore, the controller214can perform searching of content stored or to be delivered via the delivery networks.

The controller214is further coupled to control memory220(e.g., volatile or non-volatile memory, including RAM, SRAM, DRAM, ROM, programmable ROM (PROM), flash memory, electronically programmable ROM (EPROM), electronically erasable programmable ROM (EEPROM), etc.) for storing information and instruction code for controller214. Control memory220may store instructions for controller214. Control memory220may also store a database of elements, such as graphic elements containing content. The database may be stored as a pattern of graphic elements, such as graphic elements containing content, various graphic elements used for generating a displayable user interface for display interface218, and the like. Alternatively, the memory may store the graphic elements in identified or grouped memory locations and use an access or location table to identify the memory locations for the various portions of information related to the graphic elements. Additional details related to the storage of the graphic elements will be described below. Further, the implementation of the control memory220may include several possible embodiments, such as a single memory device or, alternatively, more than one memory circuit communicatively connected or coupled together to form a shared or common memory. Still further, the memory may be included with other circuitry, such as portions of bus communications circuitry, in a larger circuit.

Optionally, controller214can be adapted to extract metadata, criteria, characteristics or the like from audio and video media by using audio processor206and video processor210, respectively. That is, metadata, criteria, characteristics or the like that is contained in the vertical blanking interval, auxiliary data fields associated with video, or in other areas in the video signal can be harvested by using the video processor210with controller214to generate metadata that can be used for functions such as generating an electronic program guide having descriptive information about received video, supporting an auxiliary information service, and the like. Similarly, the audio processor206working with controller214can be adapted to recognize audio watermarks that may be in an audio signal. Such audio watermarks can then be used to perform some action such as the recognition of the audio signal, provide security which identifies the source of an audio signal, or perform some other service. Furthermore, metadata, criteria, characteristics or the like, to support the actions listed above can come from a network source which are processed by controller214.

FIG. 3and represent two exemplary input devices,300aand300b(hereinafter referred to collectively as input device300), for use with the system described inFIGS. 1 and 2. The user input device300enables operation of and interaction with the user interface process according to invention principles. The input device may be used to initiate and/or select any function available to a user related to the acquisition, consumption, access and/or modification of multimedia content.FIG. 3represents one exemplary tablet or touch panel input device300a(which is the same as the touch screen device116shown inFIG. 1and/or is an integrated example of media device108and touch screen device116). The touch panel device300amay be interfaced via the user interface216and/or touch panel interface222of the receiving device200inFIG. 2. The touch panel device300aallows operation of the receiving device or set top box based on hand movements, or gestures, and actions translated through the panel into commands for the set top box or other control device. This is achieved by the controller214generating a touch screen user interface including at least one user selectable image element enabling initiation of at least one operational command. The touch screen user interface may be pushed to the touch screen device300avia the user interface216and/or the touch panel interface222. In an alternative embodiment, the touch screen user interface generated by the controller214may be accessible via a webserver executing on one of the user interface216and/or the touch panel interface222. The touch panel300may serve as a navigational tool to navigate the grid display. In other embodiments, the touch panel300awill additionally serve as the display device allowing the user to more directly interact with the navigation through the grid display of content. The touch panel device300amay be included as part of a remote control device300bcontaining more conventional control functions such as activator and/or actuator buttons such as is shown inFIG. 4. The touch panel300acan also include at least one camera element and/or at least one audio sensing element.

In one embodiment, the touch panel300aemploys a gesture sensing controller or touch screen enabling a number of different types of user interaction. The inputs from the controller are used to define gestures and the gestures, in turn, define specific contextual commands. The configuration of the sensors may permit defining movement of a user's fingers on a touch screen or may even permit defining the movement of the controller itself in either one dimension or two dimensions. Two-dimensional motion, such as a diagonal, and a combination of yaw, pitch and roll can be used to define any three-dimensional motions, such as a swing. Gestures are interpreted in context and are identified by defined movements made by the user. Depending on the complexity of the sensor system, only simple one dimensional motions or gestures may be allowed. For instance, a simple right or left movement on the sensor as shown here may produce a fast forward or rewind function. In addition, multiple sensors could be included and placed at different locations on the touch screen. For instance, a horizontal sensor for left and right movement may be placed in one spot and used for volume up/down, while a vertical sensor for up and down movement may be placed in a different spot and used for channel up/down. In this way specific gesture mappings may be used. For example, the touch screen device300amay recognize alphanumeric input traces which may be automatically converted into alphanumeric text displayable on one of the touch screen device300aor output via display interface218to a primary display device.

The system may also be operated using an alternate input device300bsuch as the one shown inFIG. 4. The input device300bmay be used to interact with the user interfaces generated by the system and which are output for display by the display interface218to a primary display device (e.g. television, monitor, etc). The input device ofFIG. 4may be formed as a conventional remote control having a12-button alphanumerical key pad302band a navigation section304bincluding directional navigation buttons and a selector. The input device300bmay also include a set of function buttons306bthat, when selected, initiate a particular system function (e.g. menu, guide, DVR, etc). Additionally, the input device300bmay also include a set of programmable application specific buttons308bthat, when selected, may initiate a particularly defined function associated with a particular application executed by the controller214. As discussed above, the input device may also include a touch panel310bthat may operate in a similar manner as discussed above inFIG. 3A. The depiction of the input device inFIG. 4is merely exemplary and the input device may include any number and/or arrangement of buttons that enable a user to interact with the user interface process according to invention principles. Additionally, it should be noted that users may use either or both of the input devices depicted and described inFIGS. 3 and 4simultaneously and/or sequentially to interact with the system.

In another embodiment, the user input device may include at least one of an audio sensor and a visual sensor. In this embodiment, the audio sensor may sense audible commands issued from a user and translate the audible commands into functions to be executed by the user. The visual sensor may sense the user(s) present and match user information of the sensed user(s) to stored visual data in the usage database120inFIG. 1. Matching visual data sensed by the visual sensor enables the system to automatically recognize the user(s) present and retrieve any user profile information associated with those user(s). Additionally, the visual sensor may sense physical movements of at least one user present and translate those movements into control commands for controlling the operation of the system. In this embodiment, the system may have a set of pre-stored command gestures that, if sensed, enable the controller214to execute a particular feature or function of the system. An exemplary type of gesture command may include the user waving their hand in a rightward direction which may initiate a fast forward command or a next screen command or a leftward direction which may initiate a rewind or previous screen command depending on the current context. This description of physical gestures able to be recognized by the system is merely exemplary and should not be taken as limiting. Rather, this description is intended to illustrate the general concept of physical gesture control that may be recognized by the system and persons skilled in the art could readily understand that the controller may be programmed to specifically recognize any physical gesture and allow that gesture to be tied to at least one executable function of the system.

In the context of the present system, the input device300enables the user to interact with a plurality user interfaces. The user interfaces contain different types of user selectable image elements. The user selectable image element may be representative of at least one type of media object that is included in the media content being received and output for display. As used herein, the term multimedia content refers to audio-video data that may be acquired or otherwise received and which may be at least one of output for display to a user and stored in a storage device for later viewing. The multimedia content may be received live, in real-time, or may be a pre-recorded. Multimedia content is associated with an auxiliary media data file. The auxiliary media data file includes information describing at least one media object included in the multimedia content. The information in the auxiliary media data file includes position information describing the various positions of the respective media object within the multimedia content. It should be understood that the auxiliary media data file may be an actual file that is communicated from a content provider to the receiving device. Alternatively, the auxiliary media data file may be generated by the receiver device in response to receiving a data stream that includes auxiliary media data. The information may also include position description information describing each position of the media object. The information may also include media object description information that describes at least one characteristic of the media object. Additionally, the term media object refers to an item associated with the multimedia content, the selection of which results in execution of a further action. A media object may include any item, person, background, location or any other element of the multimedia content that may be displayed to a user for selection thereof. The further action resulting from the selection of an exemplary media object may be at least one of (a) skipping to another location within the media content that also includes the media object, (b) generating of a further user interface including a list of at least one other position within the multimedia content that includes the selected media object, (c) generating a further user interface that includes a listing of auxiliary information associated with the selected media object. Thus, by presenting media objects within a user interface, the user consuming the multimedia content can quickly navigate to additional positions of the multimedia content that also include the same media object. For example, the multimedia object may be a motion picture and each of the actors in the motion picture may be represented as individual media objects. A selectable image element representing each media object (actor) may be presented within a user interface and, in response to selection thereof, result in automatic skipping within the multimedia content to a subsequent position therein where that actor is once again shown (e.g. a subsequent scene). Alternatively, selection of the media object representing an actor may generate a further user interface including a list of at least one other position within the multimedia content where that media object (e.g. actor) is shown (e.g. a list of additional scenes in the motion picture that include that actor). The user interfaces according to invention principles will now be discussed with respect toFIGS. 5-7.

In the following description, it should be understood that all user interfaces including user selectable image elements may be generated by the controller214ofFIG. 2and output to the user via at least one of the use interface216, the display interface218and/or the touch panel interface222. Additionally, the interaction with the user interfaces generated by the controller214may be accomplished via the input device300aand/or300bsuch that any selection of an image element will be received and processed by the controller214resulting in one of (a) updating the currently displayed user interface in response to the data selected or entered by the user; and (b) generating a new user interface in response to the data selected or entered by the user.

FIGS. 5-7represent interactive user interfaces generated by the controller214that enable the user to selectively navigate through multimedia media content being displayed via a plurality of user selectable media objects displayed within the user interface. InFIG. 5, the controller214generates a media navigation user interface400. The media navigation user interface400includes a content section402. The content section402includes the multimedia content404selected by the user. The content section402may show multimedia media content that is actively playing showing the multimedia content in action or may display a still image representative of the multimedia content at a particular point such as a when the multimedia content is paused by a user (e.g. a screen shot of the particular position within the multimedia content). In one embodiment, the media navigation user interface400may be selectively displayed on the touch screen input device300ain response to the user pausing the active display of the multimedia content. In another embodiment, the media navigation interface400may be continually displayed on at least one of the touch screen input device300aand the display device114(FIG. 1) thereby enabling the user to view the multimedia content404within the confines of the media navigation interface400to facilitate rapid navigation within the multimedia content404.

The media navigation interface400also includes a media control section410. The media control section410includes a plurality of user selectable image elements enabling the user to control various playback operations for the particular multimedia media content404displayed within the content section402. For example, the media control section410may include a reverse seek button412, a play button414, a forward seek button416, a stop button418and a record button420. User selection of any of these buttons causes initiation of the standard operations known within the art of audio-visual display and need not be further discussed. The control section410may also include an information section422. The information section422provides time and position information associated with a current state of the multimedia content404in content section402. The information section may include a time bar426representing a length of the multimedia content404. A selection bar428may also be included and is selectively positionable along the length of the timeline426. User selection and movement within the timeline426advantageously enables the user to manually select a point within the multimedia content that is to be displayed within the content section402. Moreover, the position of the selection bar428is continually updated in response to the playback of the multimedia content. The information section422may also include a time indicator that advantageously provides the user with specific time information associated with the multimedia content404. For example, should the multimedia content be paused, as is shown inFIG. 5, the time indicator424will display the timestamp identifying the point at which the multimedia content404is paused. In the instance when the multimedia content404is being played back to the user, in addition to movement of the selection bar428, the time information424within the information section422is continually updated to reflect a current position of the multimedia content404.

The navigation interface400further includes the media object section430. The media object section430includes at least one media object image element (432,434,436,438,440and450) representing a particular media object within the multimedia content. The media object image elements are user selectable, and result in at least one further action being initiated by the controller214. In one exemplary operation, selection of a respective media object image element results in the automatic skipping within the multimedia content404to a next position that includes the selected media object. In another embodiment, selection of a respective media object image element results in generation of a further user interface such as the one shown and discussed below inFIG. 6that provides additional information about the selected media object.

Media objects within the multimedia content404are set forth in the auxiliary media data file associated with the multimedia content. The media objects in the auxiliary media data file may be provided by the content provider (e.g. cable provider, movie studio, etc). These media objects may represent any item or aspect of the multimedia content that is shown to the user. The media object may appear at least one of (a) more than one point within the multimedia content, and (b) a single point within the multimedia content. Media objects may include, but are not limited to, at least one of (a) an actor/actress in the multimedia content, (b) a character in the multimedia content, (c) an item/object in the multimedia content, (d) a particular location in the multimedia content, (e) an audio component of the multimedia content, (f) any other aspect of the multimedia content that occurs more than once within the content; and (g) an other aspect of the multimedia content that may be used to provide additional information to a user.

The auxiliary media data file may include an information element that at least one of (a) identifies a media object, (b) describes the identified media object, (c) identifies at least one position within the multimedia content at which the media object may be located, (d) describes at least one position at which the media object may occur, (e) identifies additional multimedia content that includes the particular media object (e.g. other movies, shows, etc), (f) includes access information enabling a user to access additional information associated with the media object, and (g) includes information identifying a level of relatedness of the particular media object with at least one other media object of the multimedia content. The types of information described above as being included in the auxiliary media data file are described for purposes of example only and any information describing any aspect of the multimedia content may be included as an information element in the auxiliary media data file. Additionally, each information element in the auxiliary media data file includes a unique identifier associated therewith. The unique identifier associated with each information element is used by the controller214in generating image elements for the media objects that are selectively displayed within the media object section430of media navigation interface430. In one embodiment, the unique identifier may be obtained directly from within the multimedia content and be mapped directly to a portion of the multimedia content. In another embodiment, the unique identifier may be from a 3rdparty source and communicated with the auxiliary media data associated with the multimedia content.

The operation of the navigation interface400will now be described with respect to the multimedia content404being displayed within the content section402. The following operation will be described with the multimedia content404being a motion picture. However, this description of a motion picture as the multimedia content is only to illustrate the principles of the present invention and should not be taken as limiting the present invention. Persons skilled in the art will understand the multimedia content404in content section402may be any type of audio-visual data.

The multimedia content404includes an associated auxiliary media data file. The auxiliary media data file may be received via the input signal receiver202for storage in the storage device212. In the case that the auxiliary media data is communicated to the receiving device as a data stream, the auxiliary media data may be processed by the input stream processor204and stored in the storage device212. The auxiliary media data may include a plurality of records including a plurality of fields corresponding to information elements describing a particular media object. In one embodiment, a media object record may be included for each instance that the respective media object occurs within the multimedia content. In another embodiment, the auxiliary media data file may group all of the same media objects together into respective records and further include sub-records including the information elements associated with the individual media object records. An exemplary format for the auxiliary media data file is shown in Table 1 which sets forth the field name of a particular record and a description of the type of data present in the field.

TABLE 1Aux. Media Data FileField NameField Description/Type of DataMedia Object NameThe name of the media object to whichthis record appliesMedia Object TypeType of Media Object - e.g. actor,location, etcMedia Object identifierImage file name and/or image filelocation that identifies the media objectTime Stamp - StartTime stamp data identifying a time withinthe multimedia content that the mediaobject appearsTime Stamp - DurationTime stamp identifying a duration of timethat the media object is shown; note, theduration applies directly to the time stampindicated in the Time Stamp - Start FieldTime Stamp - TotalTime stamp information identifying eachtime the media object appears throughoutthe multimedia contentOther Media ContentIncludes information identifying othermultimedia content including this mediaobject; may include multimedia contentcurrently possessed or available orcontent that can be acquired from anexternal source/3rdpartyMedia Object DescriptionInformation describing the media objectMedia Object LocationReal-world location associated with themedia object in context of the currentmultimedia content (e.g. current scenewas filmed in Maui, HI)Media Object LocationInformation about the real-world locationDescriptionlisted in Media Object Location Field;may include data for accessing additionalsources of informationMedia Object RelatednessInformation identifying the relatedness ofthis media object with any other mediaobject in the multimedia content; mayalso include information identifying othertypes of multimedia content including thisobject and any other media object in thecurrent content (e.g. media objectsrepresenting two actors in this samemovie may identify other movies that alsoinclude these two actors)
Table 1 merely illustrates the types of information element fields that may be associated with a media object within the multimedia content. However, each record in the auxiliary media data file may include any number of information element fields that include any type of information associated with the media object.

In one exemplary operation, in response to a user pausing playback of the multimedia content404, the controller214generates the navigation interface400. The pausing of content playback may occur by initiating a pause command on any user input device300. In generating the navigation interface400, the controller214identifies a current time within the multimedia content and parses the auxiliary media data file to identify the media objects that are present at the current time. Upon identifying the media objects present at the current time, the controller214obtains the media object identifier associated with a respective media object and provides the media object identifier to one of the user interface216, touch screen interface222and/or display interface218for inclusion as a user selectable media object image element in the media object section430of the navigation user interface400.

For example, the multimedia content404may be a motion picture, and, at the current time, the motion picture is displaying a scene that includes two actors dancing with a third actor playing music. The controller214will parse the auxiliary media file to identify the two actors dancing and the actor playing music as respective media objects. The male actor dancing represents the first media object431. The controller214uses the media object identifier associated with the first media object431to generate the media object image element432that may be selected by a user for inclusion in the media object section430. The male actor playing music represents the second media object433. The controller214uses the media object identifier associated with the second media object433to generate the media object image element434that may be selected by a user for inclusion in the media object section430. The female actor dancing represents the third media object435. The controller214uses the media object identifier associated with the third media object435to generate the media object image element436that may be selected by a user for inclusion in the media object section430. In parsing the auxiliary media data file, the controller214may identify the music being played as a fourth media object437, the associated media object identifier may be the instrument being played by the actor (e.g. second media object). The controller214uses the media object identifier associated with the fourth media object437to generate the media object image element438that may be selected by a user for inclusion in the media object section430. A fifth media object representing the location of the present scene may also be identified from the auxiliary media data file by the controller214. By example, additional identifiers associated additional media objects indicating additional location data can also be included in the object section430of navigation user interface400. In this embodiment, such additional media object image elements can be represented by a generic location indicator.

By presenting the media object image elements432,434,436,438and440, the user may advantageously control navigation within the multimedia content404by selecting any of the media object image elements. Selection of any of the media object image elements automatically skips within the current multimedia content to a next position at which the selected media object is displayed. For example, selection of the first media object image element432causes the controller214to search the records of the auxiliary media data file to locate the selected media object and identify a next time at which the selected media object is present. The controller214automatically causes playback of the multimedia content beginning from the next time as indicated in the auxiliary media data file. This advantageously alleviates the need to manually search through the multimedia content to locate the next occurrence while ensuring that a next occurrence of the media object is not inadvertently missed during the manual seek process. In this example, selecting the first media object image element432representing the male actor dancing results in automatic skipping to a next position within the multimedia content that includes the male actor. In one embodiment, if the multimedia content is paused within a current scene, the automatic skipping to a next position may occur using data in the Time Stamp-Start Field that occurs later than an end time indicated in the Time Stamp-Duration field.

It is important to note that, inFIG. 5, the circles shown in the multimedia content404are intended to facilitate the understanding of the relationship of identified media object with the media object image elements presented in media object section430. These circles are not actually displayed to the user within the navigation interface. Moreover, the display of five media object image elements is for purposes of example only and, depending on the particular position at which the content404is paused, there may be any number of identifiable media objects all of which may be presented in the media object section403. In the instance where there are more identified media objects than are able to be displayed simultaneously, the media content section may include a scrolling feature allowing the user to selectively scroll and selectively change which media object image elements are currently displayed within the navigation interface400.

In an optional embodiment, the media object section430may include an object tagging image element450. By selecting the object tagging image element450, the controller214initiates a tagging algorithm that enables the user to use the input device300to select at least a portion of the currently paused multimedia content404and identify the selected portion as a media object that can be selected by a user. The tagging algorithm responds to user input, for example, via touch screen interface, tracing a circle around a portion of the paused multimedia content. This automatically identifies the selected portion as a media object. Thereafter, the controller may generate a media object input user interface that includes a blank media data file record including any of the information element fields shown in Table 1. The user can selectively choose a media object identifier to be associated with the identified media object and enter data in each information element field that describes the media object identified by the user. The controller214may automatically update the auxiliary media data file to include the user-identified media object thereby enabling display of the user identified media object at any subsequent position within the multimedia content404.

In another optional embodiment, selection of the first media object image element432representing the male actor that is dancing results in the controller214generating a media object information interface500shown inFIG. 6. All data shown in the media object interface500may be derived from the auxiliary media data file associated with the multimedia content being consumed by the user. In this embodiment, the media object information interface500includes a section that includes information describing the previously selected media object431associated with the first media object image element432. For example, the interface500may include the media object identifier501providing a visual cue as to the nature of the media object as well as the name of the media object502(e.g. the name of the actor). The interface may also include at least one object information field504a,504bthat includes any information about the media object. In the case where the object is an actor, information field504amay include a birth date or age and information field504bmay include roles for which the actor is famous. These are merely described to illustrate the point that the interface500presents any type of information about the media object and any such information associated with the media object may be presented in one or more information fields504a,504b.

Also included in interface500is an object position section506that includes a listing of at least one other position in the current multimedia content that the selected media object appears. For example, in the present case, the selected media object is an actor and the object position data may represent various scenes in the movie at which the actor is present. As shown herein, the object position data is provided in an Object Position Table506. Object position table506lists the scenes in which the actor appears along with description information about what the actor is doing in those subsequent scenes and the time duration of those scenes. As shown herein, the actor appears in scene11which, based on the time stamp associated therewith and shown in the information section422ofFIG. 5, represents the current scene. A user can initiate playback of the current scene by selecting the play image element512a. Selection of play image element512amay result in playback at the beginning of the current scene. Alternatively playback can begin at the current time point. Additionally, the object position table506includes an entry for scene13which represents a subsequent position in the multimedia content in which the actor appears. In scene13, the actor is described as running and the time stamp data indicates the start time, end time and duration of scene13. A second play image element512bis provided and, selection thereof, results in the automatic skipping to and playback of the beginning of scene13as per the time stamp data.

The interface500also includes an additional media section514that presents instances of additional multimedia content that includes the selected media object in an additional media table515. The additional media table includes a first row516athat identifies the name of the additional media (“Movie 1”), the manner in which the additional media is presented (“Broadcast”), status information (“Upcoming”) and at least one action item resulting in a particular action to be taken for the respective additional media. Because the first row of table515identifies the movie as “upcoming”, the action items may include Tune520resulting in the tuner tuning a channel on which Movie 1 is to be broadcast and/or Record522resulting in the controller214scheduling a recording of Movie 1 at the indicated time. A second row516bidentifies the additional media as “TV Show 1” that has been recorded and stored. The action items include Play524, selection of which, results in the stored TV Show 1 being retrieved and played back on the display device114(FIG. 1). The action items may also include Delete which causes the stored TV Show to be deleted from the storage device212. A third row516cidentifies the additional media as “Movie 2” that is a new release and is available for purchase. The action items associated with Movie 2 include Purchase528that initiates a multimedia content purchase algorithm that enables the user to enter payment information and purchase Movie 2 from the content provider and/or a 3rdparty.

The description of data displayed inFIG. 6is described for purposes of example only and the interface500may selectively display any information element that is included in the auxiliary media data file and is associated with the selected media object.

FIG. 7represents another embodiment of the navigation user interface600in accordance with invention principles. The navigation user interface600includes the content section602, control section610and media object section630. These sections are similar in design and operation to their counterparts402,410and430, respectively, inFIG. 5and need not be further described. The additional feature of navigation interface600relates to selection of the type of media objects able to be displayed in the media object section630. Thus, interface600includes object type selection section660that includes at least one user selectable image element representing at least one type of media object included in the multimedia content currently being viewed by the user. The different object types included in the multimedia content may be included to populate section660by the controller214which parses the auxiliary data file to identify each unique type of media object included in the multimedia content. Upon identifying each type of media object, the controller214causes section660to be populated with user selectable image elements representative of the types of media objects. Selection of the image elements representative of the types of media objects results in the controller214automatically updating the media object image elements to be displayed within the media object section630. In one embodiment, the navigation interface defaults to display all types of media objects resulting in any media object image element associated with a media object at the current time position to be displayed. In response to selection of a respective type of media object from within section660, the controller214automatically modifies the media object section to omit media object image elements of the type selected in section660. One skilled in the art will appreciate that the opposite operation may also be implemented whereby the system defaults to display no media objects and, in response to selection of object types in section660, the controller214automatically modifies the media object section630to include the selected object types.

In exemplary operation, the controller has determined that the current multimedia content includes Object Type1662, Object Type2664, Object Type3665and Object Type N present in object type selection section667. In this embodiment, Object Type1662may represent actors/actresses and selection of Object Type1image element662results in the controller214displaying the first, second and third media object image elements432,434and436, respectively because these media object image elements are associated with actors/actresses. Object Type2664may represent an item in the current scene and selection of the Object Type2664image element results in the controller214displaying the fourth media object image element438because the fourth media object image element represents a musical instrument (e.g. an item in the scene). Object Type3665may represent a location at which the current scene takes place and selection of the Object Type3665image element results in the controller214displaying the fifth media object image element440because the fifth media object image element represents location information. The inclusion of Object Type N is provided only as an example to illustrate the section660may be populated with any number of object types that are present in the multimedia content thereby advantageously filtering the type of media object image elements that will be presented. Because any aspect of the multimedia content may be tagged and identified as a media object, filtering the types of media objects to be displayed in the media object section630advantageously enables the user to be more targeted in the types of objects that are used to navigate within the multimedia media content.

The above discussed operation is described with respect to the multimedia content being paused. However, in an optional embodiment, the navigation user interfaces400or600are continually displayed and output to a second screen device, e.g. a tablet, thus enabling the improved navigation within the multimedia content in real time. In this embodiment, the controller214continually queries the auxiliary media data file to identify media objects from within the multimedia content and that are presently being displayed either within the content section402/602or on a primary display device114(FIG. 1). The controller214, in response to continually identifying media objects, automatically updates the media object section430/630with media object image elements corresponding to the identified media objects thereby presenting the media object image elements in real-time and enabling real-time navigation within the multimedia content. The navigation and automatic skipping to a later position including the media object may be realized on the display device114or on the touch screen interface. In another embodiment, navigation may be realized on the display device114while the controller generates the media object information interface500ofFIG. 6and outputs the generated interface500to the touch screen display device thereby advantageously facilitating quick navigation within the multimedia content while simultaneously providing the user with media object information on the touch screen device via the media object information interface500ofFIG. 6.

FIG. 8is a flow diagram detailing operation of the system in accordance with invention principles. The flow diagram details an algorithm for operating the system that interacts with multimedia content data. At step702, a current time point for a frame of the multimedia content data is determined. In one embodiment, step702may occur after pausing playback of the multimedia content data. At step704, at least one user selectable media object included in the frame at the current time point and that occurs at least one further time point of the multimedia content data is identified.

In one embodiment, the media object includes at least one of (a) an actor/actress in the multimedia content, (b) a character in the multimedia content, (c) an item/object in the multimedia content, (d) a particular location in the multimedia content, (e) an audio component of the multimedia content, (f) any other aspect of the multimedia content that occurs more than once within the content, and (g) an other aspect of the multimedia content that may be used to provide additional information to a user.

In another embodiment, the identification step in704occurs by receiving an auxiliary media data file associated with the multimedia content data, the auxiliary media data file including at least one record corresponding to the at least one media object, parsing the at least one record of the auxiliary media data file based on the determined current time to identify a presence of at least one media object at the current time and outputting user selectable data representing at least one media object at the current time for inclusion into the user interface.

At least one media object identifier associated with the identified at least one user selectable media object is acquired in step706. In one embodiment, step706may occur using an auxiliary media data file that includes media object identifiers associated with each of the at least one media object from which the at least one media object identifier is obtained. In another embodiment, step706may include using an auxiliary media data file that includes information describing respective media objects and using the information describing respective media objects to search an external source for data to be used as the media object identifier.

A user interface display image including the frame of multimedia content and the at least one media object identifier is generated in step708. In one embodiment, step708may be performed using an auxiliary media data file associated with the multimedia content, the auxiliary media data file including all media objects included in the multimedia content data and identifies each point in time that each media object occurs in the multimedia content data. In another embodiment, step708may further include, for each media object identified at the current time, associating data representing a point in time subsequent to the current time with the media object identifier and inserting the media object identifier having the data representing the subsequent point in time into the user interface enabling the user to skip to the at least one subsequent point in time within the multimedia content upon selection of the media object identifier. In a further embodiment, step708may also include continually updating the generated user interface during playback with media object identifiers associated with identified media objects in the multimedia content data at the current time. In another optional embodiment, step708may also include generating a user selectable image element enabling a user to identify, as a media object, at least a portion of the frame of the multimedia content.

In step710, the generated user interface enables selection, by a user, of the at least one media object identifier and, in step712, playback of the multimedia content at the at least one further time point is automatically initiated in response to selection of the at least one media object identifier. In another embodiment,712may also result in generating a second user interface in response to selection of the at least one media object. In this embodiment, the second user interface includes the media object identifier associated with the selected media object, information describing the selected media object and at least one action image element for initiating at least one action associated with the media object. In a further embodiment, the information describing the selected media object includes time stamp data identifying each instance the media object appears within the multimedia content data and an action image element associated with each instance the media object appears, and activation of the action image element initiates playback of the multimedia content at a respective instance associated with the selected media object. In yet a further embodiment, the information describing the selected media object includes at least one other multimedia content data including the selected media object and provides an action image element associated with each of the at least one other multimedia content data, wherein selection of a respective action image element initiates at least one of (a) initiating a purchase of the at least one other multimedia content data, (b) tuning to a channel on which the at least one other multimedia content data is being transmitted, (c) scheduling a recording of the at least one other multimedia content data, and (d) deleting a previously recorded instance of the at least one other multimedia content data.

In an optional embodiment of the invention, Boolean logic sequences can be used to include groups of multiple objects or exclude certain selected objects from a search. For example, if a scene displays four objects, a user can select objects 1 and 2 (with an “AND”) combination and select object 3 (with a “NOT”). The basis of this search would have to require both objects 1 and 2 and exclude object 3 from any search results. Additional operators can be used such as OR, XOR, and the like. The Boolean operators can be added by using an user interface or alphanumeric entry in accordance with the disclosed principles above.

Although embodiments which incorporate the teachings of the present disclosure have been shown and described in detail herein, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings. Having described preferred embodiments of a system, method and user interface for content search (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments of the disclosure disclosed which are within the scope of the disclosure as outlined by the appended claims.