Abstract:
A video data structure and a video browser for video browsing are disclosed. The data structure includes segment group information indicating that a first segment group represents highlights sharing at least two objects in a program&#39;s content. The segment group information is used to display the highlights represented by the first segment group. The present invention can be implemented to allow users to easily understand relations between characters and changes in relations between characters in a movie or a drama by displaying significant events in the corresponding relations and by displaying character relations in a tree structure.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS  
       [0001]     The present application is a continuation of co-pending application Ser. No. 09/645,613, filed Aug. 25, 2000, the entirety of which is incorporated by reference herein. This application claims the benefit of Korea Patent Application No. 35688/1999, filed Aug. 26, 1999. 
     
    
     BACKGROUND  
       [0002]     1. Field of the Invention  
         [0003]     The present invention relates to a video browser, and more particularly to a video browser for browsing a video based on relations between characters.  
         [0004]     2. Background of the Related Art  
         [0005]     Typically, users simply view movies and/or dramas as broadcasted through a TV or played at a movie theater. However, a user may wish to view a particular movie or drama at a particular time, or wish to view only a particular section of a movie or a drama. Accordingly, various techniques which enable selective watching of a movie/drama or sections of a movie/drama have been suggested.  
         [0006]     In the related art, for example, various video data may be represented or classified into format portion, index portion, media portion, segment portion, target portion, and/or representation portion. Also, data on various characters or objects, such as a name of an object, position on the screen, or numeric data with relation to a segment of the video data in which the object appears, may be represented by the target and representation chunk. Accordingly, a user can select an object through a table and reproduce for display a particular segment where the object is shown in the video.  
         [0007]     In another related art, various additional data of a video data are obtained before, during or after the production of the video data. Thereafter, an additional information table of the obtained data is composed and provided to users. Namely, the additional data table may include a position where an actor appears, a position where a character of the actor appears, and a position where stage properties appear, such that a scene can be reproduced as selected by a user through the additional data table. For example, if a user selects a stage property, information on the selected stage property, such as the manufacturer and price, may be displayed on a screen, and the user may be able to connect with the manufacturer or a seller of the stage property through a network connection.  
         [0008]     In still another related art, recording information on each segment of a video in a video map has been suggested. That is, information, such as the degree of violence, the degree of adult content, the degree of importance of content, characters positions, and the degree of difficulty in understanding, may be indicated for each segment of a video in the video map. Thus, the user may set a degree of preference for one or more items of the video map, and only segments of the video meeting the set degree of preference would be reproduced, thereby limiting a display of particular contents to unauthorized viewers.  
         [0009]     Similarly, other techniques in the related art as described above provide items simply arranged without any relation to the objects appearing in the movie or drama, based upon the selection of the user. However, the content of a movie or drama generally builds around relations between characters, places and events. For example, relations between characters may not change from beginning to the end of the story or may continuously vary. Moreover, since one or more characters relate to a specific character in the movie or drama, the browsing method in the related art substantially fails to provide an accurate understanding of the story of the movie or drama to the user.  
         [0010]     Therefore, techniques in the related arts have disadvantages in that it is difficult to understand a video centering on relations among characters according to the development of events, changes of relations, and relations among characters and places as events develop.  
       SUMMARY  
       [0011]     Accordingly, an object of the present invention is to solve at least the problems and disadvantages of the related art.  
         [0012]     Another object of the present invention is to provide a video browser in which event segments showing changes in relations between characters are summarized and displayed based on a video data structure which includes characters in a video that significantly act on development of an event, and includes a constant and variable relations between characters.  
         [0013]     A still another object of the present invention is to provide a video browser in which main events that significantly act on variable relations between characters are connected with corresponding variable relations based on a video data structure which includes characters in a video that significantly act on development of an event, and includes a constant and variable relations between characters.  
         [0014]     A further object of the present invention is to provide a video browser in which a key frame of event segments which show a corresponding relation between characters is displayed based on a video data structure which includes characters in a video that significantly act on development of an event, and includes a constant and variable relations between characters.  
         [0015]     A further object of the present invention is to provide a video browser in which a constant relation and changes in relations between characters are displayed depending on preset relation types based on a video data structure which includes characters in a video that significantly act on development of an event, and includes a constant and variable relations between characters.  
         [0016]     Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and advantages of the invention may be realized and attained as particularly pointed out in the appended claims.  
         [0017]     To achieve the objects and in accordance with the purposes of the invention, as embodied and broadly described herein, a video browsing system for browsing a video based on a data structure in which a constant and variable relations between characters of a video are connected with characters and corresponding event segments, comprises (a) browsing and displaying a video segment showing a constant relation between characters and variable relations between characters based on an event segment showing a corresponding constant relation and variable relation, and (b) displaying a video segment of a corresponding event segment in response to a user&#39;s selection.  
         [0018]     The video browser of the present invention is characterized in that event segments showing a constant relation between characters and changes in the relations is summarized and displayed. Also, main events that significantly act on variable relations are connected with corresponding variable relations and characters.  
         [0019]     Moreover, the video browser of the present invention is characterized in that constant relation and variable relation between the characters are browsed and displayed as a constant relation and a variable relation between characters selected by the user. The constant relation and variable relation between characters may also be browsed and displayed as a constant relation and a variable relation between every characters related with a selected character.  
         [0020]     In still another video browser of the present invention, the constant relation and variable relation between characters are displayed in a tree structure. The video browser of the present invention may display a constant relation and variable relation between characters depending on preset relation types. The relation types may be set in semantic types such as “family relation,” “business relation,” and “social relation.” 
         [0021]     Furthermore, the video browser of the present invention is characterized in that event segments showing constant and variable relations between characters are displayed as key frames. Finally, each key frame may show a corresponding relation between characters. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0022]     The present invention will be described in detail with reference to the following drawings in which like reference numerals refer to like elements wherein:  
         [0023]      FIG. 1  shows an example of a video data structure based on character relation according to the present invention;  
         [0024]      FIG. 2  shows an example of a video data structure in which events showing character relations are summarized in accordance to the present invention;  
         [0025]      FIG. 3  shows a video browser based on the video data structure of  FIG. 2 ;  
         [0026]      FIG. 4  shows a video data structure of an event segment showing variable relations between characters;  
         [0027]      FIG. 5  shows a video browser based on the data structure of  FIG. 4  in accordance with the present invention;  
         [0028]      FIG. 6  shows an example of a video browser according to the present invention; and  
         [0029]      FIG. 7  shows another example of a video browser according to the present invention. 
     
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS  
       [0030]     Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.  
         [0031]      FIG. 1  shows a video data structure based on character relations in accordance with the present invention for a video browser based on content. A video browser based on content is disclosed in U.S. Pat. No. 6,602,297, entitled “Motional Video Drowsing Data Structure and Browsing Method Therefor,” and is fully incorporated herein.  
         [0032]     Referring to  FIG. 1 , a visual description scheme (DS)  101  is divided into a visualization DS  102 , a syntactic structure DS  103 , and a semantic structure DS  104 . The visualization DS  102  is organized into a highlight view DS  105  for displaying a summarized data as a video highlight, and a key frame view DS  106  for displaying summarized data based on a key frame. Because, a video plot can be summarized briefly or with greater amounts of detail, the highlight view DS  105  enables a display of multi-levels of summarized data with a level  107 , a view type  108  and a time DS  109 . Similarly, the key frame view DS  106  enables a display of multi-levels of summarized data with a level  110  and a time DS  111 .  
         [0033]     Namely, the highlight view DS  105  is organized into a level  107  which has multiple levels of highlight data based, upon a degree of detail in summarizing a video. In such case, summary data in each level may include a corresponding period in the time DS  109  for use in highlight. The key frame view DS  106  is also organized into a level  110  which has multiple levels of summarized data based upon a degree of detail. In such case, summary data in each level may include a corresponding period in the time DS  111  for use as a key frame.  
         [0034]     The syntactic structure DS  103  is used for displaying the actual video and includes actual video segments to be displayed. The syntactic structure DS  103  is organized into actual video segments in segment DSs  112 , and corresponding temporal positions of video segments in the time DS  113 .  
         [0035]     The semantic structure DS  104  includes additional information describing a video, and is organized into an event DS  114  including event information, an object DS  117  including object information, and an event/object relation graph DS  119  including information describing relations between objects and places, and corresponding events to the relations. Namely, the event DS  114  describes events, and the object DS  117  describes objects such as characters and places. The event/object relation graph DS  119  describes a constant relation or changes in a relation between characters, a relation between an object and a place, or a relation between an object and an event.  
         [0036]     Here, a constant relation means either a relation between characters that cannot change throughout a video, such as a parent to child relation, or a relation which is most representative of the relations between characters.  
         [0037]     When an event is selected to display a video segment corresponding to the event, the event DS  114  of the semantic structure DS  104  is used. The event DS  114  is divided into a Reference to Segment  115  including information necessary for displaying a segment of a video corresponding to the selected event and an annotation DS  116  including information which connects events with actual positions of the events in a video and information for explaining events in a video. Particularly, the event DS  114  is used in displaying a selected event and connects the selected event with a position of the event in a video data through the Reference to Segment  115 , and the contents of a selected event is annotated through the annotation DS  116 .  
         [0038]     The object DS  117  is organized into an object annotation DS  118  including information for describing objects such as characters or places. That is, the object DS  117  is used in displaying characters, and detailed information of the characters can be summarized or annotated through the annotation DS  118 .  
         [0039]     The event/object relation graph DS  119  includes information on a constant relation and variable relation between characters. The event/object relation graph DS  119  is organized into an entity relation  120  with a return which allows a display of character relations in a tree structure in which a relation may have subordinate relations. According to the present invention, constant relations between characters are placed in an upper level of the tree while changes of relations between the characters are placed in a lower level of the tree.  
         [0040]     The entity relation  120  is also divided into a relation  121 , a Reference to Object  124 , and a Reference to Event  125 . The relation  121  is organized into a name  122  including information on the titles of relations, and a type  123  including information on the nature of relations. For example, a nature of relation may be ‘family’ and a title of relation may be ‘spouse.’ The Reference to Object  124  connects related characters with each other and the Reference to Event  125  connects events which shows particular relations.  
         [0041]     In the above video data structure, the notation above each data such as {0,1}, {0,*}, or {1,*} indicates the number of data for the corresponding data. For example, the notation of {0,1} for the visualization DS  102  indicates that the visual DS  101  can have zero or one visualization DS. On the other hand, the notation of {0,*} for the segment DS  112  indicates that the syntactic structure DS  103  may have from zero to any number of segment DS.  
         [0042]      FIG. 2  shows a method of using a data structure of  FIG. 1  to easily understand and browse a video based on relations between characters. Referring to  FIG. 2 , a representative relation (constant relation) between ‘character  1 ’ and ‘character  2 ’ is placed at the top of the tree structure, and variable relations (relation  1 ˜relation  4 ) between ‘character  1 ’ and ‘character  2 ’ are placed at the bottom of the tree structure. Also, main events which are significant in a change of relation in a variable relation are placed between variable relations. The main events are summarized video data and are displayed by highlight data.  
         [0043]     Here, the events are connected through the Reference to Event  125  in the entity relation  120  and the video segments are connected through the Reference to Segment  115  in the event DS  114 . Thereafter, an actual video segment Is displayed by connecting the time DS  113  of the segment DS  112  with the time DS  109  of the level  107  in the highlight DS  105  of the visual DS  102 .  
         [0044]      FIG. 3  shows an example screen of a video browser based on the video data structure of  FIG. 2 . Referring to  FIG. 3 , ‘character n’ of a video is displayed on a character screen  301 , and a video is displayed on a main screen  302  according to a selection received through a user interface  303 .  
         [0045]     For example, if a user selects ‘character  1 ’ and ‘character  3 ’ from the character screen  301 , main events which are significant in bringing about a change in the variable relations between ‘character  1 ’ and ‘character  3 ’ can be browsed based on the data structures of  FIGS. 1 and 2 . As a result, contents corresponding to the main events are summarized and displayed on the main screen  302 .  
         [0046]      FIG. 4  shows another method of using a video data structure including information on variable relations of characters and main events significant in the variable relations.  
         [0047]     Referring to  FIG. 4 , a representative relation (constant relation) between ‘character  1 ’ and ‘character  2 ’ is placed at a top of a tree structure, and variable relations (relation  1 ˜relation  4 ) between ‘character  1 ’ and ‘character  2 ’ are placed at the bottom of the tree structure. Main events (event  1 ˜event  3 ) which are significant in bringing about a change in the variable relations are placed between the variable relations. The method of  FIG. 4  is realized in such a manner that a main event segment which brings a new relation is connected with the variable relations through the Reference to Event  125  of the entity relation  120 .  
         [0048]      FIG. 5  shows a video browser in accordance with the present invention based on the data structure of  FIG. 4 . Referring to  FIG. 5 , characters in a video are displayed on a character screen  501 , and connection between variable relations and events are displayed on a character relation-variable event screen  502 . Also, an event segment corresponding to a variable relation can be reproduced and displayed on a main screen  503 .  
         [0049]     For example, if a viewer selects ‘character  1 ’ and ‘character  3 ,’ variable relations between ‘character  1 ’ and ‘character  3 ,’ and main events which are significant in bringing about a change in the variable relation are respectively displayed on the screen  502 . At this time, a relation or event may be displayed by an key frame or annotation. Also, a video segment corresponding to an event such as ‘event  2 ’ selected from the screen  502  is reproduced and displayed on the main screen  503 .  
         [0050]      FIG. 6  shows another example screen of a video browser according to the present invention. Referring to  FIG. 6 , main characters of a video are displayed on a character screen  602 . Also, characters having relations with a character selected from the character screen  601 , and a constant relation and variable relations between the selected character and related characters are displayed on a relation screen  602 .  
         [0051]     At this time, a constant relation between the selected character and a related character is displayed on the top of a relation tree structure while variable relations between the selected character and the related character is displayed on the bottom of the tree structure.  
         [0052]     Furthermore, a key frame of events significant in both the constant relation and variable relations is displayed on a main scene screen  603 . Here, key frames of event segment corresponding to a type(s) of event may be displayed on the main scene screen according to a user selection through a selection screen  604 . Namely, key frames of main event segments which directly show a selected relation or key frames of secondary event segments which indirectly show the selected relation, or both can be displayed according to the user selection. For example, if a selected relation is a husband and wife, a main event segment may be a video segment of a marriage ceremony while a secondary event segment may be a video segment of third parties discussing the marriage ceremony. An event segment corresponding to an event, for example a key frame, selected from the main scene screen  603  can thus be reproduced and displayed on the main screen  605 .  
         [0053]     For example, when a user selects ‘character  1 ’ from among the characters in the character screen  601 , other characters ‘character  2 ’˜‘character  4 ’ related with ‘character  1 ’ are displayed on the relation screen  602 . If ‘relation  2 ’ with ‘character  2 ’ is selected from the relation screen  602 , event segments corresponding to ‘relation  2 ’ with ‘character  2 ’ is displayed on the main scene screen  603  as key frames. Also, a period of a video corresponding to ‘event  6 ’ selected from the main scene screen  603  is reproduced and displayed on the main screen  605 .  
         [0054]     At this time, the video browser of  FIG. 6  can be implemented by setting a relation type in the data structure of  FIG. 1 . In other words, events connected with a relation may vary depending on whether the type  123  of the relation  121  is the direct or indirect type as discussed above. Also, the events are connected through the Reference to Event  125  of the entity relation  120  and event segments are connected through the Reference to Segment  115  of the event DS  114 . Thereafter, an actual video segment is displayed by connecting the time DS  113  of the segment DS  112  with the time DS  111  of the level  110  in the key frame view DS  106  of the visual DS  102 .  
         [0055]      FIG. 7  shows still another example screen of a video browser according to the present invention. Referring to  FIG. 7 , characters of a video are displayed on a character screen  701 ; and character having relations with a selected character, and a constant relation and a variable relation between the selected character and related characters are displayed on a relation screen  702 . Here, the constant and variable relations may categorized into different natures of relations and relations corresponding to a particular nature(s) of relation selected through a selection screen  703  may be displayed in the relation screen. The nature of relation may be a family relation, a business relation, or a social relation.  
         [0056]     Thus, a key frame of events significant in the selected nature(s) of relation is displayed on a main scene screen  704 . A video segment corresponding to the selected relation and event is reproduced and displayed, on a main screen  705 , For example, when a user selects ‘character  1 ’ from characters displayed on the character screen  701 , characters (‘character  2 ’˜‘character  4 ’) ‘aving a social relation with “character  1 ” may be displayed and an event segment of ‘event  6 ’ corresponding to ‘social relation  2 ’ with ‘character  2 ’ can be reproduced and displayed on the main screen  705 .  
         [0057]     As in  FIG. 6 , a constant relation, between a selected character and a related character is displayed on the top of a relation tree structure while variable relations between the selected character and the related character is displayed on the bottom of the tree structure.  
         [0058]     The video browser of  FIG. 7  can be realized by setting a relation type  123  of the relation data  121  in the data structure of  FIG. 1 . In other words, events related with the relation data  121  may vary depending on whether the relation type  123  of the relation data  121  is a ‘family relation,’ a ‘business relation,’ or a ‘social relation.’ As a result, browsing of  FIG. 7  can be performed. At this time, the events are mutually connected through the reference to event  125  of the real relation  120 . Segments are mutually connected through the reference to segment  115  of the event DS  114 . Then, the time DS  113  of the segment DS  112  is connected with the time DS  111  of the level  107  displayed in the key frame DS  106  of the visual DS  102 .  
         [0059]     As aforementioned, the video browser based on a character relation according to the present invention has the following advantages. The event periods of the variable relation between the characters are summarized and displayed, and the main events concerned in the variable relation between the characters are connected between the variable relations and then displayed.  
         [0060]     Furthermore, the data structure for browsing a video is based on a relation between characters and a variable relation between them. A key frame of an event period showing a corresponding relation between the characters clearly or directly, or a key frame of an event period showing it with hint or indirectly can be displayed based on the data structure. Also, the relations between the characters are divided into a family relation, a social relation, and an emotion relation to be selectively provided to a user, so that video browsing can be performed based on the relations between the characters.  
         [0061]     Accordingly, contents of the video can be browsed based on characters that significantly act on development of a story of a movie or drama, the relation between the characters, and the variable relation between them. After all, the user can easily browse the video based on characters, event and relation which are concerned in development of a story and important for contents of a real video. Also, the user can understand and browse a desired video in various types based on the video data structure which represents characters and their relation.  
         [0062]     Furthermore, the present invention may be applied to a VOD system in the broadcasting field so that the user can view a desired part. This could lead to an advantage that the user can reproduce and view the desired part within short time and to effective utilization of the network source. Also, the present invention may be applied to a video player for home use and a broadcasting video player so that the video browsing environment can be provided to easily browse a desired part of a movie or drama stored in the video player.  
         [0063]     The foregoing embodiments are merely exemplary and are not to be construed as limiting the present invention. The present teachings can be readily applied to other types of apparatuses. The description of the present invention is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art.