Patent Publication Number: US-2006001833-A1

Title: Moving picture search apparatus

Description:
BACKGROUND OF THE INVENTION  
      1. Field of the Invention  
      The present invention relates to a method for effectively searching the moving picture information in the multimedia information utilizing environments, and it also relates to a system using the same.  
      2. Description of the Related Art  
      As a conventional moving picture search method, there was a method wherein entire pixels or picture elements per picture frame unit are used. With this method using the entire pixels, it is impossible to perform an effective searching of the moving picture because the amount of information of the moving picture to be calculated and processed is enormous.  
      There was another conventional method wherein a still picture is searched and the feature amount such as brightness value and dispersion value per frame is used. This method is applicable to the still picture. If this method is applied to the moving picture, it is necessary to conduct the per-frame comparison and the dynamic features with respect to time of the moving picture cannot be used.  
      In the case where the feature amount of the picture in the conventional still picture matching is applied to the moving picture search and the feature amount which is calculated from the entirety of the picture is used, there are problems in that it takes a substantive and considerable time to process the enormous moving picture information and it is impossible to search such a scene of the moving picture which has less variation. Further, with this method, it is not possible to capture the dynamic features with respect to time that are characteristics of the moving picture. Still further, since the feature amount such as the average brightness value and the dispersion value for each frame is dependent on the input data, it is necessary to change, in accordance with the input data, a threshold value which becomes a reference for the matching so as to achieve an appropriate matching, and the processes are complicated.  
      As explained above, the conventional picture matching method is a search method which is based on entirety of the picture and has a problem such that the amount of calculation process is large which problem is to be solved. The present invention is aimed at providing a high-speed moving picture search method with a small amount of information by calculating the feature amount of the moving picture.  
      Further, in this invention, as compared to the conventional method in which the average brightness value and the dispersion value per frame are used, it is easy to establish a threshold value without relying on the input data by using the means for calculating the feature amount of the moving picture as a local region unit and the means for calculating the histogram of the local region feature amount of the moving picture and then calculating the correlation value before and after the frame.  
     SUMMARY OF THE INVENTION  
      According to the invention, there is provided a method for searching moving pictures and also a system using the same, comprising: 
          for searching, a moving picture input unit for inputting the moving picture data for search;     a feature amount calculation unit including: 
            a local region extracting unit for extracting one or more square or any given shape local regions for calculating the feature amount with respect to the input moving picture data for search;     a signal extraction unit for extracting or conversion-extracting picture signal components such as brightness signals, color difference signals, each RGB color space component, each XYZ color space component, each uniform color space component, and each Munsell color space component of the feature amount calculation region extracted from the input moving picture for search; and     a feature amount table creation unit for creating a feature amount table used for the searching process, from the feature amount calculated based on the input moving picture for search;    
            for creating a database, a moving picture input unit for inputting the moving picture data for database;     a moving picture database registration unit for registering the input moving picture data for database in a moving picture database;     a database in which the input moving picture data for database is registered;     a feature amount calculation unit including: 
            a local region extracting unit for extracting one or more square or any given shape local regions for calculating the feature amount with respect to the input moving picture data for database;     a signal extraction unit for extracting or conversion-extracting picture signal components such as brightness signals, color difference signals, each RGB color space component, each XYZ color space component, each uniform color space component, and each Munsell color space component of the feature amount calculation region extracted from the input moving picture for database; and     a feature amount table creation unit for creating a feature amount table used for the searching process, from the feature amount calculated based on the input moving picture for database;    
            a table database registration unit for registering the created feature amount table for database in a feature amount table database;     a table database in which the created feature amount table for database is registered;     a search unit including: 
            a table database read out unit for reading out the database feature amount table used for search process, which table was processed by the feature amount calculation unit on the input moving picture for search and is used for the search process;     a matching unit for extracting a feature amount used for matching from the feature amount table created based on the input moving picture for search and the registered feature amount table in the feature amount table for database, outputting the extracted result to a matching result judgment unit, storing the matching result from the matching result judgment unit, and after the completion of the matching process, outputting the matching result to a moving picture database read out unit; and     a matching result judgment unit for performing judgment with respect to a threshold value, based on the feature amount of the moving picture for search and the feature amount of the moving picture for database extracted by the matching unit, and outputting the matching result to the matching unit;    
            a moving picture database read out unit for receiving the matching result from the matching result judgment unit, and reading out a search result moving picture corresponding to the matching result, from the moving picture database; and     a search result output unit for outputting the search result moving picture read out by the moving picture database read out unit.        

      With this moving picture search method and system, a local region is extracted from the picture frame and a histogram of the brightness information is calculated from the extracted local region. Then, the calculated histogram is compared before and after the frame, and the correlation value thereof is obtained. It is possible to achieve a further high-speed moving picture matching by grouping the above correlation values with respect to a certain threshold value.  
      Further, as compared to the average brightness value and the dispersion value of each frame, the correlation value does not depend upon the input data and is not necessary to be varied with respect to the threshold value input when it is grouped. As a consequent, it is possible to establish the threshold value appropriately by using the correlation value as the feature amount. Further, by using the grouped feature amount as the search parameter, it is possible to control or limit the number of parameters which are used for searching the moving picture scenes and, thus, the time required for the matching process can be effectively shortened. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       FIG. 1  is a block diagram showing the basic principle of the present invention;  
       FIG. 2  is a schematic block diagram of a hardware structure using a single terminal unit for realizing the present invention;  
       FIG. 3  is a schematic block diagram of a hardware structure using a plurality of terminal units for realizing the present invention;  
       FIG. 4  is a diagram showing local regions of an n-frame and an (n+1) frame;  
       FIG. 5  is a diagram showing one or more local regions where the feature amount per local region unit is calculated for the moving picture;  
       FIG. 6  is a diagram for explaining the way how the local region is actually extracted;  
       FIG. 7  is a diagram for explaining the way how the color space signal is extracted;  
       FIG. 8  is a diagram for explaining the grouping of the correlation value as the time relation feature amount, calculated from the signal information;  
       FIG. 9  is a diagram for explaining the creation process of the feature amount table;  
       FIG. 10  is a block diagram showing the search processes in the present invention;  
       FIG. 11  is a flow chart showing the procedures at the search data input side and the procedures of matching process;  
       FIG. 12  is a flow chart showing the procedures at the database side that deals with the feature amount data for comparison;  
       FIG. 13  is a block diagram showing the embodiment in which the local region extraction process and the signal extraction process are changed in their order;  
       FIG. 14  is a flow chart showing the procedures at the search data input side and the procedures of matching process; and  
       FIG. 15  is a flow chart showing the procedures at the database side that deals with the feature amount data for comparison. 
    
    
     PREFERRED EMBODIMENTS OF THE INVENTION  
      Now, preferred embodiments of the invention will be explained with reference to the accompanying drawings. First, we refer to  FIG. 1  which is a block diagram showing the fundamental principle of the present invention.  
      From the moving pictures  1  input by an operator, the feature amount is calculated and the feature amount table  2  is created. In relation to the moving pictures for database, a plurality of feature amount tables  3  for registering are created in advance and stored. Collation (matching process or matching judgment) between the created feature amount table  2  and the registered feature amount table  3  is performed. If the feature amount table  2  corresponds to the feature amount table  3 , the search result list is sent to the moving picture database  4 . Based on this list, the moving picture is output from the moving picture database  4  whereby the desired moving picture  5  is obtained.  
      Next, with reference to  FIG. 2 , a hardware construction consisting of a single terminal  6  is explained. Moving pictures for search are stored in an external storage unit  11  through an input and output interface  8 . The moving picture information is specified by a mouse (input device)  7 , and the feature amount thereof is calculated by a memory (internal storage unit)  12  and a central processing unit  9 . The obtained feature amount data is stored in the internal storage unit  12  or the external storage unit  11  through the input and output interface  8 .  
      On the other hand, moving pictures for database are stored in the external storage unit  11  through the input and output interface  8 , and the feature amount thereof is calculated by the memory (internal storage unit)  12  and the central processing unit  9 . This calculated feature amount is stored in the internal storage unit  12  or the external storage unit  11  through the input and output interface  8 .  
      The calculated data is read out to the memory  12  from the external storage unit  11 . The central processing unit  9  performs the search process and the search result is forwarded to the internal storage unit  12  or the external storage unit  11 .  
      The central processing unit  9  reads out the search result picture list from the internal storage unit  12  or the external storage unit  11  through the input and output interface  8 , and the search result picture is output to the external storage unit  11  or the display unit  10  through the input and output interface  8 .  
      Next, with reference to  FIG. 3 , a hardware construction having a plurality of terminals according to the present invention will be explained.  
      In relation to the moving picture for search, the moving picture information stored in an external storage unit  20  or the moving picture information stored in an external storage unit  23  at the server side  14  which is connected to a network (numerals  21 ,  22 ) through an input and output interface  17  is specified by a mouse (input device)  16  at the client side  13  through the input and output interface  17 . A memory (internal storage unit)  24  and a central processing unit  18  at the client side  13  calculate the feature amount of the specified moving picture information. The calculated feature amount data is stored in the internal storage unit  24  or the external storage unit  20  through the input and output interface  17 . Or, the calculated feature amount data is transmitted to the server side  14  and is stored in an internal storage unit  29  or the external storage unit  23  through an input and output interface  26  at the server side  14 .  
      On the other hand, in relation to the moving picture for database, the moving picture information stored in the external storage unit  23  is read out to the memory (internal storage unit)  29  through the input and output interface  26  at the server side  14 , and the feature amount thereof is calculated by the memory (internal storage unit)  29  and the central processing unit  27 . The calculated feature amount data is stored in the internal storage unit  29  or the external storage unit  23  through the input and output interface.  
      The calculated data is read out to the memory  29  from the external storage unit  23  and, then, the searching process is conducted by the central processing unit  27 . The search result is forwarded to the internal storage unit  29  or the external storage unit  23  through the input and output interface  26 .  
      The central processing unit  27  reads out the search result picture list from the internal storage unit  29  or the external storage unit  23  through the input and output interface  26 , outputs the search result picture to the external storage unit  23  through the input and output interface  26 , and transmits (numeral  22 ) the search result picture to the terminal at the client side  13  through the input and output interface  26 . The search result picture is stored in the external storage unit  20  or it is output to a display unit  19  at the terminal at the client side  13 .  
       FIGS. 4-7  are schematic diagrams for explaining the calculation of the feature amount which is the main feature of the present invention.  
      First, the extraction of the local region is explained. At the searching of the moving picture, local regions are extracted within a picture frame, and histogram of the brightness information is calculated from the local regions. The calculated histograms are compared with each other before and after the frame, and the correlation value thereof is obtained. The grouping of the correlation values with respect to a certain threshold value makes it possible to realize a high-speed matching of the moving pictures. The correlation value does not depend on the input as compared to the average value of the brightness value or the dispersion value, and it is not necessary to change the threshold value with respect to the input when the correlation values are grouped. As a result, the effective search can be realized by using the correlation value as the feature amount. In this invention, since the change in histogram for the moving picture is obtained before and after the frame in terms of time, it is possible to search such scene as a minute variation scene, a cameral panning scene or a camera zooming scene by using the histogram correlation values of the local regions, which scene cannot be searched based on the feature amount from the entire picture heretofore.  
       FIG. 4  shows an n-frame picture and an (n+1)-frame picture. For example, histograms of brightness values between the local regions, that is, between the local regions x 1 , x 2 , x 3 , x 4 , x 5  and the local regions y 1 , y 2 , y 3 , y 4 , y 5  within the picture are calculated and the correlation values thereof are obtained. By repeating this procedure, the correlation value distribution is calculated from the input moving picture information. The obtained correlation value distribution is classified into two groups with respect to a certain threshold value using the continuity in terms of time, and the structure number is used for the matching parameter. The moving picture search method and system utilizing this matching parameter is superior in the processing speed and the search precision.  
       FIG. 5  shows exemplary local regions wherein one or more feature amounts are calculated for the local region units of the moving picture. The feature amount can be calculated for the local region unit without relying on the vertical and horizontal ratio of the picture frame.  
       FIG. 6  is a diagram for explaining the actual extraction of the local regions. In the case where the local regions K are extracted from the entire region G of the moving picture, for example, only one local region (x 3  only) consisting of 8×8 pixel block or five, for example, local regions (x 1 -x 5 ) may be extracted. In the illustrated embodiment, the shape of the pixel block is square. However, the shape of the pixel block is not limitative to square. The shape of N-number pixel blocks may be any given shape such as circle or polygonal. Further, the locations of the five local regions in the illustration may be moved or changed.  
      With reference to  FIG. 7 , extraction of the color space signal is explained. Color space signals (XYZ, RGB, YCbCr, CIE L*a*b*, CIE L*u*v*, CIE UCS, etc.) are extracted for the moving pictures (pixel blocks) at the local regions in  FIG. 6 . All of the XYZ, RGB, YCbCr, CIE L*a*b*, CIE L*u*v*, CIE UCS, etc. or any one component therein may be extracted selectively. Further, the color space signal may be extracted after, for example, YCbCr color space has been once converted to RGB color space signal.  
       FIG. 8  is a diagram showing how the feature amount table is created. The feature amounts such as the signal, histogram, correlation value and their grouped processed value calculated from the input moving picture information are subjected to a grouping process using a certain threshold value. The feature amount table is created from the time duration (number of frames) of the groups, the total number of groups, the threshold values and so on.  
       FIG. 9  is a diagram in which the feature amount table of  FIG. 8  is grouped using a plurality of threshold values. Grouping into two or more groups using two or more threshold values (Th 1 , Th 2 , and Th 3  in  FIG. 9 ) is performed, and the feature amount table is created from the time duration (number of frames) of the groups, the total number of groups and the threshold values and so on.  
      Collation of the feature amount information of the input moving picture for search with the feature amount information corresponding thereto extracted from the database for comparison is performed by the following determination formula (1) or (2). 
 
|Input moving picture  H  group length−Database  H  group length|≦ Th   (1) 
 
 Th   1 ≦(Input moving picture  H  group length−Database  H  group length)≦ Ths   2   (2) 
 
 With this determination formula, a search result list is output and, based on the list, the desired moving picture is extracted. 
 
      Next, the procedures of the embodiment will be explained with reference to  FIGS. 10-12 .  FIG. 10  is a block diagram showing the search process according to the invention,  FIG. 11  is a flow chart showing the process steps at the input side and the process steps of the matching process, and  FIG. 12  is a flow chart showing the process steps at the database side dealing with the feature amount information for the comparison.  
      At a step  101 , the moving picture information for search is input. At a step  102 , it is determined whether a part of the moving picture frames of the moving picture information for search is extracted as a local region. In the case where the extraction of the local region is skipped (non-extraction), the procedure goes to a step  104 . If the local region extraction is not skipped (extraction), the procedure goes to a step  103 .  
      In the local region extraction at the step  103 , for the calculation of the feature amount with respect to the input moving picture information for search, in order to extract a part of the moving picture frames as the local region, one or more square or any given shape local regions are extracted.  
      At the step  104 , the brightness signal, the color difference signal, the RGB color space components of the local region extracted at the step  103  or the entirety of the input moving picture information are extracted. At a step  105 , it is determined whether the histogram of the signal extracted from the input moving picture information is calculated or not. If the calculation of the histogram is skipped (non-calculation), the procedure goes to a step  107 . If the calculation is not skipped (calculation), the procedure goes to a step  106 .  
      At the step  106 , the histogram of the signal extracted at the step  104  is calculated. At the step  107 , it is determined whether the correlation value of the signal extracted at the step  104  or the histogram calculated at the step  106  is calculated. If the calculation of the correlation value is skipped (non-calculation), the procedure goes to a step  109 . If such calculation is not skipped (calculation), the procedure goes to a step  108 . The correlation value is calculated between the frames positioned side by side or between between any given frames along the time lapse.  
      At a step  109 , it is determined whether the grouping process is performed on the correlation values obtained at the step  109  or the histogram at the step  106  or not. If the grouping process is skipped (non-grouping), the procedure goes to a step  111 . If such grouping is done (grouping), the procedure goes to a step  110 .  
      At the step  110 , the grouping as the unit of one or more frames in terms of time is performed on the correlation values obtained at the step  108  using a certain threshold value and, then, the feature amount such as the number of frames and the number of groups is calculated.  
      At the step  111 , a feature amount table used for the search process is created from the feature amount calculated based on the input moving picture information for search. At a step  112 , a feature amount table for database which table is for search process is read out. At a step  113 , the feature amount used for the matching process is extracted from the feature amount table stored in the database based on the feature amount table for search process created at the step  111 .  
      At a step  114 , the matching process is performed between the feature amount of the moving picture for search and the feature amount of the database moving picture. If both the feature amounts match, the procedure goes to a step  115 . If not, the procedure goes to a step  116 . At the step  115 , the matching result is stored. At the step  116 , it is determined whether the matching process is stopped or not. If the matching process is to be stopped, the procedure goes to a step  117 . If not, the procedure returns to the step  113 .  
      At the step  117 , upon receipt of the matching result, the search result moving picture corresponding to the matching result is read out from the moving picture database. At a step  118 , the search result moving picture read out by the read out process unit is output. In  FIG. 11 , there is no case where all the four steps  102 ,  105 ,  107  and  109  are performed.  
      Next, with reference to  FIG. 12 , the procedures at the database side will be explained. At a step  201 , the moving picture information to be stored in the database is input. At a step  202 , the input moving picture information is stored in the moving picture database. At a step  203 , it is decided whether the extraction of a part of the frames of the moving pictures as local regions is effected on the moving picture information to be stored in the database or not. If such extraction of the local region is skipped (non-extraction), the procedure goes to a step  205 . If not skipped (extraction), the procedure goes to a step  204 .  
      In the local region extraction performed in the step  204 , in order to calculate the feature amount with respect to the moving picture information for database and to extract portions of the frames of the moving pictures as the local regions, one or more square or any given shape local regions are extracted.  
      At the step  205 , the brightness signals, the color difference signals, or RGB color space components and so on are extracted from the local region extracted at the step  204  or the entirety of the moving picture information for database. At a step  206 , it is determined whether the histogram of the signals extracted from the moving picture information for database is calculated or not. If the calculation of the histogram is skipped (non-calculation), the procedure goes to a step  208 . If not (calculation), the procedure goes to a step  207 .  
      At the step  207 , the histogram of the signal extracted at the step  205  is calculated. At the step  208 , it is determined whether the correlation value of the signal extracted at the step  205  or the histogram calculated at the step  207  is calculated or not. If the calculation of the correlation value is skipped (non-calculation), the procedure goes to a step  210 , whereas, if not (calculation), the procedure goes to a step  209 . At the step  209 , the correlation value between the frames positioned side by side or between between any given frames along the lapse of time is calculated.  
      At the step  210 , it is decided as to whether the grouping process on the correlation values calculated at the step  209  or the histograms calculated at the step  207  is performed or not. If the grouping process is skipped (non-grouping), the procedure goes to a step  212 . If not skipped (grouped), the procedure goes to a step  211 .  
      At the step  211 , the grouping as the unit of one or more frames in terms of time is performed on the correlation values obtained at the step  209  using a certain threshold value and, then, the feature amount such as the number of frames and the number of groups is calculated.  
      At the step  212 , a feature amount table to be used for the search process is created from the feature amount calculated based on the database registration moving picture information. At a step  213 , the feature amount table for the database to be used for the search process is registered in the feature amount table database. It is to be noted that, in  FIG. 12 , there is no case where the four skip steps  203 ,  206 ,  208  and  210  are all performed.  
      Next, by referring to  FIGS. 13-15 , another embodiment of the present invention wherein the local region extraction process and the signal extraction process are changed in their order as compared to the previous embodiment will be explained.  
       FIG. 13  is a block diagram showing the search processes for the embodiment in which the local region extraction process and the signal extraction process are changed in their order.  FIG. 14  is a flow chart showing the processes at the input side of the information for search and the processes of the matching.  FIG. 15  is a flow chart showing the processes at the database side dealing with the feature amount information for comparison. With reference to  FIG. 14 , the processes at the input side are explained. At a step  301 , the moving picture information for search is input. At a step  302 , the brightness signals, the color difference signals, and/or the RGB color space components and so on with respect to the input moving picture information for search are extracted. At a step  303 , it is determined whether the extraction of a part of the frame of the input moving picture for search as the local region is performed or not. If the local region extraction is skipped (non-extraction), the procedure goes to a step  305 . If not skipped (extraction), the procedure goes to a step  304 .  
      In the local region extraction at the step  304 , for calculating the feature amount with respect to the input moving picture information for search, in order to extract parts of the moving picture frames as the local regions, one or more square or any given shape local regions are extracted.  
      At the step  305 , it is determined whether the calculation of histogram of the extracted signals such as the brightness signal, the color difference signal, and the RGB color space components of the local region extracted at the step  304  or of the entirety of the moving picture information of search is conducted or not. If such calculation of the histogram is skipped (non-calculation), the procedure goes to a step  307 . If not skipped (calculation), the procedure goes to a step  306 .  
      At the step  306 , the histogram of the signal extracted at the step  304  is calculated. At the step  307 , it is determined whether the correlation value of the signal extracted at the step  304  or the histogram calculated at the step  306  is calculated or not. The correlation value is calculated between the frames positioned side by side or between any given frames along the lapse of time. If the calculation of the correlation value is skipped (non-calculation), the procedure goes to a step  309 . If not skipped (calculation), the procedure goes to a step  308 .  
      At the step  309 , it is determined whether the grouping of the correlation values obtained at the step  308  is performed or not. If such grouping is skipped (non-grouping), the procedure goes to a step  311 . If not skipped (grouping), the procedure goes to a step  310 .  
      At the step  310 , the grouping as the unit of one or more frames in terms of time is performed on the correlation values obtained at the step  308  using a certain threshold value and, then, the feature amount such as the number of frames and the number of groups is obtained.  
      At the step  311 , a feature amount table to be used for search process is created from the feature amount calculated based on the input moving picture information for search. At a step  312 , a feature amount table for database which table is used for search process is read out. At a step  313 , the feature amount used for the matching process is extracted from the feature amount table stored in the database based on the feature amount table for search process created at the step  311 .  
      At a step  314 , the matching between the feature amount of the moving picture for search and the feature amount of the moving picture in the database. If both the feature amounts match with each other, the procedure goes to a step  315 . If not, the procedure goes to a step  316 . At the step  315 , the matching result is stored. At the step  316 , it is determined whether the matching process is stopped or not. If yes, the procedure goes to a step  317 , whereas, if not, the procedure goes to a step  313 .  
      At the step  317 , upon receipt of the matching result, the search result moving picture corresponding to the matching result is read out from the moving picture database. At a step  318 , the search result moving picture read out by the read out unit is output. It should be noted that there is no case where the four skip steps  303 ,  305 ,  307  and  309  in  FIG. 14  are all performed.  
      Next, with reference to  FIG. 15 , the procedures at the database side will be explained.  
      At a step  401 , the moving picture information to be registered in the database is input. At a step  402 , the input database registration moving picture information is registered in the moving picture database. At a step  403 , the brightness signals, the color difference signals, the RGB color space components, etc. are extracted from the input database registration moving picture information.  
      At a step  404 , as to the database registration moving picture information, it is determined whether the extraction of a part of the moving frame as a local region is performed or not. If such extraction of the local region is skipped (non-extraction), the procedure goes to a step  406 . If not skipped (extraction), the procedure goes to a step  405 .  
      In the local region extraction performed in the step  405 , in order to calculate the feature amount with respect to the database registration moving picture information and to extract portions of the frames of the moving pictures as the local regions, one or more square or any given shape local regions are extracted.  
      At the step  406 , it is determined whether the histograms such as the brightness signals, the color difference signals, the RGB color space components, etc. of the local region extracted at the step  405  or of the entirety of the database registration moving picture information are calculated or not. If the calculation of the histograms is skipped (non-calculation), the procedure goes to a step  408 . If not skipped (calculation), the procedure goes to a step  407 .  
      At the step  407 , the histogram of the signal extracted at the step  405  is calculated. At the step  408 , it is decided as to whether the correlation value of the signal extracted at the step  405  or the histogram calculated at the step  407  is calculated or not. If the calculation of the histogram is skipped (non-calculation), the procedure goes to a step  410 . If not skipped (calculation), the procedure goes to a step  411 . The correlation is calculated between the frames positioned side by side or between any given frames along the lapse of time.  
      At the step  410 , it is determined as to whether the grouping with respect to the correlation values obtained at the step  409  is effected or not. If such grouping is skipped (non-grouping), the procedure goes to a step  412 . If not skipped (grouping), the procedure goes to a step  411 .  
      At the step  411 , the grouping as the unit of one or more frames in terms of time is performed on the correlation values obtained at the step  409  using a certain threshold value and, then, the feature amount such as the number of frames and the number of groups is obtained.  
      At the step  412 , a feature amount table used for the search process is created based on the feature amounts calculated based on the database registration moving picture information. At a step  413 , the feature amount table for database used for the search process is registered in the feature amount table database. It should be noted that there is no case where the four skip steps  404 ,  406 ,  408  and  410  in  FIG. 15  are all performed.  
      As has been described hereinabove, a method for searching moving pictures and a system using the same according to the present invention comprises: 
          for searching,     a moving picture input unit for inputting the moving picture data for search;     a feature amount calculation unit including: 
            a local region extracting unit for extracting one or more square or any given shape local regions for calculating the feature amount with respect to the input moving picture data for search;     a signal extraction unit for extracting or conversion-extracting picture signal components such as brightness signals, color difference signals, each RGB color space component, each XYZ color space component, each uniform color space component, and each Munsell color space component of the feature amount calculation region extracted from the input moving picture for search; and     a feature amount table creation unit for creating a feature amount table used for the searching process, from the feature amount calculated based on the input moving picture for search;    
            for creating a database,     a moving picture input unit for inputting the moving picture data for database;     a moving picture database registration unit for registering the input moving picture data for database in a moving picture database;     a database in which the input moving picture data for database is registered;     a feature amount calculation unit including: 
            a local region extracting unit for extracting one or more square or any given shape local regions for calculating the feature amount with respect to the input moving picture data for database;     a signal extraction unit for extracting or conversion-extracting picture signal components such as brightness signals, color difference signals, each RGB color space component, each XYZ color space component, each uniform color space component, and each Munsell color space component of the feature amount calculation region extracted from the input moving picture for database; and     a feature amount table creation unit for creating a feature amount table used for the searching process, from the feature amount calculated based on the input moving picture for database;    
            a table database registration unit for registering the created feature amount table for database in a feature amount table database;     a table database in which the created feature amount table for database is registered;     a search unit including: 
            a table database read out unit for reading out the database feature amount table used for search process, which table was processed by the feature amount calculation unit on the input moving picture for search and is used for the search process;     a matching unit for extracting a feature amount used for matching from the feature amount table created based on the input moving picture for search and the registered feature amount table in the feature amount table for database, outputting the extracted result to a matching result judgment unit, storing the matching result from the matching result judgment unit, and after the completion of the matching process, outputting the matching result to a moving picture database read out unit; and     a matching result judgment unit for performing judgment with respect to a threshold value, based on the feature amount of the moving picture for search and the feature amount of the moving picture for database extracted by the matching unit, and outputting the matching result to the matching unit;    
            a moving picture database read out unit for receiving the matching result from the matching result judgment unit, and reading out a search result moving picture corresponding to the matching result, from the moving picture database; and     a search result output unit for outputting the search result moving picture read out by the moving picture database read out unit.        

      With this moving picture search method and system, a local region is extracted from the picture frame and a histogram of the brightness information is calculated from the extracted local region. Then, the calculated histogram is compared before and after the frame, and the correlation value thereof is obtained. It is possible to achieve a further high-speed moving picture matching by grouping the above correlation values with respect to a certain threshold value.  
      Further, as compared to the average brightness value and the dispersion value of each frame, the correlation value does not rely on the input data and is not necessary to be varied with respect to the threshold value input when it is grouped. As a consequent, it is possible to establish the threshold value appropriately by using the correlation value as the feature amount. Further, by using the grouped feature amount as the search parameter, it is possible to control or limit the number of parameters which are used for searching the moving picture scenes and, thus, the time required for the matching process can be effectively shortened.