Patent Publication Number: US-2017358084-A1

Title: Image analysis apparatus, image analysis system, and operation method of image analysis apparatus

Description:
CROSS REFERENCE TO RELATED APPLICATION 
     This application is a continuation application of PCT/JP2016/062486 filed on Apr. 20, 2016 and claims benefit of Japanese Application No. 2015-090620 filed in Japan on Apr. 27, 2015, the entire contents of which are incorporated herein by this reference. 
    
    
     BACKGROUND OF INVENTION 
     1. Field of the Invention 
     The present invention relates to an image analysis apparatus, an image analysis system, and an operation method of the image analysis apparatus configured to specify target elements from images of a subject to extract color components. 
     2. Description of the Related Art 
     Various image analysis apparatuses configured to specify regions in an image to analyze the image are conventionally proposed. 
     For example, an electronic endoscope system is described in Japanese Patent Application Laid-Open Publication No. 2012-152266, the electronic endoscope system including: an electronic endoscope configured to photograph inside of a subject; a change region detection section configured to detect, from image data photographed by the electronic endoscope, a change region in which a feature of an image is changed; a mask data generation section configured to generate mask data including parameters of image processing that are set for each pixel such that image processing is applied to the change region and another region in different modes based on the detected change region; and an image processing section configured to apply image processing to the image data based on the mask data. 
     An image analysis method is described in Japanese Patent Application Laid-Open Publication No. 2007-502185, the image analysis method including: picking up a digital image of dental tissue; determining a first component value of a color of a pixel and a second component value of a color of the pixel for each of a plurality of pixels in the digital image; and calculating a first function value (for example, R/G) of the pixel based on the first component value and the second component value. 
     SUMMARY OF THE INVENTION 
     An aspect of the present invention provides an image analysis apparatus including: an image input section to which images of a subject acquired over time are inputted; a region extraction section configured to specify a target element including an annular peripheral portion and a center portion that is surrounded by the peripheral portion and that is in a color different from the peripheral portion in each of a first image acquired at a first timing and a second image acquired at a second timing later than the first timing, the first image and the second image being inputted from the image input section, the region extraction section being further configured to extract only the center portion of the target element as a region to be analyzed; and a color component extraction section configured to extract respective color component values of the region to be analyzed of the first image and color component values of the region to be analyzed of the second image extracted by the region extraction section. 
     An aspect of the present invention provides an image analysis system including: an endoscope inserted into a subject and configured to pick up and acquire images of the subject; and the image analysis apparatus, wherein the images acquired by the endoscope are inputted to the image input section. 
     An aspect of the present invention provides an operation method of an image analysis apparatus, the operation method including: inputting images of a subject acquired over time to an image input section; a region extraction section specifying a target element including an annular peripheral portion and a center portion that is surrounded by the peripheral portion and that is in a color different from the peripheral portion in each of a first image acquired at a first timing and a second image acquired at a second timing later than the first timing, the first image and the second image being inputted from the image input section, the region extraction section extracting only the center portion of the target element as a region to be analyzed; and a color component extraction section extracting respective color component values of the region to be analyzed of the first image and color component values of the region to be analyzed of the second image extracted by the region extraction section. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram showing a configuration of an image analysis system according to a first embodiment of the present invention; 
         FIG. 2  is a block diagram showing a configuration of a region extraction section according to the first embodiment; 
         FIG. 3  is a flowchart showing a process using the image analysis system of the first embodiment; 
         FIG. 4  is a flowchart showing an image analysis process by an image analysis apparatus of the first embodiment; 
         FIG. 5A  is a flowchart showing a process of selecting center portions of a plurality of target elements in a selected region in the image analysis apparatus of the first embodiment; 
         FIG. 5B  is a flowchart showing a modification of the process of selecting center portions of a plurality of target elements in a selected region in the image analysis apparatus of the first embodiment; 
         FIG. 6  is a flowchart of a double closed curve edge specification process in the image analysis apparatus of the first embodiment; 
         FIG. 7  is a flowchart showing a single closed curve edge specification process in the image analysis apparatus of the first embodiment; 
         FIG. 8  is a diagram showing an example of display of images of a subject sorted in chronological order in the first embodiment; 
         FIG. 9  is a diagram showing a brightness distribution of an image of the subject and an enlarged diagram of one of the target elements in the first embodiment; 
         FIG. 10  is a diagram showing a structure of intestinal cilia that are the target elements in the first embodiment; 
         FIG. 11  is a diagram showing an example of regions to be analyzed set in the image of the subject in the first embodiment; 
         FIG. 12  is a diagram showing an example of a simulation result of brightness of an endoscope in the first embodiment; and 
         FIG. 13  is a diagram showing an example of a region suitable for extracting color component values obtained from the simulation result of the brightness of the endoscope in the first embodiment. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S) 
     Hereinafter, an embodiment of the present invention will be described with reference to the drawings. 
     First Embodiment 
       FIGS. 1 to 11  show a first embodiment of the present invention, and  FIG. 1  is a block diagram showing a configuration of an image analysis system. 
     The image analysis system includes an endoscope  20  and an image analysis apparatus  10 . 
     The endoscope  20  is inserted into a subject to pick up and acquire an image of a subject. In the present embodiment, the endoscope  20  is capable of, for example, narrow band light observation (NBI: narrow band imaging). Here, to reduce noise components to perform NBI magnified observation, a distal end hood or a distal end attachment is mounted on a distal end of the endoscope  20 , for example. In the present embodiment, to apply a load to a subject to observe a change in the subject before and after the load, the endoscope  20  acquires images of the subject over time. To more accurately perceive the change in the subject before and after the application of the load to the subject, it is desirable that setting of brightness of the endoscope  20  is in a same state. Therefore, light adjustment of a light source is not performed before and after the application of the load to the subject, and the images of the subject can be acquired with a constant amount of emitted light from the light source. 
     The image analysis apparatus  10  includes an image input section  11 , a region extraction section  12 , a color component extraction section  13 , and an image analysis section  14 . 
     The images of the subject acquired by the endoscope  20  over time are inputted to the image input section  11 . 
     The region extraction section  12  specifies target elements, each including an annular peripheral portion and a center portion that is surrounded by the peripheral portion and that is in a color different from the peripheral portion (the target element in the present embodiment is, for example, an image part of intestinal villi that is a feature region as described later), from a first image and a second image inputted from the image input section  11 , the first image acquired at a first timing and the second image acquired at a second timing later than the first timing. The region extraction section  12  extracts only the center portions of the target elements as regions to be analyzed. 
     The color component extraction section  13  extracts color component values of the regions to be analyzed of the first image and color component values of the regions to be analyzed of the second image extracted by the region extraction section  12 . 
     The image analysis section  14  calculates a degree of change between the color component values of the first image and the color component values of the second image extracted from the regions to be analyzed. 
     Next,  FIG. 2  is a block diagram showing a configuration of the region extraction section  12 . 
     The region extraction section  12  is configured to judge a difference between the colors of the peripheral portion and the center portion based on a difference in at least one of hue, saturation, and luminance. Therefore, a difference in color component value indicates a difference in color. For example, when the hue and the saturation are the same and only the luminance is different, the color is different. 
     As shown in  FIG. 2 , the region extraction section  12  includes an edge detection section  21 , a closed curve edge detection section  22 , a size filter processing section  23 , a double closed curve edge detection section  24 , a double closed curve edge specification section  25 , a single closed curve edge specification section  26 , and a region extraction control section  27 . 
     The edge detection section  21  applies, for example, an edge detection filter to the images to detect edges. 
     The closed curve edge detection section  22  further detects edges forming closed curves from the edges detected by the edge detection section  21 . 
     The size filter processing section  23  selects only closed curve edges in which the size is in a possible range of the target elements (for example, possible range of the size of intestinal villi) among the closed curve edges detected by the closed curve edge detection section  22 . 
     The double closed curve edge detection section  24  further detects double closed curve edges with double edges (that is, including an outer closed curve edge and an inner closed curve edge included in the outer closed curve edge) among the closed curve edges detected by the closed curve edge detection section  22  and further selected by, for example, the size filter processing section  23 . 
     The double closed curve edge specification section  25  specifies a region in the inner closed curve edge as a center portion when the color of the region in the inner closed curve edge in the double closed curve edge detected by the double closed curve edge detection section  24  and the color of a region between the inner closed curve edge and the outer closed curve edge are different. 
     In this case, the double closed curve edge specification section  25  is configured to further specify the region in the inner closed curve edge as a center portion when the color of the region in the inner closed curve edge is in a first color range corresponding to the center portion of the target element (for example, the first color range is a color range close to red when the target element is intestinal villi) and the color of the region between the inner closed curve edge and the outer closed curve edge is in the second color range corresponding to the peripheral portion of the target element (second color range different from the first color range) (for example, the second color range is a color range close to white when the target element is intestinal villi). 
     Note that the difference in color is judged based on the difference in at least one of the hue, the saturation, and the luminance Therefore, the color range is a range of one of the hue, the saturation, and the luminance or a range determined by a combination of two or more of the hue, the saturation, and the luminance. For example, the color range may be a range determined by a combination of the hue and the saturation, or the color range may be a luminance range (that is, the center portion and the peripheral portion may be distinguished based only on the luminance). When the target element is intestinal villi and the color range is the luminance range, the first color range can be a range with a lower luminance, and the second color range can be a range with a luminance higher than in the first color range, for example. 
     It is more preferable that the double closed curve edge specification section  25  specifies the region in the inner closed curve edge as a center portion only when the size filter processing section  23  judges that the sizes of the inner closed curve edge and the outer closed curve edge are in a possible range of the target element. 
     In the present embodiment, when the number of center portions of the target elements specified by the double closed curve edge specification section  25  is less than a predetermined number (here, the predetermined number is two or more), the single closed curve edge specification section  26  is further used to specify the center portions of the target elements (however, only the single closed curve edge specification section  26  may be used to specify the center portions of the target elements without using the double closed curve edge specification section  25 ). 
     The single closed curve edge specification section  26  specifies inside of the region surrounded by the closed curve edge as a center portion when the colors inside and outside of the region surrounded by the closed curve edge detected by the closed curve edge detection section  22  are different. 
     Note that although the single closed curve edge specification section  26  processes the closed curve edges not subjected to processing by the double closed curve edge specification section  25  in the present embodiment, all of triple, quadruple, . . . closed curve edges with more edges than the double closed curve edges are also the double closed curve edges. Therefore, the single closed curve edge specification section  26  processes single closed curve edges. 
     The single closed curve edge specification section  26  is configured to further specify a region in the single closed curve edge as a center portion when the color of the region in the single closed curve edge is in the first color range corresponding to the center portion of the target element, and the color of a region near the outside of the single closed curve edge is in the second color range corresponding to the peripheral portion of the target element. 
     It is more preferable that the single closed curve edge specification section  26  specifies the inside of the region surrounded by the single closed curve edge as a center portion only when the size filter processing section  23  judges that the size of the single closed curve edge is in the possible range of the target element. 
     The region extraction control section  27  controls respective sections in the region extraction section  12 , that is, the edge detection section  21 , the closed curve edge detection section  22 , the size filter processing section  23 , the double closed curve edge detection section  24 , the double closed curve edge specification section  25 , the single closed curve edge specification section  26 , and the like to cause the sections to perform operation as described later with reference to  FIGS. 5A to 7 . 
     Next,  FIG. 3  is a flowchart showing a process using the image analysis system. 
     When the process is started, the endoscope  20  picks up and acquires an image before the load is applied to the subject (before-load image, first image) at the first timing (step S 1 ). Here, the subject in the present embodiment is, for example, intestinal (more specifically, small intestine) villi (however, the subject is not limited to this, and some other examples include tongue, esophagus, gastric mucosa, and large intestine). At the same time as the acquisition of the image of the subject by the endoscope  20 , information of the amount of emitted light at the acquisition of the image may be recorded in, for example, the image analysis apparatus  10  or the endoscope  20 . 
     Subsequently, the load is applied to the subject (step S 2 ). Here, glucose is sprayed as the load, for example (however, the method is not limited to this, and the glucose may be intravenously injected, or other loads may be applied). When the glucose is sprayed, an amount of blood flowing through capillaries increases, and hemoglobin in the blood absorbs more light. Therefore, a part with concentrated capillaries in the villi is observed as a dark part. 
     Subsequently, the endoscope  20  picks up and acquires an image after the load is applied at a second timing later than the first timing (after-load image, second image) (step S 3 ). When the endoscope  20  acquires the image after the load is applied to the subject, the image is acquired under the same condition as in step S 1  with reference to the information of the amount of emitted light if the information of the amount of emitted light is recorded in step S 1 . Note that a function of deleting the information of the amount of emitted light recorded in step S 1  later may be included. The acquisition of the information of the amount of emitted light, the acquisition of the image using the information of the amount of emitted light, and the deletion of the information of the amount of emitted light may be realized by operation of, for example, an operation portion of the endoscope  20 , a switch provided on a control panel for controlling the image analysis system, or a foot switch for operating the endoscope  20 . 
     Whether to further acquire a next image is judged (step S 4 ). If it is judged to acquire the next image, the process returns to step S 3  to acquire a next after-load image. 
     If it is judged that the acquisition of the image is finished in step S 4 , the image analysis apparatus  10  performs image analysis (step S 5 ). The process ends when the image analysis is completed. 
       FIG. 4  is a flowchart showing an image analysis process by the image analysis apparatus  10 . 
     When the process is started, the image input section  11  inputs the images of the subject acquired over time from the endoscope  20  and sorts the images in chronological order (step S 10 ). 
       FIG. 8  is a diagram showing an example of display of the images of the subject sorted in chronological order. 
     In the example of display shown in  FIG. 8 , an image arrangement display  31 , an image acquisition time period display  32 , and an image arrangement order display  33  are provided on a display apparatus such as a monitor. 
     In the image arrangement display  31 , acquired images P 0  to P 8  of the subject are arranged and displayed in order of time period of acquisition. 
     In the image acquisition time period display  32 , time period points of the acquisition of the images P 1  to P 8  after the application of the load (spray of glucose) are disposed and shown along with, for example, acquisition time periods on a time axis. Note that although the image P 0  is an image acquired before the spray of glucose (for example, just before the spray of glucose), the image P 0  is displayed at a position of the spray of glucose for the convenience in the example illustrated in  FIG. 8  (however, it is obvious that the time axis may be extended to a time point before the spray of glucose to accurately indicate the time point of the acquisition of the image P 0 ). 
     Furthermore, the image arrangement order display  33  displays the respective images displayed in the image arrangement display  31  in association with the time points of the acquisition of the images P 0  to P 8  displayed in the image acquisition time period display  32 . 
     Next, the image analysis apparatus  10  judges whether there is an image not yet subjected to a process described later with reference to steps S 12  to S 19  (step S 11 ). 
     If it is judged that there is an unprocessed image, the region extraction section  12  inputs image data to be processed from the image input section  11  (step S 12 ). 
     Regions with inappropriate elements (inappropriate regions) IR (see  FIGS. 9, 11 , and the like), such as halation, not suitable for the extraction of color component values are excluded from the processing target (step S 13 ). Other than the regions with halation, examples of the inappropriate regions IR include regions with bubbles and regions out of focus. 
     Furthermore, a region in which an average luminance calculated for each partial region in a predetermined size in the image is equal to or greater than a predetermined value is selected as an appropriate luminance region (step S 14 ). For example, the average luminance of a region in an upper right half is lower than the predetermined value in an image Pi (here, i is one of 0 to 8 in the example shown in  FIG. 8  (that is, Pi is one of P 0  to P 8 )) as shown in  FIG. 9  (or  FIG. 11 ). Here,  FIG. 9  is a diagram showing a brightness distribution of the image of the subject and an enlarged diagram of one of the target elements. 
     Note that although the region to be analyzed is set by using the image of the subject acquired by the endoscope  20  or the like as an image indicating the performance of the image pickup apparatus configured to acquire the image inputted from the image input section  11  in the description above, the method is not limited to this. A method may also be adopted, wherein a region AR (see  FIG. 13 ) suitable for extracting the color component values from the average luminance calculated for each partial region in the predetermined size is set as the region to be analyzed based on another image indicating the performance of the image pickup apparatus (for example, an image obtained by photographing a flat object with uniform color, such as a test plate and a white balance cap, or an image serving as an index indicating the performance, such as a simulation result SI (see  FIG. 12 ) of brightness obtained from a design value of the endoscope  20 ). Furthermore, a method may also be adopted, wherein the region to be analyzed is set from the region AR suitable for extracting the color component values based on the average luminance calculated for each partial region in the predetermined size. Here,  FIG. 12  is a diagram showing an example of the simulation result SI of the brightness of the endoscope  20 , and  FIG. 13  is a diagram showing an example of the region AR suitable for extracting the color component values obtained from the simulation result SI of the brightness of the endoscope  20 . 
     Therefore, the region extraction section  12  selects, as an appropriate luminance region, a region in a lower left half of the image Pi in which the average luminance is equal to or greater than the predetermined value. As a result of the selection, a bright region suitable for extracting the color component values is selected, and a dark region not suitable for extracting the color component values is excluded. 
     Note that although the appropriate luminance range suitable for extracting the color component values is a range in which the average luminance is equal to or greater than the predetermined value here, a region that is too bright in which the average luminance is close to a saturated pixel value may also be excluded. In this case, the appropriate luminance range suitable for extracting the color component values can be a range in which the average luminance is equal to or greater than a predetermined lower limit threshold and equal to or smaller than a predetermined upper limit threshold. 
     Assuming that the tone of the luminance of the image has, for example, 256 levels of 0 to 255, the lower limit threshold of the appropriate luminance range can be set to, for example, 10 equivalent to a frame part of an endoscopic image, and the upper limit threshold can be set to, for example, 230 equivalent to halation. In this way, the color component of only the object to be analyzed can be extracted, and the accuracy of analysis can be improved. 
     Subsequently, center portions OBJc (the center portions OBJc are also elements) of a plurality of target elements (image parts of intestinal villi in the present embodiment) OBJ are selected in the selected region (step S 15 ). 
     As described later with reference to  FIGS. 5A to 7 , image analysis or the like is performed to execute an automatic process to extract and select a plurality of image parts of the intestinal villi that are the target elements OBJ (however, an option for a user to view and manually select the images may be further prepared). 
     Here, the image part of the intestinal villi that are the target element OBJ is an element including an annular (not limited to a ring shape, and an arbitrary closed curve shape is possible) peripheral portion OBJp and the center portion OBJc that is surrounded by the peripheral portion OBJp and that is in a color different from the peripheral portion OBJp. 
       FIG. 10  is a diagram showing a structure of the intestinal cilia that are the target elements. 
     In the intestinal villi, capillaries BC are distributed in a part around a center lymphatic vessel CL at a center portion, and mucosal epithelium ME is formed outside of the capillaries BC to configure the surface of the villi. 
     When the intestinal villi are magnified and observed by the NBI using light with a narrow-band wavelength that is easily absorbed by hemoglobin in the blood, the part of the capillaries BC is observed in a color different from the mucosal epithelium ME. 
     When the image part obtained by imaging the villi from above is observed, the image part of the mucosal epithelium ME is observed as the annular peripheral portion OBJp, and the image part of the capillaries BC surrounded by the mucosal epithelium ME is observed as the center portion OBJc with a color different from the mucosal epithelium ME. Therefore, as described later, the difference between the colors of the center portion OBJc and the peripheral portion OBJp is used to determine the target element OBJ. 
     A predetermined number of (five in an example shown in  FIG. 11 ) center portions OBJc with the brightness close to a median are further selected from a plurality of center portions OBJc selected in this way, and the predetermined number of selected center portions OBJc are set as regions to be analyzed OR (step S 16 ). Here,  FIG. 11  is a diagram showing an example of the regions to be analyzed OR set in the image Pi of the subject. 
     Here, the reason that the center portions OBJc with the brightness close to the median are selected is to analyze portions with brightness most appropriate as samples. Note that a luminance value calculated based on a plurality of color components may be used as the brightness, or a value obtained by simply adding a plurality of color components may be used as an index of the brightness. Other methods may be used to acquire the brightness based on a plurality of color components. In this way, the regions to be analyzed OR set here and shown in  FIG. 11  include, for example, five center portions OBJc of the image parts of the intestinal villi. 
     Next, the color component extraction section  13  extracts color component values, such as an R component value, a G component value, and a B component value, of each pixel included in the regions to be analyzed OR (step S 17 ) and further calculates an average value &lt;R&gt; of the R component values, an average value &lt;G&gt; of the G component values, and an average value &lt;B&gt; of the B component values of the regions to be analyzed OR in the first image (before-load image) and an average value &lt;R′&gt; of the R components values, an average value &lt;G′&gt; of the G component values, and an average value &lt;B′&gt; of the B component values of the regions to be analyzed OR in the second image (after-load image) (step S 18 ). 
     The image analysis section  14  then calculates an amount of change in color component average values as a degree of change from the before-load image to the after-load image as follows, for example (step S 19 ). 
     That is, the image analysis section  14  calculates the amount of change as a sum of absolute values of difference values of the color component values between the first image and the second image as shown in the following Equation 1. 
       Amount of change=|&lt; R′&gt;−&lt;R&gt;|+|&lt;G′&gt;−&lt;G&gt;|+|&lt;B′&gt;−&lt;B&gt;|   [Equation 1]
 
     Therefore, the calculated amount of change is a sum of an average value of the color component values in which the values are lower in the second image than in the first image and an average value of the color component values in which the values are higher in the second image than in the first image. 
     Note that the degree of change from the before-load image to the after-load image calculated by the image analysis section  14  is not limited to the calculation shown in Equation 1. 
     First, in a first modification, the amount of change as the degree of change is calculated as shown in the following Equation 2, wherein Min (x, y) represents a function for outputting not larger one of x and y (smaller one when x≠y). 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                      
                     
                       
                         
                           
                             
                               Min 
                                
                               
                                 ( 
                                 
                                   
                                     
                                       〈 
                                       
                                         R 
                                         ′ 
                                       
                                       〉 
                                     
                                     - 
                                     
                                       〈 
                                       R 
                                       〉 
                                     
                                   
                                   , 
                                   0 
                                 
                                 ) 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             
                               Min 
                                
                               
                                 ( 
                                 
                                   
                                     
                                       〈 
                                       
                                         G 
                                         ′ 
                                       
                                       〉 
                                     
                                     - 
                                     
                                       〈 
                                       G 
                                       〉 
                                     
                                   
                                   , 
                                   0 
                                 
                                 ) 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             Min 
                              
                             
                               ( 
                               
                                 
                                   
                                     〈 
                                     
                                       B 
                                       ′ 
                                     
                                     〉 
                                   
                                   - 
                                   
                                     〈 
                                     B 
                                     〉 
                                   
                                 
                                 , 
                                 0 
                               
                               ) 
                             
                           
                         
                       
                     
                      
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     2 
                   
                   ] 
                 
               
             
           
         
       
     
     Therefore, the calculated amount of change is a sum of only the average values of the color component values in which the values are smaller in the second image than in the first image. The calculation method is used in consideration of a characteristic of human eyes. That is, the human eyes more sharply feel a change when the brightness of image changes from bright to dark than when the brightness of image changes from dark to bright. Therefore, the characteristic of human eyes is taken into account such that a change in the image visually perceived by the user coincides with an analysis result of a change in the image obtained by the image analysis. 
     Next, in a second modification, the amount of change as the degree of change is calculated as shown in the following Equation 3. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                      
                     
                       
                         
                           
                             
                               Min 
                                
                               
                                 ( 
                                 
                                   
                                     
                                       〈 
                                       R 
                                       〉 
                                     
                                     - 
                                     
                                       〈 
                                       
                                         R 
                                         ′ 
                                       
                                       〉 
                                     
                                   
                                   , 
                                   0 
                                 
                                 ) 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             
                               Min 
                                
                               
                                 ( 
                                 
                                   
                                     
                                       〈 
                                       G 
                                       〉 
                                     
                                     - 
                                     
                                       〈 
                                       
                                         G 
                                         ′ 
                                       
                                       〉 
                                     
                                   
                                   , 
                                   0 
                                 
                                 ) 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             Min 
                              
                             
                               ( 
                               
                                 
                                   
                                     〈 
                                     B 
                                     〉 
                                   
                                   - 
                                   
                                     〈 
                                     
                                       B 
                                       ′ 
                                     
                                     〉 
                                   
                                 
                                 , 
                                 0 
                               
                               ) 
                             
                           
                         
                       
                     
                      
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     3 
                   
                   ] 
                 
               
             
           
         
       
     
     Therefore, the calculated amount of change is a sum of only the average values of the color component values in which the values are higher in the second image than in the first image. The reason that the calculation method is used is that a change in the brightness of image from dark to bright is an important analysis result in some cases. 
     Furthermore, in a third modification, respective color components illustrated on right sides of Equations 1 to 3 are multiplied by weighting factors α, β, and γ (here, α&gt;0, β&gt;0, and γ&gt;0) of respective color components. 
     For example, in accordance with Equation 1, the amount of change is calculated as shown in the following Equation 4. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                     
                        
                       
                         
                           α 
                           × 
                           
                             〈 
                             
                               R 
                               ′ 
                             
                             〉 
                           
                         
                         - 
                         
                           〈 
                           R 
                           〉 
                         
                       
                        
                     
                     + 
                     
                        
                       
                         
                           β 
                           × 
                           
                             〈 
                             
                               G 
                               ′ 
                             
                             〉 
                           
                         
                         - 
                         
                           〈 
                           G 
                           〉 
                         
                       
                        
                     
                     + 
                     
                        
                       
                         
                           γ 
                           × 
                           
                             〈 
                             
                               B 
                               ′ 
                             
                             〉 
                           
                         
                         - 
                         
                           〈 
                           B 
                           〉 
                         
                       
                        
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     4 
                   
                   ] 
                 
               
             
           
         
       
     
     Alternatively, in accordance with Equation 2, the amount of change is calculated as shown in the following Equation 5. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                      
                     
                       
                         
                           
                             
                               α 
                               × 
                               
                                 Min 
                                  
                                 
                                   ( 
                                   
                                     
                                       
                                         〈 
                                         
                                           R 
                                           ′ 
                                         
                                         〉 
                                       
                                       - 
                                       
                                         〈 
                                         R 
                                         〉 
                                       
                                     
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             
                               β 
                               × 
                               
                                 Min 
                                  
                                 
                                   ( 
                                   
                                     
                                       
                                         〈 
                                         
                                           G 
                                           ′ 
                                         
                                         〉 
                                       
                                       - 
                                       
                                         〈 
                                         G 
                                         〉 
                                       
                                     
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             γ 
                             × 
                             
                               Min 
                                
                               
                                 ( 
                                 
                                   
                                     
                                       〈 
                                       
                                         B 
                                         ′ 
                                       
                                       〉 
                                     
                                     - 
                                     
                                       〈 
                                       B 
                                       〉 
                                     
                                   
                                   , 
                                   0 
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                      
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     5 
                   
                   ] 
                 
               
             
           
         
       
     
     Alternatively, in accordance with Equation 3, the amount of change is calculated as shown in the following Equation 6. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                      
                     
                       
                         
                           
                             
                               α 
                               × 
                               
                                 Min 
                                  
                                 
                                   ( 
                                   
                                     
                                       
                                         〈 
                                         R 
                                         〉 
                                       
                                       - 
                                       
                                         〈 
                                         
                                           R 
                                           ′ 
                                         
                                         〉 
                                       
                                     
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             
                               β 
                               × 
                               
                                 Min 
                                  
                                 
                                   ( 
                                   
                                     
                                       
                                         〈 
                                         G 
                                         〉 
                                       
                                       - 
                                       
                                         〈 
                                         
                                           G 
                                           ′ 
                                         
                                         〉 
                                       
                                     
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                             + 
                           
                         
                       
                       
                         
                           
                             γ 
                             × 
                             
                               Min 
                                
                               
                                 ( 
                                 
                                   
                                     
                                       〈 
                                       B 
                                       〉 
                                     
                                     - 
                                     
                                       〈 
                                       
                                         B 
                                         ′ 
                                       
                                       〉 
                                     
                                   
                                   , 
                                   0 
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                      
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     6 
                   
                   ] 
                 
               
             
           
         
       
     
     In this case, the weighting factors α, β, and γ in Equations 4 to 6 can be adjusted to control how much each color component average value contributes to the amount of change. 
     In a fourth modification, a rate of change is calculated as the degree of change, in place of the amount of change. 
     That is, when image pickup conditions (such as exposure time period, aperture value, and illuminance of subject) of each image in a series of image groups (before-load images and after-load images) acquired over time are equal, amounts of change in the image groups, such as a first amount of change from the before-load image P 0  to the after-load image P 1  and a second amount of change from the before-load image P 0  to the after-load image P 2 , can be compared. 
     However, the brightness of image generally varies between a plurality of image groups picked up under different image pickup conditions, and the amounts of change cannot be compared as it is in some cases. For example, an amount of change in an image group acquired from a subject and an amount of change in an image group acquired from another subject are compared. If the brightness of one of the image group is twice the brightness of the other image group, the calculated amount of change of one of the image group is twice the calculated amount of change of the other image group even when pathological amounts of change are the same. 
     Therefore, the rate of change is calculated as the degree of change in the fourth modification to allow the comparison in such a case. 
     For example, in accordance with Equation 4, the amount of change is calculated as shown in the following Equation 7. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                     
                       { 
                       
                         
                           
                             
                               
                                  
                                 
                                   
                                     α 
                                     × 
                                     
                                       〈 
                                       
                                         R 
                                         ′ 
                                       
                                       〉 
                                     
                                   
                                   - 
                                   
                                     〈 
                                     R 
                                     〉 
                                   
                                 
                                  
                               
                               + 
                               
                                  
                                 
                                   
                                     β 
                                     × 
                                     
                                       〈 
                                       
                                         G 
                                         ′ 
                                       
                                       〉 
                                     
                                   
                                   - 
                                   
                                     〈 
                                     G 
                                     〉 
                                   
                                 
                                  
                               
                               + 
                             
                           
                         
                         
                           
                             
                                
                               
                                 
                                   γ 
                                   × 
                                   
                                     〈 
                                     
                                       B 
                                       ′ 
                                     
                                     〉 
                                   
                                 
                                 - 
                                 
                                   〈 
                                   B 
                                   〉 
                                 
                               
                                
                             
                           
                         
                       
                       } 
                     
                     / 
                     
                        
                       
                         
                           〈 
                           R 
                           〉 
                         
                         + 
                         
                           〈 
                           G 
                           〉 
                         
                         + 
                         
                           〈 
                           B 
                           〉 
                         
                       
                        
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     7 
                   
                   ] 
                 
               
             
           
         
       
     
     Alternatively, in accordance with Equation 5, the amount of change is calculated as shown in the following Equation 8. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                     
                        
                       
                         
                           
                             
                               
                                 α 
                                 × 
                                 
                                   Min 
                                    
                                   
                                     ( 
                                     
                                       
                                         
                                           〈 
                                           
                                             R 
                                             ′ 
                                           
                                           〉 
                                         
                                         - 
                                         
                                           〈 
                                           R 
                                           〉 
                                         
                                       
                                       , 
                                       0 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                               
                                 β 
                                 × 
                                 
                                   Min 
                                    
                                   
                                     ( 
                                     
                                       
                                         
                                           〈 
                                           
                                             G 
                                             ′ 
                                           
                                           〉 
                                         
                                         - 
                                         
                                           〈 
                                           G 
                                           〉 
                                         
                                       
                                       , 
                                       0 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                             
                           
                         
                         
                           
                             
                               γ 
                               × 
                               
                                 Min 
                                  
                                 
                                   ( 
                                   
                                     
                                       
                                         〈 
                                         
                                           B 
                                           ′ 
                                         
                                         〉 
                                       
                                       - 
                                       
                                         〈 
                                         B 
                                         〉 
                                       
                                     
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                           
                         
                       
                        
                     
                     / 
                     
                        
                       
                         
                           〈 
                           R 
                           〉 
                         
                         + 
                         
                           〈 
                           G 
                           〉 
                         
                         + 
                         
                           〈 
                           B 
                           〉 
                         
                       
                        
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     8 
                   
                   ] 
                 
               
             
           
         
       
     
     Alternatively, in accordance with Equation 6, the amount of change is calculated as shown in the following Equation 9. 
     
       
         
           
             
               
                 
                   
                     Amount 
                      
                     
                         
                     
                      
                     of 
                      
                     
                         
                     
                      
                     change 
                   
                   = 
                   
                     
                        
                       
                         
                           
                             
                               
                                 α 
                                 × 
                                 
                                   Min 
                                    
                                   
                                     ( 
                                     
                                       
                                         
                                           〈 
                                           R 
                                           〉 
                                         
                                         - 
                                         
                                           〈 
                                           
                                             R 
                                             ′ 
                                           
                                           〉 
                                         
                                       
                                       , 
                                       0 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                               
                                 β 
                                 × 
                                 
                                   Min 
                                    
                                   
                                     ( 
                                     
                                       
                                         
                                           〈 
                                           G 
                                           〉 
                                         
                                         - 
                                         
                                           〈 
                                           
                                             G 
                                             ′ 
                                           
                                           〉 
                                         
                                       
                                       , 
                                       0 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                             
                           
                         
                         
                           
                             
                               γ 
                               × 
                               
                                 Min 
                                  
                                 
                                   ( 
                                   
                                     
                                       
                                         〈 
                                         B 
                                         〉 
                                       
                                       - 
                                       
                                         〈 
                                         
                                           B 
                                           ′ 
                                         
                                         〉 
                                       
                                     
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                           
                         
                       
                        
                     
                     / 
                     
                        
                       
                         
                           〈 
                           R 
                           〉 
                         
                         + 
                         
                           〈 
                           G 
                           〉 
                         
                         + 
                         
                           〈 
                           B 
                           〉 
                         
                       
                        
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                      
                     
                         
                     
                      
                     9 
                   
                   ] 
                 
               
             
           
         
       
     
     Note that Equations 7 to 9 indicate the rates of change corresponding to the amounts of change in Equations 1 to 3 when α=β=γ=1. 
     After step S 19  is executed, the process returns to step S 11  described above. In this way, if it is judged that the processes of all images are executed in step S 11 , the process returns to a main process not shown. 
       FIG. 5A  is a flowchart showing a process of selecting center portions of a plurality of target elements in the selected region in the image analysis apparatus  10 . 
     When the image analysis apparatus  10  enters the process in step S 15  of  FIG. 4 , the edge detection section  21  applies an edge detection filter to the selected region (for example, region in the lower left half of the image Pi shown in  FIG. 9 ) to extract edge components (step S 21 ). 
     Next, the closed curve edge detection section  22  further detects edges forming closed curves from the edges detected by the edge detection section  21  (step S 22 ). 
     Subsequently, the size filter processing section  23  calculates sizes (for example, maximum diameter of closed curve, average diameter, and area of region surrounded by closed curve) of the closed curve edges detected by the closed curve edge detection section  22  and selects only the closed curve edges in which the calculated sizes are in the possible range of the target elements (for example, in the range of the possible size of intestinal villi) (step S 23 ). 
     The double closed curve edge detection section  24  detects all of the double closed curve edges from the closed curve edges passed through the size filter processing section  23  (step S 24 ). 
     Note that both the inner closed curve edges and the outer closed curve edges included in the double closed curve edges have passed through the process by the size filter processing section  23  in step S 23 , and the inner closed curve edges and the outer closed curve edges are closed curve edges judged to have sizes in the possible range of the target elements. 
     Furthermore, the double closed curve edge specification section  25  executes a process of specifying whether the double closed curve edges detected by the double closed curve edge detection section  24  are the target elements as described later with reference to  FIG. 6  (step S 25 ). 
     Subsequently, the region extraction control section  27  judges whether there is a double closed curve edge not yet subjected to the process of step S 25  among the double closed curve edges detected by the double closed curve edge detection section  24  (step S 26 ). If there is a double closed curve edge not yet subjected to the process of step S 25 , the process of step S 25  is applied to a next double closed curve edge. 
     In this way, if it is judged in step S 26  that the process of step S 25  is applied to all of the double closed curve edges, the region extraction control section  27  judges whether the number of double closed curve edges judged to be the target elements (further, the number of detected center points of the target elements) is equal to or greater than a predetermined number (five in the example shown in  FIG. 11 ) (step S 27 ). 
     Here, if it is judged that the number of double closed curve edges judged to be the target elements is less than the predetermined number, the single closed curve edge specification section  26  executes a process of specifying whether the single closed curve edges that are not the double closed curve edges (the single closed curve edges are closed curve edges passed through the process by the size filter processing section  23  in step S 23  and judged to have sizes in the possible range of the target elements) are the target elements as described later with reference to  FIG. 7  (step S 28 ). 
     Next, the region extraction control section  27  judges whether there is a single closed curve edge not yet subjected to the process of step S 25  among the single closed curve edges (step S 29 ). If there is a single closed curve edge not yet subjected to the process of step S 25 , the process of step S 28  is applied to a next single closed curve edge. 
     In this way, if it is judged in step S 29  that the process of step S 28  is applied to all of the single closed curve edges or if it is judged in step S 27  that the number of double closed curve edges judged to be the target elements is equal to or greater than the predetermined number, the process returns to the process shown in  FIG. 4 . 
     In this way, the double closed curve edges that are more likely to be the target elements are first specified, and when the number of double closed curve edges judged to be the target elements is less than the predetermined number, whether the single closed curve edges are the target elements is further specified. 
     Note that although the single closed curve edges are not specified if the number of double closed curve edges reaches the predetermined number in the process of  FIG. 5A , the single closed curve edges may be specified regardless of whether the number of double closed curve edges reaches the predetermined number. 
       FIG. 5B  is a flowchart showing a modification of the process of selecting the center portions of the plurality of target elements in the selected region in the image analysis apparatus. 
     The process of step S 27  in  FIG. 5A  is eliminated in the process shown in  FIG. 5B . As a result, not only the double closed curve edges, but also the single closed curve edges are specified. Therefore, the center portions of more target elements can be selected. 
       FIG. 6  is a flowchart showing the double closed curve edge specification process in the image analysis apparatus  10 . 
     When the image analysis apparatus  10  enters the process, the double closed curve edge specification section  25  selects one unprocessed double closed curve edge from the double closed curve edges detected by the double closed curve edge detection section  24  in step S 24  (step S 31 ). 
     The double closed curve edge specification section  25  judges whether, for example, the average value of the color component values of the respective pixels inside of the inner closed curve edge of the selected double closed curve edge is in the first color range corresponding to the center portion of the target element (step S 32 ). 
     Here, if the double closed curve edge specification section  25  judges that the average value is out of the first color range, the double closed curve edge selected in step S 31  is not identified as the target element, and the process returns to the process shown in  FIG. 5A  (or  FIG. 5B , the same applies hereinafter, and this will not be repeatedly described). 
     If the double closed curve edge specification section  25  judges that the average value is in the first color range in step S 32 , the double closed curve edge specification section  25  further judges whether, for example, the average value of the color component values of respective pixels between the outer closed curve edge and the inner closed curve edge of the selected double closed curve edge is in the second color range corresponding to the peripheral portion of the target element (step S 33 ). 
     Here, if the double closed curve edge specification section  25  judges that the average value is out of the second color range, the double closed curve edge selected in step S 31  is not identified as the target element, and the process returns to the process shown in  FIG. 5A . 
     If the double closed curve edge specification section  25  judges that the average value is in the second color range in step S 33  (therefore, if the double closed curve edge specification section  25  judges that the color of the region in the inner closed curve edge and the color of the region between the inner closed curve edge and the outer closed curve edge are different), it is determined that the double closed curve edge selected in step S 31  is the target element. The inside of the inner closed curve edge is specified as the center portion of the target element, and the region between the outer closed curve edge and the inner closed curve edge is specified as the peripheral portion of the target element (step S 34 ). The process returns to the process shown in  FIG. 5A . 
       FIG. 7  is a flowchart showing the single closed curve edge specification process in the image analysis apparatus  10 . 
     When the image analysis apparatus  10  enters the process, the single closed curve edge specification section  26  selects one unprocessed closed curve edge in the single closed curve edges other than the double closed curve edges among the closed curve edges passed through the size filter processing section  23  (step S 41 ). 
     The single closed curve edge specification section  26  then judges whether, for example, the average value of the color component values of the respective pixels inside of the selected single closed curve edge is in the first color range corresponding to the center portion of the target element (step S 42 ). 
     Here, if the single closed curve edge specification section  26  judges that the average value is out of the first color range, the single closed curve edge selected in step S 41  is not identified as the target element, and the process returns to the process shown in  FIG. 5A . 
     If the single closed curve edge specification section  26  judges that the average value is in the first color range in step S 42 , the single closed curve edge specification section  26  further judges whether, for example, the average value of the color component values of the respective pixels near the outside of the selected single closed curve edge is in the second color range (the second color range different from the first color range) corresponding to the peripheral portion of the target element (step S 43 ). 
     Here, if the single closed curve edge specification section  26  judges that the average value is out of the second color range, the single closed curve edge selected in step S 41  is not identified as the target element, and the process returns to the process shown in  FIG. 5A . 
     If the single closed curve edge specification section  26  judges that the average value is in the second color range in step S 43 , (therefore, if the single closed curve edge specification section  26  judges that the color of the inside region of the single closed curve edge and the color of the outside near region are different), the inside of the single closed curve edge selected in step S 41  is specified as the center portion of the target element (step S 44 ), and the process returns to the process shown in  FIG. 5A . 
     Note that although the respective processes, such as the edge detection (closed curve edge detection, double closed curve edge detection), the size filtering process, and the color range judgement, are executed in  FIGS. 5A to 7  to increase the detection accuracy of the target elements, any of the processes may be skipped to reduce the processing load to improve the detection speed. 
     According to the first embodiment, the region extraction section  12  specifies the target element including the annular peripheral portion and the center portion that is surrounded by the peripheral portion and that is in a color different from the peripheral portion and extracts only the center portion of the target element as the region to be analyzed (more specifically, the feature region that changes in the subject is focused to analyze the color change of the feature region). Therefore, more accurate quantitative evaluation can be performed in a necessary region. 
     The difference between the colors of the peripheral portion and the center portion is judged based on the difference in at least one of the hue, the saturation, and the luminance. Therefore, the judgement can be based on the color component values of the image. 
     Furthermore, edges are detected from the image, and the edges that form the closed curves are further detected. When the colors inside and outside of the region surrounded by the detected closed curve edge are different, the inside of the region surrounded by the closed curve edge is specified as the center portion. Therefore, the target element including the center portion and the peripheral portion in different colors can be accurately detected. 
     The inside of the region surrounded by the closed curve edge is specified as the center portion only when the size of the closed curve edge is in the possible range of the target element. Therefore, the detection accuracy of the target element can be further improved. 
     In addition, the double closed curve edge is further detected, and the region in the inner closed curve edge is specified as the center portion when the color of the region in the inner closed curve edge and the color of the region between the inner closed curve edge and the outer closed curve edge are different. Therefore, the consistency with the shape of the target element including the center portion and the peripheral portion can be higher in the detection. 
     A plurality of target elements are specified, and the center portions of the plurality of specified target elements are extracted as the regions to be analyzed. Therefore, the color component values of the regions to be analyzed are extracted from more samples, and the degree of change between the color component values of the first image and the color component values of the second image calculated based on the extracted color component values can be a more stable value. 
     Furthermore, the inappropriate elements not suitable for extracting the color component values are excluded in extracting the regions to be analyzed. Therefore, more accurate image analysis results not affected by the inappropriate elements can be obtained. 
     The center portions of the predetermined number of target elements in which the brightness is close to the median are extracted as the regions to be analyzed, and the amount of change can be more appropriately obtained. 
     The regions to be analyzed are extracted from the appropriate luminance regions in the appropriate luminance range in which the average luminance is suitable for extracting the color component values. This can prevent too bright regions and too dark regions, in which the amount of change may not be appropriately reflected on the pixel values even when there is a change in the subject, from becoming the regions to be analyzed. 
     The advantageous effects can also be attained in images of a subject picked up and acquired by the endoscope  20 . 
     Furthermore, appropriate image analysis can be performed for, for example, intestinal villi. 
     Note that the respective sections may be configured by circuits. An arbitrary circuit may be implemented as a single circuit as long as the same function can be attained, or the arbitrary circuit may be implemented by combining a plurality of circuits. Furthermore, an arbitrary circuit is not limited to a dedicated circuit for attaining the intended function, and the arbitrary circuit may be configured to cause a general-purpose circuit to execute a processing program to attain the intended function. 
     Although the image analysis apparatus (or the image analysis system, the same applies hereinafter) is mainly described above, an operation method of causing the image analysis apparatus to operate as described above may be implemented. A processing program for causing a computer to execute a process similar to the image analysis apparatus, a computer-readable non-transitory recording medium recording the processing program, and the like may also be implemented. 
     Furthermore, the present invention is not limited to the embodiment as it is, and in an execution phase, the constituent elements can be modified without departing from the scope of the present invention to embody the present invention. A plurality of constituent elements disclosed in the embodiment can be appropriately combined to form various aspects of the invention. For example, some of the constituent elements illustrated in the embodiment may be deleted. Furthermore, constituent elements across different embodiments may be appropriately combined. In this way, it is obvious that various modifications and applications can be made without departing from the scope of the invention.