Abstract:
An automatic parcel volume capture system and an automatic parcel volume capture method are provided. An automatic parcel volume capture system in accordance with an embodiment of the present invention includes stereo image input means, image processing means, feature extraction means, and volume measurement means. The stereo image input means captures images of an object from at least two different angles. The image processing means performs signal-processing on the images captured by the stereo image input means and extracts region of object in the images. The feature extraction means extracts lines and cross points of the lines from results of the image processing means. The volume measurement means generates three dimensional model on the basis of the extracted images and measures volume of the object.

Description:
TECHNICAL FIELD 
     The present invention relates to an automatic parcel volume capture system and a volume capture method using parcel image recognition. In particular, the automatic parcel volume capture system and the volume capture method utilize image recognition techniques to extract feature of a parcel and measure accurate volume of it. 
     BACKGROUND OF THE INVENTION 
     Conventional volume measurement systems have employed cameras or laser devices. Light curtain technology and PILAR (Parallel Infrared Laser Rangefinder) use laser devices. On the other hand, a method using two still images, one side view and the other front view, utilizes cameras. 
     The method with laser devices is usually used for recognizing moving objects and this method requires large cost for equipments. In the method with cameras, objects to be recognized are very sensitive with lighting variation, which actually affects image processing and feature extraction for parcel image recognition. Therefore, this method with camera is not appropriate for parcel image recognition because the environment processing parcels is not good enough. And if shape of the parcel is not rectangular in the method with two images, efficiency of recognition is deteriorated. 
     Conventional volume measurement systems have employed edge detection techniques such as Sobel operator to recognize edges of input images and the edge detection techniques are basically based upon difference between adjacent pixels in terms of brightness. However, if variation of brightness in an image is big, the conventional volume measurement systems are not able to detect edges efficiently and therefore error of volume measurement increases. 
     SUMMARY OF THE INVENTION 
     An automatic parcel volume capture system and an automatic parcel volume capture method are provided. An automatic parcel volume capture system in accordance with an embodiment of the present invention includes stereo image input means, image processing means, feature extraction means, and volume measurement means. The stereo image input means captures images of an object from at least two different angles. The image processing means performs signal-processing on the images captured by the stereo image input means and extracts region of object in the images. The feature extraction means extracts lines and cross points of the lines from results of the image processing means. The volume measurement means generates three dimensional model on the basis of the extracted images and measures volume of the object. 
     Preferably, the automatic parcel volume capture system further includes volume storage means for storing volume of the object measured by the volume measurement means. 
     Preferably, the stereo image input means includes image capturing means and image preprocessing means. The image capturing means captures images of the object from at least two different angles. The image preprocessing means averages the captured images and removes noises. 
     Preferably, the image capturing means is a charge coupled device (CCD) camera. 
     Preferably, the image processing means includes edge detecting means and region extracting means. The edge detecting means detects all edges in the captured images. The region extracting means extracts object region by comparing background image with object image in reference to the detected edges. 
     Preferably, the feature extraction means includes line extracting means and feature point extracting means. The line extracting means extracts lines of an object from result of the image processing means. The feature point extracting means extracts crossing points of an object by finding intersection points of the extracted lines. 
     Preferably, the volume measurement means includes matched junction capturing means, three dimensional model generating means, and volume calculating means. 
     The matched junction capturing means matches same crossing points among crossing points captured from the image. The three dimensional model generating means generates three dimensional model of an object on the basis of the matched junction captured by the matched junction capturing means. The volume calculating means calculates volume of the object on the basis of the three dimensional model. 
     Preferably, the volume measurement means further includes error minimizing means for compensating error of the three dimensional model generated by the three dimensional model generating means. 
     An automatic parcel volume capture method in accordance with an embodiment of the present invention includes stereo image input step, image processing step, feature extraction step, and volume measurement step. The stereo image input step captures images of an object from at least two different angles. The image processing step performs signal-processing on the images captured at the stereo image input step and extracts region of object in the images. The feature extraction step extracts lines and crossing points of the lines from results of the image processing step. The volume measurement step generates three dimensional model on the basis of the extracted images and measures volume of the object. 
     Preferably, the volume storage step stores volume of the object measured by the volume measurement step. 
     Preferably, the stereo image input step includes image capturing step and image preprocessing step. The image capturing step captures images of the object from at least two different angles. The image preprocessing step averages the captured images and removes noises. 
     Preferably, the image preprocessing step includes following steps. First step covers a W×W window around a current pixel x in N×N input image. Second step calculates local average and variation regarding all pixels in the window. Third step applies the average and the variation to the following equation 1 for MDIM (Mean difference Dynamic Image Model) and applies the average and the variation to the following equation 2 for NDIM (Normalized Dynamic Image Model). Fourth step repeats the first step to third step with increasing the current pixel x up to N×N sequentially. 
     
       
           I   m2 ( x+δx )=α( x )· I   m1 ( x ), where  I   mi ( x )= I   i ( x )− m   i ( x )  [Equation 1] 
       
     
     
       
           I   ms2 ( x+δx )= I   ms1 ( x ), where  I   msi ( x )=( I   i ( x )− m   i ( x ))/ S   1 ( x )  [Equation 2] 
       
     
     x: position of the current pixel 
     I i (x): brightness of x 
     α(x): local brightness changing component 
     S 1 (x): square value of the local variation 
       m i (x) : local average of x 
     I mi (x): difference between brightness of x I i (x) and local average of x m i (x) 
     I msi (x): I mi (x) divided by S i (x) 
     Preferably, the image processing step includes edge detecting step and region extracting step. The edge detecting step detects all edges in the captured images. The region extracting step extracts object region by comparing background image with object image in reference to the detected edges. 
     Preferably, the edge detecting step includes following steps. First step samples an N×N image, calculates average and variation regarding the sampled image, and obtains statistical feature of the image. Second step extracts candidate edge pixels among all pixels in the image, brightness of the candidate edge pixels being significantly different from brightness of adjacent pixels. Third step connects candidate edge pixels extracted at the second step. Fourth step stores the candidate edge pixels as final edge pixels if length of the connected pixels is longer than threshold length or stores the candidate edge pixels as non-edge pixels if length of the connected pixels is shorter than threshold length. 
     Preferably, the second step detects maximum value and minimum value among differences between brightness of current pixel and brightness of eight adjacent pixels, classifies the current pixel as into a non-edge pixel if the maximum value and the minimum value are smaller than threshold value, and classifies the current pixel as into an edge pixel if the maximum value and the minimum value are bigger than threshold value, the threshold value being determined by statistical feature of the image. 
     Preferably, the third step determines magnitude and direction of an edge by applying Sobel operator to the candidate edge pixels, classifies the edge pixel whose magnitude and direction are determined into a non-edge pixel if magnitude of the edge pixel is smaller than magnitude of other candidate edge pixels, and connects remaining candidate edge pixels with adjacent candidate edge pixels. 
     Preferably, the feature extraction step includes line extracting step and feature point extracting means. The line extracting step extracts lines of an object from result of the image processing step. The feature point extracting step extracts crossing points of an object by finding intersection points of the extracted lines. 
     Preferably, the volume measurement step includes matched junction capturing step, three dimensional model generating step, and volume calculating step. The matched junction capturing step matches same crossing points among crossing points captured from the image. The three dimensional model generating step generates three dimensional model of an object on the basis of the matched junction captured at the matched junction capturing step. The volume calculating step calculates volume of the object on the basis of the three dimensional model. 
     Preferably, the volume measurement step further includes error minimizing step for compensating error of the three dimensional model generated by the three dimensional model generating step. 
     Preferably, the matched junction capturing step captures matched junction by utilizing crossing points of the object captured in the image and epipolar geometry. 
     An automatic parcel volume capture method implemented in a computer system is provided. An automatic parcel volume capture method implemented in a computer system in accordance with an embodiment of the present invention includes stereo image input step, image processing step, feature extraction step, and volume measurement step. The stereo image input step captures images of an object from at least two different angles. The image processing step performs signal-processing on the images captured at the stereo image input step and extracts region of object in the images. The feature extraction step extracts lines and crossing points of the lines from results of the image processing step. The volume measurement step generates three dimensional model on the basis of the extracted images and measures volume of the object. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The embodiments of the present invention will be explained with reference to the accompanying drawings, in which: 
     FIG. 1 is a diagram illustrating an automatic parcel volume capture system in accordance with an embodiment of the present invention; 
     FIG. 2 is a flow diagram illustrating image preprocessing method of the automatic parcel volume capture system in accordance with an embodiment of the present invention; 
     FIG. 3 is a flow diagram illustrating an edge detection method of image processing unit in accordance with an embodiment of the present invention; and 
     FIG. 4 is a flow diagram illustrating parcel image recognition step of of the automatic parcel volume capture system in accordance with an embodiment of the present invention. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     FIG. 1 is a diagram illustrating an automatic parcel volume capture system in accordance with an embodiment of the present invention. 
     As shown in FIG. 1, the automatic parcel volume capture system in accordance with an embodiment of the present invention includes input image device  110 , image processing device  120 , feature extraction device  130 , volume measure device  140 , and volume save device  150 . The input image device  110  receives an image of an object and removes shadow and noise. The image processing device  120  performs edge detection and extracts region of the object from the input image. The feature extraction device  130  extracts feature from the image resulted from the image processing device  120 . The volume measure device  140  measures volume of the parcel and the volume save device  150  stores volume of the measured parcel. 
     The input image device  110  includes two stereo image capture devices  112  and  113  and two image preprocessors  114  and  115 . The two image preprocessors  114  and  115  perform average operation on the captured image. If a stereo image is averaged, an image that is insensitive to lighting variation is obtained. CCD cameras may implement the stereo image capture devices  112  and  113 . 
     Now, let&#39;s get into more details. The input image device  110  performs image capture process and image preprocessing process on the parcel  111 . The parcel  111  is moved by a tray and stopped in front of the two stereo image capture devices  112  and  113  for a second. The first stereo image capture devices  112  and the second stereo image capture devices  113  generate a stereo image of the input image. The two image preprocessors  114  and  115  perform average operation on the stereo image and convert the image in order that captured image may be well recognized regardless of lighting of the photographing environment. 
     FIG. 2 is a flow diagram illustrating image preprocessing method of the automatic parcel volume capture system in accordance with an embodiment of the present invention. Let&#39;s say that size of the parcel  111  is N×N and a variable x represents the current pixel. When an image is provided from the input image device  110  at step S 111 , following steps from S 112  to S 118  are executed as the current pixel x increases up to N×N sequentially. 
     First, a W×W window having center at pixel x is covered at step S 113  and local average and variation for all pixels in the window are calculated at step S 114  and S 115 . Using the calculated average and variation, an image based upon MDIM (Mean difference Dynamic Image Model) and an image based upon NDIM (Normalized Dynamic Image Model) are described and the current pixel x is increased by 1 at step S 116  and S 117 . If the current pixel x is larger than N×N at step S 118 , MDIM is stored at S 119 . If the current pixel x is not lager than N×N at step S 118 , the step S 113  is executed. 
     Equation 1 describes MDIM in which local average is removed. 
     
       
           I   m2 ( x+δx )=α( x )· I   m1 ( x ), where  I   mi ( x )= I   i ( x )− m   i ( x )  [Equation 1] 
       
     
     x: position of the current pixel 
     I i (x): brightness of x 
     α(x): local brightness changing component 
       m 1 (x) : local average of x 
     I mi (x): difference between brightness of x I i (x) and local average of x m i (x) 
     
       
           I   ms2 ( x+δx )= I   ms1 ( x ), where  I   msi ( x )=( I   i ( x )− m   i ( x ))/ S   1 ( x )  [Equation 2] 
       
     
     S 1 (x): square value of the local variation 
     I msi (x): I mi (x) divided by S i (x) 
     The image processing device  120  includes an edge detecting unit  121  and an region extracting unit  122 . The edge detecting unit  121  and the region extracting unit  122  detect edge of the object provided from the image preprocessors  114  and  115  and extracts region of the object. The edge detecting unit  121  utilizes statistical techniques for edge detection. The statistical techniques are advantageous because they are robust against external environmental lighting variation. In order to expedite edge detection, candidate edge pixels are estimated and magnitude and direction of the candidate edge pixels are determined. Detection for object region is performed by comparison between pre-stored background image and the image including the object. 
     FIG. 3 is a flow diagram illustrating an edge detection method of image processing unit in accordance with an embodiment of the present invention. 
     The edge detection method of image processing unit in accordance with an embodiment of the present invention includes a step for extracting feature statistically, a step for determining edge pixels and detecting edge pixels, and a step for connecting detected edge pixels and removing short edge pixels. 
     First, a N×N image is received at step S 211  and sampling is performed on the image with a particular number of pixels at step S 212 . Average and variation are calculated for the sampled pixels at step S 213 . Then, average and variation of the sampled image are set as a statistical feature of the current image. Threshold value (Th1) are determined by utilizing the statistical feature at step S 214 . 
     Once the statistical feature is determined, candidate edge pixels are determined regarding all pixels of the input image. For this, maximum value and minimum value among differences between brightness of current pixel and brightness of eight adjacent pixels are detected at step S 215  and the difference between the maximum value and the minimum value is compared with the threshold (Th1). As stated before, the threshold (Th1) is set by utilizing statistical feature of the input image. 
     If the difference between the maximum value and the minimum value is larger than the threshold (Th1) at step S 216 , the pixel is determined as a candidate edge pixel and step S 218  and step S 219  are executed. If the difference between the maximum value and the minimum value is smaller than the threshold (Th1) at step S 216 , the pixel is determined as a non-edge pixel and it is stored at non-edge pixel database  217 . 
     If the pixel is determined as a candidate edge pixel, Sobel operator is used to determine magnitude and direction of the edge at step S 218  and S 219 . At step S 219 , direction of the edge is described by GLSC (Gray Level Similarity Code)  220 . 
     Once magnitude and direction are determined, edges whose direction is different from the adjacent edges are removed at step S 221 . Such processes are called Edge Non-maximal Suppression and edge lookup table  222  is used. Edge Non-maximal Suppression is an algorithm in which direction components of current edge pixels are compared with direction components of adjacent edge pixels. If they are not identical, adjacent edge pixels are removed. As a result of the algorithm, an image described by edge lines from a thick edge image is obtained. 
     Finally, remaining candidate edge pixels are connected at step S 223 . If length of the connected edge is bigger than threshold (Th2) at step S 224 , the pixel is determined as an edge pixel  225 . If length of the connected edge is smaller than threshold (Th2) at step S 224 , the pixel is determined as a non-edge pixel  226 . The pixels determined as edge pixels by these processes usually represent boundary of objects or backgrounds. 
     The feature extraction device  130  include a line extraction unit  131  and a feature point extraction unit  132 . The line extraction unit  131  extracts lines from the result image of the image processing device  120 . The feature point extraction unit  132  determines crossing point of the extracted lines as a feature point. 
     That is, the line extraction unit  131  utilizes parcel edges obtained from the image processing device  120  and extracts lines of the parcel. The feature point extraction unit  132  finds intersections of the extracted lines and extracts crossing points. Such feature extraction units compensate blurring effect and deterioration caused by shape distortion of the parcel. 
     The volume measure device  140  acquires matched junction from the two stereo images resulted from the feature extraction unit, generates three dimensional model, repeats error minimizing process, and measure volume of the object. 
     The volume measure device  140  integrates two images obtained by the two CCD cameras and calculates volume of the parcel. The volume measure device  140  includes matched junction acquiring unit  141 , three dimensional model generating unit  142 , error minimizing unit  143 , and volume calculating unit  144  and calculated volume data is stored at the volume save device  150 . 
     The matched junction acquiring unit  141  utilizes Epipolar geometry and feature points obtained by the feature extraction device  130  and acquires matched junctions of the two stereo images. The three dimensional model generating unit  142  utilizes the matched junctions of the two stereo images, estimates relative position of the parcel  111 , and generates a three dimensional model. The three dimensional model is repeatedly applied to the error minimizing unit  143  and a precise three dimensional may be obtained through these processes. The volume calculating unit  144  utilizes the compensated three dimensional model and calculates exact volume of the parcel. 
     FIG. 4 is a flow diagram illustrating parcel image recognition step of of the automatic parcel volume capture system in accordance with an embodiment of the present invention. 
     The input image device  110  detects parcel image using two CCD cameras, which is indicated by  501  in FIG.  4 . The image processing device  120  detects edge and region of the parcel from result of the input image device  110 , which is indicated by  502  in FIG.  4 . The feature extraction device  130  extracts lines and crossing points from the result image of the image processing device  120 , which is indicated by  503  in FIG.  4 . The volume measure device  140  finds matched junction from the two result images of the feature extraction unit, which is indicated by  504  in FIG.  4  and generates three dimensional model, which is indicated by  505  in FIG.  4 . The generated three dimensional model goes through the error minimizing process to become a precise three dimensional model, which is indicated by  506  in FIG.  4 . The precise three dimensional model is used to calculate volume of the parcel and the calculated volume is stored into the volume save device  150 . 
     Although representative embodiments of the present invention have been disclosed for illustrative purpose, those who are skilled in the art will appreciate that various modifications, additions and substitutions are possible without departing from the scope and spirit of the present invention as defined in the accompanying claims.