Abstract:
A system and a method of obtaining a dimension of a target object in an image comprises receiving coordinates of a number of feature points in the image, receiving coordinates of at least one reference object in the image with a known dimension, performing a calibration to adjust the coordinates of at least one of the feature points, and receiving coordinates of the target object in the image and determining the dimension of the target object based on the coordinates of the feature points. The coordinates of at least one of the feature points are adjusted to increase an accuracy in determining the dimension of the reference object.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     This invention relates to measurement systems and methods, and more particularly, to systems and methods for obtaining a dimension of an object in an image. 
     2. Background of the Invention 
     With technology development and increase on the needs for society security, surveillance systems become a popular research topic and may be used in various applications. Many surveillance systems require a number of video cameras placed in several locations, and the recorded video images may be transmitted through cables or network to storage medium. The recorded video images may be referred to later for further analysis if an accident or incident occurred in the monitored area. Because manual identification is usually relied on for recognition of video images, it is difficult for surveillance systems to provide advance and/or preventive warning. Therefore, development of automatic analysis by computing systems has attracted a lot of attention. 
     Using visual technique to obtain geometrical information has received wide applications in recent years. Examples of its application include architectural and indoor measurements, reconstruction of objects in paintings, forensic measurements and traffic accident investigation. As an example, the technique may be used to classify people on the scene by their heights as well as for consumer target analysis. One approach to obtaining object dimension is, for example, to place one or more rulers somewhere in the monitored scene so that object dimension may later be estimated with reference to the rulers. Another approach is using a computer to analyze the captured visual information to obtain object dimension offline, sometimes with more accuracy, flexibility and efficiency. 
     There are a number of computing techniques for measuring objects from an image. For example, Criminisi et al. proposed an approach to compute object measurement from a single perspective images. A. Criminisi and A. Zisserman,  Single view metrology , International Conference on Computer Vision, Kekyrn, Greece, September 1999, pp. 434-442. It assumed that the vanishing line of a reference plane in the scene as well as a vanishing point in a reference direction may be determined from the image. Based on the vanishing line and point, distances between any plane which are parallel to the reference plane, area and length ratio on these planes and the camera&#39;s position may be computed. 
     Another approach is to use linear transformation between the camera and the 3D scene to obtain parameters which in turn may be used to compute object dimension. A. Bovyrin and K. Rodyushkin,  Human Height Prediction and Roads Estimation for Advanced Video Surveillance Systems , IEEE 2005, pp. 219-223. Wang, et al. proposed to obtain a camera projection matrix first through the homography of a reference space plan and its vertical vanishing point, and then use the matrix and some available scene constraints to retrieve geometrical entities of the scene, such as object height and distance from a point to a line. G. Wang, Z. Hu, F. Wu, and H. Tsui,  Single View Metrology From Scene Constraints , Image Vision Computing, Elsevier B.V. 2005. In another approach, object dimension is computed from the parameters obtained through the relationship between two uncalibrated images. Z. Chen, N. Pears, and B. Liang,  A Method of Visual Metrology From Uncalibrated Images , Pattern Recognition Letters, Elsevier B.V. 2006. 
     BRIEF SUMMARY OF THE INVENTION 
     One example consistent with the invention provides a method of obtaining a dimension of a target object in an image. The method may include receiving coordinates of a number of feature points in the image, receiving coordinates of at least one reference object in the image with a known dimension, performing a calibration to adjust the coordinates of at least one of the feature points, and receiving coordinates of the target object in the image and determining the dimension of the target object based on the coordinates of the feature points. The coordinates of at least one of the feature points are adjusted to increase an accuracy in determining the dimension of the reference object 
     In another example, a method of obtaining a dimension of a target object in an image is provided. The method may include receiving coordinates corresponding to a number of feature points in the image, determining coordinates corresponding to vanishing points based on the coordinates corresponding to the feature points, receiving coordinates corresponding to at least one reference object in the image with a known dimension, determining the dimension corresponding to the reference object based on the coordinates corresponding to the vanishing points, performing a calibration to adjust the coordinates corresponding to the feature points, and receiving coordinates of the target object in the image and determining the dimension of the target object based on the vanishing points. The coordinates corresponding to the feature points are adjusted to increase an accuracy in determining the dimension of the reference object 
     Another example consistent with the invention provides a system for obtaining a dimension of a target object in an image. The system may include a first device capable of providing coordinates corresponding to a number of feature points in the image and providing coordinates corresponding to at least one reference object in the image, wherein a dimension of the reference object is known, and a calibration device for receiving the coordinates corresponding to the feature points and the coordinates corresponding the reference object and for adjusting the coordinates corresponding to the feature points. The coordinates corresponding to the feature points are adjusted to increase accuracy in determining the dimension of the reference object. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
       The foregoing summary, as well as the following detailed description of the invention, will be better understood when read in conjunction with the appended, exemplary drawings. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown. 
       In the drawings: 
         FIG. 1  is a diagram of an exemplary scene for illustrating one application consistent of the present invention; 
         FIGS. 2A-2C  are diagrams for illustrating certain principles of geometry; 
         FIG. 3  illustrates an exemplary flow chart of a method for object dimension estimation in examples consistent with the present invention; 
         FIG. 4  is an exemplary illustration of a scene; 
         FIG. 5  is an exemplary block diagram for illustrating an image processing technique consistent with the present invention; 
         FIG. 6  is an illustration of a scene used for applying the object dimension estimation method in examples consistent with the invention; and 
         FIG. 7  is an illustration of a scene used for applying the object dimension estimation method in examples consistent with the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       FIG. 1  shows an exemplary example that the present invention may be implemented. With reference to  FIG. 1 , a monocular camera, such as an ordinary video CCD surveillance camera  100  and a digital video camera  102 , may be used to capture images. The camera  100  or  102  is set up in a way so that a ground plane  104  is included in the captured scene. In addition, the camera  100  or  102  is connected to a computer  106  that incorporates the present invention therein. With the camera  100  or  102  set up appropriately, optimal space calibration discussed in detail below may be performed to obtain optimal parameters for object measurement. Based on the optimal parameters, any object in the image, for example, a person  108 , a vehicle  110 , a tree  114  and a house  112 , may be estimated as long as the top and bottom coordinates of the object are provided. 
       FIGS. 2A-2C  illustrate certain principles of geometry regarding vanishing lines and vanishing points of a plane. Referring to  FIG. 2A , a reference plane in 3D space is often, but not necessary, the ground plane  200 . A set of parallel lines  202  on the ground plane  200  that are projected into a 2D image plane  210  becomes a set of concurrent lines  212 . The meeting point of these lines in the image plane  210  is called a vanishing point  214 . Connecting vanishing points  214  of all possible parallel lines on the ground plane  200  constitutes a vanishing line  216  as shown at  FIG. 2B . The vanishing line  216  and the camera center  218  may constitute a plane  220  parallel to the ground plane  200 . 
       FIG. 2B  shows an object in the 3D space between two planes relative to the distance of the camera center  218  from the ground plane  200 . Referring to  FIG. 2B , assuming that an object in 3D space has a bottom coordinate P and a top coordinate P′ with its height Z. The top and bottom coordinates are respectively on the ground plane  200  and a plane  230  parallel to the reference plane  200 . A reference direction  222  is the vector of the planes  200  and  230  as shown at  FIG. 2B . V 3  is a vanishing point at the direction parallel to the reference direction  222  as shown at  FIG. 2B . C is an intersection point of the plane  220  and a line parallel to the reference direction  222 . Since both the camera center  218  and the point C fall on the plane  220 , the distance between the point C and the ground plane  200  is equal to the distance Z c  of the camera center  218  from the ground plane  200 . Corresponding to  FIG. 2B ,  FIG. 2C  shows an object on the image plane  210  between two planes relative to the distance of the camera center  218  from one of the two planes. Points  214  are the two vanishing points of the plane  200  and constitute a vanishing line  216 . v 3  is a vanishing point at the direction perpendicular to the plane  200 . c is the intersection point of the vanishing line  216  and the line perpendicular to the plane  200  and connecting to v 3 . Upper case letters (P) are used to indicate quantities in 3D space and lower case letters (p) to indicate image quantities. 
     The four points p, p′, C, v 3  marked on  FIG. 2C  define a cross-ratio. The value of the cross-ratio provides a length ratio in 3D space which may determine the distance Z between the planes  200  and  230  relative to the camera&#39;s distance Z c  from the ground plane  200  as below: 
                           d   ⁡     (     p   ,   c     )       ×     d   ⁡     (       p   ′     ,     v   3       )             d   ⁡     (       p   ′     ,   c     )       ×     d   ⁡     (     p   ,     v   3       )           =         d   ⁡     (     P   ,   C     )       ×     d   ⁡     (       P   ′     ,   V     )             d   ⁡     (       P   ′     ,   C     )       ×     d   ⁡     (     P   ,   V     )             ,           (   1   )               
where d(x1, x2) is distance between two generic points x1 and x2. Since the back projection of the point V is a point at infinity,
 
                 d   ⁡     (       P   ′     ,   V     )         d   ⁡     (     P   ,   V     )         =   1.         
Also since d(P, C)=Z c  and d(P′, C)=Z c −Z, simple algebraic manipulation on (1) yields
 
     
       
         
           
             
               
                 
                   
                     Z 
                     
                       Z 
                       C 
                     
                   
                   = 
                   
                     1 
                     - 
                     
                       
                         
                           
                             d 
                             ⁡ 
                             
                               ( 
                               
                                 
                                   p 
                                   ′ 
                                 
                                 , 
                                 c 
                               
                               ) 
                             
                           
                           × 
                           
                             d 
                             ⁡ 
                             
                               ( 
                               
                                 p 
                                 , 
                                 
                                   v 
                                   3 
                                 
                               
                               ) 
                             
                           
                         
                         
                           
                             d 
                             ⁡ 
                             
                               ( 
                               
                                 p 
                                 , 
                                 c 
                               
                               ) 
                             
                           
                           × 
                           
                             d 
                             ⁡ 
                             
                               ( 
                               
                                 
                                   p 
                                   ′ 
                                 
                                 , 
                                 
                                   v 
                                   3 
                                 
                               
                               ) 
                             
                           
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
     According to equation (2) above, the absolute distance Z may be obtained from this distance ratio once the camera&#39;s distance Z c  is specified. Alternatively, Z c  may be computed first based on a known reference distance, such as a known object dimension, and then the computed Z c  may be applied to estimate a dimension of an object in the image. 
       FIG. 3  illustrates an exemplary flow chart of an object measurement method in examples consistent with the present invention. As shown at  FIG. 3 , initial parameters are first set up at step  302 .  FIG. 4  illustrates an exemplary example of step  302 . In an image  400 , a reference cubic box  404  on the ground plane  402  provides a number of feature points f 1 -f 6 , wherein at least two planes are perpendicular to the reference plane in the 3D scene, for example the planes {f 1 ,f 2 ,f 3 ,f 4 } and {f 3 ,f 4 ,f 5 ,f 6 }. Each plane may comprise at least two parallel lines to the reference plane in 3D scene, for example, the lines  f 1 f 3    and  f 2 f 4    in {f 1 ,f 2 ,f 3 ,f 4 } and the lines  f 3 f 5    and  f 4 f 6    in {f 3 ,f 4 ,f 5 ,f 6 }. These feature points may provide at least two vertical lines to the reference plane in the 3D scene, for example, lines  f 1 f 2    and  f 3 f 4   . The feature points f 1 -f 6  are identified either by manually clicking via a mouse  408  on the image  400  or by computer automatic detection. In the case where the coordinates corresponding to the feature points f 1 -f 6  and the reference object  406  are selected via the mouse  408 , the image may be zoomed in prior first to increase the accuracy in selecting the coordinates provided to the object dimension estimation system. The extended lines of  f 1 f 3    and  f 2 f 4   ,  f 3 f 5    and  f 4 f 6   , and  f 1 f 2    and  f 3 f 4    may result in three vanishing points. A vanishing line may be established by connecting the two vanishing points that are on the lines parallel to the ground plane  402 . In the case where there are objects which height or dimensions are known, such as the reference cubic box  404  or the flag  406  shown at  FIG. 4 , the top and bottom coordinates q and q′ of the objects  406 ,  404  and their actual dimensions may be provided to the object dimension estimation system of the present invention to complete the step  302 . 
     Referring back to  FIG. 3 , step  304  performs an optimal space calibration to avoid errors in manual selection or automatic detection of point or coordinate position that may affect the computation of the vanishing points, and eventually, object dimension estimation. The coordinates corresponding to the feature points identified at step  302  are considered as initial parameters subject to adjustment. The object dimension estimation computed based on the initial parameters is evaluated using objective function by comparing to the actual dimension of the reference object. 
     Taking the cubic box  404  at  FIG. 4  as an example. The cubic box  404  may provide the coordinates of six feature points f 1 -f 6 , collectively represented as X={f i |i=1, 2, . . . 6}ε             2 . Since each feature point f 1  has coordinates as (f i   x ,f i   y ), six feature points may provide twelve parameters. Based on the twelve parameters, three vanishing points may be computed based on lines  f 1 f 3    and  f 2 f 4   , lines  f 3 f 5    and  f 4 f 6   , and lines  f 1 f 2    and  f 3 f 4   . The two vanishing points that are obtained from lines parallel to the reference planes may establish a vanishing line. Based on the vanishing line and the third vanishing point, the intersection point c as shown at  FIG. 2C  may be obtained. According to equation (2), the distance of the camera center  218  from the ground plane  200  may be computed as below:
                     Z   C     =         Z   ×     d   ⁡     (       f   3     ,   c     )       ×     d   ⁡     (       f   4     ,     v   3       )               d   ⁡     (       f   3     ,   c     )       ×     d   ⁡     (       f   4     ,     v   3       )         -       d   ⁡     (       f   4     ,   c     )       ×     d   ⁡     (       f   3     ,     v   3       )             .             (   3   )               
Assuming that the number of reference objects with their height known is N, collectively represented as S={(q j , q j ′,h j )|j=1, 2, . . . N}, the objective function of X is:
 
                       min   .           ⁢     F   ⁡     (   X   )         =       1   N     ⁢       ∑     j   =   1     N     ⁢           ⁢            h   j     -     Z   j                  ,           (   4   )               
where Z j  is the height of j th  reference object computed based on the following equation:
 
     
       
         
           
             
               
                 
                   
                     
                       Z 
                       j 
                     
                     = 
                     
                       
                         Z 
                         C 
                       
                       × 
                       
                         ( 
                         
                           1 
                           - 
                           
                             
                               
                                 d 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       q 
                                       j 
                                       ′ 
                                     
                                     , 
                                     c 
                                   
                                   ) 
                                 
                               
                               × 
                               
                                 d 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       q 
                                       f 
                                     
                                     , 
                                     
                                       v 
                                       3 
                                     
                                   
                                   ) 
                                 
                               
                             
                             
                               
                                 d 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       q 
                                       j 
                                     
                                     , 
                                     c 
                                   
                                   ) 
                                 
                               
                               × 
                               
                                 d 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       q 
                                       j 
                                       ′ 
                                     
                                     , 
                                     
                                       v 
                                       3 
                                     
                                   
                                   ) 
                                 
                               
                             
                           
                         
                         ) 
                       
                     
                   
                   , 
                   
                     j 
                     = 
                     1 
                   
                   , 
                   2 
                   , 
                   
                     … 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       N 
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
     In order to search for an optimal parameter set for X={f i |i=1, 2, . . . 6}ε             2 , the k th  parameter f k  is set between the range of [f k   0 −δ k ,f k   0 +δ k ], where δ k  is the search space and k=1, 2, 3, . . . 12. With computer simulation, such as genetic algorithm, simulated annealing, tabu search or particle swarm optimization, a population of abstract representations of candidate parameters evolves toward a better parameter set so that the estimated dimension of the reference object would be close to its actual dimension. D. E. Goldberg,  Genetic Algorithms in Search Optimization and Machine Learning , Addison-Wesley, Reading Mass., 1989, D. T. Pham and D. Karaboga,  Intelligent Optimisation Techniques Genetic Algorithms, Tabu Search, Simulated Annealing and Neural Networks . New York: Springer-Verlag, 2000, and M. Clerc,  Particle Swarm Optimization . Hermes Science Pubns, 2006, are incorporated herein with respect to the genetic algorithm, simulated annealing, tabu search and particle swarm optimization.
     Referring back to  FIG. 3 , after optimal space calibration of step  304 , object dimension estimation on any object in the same image may be estimated provided that the top and bottom coordinates corresponding to the particular object in the image are given. At step  306 , the coordinates corresponding to the object to be measured are provided to the system either via user input devices or automatic detection of the coordinates of the selected object. At step  308 , the object dimension may be obtained in accordance with equation (2) above. 
       FIG. 5  illustrates an exemplary example of steps  306  and  308 . A series of frames of video data  500  may be first captured by a snapshot  502  to obtain a particular image. Once the image is obtained, a user may select via a mouse or keyboard the top and bottom points of an object to be measured at  504 . Then the object dimension estimation may be obtained based on equation (2) above. Alternatively, object segmentation technology  508  may be relied on to detect the area that the object to be measured is located, thereby acquiring the top and bottom coordinates of the area by computer analysis. 
       FIGS. 6 and 7  are illustrations of a exemplary scene used for applying the object dimension estimation method in examples consistent with the invention. In this example, the experiments are conducted using a Logitech QuickCam Sphere digital video camera and a CCD Pulis P2650 video camera with image capture card Winnov Videum 1000+. The image resolution for both cameras is 640×480 pixels. Genetic algorithm is applied for optimal space calibration. The search space is set to δ k =5. The number of generations to iterate the algorithm is set to 5000. The size of population N pop  is 20. The selection operation P s  is set to 0.2, crossover P c  is 0.8, and the mutation rate P m  is set to 0.1. A cubic box with its dimension of 275 mm is used as a reference box. 
     In the first experiment, the digital video camera P 4  captures a scene as shown at  FIG. 6 . The actual height of the camera is 800 mm. The cubic box may provide six feature points, and three of the feature points P 1 , P 2  and P 3  are identified in  FIG. 6  taken from a vertical view. Assuming the coordinate of point O is (0, 0), the coordinates of each point on the image are P 1 (365, 175), P 2 (345, 140), P 3 (370, 125), P 4 (75, 90), Q 1 (380, 250), Q 2 (315, 170), Q 3 (410, 135), Q 4 (285, 120). Table 1 below shows the result of object dimension estimation based on the optimal coordinates of the feature points. 
                                                               TABLE 1                       Q 1     Q 2     Q 3     Q 4                                      Actual height (mm)   300   300   300   300       Pixel numbers (pixel)   83   109   88   129       Conversion ratio   3.61   2.75   3.41   2.33       (mm/pixel)       Estimated height (mm)   305.21   308.60   302.42   308.67       Error rate (%)   1.74   2.87   0.81   2.89                    
where pixel number means the number of pixels that the reference object occupies on the image, conversion ratio is (actual height/pixel numbers), and error rate is (|estimated height−actual height|/actual height).
 
     In the second experiment, the CCD camera P 4  may be used to capture a scene similar to what is shown in  FIG. 7 . The actual height of the camera is 2650 mm in this example. Assuming the coordinate of point O is (0, 0), the coordinates of each point on the image are P 1 (505, 260), P 2 (475, 230), P 3 (495, 210), P 4 (30, 220), Q 1 (475, 320), Q 2 (635, 320), Q 3 (840, 320), Q 4 (1170, 290), Q 5 (670, 185), Q 6 (755, 170), Q 5 (755, 100) Table 2 below shows the result of object dimension estimation based on the optimal coordinates of the feature points. 
     
       
         
               
               
               
               
               
               
               
               
             
               
               
               
               
               
               
               
               
             
           
               
                   
                 TABLE 2 
               
               
                   
                   
               
               
                   
                 Q 1   
                 Q 2   
                 Q 3   
                 Q 4   
                 Q 5   
                 Q 6   
                 Q 7   
               
               
                   
                   
               
             
             
               
                   
               
             
          
           
               
                 Actual height (mm) 
                 750 
                 750 
                 750 
                 750 
                 250 
                 750 
                 750 
               
               
                 Pixel numbers (pixel) 
                 111 
                 86 
                 70 
                 58 
                 30 
                 81 
                 81 
               
               
                 Conversion ratio 
                 6.76 
                 8.72 
                 10.71 
                 12.93 
                 8.33 
                 9.26 
                 9.26 
               
               
                 (mm/pixel) 
               
               
                 Estimated height (mm) 
                 750.08 
                 750.34 
                 743.52 
                 754.92 
                 257.86 
                 759.71 
                 751.03 
               
               
                 Error rate (%) 
                 0.01 
                 0.05 
                 0.86 
                 0.66 
                 3.14 
                 1.29 
                 0.14 
               
               
                   
               
             
          
         
       
     
     According to Table 1 and Table 2 above, the present invention may provide dimension estimation of an object from an image with high degree of precision. 
     It will be appreciated by those skilled in the art that changes could be made to the examples described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular examples disclosed, but it is intended to cover modifications within the spirit and scope of the present invention as defined by the appended claims.