Abstract:
An image processing device ( 10 ) comprises the following: an image acquisition unit ( 1 ) for acquiring an image in which markers for calibration are captured; an edge detection unit ( 2 ) for detecting the edges of the markers in the image; a polygon generating unit ( 3 ) for estimating a plurality of straight lines on the basis of the edges and generating a virtual polygon region that is surrounded by the plurality of straight lines, the generation being carried out in the image in a region thereof including regions other than those where markers are installed; and a camera parameter calculation unit ( 4 ) for calculating camera parameters on the basis of the characteristic amount, with respect to the image, of the virtual polygon region and the characteristic amount, with respect to real space, of the virtual polygon region.

Description:
TECHNICAL FIELD 
       [0001]    The present invention relates to an image processing apparatus and a marker for calculating a camera parameter. 
       BACKGROUND ART 
       [0002]    Camera systems using vehicle-mounted cameras in recent years include a function of displaying a guide line indicating an estimated location of a vehicle and superimposed on an image taken by a camera (hereinafter referred to as “camera image”) or a function of generating an image from a virtual viewpoint through geometric conversion of the camera image or the like. Both of these functions use association between a three-dimensional position in a real space and a two-dimensional position projected on the camera image. Implementing such association requires calculations (hereinafter referred to as “calibration,” as appropriate) of so-called camera parameters such as a setting angle or setting position of the camera or projective transformation parameters. 
         [0003]    Calibration based on a camera image normally uses a calibration marker whose shape, size, and setting position or the like are known. As one such mode, there is a method that takes an image of a marker including a specific pattern set on a road surface and uses a projection position with respect to the camera image (e.g., see Patent Literature (hereinafter abbreviated as “PTL”) 1). 
         [0004]    Moreover, as an example of a method using a marker having a polygonal shape, a method is known which takes an image of a square marker set on a road surface using a camera, projectively transforms the camera image, generates a bird&#39;s eye image and calculates a projective transformation parameter, which is an example of a camera parameter, based on the square shape and existing position in the bird&#39;s eye image (e.g., see PTL 2). 
       CITATION LIST 
     Patent Literature 
       [0005]    PTL 1 
         [0006]    Japanese Patent Application Laid-Open No. 2011-047725 
         [0007]    PTL 2 
         [0008]    Japanese Patent Application Laid-Open No. 2010-244326 
       SUMMARY OF INVENTION 
     Technical Problem 
       [0009]    The techniques disclosed in PTLs 1 and 2 require a maker to be set on a road surface, but the range in which the marker can be set may be restricted. An example of this case will be described using  FIG. 1 . 
         [0010]      FIG. 1  is a figure illustrating an example where a vehicle and a road surface are seen from above. In  FIG. 1 , since region a is a region through which vehicles pass, no marker can be set. Meanwhile, region c is a region located at more than a predetermined distance from one side of the vehicle, so that no marker can be set. In contrast, region b is a region located at less than a predetermined distance from one side of the vehicle, so that a marker can be set. 
         [0011]    When the range in which a marker can be set is restricted in this way, a marker having necessary size for calibration cannot be set at an appropriate position, which results in a problem that calibration accuracy may deteriorate. 
         [0012]    An object of the present invention is to provide an image processing apparatus and a marker capable of keeping the accuracy for calibration even when a range in which a calibration marker can be set is restricted. 
       Solution to Problem 
       [0013]    An image processing apparatus according to an aspect of the present invention includes: an image acquiring section that acquires a captured image of a calibration marker; an edge detection section that detects an edge or a feature point of the marker in the image; a polygon generation section that estimates a plurality of straight lines based on the edge or feature point and generates a virtual polygon region surrounded by the plurality of straight lines and including a region other than a region of the image in which the marker is set; and a camera parameter calculation section that calculates a camera parameter based on a feature value of the virtual polygon region in the image and a feature value of the virtual polygon region in a real space. 
         [0014]    A marker according to an aspect of the present invention is a marker used for calibration, in which an edge or a feature point of the marker in a captured image of the marker is detected; a plurality of straight lines are estimated based on the edge or the feature point; and a virtual polygon surrounded by the plurality of straight lines is generated to be a region of the image including a region other than a region in which the marker is set. 
       Advantageous Effects of Invention 
       [0015]    According to the present invention, it is possible to keep the accuracy of calibration even when a range in which a calibration marker can be set is restricted. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0016]      FIG. 1  is a figure illustrating an example of a range in which typical markers can be set; 
           [0017]      FIGS. 2A and 2B  are figures each illustrating a calibration environment of an image processing apparatus according to Embodiment 1 of the present invention; 
           [0018]      FIG. 3  is a block diagram illustrating a configuration example of the image processing apparatus according to Embodiment 1 of the present invention; 
           [0019]      FIG. 4  is a block diagram illustrating a configuration example of an image processing apparatus according to Embodiment 2 of the present invention; 
           [0020]      FIGS. 5A and 5B  are figures illustrating examples of images before and after distortion correction/viewpoint conversion according to Embodiment 2 of the present invention; 
           [0021]      FIG. 6  is a block diagram illustrating a configuration example of an edge detection section according to Embodiment 2 of the present invention; 
           [0022]      FIGS. 7A and 7B  are figures illustrating an image after marker region specification and an image after filtering according to Embodiment 2 of the present invention; 
           [0023]      FIGS. 8A to 8E  are figures illustrating examples of images before and after extraction of a candidate point according to Embodiment 2 of the present invention; 
           [0024]      FIG. 9  is a block diagram illustrating a configuration example of a polygon generation section according to Embodiment 2 of the present invention; 
           [0025]      FIGS. 10A to 10E  are figures illustrating examples of images after estimation of straight lines and images after specification of vertexes according to Embodiment 2 of the present invention; 
           [0026]      FIG. 11  is a block diagram illustrating a configuration example of a camera parameter calculation section according to Embodiment 2 of the present invention; 
           [0027]      FIGS. 12A and 12B  are explanatory diagrams illustrating square-likeness according to Embodiment 2 of the present invention; 
           [0028]      FIGS. 13A and 13B  are figures illustrating a processing example of calculations of a roll angle and a pitch angle according to Embodiment 2 of the present invention; 
           [0029]      FIG. 14  is a figure illustrating a processing example of calculation of a height according to Embodiment 2 of the present invention; 
           [0030]      FIGS. 15A and 5B  are figures illustrating an example of comparison between a marker setup according to the related art and a marker setup according to Embodiment 2 of the present invention; 
           [0031]      FIG. 16  is a figure illustrating an example of a calibration environment of an image processing apparatus according to Embodiment 3 of the present invention; 
           [0032]      FIG. 17  is a figure illustrating an example of a virtual polygon generation position according to Embodiment 3 of the present invention; 
           [0033]      FIG. 18  is a figure illustrating another example of a virtual polygon generation position according to Embodiment 3 of the present invention; 
           [0034]      FIGS. 19A and 19B  are figures illustrating other examples of markers and a virtual polygon according to each embodiment of the present invention; 
           [0035]      FIG. 20  is a figure illustrating other examples of markers and a virtual polygon according to each embodiment of the present invention; and 
           [0036]      FIGS. 21A and 21B  are figures illustrating still other examples of markers and a virtual polygon according to each embodiment of the present invention. 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0037]    Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. 
       Embodiment 1 
       [0038]    First, Embodiment 1 will be described. 
         [0039]      FIGS. 2A and 2B  are figures illustrating a calibration environment according to the present embodiment.  FIG. 2A  is a side view of a vehicle located on a road surface and a marker set on the road surface.  FIG. 2B  is a top view of the vehicle and the marker shown in  FIG. 2A . 
         [0040]    As shown in  FIG. 2A , vehicle  100  is equipped with camera  20  at its rear and includes therein image processing apparatus  10  (an example of an image processing apparatus according to the present invention). Camera  20  takes an image of marker  60  set on road surface R. This camera image is inputted to image processing apparatus  10  and used for calibration. 
         [0041]    As shown in  FIG. 2B , marker  60  is set on one side of range (hereinafter referred to as “marker settable range”) b in which a marker can be set. In  FIG. 2B , marker  60  has a shape including, for example, two isosceles right triangles arranged side by side. 
         [0042]      FIG. 3  is a block diagram illustrating a configuration example of image processing apparatus  10  according to the present embodiment. 
         [0043]    In  FIG. 3 , image processing apparatus  10  includes image acquiring section  1 , edge detection section  2 , polygon generation section  3  and camera parameter calculation section  4 . 
         [0044]    Image acquiring section  1  acquires a camera image taken by camera  20 . This camera image is a captured image of marker  60 . 
         [0045]    Edge detection section  2  applies image processing to the camera image acquired by image acquiring section  1  and detects a plurality of edges of marker  60  from the camera image. 
         [0046]    Polygon generation section  3  estimates a plurality of straight lines based on the plurality of edges detected by edge detection section  2 . For example, polygon generation section  3  estimates straight lines that pass through a group of more points making up an edge using, for example, Hough transform or least squares method which are publicly known schemes. In the camera image, polygon generation section  3  generates a virtual polygon region surrounded by those straight lines in a region including a region outside a marker settable range (region including both the marker settable range and range outside the marker settable range or only region outside the marker settable range). 
         [0047]    Camera parameter calculation section  4  calculates a camera parameter based on a feature value in the image of the virtual polygon region generated in polygon generation section  3  and a feature value in a real space of the virtual polygon region. Examples of the calculated camera parameters include a roll angle, pitch angle, and height as to camera  20  set in vehicle  100 . 
         [0048]    The above-described “feature value in the image of the polygon region” is information on the shape of a polygon in the image, and corresponds to a length of each side of the polygon in the image when the length of the side is used to calculate the roll angle or pitch angle. For example, when the virtual polygon region is a pentagon and the vertexes thereof are assumed to be A, B, C, D and E, the feature values are the lengths of sides AB, BC, CD, DE and EA in the image. Note that the length of each side is calculated based on coordinate values of vertexes A to E specified by vertex specification section  32  which will be described later. 
         [0049]    The above-described “feature value in the real space of the polygon region” refers to information on the actual shape of the polygon, and corresponds to the actual length of each side of the polygon when, for example, the length of the side is used to calculate the roll angle or pitch angle. For example, the virtual polygon region is a pentagon and when the vertexes thereof are assumed to be A, B, C, D and E, the feature values are the actual lengths of sides AB, BC, CD, DE and EA. Alternatively, for example, when the virtual polygon region is a square, the feature value is the fact that the side lengths are the same, and the actual length of one side. 
         [0050]    For example, the following two methods are available as the above-described “calculation of camera parameters.” One of the methods is a method that uniquely calculates a camera parameter using linear simultaneous equations or a nonlinear one-to-one correspondence table from a geometric relationship based on feature values in an image and feature values in a real space. The other is a method that obtains an evaluation value indicating the degree of matching between the shape of the polygon in the image and the shape of the polygon in the real space from the feature values in the image and feature values in the real space and calculates a camera parameter through optimization based on the evaluation value. For example, calculations of the roll angle and pitch angle correspond to the latter method, and the method calculates an evaluation value from the ratio in length between the respective sides of the polygon in the real space and the ratio in length between the respective sides of the polygon in the corresponding image and obtains a combination of roll angle and pitch angle at which the evaluation value becomes a minimum. 
         [0051]    As described above, image processing apparatus  10  of the present embodiment generates a virtual polygon region in a region outside a marker settable range based on marker edges detected from a captured image of a calibration marker and calculates a camera parameter based on the polygon region. With this feature, image processing apparatus  10  can keep the accuracy of calibration even when the marker settable range is restricted. 
       Embodiment 2 
       [0052]    Next, Embodiment 2 will be described. 
         [0053]    The calibration environment in the present embodiment is similar to the calibration environment according to Embodiment 1 shown in  FIG. 2 . 
         [0054]      FIG. 4  is a block diagram illustrating a configuration example of image processing apparatus  10  according to the present embodiment. 
         [0055]    In  FIG. 4 , image processing apparatus  10  includes image acquiring section  1 , edge detection section  2 , polygon generation section  3 , camera parameter calculation section  4 , and image distortion correction/viewpoint conversion section  5 . 
         [0056]    Image acquiring section  1  acquires a camera image taken by camera  20 . This camera image is a captured image of marker  60 . 
         [0057]    Image distortion correction/viewpoint conversion section  5  corrects distortion included in a camera image acquired by image acquiring section  1 , converts the camera image into a state of central projection and converts the image as its road surface R into a viewpoint seen from above in the sky like a bird&#39;s eye view using a preset external parameter (also referred to as “extrinsic parameter,” an example of a camera parameter). Note that the extrinsic parameter used here is one of an initial temporary parameter set from the beginning in image distortion correction/viewpoint conversion section  5 , a temporary parameter set by temporary parameter setting section  44 , which will be described later, and a parameter specified by angle specification section  45 , which will be described later. For example, a design value of the camera setting angle corresponding to the vehicle is set as an initial temporary parameter. 
         [0058]      FIGS. 5A and 5B  show examples of an image before and after distortion correction and viewpoint conversion. Image distortion correction/viewpoint conversion section  5  performs distortion correction and viewpoint conversion on the camera image shown in  FIG. 5A  to thereby generate an image shown in  FIG. 5B . 
         [0059]    Here, the aforementioned distortion correction and association of coordinates in the viewpoint conversion will be described. 
         [0060]    First, coordinates of an image before distortion correction and coordinates of an image after distortion correction are associated with each other via internal parameters (also referred to as “intrinsic parameters,” example of camera parameters) made up of optical axis center coordinates, focal length and lens distortion coefficient or the like. The intrinsic parameters are stored in image distortion correction/viewpoint conversion section  5  as fixed values calculated beforehand. The intrinsic parameter calculation method may be a publicly known method. One such publicly known method is a method that takes an image of a checkerboard whose square size is known, detects intersection positions of the pattern of the checkerboard from the captured image, optimizes the pattern so that the intersection points are arranged at equal intervals as much as possible, and thereby calculates intrinsic parameters. Note that when the camera lens is a fish-eye lens, intrinsic parameters relating to projective transformation corresponding to the fish-eye lens such as so-called equidistant projection or stereographic projection may be calculated. 
         [0061]    Next, coordinates of an image before viewpoint conversion and coordinates of an image after viewpoint conversion are associated with each other via extrinsic parameters of the camera made up of a camera setting angle with respect to the road surface and a setting position. The camera setting angle is expressed by a roll angle, pitch angle and yaw angle, and the camera setting position is expressed by X coordinate, Y coordinate and Z coordinate. In a three-dimensional coordinate system with reference to the road surface and the traveling direction of the vehicle, for example, the Y coordinate is defined as a direction perpendicular to the road surface, the Z coordinate is defined as a direction along the traveling direction of the vehicle and the X coordinate is defined as a direction perpendicular to the Y coordinate and Z coordinate. 
         [0062]    To obtain coordinates after distortion correction and viewpoint conversion, processing of distortion correction and viewpoint conversion is performed in order on the coordinates. To obtain an image after distortion correction and viewpoint conversion, image conversion is performed using, for example, a lookup table. This lookup table is obtained by performing inverse conversion of viewpoint conversion and inverse conversion of distortion correction in order using the coordinates after distortion correction and viewpoint conversion as a starting point, obtaining the corresponding coordinates and describing the correspondence thereof. 
         [0063]    Returning to  FIG. 4 , edge detection section  2  will be described. 
         [0064]    Edge detection section  2  applies image processing to the image generated in image distortion correction/viewpoint conversion section  5 , and thereby detects edges of marker  60 . 
         [0065]    As shown in  FIG. 6 , edge detection section  2  includes marker region specification section  21 , filtering section  22  and candidate point extraction section  23 . 
         [0066]    Marker region specification section  21  specifies the region of marker  60  (hereinafter referred to as “marker region”) from the image (e.g., the image in  FIG. 5B ) generated in image distortion correction/viewpoint conversion section  5 . An example of the image after marker region specification is shown in  FIG. 7A . As shown in  FIG. 7A , the region of marker  60  is specified. 
         [0067]    Examples of available marker region specification methods include a method using luminance information and a method detecting feature points. The method using luminance information detects markers using a difference between the luminance of the road surface and the luminance of the markers. For example, when the luminance of the road surface in the calibration environment is low, a marker region is specified by setting markers having high luminance on the road surface, taking images of the markers using a camera and then detecting a portion whose luminance is equal to or greater than a fixed value from the camera image. On the other hand, the method detecting the feature point uses, for example, common Harris&#39; corner detection technique, and thereby detects a plurality of marker end points based on the amount of variation in luminance in the camera image. 
         [0068]    Filtering section  22  performs filtering on the image after marker region specification (e.g., the image in  FIG. 7A ) using a specific operator (e.g., laplacian operator).  FIG. 7B  shows an example of the image after the filtering. As shown in  FIG. 7B , an outer circumference of marker  60  is specified. 
         [0069]    Candidate point extraction section  23  searches for a variation in luminance from a fixed direction in the image after the filtering (e.g., the image in  FIG. 7B ), and thereby specifies edges from a set of candidate points. For example, as shown in  FIG. 8A , candidate point extraction section  23  specifies edges  11  to  14  on the outer circumference of marker  60 . In the example in  FIG. 8A , the oblique sides of two triangles making up marker  60  are specified as edges  11  to  14 , respectively.  FIG. 8B  is an image illustrating edge  11 .  FIG. 8C  is an image illustrating edge  12 .  FIG. 8D  is an image illustrating edge  13 .  FIG. 8E  is an image illustrating edge  14 . 
         [0070]    Returning to  FIG. 4 , polygon generation section  3  will be described. 
         [0071]    Polygon generation section  3  estimates a plurality of straight lines based on the plurality of edges detected in edge detection section  2 . Polygon generation section  3  then generates a virtual polygon region surrounded by a plurality of straight lines estimated from the camera image. This virtual polygon region is generated in a region including a region outside the marker settable range, that is, a region including both the marker settable range and a region outside the marker settable range or only a region outside the marker settable range. 
         [0072]    Polygon generation section  3  includes straight line estimation section  31  and vertex specification section  32  as shown in  FIG. 9 . 
         [0073]    Straight line estimation section  31  estimates a plurality of straight lines based on the plurality of edges (e.g., edges  11  to  14 ) detected in edge detection section  2 . More specifically, straight line estimation section  31  uses Hough transform, which is a common parameter estimation technique based on information on a two-dimensional coordinate sequence of edges, and thereby estimates straight line parameters.  FIGS. 10A to 10D  show examples of the respective estimated straight lines.  FIG. 10A  is an image of straight line  11 ′ estimated based on edge  11 .  FIG. 10B  is an image of straight line  12 ′ estimated based on edge  12 .  FIG. 10C  is an image of straight line  13 ′ estimated based on edge  13 .  FIG. 10D  is an image of straight line  14 ′ estimated based on edge  14 . Thus, straight lines  11 ′ to  14 ′ are lines obtained by extending edge  11  to  14  respectively. 
         [0074]    Vertex specification section  32  specifies intersection points between the straight lines estimated in straight line estimation section  31 . The intersection points between the straight lines are obtained by solving simultaneous equations derived from an equation expressing each straight line. Note that an intersection point of which straight line with which straight line is obtained is predetermined.  FIG. 10E  shows examples of the respective specified intersection points. In  FIG. 10E , intersection point A between straight line  11 ′ and straight line  12 ′, intersection point B between straight line  12 ′ and straight line  13 ′, intersection point C between straight line  13 ′ and straight line  14 ′ and intersection point D between straight line  14 ′ and straight line  11 ′ are specified respectively. Intersection positions A, B, C and D specified in this way constitute vertexes making up a virtual polygon region. Therefore, intersection points A, B, C and D will be referred to as vertexes A, B, C and D respectively. Note that in the present embodiment, the virtual polygon region generated assumes a square shape as an example. 
         [0075]    In this way, straight lines are specified by straight line estimation section  31  and vertexes are specified by vertex specification section  32 , and the polygon region surrounded by these straight lines and vertexes is generated as a virtual polygon region. Note that a specific example of the range in which this virtual polygon region is generated will be described later using  FIG. 14 . 
         [0076]    Returning to  FIG. 4 , camera parameter calculation section  4  will be described. 
         [0077]    Camera parameter calculation section  4  calculates camera parameters (e.g., roll angle, pitch angle and height of camera  20 ) based on the feature values in the image of the virtual polygon region generated in polygon generation section  3  and the feature values of the virtual polygon region in the real space. The definitions of the “feature value in an image” and the “feature value in a space” and examples of “calculation of camera parameters” referred to here are the same as described in Embodiment 1. 
         [0078]    As shown in  FIG. 11 , camera parameter calculation section  4  includes roll/pitch calculation section  41  that calculates a roll angle and a pitch angle, and height calculation section  42  that calculates a height. 
         [0079]    As shown in  FIG. 11 , roll/pitch calculation section  41  includes evaluation value calculation section  43 , temporary parameter setting section  44  and angle specification section  45 . 
         [0080]    Evaluation value calculation section  43  calculates an evaluation value indicating the degree of shape matching between the polygon in an image and the polygon in a real space based on information indicating the lengths of four sides of the virtual polygon region generated in polygon generation section  3 . When the virtual polygon region is a square, the evaluation value is a value indicating square-likeness of the polygon in the image. The present embodiment assumes that standard deviations of lengths of the four sides of the polygon are used as evaluation values. More specifically, when the lengths of the four sides of the polygon are assumed to be L 1  to L 4  and an average value of the lengths of the four sides is assumed to be L m , evaluation value E is calculated from following equation 1. 
         [0000]      (Equation 1) 
         [0000]        E =√{square root over (⅓Σ k=1   4 ( L   k   −L   m ) 2 )}  [1]
 
         [0081]    The greater the similarity of the virtual polygon region to a square, the smaller the evaluation value, that is, the value of standard deviation of lengths of the four sides of the polygon becomes.  FIGS. 12A and 12B  illustrate examples of a difference in square-likeness. When polygon region  70  in the image is compared with accurate square  80  in  FIG. 12A  and  FIG. 12B ,  FIG. 12A  shows the shape more similar to a square and the evaluation value which is the standard deviation of the lengths of the four sides of the polygon in  FIG. 12A  is also smaller. 
         [0082]    Note that although the standard deviation of the lengths of the sides making up a polygon is used above as an evaluation value, a standard deviation of angles at vertices of a polygon may be used or both the lengths of sides and the magnitudes of angles may be used and weights may be assigned to the respective values to determine an evaluation value. 
         [0083]    Temporary parameter setting section  44  sets temporary parameters of a roll angle and pitch angle, and outputs the temporary parameters to image distortion correction/viewpoint conversion section  5 . After that, image distortion correction/viewpoint conversion section  5  executes distortion correction and viewpoint conversion based on the temporary parameters as described above. 
         [0084]    Furthermore, temporary parameter setting section  44  describes a pair of the evaluation value calculated in evaluation value calculation section  43  and the temporary parameter used to calculate the evaluation value in an evaluation value list. Temporary parameter setting section  44  updates the temporary parameter values of the roll angle and pitch angle in image distortion correction/viewpoint conversion section  5 . The temporary parameter values may be updated by trying all combinations of predetermined values or temporary parameters of the next roll angle and pitch angle may be set based on tendencies of the evaluation value calculated this time and the evaluation values calculated up to this point. Upon determining that contents of the evaluation value list are sufficient to specify the roll angle and pitch angle, temporary parameter setting section  44  asks angle specification section  45  to perform the processing. 
         [0085]    Angle specification section  45  specifies the roll angle and pitch angle based on the evaluation value list stored in temporary parameter setting section  44 . As an example of the method of specifying angles, a roll angle (e.g., 10 degrees) and pitch angle (e.g., 80 degrees) corresponding to a case where the evaluation value becomes a minimum (e.g., 0.96) may be specified as solutions as shown in  FIG. 13A , for example. Alternatively, when more accurate values need to be specified, a more exact roll angle (e.g., 9.96°) may be specified by fitting a straight line from the relationship between the pitch angle (e.g., 80°) and evaluation value as shown in  FIG. 13B , for example. 
         [0086]    Angle specification section  45  outputs the roll angle and pitch angle which are the specified camera parameters to image distortion correction/viewpoint conversion section  5  and asks it to perform distortion correction and viewpoint conversion. After that, image distortion correction/viewpoint conversion section  5  performs the distortion correction and viewpoint conversion based on the camera parameters. 
         [0087]    Height calculation section  42  calculates a height based on the information indicating the lengths of the four sides of the virtual polygon region generated in polygon generation section  3 . For example, when the polygon region is a square, for example, height calculation section  42  calculates a typical value of the lengths of the sides of the square from an average value of the lengths of the four sides of the square. Height calculation section  42  inputs the typical values of the lengths of the sides and calculates the corresponding height based on a graph shown in  FIG. 14  indicating the relationship between the length of each side and the height. For example, when the length of each side of the square in the camera image has  125  pixels, a height of camera of  120  cm is obtained by looking up the graph in  FIG. 14 . Note that, suppose that the graph shown in  FIG. 14  is stored beforehand, for example, in height calculation section  42 . 
         [0088]    As described above, camera parameter calculation section  4  calculates the roll angle, pitch angle and height as camera parameters of camera  20 . 
         [0089]    The effects in the present embodiment will be described below. 
         [0090]    First, a relationship between a marker and calibration accuracy will be described. To improve the accuracy of calibration, straight lines making up a polygon need to be estimated accurately. As described above, the accuracy of estimation of straight lines by straight line estimation section  31  depends on a set of candidate points which are candidates of points making up the straight lines. Thus, the wider the range of the camera image over which a set of candidate points are distributed (in other words, the greater the length of the edge), the higher the accuracy of estimation of the straight lines performed based on the set of candidate points becomes. 
         [0091]    The related art (e.g., technique in PTL 2) and the present embodiment match in that both use a polygon for calibration and calculate camera parameters based on the polygon. 
         [0092]    On the other hand, the related art is different from the present embodiment in that while the related art needs to detect a marker itself as a polygon, the present embodiment has no such need. That is, according to the related art, the size and shape of the marker to be set need to be identical to the size and shape detected. Thus, in the related art, for example, when the marker settable range is limited to only region b as shown in  FIG. 15A , marker  61  needs to be set so that the polygon (e.g., square) does not stick out of region b. For this reason, the size of marker  61  is also restricted and length L 1  of one side of the square also becomes shorter. 
         [0093]    In contrast, in the case of the present embodiment, straight lines are estimated based on edges detected from markers and a virtual polygon is generated from straight lines thereof. That is, in the present embodiment, the size and shape of markers to be set need not be identical to the size and shape of markers detected. Thus, by adjusting the shape and setting positions of the markers, it is possible to generate a virtual polygon region that sticks out of the marker settable range. For example, in  FIG. 15B , in the same way as in  FIG. 15A , even when the marker settable range is limited to only region b, by setting marker  60  having a shape in which two triangles are arranged side by side in region b, it is possible to generate virtual polygon region  70  defined by an extension of each hypotenuse of the two triangles at a position other than region b. Although the shape of virtual polygon region  70  is a square, length L 1  of one side thereof is twice length L 1  of one side of square marker  61  shown in  FIG. 15A . Thus, according to the present embodiment, even when the marker settable range is limited, it is possible to virtually obtain a greater polygon region than the polygon region that can be obtained in the related art and thereby keep the calibration accuracy. 
         [0094]    Note that although Embodiments 1 and 2 of the present invention have described an example of detecting edges of markers and estimating straight lines based thereon, feature points of the markers may also be detected and straight lines may be estimated based thereon. 
         [0095]    The markers and the virtual polygon region in Embodiments 1 and 2 of the present invention are not limited to those described above, but can be modified in various ways. Hereinafter, the variations will be described. 
         [0096]    An example has been described in above Embodiments 1 and 2 in which marker  60  is set in only one region b (marker settable range) located on one side of vehicle  100  as shown in  FIG. 2 , but the number of set markers and the setting position are not limited to them. For example, as shown in  FIG. 16 , markers  60   a  and  60   b,  and markers  60   c  and  60   d  may be set in region b on one side and the other side of vehicle  100 . Note that markers  60   a,    60   b,    60   c  and  60   d  are the same as marker  60  shown in  FIG. 2  and  FIG. 15B . 
         [0097]    Virtual polygon region  70  generated in the case of the marker setting shown in  FIG. 16  is defined by two straight lines estimated based on marker  60   a  and marker  60   c,  and two straight lines estimated based on marker  60   b  and marker  60   d,  and placed in region a which is outside the marker settable range. Virtual polygon region  70  is a square. 
         [0098]    In the examples shown in  FIG. 16  and  FIG. 17 , the same straight line is estimated using two markers (e.g., marker  60   a  and marker  60   c ) set at separate positions, and therefore the accuracy of estimating straight lines is improved compared to the cases in  FIG. 2  and  FIG. 15B . As a result, the accuracy of calibration also improves. Note that regarding the two markers used to estimate the same straight line, inclinations of edges detected from the markers need to be the same. 
         [0099]      FIG. 18  is a figure illustrating a case where parallelogram markers  62   a  to  62   d  are set. In  FIG. 18 , markers  62   a  and  62   b  are set in one region b located on one side of vehicle  100  and markers  62   c  and  62   d  are set on other region b. In this case, as in the case of  FIG. 17 , two straight lines are estimated by marker  62   a  and marker  62   c,  and two straight lines are estimated by marker  62   b  and marker  62   d.  Virtual polygon region  70  is then generated by the estimated straight lines in region a, which is outside the marker settable range. Virtual polygon region  70  is a square. 
         [0100]    When parallelogram markers  62   a  to  62   d  shown in  FIG. 18  are set, it is possible to reduce the influence of displacement in a positional relationship between the triangles compared to the case with marker  60  where the triangles are arranged side by side, and the accuracy of calibration further improves. 
         [0101]      FIG. 19A  is a figure illustrating a case where rectilinear (elongated parallelogram) markers  63   a  to  63   j  are provided. In  FIG. 19A , markers  63   a  to  63   e  are set in one region b on one side of vehicle  100  and markers  63   f  to  63   j  are set in other region b. In this case, one straight line is estimated by marker  63   a  and marker  63   h,  and one straight line is estimated by marker  63   b  and marker  63   j.  Furthermore, one straight line is estimated by marker  63   c  and marker  63   f,  one straight line is estimated by marker  63   d  and marker  63   i,  and one straight line is estimated by marker  63   e  and marker  63   g.  Virtual polygon region  71  is then generated by the estimated straight lines in region a, which is outside the marker settable range. Virtual polygon region  71  is a pentagon. 
         [0102]    In the case of  FIG. 19A , markers may be colored to make it clear which marker edges are to be connected to each other. For example, markers are used in colors differing from one set of markers to another for connecting edges, for example, marker  63   a  and marker  63   h  being colored in yellow, and marker  63   b  and marker  63   j  being colored in blue. It is thereby possible to specify sides of a virtual polygon, too. For example, it is possible to specify side L 11  included in the straight line estimated by markers  63   a  and  63   h,  and side L 12  included in the straight line estimated by markers  63   b  and  63   j.    
         [0103]    When the setting position of the vehicle is always the same, the marker position in the camera image is always the same. For this reason, the virtual polygon generated by straight lines specified based on marker edges in the camera image is also always located at the same position. However, when the vehicle is set at a large angle relative to the virtual polygon, the virtual polygon in the camera image rotates significantly. When the degree of inclination of the vehicle setting is unclear, the association between sides of the virtual polygon and sides of the polygon in the real space is also unclear. As described above, use of markers in colors which differ from one set of markers to another for connecting edges facilitates association between sides of the virtual polygon and sides of the polygon in the real space. 
         [0104]    When a pentagon is generated as the virtual polygon region, camera parameter calculation section  4  uses the length of one specific side of the virtual polygon in the image and the length in the real space or a ratio in lengths of sides, and can thereby calculate camera parameters through a technique of calculating the evaluation values described in Embodiment 2. 
         [0105]      FIG. 19B  is a figure illustrating a case where trapezoidal markers  64   a  to  64   f  are set instead of rectilinear (elongated parallelogram) markers  63   a  to  63   j  to form the same pentagon as that in  FIG. 19A . In  FIG. 19B , for example, marker  64   a  is a trapezoid that can detect the same edges as those of marker  63   a  and marker  63   b  shown in  FIG. 19A . Two edges detected from marker  64   a  are connected to one edge detected from marker  64   e  and one edge detected from marker  64   f.  Two straight lines L 11  and L 12  are estimated in this way. Other straight lines are also estimated as with  FIG. 19A . A pentagon is generated by the straight lines as virtual polygon region  71  and placed in region a, which is outside the marker settable range. 
         [0106]    As with  FIG. 19B ,  FIG. 20  is a figure illustrating a case where trapezoidal markers  65   a  to  65   d  are set. In  FIG. 20 , a plurality of straight lines are estimated based on edges of trapezoidal markers  65   a  to  65   d  as with  FIG. 19 , whereas virtual polygon region  72  formed in region a is a triangle. 
         [0107]      FIG. 21A  is a figure illustrating a case where circular markers  66   a  to  66   h  are set. In this example, straight line estimation section  31  specifies straight lines by connecting centers of gravity of two detected markers in a camera image. For example, one straight line is estimated by marker  66   a  and marker  66   g,  and one straight line is estimated by marker  66   b  and marker  66   h . Furthermore, one straight line is estimated by marker  66   c  and marker  66   e,  and one straight line is estimated by marker  66   d  and marker  66   f  Virtual polygon region  70  is generated by the estimated straight lines in region a, which is outside the marker settable range. Virtual polygon region  70  is a square. 
         [0108]      FIG. 21B  is a figure illustrating a case where parallelogram markers  67   a,    67   d  and  67   e , and trapezoidal markers  67   b,    67   c  and  67   f  are set. In this case, polygon generation section  3  extracts specific points c 2  on sides in addition to vertexes c 1  of a virtual polygon region. For example, polygon generation section  3  estimates straight lines based on edges detected by edge detection section  2  and extracts intersection points of the straight lines, that is, vertexes c 1  and points c 2 . Camera parameter calculation section  4  calculates camera parameters based on positions of vertexes c 1  and points c 2  in the camera image and positions in the real space. 
         [0109]    Note that although an example has been described in above  FIG. 17  to  FIG. 21  in which the respective markers are set in both regions b on both sides of vehicle  100 , the markers may also be set in only one region b as with  FIG. 2B  and  FIG. 15B . 
         [0110]    Although Embodiments 1 and 2 of the present invention, and variations thereof have been described so far, the present invention is not limited to the above description, but can be modified without departing from the spirit and scope of the present invention. 
         [0111]    The disclosure of Japanese Patent Application No. 2013-164965, filled on Aug. 8, 2013, including the specification, drawings and abstract, is incorporated herein by reference in its entirety. 
       INDUSTRIAL APPLICABILITY 
       [0112]    The present invention is useful for techniques of calculating camera parameters (e.g., an apparatus, system, method and program or the like). 
       REFERENCE SIGNS LIST 
       [0113]      1  Image acquiring section 
         [0114]      2  Edge detection section 
         [0115]      3  Polygon generation section 
         [0116]      4  Camera parameter calculation section 
         [0117]      5  Image distortion correction/viewpoint conversion section 
         [0118]      10  Image processing apparatus 
         [0119]      20  Camera 
         [0120]      21  Marker region specification section 
         [0121]      22  Filtering section 
         [0122]      23  Candidate point extraction section 
         [0123]      31  Straight line estimation section 
         [0124]      32  Vertex specification section 
         [0125]      41  Roll/pitch calculation section 
         [0126]      42  Height calculation section 
         [0127]      43  Evaluation value calculation section 
         [0128]      44  Temporary parameter setting section 
         [0129]      45  Angle specification section 
         [0130]      100  Vehicle