Patent Publication Number: US-11037327-B2

Title: Camera control method, camera control device, and non-transitory computer-readable storage medium

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2017-181987, filed on Sep. 22, 2017, the entire contents of which are incorporated herein by reference. 
     FIELD 
     The embodiments discussed herein are related to a camera control method, a camera control device, and a non-transitory computer-readable storage medium. 
     BACKGROUND 
     There is a technique to identify the position of a subject (object) in real space from an image which includes the subject and is captured by a camera. When a correspondence relationship between the coordinates on an image and the position (coordinates) in real space is identified, it is possible to determine the position of an object in real space when an image is captured based on the image of the object captured by a camera. 
     In order to identify the correspondence relationship between the coordinates on the image and the coordinates in real space, for instance, calibration (correction) of the camera which acquires an image is performed. 
     Here, calibration is the process by which an object with known size and position (coordinates) in real space is photographed by a camera, the coordinates of the object on an acquired image are identified, and thus parameters for converting two-dimensional coordinates on the image to three-dimensional coordinates in real space (or the other way around) are acquired. In this manner, information on the characteristics of the camera is acquired. 
     For instance, an object (hereinafter referred to as a calibration pattern) having a shape extractable from an image by image processing or the like is photographed by camera fixed to a predetermined position in real space. In this process, when the positions (coordinates) of each of the camera and the object in real space are known, and the coordinates of the photographed object on the image are identified by image processing or the like, it is possible to identify the correspondence relationship between the coordinates on the image and the coordinates in real space. 
     Related techniques are disclosed in, for example, Japanese Laid-open Patent Publication No. 2011-101265. 
     SUMMARY 
     According to an aspect of the invention, a camera control method executed by a computer, the camera control method including performing first detection processing that detects a plurality of predetermined patterns at different positions in real space from one or more images captured by a first camera, acquiring camera characteristic information on the first camera based on one or more first predetermined patterns, included in the plurality of predetermined patterns, successfully detected in the first detection processing, correcting the one or plurality of images based on the acquired camera characteristic information, performing second detection processing to detect one or more second predetermined patterns, included in the plurality of predetermined patterns, failed to be detected in the first detection processing, the one or plurality of second predetermined patterns being included in the one or more images on which the correcting has been performed, and updating the camera characteristic information based on the detected one or more second predetermined patterns. 
     The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a schematic diagram of an embodiment according to this example; 
         FIG. 2  is a functional block diagram illustrating the functional configuration of an information processing device; 
         FIG. 3  illustrates an example object captured using a camera; 
         FIG. 4  is flowchart illustrating the flow of a series of processing of calibration processing (first stage); 
         FIG. 5  illustrates example data stored in a management information storage unit; 
         FIG. 6  illustrates example data stored in the management information storage unit when a series of processing illustrated in  FIG. 4  is performed; 
         FIG. 7A  illustrates an example image in which a flat plate is captured in its entirety near the center of the image; 
         FIG. 7B  illustrates an example image in which the flat plate is captured in its entirety near the outer edge of the image; 
         FIG. 8  illustrates an example in which the flat plate is captured within a field of view by both the camera and another camera; 
         FIG. 9A  illustrates a flowchart (part one) depicting the flow of a series of processing of the calibration processing (second stage); 
         FIG. 9B  illustrates a flowchart (part two) depicting the flow of a series of processing of the calibration processing (second stage); 
         FIG. 10  illustrates an example corrected image corresponding to the image of  FIG. 7B ; 
         FIG. 11  illustrates example data stored in the management information storage unit when a series of processing illustrated in  FIGS. 9A and 9B  is performed; 
         FIG. 12  illustrates a modification of equator setting in an equidistant cylindrical image; 
         FIG. 13A  illustrates an example azimuthal equidistant chart generated based on the equator setting described with reference to  FIG. 12 ; 
         FIG. 13B  illustrates an example in which when an azimuthal equidistant chart is generated, the flat plate is divided on an image; and 
         FIG. 14  illustrates an example hardware configuration of the information processing device in this example. 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     Here, a camera mounted with a lens having a wide range of viewing angle (for instance, a lens called a fisheye lens) may be used. 
     Although such a camera has an advantage in that a wide region in real space is covered by a single image, in an area with a larger incident angle to the lens (an area near the outer side of an image), more distortion occurs in the image. When a calibration pattern is present in an area with a larger incident angle, the shape of the calibration pattern in the image is deformed with respect to the real calibration pattern. For this reason, the calibration pattern may not be recognized by image processing, in other words, the coordinates of the captured calibration pattern on the image may not be identified. 
     In an area in which the calibration pattern is not recognizable, it is not possible to perform calibration, or even if it is possible, the accuracy of acquired information (parameters) is significantly reduced. 
     In an aspect, it is aimed to improve the accuracy of acquired camera characteristic information in consideration of distortion of an image. 
     An example of the present disclosure will be described with reference to the drawings. 
     [Overview] 
       FIG. 1  is a schematic diagram of an embodiment according to the example.  FIG. 1  illustrates an information processing device  1 , and cameras  2 - 1 ,  2 - 2 . 
     The cameras  2 - 1  and  2 - 2  are imaging devices on which fisheye lens  21 - 1  and  21 - 2  are mounted, respectively. When the viewing angle of each of the cameras  2 - 1 ,  2 - 2  is greater than or equal to 180 degrees, it is possible to obtain a 360 degree image (omnidirectional image), for instance, by installing the cameras  2 - 1 ,  2 - 2  so that the fisheye lens  21 - 1  and fisheye lens  21 - 2  face in opposite directions as illustrated in  FIG. 1 . 
     The information processing device  1  is a computer connected to the cameras  2 - 1 ,  2 - 2  via a wire or wirelessly to allow communication, and performs control of image acquisition using the cameras  2 - 1 ,  2 - 2 . The information processing device  1  is an information processing device such as a personal computer (PC), a smartphone, a personal digital assistant (PDA), for instance. 
     The various types of processing described in this example may be performed by an integrated device having the functions of the information processing device  1  and the cameras  2 - 1 ,  2 - 2 . 
     [Functional Configuration of Information Processing Device  1 ] 
       FIG. 2  is a functional block diagram illustrating the functional configuration of the information processing device  1 . The information processing device  1  includes, for instance, a communication unit  101 , an input unit  102 , an output unit  103 , a processing unit  110 , and a storage unit  120 . 
     The communication unit  101  is capable of communicating with other devices including the cameras  2 - 1 ,  2 - 2  via a wire or wirelessly. The communication unit  101  is, for instance, a communication device such as a network adaptor or a network interface controller (NIC) included in the information processing device  1 . Alternatively, the communication unit  101  is a communication port for wired connection such as Universal Serial Bus (USB), or a transmitting and receiving device for wireless communication. 
     The input unit  102  receives information input or an operation from a user. The input unit  102  is, for instance, a keyboard, a mouse, or a touchpad. 
     The output unit  103  is capable of outputting, for instance, an image acquired by the cameras  2 - 1 ,  2 - 2 , a result of image processing using the acquired image, and a result of the later-described calibration. The output unit  103  is, for instance, a liquid crystal display as hardware. 
     The processing unit  110  includes a photographic processing unit  111 , a detection unit  112 , a determination unit  113 , a calibration execution unit  114 , and a correction processing unit  115 . 
     The photographic processing unit  111  controls photographing of an image. For instance, when an omnidirectional image is captured using the cameras  2 - 1 ,  2 - 2 , the photographic processing unit  111  performs control to synchronize the timing of photographing by the cameras  2 - 1 ,  2 - 2 . 
     The detection unit  112  detects feature points (the details will be described later) used for processing of calibration, from an image acquired by the cameras  2 - 1 ,  2 - 2 . 
     The determination unit  113  makes determination on a result of detection of feature points made by the detection unit  112 . 
     The calibration execution unit  114  performs camera calibration on at least one of cameras  2 - 1 ,  2 - 2  based on the feature points detected by the detection unit  112 . As a result of calibration, the calibration execution unit  114  acquires camera parameters (the later-described internal parameter and external parameter of the camera). The acquisition unit in the present application corresponds to the calibration execution unit  114 , for instance. 
     The correction processing unit  115  performs correction processing on the acquired image using the camera  2 - 1  or camera  2 - 2 . 
     The operation of each functional unit described above will be described in detail later along with the flow of a series of processing in this example. 
     The storage unit  120  includes a camera image storage unit  121 , a management information storage unit  122 , a correction image storage unit  123 , and a parametric information storage unit  124 . 
     The camera image storage unit  121  stores the data of an image captured using the cameras  2 - 1 ,  2 - 2 . 
     The management information storage unit  122  stores information on an image captured using the cameras  2 - 1 ,  2 - 2 . 
     The correction image storage unit  123  stores a camera image corrected by the correction processing unit  115 . 
     The parametric information storage unit  124  stores camera parameters acquired as a result of calibration performed by the calibration execution unit  114 . Each camera parameter is information that indicates the characteristics of a camera, and includes, for instance, an internal parameter and an external parameter of the camera. 
     It is to be noted that the internal parameter is a parameter related to conversion of coordinates between the coordinates in real space and the coordinates on a camera image. In addition, the internal parameter is a parameter that indicates a focal length, a deviation of an image center, distortion of a lens and other factors. 
     Also, the external parameter is a parameter that indicates the position and arrangement of a camera in real space. The external parameter is represented, for instance, as a relative position (rotated, translated around or from an origin) of a camera with respect to a predetermined origin in real space. 
     The information stored in each storage unit described above will be described in detail later along with the flow of a series of processing in this example. 
     [Acquisition of Image] 
     First, acquisition (photographing) of an image used for calibration in this example will be described. Although the camera  2 - 1  (one camera) will be first described as an example, similar processing may also be performed when the camera  2 - 2  is used. 
     In calibration, an image of an object captured by the camera  2 - 1  fixed to a specific position in real space is acquired, the object having known size and position in real space. 
       FIG. 3  is a pattern illustrating an example object to be photographed using the camera  2 - 1 . An object photographed using the camera  2 - 1  may be, for instance, a flat plate  300  with a checkered pattern as illustrated in  FIG. 3 . It is to be noted that the checkered pattern of the flat plate  300  may be referred to as the “lattice pattern” in the following description. A feature point used for calibration is, for instance, a point of contact (for instance, a point  310  of  FIG. 3 ) between squares included in the checkered pattern. When each point of contact between squares included in the checkered pattern serves as a feature point, the flat plate  300  of  FIG. 3  contains a total of 95 (vertically 5×horizontally 19) feature points including the point  310 . 
     When an image for the photographic processing unit is captured with the flat plate  300  installed within an imaging range of the camera  2 - 1  in real space, and a camera image including feature points used for calibration is acquired. In this process, the entire flat plate  300  has to be included in the imaging range of the camera  2 - 1 , and the position of the flat plate  300  in real space at the time of photography and the dimensions of each square (relative positional relationship between the feature points) in the lattice pattern have to be identified beforehand. Also, it is desirable that multiple camera images be captured with different positions of the flat plate  300  in consideration of improvement of the accuracy of calibration. 
     The flat plate  300  is only an example of an object. As long as an object has a known size and feature points are detectable from a camera image by image processing such as pattern matching, the object is not uniquely limited to the aspect illustrated. 
     [Calibration Processing (First Stage)] 
     The processing of calibration in this example will be described. In the following description, it is assumed that multiple camera images used for calibration have been acquired beforehand based on the above-described aspect. Also, in the following, a description is given by once focusing attention on the images captured using the camera  2 - 1  and the camera  2 - 2 . 
       FIG. 4  is flowchart illustrating the flow of a series of processing of calibration processing (first stage). 
     First, the detection unit  112  selects one of unprocessed images from the images which have been captured using the camera  2 - 1  and stored in the camera image storage unit  121  (step S 401 ). Here, the unprocessed images refer to those images that have not undergone the later-described processing in step S 402 . 
       FIG. 5  is a table illustrating example data stored in the management information storage unit  122 . As illustrated in  FIG. 5 , the management information storage unit  122  stores, for instance, a photographic time  1221 , an image identifier (ID) (camera  2 - 1 )  1222 , resultant information (camera  2 - 1 )  1223 , an image ID (camera  2 - 2 )  1224 , resultant information camera ( 2 - 2 )  1225 , and real space position  1226  in association with each other. 
     The photographic time  1221  is information indicating a time when an image is captured.  FIG. 5  illustrates that images are captured by the two cameras  2 - 1 ,  2 - 2  at the same timing using synchronous control performed by the photographic processing unit  111 . This is because in the camera calibration for photographing an omnidirectional image, images of the object (the flat plate  300 ) at the same real space position have to be used. However, as long as images of the object at the same real space position are acquirable by the cameras  2 - 1 ,  2 - 2 , and it is possible to associate each real space position with an image, the photographic times of images do not have to agree. 
     The image ID (camera  2 - 1 )  1222  is identification information for uniquely identifying each of the images captured by the camera  2 - 1 . In  FIG. 5 , the values “1-0001” to “1-0010” are used for the sake of convenience. However, as long as each of the images is uniquely identifiable, specific values may be different from the values of  FIG. 5 . 
     The resultant information (camera  2 - 1 )  1223  is information indicating a result of detection of feature points made by the detection unit  112  for each of the images. In this example, the resultant information (camera  2 - 1 )  1223  may have three statuses (the details will be described later): unprocessed, successful, and failed. For instance, in  FIG. 5 , the status of unprocessed is denoted as “unprocessed”, which indicates an image that has not undergone the later-described processing in step S 402 . However, when the statuses are distinguishable, the data format is not limited to the format of  FIG. 5 , and may be a numerical format or a bit format, for instance. 
     The image ID (camera  2 - 2 )  1224  is identification information for uniquely identifying each of the images captured by the camera  2 - 2 . In  FIG. 5 , the values “2-0001” to “2-0010” are used for the sake of convenience. However, as long as each of the images is uniquely identifiable, specific values may be different from the values of  FIG. 5 . 
     Similarly to the resultant information (camera  2 - 1 )  1223 , the resultant information (camera  2 - 2 )  1225  is information indicating a result of detection of feature points for each of the images acquired by the camera  2 - 2 . 
     The real space position  1226  provides, for instance, real space coordinates that indicate the position of the object (flat plate  300 ) at the photographic time point of each of the images. Here, the real space coordinates may indicates for instance, three-dimensional coordinates with the origin at a predetermined point in real space, and the arrangement or orientation of the flat plate  300 . However, setting of coordinates does not have to be uniquely limited, and may be any setting as long as the real space position of each feature point along with the dimensions of the lattice pattern are calculable. 
     The data illustrated in  FIG. 5  allows the information processing device  1  to perform control of the progress of the calibration processing in this example, and to grasp a progressing state and a result of the calibration processing. 
     Returning to the description of  FIG. 4 , the processing in step S 401  may be described as processing to select one image with the value of the resultant information (camera  2 - 1 )  1223  of “unprocessed” by the detection unit  112 . 
     Subsequently, the detection unit  112  detects feature points corresponding to the lattice pattern from the image selected in step S 401  (step S 402 ). Although the detection of feature points is feasible, for instance, by image processing such as pattern matching based on the shape of the lattice pattern, a specific technique does not have to be limited to the pattern matching. 
     The determination unit  113  determines a result of detection of feature points performed in step S 402  (step S 403 ). For instance, when the number of feature points detected in step S 402  is the same as the number of points (95 for the flat plate  300  of  FIG. 3 ) detectable based on the lattice pattern, the determination unit  113  determines that the detection has been successful. On the other hand, when the number of feature points detected in step S 402  is different from (smaller than) the number of points detectable based on the lattice pattern, the determination unit  113  determines that the detection has been failed. 
     When the detection is determined to be failed (NO in step S 403 ), the determination unit  113  records a result indicating detection failure for the image selected in step S 401 . Specifically, the determination unit  113  updates the value of the resultant information (camera  2 - 1 )  1223  stored in the management information storage unit  122  for the image selected in step S 401  to “failed” (step S 404 ). Consequently, for the image selected in step S 401 , a result indicating detection failure and a result indicating that the image selected in step S 401  has been processed (the processing in step S 402  has been performed) are stored in the management information storage unit  122 . 
     On the other hand, when the detection is determined to be successful (YES in step S 403 ), the determination unit  113  records a result indicating detection success for the image selected in step S 401 . Specifically, the determination unit  113  updates the value of the resultant information (camera  2 - 1 )  1223  stored in the management information storage unit  122  for the image selected in step S 401  to “successful” (step S 405 ). Consequently, for the image selected in step S 401 , a result indicating detection success and a result indicating that the image selected in step S 401  has been processed (the processing in step S 402  has been performed) are stored in the management information storage unit  122 . 
     After the processing in step S 404  or S 405  is performed, the detection unit  112  determines whether or not all target images have been processed in step S 401  (step S 406 ). When an unprocessed image is present (the value of “unprocessed” is present in the data column of the resultant information (camera  2 - 1 )  1223 ) (NO in step S 406 ), the flow returns to step S 401 , and the information processing device  1  selects one unprocessed image, and repeatedly performs a series of processing described above. 
     On the other hand, when all target images have been processed (YES in step S 406 ), the calibration execution unit  114  performs calibration of the camera  2 - 1  using the images (the images having the value of the resultant information (camera  2 - 1 )  1223  of “successful”) with feature points successfully detected (step S 407 ). The internal parameter and the external parameter for the camera  2 - 1  are acquired by performing the calibration. The calibration execution unit  114  stores the internal parameter and the external parameter acquired by the calibration into the parametric information storage unit  124 . In the camera calibration, when the accuracy of derived camera parameters is taken into consideration, it is desirable that multiple images with successful detection of feature points be present (in other words multiple patterns of images usable for calibration be present). Thus, before execution of step S 407 , the calibration execution unit  114  may determine whether or not the number of images with successful detection of feature points is greater than or equal to a predetermined number (for instance, three), and when the number of images with successful detection of feature points is greater than or equal to a predetermined number, the calibration execution unit  114  may perform calibration. 
     After step S 407  is performed, a series of processing illustrated in  FIG. 4  is completed. 
     [Problem in Calibration Processing (First Stage)] 
       FIG. 6  is a table illustrating example data stored in the management information storage unit  122  when a series of processing (at least up to the stage in which determination of YES is made in step S 406 ) illustrated in  FIG. 4  is performed. In  FIG. 6 , a series of processing illustrated in  FIG. 4  has been performed for the camera  2 - 1  in the same manner as the above-described processing for the camera  2 - 1 . 
     For instance, when attention is focused on the resultant information (camera  2 - 1 )  1223  of  FIG. 6 , detection of feature points is successful for three images out of 10 images. Since the images with feature points successfully detected have been acquired, minimum conditions for acquiring camera parameters by calibration for the camera  2 - 1  are satisfied. 
     However, from the viewpoint of improving the accuracy of acquired camera parameters, when feature points are detected at more, various positions in real space, improvement of the accuracy is expected. Thus, the greater number of images with successful detection of feature points probably increases the accuracy of the camera parameters. However, in the example of  FIG. 6 , detection of feature points is successful for 3 images but is failed for 7 images, thus there is a possibility that the accuracy of the camera parameters may not be sufficiently ensured. 
     At least two reasons for failure of detection of feature points in an image may be considered. 
     The first reason is that, first of all, the flat plate  300  itself is not captured in an image. For instance, when an omnidirectional image is captured using the cameras  2 - 1 ,  2 - 2 , if the flat plate  300  is captured near the center of the field of view of one of the cameras in calibration, the flat plate  300  may be outside the field of view of the other camera. However, in this case, failure is due to the fact that the flat plate  300  is outside the field of view of the other camera, and if detection of feature points is successful in the one image of the camera which has captured the flat plate  300  in the field of view, no problem arises. For instance, for the camera  2 - 1 , failure of detection of feature points at photographic times T 3 , T 6 , and T 10  in  FIG. 6  is probably due to the first reason. 
     The second reason is that although the flat plate  300  is captured in an image in its entirety, detection of feature points is not possible. 
       FIGS. 7A and 7B  are each an example image in which the flat plate  300  is captured in its entirety in the image. An image captured by a camera having a lens with a wide viewing angle like a fisheye lens has such characteristics that in an area with a larger incident angle to the lens (an area near the outer side of an image), more distortion occurs in the image. Due to the characteristics, the degree of deformation (distortion) of the flat plate  300  in a captured image significantly varies according to the incident angle to the fisheye lens  21 - 1  in the area in which the flat plate  300  is installed in real space, in other words, the installation position of the flat plate  300  in real space. 
     In the image of  FIG. 7A , the position of the flat plate  300  in the image is closer to the central area of the image than the image of  FIG. 7B , that is, the incident angle to the fisheye lens  21 - 1  is smaller. In this case, the distortion of the flat plate  300  in the image is relatively small, and it is probable that feature points are successfully detected by image processing such as pattern matching. In contrast, in the image of  FIG. 7B , in which the position of the flat plate  300  in the image is closer to the outer side than the image of  FIG. 7A , that is, the incident angle to the fisheye lens  21 - 1  is larger, the shape of the flat plate  300  in the image is significantly deformed (highly distorted). In this case, the shape of the flat plate  300  is significantly different from the actual flat plate  300 , and thus detection of feature points is difficult. 
     Therefore, in an image in which the position of the flat plate  300  is located at an area on the outer side of the image, detection of feature points is failed, and as a consequence, a situation occurs in which calibration particularly for an area on the outer side of the image is not sufficiently performed. Quite simply, the accuracy of the camera parameter acquired by calibration is reduced. 
     Particularly when an omnidirectional image is captured using the cameras  2 - 1 ,  2 - 2 , failure of detection of feature points due to the second reason causes a problem. This is because when the flat plate  300  is attempted to be captured in the field of view by the two cameras, as illustrated in  FIG. 8 , the flat plate  300  is located at an area on the outer side of the image in each of the two cameras. In this case, even though the flat plate  300  is included in the camera image of each of the both cameras  2 - 1 ,  2 - 2 , it is probable that detection of feature points is failed due to the distortion of the image. For instance, in the camera  2 - 1  (and the camera  2 - 2 ), failure of detection of feature points at photographic times T 2 , T 4 , T 7 , and T 9  in  FIG. 6  is probably due to the first reason. 
     In consideration of the above-mentioned problem, the later-described calibration processing (second stage) is performed in this example. 
     [Calibration Processing (Second Stage)] 
       FIGS. 9A and 9B  are each a flowchart depicting the flow of a series of processing of the calibration processing (second stage). In the following, similarly to the description of the calibration processing (first stage), a description is given by once focusing attention on the images captured using the camera  2 - 1  and the camera  2 - 1 . 
     First, the detection unit  112  updates (changes) all data values of “failed” in the data column of the resultant information (camera  2 - 1 )  1223  to “unprocessed” (step S 901 ). It is to be noted that the processing in step S 901  is processing for distinguishing between the case (case  1 ) where the data value is “failed” because the later-described processing in step S 904  has not been performed and the case (case  2 ) where the data value is “failed” because a detection result in step S 904  indicates failure. As long as the case  1  and the case  2  become distinguishable, the processing in step S 901  may be substituted by processing different from the processing in this example. Alternatively, regardless of success or failure of detection of feature points in the calibration processing (first stage), the calibration processing (second stage) may be performed on all the images. In this case, in step S 901 , the detection unit  112  updates all data values in the data column of the resultant information (camera  2 - 1 )  1223  to “unprocessed”. 
     Subsequently, the detection unit  112  selects one of unprocessed images from the images which have been captured using the camera  2 - 1  and stored in the camera image storage unit  121  (step S 902 ). Here, the unprocessed images refer to those images that have not undergone the later-described processing in step S 904 . The processing in step S 902  may be described as processing to select one image with the value of the resultant information (camera  2 - 1 )  1223  of “unprocessed” by the detection unit  112 . 
     Subsequently, the correction processing unit  115  performs correction processing on the image selected in step S 902  (step S 903 ). The correction processing removes or reduces distortion in a selected image. The correction processing generates an equidistant cylindrical image using, for instance, the image selected in step S 902 , and the internal parameter of the camera  2 - 1  acquired in the calibration processing (first stage). It is to be noted that an image generated as a result of the correction processing, including an equidistant cylindrical image may be referred to as a “corrected image” in the following description. 
       FIG. 10  illustrates an example corrected image corresponding to the image of  FIG. 7B . For instance, the correction processing unit  115  generates an image (corrected image  1000 ) of an equidistant cylindrical image corresponding to the image of  FIG. 7B  so that a line  701  in  FIG. 7B  is the equator of the equidistant cylindrical image. The coordinates on the line  701  in  FIG. 7B  are located on a line  1001  (the equator in the corrected image  1000 ) of  FIG. 10 . It is to be noted that the shaded area in  FIG. 10  is an area that is outside the field of view of the image of  FIG. 7B  and is not captured in the image of  FIG. 7B . 
     Here, the line  701  in  FIG. 7B  passes through a central portion or its vicinity of the flat plate  300  of the image in  FIG. 7B . The correction processing unit  115  identifies a line, like the line  701 , which passes through the center of the flat plate  300  in the camera image, and sets the equator of an equidistant cylindrical image, thereby making it possible to generate an azimuthal equidistant chart in which the flat plate  300  located near the equator. Particularly an area near the equator in the azimuthal equidistant chart allows distortion of an image to be suppressed at a low level, and thus it is possible to further enhance the effect of the correction processing. 
     To determine the central position of the flat plate  300  in the camera image (before correction), the correction processing unit  115  may identify the approximate outline of the flat plate  300  in the camera image (before correction), for instance, by image processing, and may determine the center of the identified approximate outline to be the center position of the flat plate  300 . However, the center position determined in this process does not have to be the exact center of the flat plate  300 , and it is sufficient that an approximate center position be identified. This is because in order to suppress the distortion of the image of the flat plate  300  in the azimuthal equidistant chart at a low level, a sufficient effect of suppression of distortion may be expected when the position of the flat plate  300  is in the vicinity of the equator. 
     As illustrated in  FIG. 10 , in the flat plate  300  in the corrected image  1000 , the distortion of the image is reduced as compared with the flat plate  300  of the image in  FIG. 7B . Consequently, it is probable that a feature point which has not been detected in the image before the correction is detectable in the corrected image. Particularly, in the correction processing by generating an equidistant cylindrical image, it is expected that curved distortion of the flat plate  300  as illustrated in  FIG. 7B  is significantly reduced. In detection of feature points by pattern matching, curved distortion has a significant effect, whereas the detection is relatively unsusceptible to variation in aspect ratio and linear deformation using homography conversion. Therefore, possibility of detection of feature points is probably increased significantly. The correction processing unit  115  may store a generated corrected image in the correction image storage unit  123  for the subsequent processing, for instance. 
     Returning to the description of  FIG. 9A , after the correction processing in step S 903  is performed, the detection unit  112  detects feature points corresponding to the lattice pattern from the corrected image (step S 904 ). Although the detection of feature points is feasible, for instance, by image processing such as pattern matching based on the shape of the lattice pattern, a specific technique does not have to be limited to the pattern matching. 
     The determination unit  113  determines a result of detection of feature points performed in step S 904  (step S 905 ). For instance, when the number of feature points detected in step S 904  is the same as the number of points (95 for the flat plate  300  of  FIG. 3 ) detectable based on the lattice pattern, the determination unit  113  determines that the detection has been successful. On the other hand, when the number of feature points detected in step S 904  is different from (smaller than) the number of points detectable based on the lattice pattern, the determination unit  113  determines that the detection has been failed. 
     When the detection is determined to be failed (NO in step S 905 ), the determination unit  113  records a result indicating detection failure for the image selected in step S 902  (step S 906 ). Specifically, the determination unit  113  updates the value of the resultant information (camera  2 - 1 )  1223  stored in the management information storage unit  122  for the image selected in step S 902  to “failed”. Consequently, for the image selected in step S 902 , a result indicating detection failure and a result indicating that the image selected in step S 902  has been processed (the processing in step S 904  has been performed) are stored in the management information storage unit  122 . 
     On the other hand, when the detection is determined to be successful (YES in step S 905 ), the determination unit  113  records a result indicating detection success for the image selected in step S 902  (step S 907 ). Specifically, the determination unit  113  updates the value of the resultant information (camera  2 - 1 )  1223  stored in the management information storage unit  122  for the image selected in step S 902  to “successful”. Consequently, for the image selected in step S 902 , a result indicating detection success and a result indicating that the image selected in step S 902  has been processed (the processing in step S 904  has been performed) are stored in the management information storage unit  122 . 
     After the processing in step S 906  or S 907  is performed, the detection unit  112  determines whether or not all target images have been processed in step S 902  (step S 908 ). When an unprocessed image is present (the value of “unprocessed” is present in the data column of the resultant information (camera  2 - 1 )  1223 ) (NO in step S 908 ), the flow returns to step S 902 , and the information processing device  1  selects one unprocessed image, and repeatedly performs a series of processing described above. 
     On the other hand, when all target images have been processed (YES in step S 908 ), the detection unit  112  identifies the coordinates on an uncorrected image corresponding to the feature points in the corrected image, which have been successfully detected in step S 904 . (step S 909 , the flow proceeds from  FIG. 9A  to  FIG. 9B ). In the process of processing in step S 903 , the correspondence relationship between the coordinates on the uncorrected image and the coordinates on the corrected image is identified, and invertible transformation may be performed. Therefore, when the coordinates of a feature point is identifiable in the corrected image, the coordinates on the uncorrected image corresponding to the feature point is identifiable by processing corresponding to the inverse transformation of the processing in step S 904 . 
     After step S 909  is performed, the calibration execution unit  114  performs calibration of the camera  2 - 1  using the images (the images having the value of the resultant information (camera  2 - 1 )  1223  of “successful”) with feature points successfully detected (step S 910 ). In step S 910 , it is possible to perform camera calibration using both the images with feature points successfully detected in the calibration processing (first stage) and the images with feature points successfully detected in the calibration processing (second stage). 
     The internal parameter and the external parameter for the camera  2 - 1  are acquired by performing the calibration. The calibration execution unit  114  stores the internal parameter and the external parameter acquired by the calibration into the parametric information storage unit  124 . In other words, the calibration execution unit  114  updates the internal parameter and the external parameter for the camera  2 - 1 . It is to be noted that the internal parameter and the external parameter acquired by the calibration processing (first stage) may be deleted from (overwritten in) the parametric information storage unit  124  in step S 910 , and after step S 910  is performed, a series of processing illustrated in  FIG. 4  is completed. 
       FIG. 11  is a table illustrating example data stored in the management information storage unit  122  when a series of processing (at least up to the stage in which determination of YES is made in step S 908 ) illustrated in  FIGS. 9A and 9B  is performed. In  FIG. 11 , a series of processing illustrated in  FIGS. 9A and 9B  has been performed for the camera  2 - 1  in the same manner as the above-described processing for the camera  2 - 1 . 
     When the respective states illustrated in  FIG. 6  and  FIG. 11  are compared, it is seen that detection of feature points failed, for instance, at photographic times T 2 , T 4 , T 7 , and T 9  is now made possible (successful) by the calibration processing (second stage). Thus, it is seen that the occurrence of failure of detection of feature points due to (the second reason described above) distortion of the camera image is reduced. 
     As a result, for each of the cameras  2 - 1 ,  2 - 2 , the success rate of detection of feature points has increased from 30% to 70% in this example. Consequently, it is possible to improve the accuracy of each camera parameter acquired by calibration. 
     [Technical Effect on Acquisition of Omnidirectional Image] 
     The capability of detection of feature points at photographic times T 2 , T 4 , T 7 , and T 9  of  FIG. 11  for both the cameras  2 - 1 ,  2 - 2  achieves a technical effect in the calibration for acquiring an omnidirectional image. 
     To acquire an omnidirectional image, the images acquired by the cameras  2 - 1 ,  2 - 2  have to be integrated. In order to integrate the images, when camera calibration is performed, the positional relationship between the cameras  2 - 1  and  2 - 2  has to be identified. Specifically, camera calibration is performed using a pair of an image acquired by the camera  2 - 1  and an image acquired by the camera  2 - 2  that satisfy the below-described conditions, and an external parameter for each camera has to be acquired. Condition  1 : when the images are acquired, the positions of the flat plate  300  in real space are the same. Condition  2 : the flat plate  300  (in its entirety) is in the field of view (captured in the image) for both the image acquired by the camera  2 - 1  and the image acquired by the camera  2 - 2 . 
     A relative external parameter has to be determined based on the respective acquired external parameters of the camera  2 - 1  and the camera  2 - 2 . 
     In the images which satisfy the conditions  1  and  2  described above, the flat plate  300  is located at an area with a larger incident angle to the lens (an area near the outer side of an image) for both camera  2 - 1  and camera  2 - 2 . Therefore, it may be concluded that there is a high possibility that detection of feature points is failed due to the second reason described in [Problem in Calibration Processing (First Stage)]. 
     However, with the processing described in this example, it is possible to reduce the occurrence of failure of detection of feature points due to the second reason described above. Thus, it is possible to perform camera calibration for two cameras on which fisheye lens is mounted as illustrated in this example, and to acquire an external parameter of each the two cameras. Then the difference between the external parameters of two cameras acquired at this point allows the information processing device  1  to identify the difference of the positions and the arrangements or orientations in real space between the camera  2 - 1  and the camera  2 - 2  in real space. In other words, it is possible for the information processing device  1  to acquire a relative external parameter between the camera  2 - 1  and the cameras  2 - 2 . 
     [Other Modifications] 
     Various modifications may be made to the aspect described in this example. 
     For instance, in this example, the calibration processing (first stage) is performed on each of the camera  2 - 1  and camera  2 - 2 . However, for instance when the camera  2 - 1  and the camera  2 - 2  have the same model number and model type, the internal parameters of the two cameras are probably similar or the same. Thus, for instance when the information processing device  1  performs calibration for the camera  2 - 1  and the camera  2 - 2  in that order, the calibration processing (first stage) for the camera  2 - 2  may be omitted. The information processing device  1  may perform image correction processing for the camera  2 - 2  using the camera parameter (internal parameter) acquired in the calibration processing (first stage) for the camera  2 - 1 . 
     For instance, in this example, the information processing device  1  acquires the internal parameter used for correction processing of the calibration processing (second stage) by performing the calibration processing (first stage). However, when parameters serving as reference values (such as a reference value of parameter in which an individual difference between cameras is not taken into consideration) for the model type and model number of each camera used in this example are given, the correction processing of the calibration processing (second stage) may be performed utilizing an internal parameter included in the parameters serving as reference values. 
     In these modifications, it is possible to reduce the processing load associated with execution of calibration processing for multiple cameras. 
     When the correction processing described in this example is invertible transformation processing that allows distortion of an image to reduced, the correction processing may not be limited to correction processing using an equidistant cylindrical image. Also, various modifications may be adopted to correction processing using an equidistant cylindrical image. For instance,  FIG. 12  is a view illustrating a modification of equator setting in an equidistant cylindrical image. As illustrated in  FIG. 12 , it is possible to generate an azimuthal equidistant chart with the flat plate  300  located near the equator by setting a line  1201  passing through the flat plate  300  as the equator of the equidistant cylindrical image. 
       FIG. 13A  is an example azimuthal equidistant chart generated based on the equator setting described with reference to  FIG. 12 . Also, in  FIG. 13A , it is seen that in the vicinity of an equator  1301 , the distortion of the flat plate  300  in the image is lower than the distortion in  FIG. 12 . Similarly to  FIG. 10 , the shaded area in  FIG. 13A  is an area that is outside the field of view of the image of  FIG. 12  and is not captured in the image of  FIG. 12 . In this case, for instance, a line  1202  of  FIG. 12  may be a line (0 degree longitude line, for instance when both ends of the image in the azimuthal equidistant chart are 180 degrees east longitude and west longitude) corresponding to the longitude of the center of the image in the azimuthal equidistant chart. Thus, when an azimuthal equidistant chart is generated, it is possible to avoid division, as in  FIG. 13B , of the image of the flat plate  300 . As an example, a line corresponding to the longitude of the center of the image in the azimuthal equidistant chart is indicated by the line  1202  in  FIG. 12 , and the equator  1301  in  FIG. 13A . Also in these modifications, in a range where the purpose of the correction processing is achieved, a line corresponding to the longitude of the center of the equator or the image in the azimuthal equidistant chart does not have to pass through the center the of flat plate  300 . 
     [Example of Hardware Configuration] 
       FIG. 14  is an example hardware configuration of the information processing device  1  in this example. 
     The information processing device  1  is a computer including, for instance, a central processing unit (CPU)  1402 , a memory  1403 , a storage device  1404 , an NIC  1405 , a medium reading device  1406 , an input device  1407 , and a display device  1408  which are coupled to each other via a bus  1401 . 
     The CPU  1402  performs control of various operations in the information processing device  1 . The memory  1403  and the storage device  1404  store programs that executes various types of processing described in this example, and various types of data utilized for the various types of processing. The storage devices  1404  is, for instance, a memory medium such as a hard disk drive (HDD) or a solid state drive (SSD). 
     The CPU  1402  may implement each functional unit included in the processing unit  110  illustrated in  FIG. 2  by reading a program stored in the memory  1403  or the storage device  1404 , and executing processing and control. Alternatively, each of the memory  1403  and the storage device  1404  may serve as the storage unit  120  illustrated in  FIG. 2 . It is to be noted that the CPU  1402  may be replaced by a hardware circuit such as a micro processing unit (MPU), or an application specific integrated circuit (ASIC). 
     The NIC  1405  is hardware used for transmitting and receiving data via a wired or wireless network. The NIC  1405  may serve as the communication unit  101  under control of the CPU  1402 . 
     The medium reading device  1406  is a device for reading data from a recording medium, for instance, a disk drive which reads data stored in disc media such as a CD-ROM and a DVD-ROM, or a card slot which reads data stored in a memory card. Part or all of the data stored in the above-mentioned storage device  1404  may be stored in a recording medium which is readable using the medium reading device  1406 . 
     The input device  1407  is a device that receives input and designation from users (including a system administrator). As an example of the input device  1407 , for instance, a keyboard, a mouse, and a touchpad may be mentioned. The display device  1408  displays various pieces of information under the control of the CPU  1402 . The display device  1408  is, for instance, a liquid crystal display. 
     All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.