Patent Publication Number: US-8538137-B2

Title: Image processing apparatus, information processing system, and image processing method

Description:
This application is based on application No. 2009-175062 filed in Japan, the contents of which are hereby incorporated by reference. 
     BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relate to image processing techniques. 
     2. Description of the Background Art 
     Conventionally, techniques (matching point search techniques) are known in which two images are obtained by imaging the same object from different viewpoints with a so-called stereo camera and individual pixels in the two images are mutually correlated by comparison of two-dimensional image regions, e.g. by template matching. 
     In such matching point search techniques using two-dimensional image regions, one image is processed as a reference (reference image) and the other image is processed as a target (target image), for example. In such a matching point search technique, first, a window region is set in the reference image such that a point of interest in the reference image is approximately the gravity center position, and a window region of the same size is set in the target image such that a point to be processed in the target image is approximately the gravity center position. Then, the correlation between the two window regions is evaluated to match pixels between the reference and target images (for example, “A Sub-Pixel Correspondence Search Technique for Computer Vision Applications”, Kenji TAKITA, Mohammad abdul MUQUIT, Takafumi AOKI, and Tatsuo HIGUCHI, IEICE TRANS. FUNDAMENTALS, VOL E87-A, NO. 8 Aug. 2004, which is hereinafter referred to as Non-Patent Document 1). 
     Also, for such matching point search techniques, a technique is proposed in which the amount of computations required for the search of a matching point is considerably reduced and the results of matching are obtained very precisely and in short time (for example, Japanese Patent Application Laid-Open No. 2008-123141, which is hereinafter referred to as Patent Document 1). In this technique, the shape of window regions is one-dimensional. Also, a technique is proposed in which influences of noise are reduced and the three-dimensional position of an object can be very precisely measured (for example, Japanese Patent Application Laid-Open No. 208-157780, which is hereinafter referred to as Patent Document 2). In this technique, a matching point search is performed with integral images obtained by integrating individual pixels in time-sequentially taken multiple images. 
     In such matching point search techniques, the parallax about individual pixels between the reference image and target image is approximately uniform when the distance to the object taken in the two window regions set in the reference and target images is approximately uniform on the basis of the stereo camera, and then the matching point search offers precise results. Also, the amount of information used for the matching point search can be increased by increasing the size of window regions, and then the precision of the matching point search is ensured. 
     However, in the technique of Non-Patent Document 1, when the reference image and target image include a mixture of distant and near objects (distant-near competing condition), increasing the size of the window regions to ensure precise matching point search will deteriorate the precision of the matching point search. 
     Also, in the technique of Patent Document 1, while the size of window regions is reduced by adopting one-dimensional window regions, matching points are searched for by statistical processing using the results of evaluation over adjacent multiple lines, in order to ensure the amount of information. Accordingly, the precision of the matching point search might be deteriorated due to the distant-near competing condition. 
     The technique of Patent Document 2 gives no consideration at all to measures against the distant-near competing condition in matching point search. 
     SUMMARY OF THE INVENTION 
     The present invention is directed to an image processing apparatus. 
     According to the present invention, the image processing apparatus includes: an image obtaining portion that obtains a plurality of first images and a plurality of second images, the plurality of first images being obtained by time-sequentially imaging an object from a first viewpoint, the plurality of second images being obtained by time-sequentially imaging the object from a second viewpoint that is different from the first viewpoint; a region setting portion that sets reference regions including a reference point respectively in the first images with a same arrangement, and that sets comparison regions corresponding to a form of the reference regions respectively in the second images with a same arrangement; a pixel value distribution generating portion that generates one reference distribution of pixel values about two-or-more-dimensional space from distributions of pixel values about the plurality of reference regions set respectively in the plurality of first images by the region setting portion, and that generates one comparison distribution of pixel values about two-or-more-dimensional space from distributions of pixel values about the plurality of comparison regions set respectively in the plurality of second images by the region setting portion; and a matching point detecting portion that detects a matching point in the plurality of second images that corresponds to the reference point, by using the reference distribution of pixel values and the comparison distribution of pixel values. 
     This makes it possible to ensure an amount of information used in the calculations for the matching point search while suppressing the size of image regions in the respective images used in the calculations for the matching point search, which makes it possible to improve the precision of a matching point search with a plurality of images taking the same object where distant and near views coexist. 
     According to another aspect of the present invention, the image processing apparatus includes: an image obtaining portion that obtains a plurality of first images and a plurality of second images, the plurality of first images being obtained by time-sequentially imaging an object from a first viewpoint, the plurality of second images being obtained by time-sequentially imaging the object from a second viewpoint that is shifted from the first viewpoint in both of a first direction and a second direction; a first image generating portion that generates a plurality of first parallax suppressed images by suppressing parallax about the second direction occurring in each first image, and that generates a plurality of second parallax suppressed images by suppressing parallax about the second direction occurring in each second image; a second image generating portion that generates a plurality of third parallax suppressed images by suppressing parallax about the first direction occurring in each first image, and that generates a plurality of fourth parallax suppressed images by suppressing parallax about the first direction occurring in each second image; a first region setting portion that sets first reference regions that include a first reference point and that have an elongate direction corresponding to the first direction respectively in the first parallax suppressed images with a same arrangement, and also sets first comparison regions corresponding to a form of the first reference regions respectively in the second parallax suppressed images with a same arrangement; a second region setting portion that sets second reference regions that include a second reference point corresponding to the first reference point and that have an elongate direction corresponding to the second direction respectively in the third parallax suppressed images with a same arrangement, and also sets second comparison regions corresponding to a form of the second reference regions respectively in the fourth parallax suppressed images with a same arrangement; a first pixel value distribution generating portion that generates one first reference distribution of pixel values about two-or-more-dimensional space from distributions of pixel values about the plurality of first reference regions that are set respectively in the plurality of first parallax suppressed images by the first region setting portion, and also generates one first comparison distribution of pixel values about two-or-more-dimensional space from distributions of pixel values about the plurality of first comparison regions that are set respectively in the plurality of second parallax suppressed images by the first region setting portion; a second pixel value distribution generating portion that generates one second reference distribution of pixel values about two-or-more-dimensional space from distributions of pixel values about the plurality of second reference regions that are set respectively in the plurality of third parallax suppressed images by the second region setting portion, and also generates one second comparison distribution of pixel values about two-or-more-dimensional space from distributions of pixel values about the plurality of second comparison regions that are set respectively in the plurality of fourth parallax suppressed images by the second region setting portion; a first obtaining portion that, by using the first reference distribution of pixel values and the first comparison distribution of pixel values, obtains a first matching point in the plurality of second parallax suppressed images that corresponds to the first reference point, and a first reliability about the first matching point; a second obtaining portion that, by using the second reference distribution of pixel values and the second comparison distribution of pixel values, obtains a second matching point in the plurality of fourth parallax suppressed images that corresponds to the second reference point, and a second reliability about the second matching point; and a matching point detecting portion that, with a first combination of the first reference point and the first matching point and a second combination of the second reference point and the second matching point, adopts the first combination when the first reliability is higher than the second reliability, and adopts the second combination when the second reliability is higher than the first reliability. 
     This makes it possible to perform a matching point search on the basis of image regions where distant and near views coexist further less, making it possible to further improve the precision of a matching point search with a plurality of images taking the same object where distant and near views coexist. 
     The present invention is also directed to an image processing method and an information processing system. 
     The information processing system includes: an image processing apparatus according to any of the aspects described above; an imaging device having a first camera and a second camera, wherein the first camera and the second camera time-sequentially take images with same timing, so that the first camera time-sequentially takes the plurality of first images and the second camera time-sequentially takes the plurality of second images; and a projection device that projects different patterns to the object according to each timing of imaging by the first camera and the second camera. 
     Then, it is possible to take a plurality of images while time-sequentially projecting different patterns to the object. The amount of information used in the calculations for the matching point search is further increased, and it is possible to more stably and more precisely perform a matching point search with a plurality of images taking the same object where distant and near views coexist. 
     Thus, an object of the present invention is to provide a technique that can improve the precision of a matching point search with a plurality of images taking the same object where distant and near views coexist. 
     These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a diagram schematically illustrating the configuration of information processing systems according to first and second preferred embodiments and first and second modifications; 
         FIG. 2  is a block diagram illustrating the configuration of a main part of the information processing systems according to the first and second preferred embodiments and the first and second modifications; 
         FIG. 3  is a schematic diagram illustrating an example of the configuration of a stereo camera according to the first preferred embodiment and first to third modifications; 
         FIG. 4  is a block diagram illustrating the functional parts implemented by the controllers of the image processing apparatuses according to the first preferred embodiment and the first to third modifications; 
         FIG. 5  is a schematic diagram illustrating a specific example of a reference image group; 
         FIG. 6  is a schematic diagram illustrating a specific example of a target image group; 
         FIG. 7  is a schematic diagram illustrating how a reference region is set in a reference image; 
         FIG. 8  is a schematic diagram illustrating how a comparison region is set in a target image; 
         FIG. 9  is a schematic diagram illustrating how reference regions are set in the reference image group; 
         FIG. 10  is a schematic diagram illustrating how comparison regions are set in the target image group; 
         FIG. 11  is a diagram illustrating a method of generating a reference distribution of pixel values; 
         FIG. 12  is a diagram illustrating a method of generating a comparison distribution of pixel values; 
         FIG. 13  is a diagram for illustrating a matching point search using phase only correlation; 
         FIG. 14  is a diagram illustrating a distribution of POC values indicating the correlation between the reference and comparison regions; 
         FIG. 15  is a diagram for illustrating how a matching point is detected; 
         FIG. 16  is a diagram illustrating the relation between an imaging lens and an imaging element in a first camera; 
         FIG. 17  is a diagram illustrating the relation between an imaging lens and an imaging element in a second camera; 
         FIG. 18  is a flowchart illustrating an operation flow in the image processing apparatus; 
         FIG. 19  is a schematic diagram illustrating how a reference point is sequentially set in a reference image; 
         FIG. 20  is a schematic diagram illustrating the appearance of an example structure of a stereo camera according to a second preferred embodiment; 
         FIG. 21  is a block diagram illustrating the functional parts implemented by the controller of the image processing apparatus of the second preferred embodiment; 
         FIG. 22  is a schematic diagram illustrating a specific example of reference X-direction and reference Y-direction parallax suppressed image groups; 
         FIG. 23  is a schematic diagram illustrating a specific example of target X-direction and target Y-direction parallax suppressed image groups; 
         FIG. 24  is a schematic diagram illustrating how reference regions are set in the second preferred embodiment; 
         FIG. 25  is a schematic diagram illustrating how comparison regions are set in the second preferred embodiment; 
         FIG. 26  is a diagram illustrating a method of generating a reference distribution in the second preferred embodiment; 
         FIG. 27  is a diagram illustrating a method of generating a comparison distribution in the second preferred embodiment; 
         FIG. 28  is a diagram illustrating a matching point search in the second preferred embodiment; 
         FIG. 29  is a diagram illustrating how a matching point is derived in the second preferred embodiment; 
         FIG. 30  is a schematic diagram illustrating how reference regions are set in the second preferred embodiment; 
         FIG. 31  is a schematic diagram illustrating how comparison regions are set in the second preferred embodiment; 
         FIG. 32  is a diagram illustrating a method of generating a reference distribution in the second preferred embodiment; 
         FIG. 33  is a diagram illustrating a method of generating a comparison distribution in the second preferred embodiment; 
         FIG. 34  is a diagram illustrating a matching point search in the second preferred embodiment; 
         FIG. 35  is a diagram illustrating how a matching point is derived in the second preferred embodiment; 
         FIG. 36  is a flowchart illustrating an operation flow in the image processing apparatus of the second preferred embodiment; 
         FIG. 37  is a flowchart illustrating an operation flow in the image processing apparatus of the second preferred embodiment; 
         FIG. 38  is a flowchart illustrating an operation flow in the image processing apparatus of the second preferred embodiment; 
         FIG. 39  is a schematic diagram illustrating how reference regions are set in the first and second modifications; 
         FIG. 40  is a schematic diagram illustrating how comparison regions are set in the first and second modifications; 
         FIG. 41  is a diagram illustrating a method of generating a reference distribution in the first modification; 
         FIG. 42  is a diagram illustrating a method of generating a comparison distribution in the first modification; 
         FIG. 43  is a diagram illustrating a matching point search in the first modification; 
         FIG. 44  is a diagram illustrating how a matching point is detected in the first modification; 
         FIG. 45  is a diagram for illustrating a method of generating a reference distribution in the second modification; 
         FIG. 46  is a diagram for illustrating a method of generating a comparison distribution in the second modification; 
         FIG. 47  is a diagram for illustrating a method of generating a reference distribution in the second modification; 
         FIG. 48  is a diagram for illustrating a method of generating a comparison distribution in the second modification; and 
         FIG. 49  is a diagram schematically illustrating the configuration of an information processing system according to a third modification. 
     
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     The preferred embodiments of the present invention will now be described referring to the drawings. 
     &lt;(1) First Preferred Embodiment&gt; 
     &lt;(1-1) Configuration of Information Processing System&gt; 
       FIG. 1  is a diagram schematically illustrating the configuration of an information processing system  1 A according to a first preferred embodiment of the present invention, and  FIG. 2  is a block diagram illustrating the configuration of a main part of the information processing system  1 A. The information processing system  1 A includes a twin-lens stereo camera  2 A and an image processing apparatus  3 A that is connected to the stereo camera  2 A such that data can be transferred. 
     The twin-lens stereo camera  2 A is an imaging device including first and second cameras  21  and  22  having respective imaging elements. The first and second cameras  21  and  22  are separated in a given direction (in X direction here). Specifically, as shown in  FIG. 3 , the distance between the optical axis of the imaging lens  21 L of the first camera  21  and the optical axis of the imaging lens  22 L of the second camera  22  (i.e. the base length) is set as a distance L ax  in x direction.  FIG. 3  shows perpendicular three x, y and z axes in order to show the three-dimensional position of the object OB. 
     The first and second cameras  21  and  22  take images of the object OB, standing still in front of the camera, with the same timing and from different viewpoints. The object OB can be a metal product that is set at a relatively short distance from the stereo camera  2 A, for example. Two images taken at the same timing by the first and second cameras  21  and  22  are so-called stereo images, and they are referred to as “a set of images” as appropriate. 
     Herein, in the two images forming such stereo images, an image taken and acquired by the first camera  21  is referred to as “a first image” as appropriate, and an image taken and acquired by the second camera  22  is referred to as “a second image” as appropriate. Also, the viewpoint of the imaging by the first camera  21  is referred to as “a first viewpoint”, and the viewpoint of the imaging by the second camera  22  is referred to as “a second viewpoint” as appropriate. 
     With such a twin-lens stereo camera  2 A, the imaging timings of the first and second cameras  21  and  22  are synchronized, and images are taken in a time-sequential manner to obtain multiple sets of images. That is to say, a first image group G 1  including a plurality of first images is obtained by the first camera  21 , and a second image group G 2  including a plurality of second images is obtained by the second camera  22 . Then, the data about the multiple sets of images is transferred to the image processing apparatus  3 A through a data line CB. 
     The image processing apparatus  3 A is formed of a personal computer (PC), for example, and it includes an operation unit  301  including a mouse, keyboard, etc., a display  302  formed of a liquid-crystal display, for example, and an interface (I/F)  303  for receiving data from the stereo camera  2 A. The image processing apparatus  3 A also includes a storage  304 , an input/output portion  305 , and a controller  300 A. 
     The storage  304  is formed of a hard disk, for example, and it stores a program PGa for performing matching point search processing, arrangement change detecting processing, and distance measuring processing that will be described later. 
     The input/output portion  305  includes a disk drive, for example, and it accepts a storage medium  9  like an optical disk and exchange data with the controller  300 A. 
     The controller  300 A has a CPU serving as a processor and a memory for temporarily storing information, and it controls the components of the image processing apparatus  3 A in a centralized manner. In the controller  300 A, the program PGa in the storage  304  is read and executed to implement various functions and information processing operations. 
     Specifically, the controller  300 A performs an operation of searching for matching points between first images and second images that form stereo images obtained by the stereo camera  2 A (matching point search processing). Also, the controller  300 A performs an operation of detecting that the relative arrangement of the first and second cameras  21  and  22  has changed (arrangement change detecting processing). Also, the controller  300 A performs an operation of measuring the distance from the stereo camera  2 A to the object OB by calculating the three-dimensional position of the object OB by utilizing the results of the matching point search processing (distance measuring processing). The matching point search processing, arrangement change detecting processing, and distance measuring processing will be further described later. 
     The program data stored in the storage medium  9  can be stored in the memory of the controller  300 A through the input/output portion  305 . The stored program can thus be reflected to the operations of the image processing apparatus  3 A. 
     By the control of the controller  300 A, and as the arrangement change detecting processing detects a change of the relative arrangement of the first and second cameras  21  and  22 , the display  302  visually outputs a display element indicating that an adjustment operation (calibration) for the change of relative arrangement should be performed. The display  302  may visually output the results of the distance measuring processing, and may visually output a three-dimensional image of the object OB on the basis of the three-dimensional position of the object OB calculated in the distance measuring processing. 
     In this preferred embodiment, for the sake of clarity of explanation, the aberration of the stereo camera  2 A is favorably corrected, and the first and second cameras  21  and  22  are set approximately parallel (preferably, completely parallel). That is to say, the optical axes of the first and second cameras  21  and  22  are set approximately parallel (preferably, completely parallel), and the object imaged in the first and second images have approximately the same angular relations with respect to the perimeters of the first and second images (preferably, completely the same angular relations). If the actual configuration of the stereo camera  2 A is not in such a condition, the images may be transformed by image processing to stereo images taken under equivalent conditions. 
     &lt;(1-2) Functions of Image Processing Apparatus&gt; 
     In the image processing apparatus  3 A, first images are handled as reference images, and second images are handled as target images, and it performs the matching point search processing of detecting matching points on the target images that correspond to reference points sequentially set on the reference images. 
     In the matching point search processing, mainly by the following operations (1) to (3), an amount of information utilized in the calculations for matching point search is ensured while reducing the size of image regions used in the calculations for matching point search. This improves the precision of the matching point search for a plurality of images taking the same object where distant and near views coexist. 
     (1) Small windows are set respectively in multiple reference images that form a time-sequentially obtained first image group (hereinafter also referred to as “a reference image group”) G 1 , and the distributions of pixel values of the small windows are integrated to artificially generate a distribution of pixel values about a window larger than the small windows (hereinafter referred to as “a reference distribution”). 
     (2) Small windows are set respectively in multiple target images that form a time-sequentially obtained second image group (hereinafter also referred to as “a target image group”) G 2 , and the distributions of pixel values of the small windows are integrated to artificially generate a distribution of pixel values about a window larger than the small windows (hereinafter referred to as “a comparison distribution”). 
     (3) A matching point on the target images corresponding to a reference point is detected from the reference distribution and comparison distribution. 
     Now, the functional configuration for the matching point search processing, arrangement change detecting processing, and distance measuring processing by the image processing apparatus  3 A will be described. 
       FIG. 4  is a block diagram illustrating the functional configuration implemented by the controller  300 A of the image processing apparatus  3 A. As shown in  FIG. 4 , as the functional configuration, the controller  300 A includes an image obtaining block  310 , a region setting block  320 A, a pixel value distribution generating block  330 A, a matching point detecting block  340 A, an arrangement change detecting block  350 , and a distance deriving block  360 . The functional configuration will be described below. 
     &lt;(1-2-1) Image Obtaining Block  310 &gt; 
     The image obtaining block  310  obtains a reference image group G 1  obtained by time-sequentially imaging the object OB from the first viewpoint with the first camera  21 , and a target image group G 2  obtained by time-sequentially imaging the object OB from the second viewpoint with the second camera  22 . Seen from another aspect, the image obtaining block  310  obtains a plurality of image sets that are time-sequentially taken with the stereo camera  2 A. Each set of images includes a reference image and a target image in which the object OB is imaged from different viewpoints. 
       FIG. 5  is a diagram illustrating a specific example of the first image group (reference image group) G 1 . As shown in  FIG. 5 , the reference image group G 1  is formed of time-sequentially taken reference images G 11  to G 17 . Each of the reference images G 11  to G 17  is formed of a large number of pixels arranged in a matrix along mutually perpendicular X direction and Y direction, and each of the reference images G 11  to G 17  has the same rectangular form. In each of the reference images G 11  to G 17 , it is assumed that a given number (N) of pixels are arranged along X direction to form longer sides, and a given number (M) of pixels are arranged along Y direction, different from X direction, to form shorter sides. 
     In the reference images G 11  to G 17 , the upper left pixel is handled as a reference (e.g. the origin) and the positions of the individual pixels are represented by the XY coordinates (X, Y), where, for example, the value of X coordinate increases by one as the position is shifted one pixel in X direction, and the value of Y coordinate increases by one as the position is shifted one pixel in Y direction. 
       FIG. 5  also shows coordinate axes indicating the perpendicular X and Y directions. The direction of X axis, of the two axes X and Y about two-dimensional images, corresponds to the direction of x axis, of the three axes x, y and z indicating the three-dimensional position of the object OB, and the direction of Y axis about two-dimensional images corresponds to the direction of y axis indicating the three-dimensional position of the object OB. 
     Also, in  FIG. 5 , the reference images G 11  to G 17  are arranged in the order of times when they were taken, and the time is indicated by the coordinate axis T. In  FIG. 6  and other drawings, too, the perpendicular three axes formed of the coordinate axes X, Y and T are shown as needed. 
       FIG. 6  is a diagram illustrating a specific example of the second image group (target image group) G 2 . As shown in  FIG. 6 , the target image group G 2  includes time-sequentially taken target images G 21  to G 27 . The target images G 21  to G 27  have the same form, the same pixel arrangement, and the same pixel coordinate representation, as the reference images G 11  to G 17 . 
     &lt;(1-2-2) Region Setting Block  320 A&gt; 
     The region setting block  320 A has a reference region setting block  321 A and a comparison region setting block  322 A. The reference region setting block  321 A sets reference regions, as one-dimensional windows, in the individual reference images G 11  to G 17 . The comparison region setting block  322 A sets comparison regions, as one-dimensional windows, in the individual target images G 21  to G 27 . 
     Specifically, as shown in  FIG. 7 , a one-dimensional reference region BR 11  that includes a reference point P ref   1  as a center point and that is formed of a pixel string along X direction is set in the reference image G 11 . Also, as shown in  FIG. 8 , a one-dimensional comparison region CR 21  formed of a pixel string along X direction is set in the target image G 21 . 
     The reference region BR 11  and the comparison region CR 21  have a corresponding form and size (the same form and size), and the position where the reference region BR 11  is set in the reference image G 11  and the position where the comparison region CR 21  is set in the target image G 21  have a positional relation that is shifted in X direction by an amount corresponding to the base length L. 
     In this way, in the first image set included in the multiple image sets, the region setting block  320 A sets the reference region BR 11  including the reference point P ref   1  in the reference image G 11  included in the first image set, and also sets the comparison region CR 21  in the target image G 21  included in that first image set. 
     Also, as shown in  FIG. 9 , reference regions BR 12  to BR 17  are set respectively in the reference images G 12  to G 17 , and, as shown in  FIG. 10 , comparison regions CR 22  to CR 27  are set respectively in the target images G 22  to G 27 . 
     Here, the reference regions BR 12  to BR 17  have the same form and size as the reference region BR 11 , and the positional relation of the reference region BR 11  with respect to the reference image G 11 , and the positional relation of the reference regions BR 12  to BR 17  with respect to the reference images G 12  to G 17 , are the same. Also, the comparison regions CR 22  to CR 27  have the same form and size as the comparison region CR 21 , and the positional relation of the comparison region CR 21  with respect to the target image G 21 , and the positional relation of the comparison regions CR 22  to CR 27  with respect to the target images G 22  to G 27 , have the same positional relation. 
     In this way, the region setting block  320 A sets the reference regions BR 11  to BR 17  including the reference point P ref   1  in the reference images G 11  to G 17  with the same arrangement, and also sets the comparison regions CR 21  to CR 27 , corresponding to the form and size of the reference regions BR 11  to BR 17 , in the target images G 21  to G 27  with the same arrangement. 
     Here, according to the condition of arrangement of the first viewpoint and the second viewpoint separated in x direction, the elongate direction of the multiple reference regions BR 11  to BR 17  and the multiple comparison regions CR 21  to CR 27  is set in X direction corresponding to x direction. 
     &lt;(1-2-3) Pixel Value Distribution Generating Block  330 A&gt; 
     The pixel value distribution generating block  330 A generates one distribution of pixel values about two-dimensional space (a reference distribution) from the distributions of pixel values about the plurality of reference regions BR 11  to BR 17 , and also generates one distribution of pixel values about two-dimensional space (a comparison distribution) from the distributions of pixel values about the plurality of comparison regions CR 21  to CR 27 . 
     Specifically, as shown in  FIG. 11 , first, pixel strings Bi 11  to Bi 17  about the reference regions BR 11  to BR 17  are extracted from the reference images G 11  to G 17 . Here, the pixel strings Bi 11  to Bi 17  indicate the distributions of pixel values about the reference regions BR 11  to BR 17  (hereinafter also referred to as “pixel value distributions”). More specifically, the pixel value distribution Bi 11  about the reference region BR 11  is extracted from the reference image G 11 , the pixel value distribution Bi 12  about the reference region BR 12  is extracted from the reference image G 12 , the pixel value distribution Bi 13  about the reference region BR 13  is extracted from the reference image G 13 , the pixel value distribution Bi 14  about the reference region BR 14  is extracted from the reference image G 14 , the pixel value distribution Bi 15  about the reference region BR 15  is extracted from the reference image G 15 , the pixel value distribution Bi 16  about the reference region BR 16  is extracted from the reference image G 16 , and the pixel value distribution Bi 17  about the reference region BR 17  is extracted from the reference image G 17 . 
     Next, the pixel value distributions Bi 11  to Bi 17  are arranged according to given arrangement rules to generate a two-dimensional pixel value distribution (hereinafter also referred to as “a reference distribution”) D PV   1 . Here, the given arrangement rules are, for example, that the pixel value distributions Bi 11  to Bi 17  are arranged parallel to each other, and that the pixel value distribution Bi 11 , pixel value distribution Bi 12 , pixel value distribution Bi 13 , pixel value distribution Bi 14 , pixel value distribution Bi 15 , pixel value distribution Bi 16 , and pixel value distribution Bi 17  are arranged in this order and integrated, so as to form a pixel value distribution about a rectangular region. The rectangular reference distribution D PV   1  is thus generated. 
     Also, as shown in  FIG. 12 , first, pixel strings Ci 21  to Ci 27  about the comparison regions CR 21  to CR 27  are extracted from the target images G 21  to G 27 . Here, the pixel strings Ci 21  to Ci 27  indicate the distributions of pixel values about the comparison regions CR 21  to CR 27  (hereinafter also referred to as “pixel value distributions”). More specifically, the pixel value distribution Ci 21  about the comparison region CR 21  is extracted from the target image G 21 , the pixel value distribution Ci 22  about the comparison region CR 22  is extracted from the target image G 22 , the pixel value distribution Ci 23  about the comparison region CR 23  is extracted from the target image G 23 , the pixel value distribution Ci 24  about the comparison region CR 24  is extracted from the target image G 24 , the pixel value distribution Ci 25  about the comparison region CR 25  is extracted from the target image G 25 , the pixel value distribution Ci 26  about the comparison region CR 26  is extracted from the target image G 26 , and the pixel value distribution Ci 27  about the comparison region CR 27  is extracted from the target image G 27 . 
     Next, the pixel value distributions Ci 21  to Ci 27  are arranged according to given arrangement rules to generate a two-dimensional pixel value distribution (hereinafter also referred to as “a comparison distribution) D PV   2 . Here, the given arrangement rules are, for example, that the pixel value distributions Ci 21  to Ci 27  are arranged parallel to each other, and that the pixel value distribution Ci 21 , pixel value distribution Ci 22 , pixel value distribution Ci 23 , pixel value distribution Ci 24 , pixel value distribution Ci 25 , pixel value distribution Ci 26 , and pixel value distribution Ci 27  are arranged in this order and integrated, so as to form a pixel value distribution about a rectangular region. The rectangular comparison distribution D PV   2  is thus generated. 
     In the generation of the reference distribution D PV   1  and the comparison distribution D PV   2 , another order of arrangement may be adopted as long as the order of arrangement of the pixel value distributions Bi 11  to Bi 17  and the order of arrangement of the pixel value distributions Ci 21  to Ci 27  correspond to each other, for example. 
     &lt;(1-2-4) Matching Point Detecting Block  340 A&gt; 
     The matching point detecting block  340 A detects a matching point corresponding to the reference point P ref   1  in the target image group G 2  by using the reference distribution D PV   1  and the comparison distribution D PV   2 . 
     As to the method of the matching point search for detecting a matching point, for example, correlation methods in which amplitude components are suppressed are known, and it can be, for example, Phase Only Correlation (POC) or DCT sign only correlation (paper for reference: “Merger of Image Signal Processing and Image Patten Recognition-DCT Sign Only Correlation and its Applications” Hitoshi KIYA). In these correlation methods, similarity calculations are performed by using phase-only signals, with amplitude components suppressed, from frequency-resolved signals of patterns. In this preferred embodiment, a matching point search using phase only correlation is performed. 
       FIG. 13  is a diagram for illustrating a matching point search using phase only correlation. 
     Here, the reference distribution D PV   1  and the comparison distribution D PV   2  are handled as image regions in which a given number, N 1 , of pixels are arranged along X direction, and a given number, N 2 , of pixels are arranged along Y direction. These image regions are represented by Expression 1 below.
 
f(n 1 ,n 2 ), Size N 1 ×N 2  
 
g(n 1 ,n 2 ), Size N 1 ×N 2   Expression 1
 
     Where,
         n 1 =−M 1 , . . . , M 1      n 2 =M 2 , . . . , M 2          

     Here, f(n 1 , n 2 ) in Expression 1 above indicates the image region about the reference distribution D PV   1 , and g(n 1 , n 2 ) in Expression 1 indicates the image region about the comparison distribution D PV   2 . Also, N 1  and N 2  are set as N 1 =2M 1 +1, N 2 =2M 2 +1, for example. 
     First, two-dimensional Fourier transforms T 1   a  and T 1   b  using Expression 2 below are performed to the image regions of the reference distribution D PV   1  and the comparison distribution D PV   2 . 
     
       
         
           
             
               
                 
                   
                     
                       F 
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           
                             n 
                             1 
                           
                           , 
                           
                             n 
                             2 
                           
                         
                       
                       ⁢ 
                       
                         
                           f 
                           ⁡ 
                           
                             ( 
                             
                               
                                 n 
                                 1 
                               
                               , 
                               
                                 n 
                                 2 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             1 
                           
                           
                             
                               k 
                               1 
                             
                             ⁢ 
                             
                               n 
                               1 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             2 
                           
                           
                             
                               k 
                               2 
                             
                             ⁢ 
                             
                               n 
                               2 
                             
                           
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       G 
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           
                             n 
                             1 
                           
                           , 
                           
                             n 
                             2 
                           
                         
                       
                       ⁢ 
                       
                         
                           g 
                           ⁡ 
                           
                             ( 
                             
                               
                                 n 
                                 1 
                               
                               , 
                               
                                 n 
                                 2 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             1 
                           
                           
                             
                               k 
                               1 
                             
                             ⁢ 
                             
                               n 
                               1 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             2 
                           
                           
                             
                               k 
                               2 
                             
                             ⁢ 
                             
                               n 
                               2 
                             
                           
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     Where 
                     , 
                     
                       
                         W 
                         P 
                       
                       = 
                       
                         EXP 
                         ( 
                         
                           
                             - 
                             j 
                           
                           ⁢ 
                           
                             
                               2 
                               ⁢ 
                               π 
                             
                             P 
                           
                         
                         ) 
                       
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       
                         k 
                         s 
                       
                       = 
                       
                         - 
                         
                           M 
                           s 
                         
                       
                     
                     , 
                     … 
                     ⁢ 
                     
                         
                     
                     , 
                     
                       M 
                       s 
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   2 
                 
               
             
           
         
       
     
     In the note of Expression 2, N 1  and N 2  are assigned to the subscript P of W, and 1 and 2 are assigned to the subscript s of k. 
     For the image regions subjected to the Fourier transforms T 1   a  and T 1   b , normalizations T 2   a  and T 2   b  are performed to remove image amplitude components by using the expressions shown as Expression 3 below. 
     
       
         
           
             
               
                 
                   
                     
                       
                         F 
                         ′ 
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         F 
                         ⁡ 
                         
                           ( 
                           
                             
                               k 
                               1 
                             
                             , 
                             
                               k 
                               2 
                             
                           
                           ) 
                         
                       
                       
                          
                         
                           F 
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                             
                             ) 
                           
                         
                          
                       
                     
                   
                   , 
                   
                       
                   
                   ⁢ 
                   
                     
                       
                         G 
                         ′ 
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         G 
                         ⁡ 
                         
                           ( 
                           
                             
                               k 
                               1 
                             
                             , 
                             
                               k 
                               2 
                             
                           
                           ) 
                         
                       
                       
                          
                         
                           G 
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                             
                             ) 
                           
                         
                          
                       
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   3 
                 
               
             
           
         
       
     
     After normalizations T 2   a  and T 2   b , synthesis T 3  using Expression 4 below is performed, and also two-dimensional inverse Fourier transform T 4  using Expression 5 is performed. Correlation calculations between images are thus carried out and the results (POC value) are outputted. 
     
       
         
           
             
               
                 
                   
                     R 
                     ⁡ 
                     
                       ( 
                       
                         
                           k 
                           1 
                         
                         , 
                         
                           k 
                           2 
                         
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         F 
                         ′ 
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                         
                         ) 
                       
                     
                     ⁢ 
                     
                       
                         
                           G 
                           ′ 
                         
                         ⁡ 
                         
                           ( 
                           
                             
                               k 
                               1 
                             
                             , 
                             
                               k 
                               2 
                             
                           
                           ) 
                         
                       
                       _ 
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   4 
                 
               
             
             
               
                 
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         
                           k 
                           1 
                         
                         , 
                         
                           k 
                           2 
                         
                       
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       
                         
                           N 
                           1 
                         
                         ⁢ 
                         
                           N 
                           2 
                         
                       
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                         
                       
                       ⁢ 
                       
                         
                           R 
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             1 
                           
                           
                             
                               - 
                               
                                 k 
                                 1 
                               
                             
                             ⁢ 
                             
                               n 
                               1 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             2 
                           
                           
                             
                               - 
                               
                                 k 
                                 2 
                               
                             
                             ⁢ 
                             
                               n 
                               2 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   5 
                 
               
             
           
         
       
     
     By the operations above, the results (POC value) indicating the correlation between the reference distribution D PV   1  and the comparison distribution D PV   2  are obtained, and the obtained results (POC value) are as shown in  FIG. 14 , for example. 
     In  FIG. 14 , the POC value is large in the portion of the image region (N 1 ×N 2 ) where the correlation is high. In the image region about the comparison distribution D PV   2 , the position corresponding to the peak Jc of POC value is the matching point on the target image G 21  that corresponds to the center point (reference point) P ref   1  in the reference region BR 11  on the reference image G 11 . 
     Since POC value is discretely obtained, the matching point may be detected more finely by performing interpolation between adjacent pixels and estimating the position of peak Jc in the size of sub-pixel that is finer than the size of one pixel. The method of interpolation can be a method where parabola function is obtained from the distribution of discretely obtained POC values, for example. 
     By the matching point search using phase only correlation, as shown in  FIG. 15 , for example, a point P cor   2  shifted from the center point P cent   2  of the comparison distribution D PV   2  is detected as a point corresponding to the peak Jc of POC value. Then, in the target images G 21  to G 27 , points having a positional relation like the positional relation between the center point P cent   2  and the point P cor   2  are detected as matching points, on the basis of the center points of the comparison regions CR 21  to CR 27 . 
     &lt;(1-2-5) Arrangement Change Detecting Block  350 &gt; 
     The arrangement change detecting block  350  detects a change of the condition of arrangement of the first camera  21  and the second camera  22  by recognizing that the relation between the position of the reference point P ref   1  in the multiple reference images G 11  to G 17  and the position corresponding to the point P cor   2  in the multiple target images G 21  to G 27  has changed in Y direction different from X direction of the base length. That is to say, the arrangement change detecting block  350 B detects a change of the condition of arrangement about the relative arrangement of the first viewpoint and the second viewpoint. 
     For example, as shown in  FIG. 15 , in the comparison distribution D PV   2 , the arrangement change detecting block  350  detects a change of the condition of arrangement when the amount of shift in Y direction between the center point P cent   2  and the point P cor   2  has changed from a predetermined reference amount of shift (a reference amount of shift). Then, display  302  visually outputs a display element indicating that an adjustment operation (calibration) about the change of the arrangement relation should be performed. The reference amount of shift is the amount of shift in Y direction between the center point P cent   2  and the point P cor   2  when calibration is carried out, and it is updated every time calibration is performed and stored in the storage  304 . 
     The calibration will be briefly described below. 
       FIG. 16  is a diagram illustrating the relation between the imaging lens  21 L and the imaging element Se 1  in the first camera  21 , and  FIG. 17  is a diagram illustrating the relation between the imaging lens  22 L and the imaging element Se 2  in the second camera  22 . 
     Here, first, when the coordinates M of a three-dimensional point in space is taken as the coordinates m 1  of a two-dimensional point in the reference image taken and obtained by the first camera  21 , and also as the coordinates m 2  of a two-dimensional point in the target image taken and obtained by the second camera  22 , then the relations of the following expressions (1) and (2) hold with projection matrices P 1  and P 2 . 
                   Expression   ⁢           ⁢   6                             λ   ⁢       m   ~     1       =         P   1     ·     M   ~       =       (           α   1           γ   1           u   1             0         β   1           v   1             0       0       1         )     ⁢       (           R   1           t   1           )     ·     M   ~                   (   1   )                       ⁢     =       (           α   1           γ   1           u   1             0         β   1           v   1             0       0       1         )     ⁢       (         1       0       0       0           0       1       0       0           0       0       1       0         )     ·     M   ~                                     λ   ⁢       m   ~     2       =         P   2     ·     M   ~       =       (           α   2           γ   2           u   2             0         β   2           v   2             0       0       1         )     ⁢       (           R   2           t   2           )     ·     M   ~                   (   2   )               
Where,
     {tilde over (M)} is the transposed matrix of the homogeneous coordinates of the coordinates M of the three-dimensional point;   {tilde over (m)} 1 , is the transposed matrix of the homogeneous coordinates of the coordinates m 1  of the two-dimensional point;   {tilde over (m)} 2  is the transposed matrix of the homogeneous coordinates of the coordinates m 2  of the two-dimensional point;   λ is scale coefficient;   α 1 =f 1 ×sx 1 , β 1 =f 1 ×s y1 ;   α 2 =f 2 ×sx 2 , α 2 =f 2 ×sy 2 ;   f 1  is the focal length of the first camera;   f 2  is the focal length of the second camera;   sx 1  is the length in X direction of the imaging sensor of the first camera;   sy 1  is the length in Y direction of the imaging sensor of the first camera;   sx 2  is the length in X direction of the imaging sensor of the second camera;   sy 2  is the length in Y direction of the imaging sensor of the second camera;   (u 1 , v 1 ) is the optical center of the imaging lens of the first camera;   (u 2 , v 2 ) is the optical center of the imaging lens of the second camera;   γ 1  is the skew factor about the first camera;   γ 2  is the skew factor about the second camera;   R 1  is the rotation component about the arrangement of the first camera based on the first camera;   t 1  is the translation component about the arrangement of the first camera based on the first camera;   R 2  is the rotation component about the arrangement of the second camera based on the first camera; and   t 2  is the translation component about the arrangement of the second camera based on the first camera.   

     When the skews in the first and second cameras  21  and  22  are not considered, the skew factor values γ 1  and γ 2  are zero. Then, when the projection matrices P 1  and P 2  of the expressions (1) and (2) are obtained, the arrangement relation of the first camera  21  and the second camera  22  is obtained. 
     Now, a simplest method for obtaining the projection matrices P 1  and P 2  can be a method using reference and target images obtained by imaging, with the stereo camera  2 A, an object for calibration whose three-dimensional size is known. Specifically, the projection matrices P 1  and P 2  are each a 3×4 matrix and have 12 parameters. When one of the 12 parameters is normalized to 1, the 12 parameters are reduced to 11 parameters. Accordingly, the projection matrices P 1  and P 2  are uniquely obtained when six or more sets of coordinates M of a three-dimensional point in space and coordinates m 1  and m 2  of the corresponding two-dimensional points are given about reference and target images. The method for detecting the coordinates m 1  and m 2  of two-dimensional points corresponding to the coordinates M of the three-dimensional point can be a method in which a portion of the coordinates M about a characteristic shape of the calibration object is detected in the reference and target images, for example. 
     In this way, the operation of calculating the projection matrices P 1  and P 2  indicating the arrangement relation of the first camera  21  and the second camera  22  corresponds to the calibration. In the calibration, the calculated projection matrices P 1  and P 2  are stored in the storage  304  as transform data D trans  and utilized in the distance measuring processing. 
     The various information processing operations in the calibration are started as a user operates the operation unit  301 , with the stereo camera  2 A and an object for calibration appropriately arranged, and the operations are executed under the control by the controller  300 A. 
     &lt;(1-2-6) Distance Calculating Block  360 &gt; 
     The distance calculating block  360  calculates the distance from the stereo camera  2 A to the portion of the object OB that corresponds to the reference point P ref   1  on the basis of the coordinates of the reference point P ref   1  in the reference image G 11  and the coordinates of the matching point in the target image G 21  detected by the matching point detecting block  340 A. 
     Specifically, the transform data D trans  obtained by the calibration and stored in the storage  304  (specifically, the projection matrices P 1  and P 2 ) is substituted into the expressions (1) and (2), and the coordinate values of the reference point P ref   1  are substituted into the expression (1) as the values of the coordinates m 1 , and the coordinate values of the matching point corresponding to the reference point P ref   1  are substituted into the expression (2) as the values of the coordinates m 2 . Then, the coordinates M of the three-dimensional point about the portion of the object OB that corresponds to the reference point P ref   1  are obtained as the two expressions are solved. Then, for example, when the three-dimensional coordinates of the stereo camera  2 A is taken as the origin, it is easy to calculate the distance from the stereo camera  2 A to the portion of the object OB that corresponds to the reference point P ref   1 , on the basis of the obtained coordinates M. 
     The distance data L data  calculated here is stored in the storage  304  in association with the coordinates M of the corresponding three-dimensional point. 
     &lt;(1-3) Operations of Image Processing Apparatus&gt; 
       FIG. 18  is a flowchart illustrating the operation flow of the matching point search processing, arrangement change detecting processing, and distance measuring processing in the image processing apparatus  3 A. This operation flow is implemented as the controller  300 A executes the program PGa, and the operation flow is started as a user variously operates the operation unit  301  and the flow moves to step ST 1  in  FIG. 18 . 
     In step ST 1 , the image obtaining block  310  obtains a reference image group G 1  and a target image group G 2 . 
     In step ST 2 , the region setting block  320 A sets one pixel in the reference image G 11  as a reference point P ref   1 . 
     In step ST 3 , the region setting block  320 A sets windows in the reference image G 11  and the target image G 21  that form the first set of images. Here, a reference region BR 11  as a one-dimensional window is set in the reference image G 11 , and a comparison region CR 21 , having the same form and size as the reference region BR 11 , is set in the target image G 21 . 
     In step ST 4 , the region setting block  320 A sets windows in the reference images G 12  to G 17  and the target images G 22  to G 27  that form the second and following sets of images. Here, reference regions BR 12  to BR 17 , corresponding to the form, size, and arrangement position of the reference region BR 11 , are set in the reference images G 12  to G 17 , and comparison regions CR 22  to CR 27 , corresponding to the form, size and arrangement position of the comparison region CR 21 , are set in the target images G 22  to G 27 . 
     In step ST 5 , the pixel value distribution generating block  330 A generates one reference distribution D PV   1  about two-dimensional space from the distributions of pixel values about the plurality of reference regions BR 11  to BR 17 , and also generates one comparison distribution D PV   2  about two-dimensional space from the distributions of pixel values about the plurality of comparison regions CR 21  to CR 27 . 
     In step ST 6 , on the basis of the reference distribution D PV   1  and the comparison distribution D PV   2 , the matching point detecting block  340 A detects a matching point corresponding to the reference point P ref   1  from the target image group G 2 . 
     In step ST 7 , on the basis of the coordinates of the reference point P ref   1  in the reference image G 11  and the coordinates of the matching point in the target image G 21  detected in step ST 6 , the distance calculating block  360  calculates the three-dimensional coordinates of the portion of the object OB that corresponds to the reference point P ref   1 , and the distance to the portion of the object OB corresponding to the reference point P ref   1 , on the basis of the stereo camera  2 A. The data indicating the three-dimensional coordinates and distance thus calculated is stored in the storage  304  in association with information indicating the reference point P ref   1 . 
     In step ST 8 , the arrangement change detecting block  350  judges whether the relation between the position of the reference point P ref   1  in the multiple reference images G 11  to G 17  and the position about the point P cor   2  in the multiple target images G 21  to G 27  has changed in Y direction different from X direction based on the base length. When the amount of shift in Y direction between the positions about the reference point P ref   1  and the point P cor   2  has changed from the predetermined reference amount of shift, the flow moves to step ST 9 ; when the amount of shift in Y direction between the positions about the reference point P ref   1  and the point P cor   2  has not changed from the predetermined reference amount of shift, the flow moves to step ST 10 . 
     In step ST 9 , according to a signal output from the arrangement change detecting block  350 , the display  302  visually outputs a display element indicating that an adjustment operation (calibration) should be performed. The visual output of the display element indicating that the calibration should be performed may be continued until calibration is performed, for example. The information that calibration should be performed may be given by voice. 
     In step ST 10 , the region setting block  320 A judges whether there remain any points to be set as reference point P ref   1  (points to be processed) in the reference image G 11 . For example, it judges whether all pixels of the reference image G 11  have been set as reference point P ref   1 . When there remains any point to be processed, the flow returns to step ST 2  and another pixel in the reference image G 11  is set as reference point P ref   1  and steps ST 3  to ST 9  are performed. When no point to be processed remains, the operation flow is ended. 
     According to the operation flow above, as shown in the arrows in  FIG. 19  in the reference image G 11 , for example, the reference point P ref   1  is time-sequentially set, while shifted one pixel at a time, sequentially from the top (−Y direction) and from left to right (X direction), and matching points corresponding to the reference points P ref   1  are detected on the target images G 21  to G 27 . Matching points may be detected about reference points at intervals of a given number of pixels in the reference image G 11 . 
     As described so far, according to the image processing apparatus  3 A of the first preferred embodiment, the reference regions BR 11  to BR 17  set in the reference images G 11  to G 17  and the comparison regions CR 21  to CR 27  set in the target images G 21  to G 27  are relatively small regions about pixel strings along X direction. Accordingly, it is possible to suppress coexistence of distant and near views in the image regions of the reference regions BR 11  to BR 17  and the comparison regions CR 21  to CR 27  used in the calculations for the matching point search. 
     Also, the amount of information used for the matching point search is maintained by utilizing the information about the reference regions BR 11  to BR 17  set in the time-sequentially obtained reference images G 11  to G 17  and the information about the comparison regions CR 21  to CR 27  set in the time-sequentially obtained target images G 21  to G 27 . That is to say, it is possible to ensure an amount of information used in the calculations for matching point search, while reducing the size of the image regions used in the calculations for the matching point search, in the reference and target images. Also, influences of noise on the images are reduced. 
     Thus, the amount of information used in the calculations for matching point search is ensured while reducing the size of the image regions used in the calculations for matching point search in individual images, and therefore the precision of the matching point search is improved while reducing the influences of coexistence of distant and near views (distant-near competing). That is to say, it is possible to improve the precision of the matching point search with a plurality of images taking the same object where distant and near views coexist. 
     In the matching point search, two artificial two-dimensional regions are used for the calculations, so that the precision of the matching point search is improved without increasing the amount of calculations in comparison to conventional ones. 
     Then, as a result of the improved precision of the matching point search, the distance to the object OB can be more accurately obtained from the reference images G 11  to G 17  and the target images G 21  to G 27  taking the same object where distant and near views coexist. 
     Also, when the relative arrangement between the first and second viewpoints of the stereo camera  2 A has shifted due to aged deterioration, impact, etc., information indicating that calibration should be performed is given. Accordingly, the user can perform adjustment operations according to the condition of arrangement. 
     &lt;(2) Second Preferred Embodiment&gt; 
     &lt;(2-1) Differences from First Preferred Embodiment&gt; 
     In the information processing system  1 A of the first preferred embodiment, the amount of shift between the optical axis of the imaging lens  21 L and the optical axis of the imaging lens  22 L in the stereo camera  2 A is the distance L ax  in x direction, and the elongate direction of the reference regions BR 11  to BR 17  and the comparison regions CR 21  to CR 27  is set in X direction in the matching point search processing. 
     On the other hand, in an information processing system  1 B of a second preferred embodiment, as shown in  FIG. 20 , the amount of shift between the optical axis of an imaging lens  21 L and the optical axis of an imaging lens  22 L in a stereo camera  2 B is set as a distance L bx  in x direction and a distance L by  in y direction. Also, in the image processing apparatus  3 B of the second preferred embodiment, for each reference point, a matching point search using reference regions and comparison regions whose elongate direction is set in X direction, and a matching point search using reference regions and comparison regions whose elongate direction is set in Y direction, are performed, and the result of a matching point search with higher reliability is determined as a matching point corresponding to the reference point. This further enhances the precision of the matching point search with a plurality of images taking the same object where distant and near views coexist. 
     Specifically, in the information processing system  1 B of the second preferred embodiment, as compared with the information processing system  1 A of the first preferred embodiment, the stereo camera  2 A is replaced by the stereo camera  2 B where the arrangement of the first and second cameras  21  and  22  is changed, and the image processing apparatus  3 A is replaced by an image processing apparatus  3 B where the controller  300 A is changed to a controller  300 B having different functions. In the controller  300 B, various functions are implemented as a program PGb stored in a storage  304  is read and executed. Other parts of the information processing system  1 B of the second preferred embodiment are the same as those of the information processing system  1 A of the first preferred embodiment. Therefore the same parts are shown by the same reference characters and the same explanations are not repeated. 
     &lt;(2-2) Functions of Image Processing Apparatus&gt; 
       FIG. 21  is a block diagram illustrating the functional configuration implemented in the controller  300 B of the image processing apparatus  3 B. As show in  FIG. 21 , as the functional configuration, the controller  300 B has an image obtaining block  310 B, a first rectification block  311 B, a second rectification block  312 B, a first region setting block  321 B, a second region setting block  322 B, a first pixel value distribution generating block  331 B, a second pixel value distribution generating block  332 B, a first matching point deriving block  341 B, a second matching point deriving block  342 B, a reliability comparing block  343 B, a matching point detecting block  344 B, an arrangement change detecting block  350 B, and a distance calculating block  360 B. The functional configuration will be described below. 
     &lt;(2-2-1) Image Obtaining Block  310 B&gt; 
     Like the image obtaining block  310  of the first preferred embodiment, the image obtaining block  310 B obtains a reference image group G 1  by time-sequentially imaging an object OB from a first viewpoint with the first camera  21 , and a target image group G 2  by time-sequentially imaging the object OB from a second viewpoint with the second camera  22 . 
     &lt;(2-2-2) First Rectification Block  311 B&gt; 
     From a plurality of reference images G 11  to G 17  of the reference image group G 1 , the first rectification block  311 B generates a plurality of images (reference Y-direction parallax suppressed images) G 11   x  to G 17   x  in which parallax in Y direction is suppressed.  FIG. 22  is a diagram illustrating a specific example of an image group (reference Y-direction parallax suppressed image group) G 1   x  formed of a plurality of reference Y-direction parallax suppressed images G 11   x  to G 17   x.    
     Also, from a plurality of target images G 21  to G 27  of the target image group G 2 , the first rectification block  311 B generates a plurality of images (target Y-direction parallax suppressed images) G 21   x  to G 27   x  in which parallax in Y direction is suppressed.  FIG. 23  is a diagram illustrating a specific example of an image group (target Y-direction parallax suppressed image group) G 2   x  formed of a plurality of target Y-direction parallax suppressed images G 21   x  to G 27   x.    
     In the description below, the processing in which the first rectification block  311 B generates a plurality of reference Y-direction parallax suppressed images G 11   x  to G 17   x  from a plurality of reference images G 11  to G 17  and generates a plurality of target Y-direction parallax suppressed images G 21   x  to G 27   x  from a plurality of target images G 21  to G 27  is referred to as “a first rectification processing” as needed. 
     Now, an example of a method of suppressing parallax in Y direction in the plurality of reference images G 11  to G 17  and the plurality of target images G 21  to G 27  will be briefly described. 
     A stereo camera in which the first camera  21  and the second camera  22  are separated only in x direction (rectification stereo camera) has, as shown in the expressions (3) and (4) below, a relation in which a rectification matrix S 1  is added to the right side of the expression (1), and a rectification matrix S 2  is added to the right side of the expression (2). 
                   Expression   ⁢           ⁢   7                             λ   ⁢       m   ~     1       =         S   1     ·     P   1     ·     M   ~       =       P     1   ⁢   a       ·     M   ~                 (   3   )                 λ   ⁢       m   ~     2       =         S   2     ·     P   2     ·     M   ~       =       P     2   ⁢   a       ·     M   ~                 (   4   )                 P     1   ⁢   a       =       (         α       γ       u           0       β       v           0       0       1         )     ⁢     (                                 0                       R       0                                   0         )               (   5   )                 P     2   ⁢   a       =       (         α       γ       u           0       β       v           0       0       1         )     ⁢     (                                 B                       R       0                                   0         )               (   6   )               
Where S 1  and S 2  are rectification matrices.
 
     As shown by the expression (3) above, when the product of the rectification matrix S 1  and the projection matrix P 1  is a projection matrix P 1a , the projection matrix P 1a  is represented by the expression (5) above. Also, as shown by the expression (4), when the product of the rectification matrix S 2  and the projection matrix P 2  is a projection matrix P 2a , the projection matrix P 2a  is represented by the expression (6). In the expressions (5) and (6), R represents a rotation component about the arrangement of the first and second cameras  21  and  22  on the basis of the perpendicular three axes, and B represents a translation x component corresponding to the base length in x direction between the first camera  21  and the second camera  22 . Also, in the expressions (5) and (6) above, assuming that the internal setting conditions in the first camera  21  and the second camera  22  are identical, α 1  and α 2  have the same value α, β 1  and β 2  have the same value β, γ 1  and γ 2  have the same value γ, u 1  and u 2  have the same value u, and v 1  and v 2  have the same value v. 
     Then, between the projection matrix P 1a  and the projection matrix P 2a , there is only a difference of an external parameter, translation x component B about the first and second cameras  21  and  22 . 
     When the three-dimensional coordinates are rotated such that the external parameter rotation component is an identity matrix, the projection matrix P 1aa  about the first camera  21  of the rectification stereo camera is represented by the expression (7) below, and the projection matrix P 2aa  about the second camera  22  of the rectification stereo camera is represented by the expression (8) below. 
     
       
         
           
             
               
                 
                   
                       
                   
                   ⁢ 
                   
                     Expression 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     8 
                   
                 
               
               
                 
                     
                 
               
             
             
               
                 
                   
                     P 
                     
                       1 
                       ⁢ 
                       aa 
                     
                   
                   = 
                   
                     
                       
                         P 
                         
                           1 
                           ⁢ 
                           a 
                         
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             
                               
                                 R 
                                 T 
                               
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               1 
                             
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         
                           ( 
                           
                             
                               
                                 α 
                               
                               
                                 γ 
                               
                               
                                 u 
                               
                             
                             
                               
                                 0 
                               
                               
                                 β 
                               
                               
                                 v 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                             
                           
                           ) 
                         
                         ⁢ 
                         
                           ( 
                           
                             
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                             
                           
                           ) 
                         
                       
                       = 
                       
                         ( 
                         
                           
                             
                               α 
                             
                             
                               γ 
                             
                             
                               u 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               β 
                             
                             
                               v 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
             
               
                 
                   
                     P 
                     
                       2 
                       ⁢ 
                       aa 
                     
                   
                   = 
                   
                     
                       
                         P 
                         
                           2 
                           ⁢ 
                           a 
                         
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             
                               
                                 R 
                                 T 
                               
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               1 
                             
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         
                           ( 
                           
                             
                               
                                 α 
                               
                               
                                 γ 
                               
                               
                                 u 
                               
                             
                             
                               
                                 0 
                               
                               
                                 β 
                               
                               
                                 v 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                             
                           
                           ) 
                         
                         ⁢ 
                         
                           ( 
                           
                             
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 B 
                               
                             
                             
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                             
                           
                           ) 
                         
                       
                       = 
                       
                         ( 
                         
                           
                             
                               α 
                             
                             
                               γ 
                             
                             
                               u 
                             
                             
                               
                                 B 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 α 
                               
                             
                           
                           
                             
                               0 
                             
                             
                               β 
                             
                             
                               v 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
     Then, by using the 3×4 projection matrices P 1  and P 2  obtained by the calibration described in the first preferred embodiment, the rectification matrices S 1  and S 2  are calculated. 
     Specifically, the relations of expressions (9) and (10) below hold when the rectification matrix S 1  is a 3×3 matrix including 9 parameters s 1   11 , s 1   12 , s 1   13 , s 1   21 , s 1   22 , s 1   23 , s 1   31 , s 1   32 , s 1   33  and the rectification matrix S 2  is a 3×3 matrix including 9 parameters s 2   11 , s 2   12 , s 2   13 , s 2   21 , s 2   22 , s 2   23 , s 2   31 , s 2   32 , s 2   33 . 
     
       
         
           
             
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   9 
                 
               
               
                 
                     
                 
               
             
             
               
                 
                   
                     
                       S 
                       1 
                     
                     · 
                     
                       P 
                       1 
                     
                   
                   = 
                   
                     
                       
                         ( 
                         
                           
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   11 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   12 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   13 
                                 
                               
                             
                           
                           
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   21 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   22 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   23 
                                 
                               
                             
                           
                           
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   31 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   32 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   1 
                                   33 
                                 
                               
                             
                           
                         
                         ) 
                       
                       · 
                       
                         P 
                         1 
                       
                     
                     = 
                     
                       
                         ( 
                         
                           
                             
                               α 
                             
                             
                               γ 
                             
                             
                               u 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               β 
                             
                             
                               v 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                           
                         
                         ) 
                       
                       = 
                       
                         P 
                         
                           1 
                           ⁢ 
                           aa 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       S 
                       2 
                     
                     · 
                     
                       P 
                       2 
                     
                   
                   = 
                   
                     
                       
                         ( 
                         
                           
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   11 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   12 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   13 
                                 
                               
                             
                           
                           
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   21 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   22 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   23 
                                 
                               
                             
                           
                           
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   31 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   32 
                                 
                               
                             
                             
                               
                                 s 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   2 
                                   33 
                                 
                               
                             
                           
                         
                         ) 
                       
                       · 
                       
                         P 
                         2 
                       
                     
                     = 
                     
                       
                         ( 
                         
                           
                             
                               α 
                             
                             
                               γ 
                             
                             
                               u 
                             
                             
                               
                                 B 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 α 
                               
                             
                           
                           
                             
                               0 
                             
                             
                               β 
                             
                             
                               v 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                           
                         
                         ) 
                       
                       = 
                       
                         P 
                         
                           2 
                           ⁢ 
                           aa 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
     Here, the number of unknown parameters of the rectification matrix S 1  is 9, the number of unknown parameters of the rectification matrix S 2  is 9, and the number of other unknown parameters (α, β, γ, u, v, B) is 6, and so the total number of unknown parameters is 24. Then, when the 3×4 projection matrices P 1  and P 2  obtained by the calibration are applied to the expressions (9) and (10) above, 24 expressions are obtained, and the numerical values of the 24 unknown parameters are derived as the 24 expressions are solved. The rectification matrices S 1  and S 2  are thus obtained. 
     By using the rectification matrices S 1  and S 2  thus obtained, the two-dimensional coordinates (x 1 , y 1 ) in the reference images G 11  to G 17  and the two-dimensional coordinates (X 1 , Y 1 ) in the reference Y-direction parallax suppressed images G 11   x  to G 17   x  have the relation of the expression (11) below. Also, the two-dimensional coordinates (x 2 , y 2 ) in the target images G 21  to G 27  and the two-dimensional coordinates (X 2 , Y 2 ) in the target Y-direction parallax suppressed images G 21   x  to G 27   x  have the relation of expression (12) below. 
     Expression 10
 
( X   1   ,Y   1 ,1) T   =S   1 ·( x   1   ,y   1 ,1) T   (11)
 
( X   2   ,Y   2 ,1) T   =S   2 ·( x   2   ,y   2 ,1) T   (12)
 
     Thus, by using the expressions (11) and (12), the reference Y-direction parallax suppressed images G 11   x  to G 17   x , where parallax in Y direction is suppressed, are generated from the reference images G 11  to G 17 , and the target Y-direction parallax suppressed images G 21   x  to G 27   x , where parallax in Y direction is suppressed, are generated from the target images G 21  to G 27 . 
     &lt;(2-2-3) Second Rectification Block  312 B&gt; 
     From the plurality of reference images G 11  to G 17  of the reference image group G 1 , the second rectification block  312 B generates a plurality of images (reference X-direction parallax suppressed images) G 11   y  to G 17   y  in which parallax in X direction is suppressed. Then, as shown in  FIG. 22 , the plurality of reference X-direction parallax suppressed images G 11   y  to G 17   y  form an image group (reference X-direction parallax suppressed image group) G 1   y.    
     Also, from the plurality of target images G 21  to G 27  of the target image group G 2 , the second rectification block  312 B generates a plurality of images (target X-direction parallax suppressed images) G 21   y  to G 27   y  in which parallax in X direction is suppressed. Then, as shown in  FIG. 23 , the plurality of target X-direction parallax suppressed images G 21   y  to G 27   y  form an image group (target X-direction parallax suppressed image group) G 2   y.    
     In the description below, the processing in which the second rectification block  312 B generates a plurality of reference X-direction parallax suppressed images G 11   y  to G 17   y  from a plurality of reference images G 11  to G 17  and generates a plurality of target X-direction parallax suppressed images G 21   y  to G 27   y  from a plurality of target images G 21  to G 27  is referred to as “a second rectification processing” as needed. 
     As to the method of suppressing parallax in X direction in the plurality of reference images G 11  to G 17  and the plurality of target images G 21  to G 27 , a method like that of suppressing parallax in Y direction described above can be adopted, for example. However, while the external parameter, translation x component in expression (6) is B in the Y-direction parallax suppressing method, it is necessary in the X-direction parallax suppressing method to change the external parameter, translation y component in expression (6) to B, and to change the x component to 0. 
     &lt;(2-2-4) First Region Setting Block  321 B&gt; 
     The first region setting block  321 B includes a reference region setting block  3211 B and a comparison region setting block  3212 B. The reference region setting block  3211 B sets reference regions as one-dimensional windows respectively in the reference Y-direction parallax suppressed images G 11   x  to G 17   x  generated by the first rectification block  311 B. The comparison region setting block  3212 B sets comparison regions as one-dimensional windows respectively in the target Y-direction parallax suppressed images G 21   x  to G 27   x  generated by the first rectification block  311 B. 
     Specifically, as shown in  FIG. 24 , a one-dimensional reference region BR 11   x  that includes a reference point P ref   1   x  as a center point and that is about a pixel string along X direction is set in the reference Y-direction parallax suppressed image G 11   x . Also, as shown in  FIG. 25 , a one-dimensional comparison region CR 21   x  about a pixel string along X direction is set in the target Y-direction parallax suppressed image G 21   x.    
     The reference region BR 11   x  and the comparison region CR 21   x  have a corresponding form and size (the same form and size), and the position of setting of the reference region BR 11   x  in the reference Y-direction parallax suppressed image G 11   x  and the position of setting of the comparison region CR 21   x  in the target Y-direction parallax suppressed image G 21   x  have a positional relation that is shifted in X direction by an amount corresponding to the amount of shift L bx  in x direction between the optical axis of the imaging lens  21 L and the optical axis of the imaging lens  22 L. 
     In this way, the first region setting block  321 B sets the reference region BR 11   x  and the comparison region CR 21   x  in the first image set formed of the reference Y-direction parallax suppressed image G 11   x  and target Y-direction parallax suppressed image G 21   x.    
     Also, as shown in  FIG. 24 , reference regions BR 12   x  to BR 17   x  are set respectively in the reference Y-direction parallax suppressed images G 12   x  to G 17   x , and as shown in  FIG. 25 , comparison regions CR 22   x  to CR 27   x  are set respectively in the target Y-direction parallax suppressed images G 22   x  to G 27   x.    
     Here, the reference regions BR 12   x  to BR 17   x  have the same form and size as the reference region BR 11   x , and the positional relation of the reference region BR 11   x  in the reference Y-direction parallax suppressed image G 11   x  and the positional relation of the reference regions BR 12   x  to BR 17   x  in the reference Y-direction parallax suppressed images G 12   x  to G 17   x  are the same. Also, the comparison regions CR 22   x  to CR 27   x  have the same form and size as the comparison region CR 21   x , and the positional relation of the comparison region CR 21   x  in the target Y-direction parallax suppressed image G 21   x  and the positional relation of the comparison regions CR 22   x  to CR 27   x  in the target Y-direction parallax suppressed images G 22   x  to G 27   x  are the same. 
     In this way, the first region setting block  321 B sets the reference regions BR 11   x  to BR 17   x  including the reference point P ref   1   x  respectively in the reference Y-direction parallax suppressed images G 11   x  to G 17   x  with the same arrangement, and also sets the comparison regions CR 21   x  to CR 27   x , corresponding to the form and size of the reference regions BR 11   x  to BR 17   x , respectively in the target Y-direction parallax suppressed images G 21   x  to G 27   x  with the same arrangement. 
     For the reference Y-direction parallax suppressed images G 11   x  to G 17   x  and the target Y-direction parallax suppressed images G 21   x  to G 27   x , the first viewpoint and the second viewpoint are artificially arranged as being separated only in x direction as the result of the rectification, and so the elongate direction of the plurality of reference regions BR 11   x  to BR 17   x  and the plurality of comparison regions CR 21   x  to CR 27   x  is set in X direction corresponding to x direction. 
     &lt;(2-2-5) First Pixel Value Distribution Generating Block  331 B&gt; 
     The first pixel value distribution generating block  331 B generates one pixel value distribution (reference distribution) about two-dimensional space from the distributions of pixel values about the plurality of reference regions BR 11   x  to BR 17   x , and also generates one pixel value distribution (comparison distribution) about two-dimensional space from the distributions of pixel values about the plurality of comparison regions CR 21   x  to CR 27   x.    
     Specifically, as shown in  FIG. 26 , first, pixel strings Bi 11   x  to Bi 17   x  about the reference regions BR 11   x  to BR 17   x  are extracted from the reference Y-direction parallax suppressed images G 11   x  to G 17   x . Here, the pixel strings Bi 11   x  to Bi 17   x  indicate the distributions of pixel values (pixel value distributions) about the reference regions BR 11   x  to BR 17   x . Specifically, the pixel value distribution Bi 11   x  about the reference region BR 11   x  is extracted from the reference Y-direction parallax suppressed image G 11   x , the pixel value distribution Bi 12   x  about the reference region BR 12   x  is extracted from the reference Y-direction parallax suppressed image G 12   x , the pixel value distribution Bi 13   x  about the reference region BR 13   x  is extracted from the reference Y-direction parallax suppressed image G 13   x , the pixel value distribution Bi 14   x  about the reference region BR 14   x  is extracted from the reference Y-direction parallax suppressed image G 14   x , the pixel value distribution Bi 15   x  about the reference region BR 15   x  is extracted from the reference Y-direction parallax suppressed image G 15   x , the pixel value distribution Bi 16   x  about the reference region BR 16   x  is extracted from the reference Y-direction parallax suppressed image G 16   x , and the pixel value distribution Bi 17   x  about the reference region BR 17   x  is extracted from the reference Y-direction parallax suppressed image G 17   x.    
     Next, the pixel value distributions Bi 11   x  to Bi 17   x  are arranged according to given arrangement rules, so as to generate a two-dimensional pixel value distribution (reference distribution) D PV   1   x . The given arrangement rules are, for example, that the pixel value distributions Bi 11   x  to Bi 17   x  are arranged parallel to each other, and that the pixel value distribution Bi 11   x , pixel value distribution Bi 12   x , pixel value distribution Bi 13   x , pixel value distribution Bi 14   x , pixel value distribution Bi 15   x , pixel value distribution Bi 16   x , and pixel value distribution Bi 17   x  are arranged in this order and integrated, so as to form a pixel value distribution about a rectangular region. The rectangular reference distribution D PV   1   x  is thus generated. 
     Also, as shown in  FIG. 27 , first, pixel strings Ci 21   x  to Ci 27   x  about the comparison regions CR 21   x  to CR 27   x  are extracted from the target Y-direction parallax suppressed images G 21   x  to G 27   x . Here, the pixel strings Ci 21   x  to Ci 27   x  indicate the distributions of pixel values (pixel value distributions) about the comparison regions CR 21   x  to CR 27   x . Specifically, the pixel value distribution Ci 21   x  about the comparison region CR 21   x  is extracted from the target Y-direction parallax suppressed image G 21   x , the pixel value distribution Ci 22   x  about the comparison region CR 22   x  is extracted from the target Y-direction parallax suppressed image G 22   x , the pixel value distribution Ci 23   x  about the comparison region CR 23   x  is extracted from the target Y-direction parallax suppressed image G 23   x , the pixel value distribution Ci 24   x  about the comparison region CR 24   x  is extracted from the target Y-direction parallax suppressed image G 24   x , the pixel value distribution Ci 25   x  about the comparison region CR 25   x  is extracted from the target Y-direction parallax suppressed image G 25   x , the pixel value distribution Ci 26   x  about the comparison region CR 26   x  is extracted from the target Y-direction parallax suppressed image G 26   x , and the pixel value distribution Ci 27   x  about the comparison region CR 27   x  is extracted from the target Y-direction parallax suppressed image G 27   x.    
     Next, the pixel value distributions Ci 21   x  to Ci 27   x  are arranged according to given arrangement rules, so as to generate a two-dimensional pixel value distribution (comparison distribution) D PV   2   x . The given arrangement rules are, for example, that the pixel value distributions Ci 21   x  to Ci 27   x  are arranged parallel to each other, and that the pixel value distribution Ci 21   x , pixel value distribution Ci 22   x , pixel value distribution Ci 23   x , pixel value distribution Ci 24   x , pixel value distribution Ci 25   x , pixel value distribution Ci 26   x , and pixel value distribution Ci 27   x  are arranged in this order and integrated, so as to form a pixel value distribution about a rectangular region. The rectangular comparison distribution D PV   2   x  is thus generated. 
     In the generation of the reference distribution D PV   1   x  and the comparison distribution D PV   2   x , another order of arrangement may be adopted as long as the order of arrangement of the pixel value distributions Bi 11   x  to Bi 17   x  and the order of arrangement of the pixel value distributions Ci 21   x  to Ci 27   x  correspond to each other, for example. 
     &lt;(2-2-6) First Matching Point Deriving Block  341 B&gt; 
     By using the reference distribution D PV   1   x  and the comparison distribution D PV   2   x , the first matching point deriving block  341 B calculates a matching point corresponding to the reference point P ref   1   x  in the target Y-direction parallax suppressed image G 21   x , and a reliability about that matching point. 
       FIG. 28  is a diagram illustrating a matching point search using phase only correlation in the first matching point deriving block  341 B. In the matching point search in the first matching point deriving block  341 B, as compared with the matching point search using phase only correlation of the first preferred embodiment, the combination of image regions used for the matching point search (specifically, the combination of the image region of the reference distribution D PV   1  and the image region of the comparison distribution D PV   2 ) is changed to a different combination of image regions (specifically, the combination of the image region of the reference distribution D PV   1   x  and the image region of the comparison distribution D PV   2   x ). 
     In the matching point search in the first matching point deriving block  341 B, a distribution of POC value indicating the correlation between the reference distribution D PV   1   x  and the comparison distribution D PV   2   x  is obtained. Then, the position corresponding to the peak of POC value in the image region of the comparison distribution D PV   2   x  is derived as a matching point on the target Y-direction parallax suppressed image G 21   x  that corresponds to the center point (reference point) P ref   1   x  of the reference region BR 11   x  on the reference Y-direction parallax suppressed image G 11   x.    
     By the matching point search using phase only correlation, as shown in  FIG. 29 , for example, a point P cor   2   x  shifted from the center point P cent   2   x  of the comparison distribution D PV   2   x  is detected as a point corresponding to the peak of POC value. Then, in the target Y-direction parallax suppressed images G 21   x  to G 27   x , points in a positional relation like the positional relation between the center point P cent   2   x  and the point P cor    2   x  are derived as matching points, on the basis of the center points of the comparison regions CR 21   x  to CR 27   x . Also, the peak value of POC value about the point P cor   2   x  is provided as an index (reliability) indicating the reliability as a matching point. 
     In the description below, a matching point derived by the first matching point deriving block  341 B is referred to as “a first matching point” as needed, and the reliability about the first matching point is referred to as “a first reliability” as needed. Also, the processing of searching for a matching point by the series of operations by the first region setting block  321 B, the first pixel value distribution generating block  331 B, and the first matching point deriving block  341 B, is referred to as “first matching point search processing” as needed. 
     &lt;(2-2-7) Second Region Setting Block  322 B&gt; 
     The second region setting block  322 B includes a reference region setting block  3221 B and a comparison region setting block  3222 B. The reference region setting block  3221 B sets reference regions as one-dimensional windows respectively in the reference X-direction parallax suppressed images G 11   y  to G 17   y  generated by the second rectification block  312 B. The comparison region setting block  3222 B sets comparison regions as one-dimensional windows respectively in the target X-direction parallax suppressed images G 21   y  to G 27   y  generated by the second rectification block  312 B. 
     Specifically, as shown in  FIG. 30 , a one-dimensional reference region BR 11   y  that includes a reference point P ref   1   y  as a center point and that is about a pixel string along Y direction is set in the reference X-direction parallax suppressed image G 11   y . Also, as shown in  FIG. 31 , a one-dimensional comparison region CR 21   y  about a pixel string along Y direction is set in the target X-direction parallax suppressed image G 21   y.    
     The reference region BR 11   y  and the comparison region CR 21   y  have a corresponding form and size (the same form and size), and the position of setting of the reference region BR 11   y  in the reference X-direction parallax suppressed image G 11   y  and the position of setting of the comparison region CR 21   y  in the target X-direction parallax suppressed image G 21   y  have a positional relation that is shifted in Y direction by an amount corresponding to the amount of shift L by  in y direction between the optical axis of the imaging lens  21 L and the optical axis of the imaging lens  22 L. 
     In this way, the second region setting block  322 B sets the reference region BR 11   y  and the comparison region CR 21   y  in the first image set formed of the reference X-direction parallax suppressed image G 11   y  and the target X-direction parallax suppressed image G 21   y.    
     Also, as shown in  FIG. 30 , reference regions BR 12   y  to BR 17   y  are set respectively in the reference X-direction parallax suppressed images G 12   y  to G 17   y , and as shown in  FIG. 31 , comparison regions CR 22   y  to CR 27   y  are set respectively in the target X-direction parallax suppressed images G 22   y  to G 27   y.    
     Here, the reference regions BR 12   y  to BR 17   y  have the same form and size as the reference region BR 11   y , and the positional relation of the reference region BR 11   y  in the reference X-direction parallax suppressed image G 11   y  and the positional relation of the reference regions BR 12   y  to BR 17   y  in the reference X-direction parallax suppressed images G 12   y  to G 17   y  are the same. Also, the comparison regions CR 22   y  to CR 27   y  have the same form and size as the comparison region CR 21   y , and the positional relation of the comparison region CR 21   y  in the target X-direction parallax suppressed image G 21   y  and the positional relation of the comparison regions CR 22   y  to CR 27   y  in the target X-direction parallax suppressed images G 22   y  to G 27   y  are the same. 
     In this way, the second region setting block  322 B sets the reference regions BR 11   y  to BR 17   y  including the reference point P ref   1   y  respectively in the reference X-direction parallax suppressed images G 11   y  to G 17   y  with the same arrangement, and also sets the comparison regions CR 21   y  to CR 27   y , corresponding to the form and size of the reference regions BR 11   y  to BR 17   y , respectively in the target X-direction parallax suppressed images G 21   y  to G 27   y  with the same arrangement. 
     For the reference X-direction parallax suppressed images G 11   y  to G 17   y  and the target X-direction parallax suppressed images G 21   y  to G 27   y , the first viewpoint and the second viewpoint are artificially arranged as being separated only in y direction as the result of the rectification, and so the elongate direction of the plurality of reference regions BR 11   y  to BR 17   y  and the plurality of comparison regions CR 21   y  to CR 27   y  is set in Y direction corresponding to y direction. 
     &lt;(2-2-8) Second Pixel Value Distribution Generating Block  332 B&gt; 
     The second pixel value distribution generating block  332 B generates one pixel value distribution (reference distribution) about two-dimensional space from the distributions of pixel values about the plurality of reference regions BR 11   y  to BR 17   y , and also generates one pixel value distribution (comparison distribution) about two-dimensional space from the pixel value distributions about the plurality of comparison regions CR 21   y  to CR 27   y.    
     Specifically, as shown in  FIG. 32 , first, pixel strings Bi 11   y  to Bi 17   y  about the reference regions BR 11   y  to BR 17   y  are extracted from the reference X-direction parallax suppressed images G 11   y  to G 17   y . Here, the pixel strings Bi 11   y  to Bi 17   y  indicate the distributions of pixel values (pixel value distributions) about the reference regions BR 11   y  to BR 17   y . Specifically, the pixel value distribution Bi 11   y  about the reference region BR 11   y  is extracted from the reference X-direction parallax suppressed image G 11   y , the pixel value distribution Bi 12   y  about the reference region BR 12   y  is extracted from the reference X-direction parallax suppressed image G 12   y , the pixel value distribution Bi 13   y  about the reference region BR 13   y  is extracted from the reference X-direction parallax suppressed image G 13   y , the pixel value distribution Bi 14   y  about the reference region BR 14   y  is extracted from the reference X-direction parallax suppressed image G 14   y , the pixel value distribution Bi 15   y  about the reference region BR 15   y  is extracted from the reference X-direction parallax suppressed image G 15   y , the pixel value distribution Bi 16   y  about the reference region BR 16   y  is extracted from the reference X-direction parallax suppressed image G 16   y , and the pixel value distribution Bi 17   y  about the reference region BR 17   y  is extracted from the reference X-direction parallax suppressed image G 17   y.    
     Next, the pixel value distributions Bi 11   y  to Bi 17   y  are arranged according to given arrangement rules, so as to generate a two-dimensional pixel value distribution (reference distribution) D PV   1   y . The given arrangement rules are, for example, that the pixel value distributions Bi 11   y  to Bi 17   y  are arranged parallel to each other, and that the pixel value distribution Billy, pixel value distribution Billy, pixel value distribution Bi 13   y , pixel value distribution Bi 14   y , pixel value distribution Bi 15   y , pixel value distribution Bi 16   y , and pixel value distribution Bi 11   y  are arranged in this order and integrated, so as to form a pixel value distribution about a rectangular region. The rectangular reference distribution D PV   1   y  is thus generated. 
     Also, as shown in  FIG. 33 , first, pixel strings Ci 21   y  to Ci 27   y  about the comparison regions CR 21   y  to CR 27   y  are extracted from the target X-direction parallax suppressed images G 21   y  to G 27   y . Here, the pixel strings Ci 21   y  to Ci 27   y  indicate the distributions of pixel values (pixel value distributions) about the comparison regions CR 21   y  to CR 27   y . Specifically, the pixel value distribution Ci 21   y  about the comparison region CR 21   y  is extracted from the target X-direction parallax suppressed image G 21   y , the pixel value distribution Ci 22   y  about the comparison region CR 22   y  is extracted from the target X-direction parallax suppressed image G 22   y , the pixel value distribution Ci 23   y  about the comparison region CR 23   y  is extracted from the target X-direction parallax suppressed image G 23   y , the pixel value distribution Ci 24   y  about the comparison region CR 24   y  is extracted from the target X-direction parallax suppressed image G 24   y , the pixel value distribution Ci 25   y  about the comparison region CR 25   y  is extracted from the target X-direction parallax suppressed image G 25   y , the pixel value distribution Ci 26   y  about the comparison region CR 26   y  is extracted from the target X-direction parallax suppressed image G 26   y , and the pixel value distribution Ci 27   y  about the comparison region CR 27   y  is extracted from the target X-direction parallax suppressed image G 27   y.    
     Next, the pixel value distributions Ci 21   y  to Ci 27   y  are arranged according to given arrangement rules, so as to generate a two-dimensional pixel value distribution (comparison distribution) D PV   2   y . The given arrangement rules are, for example, that the pixel value distributions Ci 21   y  to Ci 27   y  are arranged parallel to each other, and that the pixel value distribution Ci 21   y , pixel value distribution Ci 22   y , pixel value distribution Ci 23   y , pixel value distribution Ci 24   y , pixel value distribution Ci 25   y , pixel value distribution Ci 26   y , and pixel value distribution Ci 27   y  are arranged in this order and integrated, so as to form a pixel value distribution about a rectangular region. The rectangular comparison distribution D PV   2   y  is thus generated. 
     In the generation of the reference distribution D PV   1   y  and the comparison distribution D PV   2   y , another order of arrangement may be adopted as long as the order of arrangement of the pixel value distributions Bi 11   y  to Bi 17   y  and the order of arrangement of the pixel value distributions Ci 21   y  to Ci 27   y  correspond to each other, for example. 
     &lt;(2-2-9) Second Matching Point Deriving Block  342 B&gt; 
     By using the reference distribution D PV   1   y  and the comparison distribution D PV   2   y , the second matching point deriving block  342 B calculates a matching point corresponding to the reference point P ref   1   y  in the target X-direction parallax suppressed image G 21   y , and a reliability about that matching point. 
       FIG. 34  is a diagram illustrating a matching point search using phase only correlation in the second matching point deriving block  342 B. In the matching point search in the second matching point deriving block  342 B, as compared with the matching point search using phase only correlation of the first preferred embodiment, the combination of image regions used for the matching point search (specifically, the combination of the image region of the reference distribution D PV   1  and the image region of the comparison distribution D PV   2 ) is changed to a different combination of image regions (specifically, the combination of the image region of the reference distribution D PV   1   y  and the image region of the comparison distribution D PV   2   y ). 
     In the matching point search in the second matching point deriving block  342 B, a distribution of POC value indicating the correlation between the reference distribution D PV   1   y  and the comparison distribution D PV   2   y  is obtained. Then, the position corresponding to the peak of POC value in the image region of the comparison distribution D PV   2   y  is derived as a matching point on the target X-direction parallax suppressed image G 21   y  that corresponds to the center point (reference point) P ref   1   y  of the reference region BR 11   y  on the reference X-direction parallax suppressed image G 11   y.    
     By the matching point search using phase only correlation, as shown in  FIG. 35 , for example, a point P cor   2   y  shifted from the center point P cent   2   y  of the comparison distribution D PV   2   y  is detected as a point corresponding to the peak of POC value. Then, in the target X-direction parallax suppressed images G 21   y  to G 27   y , points in a positional relation like the positional relation between the center point P cent   2   y  and the point P cor   2   y  are derived as matching points, on the basis of the center points of the comparison regions CR 21   y  to CR 27   y . Also, the peak value of POC value about the point P cor   2   y  is provided as an index (reliability) indicating the reliability as a matching point. 
     In the description below, a matching point derived by the second matching point deriving block  342 B is referred to as “a second matching point” as needed, and the reliability about the second matching point is referred to as “a second reliability” as needed. Also, the processing of searching for a matching point by the series of operations by the second region setting block  322 B, the second pixel value distribution generating block  332 B, and the second matching point deriving block  342 B, is referred to as “second matching point search processing” as needed. 
     &lt;(2-2-10) Reliability Comparing Block  343 B&gt; 
     The reliability comparing block  343 B compares the first reliability calculated by the first matching point deriving block  341 B and the second reliability calculated by the second matching point deriving block  342 B and recognizes a relatively larger reliability. The combination of the first reliability and the second reliability to be compared is a combination of the first reliability about the matching point of the reference point P ref   1   x  corresponding to one pixel in the reference image G 11  and the second reliability about the matching point of the reference point P ref   1   y  corresponding to that one pixel. 
     &lt;(2-2-11) Matching Point Detecting Block  344 B&gt; 
     On the basis of the result of comparison in the reliability comparing block  343 B, the matching point detecting block  344 B adopts one of the combination of the reference point P ref   1   x  and the first matching point and the combination of the reference point P ref   1   y  and the second matching point. For example, when the first reliability is higher than the second reliability, the combination of the reference point P ref   1   x  and the first matching point is adopted. On the other hand, when the second reliability is higher than the first reliability, the combination of the reference point P ref   1   y  and the second matching point is adopted. 
     In the matching point detecting block  344 B, when the combination of the reference point P ref   1   x  and the first matching point is adopted, coordinate transformation is performed according to the expressions (11) and (12), and the relation between the reference point in the reference image group G 1  and the matching point in the target image group G 2  corresponding to that reference point is obtained. On the other hand, when the combination of the reference point P ref   1   y  and the second matching point is adopted, coordinate transformation is performed according to the expressions (11) and (12), and the relation between the reference point in the reference image group G 1  and the matching point in the target image group G 2  corresponding to that reference point is obtained. Thus, matching points on the plurality of reference images G 21  to G 27  that correspond to the reference point on the plurality of reference images G 11  to G 17  are detected. 
     &lt;(2-2-12) Arrangement Change Detecting Block  3508 &gt; 
     The arrangement change detecting block  350 B detects a change of the condition of arrangement of the first camera  21  and the second camera  22  by recognizing that the relation between the position of the reference point P ref   1   x  in the plurality of reference Y-direction parallax suppressed images G 11   x  to G 17   x  and the position of the first matching point in the plurality of target Y-direction parallax suppressed images G 21   x  to G 27   x  has changed in Y direction. Also, the arrangement change detecting block  350 B detects a change of the condition of arrangement of the first camera  21  and the second camera  22  by recognizing that the relation between the position of the reference point P ref   1   y  in the plurality of reference X-direction parallax suppressed images G 11   y  to G 17   y  and the position of the second matching point in the plurality of target X-direction parallax suppressed images G 21   y  to G 27   y  has changed in X direction. That is to say, the arrangement change detecting block  350 B detects a change of the condition of arrangement about the relative arrangement of the first viewpoint and the second viewpoint. 
     For example, as shown in  FIG. 29 , in the comparison distribution D PV   2   x , when the amount of shift in Y direction between the center point P cent   2   x  and the point P cor   2   x  has changed from a predetermined reference amount of shift (reference amount of shift), the arrangement change detecting block  350 B detects a change of the condition of arrangement. Also, as shown in  FIG. 35 , in the comparison distribution D PV   2   y , when the amount of shift in X direction between the center point P cent   2   y  and the point P cor   2   y  has changed from a predetermined reference amount of shift (reference amount of shift), the arrangement change detecting block  350 B detects a change of the condition of arrangement. 
     When the arrangement change detecting block  350 B detects a change of the condition of arrangement, the display  302  visually outputs a display element indicating that an adjustment operation (calibration) about the change of the condition of arrangement should be performed. The reference amount of shift in Y direction is the amount of shift in Y direction between the center point P cent   2   x  and the point P cor   2   x  when calibration is performed, and the reference amount of shift in X direction is the amount of shift in X direction between the center point P cent   2   y  and the point P cor   2   y  when calibration is performed. The reference amounts of shift in X direction and Y direction are updated and stored in the storage  304  each time calibration is performed. The method of the calibration is similar to that described in the first preferred embodiment. 
     &lt;(2-2-13) Distance Calculating Block  360 B&gt; 
     The distance deriving block  360 B derives the distance from the stereo camera  2 B to the portion of the object OB that corresponds to the reference point on the basis of the combination of the coordinates of the reference point in the reference image G 11  and the coordinates of the matching point in the target image G 21  detected by the matching point detecting block  344 B. 
     Specifically, the transform data D trans  obtained by the calibration and stored in the storage  304  (specifically the projection matrices P 1  and P 2 ) is substituted into the expressions (1) and (2), and the coordinate values of the reference point are substituted into the expression (1) as the values of the coordinates m 1 , and the coordinate values of the matching point corresponding to the reference point are substituted into the expression (2) as the values of the coordinates m 2 . Then, the coordinates M of the three-dimensional point about the portion of the object OB that corresponds to the reference point are obtained as the two expressions are solved. Then, for example, when the three-dimensional coordinates of the stereo camera  2 B is taken as the origin, it is easy to derive the distance from the stereo camera  2 B to the portion of the object OB that corresponds to the reference point, on the basis of the obtained coordinates M. 
     The distance data L data  calculated here is stored in the storage  304  in association with the coordinates M of the corresponding three-dimensional point. 
     In this way, the distance from the stereo camera  2 B to the portion of the object OB that corresponds to the reference point is calculated on the basis of the combination of the reference point and matching point adopted in the matching point detecting block  344 B (specifically, the combination of the reference point P ref   1   x  and the first matching point, or the combination of the reference point P ref   1   y  and the second matching point). 
     &lt;(2-3) Operations of Image Processing Apparatus&gt; 
       FIGS. 36 to 38  are flowcharts illustrating the operation flow of the matching point search processing, the arrangement change detecting processing, and the distance measuring processing in the image processing apparatus  3 B. This operation flow is implemented as the program PGb is executed in the controller  300 B, and the operation flow is started as a user variously operates the operation unit  301  and the flow moves to step SP 1  in  FIG. 36 . 
     In step SP 1 , the image obtaining block  310 B obtains a reference image group G 1  and a target image group G 2 . 
     In step SP 2 , the first and second rectification blocks  311 B and  312 B perform the first and second rectification operations. That is to say, a plurality of reference Y-direction parallax suppressed images G 11   x  to G 17   x , a plurality of target Y-direction parallax suppressed images G 21   x  to G 27   x , a plurality of reference X-direction parallax suppressed images G 11   y  to G 17   y , and a plurality of target X-direction parallax suppressed images G 21   y  to G 27   y  are generated from the reference image group G 1  and the target image group G 2 . 
     In step SP 3 , the first region setting block  321 B, the first pixel value distribution generating block  331 B, and the first matching point deriving block  341 B perform the first matching point search processing. In this step SP 3 , the operation flow shown in  FIG. 37  is performed. 
     In step SP 31  in  FIG. 37 , the first region setting block  321 B sets one pixel of the reference Y-direction parallax suppressed image G 11   x  as a reference point P ref   1   x.    
     In step SP 32 , the first region setting block  321 B sets windows in the reference Y-direction parallax suppressed image G 11   x  and the target Y-direction parallax suppressed image G 21   x  that form the first image set. Here, a reference region BR 11   x  as a one-dimensional window is set in the reference Y-direction parallax suppressed image G 11   x , and a comparison region CR 21   x  as a one-dimensional window is set in the target Y-direction parallax suppressed image G 21   x.    
     In step SP 33 , the first region setting block  321 B sets windows in the reference Y-direction parallax suppressed images G 12   x  to G 17   x  and the target Y-direction parallax suppressed images G 21   x  to G 27   x  that form the second and following image sets. Here, reference regions BR 12   x  to BR 17   x , corresponding to the form, size and arrangement position of the reference region BR 11   x , are set in the reference Y-direction parallax suppressed images G 12   x  to G 17   x , and comparison regions CR 22   x  to CR 27   x , corresponding to the form, size, and arrangement position of the comparison region CR 21   x , are set in the target Y-direction parallax suppressed images G 22   x  to G 27   x.    
     In step SP 34 , the first pixel value distribution generating block  331 B generates one reference distribution D PV   1   x  about two-dimensional space from the distributions of pixel values in the plurality of reference regions BR 11   x  to BR 17   x , and also generates one comparison distribution D PV   2   x  about two-dimensional space from the distributions of pixel values in the plurality of comparison regions CR 21   x  to CR 27   x.    
     In step SP 35 , on the basis of the reference distribution D PV   1   x  and the comparison distribution D PV   2   x , the first matching point deriving block  341 B obtains a first matching point corresponding to the reference point P ref   1   x  in the target Y-direction parallax suppressed images G 21   x  to G 27   x , and a first reliability about the first matching point. 
     In step SP 4 , the second region setting block  322 B, the second pixel value distribution generating block  332 B, and the second matching point deriving block  342 B perform the second matching point search processing. In this step SP 4 , the operation flow shown in  FIG. 38  is performed. 
     In step SP 41  in  FIG. 38 , the second region setting block  322 B sets one pixel in the reference X-direction parallax suppressed image G 11   y  as a reference point P ref   1   y . The reference point P ref   1   y  set here is a point corresponding to the reference point P ref   1   x  set in step SP 31 . More specifically, the reference point P ref   1   x  set in step SP 31  and the reference point P ref   1   y  set in step SP 41  are points about the same pixel of the reference image G 11 . 
     In step SP 42 , the second region setting block  322 B sets windows in the reference X-direction parallax suppressed image G 11   y  and the target X-direction parallax suppressed image G 21   y  that form the first image set. Here, a reference region BR 11   y  as a one-dimensional window is set in the reference X-direction parallax suppressed image G 11   y , and a comparison region CR 21   y  as a one-dimensional window is set in the target X-direction parallax suppressed image G 21   y.    
     In step SP 43 , the second region setting block  322 B sets windows in the reference X-direction parallax suppressed images G 12   y  to G 17   y  and the target X-direction parallax suppressed images G 21   y  to G 27   y  that form the second and following image sets. Here, reference regions BR 12   y  to BR 17   y , corresponding to the form, size and arrangement position of the reference region BR 11   y , are set in the reference X-direction parallax suppressed images G 12   y  to G 17   y , and comparison regions CR 22   y  to CR 27   y , corresponding to the form, size, and arrangement position of the comparison region CR 21   y , are set in the target X-direction parallax suppressed images G 22   y  to G 27   y.    
     In step SP 44 , the second pixel value distribution generating block  332 B generates one reference distribution D PV   1   y  about two-dimensional space from the distributions of pixel values in the plurality of reference regions BR 11   y  to BR 17   y , and also generates one comparison distribution D PV   2   y  about two-dimensional space from the distributions of pixel values in the plurality of comparison regions CR 21   y  to CR 27   y.    
     In step SP 45 , on the basis of the reference distribution D PV   1   y  and the comparison distribution D PV   2   y , the second matching point deriving block  342 B derives a second matching point corresponding to the reference point P ref   1   y  in the target X-direction parallax suppressed images G 21   y  to G 27   y , and a second reliability about the second matching point. 
     In step SP 5 , the reliability comparing block  343 B compares the first reliability obtained in step SP 3  and the second reliability obtained in step SP 4  and recognizes a relatively larger reliability. 
     In step SP 6 , on the basis of the result of comparison in the reliability comparing block  343 B, the matching point detecting block  344 B adopts one of the combination of the reference point P ref   1   x  and the first matching point and the combination of the reference point P ref   1   y  and the second matching point, and the matching points on the plurality of target images G 21  to G 27  that correspond to the reference point on the plurality of reference images G 11  to G 17  are detected from the adopted combination. 
     In step SP 7 , on the basis of the coordinates of the reference point and the matching point detected in step SP 6 , the distance calculating block  360 B calculates the three-dimensional coordinates of the portion of the object OB that corresponds to the reference point, and the distance to the portion of the object OB that corresponds to the reference point, on the basis of the stereo camera  2 B. The data indicating the three-dimensional coordinates and distance calculated here is stored in the storage  304  in association with information indicating the reference point. 
     In step SP 8 , the arrangement change detecting block  350 B judges whether a change of the condition of arrangement about the relative arrangement of the first viewpoint and the second viewpoint has been detected. Here, a change of the condition of arrangement of the first camera  21  and the second camera  22  is detected when the amount of shift in Y direction between the position of the reference point P ref   1   x  in the plurality of reference Y-direction parallax suppressed images G 11   x  to G 17   x  and the position of the first matching point in the plurality of target Y-direction parallax suppressed images G 21   x  to G 27   x  has changed from a predetermined reference amount of shift. Also, a change of the condition of arrangement of the first camera  21  and the second camera  22  is detected when the amount of shift in X direction between the position of the reference point P ref   1   y  in the plurality of reference X-direction parallax suppressed images G 11   y  to G 17   y  and the position of the second matching point in the plurality of target X-direction parallax suppressed images G 21   y  to G 27   y  has changed from a predetermined reference amount of shift. Then, when a change of the condition of arrangement is detected, the flow moves to step SP 9 , and when a change of the condition of arrangement is not detected, the flow moves to step SP 10 . 
     In step SP 9 , according to a signal output from the arrangement change detecting block  350 B, the display  302  visually outputs a display element indicating that an adjustment operation (calibration) should be performed. The visual output of the display element indicating that the calibration should be performed may be continued until calibration is performed, for example. The information that calibration should be performed may be given by voice. 
     In step SP 10 , the first and second region setting blocks  321 B and  322 B judge whether there remains any point to be set as reference point P ref   1   x  (a point to be processed) in the reference Y-direction parallax suppressed image G 11   x , and whether there remains any point to be set as reference point P ref   1   y  (a point to be processed) in the reference X-direction parallax suppressed image G 11   y . For example, it is judged whether all pixels forming the reference Y-direction parallax suppressed image G 11   x  have been set as the reference point P ref   1   x . When a point to be processed remains, the flow returns to step SP 3 , and in step SP 3 , another pixel in the reference Y-direction parallax suppressed image G 11   x  is set as reference point P ref   1   x  and step SP 3  is performed, and also in step SP 4 , another pixel in the reference X-direction parallax suppressed image G 11   y  is set as reference point P ref   1   y  and step SP 4  is performed. On the other hand, when no point to be processed remains, this operation flow is ended. 
     According to the operation flow described above, matching points on the plurality of target images G 21  to G 27  are detected in correspondence with reference points on the reference images G 11  to G 17 . The matching points may be detected about reference points at intervals of a given number of pixels on the reference image G 11 . 
     As described above, the image processing apparatus  3 B of the second preferred embodiment performs a matching point search based on image regions where distant and near views coexist further less. Accordingly, the precision of the matching point search about a plurality of images taking the same object where distant and near views coexist is further improved. 
     &lt;(3) Modifications&gt; 
     The present invention is not limited to the above-described preferred embodiments, and various modifications and variations are possible without departing from the scope of the present invention. 
     &lt;(3-1) First Modification&gt; 
     In the first and second preferred embodiments, one reference distribution about two-dimensional space is generated from the distributions of pixel values about one-dimensional space in a plurality of reference regions, and one comparison distribution about two-dimensional space is generated from the distributions of pixel values about one-dimensional space in a plurality of comparison regions, but this is meant to be illustrative and not restrictive. 
     For example, one reference distribution about three-dimensional space may be generated from the distributions of pixel values about two-dimensional space in a plurality of reference regions, and one comparison distribution about three-dimensional space may be generated from the distributions of pixel values about two-dimensional space in a plurality of comparison regions. That is to say, the number of dimensions of the space of a reference distribution of pixel values is larger than the number of dimension(s) of the space of a plurality of reference regions, and the number of dimensions of the space of a comparison distribution of pixel values is larger than the number of dimension(s) of the space of a plurality of comparison regions. 
     Now, a specific example of a matching point search will be described in which the reference regions and comparison regions are two-dimensional regions where a plurality of pixels are arranged in both of X direction and Y direction, and the reference distribution of pixel values is a three-dimensional distribution of pixel values generated by layering the pixel value distributions in the plurality of reference regions according to given arrangement rules, and the comparison distribution of pixel values is a three-dimensional distribution of pixel values generated by layering the distributions of pixel values in the plurality of comparison regions according to given arrangement rules. 
     In an information processing system  1 C of a first modification, as compared with the information processing system  1 A of the first preferred embodiment, the image processing apparatus  3 A is changed to an information processing apparatus  3 C in which the controller  300 A is replaced by a controller  300 C having different functions. Specifically, as shown in  FIG. 4 , the region setting block  320 A of the first preferred embodiment is changed to a region setting block  320 C, the pixel value distribution generating block  330 A of the first preferred embodiment is changed to a pixel value distribution generating block  330 C, and the matching point detecting block  340 A of the first preferred embodiment is changed to a matching point detecting block  340 C. 
     In the controller  300 C, a program PGc stored in a storage  304  is read and executed to implement various functions. In other respects, the configuration of the information processing system  1 C of the first modification is the same as that of the information processing system  1 A of the first preferred embodiment. 
     Accordingly, in the information processing system  1 C of the first modification, the same parts as those of the information processing system  1 A of the first preferred embodiment will be shown by the same reference characters and will not be described again here, and parts different from those of the information processing system  1 A of the first preferred embodiment will be described. 
     The region setting block  320 C has a reference region setting block  321 C and a comparison region setting block  322 C. The reference region setting block  321 C sets reference regions, as two-dimensional windows, in individual reference images G 11  to G 17 . The comparison region setting block  322 C sets comparison regions, as two-dimensional windows, in individual target images G 21  to G 27 . 
     Specifically, as shown in  FIG. 39 , a reference region BR 11   c  that includes a reference point P ref   1  as a center point and that is approximately a rectangle extending in X direction is set in the reference image G 11 . Also, as shown in  FIG. 40 , a comparison region CR 21   c  that is approximately a rectangle extending in X direction is set in the target image G 21 . The size in Y direction of the reference region BR 11   c  and the comparison region CR 21   c  can be two pixels, for example. 
     The reference region BR 11   c  and the comparison region CR 21   c  have a corresponding form and size (the same form and size), and the position where the reference region BR 11   c  is set in the reference image G 11  and the position where the comparison region CR 21   c  is set in the target image G 21  have a positional relation that is shifted in X direction by an amount corresponding to a base length L ax . 
     In this way, in the first image set included in the multiple image sets, the region setting block  320 C sets the reference region BR 11   c  including the reference point P ref   1  in the reference image G 11  included in the first image set, and also sets the comparison region CR 21   c  in the target image G 21 . 
     Also, as shown in  FIG. 39 , reference regions BR 12   c  to BR 17   c  are set respectively in the reference images G 12  to G 17 , and, as shown in  FIG. 40 , comparison regions CR 22   c  to CR 27   c  are set respectively in the target images G 22  to G 27 . 
     Here, the reference regions BR 12   c  to BR 17   c  have the same form and size as the reference region BR 11   c , and the positional relation of the reference region BR 11   c  with respect to the reference image G 11 , and the positional relation of the reference regions BR 12   c  to BR 17   c  with respect to the reference images G 12  to G 17 , are the same. Also, the comparison regions CR 22   c  to CR 27   c  have the same form and size as the comparison region CR 21   c , and the positional relation of the comparison region CR 21   c  with respect to the target image G 21 , and the positional relation of the comparison regions CR 22   c  to CR 27   c  with respect to the target images G 22  to G 27 , are the same. 
     In this way, the region setting block  320 C sets the reference regions. BR 11   c  to BR 17   c  including the reference point P ref   1  in the reference images G 11  to G 17  with the same arrangement, and also sets the comparison regions CR 21   c  to CR 27   c , corresponding to the form and size of the reference regions BR 11   c  to BR 17   c , in the target images G 21  to G 27  with the same arrangement. 
     Here, according to the condition of arrangement of the first viewpoint and the second viewpoint separated in x direction, the elongate direction of the multiple reference regions BR 11   c  to BR 17   c  and the multiple comparison regions CR 21   c  to CR 27   c  is set in X direction corresponding to x direction. 
     The pixel value distribution generating block  330 C generates one pixel value distribution about three-dimensional space (reference distribution) from the distributions of pixel values about the plurality of reference regions BR 11   c  to BR 17   c , and also generates one pixel value distribution about three-dimensional space (comparison distribution) from the distributions of pixel values about the plurality of comparison regions CR 21   c  to CR 27   c.    
     Specifically, as shown in  FIG. 41 , first, pixel value distributions Bi 11   c  to Bi 17   c  about the reference regions BR 11   c  to BR 17   c  are extracted from the reference images G 11  to G 17 . Next, the pixel value distributions Bi 11   c  to Bi 17   c  are arranged according to given arrangement rules to generate a three-dimensional pixel value distribution (reference distribution) D PV   1   c . Here, the given arrangement rules are, for example, that the pixel value distributions Bi 11   c  to Bi 17   c  are arranged parallel to each other, and that the pixel value distribution Bi 11   c , pixel value distribution Bi 12   c , pixel value distribution Bi 13   c , pixel value distribution Bi 14   c , pixel value distribution Bi 15   c , pixel value distribution Bi 16   c , and pixel value distribution Bi 17   c  are arranged like layers in this order, so as to form a pixel value distribution about a rectangular parallelepiped region. The rectangular parallelepiped reference distribution D PV   1   c  is thus generated. 
     Also, as shown in  FIG. 42 , first, pixel value distributions Ci 21   c  to Ci 27   c  about the comparison regions CR 21   c  to CR 27   c  are extracted from the target images G 21  to G 27 . Next, the pixel value distributions Ci 21   c  to Ci 27   c  are arranged according to given arrangement rules to generate a three-dimensional pixel value distribution (comparison distribution) D PV   2   c . Here, the given arrangement rules are, for example, that the pixel value distributions Ci 21   c  to Ci 27   c  are arranged parallel to each other, and that the pixel value distribution Ci 21   c , pixel value distribution Ci 22   c , pixel value distribution Ci 23   c , pixel value distribution Ci 24   c , pixel value distribution Ci 25   c , pixel value distribution Ci 26   c , and pixel value distribution Ci 27   c  are arranged like layers in this order, so as to form a pixel value distribution about a rectangular parallelepiped region. The rectangular parallelepiped comparison distribution D PV   2   c  is thus generated. 
     In the generation of the reference distribution D PV   1   c  and the comparison distribution D PV   2   c , another order of arrangement may be adopted as long as the order of arrangement of the pixel value distributions Bi 11   c  to Bi 17   c  and the order of arrangement of the pixel value distributions Ci 21   c  to Ci 27   c  correspond to each other, for example. 
     The matching point detecting block  340 C detects a matching point corresponding to the reference point P ref   1  in the target image group G 2  by using the reference distribution D PV   1   c  and the comparison distribution D PV   2   c.    
       FIG. 43  is a diagram illustrating the matching point search using phase only correlation in the matching point detecting block  340 C. In the matching point search in the matching point detecting block  340 C, as compared with the matching point search using phase only correlation of the first preferred embodiment, the combination of image regions used in the matching point search (specifically, the combination of the image region of the reference distribution D PV   1  and the image region of the comparison distribution D PV   2 ) is changed to a different combination of image regions (specifically, the combination of the image region of the reference distribution D PV   1   c  and the image region of the comparison distribution D PV   2   c ). 
     In the matching point search in the matching point detecting block  340 C, a distribution of POC values indicating the correlation between the reference distribution D PV   1   c  and the comparison distribution D PV   2   c  is obtained. Then, the position corresponding to the peak of POC value in the image region about the comparison distribution D PV   2   c  is detected as a matching point on the target image G 21   x  that corresponds to the center point (reference point) P ref   1  of the reference region BR 11   c  on the reference image G 11 . 
     Then, by the matching point search using phase only correlation, as shown in  FIG. 44 , for example, a point P cor   2   c  shifted from the center point P cent   2   c  of the comparison distribution D PV   2   c  is detected as a point corresponding to the peak of POC value. Then, in the target images G 21  to G 27 , points in a positional relation like the positional relation between the center point P cent   2   c  and the point P cor   2   c  are obtained as matching points, on the basis of the center points of the comparison regions CR 21   c  to CR 27   c.    
     The matching point search of the first preferred embodiment adopts phase only correlation about a two-dimensional region, but the matching point search of this modification adopts phase only correlation about a three-dimensional region. Now, a method of calculating POC value in the matching point search in the matching point detecting block  340 C of this modification will be described. 
     Here, the reference distribution D PV   1   c  and the comparison distribution D PV   2   c  are handled as rectangular parallelepiped image regions in which a given number, N 1 , of pixels are arranged along X direction, a given number, N 2 , of pixels are arranged along Y direction, and a given number, N 3 , of pixels are arranged along T direction. These image regions are represented by Expression 11 below.
 
f(n 1 ,n 2 ,n 3 ),Size N 1 ×N 2 ×N 3  
 
g(n 1 ,n 2 ,n 3 ),Size N 1 ×N 2 ×N 3   Expression 11
 
     Where,
         n 1 =M 1 , . . . M 1      n 2 =−M 2 , . . . , M 2      n 3 =−M 3 , . . . , M 3          

     Here, f(n 1 , n 2 , n 3 ) in Expression 11 above indicates the image region about the reference distribution D PV   1   c , and g(n 1 , n 2 , n 3 ) in Expression 11 indicates the image region about the comparison distribution D PV   2   c . Also, N 1 , N 2  and N 3  are set as N 1 =2M 1 +1, N 2 =2M 2 +1, and N 3 =2M 3 +1, for example. 
     First, three-dimensional Fourier transforms T 1   ac  and T 1   bc  using Expression 12 below are performed to the image regions of the reference distribution D PV   1   c  and the comparison distribution D PV   2   c . 
     
       
         
           
             
               
                 
                   
                     
                       F 
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                           , 
                           
                             k 
                             3 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           
                             n 
                             1 
                           
                           , 
                           
                             n 
                             2 
                           
                           , 
                           
                             n 
                             3 
                           
                         
                       
                       ⁢ 
                       
                         
                           f 
                           ⁡ 
                           
                             ( 
                             
                               
                                 n 
                                 1 
                               
                               , 
                               
                                 n 
                                 2 
                               
                               , 
                               
                                 n 
                                 3 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             1 
                           
                           
                             
                               k 
                               1 
                             
                             ⁢ 
                             
                               n 
                               1 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             2 
                           
                           
                             
                               k 
                               2 
                             
                             ⁢ 
                             
                               n 
                               2 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             3 
                           
                           
                             
                               k 
                               3 
                             
                             ⁢ 
                             
                               n 
                               3 
                             
                           
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       G 
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                           , 
                           
                             k 
                             3 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           
                             n 
                             1 
                           
                           , 
                           
                             n 
                             2 
                           
                           , 
                           
                             n 
                             3 
                           
                         
                       
                       ⁢ 
                       
                         
                           g 
                           ⁡ 
                           
                             ( 
                             
                               
                                 n 
                                 1 
                               
                               , 
                               
                                 n 
                                 2 
                               
                               , 
                               
                                 n 
                                 3 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             1 
                           
                           
                             
                               k 
                               1 
                             
                             ⁢ 
                             
                               n 
                               1 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             2 
                           
                           
                             
                               k 
                               2 
                             
                             ⁢ 
                             
                               n 
                               2 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             3 
                           
                           
                             
                               k 
                               3 
                             
                             ⁢ 
                             
                               n 
                               3 
                             
                           
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   
                     Where 
                     , 
                     
                       
                         W 
                         p 
                       
                       = 
                       
                         EXP 
                         ( 
                         
                           
                             - 
                             j 
                           
                           ⁢ 
                           
                             
                               2 
                               ⁢ 
                               π 
                             
                             P 
                           
                         
                         ) 
                       
                     
                     , 
                     
                         
                     
                     ⁢ 
                     
                       
                         k 
                         s 
                       
                       = 
                       
                         - 
                         
                           M 
                           s 
                         
                       
                     
                     , 
                     … 
                     ⁢ 
                     
                         
                     
                     , 
                     
                       M 
                       s 
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   12 
                 
               
             
           
         
       
     
     In the note of Expression 12, N 1 , N 2  and N 3  are assigned to the subscript P of W, and 1, 2 and 3 are assigned to the subscript s of k. 
     For the image regions subjected to the Fourier transforms T 1   ac  and T 1   bc , normalizations T 2   ac  and T 2   b c are performed to remove image amplitude components by using the expressions shown as Expression 13 below. 
     
       
         
           
             
               
                 
                   
                     
                       
                         F 
                         ′ 
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                           , 
                           
                             k 
                             3 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         F 
                         ⁡ 
                         
                           ( 
                           
                             
                               k 
                               1 
                             
                             , 
                             
                               k 
                               2 
                             
                             , 
                             
                               k 
                               3 
                             
                           
                           ) 
                         
                       
                       
                          
                         
                           F 
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                               , 
                               
                                 k 
                                 3 
                               
                             
                             ) 
                           
                         
                          
                       
                     
                   
                   , 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         G 
                         ′ 
                       
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                           , 
                           
                             k 
                             3 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         G 
                         ⁡ 
                         
                           ( 
                           
                             
                               k 
                               1 
                             
                             , 
                             
                               k 
                               2 
                             
                             , 
                             
                               k 
                               3 
                             
                           
                           ) 
                         
                       
                       
                          
                         
                           G 
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                               , 
                               
                                 k 
                                 3 
                               
                             
                             ) 
                           
                         
                          
                       
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   13 
                 
               
             
           
         
       
     
     After normalizations T 2   ac  and T 2   bc , synthesis T 3   c  using Expression 14 below is performed, and also three-dimensional inverse Fourier transform T 4   c  using Expression 15 is performed. Correlation calculations between images are thus carried out and the results (POC value) are outputted. 
     
       
         
           
             
               
                 
                   
                       
                   
                   ⁢ 
                   
                     
                       R 
                       ⁡ 
                       
                         ( 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                           , 
                           
                             k 
                             3 
                           
                         
                         ) 
                       
                     
                     = 
                     
                       
                         
                           F 
                           ′ 
                         
                         ⁡ 
                         
                           ( 
                           
                             
                               k 
                               1 
                             
                             , 
                             
                               k 
                               2 
                             
                             , 
                             
                               k 
                               3 
                             
                           
                           ) 
                         
                       
                       ⁢ 
                       
                         
                           
                             G 
                             ′ 
                           
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                               , 
                               
                                 k 
                                 3 
                               
                             
                             ) 
                           
                         
                         _ 
                       
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   14 
                 
               
             
             
               
                 
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         
                           k 
                           1 
                         
                         , 
                         
                           k 
                           2 
                         
                         , 
                         
                           k 
                           3 
                         
                       
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       
                         
                           N 
                           1 
                         
                         ⁢ 
                         
                           N 
                           2 
                         
                         ⁢ 
                         
                           N 
                           3 
                         
                       
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           
                             k 
                             1 
                           
                           , 
                           
                             k 
                             2 
                           
                           , 
                           
                             k 
                             3 
                           
                         
                       
                       ⁢ 
                       
                         
                           R 
                           ⁡ 
                           
                             ( 
                             
                               
                                 k 
                                 1 
                               
                               , 
                               
                                 k 
                                 2 
                               
                               , 
                               
                                 k 
                                 3 
                               
                             
                             ) 
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             1 
                           
                           
                             
                               - 
                               
                                 k 
                                 1 
                               
                             
                             ⁢ 
                             
                               n 
                               1 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             2 
                           
                           
                             
                               - 
                               
                                 k 
                                 2 
                               
                             
                             ⁢ 
                             
                               n 
                               2 
                             
                           
                         
                         ⁢ 
                         
                           W 
                           
                             N 
                             3 
                           
                           
                             
                               - 
                               
                                 k 
                                 3 
                               
                             
                             ⁢ 
                             
                               n 
                               3 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Expression 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   15 
                 
               
             
           
         
       
     
     As described above, in the image processing apparatus  3 C of the first modification, the reference regions BR 11   c  to BR 17   c  and the comparison regions CR 21   c  to CR 27   c  are relatively small rectangular regions whose elongate direction is along X direction. Accordingly, a matching point corresponding to a reference point is less likely to be out of the comparison regions. As a result, the matching point search can be performed even when viewpoints about a plurality of images are somewhat shifted. 
     The method of matching point search of this modification may be applied to the first and second matching point search operations of the image processing apparatus  313  of the second preferred embodiment. 
     &lt;(3-2) Second Modification&gt; 
     In the first and second preferred embodiments and the first modification, the number of dimensions of the space of a reference distribution of pixel values is larger than the number of dimension(s) of the space of a plurality of reference regions, and the number of dimensions of the space of a comparison distribution of pixel values is larger than the number of dimension(s) of the space of a plurality of comparison regions, but this is meant only to be illustrative and not restrictive. 
     For example, with reference regions and comparison regions being two-dimensional regions, the reference distribution of pixel values may be a two-dimensional distribution of pixel values generated by arranging the distributions of pixel values of the plurality of reference regions according to given arrangement rules, and the comparison distribution of pixel values may be a two-dimensional distribution of pixel values generated by arranging the distributions of pixel values about the plurality of comparison regions according to given arrangement rules. Such a method of generating a reference distribution and a comparison distribution will be described below. 
     In an information processing system  1 D of a second modification, the image processing apparatus  3 C of the information processing system  1 C of the first modification is changed to an information processing apparatus  3 D in which the controller  300 C is replaced by a controller  300 D having different functions. Specifically, as shown in  FIG. 4 , the pixel value distribution generating block  330 C of the first modification is changed to a pixel value distribution generating block  330 D, and the matching point detecting block  340 C of the first modification is changed to a matching point detecting block  340 D. 
     In the controller  300 D, a program PGd stored in a storage  304  is read and executed to implement various functions. The functions implemented by the controller  300 D that are the same as those of the first modification are shown by the same reference characters and will not be described again, and the pixel value distribution generating block  330 D and the matching point detecting block  340 D, which are different from those of the information processing system  1 C of the first modification, will be described. 
     The pixel value distribution generating block  330 D generates one pixel value distribution about two-dimensional space (reference distribution) from the pixel value distributions about a plurality of reference regions BR 11   c  to BR 17   c , and also generates one pixel value distribution about two-dimensional space (comparison distribution) from the pixel value distributions about a plurality of comparison regions CR 21   c  to CR 27   c.    
     Specifically, as shown in  FIG. 45 , pixel value distributions Bi 11   c  to Bi 17   c  are arranged according to given arrangement rules to generate a two-dimensional pixel value distribution (reference distribution) D PV   1   d . The given arrangement rules are, for example, that the pixel value distributions Bi 11   c  to Bi 17   c  are arranged parallel to each other, and that the pixel value distribution Bi 11   c , pixel value distribution Bi 12   c , pixel value distribution Bi 13   c , pixel value distribution Bi 14   c , pixel value distribution Bi 15   c , pixel value distribution Bi 16   c , and pixel value distribution Bi 17   c  are arranged in this order, so as to form a pixel value distribution about a rectangular region. The rectangular reference distribution D PV   1   d  is thus generated. 
     Also, as shown in  FIG. 46 , pixel value distributions Ci 21   c  to Ci 27   c  are arranged according to given arrangement rules to generate a two-dimensional pixel value distribution (comparison distribution) D PV   2   d . The given arrangement rules are, for example, that the pixel value distributions Ci 21   c  to Ci 27   c  are arranged parallel to each other, and that the pixel value distribution Ci 21   c , pixel value distribution Ci 22   c , pixel value distribution Ci 23   c , pixel value distribution Ci 24   c , pixel value distribution Ci 25   c , pixel value distribution Ci 26   c , and pixel value distribution Ci 27   c  are arranged in this order, so as to form a pixel value distribution about a rectangular region. The rectangular comparison distribution D PV   2   d  is thus generated. 
     As to the method of matching point search using phase only correlation in the matching point detecting block  340 D, a method like that of the matching point search in the matching point detecting block  340 A of the first preferred embodiment is adopted. However, the combination of image regions used in the matching point search (specifically, the combination of the image region of the reference distribution D PV   1  and the image region of the comparison distribution D PV   2 ) is changed to a different combination of image regions (specifically, the combination of the image region of the reference distribution D PV   1   d  and the image region of the comparison distribution D PV   2   d ). 
     In the generation of the reference distribution D PV   1   d  and the comparison distribution D PV   2   d , another order of arrangement may be adopted as long as the order of arrangement of the pixel value distributions Bi 11   c  to Bi 17   c  and the order of arrangement of the pixel value distributions Ci 21   c  to Ci 27   c  correspond to each other, for example. 
     Also, while, as shown in  FIG. 45 , the reference distribution D PV   1   d  is generated by arranging the pixel value distributions Bi 11   c  to Bi 17   c  in a line, and, as shown in  FIG. 46 , the comparison distribution D PV   2   d  is generated by arranging the pixel value distributions Ci 21   c  to Ci 27   c  in a line, this is not restrictive. For example, as shown in  FIG. 47 , a reference distribution D PV   1   dd  may be generated by arranging the pixel value distributions Bi 11   c  to Bi 13   c  in a first line and arranging the pixel value distributions Bi 14   c  to Bi 16   c  in a second line, and, as shown in  FIG. 48 , a comparison distribution D PV   2   dd  may be generated by arranging the pixel value distributions Ci 21   c  to Ci 23   c  in a first line and the pixel value distributions Ci 24   c  to Ci 26   c  in a second line. 
     &lt;(3-3) Third Modification&gt; 
     In the first and second preferred embodiments and the first and second modifications, the object OB is intactly imaged in a time-sequential manner with the stereo camera  2 A or  2 B, but this is illustrative and not restrictive. For example, a projection device may be further provided to project different patterns to the object OB according to the timing of imaging by the first and second cameras  21  and  22 . 
       FIG. 49  is a diagram schematically illustrating the configuration of an information processing system  1 E of a third modification in which a projection device  23  is further provided in the information processing system  1 A of the first preferred embodiment. 
     The timing of projection to the object OB by the projection device  23  is controlled by the image processing apparatus  3 A. For example, in synchronization with the timing of imaging by the first and second cameras  21  and  22 , the projection device  23  projects different patterns to the object OB according to individual timings of imaging. The patterns projected by the projection device  23  can be patterns in which dots are randomly arranged (random dot patterns) or noise patterns of normal distribution (Gaussian noise patterns), for example. 
     As described above, according to the information processing system  1 E of this modification, a plurality of images are taken while time-sequentially projecting different patterns to the object, whereby the amount of information used in the calculations of the matching point search is further increased. This makes it possible to more stably and precisely perform a matching point search with a plurality of images taking the same object where distant and near views coexist. 
     &lt;(3-4) Other Modifications&gt; 
     In the information processing systems  1 A to  1 E of the first and second preferred embodiments and the first to third modifications, a matching point in the target image group G 2  corresponding to a reference point in the reference image group G 1  is detected in a single step of process, but this is only illustrative and not restrictive. For example, a matching point corresponding to a reference point may be detected by multiple process steps including the following steps (I) to (III). 
     (I) First, with a reference image group G 1 , reference image groups of multiple levels of resolution are generated by adopting pixel lines at given intervals, and with a target image group G 2 , target image groups of multiple levels of resolution are generated by adopting pixel lines at given intervals. 
     (II) Next, a matching point corresponding to a reference point is detected between reference and target images at low resolution. 
     (III) Furthermore, between reference and target images at relatively high resolution, on the basis of the reference and matching points between the low-resolution reference and target images, reference and comparison regions are set and a matching point corresponding to the reference point is detected between the reference and comparison regions. 
     In such a configuration, when the reference and comparison regions are somewhat extended in two-dimensional space, like the reference regions BR 11   c  to BR 17   c  and the comparison regions CR 21   c  to CR 27   c  of the second and third modifications, the matching point is less likely to be out of the comparison regions and the precision of the matching point search is less likely to deteriorate. 
     Also, in the information processing systems  1 A to  1 E of the first and second preferred embodiments and the first to third modifications, the reference regions and comparison regions are set on the basis of the perpendicular X direction and Y direction, and the reference distribution and comparison distribution are generated, but this is only illustrative and not restrictive. For example, reference regions and comparison regions may be set on the basis of X direction and a direction that intersects X direction at an acute angle, and a reference distribution and comparison distribution may be generated. 
     Also, in the information processing systems  1 A to  1 E of the first and second preferred embodiments and the first to third modifications, a matching point corresponding to a reference point is detected on the basis of a plurality of image sets each including a reference image and a target image, but this is only illustrative and not restrictive. For example, each image set may include three or more images and a matching point corresponding to a reference point may be detected among the three or more images. That is to say, each image set includes two or more images, and a matching point corresponding to a reference point is detected among the two or more images. 
     Needless to say, all or part of the first and second preferred embodiments, the first to third modifications, and other modifications can be used in combination as long as contradictions do not occur. 
     While the invention has been described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is understood that numerous other modifications and variations can be devised without departing from the scope of the invention.