Patent Description:
Continuous maintenance is essential for infrastructure/structures such as roads, bridges, buildings, towers, walls, dams, tunnels, and so on. In continuous maintenance for infrastructures, it is important to identify, record, and observe cracks and other defective portions.

Examples of inspection methods for infrastructures employed in many cases include a method for capturing images of an infrastructure, identifying cracks and other defects based on the captured images, and evaluating the identified defects. However, if the resolution per unit area on the surface of the infrastructure under inspection is low in the captured image, a difficulty arises in recognizing defects, for example, because of a blurred crack pattern. Therefore, it is demanded that the resolution of a captured image per unit area is equal to or higher than a predetermined value on the surface under inspection. For example, if an infrastructure has a large scale or complicated shape, it is difficult to capture an image of the infrastructure at one time with a predetermined resolution or higher. In such a case, the infrastructure is imaged in split imaging, and defects are identified and evaluated by using the images captured in split imaging.

For example, <CIT> discusses a technique for estimating the resolutions in the images based on the distance to an object and comparing the average value of the estimated resolutions with a predetermined resolution threshold value to determine whether image acquisition is possible. More specifically, when the average value of the estimated resolutions exceeds the resolution threshold value, the image acquisition is acknowledged and images are stored. On the other hand, when the average value is below the resolution threshold value, the image acquisition is rejected.

The technique discussed in <CIT> determines whether image acquisition is possible based on the estimated image resolution. However, if whether image acquisition is possible is determined based only on the resolution, the number of images available for the defect evaluation of an object such as an infrastructure may become insufficient, possibly disabling the recognition of defects of the object.

<CIT> discusses a method includes receiving a first image and a plurality of other images of a planar surface of a specimen. The method also includes identifying first scale-invariant features in the first image. The first scale-invariant features are based on a scale-invariant feature transform of the first image. The method includes storing the first scale-invariant features in a grouping. The method includes, for each respective image of the plurality of other images, identifying scale-invariant features based on a scale-invariant feature transform of the respective image; matching the scale-invariant features to the grouping; based on the matching, determining a planar homography of the respective image with respect to the grouping; and adding the scale-invariant features of the respective image to the grouping. The method also includes stitching the first image and the plurality of other images into a composite image based on the planar homographies. A field of view of the specimen in the composite image is greater than a field of view of at least one of the first image or the plurality of other images.

<CIT> discusses a method for acquiring and processing images of the ocular fundus by means of a portable electronic device, wherein the portable electronic device comprises a photographic camera or video camera capable of acquiring a plurality of images at a predetermined frequency, and a processor capable of processing these images, the method comprising: acquiring a plurality of images of the ocular fundus by means of the photographic camera or video camera of the electronic device, mapping certain images selected from the plurality of acquired images, wherein the selected images are positioned relative to one another so as to form a map of the ocular fundus formed by the assembly of the selected images, and the positioning of each image relative to the other, previously selected, images is carried out within a maximum time interval of <NUM> milliseconds, and joining the selected images or images corresponding to them according to the mutual positioning defined by the mapping, so as to generate a single image of the ocular fundus.

A paper titled "<NPL>, discusses a method to generate a panorama of a construction site by using an image stitching technique with a focus on preprocessing. To create high-quality panoramas, blurred frames of videos are filtered out, key frames are selected, and camera lens distortion is corrected. The proposed method produced a high-quality panorama of a construction site, which was evaluated by comparing it with an aerial photograph and the panorama produced by the existing image stitching technique. The proposed method is expected to help managers to easily identify various construction site conditions with the help of high-quality image data.

A paper titled "<NPL>, discusses an approach to stitch drone-captured indoor video frames, where feature based stitching fails. In order to achieve this, the image feature data extracted is fused with drone inertial measurement unit (IMU) data. The approach is tested in a warehouse and the performance is compared with other state-of-the-art image stitching algorithms. The proposed approach shows robust performance in cases of highly non-planar scenes.

According to a first aspect of the present invention, there is provided an information processing apparatus as specified in claims <NUM> to <NUM>. According to a second aspect of the present invention, there is provided an information processing method as specified in claim <NUM>. According to a third aspect of the present invention, there is provided a storage medium storing a program as specified in claim <NUM>.

Each of the embodiments of the present invention described below can be implemented solely or as a combination of a plurality of the embodiments or features thereof where necessary or where the combination of elements or features from individual embodiments in a single embodiment is beneficial.

The present invention will be described in detail below based on exemplary embodiments with reference to the accompanying drawings. Configurations described in the following exemplary embodiments are to be considered as illustrative, and the present invention is not limited to illustrated configurations.

<FIG> is a block diagram illustrating an example of a hardware configuration of an information processing apparatus <NUM> according to a first exemplary embodiment. The information processing apparatus <NUM> includes a central processing unit (CPU) <NUM>, a random access memory (RAM) <NUM>, a read only memory (ROM) <NUM>, an auxiliary storage device <NUM>, an input/output interface <NUM>, a communication interface <NUM>, a display apparatus <NUM>, a bus <NUM>, an input controller <NUM>, and an input apparatus <NUM>.

The CPU <NUM> executes and controls each function of the information processing apparatus <NUM>. The RAM <NUM> temporarily stores programs and data supplied from an external apparatus. The ROM <NUM> stores programs and various kinds of parameters without the need of change. The display apparatus <NUM> displays graphics drawn by the CPU <NUM>. The auxiliary storage device <NUM> stores various kinds of information. The input/output interface <NUM> transmits and receives data to/from an external apparatus. The communication interface <NUM> for connection to a network transmits and receives data to/from an external apparatus via the network. The bus <NUM> as a system bus connects the CPU <NUM>, the RAM <NUM>, the ROM <NUM>, the auxiliary storage device <NUM>, the input/output interface <NUM>, the communication interface <NUM>, the display apparatus <NUM>, and the input controller <NUM>. The input controller <NUM> controls input signals from the input apparatus <NUM> (described below). The input apparatus <NUM> is an external input apparatus for receiving operation instructions from a user, such as a keyboard and a mouse. Functions and the processing of the information processing apparatus <NUM> (described below) are implemented when the CPU <NUM> loads a program stored in the ROM <NUM> and executes the program.

Exemplary embodiments will be described below centering on a case where images of an infrastructure as an object under inspection are captured, and cracks and other defects are identified and evaluated based on the captured images. According to the present exemplary embodiment, if an infrastructure has a large scale or a complicated shape, defects are identified and evaluated by using an image obtained by stitching images of the infrastructure captured through split imaging. When evaluating defects by using captured images, as described above, it is necessary to acquire images having a resolution per unit area on the surface under inspection equal to or larger than a predetermined value. When stitching images captured through split imaging, the images are mutually positioned based on feature points in the images before image stitching. However, if whether image acquisition is possible is determined by using only the image resolution, like the technique discussed in <CIT>, images may be stored with insufficient feature points. In this case, positioning based on feature points cannot be performed, possibly resulting in image stitching failure.

Therefore, according to the first exemplary embodiment, feature points are extracted from input images captured through split imaging and then evaluated, and the image quality on an object on the input images is evaluated. Then, based on these evaluation results, images available for inspection of the object are evaluated. Then, according to the first exemplary embodiment, a three-dimensional shape of an infrastructure is estimated and, for the estimated three-dimensional shape, input images are mapped (projected) based on the image evaluation results to generate a stitched image.

<FIG> is a function block diagram illustrating a functional configuration of the information processing apparatus <NUM> according to the first exemplary embodiment. The functional configuration of the information processing apparatus <NUM> according to the first exemplary embodiment will be described below with reference to <FIG>.

An image input unit <NUM> acquires images of an object captured by using a camera. An object refers to a target to be inspected by using the captured images. Examples of objects include infrastructure/structures such as concrete surfaces of bridges, dams, and tunnels. However, objects are not limited thereto, and may be concrete, or any other material (e.g. brick), wall surfaces of a variety of structures. Each image acquired by the image input unit <NUM> includes internal parameters of the camera, i.e., the image resolution, the focal length of the camera, the size of the image sensor, the position of the image center, and information about lens distortion.

A feature extraction unit <NUM> extracts feature points from input images acquired by the image input unit <NUM>, performs feature point matching between the images, and outputs information about feature points and information about feature point matching to a position and orientation estimation unit <NUM> and a feature evaluation unit <NUM>. The feature extraction unit <NUM> will be described in detail below.

The feature evaluation unit <NUM> evaluates the feature points extracted by the feature extraction unit <NUM> for each image. The feature evaluation unit <NUM> will be described in detail below.

The position and orientation estimation unit <NUM> estimates the position and orientation of the camera which has captured images, based on the information about feature points extracted by the feature extraction unit <NUM> and the information about feature point matching. Techniques for estimating the camera position and orientation by using image feature points and matching information are generally known, and therefore descriptions thereof will be omitted.

A shape estimation unit <NUM> estimates the three-dimensional shape of the object under inspection based on the camera position and orientation estimated by the position and orientation estimation unit <NUM> and the images acquired by the image input unit <NUM>. The shape estimation unit <NUM> will be described in detail below.

An image quality evaluation unit <NUM> evaluates the image quality on the object under inspection based the input images acquired by the image input unit <NUM>, the three-dimensional shape of the object calculated by the shape estimation unit <NUM>, and the camera position and orientation estimated by the position and orientation estimation unit <NUM>. The image quality evaluation unit <NUM> will be described in detail below.

A threshold value holding unit <NUM> holds threshold values for the result of evaluation by the feature evaluation unit <NUM> and the result of evaluation by the image quality evaluation unit <NUM>. The threshold values define allowable conditions for determining whether the quality of images is permissible as images to be used for inspection of the infrastructure. Allowable conditions are defined by using the resolution as an index. The threshold value holding unit <NUM> will be described in detail below.

An image evaluation unit <NUM> identifies regions available for inspection in the images acquired by the image input unit <NUM> based on the result of evaluation by the feature evaluation unit <NUM>, the result of evaluation by the image quality evaluation unit <NUM>, and the threshold values held by the threshold value holding unit <NUM>. Then, the image evaluation unit <NUM> outputs information about the regions available for inspection to an image stitching unit <NUM>. The image evaluation unit <NUM> will be described in detail below.

The image stitching unit <NUM> stitches a plurality of images based on the images input from the image input unit <NUM>, the camera position and orientation from the position and orientation estimation unit <NUM>, the three-dimensional shape from the shape estimation unit <NUM>, and the information about the regions available for inspection from the image evaluation unit <NUM>. The image stitching unit <NUM> will be described in detail below.

The feature extraction unit <NUM>, the shape estimation unit <NUM>, the feature evaluation unit <NUM>, the image quality evaluation unit <NUM>, the threshold value holding unit <NUM>, the image evaluation unit <NUM>, and the image stitching unit <NUM> will be described in detail below.

The feature extraction unit <NUM> extracts, for example, Scale-Invariant Feature Transform (SIFT) feature points from the input images acquired by the image input unit <NUM>. SIFT feature points are generally known feature points, and descriptions thereof will be omitted. Feature points used in the present exemplary embodiment are feature points for performing local region matching between object images captured from different image capturing directions and at different image capturing angles. According to the present exemplary embodiment, SIFT feature points are used because of the robustness to the rotation, enlargement, and reduction of an object. According to the present exemplary embodiment, feature points extracted by the feature extraction unit <NUM> may be feature points other than SIFT feature points as long as local region matching between images can be performed.

Firstly, the shape estimation unit <NUM> estimates a three-dimensional point group based on the camera position and orientation and the images associated with the camera position and orientation, by using the multi-baseline stereo method. The multi-baseline method performed by the shape estimation unit <NUM> makes it possible to perform window-based matching centering on image pixels to estimate a three-dimensional point group based on the principle of triangulation.

In this case, Sum of Squared Differences (SSD) in luminance is used as the window-based matching. The shape estimation unit <NUM> further meshes the three-dimensional point group by using the general Delaunay triangulation. Then, the shape estimation unit <NUM> outputs the meshed three-dimensional shape to the image quality evaluation unit <NUM> and the image stitching unit <NUM>.

The feature evaluation unit <NUM> evaluates the feature points by using the kurtosis of the positions of two-dimensional coordinates of feature points in the images and the number of feature points. More specifically, the feature evaluation unit <NUM> calculates an evaluation value V for each image by using Equation (<NUM>).

Referring to Equation (<NUM>), a and β are arbitrary coefficients indicating the weights of the first and the second terms, respectively. According to the present exemplary embodiment, both α and β are <NUM>. Referring to Equation (<NUM>), Ku denotes the kurtosis of the position distribution in two-dimensional coordinates of features points in the images, a coefficient j denotes the dimension of the images, and N denotes the number of feature points. In Equation (<NUM>), the evaluation increases with decreasing kurtosis of the distribution (Ku) and increasing number of feature points (N). More specifically, Equation (<NUM>) indicates that a higher evaluation value results for images having feature points that are uniformly distributed in the images and having a larger number of feature points. N may be the number of feature points that can be matched with feature points of other images. Likewise, also for feature points to be used for the calculation of the kurtosis of position distribution of two-dimensional coordinates of features points in the images, only feature points that can be matched with feature points of other images may be used for the calculation.

The image quality evaluation unit <NUM> estimates the resolution of the image capturing target object appearing in the input images acquired by the image input unit <NUM>. According to the present exemplary embodiment, the image quality evaluation unit <NUM> estimates how many pixels of the image corresponds to a length of <NUM> on the object. For example, when a <NUM>,<NUM> by <NUM>,<NUM> region on the object is imaged with a camera with <NUM>,<NUM> * <NUM>,<NUM> pixels, the object is imaged with a resolution of <NUM> pixel per <NUM>. The number of pixels corresponding to a length of <NUM> on the object can be geometrically calculated based on information about the three-dimensional shape estimated by the shape estimation unit <NUM>, the camera position and orientation estimated by the position and orientation estimation unit <NUM>, and the size and internal camera parameters of the images input to the image input unit <NUM>.

The threshold value holding unit <NUM> holds the threshold values in tabular form. <FIG> illustrates an example of a table <NUM> held by the threshold value holding unit <NUM>. The table <NUM> holds a threshold value <NUM> for the feature point evaluation value and a threshold value <NUM> for the image quality evaluation value. The threshold value <NUM> for the image quality evaluation value in the table <NUM> is the resolution on the object, which means, in the example shown in <FIG>, <FIG> pixel per <NUM>.

The image evaluation unit <NUM> compares the evaluation value of the result of evaluation by the feature evaluation unit <NUM> with the threshold value <NUM> for the feature point evaluation value held by the threshold value holding unit <NUM>. When the feature point evaluation value is equal to or larger than the threshold value <NUM>, the image evaluation unit <NUM> further compares the image resolution of each region supplied by the image quality evaluation unit <NUM> with the threshold value <NUM> for the image quality evaluation value held by the threshold value holding unit <NUM> to identify pixels having an image quality evaluation value equal to or larger than the threshold value <NUM>.

According to the present exemplary embodiment, the image resolution for each region is assumed to be the resolution supplied for each pixel in the images. Then, the image evaluation unit <NUM> outputs position information for pixels identified to have an image quality evaluation value equal to or larger than the threshold value to the image stitching unit <NUM> as information about the regions available for defect inspection. The information about the regions available for inspection specifically refers to a two-dimensional array including the same number of pixels as the images input to the image input unit <NUM>, where each pixel available for inspection is supplied with "<NUM>" and each pixel not available for inspection is supplied with "<NUM>".

The image evaluation unit <NUM> is also able to hold values between <NUM> to <NUM> on a multi-value basis, instead of supplying "<NUM>" to the regions available for inspection and supplying "<NUM>" to the regions not available for inspection in the images. With this method, an image portion where there is no region available for inspection can be compensated with an image portion having a value of the region available for inspection closest to "<NUM>" in other images when the image stitching unit <NUM> performs image stitching.

<FIG> illustrates image stitching processing performed by the image stitching unit <NUM>. A three-dimensional shape <NUM> is the three-dimensional shape estimated by the shape estimation unit <NUM>. Referring to <FIG>, positions <NUM> and <NUM> denote the positions of principal points of the camera(s) at different positions. Imaging planes <NUM> and <NUM> represent imaging planes of the camera(s) at these different positions. Although the imaging planes <NUM> and <NUM> are originally positioned behind the principal points, the following descriptions will be made on the premise that these imaging planes are disposed in front of the principal points in consideration of the understandability of the drawing. Referring to <FIG>, the imaging planes <NUM> and <NUM> represent crack images <NUM> and <NUM>, respectively, as defect images formed on respective surfaces. <FIG> illustrates that the crack images <NUM> and <NUM> are captured and recorded.

Regions <NUM> and <NUM> are regions available for inspection in a two-dimensional array input from the image evaluation unit <NUM>. Projection images <NUM> and <NUM> represent the projection of the regions <NUM> and <NUM> available for inspection onto the three-dimensional shape <NUM>, out of the images on the imaging planes <NUM> and <NUM>, respectively. A region <NUM> illustrates a region where the projection images <NUM> and <NUM> overlap with each other.

The image stitching unit <NUM> generates the projection images <NUM> and <NUM> of all of the images where the regions available for inspection exist, for the three-dimensional shape <NUM>, and projects these projection images onto the three-dimensional shape <NUM>. The image stitching unit <NUM> also performs blending processing on a region where images are overlapping, such as the region <NUM>. As the blending processing, for example, multi-band blending processing can be used.

The images of the imaging planes <NUM> and <NUM> in the information processing apparatus <NUM> according to the present exemplary embodiment are subjected to the feature point evaluation by the feature evaluation unit <NUM>. Thus, the projection images <NUM> and <NUM> having a small amount of positional deviation in the images can be obtained. By obtaining projection images having a small amount of positional deviation in the images, an effect of preventing degradation of the accuracy in crack evaluation as defect evaluation can be obtained. The reason is as follows. If a position deviation occurs between images in crack inspection, one crack may be broken and accordingly recognized as two different cracks. The present exemplary embodiment makes it possible to reduce the amount of position deviation.

Even when the object is captured from an oblique position such as the position <NUM> of the principal point of the camera, the information processing apparatus <NUM> according to the present exemplary embodiment is able to obtain the projection image <NUM> since the region <NUM> available for inspection exists. This is effective in a case where the camera is able to image the object only from an oblique position because of the presence of an obstacle or other physical reasons.

The flow of processing performed by the information processing apparatus <NUM> according to the first exemplary embodiment will be described below with reference to the flowcharts illustrated in <FIG>, <FIG>, <FIG>, and <FIG>.

A flow of overall processing of the information processing apparatus <NUM> will be described below with reference to the flowchart illustrated in <FIG>.

In step S501, the image input unit <NUM> acquires images captured by the camera. The image input unit <NUM> may acquire images from a storage medium storing images captured by a camera, or acquire images captured by a camera via a network. Upon completion of step S501, the processing of the information processing apparatus <NUM> proceeds to step S502.

In step S502, the feature extraction unit <NUM> performs processing for extracting feature points on all of the images acquired by the image input unit <NUM>. In step S503, the feature extraction unit <NUM> performs matching between feature points. The feature extraction unit <NUM> outputs information about feature points and information about feature point matching to the position and orientation estimation unit <NUM> and the feature evaluation unit <NUM>. Upon completion of step S502, the processing of the information processing apparatus <NUM> proceeds to step S504.

In step S504, the position and orientation estimation unit <NUM> estimates the camera position and orientation based on the information about feature points and the information about feature point matching input from the feature extraction unit <NUM>. The position and orientation estimation unit <NUM> further outputs the information about the camera position and orientation to the shape estimation unit <NUM>, the image quality evaluation unit <NUM>, and the image stitching unit <NUM>. Upon completion of step S504, the processing of the information processing apparatus <NUM> proceeds to step S505.

In step S505, the shape estimation unit <NUM> estimates the three-dimensional shape of the object based on the information about the camera position and orientation input from the position and orientation estimation unit <NUM> and the images input from the image input unit <NUM>. Upon completion of step S505, the processing of the information processing apparatus <NUM> proceeds to step S506.

In step S506, the information processing apparatus <NUM> performs processing for evaluating the images available for inspection. The processing in step S506 will be described in detail below with reference to the flowcharts illustrated in <FIG> and <FIG>. Upon completion of step S506, the processing of the information processing apparatus <NUM> proceeds to step S507.

In step S507, the information processing apparatus <NUM> performs image stitching as post-processing. The processing in step S507 will be described in detail below with reference to the flowchart illustrated in <FIG>. Upon completion of the processing in step S507, the processing of the information processing apparatus <NUM> ends.

<FIG> is a detailed flowchart illustrating processing for evaluating the images available for inspection in step S506 illustrated in <FIG>.

In step S601 illustrated in <FIG>, the image evaluation unit <NUM> determines whether the image evaluation is completed for all of the images input from the image input unit <NUM>. When the image evaluation unit <NUM> determines that the image evaluation is completed for all of the input images (YES in step S601), the image evaluation unit <NUM> ends the processing for evaluating the images available for inspection. Then, the processing of the information processing apparatus <NUM> proceeds to step S507. On the other hand, when the image evaluation unit <NUM> determines that the image evaluation is not completed for all of the input images (NO in step S601), the processing of the information processing apparatus <NUM> proceeds to step S602.

In step S602, the feature evaluation unit <NUM> acquires the information about feature points from the feature extraction unit <NUM> and calculates the evaluation value of the feature points. An evaluation Equation denoted by Equation (<NUM>) is used to calculate the evaluation value of feature points. Upon completion of step S602, the processing of the information processing apparatus <NUM> proceeds to step S603.

In step S603, the image evaluation unit <NUM> compares the feature point evaluation value calculated by the feature evaluation unit <NUM> with the threshold value <NUM> for the feature point evaluation value held by the threshold value holding unit <NUM> to determine whether the feature point evaluation value is equal to or larger than the threshold value. When the image evaluation unit <NUM> determines that the feature point evaluation value is equal to or larger than the threshold value (YES in step S603), the processing of the information processing apparatus <NUM> proceeds to step S604. On the other hand, when the image evaluation unit <NUM> determines that the feature point evaluation value is smaller than the threshold value (NO in step S603), the processing of the information processing apparatus <NUM> returns to step S601.

In step S604, the information processing apparatus <NUM> performs the image quality evaluation processing. Then, the processing of the information processing apparatus <NUM> returns to step S601.

<FIG> is a detailed flowchart illustrating the image quality evaluation processing in step S604 illustrated in <FIG>.

In step S701, the image quality evaluation unit <NUM> estimates the resolution distribution on the object based on the images input from the image input unit <NUM>. Upon completion of step S701, the processing of the information processing apparatus <NUM> proceeds to step S702.

In step S702, the image evaluation unit <NUM> compares the resolution for each pixel in the images supplied by the image quality evaluation unit <NUM> with the threshold value <NUM> for the image quality evaluation value held by the threshold value holding unit <NUM> to determine whether there exists a pixel having a value equal to or larger than the threshold value. When the image evaluation unit <NUM> determines that there exists a pixel having a value equal to or larger than the threshold value (YES in step S702), the processing of the information processing apparatus <NUM> proceeds to step S703. On the other hand, when the image evaluation unit <NUM> determines that there exists no pixel having a value equal to or larger than the threshold value (NO in step S702), the information processing apparatus <NUM> ends the image quality evaluation processing.

In step S703, the image evaluation unit <NUM> outputs position information for a pixel identified to have a value equal to or larger than the threshold value in S702 to the image stitching unit <NUM> as information about the region available for inspection. Then, the information processing apparatus <NUM> ends the image quality evaluation processing.

<FIG> is a detailed flowchart illustrating the post-processing in step S507 illustrated in <FIG>.

In step S801, the image stitching unit <NUM> reads the three-dimensional shape from the shape estimation unit <NUM>. Then, the processing proceeds to step S802.

In step S802, the image stitching unit <NUM> reads the camera position and orientation estimated by the position and orientation estimation unit <NUM>. Then, the processing proceeds to step S803.

In step S803, the image stitching unit <NUM> acquires the images input from the image input unit <NUM>. The image stitching unit <NUM> also acquires the information about the regions available for inspection corresponding to the images input from the image input unit <NUM>, from the image evaluation unit <NUM>. Then, the processing proceeds to step S804.

In step S804, the image stitching unit <NUM> projects the images of the regions available for inspection corresponding to the images input from the image input unit <NUM> onto the three-dimensional shape input from the shape estimation unit <NUM>, based on the camera position and orientation. Then, the processing proceeds to step S805.

In step S805, the image stitching unit <NUM> performs the blending processing on the overlapping portion of the images projected onto the three-dimensional shape input from the shape estimation unit <NUM>. Then, the processing of the information processing apparatus <NUM> ends.

As described above, the information processing apparatus <NUM> according to the first exemplary embodiment evaluates the feature points of the images and the image resolution, and projects the images having preferable evaluation results to a three-dimensional shape. More specifically, the present exemplary embodiment makes it possible to reduce image stitching failure through the stitching processing using the feature points available for accurate positioning and to stitch images having a resolution available for defect inspection. Thus, the present exemplary embodiment makes it possible to acquire images available for defect evaluation of an object such as an infrastructure, and therefore to recognize defects of the object.

As a first modification of the first exemplary embodiment, an example where the index of the image quality to be evaluated by the image quality evaluation unit <NUM> is the defocus amount, i.e., an example where the degree of focusing on the object is evaluated by the image quality evaluation unit <NUM> is described. The defocus amount refers to a numerical representation of the amount of anteroposterior deviation in focusing on the object at pixel positions in the images. At the time of image capturing, the defocus amount can be acquired as defocus amount information at pixel positions by using an imaging plane phase difference image sensor. As a method for acquiring the defocus amount information, a known technique can be used. For example, an automatic focus technique using the amount of anteroposterior deviation in focusing detected by the imaging plane phase difference image sensor has already been widely used.

<FIG> illustrates an example of a defocus map for visualizing the defocus amount information for the images input from the image input unit <NUM>. According to the first modification, the defocus amount information is detected based on the size of a permissible circle-of-confusion focused on the pixels of the image sensor. The defocus amount information accompanies the images input from the image input unit <NUM>. An outer frame <NUM> is a frame corresponding to the image size (image boundary) of the input images. A numerical value <NUM> indicates information about the defocus amount in each region. A line <NUM> indicates a boundary line between regions having different defocus amounts.

In the defocus map, a region to which "<NUM>" is assigned indicates an error-free region in the focusing unit of an imaging apparatus, a region to which "-<NUM>" is assigned indicates a region of the front focus in one focusing unit of the imaging apparatus, and a region to which "<NUM>" is assigned indicates a region of the rear focus in one focusing unit of the imaging apparatus. The focusing unit may be any level-based unit that enables the determination of the focusing state (degree). For example, the width of the defocus amount, such as <NUM> or <NUM> in the depth direction, can be defined as one unit. There is a value determined by the ratio of the size of the image sensor of the imaging apparatus to the size of the permissible circle-of-confusion of an optical lens system including a diaphragm.

In the following descriptions, configurations which have already been described above in the first exemplary embodiment are assigned the same reference numerals, and redundant descriptions thereof will be omitted. The following descriptions will be made centering on differences from the first exemplary embodiment. The functional configuration of the information processing apparatus <NUM> according to the first modification is similar to that according to the first exemplary embodiment illustrated in <FIG>. However, the modification differs from the first exemplary embodiment in the operations of the image quality evaluation unit <NUM> and in the threshold values held by the threshold information holding unit <NUM>. The image quality evaluation unit <NUM> according to the first modification acquires the defocus amount information accompanying input images acquired by the image input unit <NUM>. According to the first modification, the defocus amount information is information accompanying images. When images are RAW data holding information about an imaging plane phase difference image sensor, the defocus amount information is calculated based on phase difference information.

The threshold information holding unit <NUM> holds threshold values in tabular form. <FIG> illustrates an example of a table <NUM> held by the threshold value holding unit <NUM> according to the first modification. The table <NUM> holds a threshold value <NUM> for the feature point evaluation value and a threshold value <NUM> for the image quality evaluation value. The threshold value <NUM> for the image quality evaluation value in the table <NUM> refers to a threshold value for the value of the defocus amount corresponding to the image. According to the first modification, regions where the value of the defocus amount is <NUM> or less are regions available for inspection. More specifically, referring to the defocus map illustrated in <FIG>, for example, image regions other than some regions on the right-hand side, where the value of the defocus amount is <NUM> or larger, are regions available for inspection.

<FIG> illustrates the image stitching processing performed by the image stitching unit <NUM> and corresponds to <FIG> according to the first exemplary embodiment. A three-dimensional shape <NUM> is the three-dimensional shape estimated by the shape estimation unit <NUM>. Referring to <FIG>, positions <NUM> and <NUM> indicate the positions of the principal points of the camera(s) at different positions. Imaging planes <NUM> and <NUM> represent imaging planes of the camera(s) at these different positions. Referring to <FIG>, like <FIG>, the imaging planes are disposed in front of the principal points in consideration of the understandability of the drawing. In addition, the imaging planes <NUM> and <NUM> represent crack images <NUM> and <NUM>, respectively, as defect images focused on respective planes, and the crack images <NUM> and <NUM> are captured and recorded.

Regions <NUM> and <NUM> are regions available for inspection in a two-dimensional array input from the image evaluation unit <NUM>. Projection images <NUM> and <NUM> represent the projection of the regions <NUM> and <NUM> available for inspection, out of the images on the imaging planes <NUM> and <NUM>, respectively, onto the three-dimensional shape <NUM>.

The image stitching unit <NUM> generates the projection images <NUM> and <NUM> in all of the images where the regions available for inspection exist for the three-dimensional shape <NUM>, and projects these projection images onto the three-dimensional shape <NUM>. Like the first exemplary embodiment, the image stitching unit <NUM> also performs blending processing on a region where images are overlapping, such as the region <NUM>. Also, in the first modification, the images on the imaging planes <NUM> and <NUM> in the information processing apparatus <NUM> are subjected to the feature point evaluation by the feature evaluation unit <NUM>. Thus, the projection images <NUM> and <NUM> having a small amount of positional deviation in the images can be obtained. Even when the object is captured from an oblique position such as the position <NUM> of the principal point of the camera, the information processing apparatus <NUM> according to the first modification makes it possible to obtain the projection image <NUM> since the region <NUM> available for inspection exists.

The processing of the information processing apparatus <NUM> according to the first modification of the first exemplary embodiment is performed according to the flowcharts illustrated in <FIG>, <FIG>, <FIG>, and <FIG>. The flowcharts illustrated in <FIG> and <FIG> are similar to those according to the first exemplary embodiment, and redundant descriptions thereof will be omitted. In the image quality evaluation processing illustrated in <FIG>, the processing of the flowchart illustrated in <FIG> is performed instead of the flowchart illustrated in <FIG>.

The image quality evaluation processing in step S604 illustrated in <FIG> according to the first modification will be described in detail below with reference to the flowchart illustrated in <FIG>. In step S2101, the image quality evaluation unit <NUM> acquires defocus amount distribution information for the images input from the image input unit <NUM>. In step S2102, the image evaluation unit <NUM> compares information for the defocus amount for each pixel in the images supplied by the image quality evaluation unit <NUM> with the threshold value <NUM> for the image quality evaluation value held by the threshold value holding unit <NUM> to determine whether there exists a pixel having a defocus amount larger than the threshold value. When the image evaluation unit <NUM> determines that there exists a pixel having a defocus amount larger than the threshold value (YES in step S2102), the processing of the information processing apparatus <NUM> proceeds to step S2103. On the other hand, when the image evaluation unit <NUM> determines that there exists no pixel having a defocus amount larger than the threshold value (NO in step S2102), the information processing apparatus <NUM> ends the image quality evaluation processing.

In step S2103, the image evaluation unit <NUM> outputs position information for a pixel identified to have a defocus amount larger than the threshold value in S2102 to the image stitching unit <NUM>, as information about the region available for inspection. Then, the information processing apparatus <NUM> ends the image quality evaluation processing.

As discussed above, the information processing apparatus <NUM> according to the first modification of the first exemplary embodiment evaluates the feature points and the defocus amount of images, and projects images with the preferable evaluation results to a three-dimensional shape. More specifically, the first modification makes it possible to reduce image stitching failure through the stitching processing using the feature points available for accurate positioning and to stitch images having a defocus amount available for defect inspection.

The use of the defocus amount for the evaluation in image stitching makes it possible to use only the regions where the focus is within a threshold value, thus eliminating image failure caused due to a physical error factor that had been unavoidable in the geometric resolution estimation according to the first exemplary embodiment. If a stitched image becomes an in-focus unclear image, this problem can be handled as a report item when performing image capturing again as a problem of image shake in image capturing. Alternatively, if the amount of image shake width can be confirmed in the unclear image, it is possible to perform defect inspection in consideration of image shake instead of performing image capturing again. Thus, the first modification makes it possible to acquire defocus-amount-based available images for the defect evaluation on an object such as an infrastructure and therefore to recognize defects of the object.

According to the first exemplary embodiment and the first modification, the processing of the information processing apparatus <NUM> ends when the image stitching unit <NUM> projects input image regions where the image quality satisfies the allowable conditions, onto the three-dimensional shape estimated based on the feature amount of the input images. A second modification of the first exemplary embodiment will be described below centering on a case where processing for generating inspection images to be used for object inspection works is added.

Inspection of an object specifically refers to identifying and evaluating cracks and other defects based on captured images of a structure (infrastructure) as an object under inspection. Defect identification works are manually performed. Alternatively, defect identification works are performed by detecting defects in the images by using a learnt model that have learned. In either case, it is preferable to use images viewed from a viewpoint facing the surface under inspection of the structure. For example, when defects are cracks, a crack width is viewed in different ways between a case where the structure is viewed from a viewpoint facing the structure and a case where the structure is viewed from a viewpoint tilted by a tilt angle. More specifically, in an image captured in tilt image capturing, cracks having the same width existing on the near and far sides in the image are viewed in different ways. Therefore, it is necessary to determine the crack width based on different criteria. When performing defect identification works within a wide range of a large structure, it is troublesome to minutely change the determination criterion in consideration of the tilt angle for each region. To identify defects based on determination criterion unified as much as possible on the entire portion of a large structure in this way, it is preferable to use inspection images equivalent to images of the structure captured from a position facing each region of the structure.

According to the first exemplary embodiment and the first modification, the image stitching unit <NUM> can generate a three-dimensional model in which input image regions where the image quality satisfies the allowable conditions are projected onto the three-dimensional shape estimated based on the feature amounts of the input images. Therefore, inspection images for the structure can be generated by a generation unit as a function of the CPU <NUM>. More specifically, with the position of a virtual camera arbitrarily set, the generation unit clips partial images viewed from a position facing the structure to a size suitable for processing in the subsequent stage to generate inspection images. The information processing apparatus <NUM> displays the generated inspection images on the display apparatus <NUM>. When manually performing inspection, an inspector identifies crack portions based on the displayed inspection images and inputs crack information by using the input apparatus <NUM>. For example, cracks of the inspection images are traced by using a pointing device.

A second exemplary embodiment will be described below.

According to the second exemplary embodiment, unlike the first exemplary embodiment, the image quality evaluation unit <NUM> also evaluates not only the resolution on the object but also the angle between light incident to the camera and a surface of an object. In the following descriptions, configurations which have already been described above in the first exemplary embodiment are assigned the same reference numerals, and redundant descriptions thereof will be omitted. The image quality evaluation unit <NUM> different from that according to the first exemplary embodiment will be described in detail below.

For each pixel in the images input from the image input unit <NUM>, the image quality evaluation unit <NUM> estimates the angle between light incident to the camera and the object surface based on the three-dimensional shape estimated by the shape estimation unit <NUM> and the camera position and orientation estimated by the position and orientation estimation unit <NUM>.

The angle between light incident to the camera and the object surface will be described below with reference to <FIG>.

The angle between light incident to the camera and the object surface refers to an angle <NUM> illustrated in <FIG>. The angle <NUM> is an angle corresponding to a position <NUM> in the image on the imaging plane <NUM>. The angle <NUM> is an angle made by lines <NUM> and <NUM>. The line <NUM> is a line starting from the principal point <NUM> of the camera and passing through the position <NUM> in the image. The line <NUM> is a line connecting intersections <NUM> and <NUM>. The intersection <NUM> is an intersection between the line <NUM> and the three-dimensional shape <NUM> of the object surface. The intersection <NUM> is an intersection between a line <NUM> and the three-dimensional shape <NUM> of the object surface. The line <NUM> is a line starting from the principal point <NUM> of the camera and passing through the image center.

When the angle <NUM> decreases, cracks appearing on the imaging plane <NUM> appears thin in the image. Therefore, when actually performing inspection by using the stitched image generated by the information processing apparatus <NUM>, the crack width may be incorrectly recognized thinner than it actually is.

Therefore, the present exemplary embodiment is intended to reduce the possibility that the crack width is incorrectly recognized thinner than it actually is by setting a predetermined value or larger to the angle between light incident to the camera and the object surface. According to the present exemplary embodiment, the predetermined value is, for example, <NUM> degrees or more.

The information processing apparatus <NUM> according to the present exemplary embodiment holds the threshold value for the angle between light incident to the camera and the object surface in tabular form in the threshold value holding unit <NUM>, and adds the threshold value, for example, to the line below the threshold value <NUM> for the image quality evaluation value illustrated in <FIG>.

Then, for each pixel in the images input from the image input unit <NUM>, the image quality evaluation unit <NUM> evaluates whether the angle between light incident to the camera and the object surface is equal to or larger than a threshold value, in addition to the resolution on the object. In the case of the example illustrated in <FIG>, the region where an angle of light incident to the camera with respect to the object surface is equal to or larger than the threshold value is inside a region <NUM>.

<FIG> is a flowchart illustrating the flow of the image quality evaluation processing of the information processing apparatus <NUM> according to the second exemplary embodiment. The processing in steps S702 and S703 of the flowchart illustrated in <FIG> is similar to the processing in steps S702 and S703 of the flowchart illustrated in <FIG>, respectively, and redundant descriptions thereof will be omitted. According to the second exemplary embodiment, processing other than the image quality evaluation processing in step S604 according to the first exemplary embodiment is similar to the processing flow according to the first exemplary embodiment.

The image quality evaluation processing in step S604 according to the second exemplary embodiment will be described below with reference to the flowchart illustrated in <FIG>.

In step S1001, with respect to the images input from the image input unit <NUM>, the image quality evaluation unit <NUM> calculates the angle between light incident to the camera and the object surface for each pixel. Then, the processing of the image quality evaluation unit <NUM> proceeds to step S1002.

In step S1002, the image quality evaluation unit <NUM> reads the threshold value for the angle between light incident to the camera and the object surface, from the threshold value holding unit <NUM>. Then, the image quality evaluation unit <NUM> identifies regions where the angle between light incident to the camera and the object surface is equal to or larger than the threshold value. The image quality evaluation unit <NUM> further estimates the resolution distribution on the object for pixels belonging to the regions where the angle between light incident to the camera and the object surface is equal to or larger than the threshold value. Then, the processing of the information processing apparatus <NUM> proceeds to step S702. The processing in steps S702 and S703 is similar to the processing in the flowchart according to the first exemplary embodiment illustrated in <FIG>, and redundant descriptions thereof will be omitted.

As described above, the information processing apparatus <NUM> according to the second exemplary embodiment makes it possible to reduce the possibility that the crack width is incorrectly recognized thinner than it actually is by evaluating the angle between light incident to the camera and the object surface.

A third exemplary embodiment will be described below. In the following descriptions, configurations which have already been described above in the first exemplary embodiment are assigned the same reference numerals, and redundant descriptions thereof will be omitted.

<FIG> is a function block diagram illustrating a functional configuration of the information processing apparatus <NUM> according to the third exemplary embodiment. The configuration illustrated in <FIG> includes the configuration according to the first exemplary embodiment illustrated in <FIG> and an additional display unit <NUM>. The display unit <NUM> will be mainly described below.

According to the third exemplary embodiment, the information processing apparatus <NUM> is connected, for example, to a camera. The image input unit <NUM> sequentially acquires images of an object captured by the camera. Then, the display unit <NUM> instructs the display apparatus <NUM> to display a stitched image generated by the image stitching unit <NUM>. More specifically, the display unit <NUM> instructs the display apparatus <NUM> to display the stitched image generated by the image stitching unit <NUM> to present the stitched image to the user who is imaging the target object. The information processing apparatus <NUM> according to the third exemplary embodiment may be built in the camera.

<FIG> illustrates display contents on the display unit <NUM> according to the third exemplary embodiment. The display unit <NUM> displays the three-dimensional shape <NUM> estimated by the shape estimation unit <NUM>, and the projection images <NUM> and <NUM> generated by the image stitching unit <NUM>, on a display screen <NUM> of the display apparatus <NUM>. More specifically, the display unit <NUM> continues the display corresponding to the sequentially input images to the image input unit <NUM> while sequentially updating the display.

According to the third exemplary embodiment, the image evaluation unit <NUM> evaluates new images sequentially acquired by the image input unit <NUM> but does not re-evaluate images that have already been evaluated. Therefore, according to the third exemplary embodiment, when the image evaluation is completed for all of the images in the processing for evaluating the images available for inspection in step S601 illustrated in <FIG> according to the first exemplary embodiment (YES in step S601), the image evaluation unit <NUM> ends the processing. On the other hand, when the image evaluation is not completed for all of the images (NO in step S601), the processing proceeds to step S602.

If the shape estimation unit <NUM> does not acquired at least two images from the image input unit <NUM>, the shape estimation unit <NUM> cannot estimate the three-dimensional shape. Therefore, the display unit <NUM> starts displaying the estimated three-dimensional shape <NUM> after a predetermined number of images are input from the image input unit <NUM> and the shape estimation is performed by the shape estimation unit <NUM>. According to the present exemplary embodiment, the display unit <NUM> starts the display after <NUM> images have been input from the image input unit <NUM>.

In the processing of the information processing apparatus <NUM> according to the third exemplary embodiment, processing other than the post-processing in step S507 according to the first exemplary embodiment is similar to the processing according to the first exemplary embodiment. Therefore, the post-processing in step S507 according to the present exemplary embodiment will be described below with reference to the flowchart illustrated in <FIG>. Processing in steps S801, S802, S803, and S804 illustrated in <FIG> is similar to the processing assigned the same reference numerals illustrated in <FIG>, and redundant descriptions thereof will be omitted.

Referring to <FIG>, upon completion of the processing in step S804, the processing of the information processing apparatus <NUM> proceeds to step S1301.

In step S1301, the display unit <NUM> displays the three-dimensional shape <NUM> and the projection images <NUM> and <NUM> on the display screen <NUM>. Then, the information processing apparatus <NUM> ends the post-processing in step S507.

As discussed above, the display unit <NUM> of the information processing apparatus <NUM> according to the third exemplary embodiment displays the three-dimensional shape <NUM> and the projection images <NUM> and <NUM> corresponding to the sequentially acquired input images, on the display screen <NUM>. This enables the user to reference the display on the display screen <NUM> during image capturing of the object to recognize regions where inspectable images have been captured and regions where inspectable images have not been captured on the object.

A fourth exemplary embodiment will be described below. In the following descriptions, configurations which have already been described above in the first exemplary embodiment are assigned the same reference numerals, and redundant descriptions thereof will be omitted.

<FIG> is a function block diagram illustrating a functional configuration of the information processing apparatus <NUM> according to the fourth exemplary embodiment. The configuration according to the fourth exemplary embodiment includes the functional configuration according to the first exemplary embodiment illustrated in <FIG> and a defect detection unit <NUM>, an estimation error holding unit <NUM>, and a display unit <NUM> as additional configurations, instead of the image stitching unit <NUM>. Therefore, the defect detection unit <NUM>, the estimation error holding unit <NUM>, and the display unit <NUM> will be described below.

The defect detection unit <NUM> detects cracks as examples of defects based on the images input from the image input unit <NUM>. As a crack detection method, a method for extracting edges by using a Sobel filter is applicable. In addition, the defect detection unit <NUM> performs noise removal processing and labeling processing as labeling to the result of crack detection to remove small-area labels. With respect to the labeled cracks, the defect detection unit <NUM> further references resolution information on the object input from the image quality evaluation unit <NUM>. Then, the defect detection unit <NUM> reads the resolution information on the object corresponding to the labeled crack positions, associates the resolution information with the labeled cracks, and outputs the resolution information to the display unit <NUM>.

If there are two or more pieces of resolution information on the object corresponding to the labeled cracks, the defect detection unit <NUM> supplies information with the lowest resolution. For example, when there are two different resolutions "<NUM> pixel per <NUM>" and "<NUM> pixels per <NUM>", the defect detection unit <NUM> supplies "<NUM> pixels per <NUM>".

The estimation error holding unit <NUM> holds a table for the estimation accuracy of the crack width corresponding to the resolution on the object. <FIG> illustrates an example of a table <NUM> held by the estimation error holding unit <NUM>. Referring to the table <NUM>, accuracies <NUM>, <NUM>, and <NUM> represent the estimation accuracy of the crack width corresponding to the resolution on the object. For example, the accuracy <NUM> means that, when image capturing is performed with a resolution of <NUM> pixels per <NUM> on the object, the estimation accuracy of the crack width becomes ±<NUM>.

The display unit <NUM> displays information on the display screen to the user who is imaging the object. The display unit <NUM> according to the fourth exemplary embodiment continues the display corresponding to the images sequentially input to the image input unit <NUM> while updating the display. In the fourth exemplary embodiment, like the third exemplary embodiment, the display unit <NUM> starts the display after <NUM> images have been input from the image input unit <NUM>.

<FIG> illustrates detailed contents displayed on a display screen <NUM> of a camera <NUM> by the display unit <NUM>. More specifically, referring to <FIG>, the information processing apparatus <NUM> according to the present exemplary embodiment is built in the camera <NUM>. The display unit <NUM> displays the images input from the image input unit <NUM>, on the display screen <NUM> of the camera <NUM>. In the example illustrated in <FIG>, the display screen <NUM> displays an image capturing target object <NUM> on the images input from the image input unit <NUM>. In this example, the camera <NUM> performs image capturing from an oblique direction with respect to the surface of the image capturing target object <NUM>.

The display unit <NUM> superimposes information <NUM> about the regions available for inspection output from the image evaluation unit <NUM> onto the images input from the image input unit <NUM>. This display enables the user to grasp which range of the image capturing target object <NUM> can be captured as regions available for inspection.

In addition, the display unit <NUM> reads information about the evaluation of feature points from the image evaluation unit <NUM> and displays feature point evaluation information <NUM> on the display screen <NUM>. The feature point evaluation information <NUM> displays, for example, "o" and "x" based on the threshold value <NUM> for the feature point evaluation held by the threshold value holding unit <NUM>. "o" indicates that the feature point evaluation information <NUM> is equal to or larger than the threshold value <NUM>, and "x" indicates that the feature point evaluation information <NUM> is less than the threshold value <NUM>.

In addition, the display unit <NUM> acquires the information about the labeled cracks and the resolution information on the object associated with the labels, input from the defect detection unit <NUM>. Then, the display unit <NUM> references the table of the estimation error holding unit <NUM> and acquires crack width estimation error information based on the resolution information on the object.

The display unit <NUM> displays crack portions based on the crack information input from the defect detection unit <NUM>, on the display screen <NUM> as crack information <NUM> and <NUM>.

Finally, the display unit <NUM> superimposes crack width estimation error information <NUM> and <NUM> corresponding to the crack information <NUM> and <NUM>, respectively, on the display screen <NUM>.

<FIG> is a flowchart illustrating the post-processing flow in the information processing apparatus <NUM> according to the fourth exemplary embodiment. In the processing flow of the information processing apparatus <NUM> according to the fourth exemplary embodiment, processing other than the post-processing in step S507 according to the first exemplary embodiment is similar to the processing according to the first exemplary embodiment. The post-processing in step S507 according to the fourth exemplary embodiment will be described below.

In step S1701, the display unit <NUM> displays the images input from the image input unit <NUM> on the display screen <NUM>. Then, the processing of the display unit <NUM> proceeds to step S1702.

In step S1702, the display unit <NUM> superimposes the information <NUM> about the regions available for inspection output from the image evaluation unit <NUM>, onto the images input from the image input unit <NUM>. Then, the processing of the display unit <NUM> proceeds to step S1703.

In step S1703, the display unit <NUM> reads the feature point evaluation information from image evaluation unit <NUM> and displays the feature point evaluation information <NUM> on the display screen <NUM>. Upon completion of step S1703, the processing of the information processing apparatus <NUM> proceeds to step S1704.

In step S1704, the defect detection unit <NUM> detects cracks based on the images input from the image input unit <NUM>. Then, the processing of the defect detection unit <NUM> proceeds to step S1705.

In step S1705, the defect detection unit <NUM> references the resolution information on the object input from the image quality evaluation unit <NUM>. Then, the defect detection unit <NUM> reads the resolution information on the object corresponding to the labeled crack positions, associates the resolution information with the labeled cracks, and outputs the resolution information to the display unit <NUM>. Upon completion of step S1705, the processing of the information processing apparatus <NUM> proceeds to step S1706.

In step S1706, the display unit <NUM> displays crack portions based on the crack information input from the defect detection unit <NUM>, on the display screen <NUM>. Then, the processing of the display unit <NUM> proceeds to step S1707.

In step S1707, the display unit <NUM> superimposes the crack width estimation error information on the display screen <NUM>. Then, the information processing apparatus <NUM> ends the post-processing in step S507.

As discussed above, the information processing apparatus <NUM> according to the fourth exemplary embodiment displays the information about the regions available for inspection, the feature point evaluation information, the crack information, and the crack width estimation error information on the display screen <NUM> of the display unit <NUM>. The information displayed in this way enables the user to reference the display screen <NUM> at the time of image capturing of the object to recognize which region of the object can be captured with an image quality available for inspection at the current capturing angle. The information also enables the user to reference the feature point evaluation information to recognize whether the captured image can be accurately projected at the time of image projection. In addition, the information enables the user to reference the crack information and the crack width estimation error information to confirm whether cracks can be detected and at the same time confirm a measure of the crack width estimation accuracy.

Although, in the fourth exemplary embodiment, the defect detection processing is performed instead of the above-described image stitching processing according to the first to third exemplary embodiments, the present invention is not limited thereto. The above-described defect detection unit <NUM> may be used for the defect detection processing for stitched images or the defect detection processing for inspection images according to the second modification of the first exemplary embodiment.

According to the present invention, it becomes possible to acquire images available for the evaluation of defects of an object such as an infrastructure and recognize defects of the object.

Claim 1:
An information processing apparatus comprising:
feature extraction means (<NUM>) configured to extract feature points from a plurality of images of an object to be inspected captured from a plurality of viewpoints by extracting feature points for each of the plurality of images;
image quality evaluation means (<NUM>) configured to, for each of the plurality of images, each of which including a plurality of regions, evaluate whether an image quality based on a predetermined index satisfies allowable conditions for inspection works of the object, the image quality evaluation means (<NUM>) evaluating a resolution per length unit or area unit of the object for each region based on internal parameters, including image resolution, of a camera used to capture the plurality of images, and using the evaluated resolution as the predetermined index, and
wherein, for each of the plurality of regions in each of the plurality of images, the image quality evaluation means evaluates whether the image quality based on the predetermined index satisfies the allowable conditions for inspection works of the object; and
image stitching means (<NUM>) configured to stitch at least a part of images included in the plurality of images and having the image quality that satisfies the allowable conditions according to a positional relation based on the feature points extracted by the extraction means (<NUM>), the image stitching means (<NUM>) stitching regions included in the plurality of images and having the image quality that satisfies the allowable conditions according to the positional relation based on the extracted feature points.