Patent Description:
Conventionally, various techniques for inspecting the appearance of a to-be-inspected object such as a tire have been studied (see, for example, <CIT>).

<CIT> relates to appearance inspection for tires wherein pictures are taken from the tires to be inspected and the tires are inspected with reference to a master image for detecting shape anomaly. Red illumination and blue illumination having different wave lengths from each other are directed to the surface of a rotating tire from different directions so as to overlap the cast lights with each other. Then, the surface images are taken, and the red component and the blue component are separated from each other by means of image processing, and the presence or absence of unevenness of the tire surface is determined from the ratio between the intensities of the red component and the blue component.

The publication "<NPL>, discusses a method of classifying defects and non-defectives generated on background patterns on an inner tire surface using data obtained by imaging the entire inner tire surface. In this regard, a method of imaging and a method of image creation, particularly the generation of 3D shape images, are being discussed. Then, based on the image, it is detected whether or not defects are present, and the objects/tires are classified as good parts, quasi-good parts and defective parts.

<CIT> refers to a method in which acceptability of shape of a tire surface is determined by comparing an image against a reference image.

In the appearance inspection, further improvement in inspection accuracy is expected.

The present invention has been made in view of the above circumstances, and a main object of the present invention is to provide an appearance inspection method and an appearance inspection apparatus that are capable of detecting an appearance defect with high accuracy.

A first aspect of the present invention is directed to a method for inspecting appearance of a to-be-inspected object, the appearance inspection method being executed using an appearance inspection apparatus and including: an imaging step of imaging a plurality of the to-be-inspected objects using a camera of an imaging unit and acquiring images thereof; a first sorting step of sorting the images as non-defective product estimation images within a predetermined standard and defective product estimation images outside the standard by using an algorithm of one class learning, the first sorting step being executed using a processing unit of a computer device; and a second sorting step of detecting a defective product from the to-be-inspected objects by extracting features of defects from the defective product estimation images using an algorithm of object detection, and searching the images for the features by comparing the image data acquired in the imaging step with the features of the defects from the defective product estimation images sorted in the first sorting step, the second sorting step being executed using the processing unit.

In the appearance inspection method according to the present invention, in the second sorting step, the features are preferably classified and accumulated for each of types of the defects.

In the appearance inspection method according to the present invention, in the second sorting step, the feature extracted from the image of the to-be-inspected object detected as the defective product in the second sorting step after being sorted as the non-defective product estimation image in the first sorting step is preferably accumulated.

In the appearance inspection method according to the present invention, the to-be-inspected objects are preferably tires.

In the appearance inspection method according to the present invention, an annular imaging area at a sidewall portion is preferably imaged in the imaging step.

In the appearance inspection method according to the present invention, the imaging area is preferably converted into a band shape in the imaging step.

A second aspect of the present invention is directed to an appearance inspection apparatus for inspecting appearance of a to-be-inspected object, the appearance inspection apparatus including: an imaging unit configured to image a plurality of the to-be-inspected objects and acquire images thereof; a storage unit configured to store the images; a first sorting unit configured to sort the images as non-defective product estimation images within a predetermined standard and defective product estimation images outside the standard by using an algorithm of one class learning; and a second sorting unit configured to detect a defective product from the to-be-inspected objects by extracting features of defects from the defective product estimation images using an algorithm of object detection, and searching the images for the features by comparing the image data acquired by the imaging unit with the features of the defects from the defective product estimation images sorted by the first sorting unit.

In the appearance inspection apparatus according to the present invention, preferably, the second sorting unit classifies the features by types of the defects, and the storage unit accumulates the features for each of the types.

In the appearance inspection apparatus according to the present invention, the imaging unit preferably includes a 2D camera.

In the appearance inspection apparatus according to the present invention, the imaging unit preferably includes a 3D camera.

The appearance inspection method according to the first aspect includes the first sorting step using the algorithm of the one class learning, and the second sorting step using the algorithm of the object detection. In the first sorting step, each of the images acquired in the imaging step is sorted as the non-defective product estimation image or the defective product estimation image. In the second sorting step, the features of the defects are extracted from the defective product estimation images. Furthermore, the images are searched for the features, and the presence/absence of the defects in the to-be-inspected objects is determined. Accordingly, the defective product is accurately detected from the to-be-inspected objects.

In the first aspect, as the acquired images are accumulated with progress of inspection, the number of the defective product estimation images obtained in the first sorting step increases, and the number of the features of the defects extracted in the second sorting step increases automatically. Therefore, by inspecting many to-be-inspected objects, the accuracy of detecting the defective product is easily improved.

The appearance inspection apparatus according to the second aspect includes the first sorting unit using the algorithm of the one class learning, and the second sorting unit using the algorithm of the object detection. The first sorting unit sorts each of the images acquired by the imaging unit, as the non-defective product estimation image or the defective product estimation image. The second sorting unit extracts the features of the defects from the defective product estimation images. Furthermore, the second sorting unit searches the images for the features, and determines the presence/absence of the defects in the to-be-inspected objects. Accordingly, the defective product is accurately detected from the to-be-inspected objects.

In the second aspect, as the acquired images are accumulated with progress of inspection, the number of the defective product estimation images obtained by the first sorting unit increases, and the number of the features of the defects extracted by the second sorting unit increases automatically. Therefore, by inspecting many to-be-inspected objects, the accuracy of detecting the defective product is easily improved.

<FIG> shows an example of a processing procedure of an embodiment of the appearance inspection method according to the first aspect. The appearance inspection method is a method for inspecting the appearance of a to-be-inspected object. Inspecting the appearance of the to-be-inspected object is to determine whether a defect (abnormality) has occurred in the appearance of the to-be-inspected object. The appearance inspection method is performed, for example, in combination with or instead of visual inspection by an operator.

The to-be-inspected object is not particularly limited. In the present embodiment, industrial products such as tires are used as the to-be-inspected object. In this case, the appearance inspection method according to the first aspect is performed at a production line for the industrial products. A product whose appearance is determined to be defective is discarded without being shipped, or is repaired for the defect and then shipped.

Hereinafter, inspection of the appearance of tires will be described, but the description can be applied to inspection of the appearance of other industrial products, by replacing the tires with the industrial products.

The appearance inspection method includes an imaging step S1, a first sorting step S2, and a second sorting step S3. The appearance inspection method is executed using an appearance inspection apparatus <NUM>. The appearance inspection apparatus <NUM> is an apparatus for inspecting the appearance of a to-be-inspected object.

<FIG> shows a schematic configuration of the appearance inspection apparatus <NUM>. The appearance inspection apparatus <NUM> is an apparatus for inspecting the appearance of a to-be-inspected object. The appearance inspection apparatus <NUM> includes an imaging unit <NUM> and a calculation unit <NUM>. For example, a computer device <NUM> is used as the calculation unit <NUM>.

The imaging unit <NUM> includes a camera <NUM> including an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera <NUM> images a plurality of tires <NUM> as to-be-inspected objects <NUM> and acquires images thereof. More specifically, the camera <NUM> converts, for example, light reflected by the tire <NUM>, into an electric signal and acquires electronic data of the image. The image data acquired by the camera <NUM> is transmitted to the computer device <NUM> and stored therein.

The imaging unit <NUM> is provided with an illumination device <NUM> for illuminating the tire <NUM>, which is a subject of the camera <NUM>, as required. In addition, the imaging unit <NUM> is provided with a device (not shown) for positioning and rotating the conveyed tire <NUM>. The camera <NUM>, the illumination device <NUM>, the above-described device, and the like are controlled, for example, by the computer device <NUM>.

<FIG> shows a schematic configuration of the computer device <NUM>. The computer device <NUM> includes a processing unit <NUM> that executes various kinds of arithmetic processing and information processing, a storage unit <NUM> in which a program for controlling operation of the processing unit <NUM> and various kinds of information are stored, an input unit <NUM> for inputting various commands and various kinds of information to the processing unit <NUM>, an output unit <NUM> for outputting results of processing by the processing unit <NUM>, etc. The processing unit <NUM> of the present embodiment also controls the above-described camera <NUM>, the above-described illumination device <NUM>, etc..

The processing unit <NUM> includes, for example, a central processing unit (CPU) and a memory. In addition to the CPU or instead of the CPU, a graphics processing unit (GPU) that excels in specific data processing may be used in the processing unit <NUM>. For example, a largecapacity hard disk drive is used as the storage unit <NUM>.

A device such as a keyboard and a mouse, and a device that is connected to the imaging unit <NUM> to receive an input of image data from the imaging unit <NUM> are used as the input unit <NUM>. A display device such as a liquid crystal display (LCD) is used as the output unit <NUM>.

The computer device <NUM> may be provided with a communication unit (not shown) for exchanging various kinds of information (for example, information to be used for inspecting the appearance of the tires <NUM>) with another computer device (not shown). For example, the communication unit is connected to the other computer device via a local area network (LAN). In this case, the calculation unit <NUM> is composed of a network including the computer device <NUM> and the other computer device.

Hereinafter, a processing procedure of the appearance inspection method, that is, operation of the appearance inspection apparatus <NUM>, will be described with reference to <FIG>.

In the imaging step S1 shown in <FIG>, the plurality of tires <NUM> are imaged by the camera <NUM>, and images thereof are acquired as electronic data. The image data acquired in the imaging step S1 is transmitted to the computer device <NUM> and stored in the storage unit <NUM> by the processing unit <NUM>.

The first sorting step S2 and the second sorting step S3 are executed by the processing unit <NUM> of the computer device <NUM>.

In the first sorting step S2, each image acquired in the imaging step S1 is sorted as a non-defective product estimation image <NUM> or a defective product estimation image <NUM> using an algorithm of one class learning (One Class Support Vector Machine). The non-defective product estimation image <NUM> is the image of the tire <NUM> whose appearance is estimated to be good by the one class learning, and is stored, for example, in a non-defective product image folder <NUM> of the storage unit <NUM>. On the other hand, the defective product estimation image <NUM> is the image of the tire <NUM> whose appearance is estimated not to be good by the one class learning, and is stored, for example, in a defective product image folder <NUM> of the storage unit <NUM>.

The one class learning is an algorithm for detecting outliers from learning values of non-defective products learned without supervision, and is a method suitable for abnormality inspection of industrial products, such as the tires <NUM>, the majority of which are non-defective products. In the present embodiment, for example, the learning values are acquired by learning image data of non-defective tires. For learning image data, for example, a method of deep learning is used.

<FIG> shows an example of a learning model generated by deep learning. In the present embodiment, a learning model <NUM> is generated by calculation of the processing unit <NUM> and stored in the storage unit <NUM>. The learning model <NUM> may be, for example, generated by calculation of another computer device outside the computer device <NUM>, inputted into the computer device <NUM>, and stored in the storage unit <NUM>.

The learning model <NUM> is defined, for example, by an intermediate layer <NUM> generated by machine learning using image data 701a, 701b, 701c, 701d, 701e. of a plurality of non-defective tires as an input layer <NUM> and learning values 703a, 703b, 703c. of the non-defective products as an output layer <NUM>.

The intermediate layer <NUM> of the present embodiment includes a combination of a plurality of neurons (nodes) <NUM> hierarchized in multiple stages and optimized weighting factors <NUM> (parameters). The respective neurons <NUM> are connected by the weighting factors <NUM>. Such an intermediate layer <NUM> is referred to as a convolutional neural network. That is, the learning model <NUM> of the present embodiment includes a convolutional neural network.

In the first sorting step S2, outliers from the learning values 703a, 703b, 703c. of the non-defective products are detected by using the one class learning, and thus new (unlearned) defects that are difficult to assume can also be detected. Therefore, it is possible to easily collect various types of defective product estimation images <NUM>.

In the second sorting step S3, the tire <NUM> that is a defective product is detected by using an algorithm of object detection. The object detection is an algorithm for detecting an object (a defect in the present embodiment) in an image of a to-be-inspected object. For example, a learning model including a convolutional neural network is used in the object detection.

In the second sorting step S3 of the present embodiment, the image data acquired in the imaging step S1 is searched for features of defects. The features of the defects are sorted in the first sorting step S2 and extracted from the defective product estimation images <NUM> stored in the storage unit <NUM>. That is, the processing unit <NUM> extracts features of defects from the defective product estimation images <NUM> and, for example, stores the features of the defects in a defect folder <NUM> of the storage unit <NUM>. The extracted features of the defects include features that cannot be understood by humans due to the convolution processing. Then, the processing unit <NUM> searches the image data for defects by comparing the image data acquired in the imaging step S1 with the features of the defects, and detects a defective product from the imaged tires <NUM>. That is, if a defect exists in the image data, the tire <NUM> from which the image data is acquired is determined to be a defective product.

<FIG> shows examples of a defect <NUM> detected in the second sorting step S3. In the example, a defect <NUM> that occurs at a part of a sidewall portion <NUM> (see <FIG>) of the tire <NUM> is shown. In <FIG>, occurrence of the defect <NUM> on the appearance due to poor rubber flow or the like at a character "E" formed on the surface of the sidewall portion <NUM> is confirmed. In addition, occurrence of a similar defect <NUM> near an indication "<NUM>/60R16" indicating the tire size is also confirmed. According to the present embodiment, not only the defects <NUM> shown in <FIG> but also defects such as foreign matter mixed into the sidewall portion <NUM> can be detected.

As for industrial products such as the tires <NUM>, the majority of the products are usually non-defective products, and the frequency of occurrence of defective products is low. Therefore, it is generally difficult to collect features of various defects <NUM> and store the features in the storage unit <NUM>. However, in the present embodiment, as the images captured by the camera <NUM> in the imaging step S1 are accumulated with progress of inspection, the number of defective product estimation images <NUM> obtained in the first sorting step S2 increases, and the number of features of defects <NUM> extracted in the second sorting step S3 and accumulated in the storage unit <NUM> also increases. Therefore, by inspecting many tires <NUM>, the accuracy of detecting a defective product is easily improved.

In the present embodiment, in the first sorting step S2, the processing unit <NUM> serves as a first sorting unit that sorts images acquired by the camera <NUM> as non-defective product estimation images <NUM> and defective product estimation images <NUM> by using the algorithm of the one class learning. In addition, in the second sorting step S3, the processing unit <NUM> serves as a second sorting unit that detects a defective product from the tires <NUM> imaged by the camera <NUM>, by using the algorithm of the object detection. That is, the first sorting unit and the second sorting unit are realized by the processing unit <NUM> and software or the like that controls operation of the processing unit <NUM>. The algorithm of the one class learning and the algorithm of the object detection can also be introduced by using known software.

In the second sorting step S3, the features of the defects <NUM> are preferably classified and accumulated for each of the types thereof (for example, the above-described poor rubber flow, mixing of foreign matter, etc.). In the present embodiment, the features of the defects <NUM> are classified by the processing unit <NUM> (second sorting unit) and accumulated in the defect folder <NUM> of the storage unit <NUM> for each type. By comparing the image data acquired in the imaging step S1 using the features of the defects <NUM> classified by type, the accuracy of detecting a defective product is further improved.

In the second sorting step S3, not only the defective product estimation images <NUM> but also the non-defective product estimation images <NUM> are preferably compared with the features of the defects <NUM>. In this case, the tires <NUM> determined to be non-defective products in the first sorting step S2 are also searched for the defects <NUM> in the second sorting step S3, so that the accuracy of detecting a defective product is improved. In addition, the tire <NUM> determined to be a defective product in the first sorting step S2 and the second sorting step S3 may be handled as a defective product, and the tire <NUM> determined to be a defective product in the first sorting step S2 or the second sorting step S3 may be handled as a defective product.

In these cases, the features of the defects <NUM> extracted from the images of the tires <NUM> that are detected as defective products in the second sorting step S3 after being sorted as non-defective product estimation images in the first sorting step S2 may be accumulated in the defect folder <NUM>. With such a configuration, an even wider variety of features of defects <NUM> are accumulated in the defect folder <NUM>, so that the accuracy of detecting a defective product is further improved.

Meanwhile, the images of the tires <NUM> determined to be non-defective products in the second sorting step S3 may be added to images of non-defective tires in the one class learning of the first sorting step S2. With such a configuration, the accuracy of sorting images in the first sorting step S2 is further improved.

<FIG> shows an imaging area <NUM> of the tire <NUM> imaged by the camera <NUM>. In <FIG>, the imaging area <NUM> is drawn by hatching with a dot pattern. In the present embodiment, in the imaging step S1, the annular imaging area <NUM> at the sidewall portion <NUM> of the tire <NUM> is imaged. Accordingly, the presence/absence of an appearance defect at the sidewall portion <NUM> of the tire <NUM> is easily inspected. In this case, in the imaging step S1, the annular imaging area <NUM> is preferably converted into a band shape.

<FIG> shows an image of the sidewall portion <NUM> converted into a band shape. The conversion of the image is executed by the processing unit <NUM>, and the converted image is stored in the storage unit <NUM>. Since the annular image is converted into a band shape in the imaging step S1, it is easy to scroll the image on the display device, and an operator can easily check an appearance defect.

In the imaging step S1, a uniformity peak mark, a light dot mark, etc., that are put on the sidewall portion <NUM> are preferably subjected to a masking process. In the present embodiment, in the case where the to-be-inspected object <NUM> is the tire <NUM>, areas corresponding to the peak mark, the light dot mark, and the like in an image captured by the camera <NUM> are masked by painting in the same color as the tire <NUM> itself, for example, in black. These marks are portions peculiar in terms of color but not defects of the tire <NUM>, and thus it is not desirable to detect the marks as abnormalities. Therefore, in the present embodiment, by performing the masking process, the presence of the marks in the image is ignored in the first sorting step S2 and the second sorting step S3, so that false detection of a defective product is inhibited.

Meanwhile, in the present embodiment, the position of the camera <NUM> may be changed such that a tread portion <NUM> (see <FIG>) of the tire <NUM> is imaged. With such a configuration, it is also possible to inspect the presence/absence of an appearance defect at the tread portion <NUM>.

A 2D camera, a 3D camera, or a 2D camera and a 3D camera are used as the camera <NUM>. With a 2D camera, in the first sorting step S2 and the second sorting step S3, the defect <NUM> is detected mainly in terms of abnormality relating to color. Meanwhile, with a 3D camera, in the first sorting step S2 and the second sorting step S3, the defect <NUM> is detected mainly in terms of abnormality relating to shape. By using both cameras in combination, the defect <NUM> is detected in terms of abnormality relating to color and shape.

Claim 1:
An appearance inspection method for inspecting appearance of a to-be-inspected object (<NUM>), the appearance inspection method being executed using an appearance inspection apparatus (<NUM>) and comprising:
an imaging step (S1) of imaging a plurality of the to-be-inspected objects (<NUM>) using a camera (<NUM>) of an imaging unit (<NUM>) and acquiring images thereof;
a first sorting step (S2) of sorting the images as non-defective product estimation images (<NUM>) within a predetermined standard and defective product estimation images (<NUM>) outside the standard by using an algorithm of one class learning, the first sorting step (S2) being executed using a processing unit (<NUM>) of a computer device (<NUM>); and
a second sorting step (S3) of detecting a defective product from the to-be-inspected objects (<NUM>) by extracting features of defects (<NUM>) from the defective product estimation images (<NUM>) using an algorithm of object detection, and searching the images for the features by comparing the image data acquired in the imaging step (S1) with the features of the defects (<NUM>) from the defective product estimation images (<NUM>) sorted in the first sorting step (S2), the second sorting step (S3) being executed using the processing unit (<NUM>).