Inspection system, inspection method, and inspection program

An inspection system includes a memory; and a processor, wherein the memory is configured to hold an image reproducing model trained to reproduce, from a first masked image generated by masking part, of a first image determined as including no defect from among images that capture an object to be inspected, the first image before being masked, and wherein the processor is configured to determine, based on a reproduced image reproduced by inputting a second masked image generated by masking part of a second image that captures a new object to be inspected into the image reproducing model, and the second image, whether the second image includes a defect.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is based upon and claims the benefit of priority under 35 USC 119 of Japanese Patent Application No. 2021-010592 filed on Jan. 26, 2021, the entire contents of which, including the description, claims, drawings, and abstract, are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an inspection system, an inspection method, and an inspection program.

BACKGROUND ART

Conventionally, an inspection system has been known, in which an object to be inspected, such as a printed circuit board, is captured in an image to be compared with a reference image, so as to inspect whether the object to be inspected is a good product or a defective product.

In the inspection system, for example, in order to avoid taking a defective product as a good product, candidate defects are detected in excess, and the images of the detected candidate defects are finally visually inspected by an inspector, to realize prevention of missing defects.

RELATED ART DOCUMENTS

Patent Documents

However, in the case of the inspection system described above, manufacturing variations and the like within a range of good products are also detected as candidate defects; therefore, there have been problems such that there are many images of candidate defects that need to be visually inspected by the inspector, and hence, the workload of the inspector is high.

SUMMARY

According to one embodiment, an inspection system includes a memory; and a processor, wherein the memory is configured to hold an image reproducing model trained to reproduce, from a first masked image generated by masking part of a first image determined as including no defect from among images that capture an object to be inspected, the first image before being masked, and wherein the processor is configured to determine, based on a reproduced image reproduced by inputting a second masked image generated by masking part of a second image that captures a new object to be inspected into the image reproducing model, and the second image, whether the second image includes a defect.

EMBODIMENTS OF THE INVENTION

In the following, embodiments will be described with reference to the accompanying drawings. Note that throughout the description and the drawings, for elements having substantially the same functional configurations, the same reference signs are assigned, and duplicate descriptions are omitted.

According to an embodiment in the present disclosure, the workload of an inspector in an inspection system can be reduced.

First Embodiment

System Configuration of Inspection System in Learning Phase

First, a system configuration of an inspection system in a learning phase according to a first embodiment will be described.FIG.1is a diagram illustrating an example of a system configuration of an inspection system in a learning phase.

As illustrated inFIG.1, an inspection system100in a learning phase includes an automated optical inspection (AOI) device110and a learning device120.

The AOI device110executes automated appearance inspection of a printed circuit board130. The AOI device110scans the printed circuit board130with a camera, and inspects various inspection items, to detect candidate defects. The inspection items inspected by the AOI device110include, for example, circuit widths, circuit spacing, missing pads/no pads, short circuits, and the like.

An image140of each region that includes a candidate defect detected by the AOI device110is transmitted to the learning device120, and also transmitted to an inspection line. On the inspection line, an inspector111as one of the inspectors visually inspects the image140of each region that includes a candidate defect. Note that the AOI device110is assumed to be configured to in excess detect images of the respective regions that include candidate defects so as not to take a defective product as a good product.

The inspector111visually inspects the images140of the respective regions to determine whether any defect is included, and to finally determine whether the printed circuit board130is a good product or a defective product. Specifically, in the case where none of the images140of the respective regions that include the candidate defects includes a defect, the printed circuit board130is determined as a good product. Alternatively, in the case where any one of the images140of the respective regions that include the candidate defects includes a defect, the printed circuit board130is determined as a defective product.

Note that the inspector111informs the learning device120of results of visual inspection (results of determining whether the images140of the respective regions include defects). In the example inFIG.1, “visual inspection result: OK” means that it is determined that a defect, is not included in the images of the regions that include the candidate defects, whereas “visual inspection result: NG” means that it is determined that a defect is included in the images of the regions that include the candidate defects.

A learning program is installed in the learning device120, and by executing the program, the learning device120functions as a training dataset generating unit121and a learning unit122.

The training dataset generating unit121extracts images that are determined as not including defects as the results of visual inspection performed by the inspector111, from among the images140of the respective regions that include the candidate defects transmitted from the AOI device110. Also, the training dataset generating unit121associates the extracted image of each region with a corresponding result of the visual inspection, and stores the associated data in the training dataset storage unit123as a training dataset.

The learning unit122reads the image of each region included in the training dataset stored in the training dataset storage unit123. Also, the learning unit122generates a masked image by masking part of the image of the region, and from the generated masked images, executes a learning process for a model so as to reproduce the image before being masked.

Note that as the model for which the learning process is executed by the learning unit122, for example, a model for complementing a blanked-out region of an image is used. In the model, in the case of receiving as input an image having part of it blanked out, the blanked-out region is complemented based on a surrounding region of the blanked-out region, to reproduce the original image before it is blanked out. In the following, such a model will be referred to as an “image reproducing unit” in the present description, as an example of “image reproducing model” in the claims. Note that the method implementing the learning process includes, for example, decision trees, support vector machines, logistic regression, linear regression, neural networks, deep learning, and the like, though not limited as such.

Hardware Configuration of Learning Device

Next, a hardware configuration of the learning device120will be described.FIG.2is a diagram illustrating an example of a hardware configuration of the learning device120. As illustrated inFIG.2, the learning device120includes a processor201, a memory202, an auxiliary storage device203, an interface (I/F) device204, a communication device205, and a drive device206. Note that the respective hardware units of the learning device120are interconnected via a bus207.

The processor201includes various arithmetic/logic devices such as a central processing unit (CPU) and a graphics processing unit (GPU). The processor201reads various programs (e.g., a learning program, etc.) to be loaded onto the memory202, and executes the programs.

The memory202includes main memory devices such as a read-only memory (ROM), a random access memory (RAM), and the like. The processor201and the memory202constitute what-is-called a computer, and by causing the processor201to execute the various programs loaded on the memory202, the computer implements, for example, the functions described above (as implemented in the training dataset generating unit121and the learning unit122).

The auxiliary storage device203stores the various programs and various items of data used when the various programs are executed by the processor201. For example, the training dataset storage unit123is implemented in the auxiliary storage device203.

The I/F device204is a connection device that connects an operation device210as an example of an external device and the display device211with the learning device120. The I/F device204receives an operation performed on the learning device120via the operation device210(e.g., an operation performed by the inspector111to enter a result of visual inspection; an operation performed by an administrator (not illustrated) of the learning device120to enter a command for the learning process; or the like). Also, the I/F device204outputs results of the learning process and the like executed by the learning device120, and displays the results for the administrator of the learning device120via the display device211.

The communication device205is a communication device for communicating with another device (in the present embodiment, the AOI device110).

The drive device206is a device for setting a recording medium212. The recording medium212here includes a medium to record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, or the like. Further, the recording medium212may include a semiconductor memory or the like to record information electrically, such as a ROM, a flash memory, or the like.

Note that the various programs installed in the auxiliary storage device203are installed by, for example, setting a distributed recording medium212in the drive device206, and reading the various programs recorded on the recording medium212by the drive device206. Alternatively, the various programs installed in the auxiliary storage device203may be installed by downloading from a network via the communication device205.

Details of Units of Learning Device

Next, the respective units of the learning device120(here, the training dataset generating unit121and the learning unit122) will be described in detail.

(1) Specific Example of Processing Executed by the Training Dataset Generating Unit

FIG.3is a diagram illustrating a specific example of processing executed by the training dataset generating unit121of the learning device120. As illustrated inFIG.3, for example, when images301to306of respective regions that include candidate defects are transmitted from the AOI device110, the training dataset generating unit121extracts images of “visual inspection result: OK”.

The example inFIG.3illustrates that from among the images301to306of the respective regions, the images301,302,304, and305are images of “visual inspection result: NG”. Therefore, the training dataset generating unit121extracts the images303and306of corresponding regions (images of “visual inspection result: OK”), to generate a training dataset310.

As illustrated inFIG.3, the training dataset310includes, as fields of information, “ID”, “image”, and “visual inspection result”.

The field of “ID” stores an identifier to identify an image of each region. The field of “image” stores the image of the region. The field of “visual inspection result” stores a result of visual inspection with respect to the image of the region. Note that only images of “visual inspection result: OK” are stored in the training dataset310; therefore, only “OK” is stored in the field of “visual inspection result”.

In this way, the training dataset generating unit121generates the training dataset310by usingimages of the respective regions that include candidate defects detected by the AOI device110; andimages that are determined by the inspector111as including no defect (images of “visual inspection result: OK”).

(2) Specific Example of Processing Executed by the Learning Unit

FIG.4is a first diagram illustrating a specific example of processing executed by the learning unit122of the learning device120. As illustrated inFIG.4, the learning unit122includes a mask unit410, an image reproducing unit420, and a comparison/change unit430.

The mask unit410reads the image of each region (an example of a first image, e.g., an image440) stored in the field of “image” in the training dataset310stored in the training dataset storage unit123. Also, the mask unit410masks part of the read image440, to generate masked images (examples of a first masked image, e.g., masked images440_1to440_n). Also, the mask unit410inputs the generated masked images440_1to440_ninto the image reproducing unit420. Note that when the mask unit410masks the read image440, the positions to be masked are selected randomly; for example, the mask unit410executes masking at around 9 to 16 positions.

Based on the masked images440_1to440_n, the image reproducing unit420reproduces images before being masked441_1to441_n, and outputs these images to the comparison/change unit430.

The comparison/change unit430compares each of the images441_1to441_nreproduced by the image reproducing unit420, with the image before being masked440read by the mask unit410, and updates model parameters of the image reproducing unit420so as to make both images consistent with each other.

In this way, for the image reproducing unit420, a learning process is executed so as to reproduce the image before being masked440from the masked images440_1to440_ngenerated by the mask unit410. Thus, by adopting a configuration that uses the image reproducing unit420, according to the learning device120, an unsupervised learning process can foe executed. Also, the learning process can be executed using only images of the respective regions that do not include defects (images of “visual inspection result: OK”). Therefore, compared to the case of collecting images of the respective regions that include various types of defects (images of “visual inspection result: NG”) and executing a learning process, the cost of learning can be reduced.

Note that the trained image reproducing unit for which the learning process has been executed to reproduce an image before being masked is used in an inspection phase that will be described later.

System Configuration of Inspection System in Inspection Phase

Next, a system configuration of an inspection system in an inspection phase according to the first embodiment will be described.FIG.5is a diagram illustrating an example of a system configuration of an inspection system in an inspection phase.

As illustrated inFIG.5, an inspection system500in the inspection phase includes an AOI device110and an inference device510.

Among the elements, the AOI device110is the same as the AOI device110of the inspection system100in the learning phase; therefore, the description is omitted here.

An inspection program is installed in the inference device510, and by executing the program, the inference device510functions as an inference unit511and an output unit512.

The inference unit511includes a trained image reproducing unit that has been generated in the learning phase. The inference unit511obtains images540of the respective regions of a printed circuit board530as an object to be inspected from the AOI device110, transmitted when automated appearance inspection is executed with respect to the printed circuit board530. Also, the inference unit511masks part of the obtained image540of each region, to generate masked images, and inputs these images into the trained image reproducing unit. Also, the inference unit511compares each image reproduced by the trained image reproducing unit, with an image before being masked540, to determine whether the image before being masked540of each region includes a defect. Further, the inference unit511informs the output unit512of the determination results.

The output unit512outputs the determination results informed from the inference unit511to the inspection line. On the inspection line, the inspector111visually inspects the image of each region that includes a candidate defect. However, in the inspection phase, on the inspection line, with reference to the determination results output by the output unit512, images determined as including no defect by the inference device510are excluded, from among the images540of the respective regions that include candidate defects. Then, on the inspection line, from among the images540of the respective regions that include candidate defects, images550determined as including no defect by the inference device510are assigned to visual inspection. In other words, the output unit512outputs each image550determined as including a defect, so as to have the image550undergo the visual inspection.

In this way, the automated appearance inspection is executed by the AOI device110for the printed circuit board530, and in the case where an image of a region that includes a candidate defect is detected, on the inspection line, the image determined as including a defect by the inference device510is assigned to visual inspection. As a result, according to the inspection system500, the number of images assigned to the visual inspection can be reduced, and thereby, the workload of the visual inspection by the inspector111can be reduced.

Hardware Configuration of Inference Device

Next, a hardware configuration of the inference device510will be described.FIG.6is a diagram illustrating an example of a hardware configuration of the inference device510. Note that as illustrated inFIG.6, the hardware configuration of the inference device510is substantially the same as the hardware configuration of the learning device120; therefore, here, differences from the hardware configuration of the learning device120will be mainly described.

The processor601reads various programs (e.g., an inspection program, etc.) to be loaded onto the memory602, and executes the programs. By causing the processor601to execute various programs loaded on the memory602, the computer constituted with the processor601and the memory602implements, for example, the functions described above (as implemented in the inference unit511and the output unit512).

Details of Units of Inference Device

Next, the respective units of the inference device510(here, the inference unit511) will be described in detail.FIG.7is a first diagram illustrating a specific example of processing executed by the inference unit511of the inference device510. As illustrated inFIG.7, the inference unit511includes a mask unit710, a trained image reproducing unit720, and a determination unit730.

The mask unit710obtains the image of each region (an example of a second image, e.g., an image741) that includes a candidate defect transmitted from the AOI device110, and masks part of the obtained image, to generate masked images (examples of second masked images, e.g., the masked images741_1to741_n). Also, the mask unit710inputs the generated masked images741_1to741_ninto the trained image reproducing unit720.

The trained image reproducing unit720is a trained model that has been generated by executing a learning process for the image reproducing unit420during the learning phase. Based on the masked images440_1to440_n, the trained image reproducing unit720reproduces images before being masked (examples of a reproduced image of a second image, e.g., masked images751_1to751_n).

The determination unit730compares each of the images751_1to751_nreproduced by the trained image reproducing unit720with the image before being masked741obtained by the mask unit710, to determine whether a defect is included.

Specifically, first, the determination unit730calculates the mean squared error (MSE) of pixel values of the reproduced image751_1and the image before being masked741. Next, the determination unit730calculates the mean squared error (MSE) of pixel values of the reproduced image751_2and the image before being masked741. Thereafter, similarly, the determination unit730calculates the mean squared error (MSE) of pixel values of the reproduced image751_3and the image before being masked741, and so on, up to the mean squared error (MSE) of pixel values of the reproduced image751_nand the image before being masked741.

Next, the determination unit730determines whether each of the calculated MSEs is less than or equal to a predetermined threshold value (Th). In the case where it is determined that every calculated MSE is less than or equal to the predetermined threshold value, the image741is determined as not having a defect (“visual inspection result: OK”). On the other hand, in the case where it is determined that any one of the calculated MSEs exceeds the predetermined threshold value, the image741is determined as including a defect (“visual inspection result: NG”).

In this way, the inference unit511compares a reproduced image with the image before being masked, to determine whether a defect is included. Thus, for example, compared to the case where an image before being masked is compared with a reference image to determine whether a defect is included, it becomes possible to output a determination result corresponding to manufacturing variations within a range of good products.

In other words, according to the inspection system500, for images of the respective regions including candidate defects that are detected so as to implement prevention of missing defects, images that are expected to be “visual inspection result: OK” (images of “visual inspection result: OK”) can be excluded appropriately from assignment to visual inspection. As a result, according to the inspection system500, the number of images assigned to the visual inspection can be reduced, and thereby, the workload of the inspector can be reduced.

Flow of Learning Process

Next, a flow of the learning process in the inspection system100will be described.FIG.8is a first flow chart illustrating a flow of the learning process in the inspection system100.

At Step S801, the training dataset generating unit121of the learning device120obtains images of the respective regions that include candidate defects from the AOI device110.

At Step S802, the training dataset generating unit121of the learning device120extracts images of “visual inspection result: OK” from among the obtained images of the respective regions, to generate a training dataset.

At Step S803, the learning unit122of the learning device120generates a masked image by masking part of the image of each region included in the training dataset, and from the generated masked images, executes a learning process for the image reproducing unit so as to reproduce the image before being masked.

At Step S804, the learning unit122of the learning device120determines whether to end the learning process. If it is determined at Step S804to continue the learning process (if NO is determined at Step S804), the process returns to Step S801.

On the other hand, if it is determined at Step S804to end the learning process (if YES is determined at Step S804), the process proceeds to Step S805.

At Step S805, the learning unit122of the learning device120outputs the trained image reproducing unit, and ends the learning process.

Flow of Inspection Process

Next, a flow of the inspection process in the inspection system500will foe described.FIG.9is a first flow chart illustrating a flow of the inspection process in the inspection system500.

At Step S901, the inference unit511of the inference device510obtains images of the respective regions that include candidate defects from the AOI device110.

At Step S902, the inference unit511of the inference device510masks part of the obtained image of each region, to generate a masked image, and inputs the generated masked images into the trained image reproducing unit, to reproduce the image before being masked.

At Step S903, the inference unit511of the inference device510compares the reproduced image with the image before being masked, and calculates the MSE, to determine whether the image before being masked includes a defect.

Specifically, the inference unit511of the inference device510partitions the obtained image into, for example, nine regions of three partitions in the vertical direction times three partitions in the horizontal direction. Next, the inference unit511of the inference device510executes the process of calculating the MSE for the masked image generated by masking one of the partitioned regions, for nine times while changing the partitioned region to be masked in order. Then, if the MSE for any one of the masked images exceeds the threshold value, the inference unit511of the inference device510determines that the obtained image includes a defect (“visual inspection result: NG”).

At Step S904, the output unit512of the inference device510informs the determination result. In this way, the images of the respective regions that include candidate defects detected by the AOI device110upon executing automated appearance inspection, are classified into images that are determined as including no defect (images of “visual inspection result: OK”) and images that are determined as including a defect (images of “visual inspection result: NG”). As a result, it becomes possible to assign only images determined as including a defect (images of “visual inspection result: NG”) to visual inspection.

At Step S905, the inference unit511of the inference device510determines whether to end the inspection process. If it is determined at Step S905to continue the inspection process (if NO is determined at Step S905), the process returns to Step S901.

On the other hand, if it is determined at Step S905to end the inspection process (if YES is determined at Step S905), the inspection process ends.

Summary

As clarified from the above description, the inspection system according to the first embodiment,generates a masked image by masking part of an image that is determined as including no defect from among images that capture a printed circuit board, and then, executes a learning process for the image reproducing unit so as to reproduce the image before being masked from the generated masked image; andgenerates a masked image by masking part of an image that captures a new printed circuit board, and then, by inputting the generated masked image into the trained image reproducing unit to reproduce an image before being masked, and comparing the reproduced image with the image before being masked, determines whether the image before being masked includes a defect.

In this way, by comparing a reproduced image with the image before being masked to determine whether a defect is included, it becomes possible to output a determination result corresponding to manufacturing variations within a range of good products.

In other words, according to the first embodiment, for images of the respective regions including candidate defects that are detected so as to implement prevention of missing defects, images that are expected to be “visual inspection result: OK” (images of “visual inspection result: OK”) can be excluded appropriately from assignment to visual inspection. As a result, according to the first embodiment, the number of images assigned to the visual inspection can be reduced, and thereby, the workload of the inspector can be reduced.

Second Embodiment

In the description of the first embodiment above, it is assumed that a masked image generated by masking part of the image of each region is input into the image reproducing unit. In contrast, in the second embodiment, a case will be described in which CAD (Computer-aided design) data is superimposed on a masked image generated by masking part of the image of each region, and then, input into the image reproducing unit. In the following, the second embodiment will be described focusing on differences from the first embodiment described above.

Specific Example of Processing Executed by the Learning Unit

First, from among the respective units of a learning device120according to the second embodiment, a learning unit122will be described in terms of a specific example of processing.FIG.10is a second diagram illustrating a specific example of processing executed by the learning unit122of the learning device120.

As illustrated inFIG.10, in the second embodiment, it is assumed that when the learning unit122executes processing, a training dataset1010is stored in a training dataset storage unit123.

The training dataset1010includes “CAD data” as a field of information in addition to the fields of information in the training dataset310. The field of “CAD data” stores CAD data of each region corresponding to an image stored in the field of “image”. The example inFIG.10illustrates a state in which CAD data1011of a region corresponding to an image1012is stored.

Also, as illustrated inFIG.10, the learning unit122includes a mask unit410, a superimposing unit1020, an image reproducing unit1030, and a comparison/change unit1040.

The mask unit410is substantially the same as the mask unit410described inFIG.4, and the example inFIG.10illustrates a state in which a masked image1031is generated by reading the image1012stored in the field of “image” of the training dataset1010, and masking part of it.

The superimposing unit1020superimposes a corresponding region of the CAD data on the masked region of the masked image that is generated by the mask unit410, to generate a superimposed image. The example inFIG.10illustrates a state in which a superimposed image1032is generated by superimposing the corresponding region of the CAD data1011on the masked region of the masked image1031.

Based on the superimposed image generated by the superimposing unit1020, the image reproducing unit1030reproduces an image before being masked, and outputs the image to the comparison/change unit1040. The example inFIG.10illustrates a state in which an image before being masked1033is reproduced based on the superimposed image1032generated by the superimposing unit1020, and output to the comparison/change unit1040.

The comparison/change unit1040compares the image1033reproduced by the image reproducing unit1030, with the image before being masked1012read by the mask unit410, and updates model parameters of the image reproducing unit1030so as to make both images consistent with each other.

In this way, for the image reproducing unit1030, a learning process is executed so as to reproduce the image before being masked1012from the superimposed image1032generated by the superimposing unit1020. Note that the trained image reproducing unit for which the learning process has been executed to reproduce an image before being masked is used in an inspection phase that will be described later.

Specific Example of Processing of Units of the Inference Device

Next, a specific example of processing executed by the inference unit511of the inference device510will be described.FIG.11is a second diagram illustrating a specific example of processing executed by the inference unit511of the inference device510. As illustrated inFIG.11, the inference unit511includes a mask unit710, a superimposing unit1110, a trained image reproducing unit1120, and a determination unit1130.

The mask unit710is substantially the same as the mask unit710described inFIG.7, and the example inFIG.11illustrates a state in which a masked image1141is generated by masking part of an image1101transmitted from the AOI device110. Note that the image1101is an image that includes a defect such as a short circuit.

The superimposing unit1110superimposes a corresponding region of the CAD data on the masked region of the masked image that is generated by the mask unit710, to generate a superimposed image. The example inFIG.11illustrates a state in which a superimposed image1142is generated by superimposing the corresponding region of the CAD data1102on the masked region of the masked image1141.

The trained image reproducing unit1120is a trained model that has been generated by executing a learning process for the image reproducing unit1030during the learning phase. Based on the superimposed image informed by the superimposing unit1110, the trained image reproducing unit1120reproduces an image before being masked. The example inFIG.11illustrates a state in which an image before being masked1143is reproduced, based on the superimposed image1142informed by the superimposing unit1110.

The determination unit1130compares the image1143reproduced by the trained image reproducing unit1120with the image before being masked1101obtained by the mask unit710, to determine whether a defect is included. The example inFIG.11illustrates a state in which as a result of comparing the image1143reproduced by the trained image reproducing unit1120with the image before being masked1101obtained by the mask unit710, the MSE exceeds a predetermined threshold value, and thereby, the image is determined as including a defect (“visual inspection result: NG”).

In this way, in the case of the image1101that includes a defect such as a short circuit, only with the masked image described in the first embodiment, it is difficult to operate the device to reproduce an image of “visual inspection result: OK”. This is because in the case of a defect such as a short circuit, the surroundings of the masked region do not include information necessary to complement the masked region.

In contrast, as described above, by superimposing a corresponding region of the CAD data on the masked region, it becomes possible to operate the device to reproduce an image of “visual inspection result: OK”. As a result, the error between the reproduced image and the image before being masked becomes greater, and thereby, it becomes possible to determine that a defect such as a short circuit is included more precisely.

Flow of Learning Process

Next, a flow of the learning process in the inspection system100according to the second embodiment will be described.FIG.12is a second flow chart illustrating a flow of the learning process in the inspection system100. Differences from the first flow chart illustrated inFIG.8are Steps S1201to S1203.

At Step S1201, the training dataset generating unit121of the learning device120obtains CAD data corresponding to images of the respective regions obtained at Step S801.

At Step S1202, the training dataset generating unit121of the learning device120extracts images of “visual inspection result: OK” from among the obtained images of the respective regions. Also, the training dataset generating unit121of the learning device120associates each of the extracted images of the respective regions with a corresponding region of the CAD data, to generate a training dataset.

At Step S1203, the learning unit122of the learning device120generates a masked image by masking part of the image of each region, and then, generates a superimposed image included in the training dataset by superimposing the corresponding region of the CAD data.

The following Steps S803to S805have been described with reference toFIG.8; therefore, the description is omitted here.

Flow of Inspection Process

Next, a flow of the inspection process in the inspection system500according to the second embodiment will be described.FIG.13is a second flow chart illustrating a flow of the inspection process in the inspection system500. Differences from the first flow chart illustrated inFIG.9are Steps S1301to S1302.

At Step S1301, the inference unit511of the inference device510obtains regions of CAD data corresponding to images of the respective regions obtained at Step S901.

At Step S1302, the inference unit511of the inference device510generates a masked image by masking part of the obtained image of each region, and then, superimposes the corresponding region of the CAD data, to generate a superimposed image.

The following Steps S902to S905have been described with reference toFIG.9; therefore, the description is omitted here.

Summary

As clarified from the description, the inspection system for the second embodiment,generates a masked image by masking part of an image determined as including no defect from among images that capture a printed circuit board, and then, superimposes the corresponding region of CAD data, to generate a superimposed image, and then, executes a learning process for the image reproducing unit so as to reproduce an image before being masked from the generated superimposed image; andgenerates a masked image by masking part of an image that captures a new printed circuit board, and then, superimposes the corresponding region of the CAD data, to generate a superimposed image, and then, by inputting the generated superimposed image into the trained image reproducing unit to reproduce an image before being masked, and comparing the reproduced image with the image before being masked, determines whether the image before being masked includes a defect.

In this way, by adopting a configuration in which a learning process is executed so as to be capable of reproducing an image of “visual inspection result: OK” from a superimposed image, it becomes possible to determine a defect including a short circuit or the like more precisely. As a result, according to the second embodiment, while enjoying substantially the same effects as in the first embodiment described above, it becomes possible to avoid an erroneous determination.

Third Embodiment

In the description of the first embodiment above, in the learning phase, it is assumed that the mask unit410generates multiple masked images (e.g., the masked images440_1to440_n) for one image (e.g., the image440). However, the masked images that are generated from one image are not limited to multiple images, for example, the mask unit410may be configured to generate only one masked image from one image.

Also, in the first embodiment described above, although a specific number of iterations and the like in the learning process executed by the learning unit122are not mentioned, for example, in the case of generating one masked image from one image,10,000 images are prepared as a training dataset310,from among these 10,000 images, 20 images are selected each of which is masked at a random position with a random size,a process of updating the model parameters of the image reproducing unit420using these 20 masked images is executed for 500 times (=10,000/20), andone set of these steps is iterated for 10,000 sets until the error converges.

In this way, the inference unit511can output a highly precise determination result.

In the second embodiment described above, although it has been described assuming that CAD data is superimposed on a masked image, data to be superimposed on a masked image is not limited to CAD data. For example, in the inspection phase, the processing may be configured to have a corresponding region of an image of “visual inspection result: OK” superimposed on the masked image.

Also, in the first and second embodiments described above, although they have been described for the case in which an object to be inspected is a printed circuit board, the object to be inspected is not limited to a printed circuit board.

Also, in the first and second embodiments described above, they have been described assuming that an image assigned to visual inspection is determined based on the determination result. However, the method of using the determination result is not limited as such; for example, the method may be configured to display the determination result for the inspector111, so as to support the visual inspection.

Also, in the first and second embodiments described above, although the learning device120and the inference device510are configured as separate devices, the learning device120and the inference device510may be configured as a unified device.

Note that the present inventive concept is not limited to the configurations described herein including the configurations exemplified in the above embodiments; those combined with other elements; and the like. In these regards, it is possible to alter a configuration within a range not deviating from the gist of the present inventive concept, and the range can be appropriately determined according to the application form.