Patent Description:
Conventionally, an inspection system has been known, in which an object to be inspected, such as a printed circuit board, is captured in an image to be compared with a reference image, so as to inspect whether the object to be inspected is a good product or a defective product.

In the inspection system, for example, in order to avoid taking a defective product as a good product, candidate defects are detected in excess, and the images of the detected candidate defects are finally visually inspected by an inspector, to realize prevention of missing defects. <CIT> discloses a method, a device and a computer program product for image processing. In the method, whether a first image indicates a defect associated with a target object is determined. In response to determining that the first image indicates the defect, a second image absent from the defect is obtained based on the first image. The defect is identified by comparing the first image with the second image. In this way. the defect associated with the target object in the image can be accurately and efficiently identified or segmented.

However, in the case of the inspection system described above, manufacturing variations and the like within a range of good products are also detected as candidate defects; therefore, there have been problems such that there are many images of candidate defects that need to be visually inspected by the inspector, and hence, the workload of the inspector is high.

According to one aspect, it is an object to reduce the workload of an inspector in an inspection system.

According to one embodiment, an inspection system includes a memory; and a processor, wherein the memory is configured to hold an image reproducing model trained to reproduce, from a first masked image generated by masking part of a first image determined as including no defect from among images that capture an object to be inspected, the first image before being masked, and wherein the processor is configured to determine, based on a reproduced image reproduced by inputting a second masked image generated by masking part of a second image that captures a new object to be inspected into the image reproducing model, and the second image, whether the second image includes a defect, wherein the image reproducing model is trained so as to reproduce the first image before being masked from a superimposed image obtained by having CAD data corresponding to the masked part of the first image superimposed on the generated first masked image, and wherein the processor determines whether the second image includes a defect, based on a reproduced image reproduced by inputting a superimposed image obtained by having CAD data corresponding to the masked part of the second image superimposed on the generated second masked image, and the second image.

The workload of an inspector in an inspection system can be reduced.

In the following, embodiments will be described with reference to the accompanying drawings. Note that throughout the description and the drawings, for elements having substantially the same functional configurations, the same reference signs are assigned, and duplicate descriptions are omitted.

First, a system configuration of an inspection system in a learning phase according to a first embodiment not covered by the claimed invention will be described. <FIG> is a diagram illustrating an example of a system configuration of an inspection system in a learning phase.

As illustrated in <FIG>, an inspection system <NUM> in a learning phase includes an automated optical inspection (AOI) device <NUM> and a learning device <NUM>.

The AOI device <NUM> executes automated appearance inspection of a printed circuit board <NUM>. The AOI device <NUM> scans the printed circuit board <NUM> with a camera, and inspects various inspection items, to detect candidate defects. The inspection items inspected by the AOI device <NUM> include, for example, circuit widths, circuit spacing, missing pads/no pads, short circuits, and the like.

An image <NUM> of each region that includes a candidate defect detected by the AOI device <NUM> is transmitted to the learning device <NUM>, and also transmitted to an inspection line. On the inspection line, an inspector <NUM> as one of the inspectors visually inspects the image <NUM> of each region that includes a candidate defect. Note that the AOI device <NUM> is assumed to be configured to in excess detect images of the respective regions that include candidate defects so as not to take a defective product as a good product.

The inspector <NUM> visually inspects the images <NUM> of the respective regions to determine whether any defect is included, and to finally determine whether the printed circuit board <NUM> is a good product or a defective product. Specifically, in the case where none of the images <NUM> of the respective regions that include the candidate defects includes a defect, the printed circuit board <NUM> is determined as a good product. Alternatively, in the case where any one of the images <NUM> of the respective regions that include the candidate defects includes a defect, the printed circuit board <NUM> is determined as a defective product.

Note that the inspector <NUM> informs the learning device <NUM> of results of visual inspection (results of determining whether the images <NUM> of the respective regions include defects). In the example in <FIG>, "visual inspection result: OK" means that it is determined that a defect is not included in the images of the regions that include the candidate defects, whereas "visual inspection result: NG" means that it is determined that a defect is included in the images of the regions that include the candidate defects.

A learning program is installed in the learning device <NUM>, and by executing the program, the learning device <NUM> functions as a training dataset generating unit <NUM> and a learning unit <NUM>.

The training dataset generating unit <NUM> extracts images that are determined as not including defects as the results of visual inspection performed by the inspector <NUM>, from among the images <NUM> of the respective regions that include the candidate defects transmitted from the AOI device <NUM>. Also, the training dataset generating unit <NUM> associates the extracted image of each region with a corresponding result of the visual inspection, and stores the associated data in the training dataset storage unit <NUM> as a training dataset.

The learning unit <NUM> reads the image of each region included in the training dataset stored in the training dataset storage unit <NUM>. Also, the learning unit <NUM> generates a masked image by masking part of the image of the region, and from the generated masked images, executes a learning process for a model so as to reproduce the image before being masked.

Note that as the model for which the learning process is executed by the learning unit <NUM>, for example, a model for complementing a blanked-out region of an image is used. In the model, in the case of receiving as input an image having part of it blanked out, the blanked-out region is complemented based on a surrounding region of the blanked-out region, to reproduce the original image before it is blanked out. In the following, such a model will be referred to as an "image reproducing unit" in the present description, as an example of "image reproducing model" in the claims. Note that the method implementing the learning process includes, for example, decision trees, support vector machines, logistic regression, linear regression, neural networks, deep learning, and the like, though not limited as such.

Next, a hardware configuration of the learning device <NUM> will be described. <FIG> is a diagram illustrating an example of a hardware configuration of the learning device <NUM>. As illustrated in <FIG>, the learning device <NUM> includes a processor <NUM>, a memory <NUM>, an auxiliary storage device <NUM>, an interface (I/F) device <NUM>, a communication device <NUM>, and a drive device <NUM>. Note that the respective hardware units of the learning device <NUM> are interconnected via a bus <NUM>.

The processor <NUM> includes various arithmetic/logic devices such as a central processing unit (CPU) and a graphics processing unit (GPU). The processor <NUM> reads various programs (e.g., a learning program, etc.) to be loaded onto the memory <NUM>, and executes the programs.

The memory <NUM> includes main memory devices such as a read-only memory (ROM), a random access memory (RAM), and the like. The processor <NUM> and the memory <NUM> constitute what-is-called a computer, and by causing the processor <NUM> to execute the various programs loaded on the memory <NUM>, the computer implements, for example, the functions described above (as implemented in the training dataset generating unit <NUM> and the learning unit <NUM>).

The auxiliary storage device <NUM> stores the various programs and various items of data used when the various programs are executed by the processor <NUM>. For example, the training dataset storage unit <NUM> is implemented in the auxiliary storage device <NUM>.

The I/F device <NUM> is a connection device that connects an operation device <NUM> as an example of an external device and the display device <NUM> with the learning device <NUM>. The I/F device <NUM> receives an operation performed on the learning device <NUM> via the operation device <NUM> (e.g., an operation performed by the inspector <NUM> to enter a result of visual inspection; an operation performed by an administrator (not illustrated) of the learning device <NUM> to enter a command for the learning process; or the like). Also, the I/F device <NUM> outputs results of the learning process and the like executed by the learning device <NUM>, and displays the results for the administrator of the learning device <NUM> via the display device <NUM>.

The communication device <NUM> is a communication device for communicating with another device (in the present embodiment, the AOI device <NUM>).

The drive device <NUM> is a device for setting a recording medium <NUM>. The recording medium <NUM> here includes a medium to record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, or the like. Further, the recording medium <NUM> may include a semiconductor memory or the like to record information electrically, such as a ROM, a flash memory, or the like.

Note that the various programs installed in the auxiliary storage device <NUM> are installed by, for example, setting a distributed recording medium <NUM> in the drive device <NUM>, and reading the various programs recorded on the recording medium <NUM> by the drive device <NUM>. Alternatively, the various programs installed in the auxiliary storage device <NUM> may be installed by downloading from a network via the communication device <NUM>.

Next, the respective units of the learning device <NUM> (here, the training dataset generating unit <NUM> and the learning unit <NUM>) will be described in detail.

<FIG> is a diagram illustrating a specific example of processing executed by the training dataset generating unit <NUM> of the learning device <NUM>. As illustrated in <FIG>, for example, when images <NUM> to <NUM> of respective regions that include candidate defects are transmitted from the AOI device <NUM>, the training dataset generating unit <NUM> extracts images of "visual inspection result: OK".

The example in <FIG> illustrates that from among the images <NUM> to <NUM> of the respective regions, the images <NUM>, <NUM>, <NUM>, and <NUM> are images of "visual inspection result: NG". Therefore, the training dataset generating unit <NUM> extracts the images <NUM> and <NUM> of corresponding regions (images of "visual inspection result: OK"), to generate a training dataset <NUM>.

As illustrated in <FIG>, the training dataset <NUM> includes, as fields of information, "ID", "image", and "visual inspection result".

The field of "ID" stores an identifier to identify an image of each region. The field of "image" stores the image of the region. The field of "visual inspection result" stores a result of visual inspection with respect to the image of the region. Note that only images of "visual inspection result: OK" are stored in the training dataset <NUM>; therefore, only "OK" is stored in the field of "visual inspection result".

In this way, the training dataset generating unit <NUM> generates the training dataset <NUM> by using.

<FIG> is a first diagram illustrating a specific example of processing executed by the learning unit <NUM> of the learning device <NUM>. As illustrated in <FIG>, the learning unit <NUM> includes a mask unit <NUM>, an image reproducing unit <NUM>, and a comparison/change unit <NUM>.

The mask unit <NUM> reads the image of each region (an example of a first image, e.g., an image <NUM>) stored in the field of "image" in the training dataset <NUM> stored in the training dataset storage unit <NUM>. Also, the mask unit <NUM> masks part of the read image <NUM>, to generate masked images (examples of a first masked image, e.g., masked images 440_1 to 440_n). Also, the mask unit <NUM> inputs the generated masked images 440_1 to 440_n into the image reproducing unit <NUM>. Note that when the mask unit <NUM> masks the read image <NUM>, the positions to be masked are selected randomly; for example, the mask unit <NUM> executes masking at around <NUM> to <NUM> positions.

Based on the masked images 440_1 to 440_n, the image reproducing unit <NUM> reproduces images before being masked 441_1 to 441_n, and outputs these images to the comparison/change unit <NUM>.

The comparison/change unit <NUM> compares each of the images 441_1 to 441_n reproduced by the image reproducing unit <NUM>, with the image before being masked <NUM> read by the mask unit <NUM>, and updates model parameters of the image reproducing unit <NUM> so as to make both images consistent with each other.

In this way, for the image reproducing unit <NUM>, a learning process is executed so as to reproduce the image before being masked <NUM> from the masked images 440_1 to 440_n generated by the mask unit <NUM>. Thus, by adopting a configuration that uses the image reproducing unit <NUM>, according to the learning device <NUM>, an unsupervised learning process can be executed. Also, the learning process can be executed using only images of the respective regions that do not include defects (images of "visual inspection result: OK"). Therefore, compared to the case of collecting images of the respective regions that include various types of defects (images of "visual inspection result: NG") and executing a learning process, the cost of learning can be reduced.

Note that the trained image reproducing unit for which the learning process has been executed to reproduce an image before being masked is used in an inspection phase that will be described later.

Next, a system configuration of an inspection system in an inspection phase according to the first embodiment will be described. <FIG> is a diagram illustrating an example of a system configuration of an inspection system in an inspection phase.

As illustrated in <FIG>, an inspection system <NUM> in the inspection phase includes an AOI device <NUM> and an inference device <NUM>.

Among the elements, the AOI device <NUM> is the same as the AOI device <NUM> of the inspection system <NUM> in the learning phase; therefore, the description is omitted here.

An inspection program is installed in the inference device <NUM>, and by executing the program, the inference device <NUM> functions as an inference unit <NUM> and an output unit <NUM>.

The inference unit <NUM> includes a trained image reproducing unit that has been generated in the learning phase. The inference unit <NUM> obtains images <NUM> of the respective regions of a printed circuit board <NUM> as an object to be inspected from the AOI device <NUM>, transmitted when automated appearance inspection is executed with respect to the printed circuit board <NUM>. Also, the inference unit <NUM> masks part of the obtained image <NUM> of each region, to generate masked images, and inputs these images into the trained image reproducing unit. Also, the inference unit <NUM> compares each image reproduced by the trained image reproducing unit, with an image before being masked <NUM>, to determine whether the image before being masked <NUM> of each region includes a defect. Further, the inference unit <NUM> informs the output unit <NUM> of the determination results.

The output unit <NUM> outputs the determination results informed from the inference unit <NUM> to the inspection line. On the inspection line, the inspector <NUM> visually inspects the image of each region that includes a candidate defect. However, in the inspection phase, on the inspection line, with reference to the determination results output by the output unit <NUM>, images determined as including no defect by the inference device <NUM> are excluded, from among the images <NUM> of the respective regions that include candidate defects. Then, on the inspection line, from among the images <NUM> of the respective regions that include candidate defects, images <NUM> determined as including no defect by the inference device <NUM> are assigned to visual inspection. In other words, the output unit <NUM> outputs each image <NUM> determined as including a defect, so as to have the image <NUM> undergo the visual inspection.

In this way, the automated appearance inspection is executed by the AOI device <NUM> for the printed circuit board <NUM>, and in the case where an image of a region that includes a candidate defect is detected, on the inspection line, the image determined as including a defect by the inference device <NUM> is assigned to visual inspection. As a result, according to the inspection system <NUM>, the number of images assigned to the visual inspection can be reduced, and thereby, the workload of the visual inspection by the inspector <NUM> can be reduced.

Next, a hardware configuration of the inference device <NUM> will be described. <FIG> is a diagram illustrating an example of a hardware configuration of the inference device <NUM>. Note that as illustrated in <FIG>, the hardware configuration of the inference device <NUM> is substantially the same as the hardware configuration of the learning device <NUM>; therefore, here, differences from the hardware configuration of the learning device <NUM> will be mainly described.

The processor <NUM> reads various programs (e.g., an inspection program, etc.) to be loaded onto the memory <NUM>, and executes the programs. By causing the processor <NUM> to execute various programs loaded on the memory <NUM>, the computer constituted with the processor <NUM> and the memory <NUM> implements, for example, the functions described above (as implemented in the inference unit <NUM> and the output unit <NUM>).

Next, the respective units of the inference device <NUM> (here, the inference unit <NUM>) will be described in detail. <FIG> is a first diagram illustrating a specific example of processing executed by the inference unit <NUM> of the inference device <NUM>. As illustrated in <FIG>, the inference unit <NUM> includes a mask unit <NUM>, a trained image reproducing unit <NUM>, and a determination unit <NUM>.

The mask unit <NUM> obtains the image of each region (an example of a second image, e.g., an image <NUM>) that includes a candidate defect transmitted from the AOI device <NUM>, and masks part of the obtained image, to generate masked images (examples of second masked images, e.g., the masked images 741_1 to 741_n). Also, the mask unit <NUM> inputs the generated masked images 741_1 to 741_n into the trained image reproducing unit <NUM>.

The trained image reproducing unit <NUM> is a trained model that has been generated by executing a learning process for the image reproducing unit <NUM> during the learning phase. Based on the masked images 440_1 to 440_n, the trained image reproducing unit <NUM> reproduces images before being masked (examples of a reproduced image of a second image, e.g., masked images 751_1 to 751_n).

The determination unit <NUM> compares each of the images 751_1 to 751_n reproduced by the trained image reproducing unit <NUM> with the image before being masked <NUM> obtained by the mask unit <NUM>, to determine whether a defect is included.

Specifically, first, the determination unit <NUM> calculates the mean squared error (MSE) of pixel values of the reproduced image 751_1 and the image before being masked <NUM>. Next, the determination unit <NUM> calculates the mean squared error (MSE) of pixel values of the reproduced image 751_2 and the image before being masked <NUM>. Thereafter, similarly, the determination unit <NUM> calculates the mean squared error (MSE) of pixel values of the reproduced image 751_3 and the image before being masked <NUM>, and so on, up to the mean squared error (MSE) of pixel values of the reproduced image 751_n and the image before being masked <NUM>.

Next, the determination unit <NUM> determines whether each of the calculated MSEs is less than or equal to a predetermined threshold value (Th). In the case where it is determined that every calculated MSE is less than or equal to the predetermined threshold value, the image <NUM> is determined as not having a defect ("visual inspection result: OK"). On the other hand, in the case where it is determined that any one of the calculated MSEs exceeds the predetermined threshold value, the image <NUM> is determined as including a defect ("visual inspection result: NG").

In this way, the inference unit <NUM> compares a reproduced image with the image before being masked, to determine whether a defect is included. Thus, for example, compared to the case where an image before being masked is compared with a reference image to determine whether a defect is included, it becomes possible to output a determination result corresponding to manufacturing variations within a range of good products.

In other words, according to the inspection system <NUM>, for images of the respective regions including candidate defects that are detected so as to implement prevention of missing defects, images that are expected to be "visual inspection result: OK" (images of "visual inspection result: OK") can be excluded appropriately from assignment to visual inspection. As a result, according to the inspection system <NUM>, the number of images assigned to the visual inspection can be reduced, and thereby, the workload of the inspector can be reduced.

Next, a flow of the learning process in the inspection system <NUM> will be described. <FIG> is a first flow chart illustrating a flow of the learning process in the inspection system <NUM>.

At Step S801, the training dataset generating unit <NUM> of the learning device <NUM> obtains images of the respective regions that include candidate defects from the AOI device <NUM>.

At Step S802, the training dataset generating unit <NUM> of the learning device <NUM> extracts images of "visual inspection result: OK" from among the obtained images of the respective regions, to generate a training dataset.

At Step S803, the learning unit <NUM> of the learning device <NUM> generates a masked image by masking part of the image of each region included in the training dataset, and from the generated masked images, executes a learning process for the image reproducing unit so as to reproduce the image before being masked.

At Step S804, the learning unit <NUM> of the learning device <NUM> determines whether to end the learning process. If it is determined at Step S804 to continue the learning process (if NO is determined at Step S804), the process returns to Step S801.

On the other hand, if it is determined at Step S804 to end the learning process (if YES is determined at Step S804), the process proceeds to Step S805.

At Step S805, the learning unit <NUM> of the learning device <NUM> outputs the trained image reproducing unit, and ends the learning process.

Next, a flow of the inspection process in the inspection system <NUM> will be described. <FIG> is a first flow chart illustrating a flow of the inspection process in the inspection system <NUM>.

At Step S901, the inference unit <NUM> of the inference device <NUM> obtains images of the respective regions that include candidate defects from the AOI device <NUM>.

At Step S902, the inference unit <NUM> of the inference device <NUM> masks part of the obtained image of each region, to generate a masked image, and inputs the generated masked images into the trained image reproducing unit, to reproduce the image before being masked.

At Step S903, the inference unit <NUM> of the inference device <NUM> compares the reproduced image with the image before being masked, and calculates the MSE, to determine whether the image before being masked includes a defect.

Specifically, the inference unit <NUM> of the inference device <NUM> partitions the obtained image into, for example, nine regions of three partitions in the vertical direction times three partitions in the horizontal direction. Next, the inference unit <NUM> of the inference device <NUM> executes the process of calculating the MSE for the masked image generated by masking one of the partitioned regions, for nine times while changing the partitioned region to be masked in order. Then, if the MSE for any one of the masked images exceeds the threshold value, the inference unit <NUM> of the inference device <NUM> determines that the obtained image includes a defect ("visual inspection result: NG").

At Step S904, the output unit <NUM> of the inference device <NUM> informs the determination result. In this way, the images of the respective regions that include candidate defects detected by the AOI device <NUM> upon executing automated appearance inspection, are classified into images that are determined as including no defect (images of "visual inspection result: OK") and images that are determined as including a defect (images of "visual inspection result: NG"). As a result, it becomes possible to assign only images determined as including a defect (images of "visual inspection result: NG") to visual inspection.

At Step S905, the inference unit <NUM> of the inference device <NUM> determines whether to end the inspection process. If it is determined at Step S905 to continue the inspection process (if NO is determined at Step S905), the process returns to Step S901.

On the other hand, if it is determined at Step S905 to end the inspection process (if YES is determined at Step S905), the inspection process ends.

As clarified from the above description, the inspection system according to the first embodiment,.

In this way, by comparing a reproduced image with the image before being masked to determine whether a defect is included, it becomes possible to output a determination result corresponding to manufacturing variations within a range of good products.

In other words, according to the first embodiment, for images of the respective regions including candidate defects that are detected so as to implement prevention of missing defects, images that are expected to be "visual inspection result: OK" (images of "visual inspection result: OK") can be excluded appropriately from assignment to visual inspection. As a result, according to the first embodiment, the number of images assigned to the visual inspection can be reduced, and thereby, the workload of the inspector can be reduced.

In the description of the first embodiment above, it is assumed that a masked image generated by masking part of the image of each region is input into the image reproducing unit. In contrast, in the second embodiment, a case will be described in which CAD (Computer-aided design) data is superimposed on a masked image generated by masking part of the image of each region, and then, input into the image reproducing unit. In the following, the second embodiment will be described focusing on differences from the first embodiment described above.

First, from among the respective units of a learning device <NUM> according to the second embodiment, a learning unit <NUM> will be described in terms of a specific example of processing. <FIG> is a second diagram illustrating a specific example of processing executed by the learning unit <NUM> of the learning device <NUM>.

As illustrated in <FIG>, in the second embodiment, it is assumed that when the learning unit <NUM> executes processing, a training dataset <NUM> is stored in a training dataset storage unit <NUM>.

The training dataset <NUM> includes "CAD data" as a field of information in addition to the fields of information in the training dataset <NUM>. The field of "CAD data" stores CAD data of each region corresponding to an image stored in the field of "image". The example in <FIG> illustrates a state in which CAD data <NUM> of a region corresponding to an image <NUM> is stored.

Also, as illustrated in <FIG>, the learning unit <NUM> includes a mask unit <NUM>, a superimposing unit <NUM>, an image reproducing unit <NUM>, and a comparison/change unit <NUM>.

The mask unit <NUM> is substantially the same as the mask unit <NUM> described in <FIG>, and the example in <FIG> illustrates a state in which a masked image <NUM> is generated by reading the image <NUM> stored in the field of "image" of the training dataset <NUM>, and masking part of it.

The superimposing unit <NUM> superimposes a corresponding region of the CAD data on the masked region of the masked image that is generated by the mask unit <NUM>, to generate a superimposed image. The example in <FIG> illustrates a state in which a superimposed image <NUM> is generated by superimposing the corresponding region of the CAD data <NUM> on the masked region of the masked image <NUM>.

Based on the superimposed image generated by the superimposing unit <NUM>, the image reproducing unit <NUM> reproduces an image before being masked, and outputs the image to the comparison/change unit <NUM>. The example in <FIG> illustrates a state in which an image before being masked <NUM> is reproduced based on the superimposed image <NUM> generated by the superimposing unit <NUM>, and output to the comparison/change unit <NUM>.

The comparison/change unit <NUM> compares the image <NUM> reproduced by the image reproducing unit <NUM>, with the image before being masked <NUM> read by the mask unit <NUM>, and updates model parameters of the image reproducing unit <NUM> so as to make both images consistent with each other.

In this way, for the image reproducing unit <NUM>, a learning process is executed so as to reproduce the image before being masked <NUM> from the superimposed image <NUM> generated by the superimposing unit <NUM>. Note that the trained image reproducing unit for which the learning process has been executed to reproduce an image before being masked is used in an inspection phase that will be described later.

Next, a specific example of processing executed by the inference unit <NUM> of the inference device <NUM> will be described. <FIG> is a second diagram illustrating a specific example of processing executed by the inference unit <NUM> of the inference device <NUM>. As illustrated in <FIG>, the inference unit <NUM> includes a mask unit <NUM>, a superimposing unit <NUM>, a trained image reproducing unit <NUM>, and a determination unit <NUM>.

The mask unit <NUM> is substantially the same as the mask unit <NUM> described in <FIG>, and the example in <FIG> illustrates a state in which a masked image <NUM> is generated by masking part of an image <NUM> transmitted from the AOI device <NUM>. Note that the image <NUM> is an image that includes a defect such as a short circuit.

The trained image reproducing unit <NUM> is a trained model that has been generated by executing a learning process for the image reproducing unit <NUM> during the learning phase. Based on the superimposed image informed by the superimposing unit <NUM>, the trained image reproducing unit <NUM> reproduces an image before being masked. The example in <FIG> illustrates a state in which an image before being masked <NUM> is reproduced, based on the superimposed image <NUM> informed by the superimposing unit <NUM>.

The determination unit <NUM> compares the image <NUM> reproduced by the trained image reproducing unit <NUM> with the image before being masked <NUM> obtained by the mask unit <NUM>, to determine whether a defect is included. The example in <FIG> illustrates a state in which as a result of comparing the image <NUM> reproduced by the trained image reproducing unit <NUM> with the image before being masked <NUM> obtained by the mask unit <NUM>, the MSE exceeds a predetermined threshold value, and thereby, the image is determined as including a defect ("visual inspection result: NG").

In this way, in the case of the image <NUM> that includes a defect such as a short circuit, only with the masked image described in the first embodiment, it is difficult to operate the device to reproduce an image of "visual inspection result: OK". This is because in the case of a defect such as a short circuit, the surroundings of the masked region do not include information necessary to complement the masked region.

In contrast, as described above, by superimposing a corresponding region of the CAD data on the masked region, it becomes possible to operate the device to reproduce an image of "visual inspection result: OK". As a result, the error between the reproduced image and the image before being masked becomes greater, and thereby, it becomes possible to determine that a defect such as a short circuit is included more precisely.

Next, a flow of the learning process in the inspection system <NUM> according to the second embodiment will be described. <FIG> is a second flow chart illustrating a flow of the learning process in the inspection system <NUM>. Differences from the first flow chart illustrated in <FIG> are Steps S1201 to S1203.

At Step S1201, the training dataset generating unit <NUM> of the learning device <NUM> obtains CAD data corresponding to images of the respective regions obtained at Step S801.

At Step S1202, the training dataset generating unit <NUM> of the learning device <NUM> extracts images of "visual inspection result: OK" from among the obtained images of the respective regions. Also, the training dataset generating unit <NUM> of the learning device <NUM> associates each of the extracted images of the respective regions with a corresponding region of the CAD data, to generate a training dataset.

At Step S1203, the learning unit <NUM> of the learning device <NUM> generates a masked image by masking part of the image of each region, and then, generates a superimposed image included in the training dataset by superimposing the corresponding region of the CAD data.

The following Steps S803 to S805 have been described with reference to <FIG>; therefore, the description is omitted here.

Next, a flow of the inspection process in the inspection system <NUM> according to the second embodiment will be described. <FIG> is a second flow chart illustrating a flow of the inspection process in the inspection system <NUM>. Differences from the first flow chart illustrated in <FIG> are Steps S1301 to S1302.

At Step S1301, the inference unit <NUM> of the inference device <NUM> obtains regions of CAD data corresponding to images of the respective regions obtained at Step S901.

At Step S1302, the inference unit <NUM> of the inference device <NUM> generates a masked image by masking part of the obtained image of each region, and then, superimposes the corresponding region of the CAD data, to generate a superimposed image.

The following Steps S902 to S905 have been described with reference to <FIG>; therefore, the description is omitted here.

As clarified from the description, the inspection system for the second embodiment,.

In this way, by adopting a configuration in which a learning process is executed so as to be capable of reproducing an image of "visual inspection result: OK" from a superimposed image, it becomes possible to determine a defect including a short circuit or the like more precisely. As a result, according to the second embodiment, while enjoying substantially the same effects as in the first embodiment described above, it becomes possible to avoid an erroneous determination.

In the description of the first embodiment above, in the learning phase, it is assumed that the mask unit <NUM> generates multiple masked images (e.g., the masked images 440_1 to 440_n) for one image (e.g., the image <NUM>). However, the masked images that are generated from one image are not limited to multiple images, for example, the mask unit <NUM> may be configured to generate only one masked image from one image.

Also, in the first embodiment described above, although a specific number of iterations and the like in the learning process executed by the learning unit <NUM> are not mentioned, for example, in the case of generating one masked image from one image,.

In this way, the inference unit <NUM> can output a highly precise determination result.

In the second embodiment described above, although it has been described assuming that CAD data is superimposed on a masked image, data to be superimposed on a masked image is not limited to CAD data. For example, in the inspection phase, the processing may be configured to have a corresponding region of an image of "visual inspection result: OK" superimposed on the masked image.

Also, in the first and second embodiments described above, although they have been described for the case in which an object to be inspected is a printed circuit board, the object to be inspected is not limited to a printed circuit board.

Also, in the first and second embodiments described above, they have been described assuming that an image assigned to visual inspection is determined based on the determination result. However, the method of using the determination result is not limited as such; for example, the method may be configured to display the determination result for the inspector <NUM>, so as to support the visual inspection.

Claim 1:
An inspection system (<NUM>) comprising:
a memory (<NUM>, <NUM>); and
a processor (<NUM>, <NUM>),
wherein the memory (<NUM>, <NUM>) is configured to hold an image reproducing model (<NUM>, <NUM>) trained to reproduce, from a first masked image (440_1 ... 440_n) generated by masking part of a first image (<NUM>) determined as including no defect from among images that capture an object to be inspected, the first image (441_1 ... 441_n) before being masked,
wherein the processor (<NUM>, <NUM>) is configured to determine, based on a reproduced image (751_1...751_n) reproduced by inputting a second masked (741_1...741_n) image generated by masking part of a second image (<NUM>) that captures a new object to be inspected into the image reproducing model, and the second image, whether the second image includes a defect,
wherein the image reproducing model is trained so as to reproduce the first image before being masked from a superimposed image obtained by having CAD data corresponding to the masked part of the first image superimposed on the generated first masked image, and
wherein the processor (<NUM>, <NUM>) determines whether the second image includes a defect, based on a reproduced image reproduced by inputting a superimposed image obtained by having CAD data corresponding to the masked part of the second image superimposed on the generated second masked image, and the second image.