Stored image reclassification system and reclassification method

A component picked up by a suction nozzle of a component mounting machine is imaged by a camera, the captured image is processed by an image recognition system to recognize the component, the image is determined to be normal or abnormal based on the recognition result, and in addition to classifying the image as a normal image or an abnormal image and storing the image in a storage device, component mounting boards, unloaded from a component mounting machine, are inspected with an inspection device. A stored image reclassification computer acquires the inspection result from the inspection device, reclassifies the normal image stored in the storage device, based on the inspection result, as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect, and then stores the normal image in the storage device.

TECHNICAL FIELD

The present specification discloses a stored image reclassification system and method in which components picked up by a suction nozzle of a component mounting machine are imaged by a camera, and the captured images are classified as normal images or abnormal images and stored in a storage device, thereafter the stored images are reclassified.

BACKGROUND ART

Conventionally, in a component mounting machine, when a component supplied from a feeder is picked up by a suction nozzle and conveyed onto a circuit board, the component picked up by the suction nozzle is imaged by a camera, the captured image is processed to recognize the component, the component suction orientation or the like is determined to be normal or abnormal based on the recognition result, and in addition to mounting the component onto the circuit board, it is possible to examine the images captured at the time of an abnormality occurrence when investigating the cause of various abnormalities that have occurred in the component mounting machine, and images captured during production are classified as a normal image or an abnormal image and stored in a storage device. The stored images are also used to improve shape data of the component for image processing, to improve the recognition accuracy of the component suction orientation (i.e., the position and angle), and the like.

However, if all the images captured during production are stored in the storage device for a long period of time, the number of stored images becomes enormous, and therefore, it takes a great deal of time and labor to examine the stored images when investigating the cause of various abnormalities that have occurred in the component mounting machine.

In Patent Literature 1 (JP-A-2012-169394), only images for events that may cause changes in the operational state of a feeder or a mounting head are selected and stored in a storage device for a long period of time.

PATENT LITERATURE

SUMMARY OF INVENTION

Since various devices other than the feeder and the mounting head are mounted on the component mounting machine, the causes of various abnormalities occurring in the component mounting machine are not limited to only the feeder and the mounting head. Therefore, as described in Patent Document 1, if only images for events having caused changes in the operational state of a feeder or a mounting head are selected and stored, it is difficult to ascertain the cause of an abnormality caused by a device other than the feeder or the mounting head even if the stored images are examined.

However, as described above, when all images taken during production are saved, the number of saved images becomes enormous, and therefore, it takes a great deal of time and labor to examine the saved images when investigating the causes of various abnormalities occurring in the component mounting machine.

Further, component recognition results of the image recognition system of a component mounting machine are not necessarily accurate, rather there is a possibility that the image recognition system is misrecognizing. For example,FIGS. 2 to 4are all images of components with leads that are classified as normal images, however, only the image ofFIG. 2is an image in which the component suction orientation (i.e., the position and angle) is correctly recognized, and the image ofFIG. 3is an example in which the component suction orientation is misrecognized by erroneously determining a foreign particle (e.g., a piece of dust) to be a lead, and the image ofFIG. 4is another example in which the component suction orientation is misrecognized by erroneously determining an abrasion mark on the lower surface of the suction nozzle, by which the component was not picked up due to a component pickup error, to be a lead. As described above, since images whose determination as a normal image is suspect are mixed in among the large number of stored normal images, when using the stored normal images, it is necessary for an operator to visually check the large number of normal images one by one and to reclassify the images whose determination as a normal image is suspect, such as shown inFIGS. 3 and 4, and it takes a great deal of time and effort to perform the operation.

Solution to Problem

In order to solve the above-mentioned problem, the present invention may be configured to: image a component by a camera, the component being picked up by a suction nozzle of a component mounting machine; recognize the component by processing the captured image using an image recognition system; determine the image to be normal or abnormal based on the recognition result; classify the image as a normal image or an abnormal image and store the image in a storage device, and inspect a component mounting board unloaded from the component mounting machine with an inspection device, wherein the stored image reclassification system has a reclassification section configured to reclassify the normal image, based on the inspection result of the inspection device, as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect, and stores the normal image in the storage device.

For example, among the components determined to have a mounting defect (i.e., a deviation in mounting position) or to have not been mounted, whose determinations are made by the inspection device inspecting component mounting boards unloaded from the component mounting machine, there is a component for which the image recognition system of the component mounting machine may have erroneously determined the image of the component to be normal in a way such that the component suction orientation was misrecognized due to a foreign particle (e.g., a piece of dust), as shown inFIG. 3, or the component was misrecognized due to an abrasion mark on the lower surface of the suction nozzle for which a component pickup error had occurred, as shown inFIG. 4.

Therefore, an image that is classified as normal by the image recognition system of the component mounting machine is automatically reclassified based on the inspection result of the inspection device as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect, after which the reclassified image is stored in the storage device. This eliminates the need for an operator to visually check and reclassify a large number of normal images stored in the storage device one by one, and when investigating the cause of a mounting failure or the like, the investigation target can be narrowed down to the images whose determination as a normal image is suspect, thereby the investigation of the cause can be proceeded efficiently in a relatively short time.

In this case, when an inspection device for inspecting whether the mounted state for each component mounted on the circuit board is acceptable is used as an inspection device, the reclassification section reclassifies, from among the normal images, the normal image of the component whose inspection result of the inspection device is unacceptable as the normal image for which the determination is suspect.

Further, the present invention may be configured to have the stored image reclassification system including a component suction orientation measurement section configured to measure positions and angles of components of multiple normal images stored in the storage device, and a reclassification section configured to statistically process the positions and the angles of the components of the normal images measured by the component suction orientation measurement section for each component type, and reclassify the normal image as an image for which the determination as a normal image is suspect or as an image for which the determination as a normal image is not suspect, based on whether the position or the angle of the component falls outside a predetermined variation range, and stores the normal image in the storage device. In general, it is presumed that components which have been stably mounted on mounting positions of boards by a component mounting machine have been picked up by a suction nozzle in a substantially similar orientation. Accordingly, the component suction orientation (i.e., the position and angle) of the multiple normal images stored in the storage device are measured, the measured values are then statistically processed for each component type, and though the image is determined to be a normal image by the image recognition system of the component mounting machine, if it is determined that the normal image has any measured value of the component suction orientation falling outside the predetermined variation range, the image is reclassified as an image whose determination as a normal image is suspect, because there is a possibility of erroneous image recognition.

In this case, the predetermined variation range may be set based on the standard deviation from an average value for the position and the angle of the component. In place of the average value, the median value, or the mode value, in other words, any value indicating the central position of the distribution of the measured values may be used.

The component suction orientation measurement section may also measure the position and angle of the component of the abnormal image stored in the storage device, and the reclassification section may reclassify the abnormal image as an image whose determination as an abnormal image is suspect or as an image whose determination as an abnormal image is not suspect, based on whether both the position and the angle of the component of the abnormal image measured with the component suction orientation measurement section are within a predetermined variation range of the normal image of the same component type as the component, and store the abnormal image in the storage device In other words, the component suction orientation (i.e., the position and angle) of the abnormal image stored in the storage device is measured, and if the measured values of the component suction orientation of the abnormal image are within the predetermined variation range of the component suction orientation of a normal image of the same component type, the abnormal image may be within the predetermined variation range of a normal image, there is a possibility that the component suction orientation was misrecognized, thus the determination of the abnormal image by the image recognition system of the component mounting machine is reclassified as being suspect.

Further, a configuration of the present invention may include: a normal image selection section configured to select a normal image whose determination as a normal image is not suspect from among multiple normal images stored in a storage device, and a template image creation section configured to create a template image by moving and rotating the normal image so that the position and the angle of the component of the normal image selected by the normal image selection section coincide with the position and angle of a reference position; a matching processing section for matching remaining normal images using the template image created by the template image creation section; and a reclassification section for reclassifying the normal image as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect based on the processing result of the matching processing section, and store the image in the storage device. In this manner, a template image is created by selecting a normal image whose determination as a normal image is not suspect, from among multiple normal images stored in the storage device, and a matching process is performed on the remaining normal images using the template image, by which a correlation value and a deviation amount, serving as indices indicating the correlation (i.e., the similarity) between the two images, can be obtained. When the correlation between the two images is low (i.e., the correlation value is low, or the deviation amount is very different from the recognition result of the image recognition system of the component mounting machine), though the image recognition system of the component mounting machine determines that the image is a normal image, there is a possibility that the image was misrecognized, and the image is therefore reclassified as an image whose determination as a normal image is suspect.

In this case, the normal image selection section may select a normal image whose determination as a normal image is not suspect from among multiple normal images stored in the storage device based on the inspection result of the inspection device, which inspects component mounting boards unloaded from the component mounting machine.

Alternatively, the normal image selection section may measure the positions and angles of components of multiple normal images stored in the storage device, perform statistical processing on the measured values for each component type, and select a normal image whose determination as a normal image is not suspect (e.g., a normal image located at the center of the distribution) from the multiple normal images based on the processing result.

Alternatively, the normal image selection section may create an average normal image from multiple normal images stored in the storage device and select the average normal image as a normal image whose determination as a normal image is not suspect.

Further, the matching processing section may perform a matching process on the abnormal image using the template image, and the reclassification section may reclassify the abnormal image as an image whose determination as an abnormal image is suspect or as an image whose determination as an abnormal image is not suspect based on the processing result of the matching processing section, and store the image in the storage device. In other words, a matching process is performed on an abnormal image using the template image created using a normal image whose determination as a normal image is not suspect, and in the case where the correlation between the two images is high, even if the image recognition system of the component mounting machine determines that the image is an abnormal image, the image is reclassified as an image whose determination as an abnormal image is suspect since there is a possibility that the image was misrecognized.

Alternatively, a classifier for recognizing a component may be created by learning a normal image reclassified by the reclassification section as a normal image whose determination as a normal image is not suspect. By doing so, it is possible to learn only the normal image whose determination as a normal image is not suspect as learning data for creating a high-precision classifier, and by mounting this classifier in the image recognition system of the component mounting machine, it is possible to improve the component recognition accuracy of the image recognition system of the component mounting machine.

DESCRIPTION OF EMBODIMENTS

Embodiment 1 will be described with reference toFIGS. 1 to 5. First, the configuration of component mounting line10will be described with reference toFIG. 1. Component mounting line10is configured by arranging one or multiple component mounting machines12and mounting-related machines such as solder printer13and a flux coating device (not shown) along a conveying direction of circuit board11. Inspection device14, which can be an appearance inspection device for inspecting whether the mounted state of each component mounted on circuit board11is acceptable, is installed on the side of component mounting line10from which the board is unloaded. The number of inspection devices14installed in component mounting line10is not limited to one but may be a multiple, and some inspection devices14may be installed between component mounting machines12in the middle of component mounting line10.

Component mounting machine12, solder printer13, and inspection device14of component mounting line10are connected via network16to production management computer21in a mutually communicable manner, and production by component mounting line10is managed by production management computer21. In accordance with the production job transmitted from production management computer21, a predetermined number of components can be mounted on circuit board11, such that each component mounting machine12repeats the operations of: moving mounting head18along the path consisting of a component pickup position→a component imaging position→a component mounting position; picking up the component supplied by feeder19with a suction nozzle (not shown) of mounting head18and imaging the component with the component imaging camera (not shown); processing the captured image with image recognition system17of component mounting machine12to measure the suction orientation (i.e., the position X, Y and the angle θ) of the component; correcting any deviation of the position X, Y and the angle θ of the component, and mounting the component on circuit board11.

Further, image recognition system17of each component mounting machine12determines whether the image captured with the component imaging camera is normal or abnormal based on a component recognition result, classifies the image as a normal image or an abnormal image, and stores the normal image or the abnormal image in storage device20. Storage device20has a storage capacity capable of storing the number of images necessary to investigate a cause when an abnormality of component mounting machine12occurs, which is configured by a rewritable nonvolatile storage medium, such as a hard disk device or the like, with holding stored data even in a power-off state. When the number of stored images in storage device20exceeds a predetermined number or a predetermined memory capacity, the oldest stored image is automatically deleted and the most recent image is stored.

The component mounting board produced by sequentially passing through component mounting machines12of component mounting line10is loaded into inspection device14, and whether the mounting state is acceptable is inspected for each component mounted on circuit board11. Stored image reclassification computer22is connected to network16, and the inspection result (pass/fail) of inspection device14is transmitted to production management computer21and stored image reclassification computer22.

Among the components determined to have a mounting defect (i.e., a deviation in mounting position) or to have not been mounted, whose determinations are made by inspection device14inspecting component mounting boards unloaded from component mounting machine12, there is a component for which image recognition system17of the component mounting machine12may have erroneously determined the image of the component to be normal in a way such that the component suction orientation was misrecognized due to a foreign particle (e.g., a piece of dust), as shown inFIG. 3, or the component was misrecognized due to an abrasion mark on the lower surface of the suction nozzle to which a component pickup error had occurred, as shown inFIG. 4.

For this reason, stored image reclassification computer22executes the stored image reclassification program ofFIG. 5, which will be described later, to thereby function as a “reclassification section” for reading out the normal image stored in storage device20, reclassifying the normal image as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect based on the inspection result of inspection device14, and storing the normal image in storage device20. Here, “determination as a normal image is suspect” means “there is a possibility that the image is not a normal image”. When storing the results of the reclassification, the file names of the reclassified images may be sorted according to their classification and written in a text file, the storage area for storing the reclassified images may be divided for according to classification, a specific character string indicating the classification may be added to the file name of the reclassified image, and in short, the operator may easily visually confirm the result of the reclassification.

Further, in embodiment 1, in order for inspection device14to inspect whether the mounted state for each component mounted on circuit board11is acceptable, stored image reclassification computer22reclassifies, as images whose determination as a normal image is suspect, the normal images of components for which the inspection result of inspection device14has failed from among the normal images stored in storage device20of component mounting machine12.

When image recognition system17of component mounting machine12determines that an image of a component picked up by the suction nozzle is an abnormal image, the component is discarded and is not mounted on circuit board11so that a component whose image was determined to be abnormal is not inspected by inspection device14.

The reclassification of the normal image of embodiment 1 described above is executed by stored image reclassification computer22, in accordance with the stored image reclassification program ofFIG. 5, as follows. The stored image reclassification program shown inFIG. 5is started during production (i.e., during operation of the component mounting machine12) or after production is completed. When there is space in the CPU processing capacity of the control device, which is the main part of image recognition system17of component mounting machine12, the stored image reclassification program ofFIG. 5may be executed by the control device of component mounting machine12. In this case, since the control device of component mounting machine12also functions as a “reclassification unit”, it is not necessary to provide stored image reclassification computer22. Alternatively, the stored image reclassification program ofFIG. 5may be executed by production management computer21. In this case, since production management computer21also functions as a “reclassification unit”, it is not necessary to provide stored image reclassification computer22.

When the stored image reclassification program ofFIG. 5starts, first, in step101, a normal image stored in storage device20of component mounting machine12is acquired, and in the next step, step102, the inspection result for the component in the normal image is acquired from inspection device14.

Thereafter, the process proceeds to step103, where it is determined whether the inspection result of the component fails, and if the inspection result is determined to have failed, the process proceeds to step104, where the normal image is reclassified as an image whose determination as a normal image is suspect and then the image is stored in storage device20, after which the process proceeds to step106. On the other hand, if it is determined in step103that the inspection result of the component passes, the process proceeds to step105, where the normal image is reclassified as a normal image and stored in the storage device20, after which the process proceeds to step106.

In step106, it is determined whether a normal image to be reclassified remains, and if it is determined that a normal image remains, the above-described process of step101and subsequent steps is repeated, the normal image is reclassified as an image for which the determination as a normal image is suspect or not suspect based on the inspection result of inspection device14, and the process of storing the normal image in storage device20is repeated. Thereafter, when it is determined in step106that no normal image to be reclassified remains, the program ends.

In the stored image reclassification program ofFIG. 5, the acquisition of the normal image and the acquisition of the inspection result are performed one by one in steps101to102, but all the normal images and all the inspection results that can be acquired at that time may be acquired collectively.

In embodiment 1 described above, since the normal image classified by image recognition system17of component mounting machine12is automatically reclassified into images whose determination as a normal image is suspect or images whose determination as a normal image is not suspect, based on the inspection result of inspection device14, and stored in storage device20by the stored image reclassification program ofFIG. 5, it becomes unnecessary for an operator to visually check and reclassify a large number of normal images stored in storage device20one by one, and when investigating the cause of a mounting failure or the like, the investigation target can be narrowed down to the images whose determination as a normal image is suspect, thereby the investigation of the cause can be proceeded efficiently in a relatively short time.

As another method for using the normal image reclassified by the stored image reclassification program ofFIG. 5, a classifier for recognizing a component may be created by learning a normal image reclassified by the reclassification section as a normal image whose determination as a normal image is not suspect. By doing so, it is possible to learn only the normal image for which the determination as being normal is not suspect as learning data for creating a high-precision classifier, and by mounting this classifier in image recognition system17of component mounting machine12, it is possible to improve the component recognition accuracy of image recognition system17of component mounting machine12.

In embodiment 1, inspection device14inspects whether the mounted state for each component mounted on circuit board11is acceptable, but when there is a mounting area on the mounting surface of circuit board11that cannot be inspected for each component or when inspecting each mounting area including multiple components, the normal images of all the components included in the mounting area in which the inspection result has failed may be reclassified as images for which the determination as normal is suspect.

Next, embodiment 2 will be described with reference toFIGS. 6 to 8. The same reference numerals are assigned to substantially the same parts as those in embodiment 1, and description thereof is omitted or simplified, and mainly different parts will be described.

In general, it is presumed that components which have been stably mounted on mounting positions of circuit boards11by the component mounting machine were picked up by the suction nozzle of component mounting machine12in a substantially similar orientation. In consideration of this characteristic, in embodiment 2, stored image reclassification computer22executes the stored image reclassification program ofFIGS. 6 and 7, which will be described later, to measure component suction orientations (i.e., positions X, Y and angle θ) of multiple normal images stored in storage device20of component mounting machine12, the measured values are then statistically processed for each component type to evaluate the distribution (seeFIG. 8), and if it is determined that a normal image has any of the measured values X, Y, and θ falling outside the predetermined variation range, because there is a possibility of misrecognition, the image is reclassified as an image whose determination as a normal image is suspect, even if the image is determined to be a normal image by image recognition system17of component mounting machine12.

In embodiment 2, the predetermined variation range is set by the following equation in accordance with the standard deviations (σX, σY, σθ) about the respective average values (Xav, Yav, θav) of the respective component positions X, Y and angles θ.

Here, aX, aY, aθ are coefficients for adjusting the size of the predetermined variation range. The coefficients aX, aY, aθ may be fixed values that are set in advance, or may be coefficients that can be adjusted by an input operation by an operator. In place of the average value, the median value, or the mode value, in other words, any value indicating the central position of the distribution of the measured values X, Y, and θ may be used.

Further, in embodiment 2, also for the abnormal image stored in storage device20of component mounting machine12, the component suction orientation (X, Y, θ) of the abnormal image is measured, and the abnormal image is reclassified as an image whose determination as an abnormal image is suspect or as an image whose determination as an abnormal image is not suspect based on whether the measured values X, Y, θ are within a predetermined variation range of the component suction orientation of a normal image of the same component type as the component, and stored in the storage device20. In short, the component suction orientation (X, Y, θ) of the abnormal image stored in storage device20is measured, and if the measured values X, Y, θ are within the predetermined variation range of the component suction orientation (X, Y, θ) of the normal image of the same component type as the component, the abnormal image falls within the predetermined variation range of the normal image and there is a possibility that the component suction orientation was misrecognized, and the determination of the abnormality by image recognition system17of component mounting machine12is reclassified as being suspect.

The reclassification of the normal image of embodiment 2 described above is executed by stored image reclassification computer22in accordance with the stored image reclassification program ofFIGS. 6 and 7, as follows. The stored image reclassification program ofFIGS. 6 and 7may be executed by the control device of component mounting machine12or production management computer21.

The stored image reclassification program ofFIGS. 6 and 7is started after production is completed and functions as a “reclassification section”. Even during production, the stored image reclassification program ofFIGS. 6 and 7may be executed after the number of images stored in storage device20of component mounting machine12reaches the number of images required to execute statistical processing (i.e., calculation of an average value and a standard deviation) with accuracy.

When the stored image reclassification program ofFIGS. 6 and 7is started, first, in step201, all the normal images stored in storage device20of component mounting machine12are acquired, and then, in step202, the component suction orientation (X, Y, θ) of each normal image is measured. The process of step202serves as a “component suction orientation measurement section”.

Thereafter, the process proceeds to step203, where the measured values of the component suction orientation (X, Y, θ) are statistically processed for each component type, and the average values (Xav, Yav, θav) and the standard deviations (σX, σY, σθ) are calculated. Thereafter, the process proceeds to step204, and a predetermined variation range of the component suction orientation (X, Y, θ) is calculated using the above-described equations (1) to (3).

Thereafter, the process proceeds to step205, where it is determined whether the measured values of the component suction orientation (X, Y, θ) of the normal image to be reclassified fall outside the predetermined variation range, and as a result, if it is determined that a measured value of the component suction orientation (X, Y, θ) of the normal image falls outside the predetermined variation range, the process proceeds to step206, where the normal image is reclassified as an image whose determination as a normal image is suspect, after which the image is stored in storage device20and the process proceeds to step208. On the other hand, if it is determined in step205that the measured values of the component suction orientation (X, Y, θ) of the normal image is within the predetermined variation range, the process proceeds to step207, where the normal image is reclassified as a normal image and stored in storage device20, and the process proceeds to step208.

In step208, it is determined whether a normal image to be reclassified remains, and if it is determined that a normal image remains, the above-described process of step205and subsequent steps is repeated, the normal image is reclassified as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect based on whether the measured values of the component suction orientation (X, Y, θ) of the normal image falls outside the predetermined variation range, and the process of storing the normal image in storage device20is repeated.

Thereafter, when it is determined in step208that no normal image to be reclassified remains, the process in step209and subsequent steps ofFIG. 7is executed, the abnormal image stored in storage device20of component mounting machine12is also reclassified as an image whose determination as an abnormal image is suspect or as an image whose determination as an abnormal image is not suspect, and the image is stored in storage device20as follows.

First, in step209, all the abnormal images stored in storage device20of component mounting machine12are acquired, and then the process proceeds to step210, where the component suction orientations (X, Y, θ) of the abnormal images are measured. The process of step210serves as a “component suction orientation measurement section”. Thereafter, the process proceeds to step211, where it is determined whether the measured values of the component suction orientation (X, Y, θ) of the abnormal images to be reclassified are within a predetermined variation range of the component suction orientation of a normal image of the same component type. In this case, as the predetermined variation range, the values calculated in step204may be used as is, or the size of the predetermined variation range may be changed by changing the coefficients aX, aY, aθ for adjusting the size of the predetermined variation range.

If it is determined in step211that the measured values of the component suction orientation (X, Y, θ) of the abnormal image are within the predetermined variation range of the normal image, the process proceeds to step212, where the abnormal image is reclassified as an image whose determination as an abnormal image is suspect and the image is stored in storage device20, and the process proceeds to step214. On the other hand, if it is determined in step211that the measured values of the component suction orientation (X, Y, θ) of the abnormal image falls outside the predetermined variation range of the normal image, the process proceeds to step213, where the abnormal image is reclassified as an image whose determination as an abnormal image is not suspect and the image is stored in storage device20, and the process proceeds to step214.

In step214, it is determined whether an abnormal image to be reclassified remains, and if it is determined that an abnormal image remains, the above-described process from step211and subsequent steps are repeated, the abnormal image is reclassified as an image whose determination as an abnormal image is suspect or as an image whose determination as an abnormal image is not suspect based on whether the measured values for the component suction orientation (X, Y, θ) of the abnormal image fall within the predetermined variation range of the normal image, and the process of storing the abnormal image in storage device20is repeated. Thereafter, when it is determined in step214that no abnormal image to be reclassified remains, the program ends.

Also in embodiment 2 described above, the same effects as in embodiment 1 can be obtained. Further, in embodiment 2, since the abnormal images stored in storage device20of component mounting machine12, based on whether the measured values of the component suction orientation (X, Y, θ) are within a predetermined variation range of the component suction orientation of a normal image of the same component type, are also reclassified into images whose determinations as the abnormal image are suspect or images whose determinations as the abnormal image are not suspect, it becomes unnecessary for an operator to visually check and reclassify the abnormal images stored in the storage device20one by one, and when investigating the cause of a mounting failure or the like, the investigation target can be narrowed down to images whose determination as an abnormal image is suspect, thereby the investigation of the cause can be proceeded efficiently in a relatively short time.

Note that, when image recognition system17of component mounting machine12determines that the image is an abnormal image, since the component picked up by the suction nozzle is discarded without being mounted on circuit board11, if the number of erroneous determinations of abnormal images increases, wasteful discarding of components increases, production time becomes longer, and production does not end as scheduled. Therefore, when production delays become remarkable, the operator needs to investigate the cause. In this case, the operator may visually check the abnormal images stored in storage device20of component mounting machine12to examine whether there is a large amount of wasteful component discards, but visually checking the many abnormal images stored in storage device20one by one is very laborious.

In this regard, in embodiment 2, since the abnormal images stored in storage device20of component mounting machine12are reclassified into images whose determination as abnormal images are suspect or images whose determination as abnormal images are not suspect, when there are many wasteful component discards (i.e., when the production is not completed as scheduled), the operator can narrow down the target images to examine to abnormal images whose determination as abnormal images is suspect, and quickly becomes aware of the cause of the large amount of wasteful component discards.

As another method for using the normal image reclassified by the stored image reclassification program ofFIGS. 6 and 7, a classifier for recognizing a component may be created by learning a normal image reclassified by the reclassification section as a normal image whose determination as a normal image is not suspect and this classifier may be mounted in image recognition system17of component mounting machine12.

Next, embodiment 3 will be described with reference toFIGS. 9 and 10. However, the same reference numerals are assigned to substantially the same parts as in embodiment 1, and description thereof is omitted or simplified, and mainly the parts which differ will be described.

In embodiment 3, by executing the stored image reclassification program ofFIGS. 9 and 10with stored image reclassification computer22, a template image is created by selecting one of multiple normal images stored in storage device20of component mounting machine12whose determination as a normal image is not suspect, and by performing a matching process of matching the remaining normal images with the template image, the normal image is reclassified and stored in storage device20as a normal image whose determination as a normal image is not suspect, and also by performing a matching process of the abnormal image stored in storage device20with the template image, the abnormal image is reclassified and stored in storage device20as an abnormal image whose determination as an abnormal image is suspect or not suspect.

As described above, if the matching process is performed on the remaining normal images using the template image created using a normal image whose determination as a normal image is not suspect, the correlation value and the deviation amount, both of which serve as indices indicating the correlation (i.e., the similarity) between the two images, can be obtained. When the correlation between the two images is low (i.e., when the correlation value is low or when the deviation amount is very different from the recognition result of image recognition system17of component mounting machine12), even if image recognition system17of component mounting machine12determines that the image is a normal image, there is a possibility that the image was misrecognized and therefore the image can be reclassified as an image whose determination as a normal image is suspect.

Further, a matching process is performed on the abnormal image with a template image created using a normal image whose determination as a normal image is not suspect, and in the case where the correlation between the two images is high, even if image recognition system17of component mounting machine12determines that the image is an abnormal image, the image may be reclassified as an image whose determination as an abnormal image is suspect since there is a possibility that the image was misrecognized. Here, “determination as an abnormal image is suspect” means “there is a possibility that the image is not an abnormal image”.

The reclassification of the stored image of embodiment 3 described above is executed by stored image reclassification computer22in accordance with the stored image reclassification program ofFIGS. 9 and 10. The stored image reclassification program ofFIGS. 9 and 10is started during production (i.e., during operation of component mounting machine12) or after production is completed, and functions as a “reclassification section”. The stored image reclassification program ofFIGS. 9 and 10may be executed by the control device of component mounting machine12or production management computer21.

When the stored image reclassification program ofFIGS. 9 and 10is started, first, in step301, all the normal images stored in the storage device20of the component mounting machine12are acquired, then the process proceeds to step302, in which one of the acquired normal images whose determination as a normal image is not suspect is selected by any one of the following methods (1) to (3). The process of step302serves as a “normal image selection section”.

(1) Similarly to embodiment 1, one of the multiple normal images acquired in step301whose determination as a normal image is not suspect is selected based on the inspection result of inspection device14, which inspects component mounting boards carried out from component mounting device12. That is, the normal image of the component for which the inspection result of the inspection device14has passed is selected as a normal image whose determination as a normal image is not suspect.

(2) Similarly to embodiment 2, the component suction orientations (X, Y, θ) of the multiple normal images acquired in step301are measured, the measured values are statistically processed for each component type, and one of the multiple normal images (for example, a normal image located at the center of the distribution) whose determination as a normal image is not suspect is selected from the multiple normal images based on the processing result.

(3) An average normal image is created from the multiple normal images acquired in step301, and the average normal image is selected as a normal image whose determination as a normal image is not suspect. In this case, an average image created by simply averaging the multiple normal images may be used, or an average normal image may be created by using another method (for example, KL expansion).

Thereafter, the process proceeds to step303, where the normal image is moved and rotated so that the position (X, Y) and the angle θ of the component of the normal image selected in step301, whose determination as a normal image is not suspect, respectively coincide with the position and angle of a reference position, thereby creating a template image. The process of step302serves as a “template image creation section”.

Thereafter, the process proceeds to step304, in which a matching process (e.g., normalized correlation, phase-limited correlation, etc.) is performed on the normal image to be reclassified using the template image, and a correlation value and a deviation amount, both of which are indices indicating the correlation between the two images, are obtained.

Then, in the next step, step305, it is determined whether the correlation between the normal image to be reclassified and the template image is low based on the correlation value and the deviation amount. In this case, when the correlation value is smaller than a predetermined value or when the deviation amount is significantly different from the recognition result of image recognition system17of component mounting machine12by a predetermined value or more, it is determined that the correlation between the two images is low, and the process proceeds to step306, where the normal image is reclassified as an image whose determination as a normal image is suspect and is stored in storage device20, and the process proceeds to step308. On the other hand, if it is determined in step305that the correlation between the two images is high, the process proceeds to step307, where the normal image is reclassified as a normal image and stored in storage device20, and the process proceeds to step308.

In step308, it is determined whether a normal image to be reclassified remains, and if it is determined that a normal image remains, the above-described process from step304and subsequent steps is repeated, the normal image is reclassified as an image whose determination as a normal image is suspect or as an image whose determination as a normal image is not suspect based on whether the correlation between the normal image to be reclassified and the template image is low, and the process of storing the abnormal image in storage device20is repeated. Thereafter, when it is determined in step308that no normal image to be reclassified remains, the process in step309and subsequent steps inFIG. 10is executed, and the abnormal image stored in storage device20of component mounting machine12is also reclassified, as will be described next, as an image whose determination as an abnormal image is suspect or not suspect, and stored in the storage device20.

First, in step309, after all the abnormal images stored in storage device20of component mounting machine12are acquired, the process proceeds to step310, in which a matching process (e.g., normalized correlation, phase limited correlation, etc.) is performed on the abnormal images to be reclassified using the template image created in step303, and a correlation value and a deviation amount, both of which serve as indices indicating the correlation between the two images, are obtained.

Then, in the next step, step311, it is determined whether the correlation between the abnormal image to be reclassified and the template image is high based on the correlation value and the deviation amount. In this case, when the correlation value is larger than the predetermined value or when the deviation amount is not very different from the recognition result of image recognition system17of component mounting machine12, it is determined that the correlation between the two images is high, and the process proceeds to step312, where the abnormal image is reclassified as an image whose determination as an abnormal image is suspect and is stored in storage device20, after which the process proceeds to step314. On the other hand, if it is determined in step311that the correlation between the two images is low, the process proceeds to step312, where the abnormal image is reclassified as an image whose determination as an abnormal image is not suspect and then stored in the storage device20, and the process proceeds to step314.

In step314, it is determined whether an abnormal image to be reclassified remains, and if it is determined that an abnormal image remains, the above-described process from step310and subsequent steps is repeated, the abnormal image is reclassified as an image whose determination as an abnormal image is suspect or as an image whose determination as an abnormal image is not suspect based on whether the correlation between the abnormal image to be reclassified and the template image is high, and the process of storing the abnormal image in storage device20is repeated. Thereafter, when it is determined in step314that no abnormal image to be reclassified remains, the program ends. The same effects as those of embodiments 1 and 2 can be obtained also in embodiment 3 described above.

As another method for using the normal image reclassified by the stored image reclassification program ofFIGS. 9 and 10, a classifier for recognizing a component may be created by learning a normal image reclassified by the reclassification section as a normal image whose determination as a normal image is not suspect and this classifier may be mounted in image recognition system17of component mounting machine12.

In above-described embodiments 1 to 3, the normal image and the abnormal image are stored in storage device20of component mounting machine12, but may be stored in the storage device of stored image reclassification computer22, the storage device of production management computer21, or another server.

Further, the present invention is not limited to above-mentioned embodiments 1 to 3, and it is needless to state that the present invention can be implemented by various modifications within a range that does not deviate from the gist, for example, the configuration of component mounting line10may be changed.

DESCRIPTION OF REFERENCE NUMERALS