Method of inspecting jointed portion

Abstract: A bonded portion of a bonding wire (3) jointed to a semiconductor chip (2), for example, is inspected by utilizing an image pickup device by a method which comprises the steps of image picking up a bonded portion (4) plastically deformed, extracting the bonded portion as a mass of pixels each having predetermined brightness from an image picked up by the image pickup device, decreasing a size of the mass of the extracted pixels by carrying out, by at least one step, logical brightness conversion, i.e. filtering, operation on the basis of the relationship in brightness of one of the pixels with respect to other pixels which surround the one of the pixels, and repeating the brightness conversion operation till the mass of the extracted pixels is extinguished. A size of the bonded portion is detected in dependency on the number of times of the brightness conversion operations carried out till the mass of the pixels is extinguished to thereby evaluate the quality of the bonded portion. A center of the mass of the pixels is simultaneously detected as a center of the bonded portion just before a time when the mass thereof is extinguished to thereby inspect a position of the bonded portion.

Description: The present invention relates to a method of inspecting a joint portion of a first member which is plastically deformed and joined to a second member by utilizing an image pickup device, comprising the steps of: image picking up the joint portion plastically deformed; and extracting the joint portion as a mass of pixels each having predetermined brightness from the image picked up by the image pickup device. Such method is disclosed in JP-A-61 128 143 where it is suggested that image data of a connected part are compared with stored proper image data. Then a difference of the number of picture elements (pixels) in the connected part image data and the stored proper image data is calculated, and this difference is finally compared with a predetermined value. If the difference is equal to or larger than this predetermined value, it is judged that the joint portion is inferior. JP-A-61 132 843 discloses a printed board inspecting method which also relies on a comparison of image data of a reference sample with image data of a printed board to be inspected for determining if through-holes in a printed board are defective or not. The background of the invention will additionally be described hereinafter for facilitating to understand the present invention. In the prior art, when a semiconductor device is manufactured, one end of a bonding wire is bonded to a surface of a bonding pad (electrode) of the semiconductor chip and the other end of the bonding wire, as an external terminal, is bonded to an inner lead of a lead frame. In this manufacturing process, the one end of the bonding wire is plastically deformed by means of melting to form a bonding ball as a bonding portion and the thus formed bonding ball is bonded to the surface to the bonding pad by pressing same. In such bonding process, it is necessary to inspect the size and the position of the bonding ball to thereby evaluate the quality of the bonded portion. However, in the conventional technology, such inspection is carried out by visually observing the outer appearance of the bonded portion by utilizing a stereo-microscope. Such visual observation involves defects or problems such as observation mistakes and different evaluation by different viewers. This inspection method further lacks real-time performance from the bonding time to the inspection time. In order to obviate these defects or problems, the provision of an exact automatic inspection method has been highly required. Further in the known art, to solve the above problem, there is provided a method in which the bonded portion of the bonding ball is observed as an image by an image pickup device and an area of the bonded portion in the image is measured, the measured value being then compared with a reference value to thereby evaluate the quality of the bonded portion. This method, however, also involves a problem with respect to the extinction of the bonded portion from the other portion in the image. Such extinction generally requires a complicated pretreatment, though being dependent on the quality of the image, resulting in lowering the inspection speed and increasing the cost of an inspection device to be used. Furthermore, in this method, in order to discriminate the bonded portion from the other portion in the image, it is required to carry out an image pickup procedure with relatively high magnitude so that the bonded portion is imaged with more than 50 pixels. In case of a number of pixels less than 50, it is extremely hard to discriminate the bonded portion from the other portion in the image. Furthermore, in the inspection of the size of the bonding ball, when the bonding ball has the image picked up from the upper portion thereof, a part of the image of the bonding ball is concealed by the bonding wire, which results in complicated and difficult preprocessing for specifying the outer contour of the bonding ball, as well as specifying the portion concealed by the bonding wire. Moreover, in addition to the above problem, in a case where the contour of the image of the bonding ball is not clear and is not shown with a smooth line, the area of the bonding ball is measured differently to a great extent, repending on the selection of the contour line, and particularly, with respect to the image picked up with low magnitude, the measuring, i.e. inspection, performance will be degraded. The difficulty of specifying the contour line results in the difficulty of the exact positioning of the bonded portion by the inspection of the central portion thereof. SUMMARY OF THE INVENTION An object of the present invention is to substantially eliminate defects or drawbacks encountered in the prior art and to provide a method of inspecting a joint or bonded portion capable of easily evaluating the quality of a bonded portion and detecting the position thereof. This and other objects can be achieved according to the present invention by providing a method of inspecting a joint portion of a first member which is plastically deformed and joined to a second member by utilizing an image pickup device, comprising the steps of: image picking up the joint portion plastically deformed; extracting the joint portion as a mass of pixels each having predetermined brightness from an image picked up by the image pickup device; decreasing the size of the mass of the extracted pixels by carrying out, by at least one step, logical brightness conversion operation on the basis of the brightness relationship of one of the pixels with respect to other pixels which surround the one of the pixels; repeating the brightness conversion operation until the mass of the extracted pixels is extinguished; detecting the size of the joint portion in dependency on the number of times of the brightness conversion operations carried out until the mass of the pixels is extinguished to thereby evaluate the quality of the joint portion; and/or detecting a center of the mass of the pixels as a center of the joint portion just before a time when the mass thereof is extinguished to thereby inspect a position of the joint portion. According to the present invention, the evaluation of the quality of the jointed portion and the position thereof can be both simultaneously carried out. Preferred embodiments of the invention are indicated in the dependent claims. In particular, according to a preferred embodiment, the method may include a step of temporarily increasing the size of the mass of the pixels before the step of decreasing the size of the mass of the pixels by a reverse logical brightness conversion operation to that of the decreasing step. According to the present invention described above, the size of the joint portion can be inspected in accordance with the number of times of logical brightness conversion operations carried out until the mass of the extracted pixels is extinguished, without specifying the contour of the image of the joint portion by utilizing the image pickup device. Accordingly, the size of the joint portion and the quality of the joint portion can be surely inspected by the image pickup operation with relatively low magnitude without effecting a complicated preprocessing. Furthermore, the position of the joint portion can also be easily inspected substantially simultaneously in dependency of the mass of the pixels which gradually decreases in accordance with the repeated brightness conversion operations. In addition, a portion of the image having a locally erroneous brightness due to noise, for example, can be corrected by temporarily increasing the size of the mass of the pixels before the decreasing step. In this case, the temporarily increase of the size of the mass of the pixels can be amended by considering the extra number of times of the brightness conversion operations. BRIEF DESCRIPTION OF THE DRAWINGS In the accompanying drawings: Fig. 1 is a view showing an image extracted as a mass of pixels having predetermined brightness of an image picked-up portion; Figs. 2A, 2B and 2C, represent tables to be utilized for a brightness conversion operation for decreasing the size of the mass of the pixels; Fig. 3 is a view of an image after the brightness conversion operation is carried out as to the image shown in Fig. 1 in dependency on the table of Figs. 2A, 2B and 2C; Fig. 4 is a view of an image after the brightness conversion operation is carried out as to the image shown in Fig. 3 in dependency on the table of Figs. 2A, 2B and 2C; Fig. 5 is a view of an image after four brightness conversion operations have been carried out as to the image shown in Fig. 4 in dependency on the table of Figs. 2A, 2B and 2C; Fig. 6 is a view similar to that of Fig. 1, but shows a case in which the brightness is erroneously represented due to noise or the like; Figs. 7A, 7B and 7C are views of tables utilized for carrying out a reverse brightness conversion operation for temporarily increasing the size of the mass of the pixels; Fig. 8 is a plan view of a bonding wire which is bonded to a semiconductor chip, for example; Fig. 9 is a sectional view taken along the line IX-IX of Fig. 8; and Fig. 10 is a plan view showing a condition of an image of a bonding ball of the bonding wire having an irregular contour. DESCRIPTION OF THE PREFERRED EMBODIMENT In advance of the detailed disclosure of the present invention, the conventional technology of the inspection method will be first described with reference to Figs. 8 to 10 for a better understanding of the present invention. Referring to Figs. 8 and 9, when a semiconductor device is manufactured, one end of a bonding wire 3 is bonded to a surface of a bonding pad (electrode) 2 of the semiconductor chip 1 and the other end of the bonding wire 3, as an external terminal, is bonded to an inner lead of a lead frame. In this manufacturing process, the one end of the bonding wire 3 is plastically deformed by means of melting to form a bonding ball 4 as a bonding portion and the thus formed bonding ball 4 is bonded to the surface of the bonding pad 2 by pressing same. In such a bonding process, it is necessary to inspect the size and the position of the bonding ball 4 to thereby evaluate the defectiveness or quality of the bonding portion. However, the conventional technology has the problems described hereinbefore. Further in the known art, to solve the above problems, there is provided a method in which the bonded portion of the bonding ball 4 is observed as an image by an image pickup device and an area of the bonded portion in the image is measured, the measured value being then compared with a reference value thereby to evaluate the defectiveness or quality of the bonded portion. This method, however, also involves the problem described hereinbefore. Furthermore, as shown in Figs. 8 and 9, in the inspection of the size of the bonding ball 4 as the bonding portion, when the bonding ball 4 is image picked up from the upper portion thereof, a part of the image of the bonding ball 4 is concealed by the bonding wire 3, which results in complicated and difficult preprocessing for specifying the outer contour of the bonding ball 4, as well as specifying the portion concealed by the bonding wire 3. Moreover, as shown in Fig. 10, in addition to the above problem, in the case where the contour 4a of the image of the bonding ball 4 is not clear and is not shown by a smooth line, the area of the bonding ball 4 is measured differently to a great extent repending on the selection of the contour lines. Namely, referring to Fig. 10, the area largely differs by selecting either one of the contour lines 4b or 4c, and particularly, with respect to the image picked up with low magnitude, the measuring, i.e. inspection, performance will be degraded. The difficulty of specifying the contour line will result in a difficulty of the exact positioning of the bonded portion by the inspection of the central portion thereof. The embodiment of the present invention conceived to solve or substantially eliminate the defects or problems encountered in the prior art described hereinabove will be described hereunder with reference to Figs. 1 to 7. The embodiment is for the inspection of the size and the position of the bonding ball 4 as a bonding portion to be inspected shown in Figs. 8 and 9. According to the present invention, the bonding ball 4, shown in Fig. 8 or 9, is first image picked up as a portion to be bonded and the bonding ball 4 is extracted as a mass of pixels, each having a predetermined brightness, from the thus picked up image. The image of Fig. 1 is obtained by binarizing the mass of pixels, or distinguishing a portion having a predetermined brightness [1] of the bonding ball 4 and the bonding wire 3 bonded thereto from the other portion having a brightness [0]. In Fig. 1, checkers represent quantized grids. The image pickup operation is performed with a relatively low magnification. A brightness conversion operation, a so-called "logical filtering", to be effected to the image of Fig. 1 will be explained hereunder. This operation includes a step in which, with respect to the binarized image, in combination of brightnesses ([1] or [0]) of nine pixels consisting of a certain central one pixel and other eight pixels surrounding the one pixel, a new brightness of the central pixel is determined and such step is then performed with the whole pixels. The determination, i .e. conversion, of the brightness of the central pixel due to the combination of these nine pixels is performed on the basis of a table prepared preliminarily independently as shown in Fig. 2. For example, when a new brightness of the pixel 1 ○ shown in Fig. 2A is determined, the determination of the brightness of the pixel 1 ○ is performed on the combination of the pixel 1 ○ and the other eight pixels 2 ○ to 9 ○ surrounding the pixel 1 ○. For this purpose, a table such as shown in Fig. 2B is preliminarily prepared. The table of Fig. 2B represents the fact that, with the total of 512 combinations of these pixels 1 ○ to 9 ○ (i.e. 512= 2⁹), when the brightnesses of these pixels are all values of [1], a new brightness of the pixel 1 ○ is made to [1] and, on the other hand, when the pixels includes even one pixel having a brightness [0], the new brightness of the pixel 1 ○ is made to [0]. This operation is repeated, for the image shown in Fig. 1, to determine and then convert the new brightnesses of the respective pixels by shifting a 3 × 3 pixel area with respect to each one pixel so that each of the whole pixels is in accordance with the pixel 1 ○ positioned at the central portion as shown in Fig. 2A. According to these repeated operations, an image shown in Fig. 3 will be obtained, in which the brightness [0] is omitted. When such brightness conversion operation is performed with respect to the image of Fig. 3 by one further stage, an image such as shown in Fig. 4 will be obtained. When this brightness conversion operation is repeated subsequently, an image corresponding to the bonding ball 4 having the brightness [1] is extinguished in dependency on the number of times of the "logical filtering" operations in accordance with the size of the bonding ball 4 without being influenced with a portion corresponding to the bonding wire 3 shown in the original image of Fig. 1 and the irregularity of the contour 4a of the bonding ball 4. Namely, when additional four brightness conversion operations are performed with respect to the image shown in Fig. 4, an image shown in Fig. 5 will be obtained, and when further one brightness conversion operation is performed with respect to the image of Fig. 5, the image of the bonding ball 4 having the brightness [1] is extinguished. As can be understood from the above disclosure, the number of times of the brightness conversion operations constitutes the criterion for representing the size of the bonding ball 4, i.e. the bonding portion of the bonding wire 3. Accordingly, the defectiveness or quality of the bonded portion will be evaluated by preliminarily setting an allowable range (number of times) of these brightness conversion operations. In other words, supposing that the reference number of stages or steps of the operation including the allowable minimum number of stages and the allowable maximum number of stages is preliminarily determined, the bonded portion is evaluated to be "good (not defective)" when the image having the brightness [1] is extinguished within the reference number of operation times, and also evaluated to be "bad (defective)" when not extinguished. Furthermore, there may be a case, as shown in Fig. 6, wherein an area having the brightness [0] is generated within an image area having the brightness [1] extracted from the original image due to noise, for example. In such a case, if the brightness conversion operation is repeated subsequently, the area having the brightness [0] due to noise is gradually enlarged, and in comparison with the image having no area of brightness [0] as essentially shown in Fig. 1, the image area of the brightness [1] of the bonding ball 4, i.e. the bonded portion, may be extinguished by a lesser number of times of the brightness conversion operations, resulting in erroneous evaluation of the quality of the bonded portion to be defective. In order to obviate such evaluation error, another table such as shown in Figs. 7A, 7B and 7C may be prepared. The image is once enlarged by utilizing the tables of Figs. 7A, 7B and 7C and the brightness conversion operation is thereafter carried out by utilizing the table shown in Figs. 2A, 2B and 2C. Namely, according to the tables of Figs. 7A, 7B and 7C, only in a case where all the brightnesses of the pixels 1 ○ to 9 ○ are made [0], the brightness of the pixel 1 ○ is made [0], and in other cases, the brightness of the pixel 1 ○ is made [1]. The brightness conversion operation of Figs. 7A, 7B and 7C is reverse to the brightness conversion operation of Figs. 2A, 2B and 2C. According to the manner described above, the area A having the brightness [0] shown in Fig. 6 is amended to [1] to thereby prevent the influence of noise, for example, thus being capable of performing a correct treatment thereafter. In this case, although the area of the contour portion having the brightness [1] is increased, this problem can be solved by considering the extra number of times of the brightness conversion operations. Further, an image constituted by a pixel having the brightness [1] just before the extinction such as shown in Fig. 5 is converged to the central portion of the bonding ball 4, so that the center of this image can be deemed the central portion of the bonded portion at a time when the position of the bonded portion is measured, whereby the position of the bonded portion can be easily inspected at the same time as the inspection of the quality of the bonded portion. According to the present embodiments, the inspections of the quality and the position of the bonded portion can be made possible during the same procedure irrespective of the extending direction of the bonding wire 3 and the irregular shape of the contour of the extracted pixel of the bonding ball 4 (bonded portion), resulting in a shortage of the inspection time and an improvement of the inspection performance, and, hence, the inspection device is itself made compact. In the described embodiment, for the determination of the brightness of a certain pixel, the combination of nine pixels including this one central pixel and other eight pixels surrounding the same is used, the new brightness of the central pixel is determined, but in a modification, the brightness of the central pixel may be determined based on the combination of the central pixel and other four pixels positioned in the upper and lower and left and right portions of the central pixels. Reference signs in the claims are intended for better understanding and shall not limit the scope.

Claims: A method of inspecting a joint portion (4) of a first member (3) which is plastically deformed and joined to a second member (2) by utilizing an image pickup device, comprising the steps of: image picking up the joint portion (4) plastically deformed; extracting the joint portion (4) as a mass of pixels each having predetermined brightness from the image picked up by the image pickup device; characterized by the steps of decreasing the size of the mass of the extracted pixels by carrying out, by at least one step, logical brightness conversion operation on the basis of the relationship in brightness of one of the pixels with respect to other pixels which surround the one of the pixels; repeating the brightness conversion operation until the mass of the extracted pixels is extinguished; detecting the size of the joint portion (4) in dependency on the number of times of the brightness conversion operations carried out until the mass of the pixels is extinguished to thereby evaluate the quality of the joint portion (4); and/or detecting a center of the mass of the pixels as a center of the joint portion (4) just before a time when the mass thereof is extinguished to thereby inspect a position of the joint portion (4). A method according to claim 1, wherein the brightness of the pixels are represented by binarized values. A method according to claim 1, wherein the logical brightness conversion operation is carried out by logical filtering operation. A method according to claim 1, wherein the evaluation of the quality of the joint portion (4) is made in accordance with a preliminary determined allowable reference number of times of the brightness conversion operations. A method according to claim 1, wherein the joint portion (4) of the first member (3) is bonded to the second member (2). A method according to claim 5, wherein the first member is a bonding wire (3) and the second member is a semiconductor chip (1). A method according to claim 1, further comprising a step of temporarily increasing the size of the mass of the pixels, before the step of decreasing the size of the mass of the pixels, by a reverse logical brightness conversion operation to that of the decreasing step.

Inventors: YAMANAKA KAZUYUKI, YAMANAKA, KAZUYUKI
Applicants: TOSHIBA KK, KABUSHIKI KAISHA TOSHIBA
Application year: 1991
Main classification: G01R31
IPC codes: G01B11, G01R31, H01L21