Source: http://www.google.com/patents/US6064756?dq=6,891,551
Timestamp: 2016-05-31 08:55:39
Document Index: 287531018

Matched Legal Cases: ['art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 402', 'art 402', 'art 402', 'art 402', 'art 402', 'art 70', 'art 70', 'art 70']

Patent US6064756 - Apparatus for three dimensional inspection of electronic components - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA three dimensional inspection apparatus for ball array devices, where the ball array device is positioned in a fixed optical system. An illumination apparatus is positioned for illuminating the ball array device. A first camera is disposed in a fixed focus position relative to the ball array device...http://www.google.com/patents/US6064756?utm_source=gb-gplus-sharePatent US6064756 - Apparatus for three dimensional inspection of electronic componentsAdvanced Patent SearchPublication numberUS6064756 APublication typeGrantApplication numberUS 09/321,838Publication dateMay 16, 2000Filing dateMay 28, 1999Priority dateJan 16, 1998Fee statusLapsedAlso published asCN1287643A, EP1046127A1, EP1046127A4, US6064757, US6072898, US6862365, US7508974, US20050189657, WO1999036881A1Publication number09321838, 321838, US 6064756 A, US 6064756A, US-A-6064756, US6064756 A, US6064756AInventorsElwin M. Beaty, David P. MorkOriginal AssigneeElwin M. BeatyExport CitationBiBTeX, EndNote, RefManPatent Citations (31), Non-Patent Citations (1), Referenced by (34), Classifications (28), Legal Events (10) External Links: USPTO, USPTO Assignment, EspacenetApparatus for three dimensional inspection of electronic components
US 6064756 AAbstract
A three dimensional inspection apparatus for ball array devices, where the ball array device is positioned in a fixed optical system. An illumination apparatus is positioned for illuminating the ball array device. A first camera is disposed in a fixed focus position relative to the ball array device for taking a first image of the ball array device to obtain a characteristic circular doughnut shape image from a ball. A second camera is disposed in a fixed focus position relative to the ball array device for taking a second image of the ball array device to obtain a top surface image of the ball. A processor applies triangulation calculations on related measurements of the first image and the second image to calculate a three dimensional position of the ball with reference to a pre-calculated calibration plane.
1. A three dimensional inspection apparatus for ball array devices having a plurality of balls, wherein the ball array device is positioned in a fixed optical system, the apparatus comprising:a) an illumination apparatus positioned for illuminating the ball array device; b) a first camera disposed in a fixed focus position relative to the ball array device for taking a first image of the ball array device to obtain a characteristic circular doughnut shape image from at least one ball; c) a second camera disposed in a fixed focus position relative to the ball array device for taking a second image of the ball array device to obtain a side view image of the at least one ball; and d) a processor, coupled to receive the first image and the second image, that applies triangulation calculations on related measurements of the first image and the second image to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane. 2. The three dimensional inspection apparatus of claim 1 wherein the second image comprises a segment having a crescent shape.
3. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.
4. The three dimensional inspection apparatus of claim 3 wherein the triangulation calculations are based on determining a center of the ball in the first image and determining a ball top location in the second image.
5. The three dimensional inspection apparatus of claim 1 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
6. The three dimensional inspection apparatus of claim 1 wherein a mirror reflects light between the ball array device and the second camera.
7. The three dimensional inspection apparatus of claim 1 wherein the second image is obtained at a low angle of view.
8. The three dimensional inspection apparatus of claim 1 wherein the first camera and the second camera are fixed at different angles relative to the calibration plane.
9. The three dimensional inspection apparatus of claim 1 wherein the first camera and the second camera each comprise a charged coupled device array.
10. The three dimensional inspection apparatus of claim 1 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.
11. The three dimensional inspection apparatus of claim 1 wherein the illumination apparatus further comprises a diffuser.
12. The three dimensional inspection apparatus of claim 1 wherein the ball array devices comprise ball grid array devices.
13. The three dimensional inspection apparatus of claim 1 wherein the ball array devices comprise bump on wafer devices.
14. A three dimensional inspection apparatus for ball array devices having a plurality of balls, the apparatus comprising:(a) an illuminator positioned to produce reflections from the ball array device; (b) a first camera, for taking a first image of the ball array device, disposed in a first fixed position relative to the ball array device to obtain a circular doughnut shape view of the ball array device; (c) a second camera, for taking a second image of the ball array device, disposed in a second fixed position non-parallel to the first fixed position to obtain a side view of the ball array device; (d) a first frame grabber coupled to the first camera to acquire first image; (e) a second frame grabber coupled to the second camera to acquire the second image; and (f) a processor, coupled to the first and second frame grabbers, that applies triangulation calculations to related measurements on the first image and the second image so as to calculate a three dimensional position of at least one ball with reference to a pre-calculated calibration plane. 15. The three dimensional inspection apparatus of claim 14 wherein the second image comprises a segment having a crescent shape.
16. The three dimensional inspection apparatus of claim 14 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.
17. The three dimensional inspection apparatus of claim 14 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
18. The three dimensional inspection apparatus of claim 14 wherein a mirror is positioned to reflect light between the ball array device and the second camera.
19. The three dimensional inspection apparatus of claim 14 wherein the first camera and the second camera each comprise a charged coupled device array.
20. The three dimensional inspection apparatus of claim 14 wherein the illuminator further comprises a diffuser.
21. The three dimensional inspection apparatus of claim 14 wherein the ball array devices comprise ball grid array devices.
22. The three dimensional inspection apparatus of claim 14 wherein the ball array devices comprise bump on wafer devices.
23. The three dimensional inspection apparatus of claim 14 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.
24. A three dimensional inspection apparatus for ball array devices having a plurality of balls, the apparatus comprising:(a) an illuminator disposed to illuminate a ball array device; (b) a first camera for taking a first image of the ball array device, the first camera disposed in a first fixed position relative to the ball array device to obtain a circular doughnut shape view of the ball array device, wherein the first camera includes a charged coupled device array; (c) a second camera disposed for taking a second image of the ball array device, the second camera disposed in a second fixed position non-parallel to the first fixed position to obtain a side view of he ball array device, wherein a fixed mirror is interposed to reflect light between the ball array device and the second camera, and wherein the second camera includes a charged coupled device array; (d) a first image acquisition apparatus coupled to the first camera to acquire first image, and a second image acquisition apparatus coupled to the second camera to acquire the second image; and (e) a processor coupled to receive information from the first and second image acquisition apparatus, where the processor operates to make measurements from the first image and the second image so as to calculate a three dimensional position of at least one ball using a triangulation method with reference to a pre-calculated calibration plane, wherein the calibration plane comprises a coordinate system having X, Y and Z axes, and wherein an X measurement value is proportional to a Z measurement value. 25. The three dimensional inspection apparatus of claim 24 wherein the second image comprises a segment having a crescent shape.
26. The three dimensional inspection apparatus of claim 24 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
27. The three dimensional inspection apparatus of claim 24 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.
28. The three dimensional inspection apparatus of claim 24 wherein the illuminator further comprises a diffuser.
29. The three dimensional inspection apparatus of claim 24 wherein the ball array devices comprise ball grid array devices.
30. The three dimensional inspection apparatus of claim 24 wherein the ball array devices comprise bump on wafer devices.
This application is a divisional of pending U.S. application Ser. No. 09/008,243 filed Jan. 16, 1998, entitled "Method and Apparatus for Three Dimensional Inspection of Electronic Components," incorporated by reference herein.
Structured light systems project precise bands of light onto the part to be inspected. The deviation of the light band from a straight line is proportional to the distance from a reference surface. The light bands are moved across the part, or alternately the part is moved with respect to the light bands, and successive images are acquired. The maximum deviation of the light band indicates the maximum height of a ball. This method suffers from specular reflections due to the highly focused nature of the light bands resulting in erroneous delta. This method further suffers from increased inspection times due to the number of images required.
The invention provides a calibration and part inspection method and apparatus for the inspection of BGA devices. The invention includes two cameras to image a precision pattern mask with dot patterns deposited on a transparent reticle to be inspected and provides information needed for calibration. A light source and overhead light reflective diffuser provide illumination that enhances the outline of the ball grid array. A first camera images the reticle precision pattern mask from directly below. An additional mirror or prism located below the bottom plane of the reticle reflects the reticle pattern mask from a side view, through prisms or reflective surfaces, into a second camera. A second additional mirror or prism located below the bottom plane of the reticle reflects the opposite side view of the reticle pattern mask through prisms or mirrors into a second camera. By imaging more than one dot pattern, the missing state values of the system can be resolved using a trigonometric solution. The reticle with the pattern mask is removed after calibration and a BGA to be inspected is placed with the balls facing downward, in such a manner as, to be imaged by the two cameras. The scene of the part can thus be triangulated and the dimensions of the BGA are determined.
The invention also provides a method for three dimensional inspection of a lead on a part mounted on a reticle. The method comprises the steps of: locating a first camera to receive an image of the lead; transmitting an image of the lead to a first frame grabber; providing fixed optical elements to obtain two side perspective views of the lead; locating a second camera to receive an image of the two side perspective views of the lead; transmitting the two side perspective views of the lead to a second frame grabber; operating a processor to send a command to the first frame grabber and second flame grabber to acquire images of pixel values from the first camera and the second camera; and processing the pixel values with the processor to obtain three dimensional data about the lead.
The invention also provides a method to inspect a ball grid array device comprising the steps of: locating a point on a world plane determined by a bottom view ray passing through a center of a ball on the ball grid array device; locating a side perspective view point on the world plane determined by a side perspective view ray intersecting a ball reference point on the ball and intersecting the bottom view ray at a virtual point where the side perspective view ray intersects the world plane at an angle determined by a reflection of the side perspective view ray off of a back surface of a prism where a value of the angle was determined during a calibration procedure; calculating a distance L1 as a difference between a first world point, defined by an intersection of the bottom view ray with a Z=0 world plane, and a second world point, defined by the intersection of the side perspective view ray and the Z=0 world plane, where a value Z is defined as a distance between a third world point and is related to L1 as follows: ##EQU1## wherein Z is computed based on the angle; computing an offset E as the difference between the virtual point defined by the intersection of the bottom view ray and the side perspective view ray and a crown of a ball at a crown point that is defined by the intersection of the bottom view ray with the crown of the ball, and can be calculated from a knowledge of the angle and ideal dimensions of the ball where a final value of Z for the ball is:
ZFinal =Z-E.
The invention also provides a method for finding a location and dimensions of a ball in a ball grid array from a bottom image comprising the steps of: defining a region of interest in the bottom image of an expected position of a ball where a width and a height of the region of interest are large enough to allow for positioning tolerances of the ball grid array for inspection; imaging the ball, wherein the ball is illuminated to allow a spherical shape of the ball to present a donut shaped image, wherein the region of interest includes a perimeter of the ball wherein the bottom image comprises camera pixels of higher grayscale values and where a center of the bottom image comprises camera pixels of lower grayscale value and wherein a remainder of the region of interest comprises camera pixels of lower grayscale values; finding an approximate center of the ball by finding an average position of pixels having pixel values that are greater than a predetermined threshold value; converting the region of lower grayscale pixel values to higher grayscale values using coordinates of the approximate center of the ball; and finding he center of the ball.
Refer now to FIG. 1A which shows the apparatus of the invention configured with a calibration reticle for use during calibration of the state values of the system. The apparatus obtains what is known as a bottom image 50 of the calibration reticle 20. To take the bottom image 50 the apparatus includes a camera 10 with a lens 11 and calibration reticle 20 with a calibration pattern 22 on the bottom surface. The calibration pattern 22 on the reticle 20 comprises precision dots 24. The camera 10 is located below the central part of the calibration reticle 20 to receive an image 50 described in conjunction with FIG. 1B. In one embodiment the camera 10 comprises an image sensor. The image sensor may be a charged coupled device array. The camera 10 is connected to a frame grabber board 12 to receive the image 50. The frame grabber board 12 provides an image data output to a processor 13 to perform a two dimensional calibration as described in conjunction with FIG. 2A. The processor 13 may store an image in memory 14. The apparatus of the invention obtains an image of a pair of side perspective views and includes using a camera 15 with a lens 16 and a calibration reticle 20. The camera 15 is located to receive an image 60, comprising a pair of side perspective views, described in conjunction with FIG. 1B. Fixed optical elements 30, 32 and 38 provide a first side perspective view and fixed optical elements 34, 36, 38 for a second side perspective view. The fixed optical elements 30, 32, 34, 36 and 38 may be mirrors or prisms. As will be appreciated by those skilled in the art additional optical elements may be incorporated. The camera 15 is connected to a frame grabber board 17 to receive the image 60. The frame grabber board 17 provides an image data output to a processor 13 to perform a two dimensional inspection as described in conjunction with FIG. 2B. The processor 13 may store an image in memory 14. In one embodiment of the invention, the apparatus may contain a nonlinear optical element 39 to magnify the side perspective image 60 in one dimension as shown in FIG. 8A. In another embodiment of the invention optical element 38 may be a nonlinear element. The nonlinear optical elements 38 and 39 may be a curved mirror or a lens.
FIG. 2A shows a flow diagram for the calibration of the bottom view of the system. The method starts in step 1101 by providing a transparent reticle 20 having a bottom surface containing a dot pattern 22, comprising precision dots 24 of known dimensions and spacing. The method in step 102 provides a camera 10 located beneath the transparent reticle 20 to receive an image 50. In step 103 the processor 13 sends a command to a frame grabber 12 to acquire an image 50, comprising pixel values from the camera 10. The method then proceeds to step 104 and processes the pixel values with a processor 13.
FIG. 2C shows a flow diagram for the calibration of the side perspective views of the system. The method starts in step 121 by providing a transparent reticle 20 having a bottom surface containing a dot pattern 22, comprising precision dots 24 of known dimensions and spacing. The method in step 122 provides fixed optical elements 30, 32, 34, 36 and 38 to reflect two perspective images of the precision dot pattern 22 into camera 15. The method in step 123 provides a camera 15 located to receive an image 60. In step 124 the processor 13 sends a command to a frame grabber 12 to acquire an image 60, comprising pixel values from the camera 15. The method then proceeds to step 125 and processes the pixel values with processor 13.
FIG. 2E shows the relationship of a side perspective angle to the ratio of the perspective dimension to the non-perspective dimension. Ray 171, 172, and 173 defining point 181 is parallel to ray 174, 175 and 176 defining point 182. Point 181 and point 182 lie on a plane 170 parallel to a plane 180. The intersection of ray 175 and ray 176 define point 186. The intersection of ray 176 and ray 172 define point 184. The intersection of ray 173 and ray 172 define point 187. The intersection of ray 174 and ray 172 define point 183. The reflecting plane 179 intersecting plane 180 at an angle D is defined by ray 172 and ray 175 and the law of reflectance. Ray 172 and ray 175 intersect plane 170 at an angle 177. Referring to FIG. 2E it can be shown: ##EQU2## Substituting: ##EQU3##
FIG. 2F shows a bottom view and a side perspective view of precision dots used in the method for determining a side perspective view angle 177 as shown in FIG. 2E of the system. A bottom view image 200 comprising precision dots 201, 202 and 203 of known spacing and dimensions from the calibration method described earlier can be used to provide a reference for determination of a side perspective view angle 177. The value DH and DB are known from the bottom view calibration. A side perspective view image 210 comprising precision dots 211, 212 and 213, corresponding to bottom view dots 201, 202 and 203 respectively, of known spacing and dimensions DS and Dh from the calibration method described earlier, can be used to determine the side view perspective angle. The ratio of (Dh /DH) from the bottom image 200 and the side perspective image 210 can be used in the bottom view to calibrate DB in the same units as the side perspective view as follows:
DBcal =DB (Dh /DH)
Substituting into the equation for the side perspective view angle 177 described earlier yields: ##EQU4##
FIG. 3A shows the apparatus of the invention for a three dimensional inspection of the balls of a ball grid array. The apparatus of the invention includes a part 70 to be inspected. The apparatus further includes a camera 10 with a lens 11, located below the central area of part 70, to receive a bottom image 80, described in conjunction with FIG. 3B, of part 70. The camera 10 is connected to a frame grabber board 12 to receives the image 80. The frame grabber board 12 provides an image data output to a processor 13 to perform a two dimensional inspection as described in conjunction with FIG. 3A. The processor 13 may store an image in memory 14. The apparatus of the invention obtains an image of a pair of side perspective views with a camera 15 and a lens 16. The camera 15 is located to receive an image 90, comprising a pair of side perspective views, described in conjunction with FIG. 3B and utilizing fixed optical elements 30, 32 and 38 for a first side perspective view and fixed optical elements 34, 36 and 38 for a second side perspective view. In one embodiment of the invention, the apparatus may contain a nonlinear optical element 39 to magnify the side perspective image 60 in one dimension as shown in FIG. 8B. In another embodiment of the invention optical element 38 may be the nonlinear element. The fixed optical elements 30, 32, 34, 36 and 38 may be mirrors or prisms. As will be appreciated by those skilled in the art additional optical elements may be incorporated without deviating from the spirit and scope of the invention. The camera 15 is connected to a frame grabber board 17 to receive the image 90. The frame grabber board 17 provides an image data output to a processor 13 to calculate the Z position of the balls, described in conjunction with FIG. 3B. The processor 13 may store an image in memory 14.
By using the measurement values from the part definition file and the placement values determined in step 154, the processor calculates an expected position for each ball of the part for the bottom view contained in image 801. The processor employs a search procedure on the image data to locate the balls 81 in image 30. The processor then determines each ball's center location and diameter in pixel values using grayscale blob techniques as described in FIG. 7A. The results are stored in memory 14.
In step 161 these part values are compared to the ideal values defined in the part file to calculate the deviation of each ball center from its ideal location. In one examples embodiment of the invention the deviation values may include ball diameter in several orientations with respect to the X and Y part coordinates, ball center in the X direction, Y direction and radial direction, ball pitch in the X direction and Y direction and missing and deformed balls. The Z world data can be used to define a seating plane, using well known mathematical formulas, from which the Z dimension of the balls with respect to the seating plane can be calculated. Those skilled in the art will recognize that there are several possible definitions for seating planes from the data that may be used without deviating from the spirit and scope of the invention.
FIGS. 6A and 6B show an example ball of a ball grid array and associated geometry use in a method of the invention for determining the Z position of the ball. The method determines the Z position of a ball with respect to the world coordinates defined during calibration. Using parameters determined from the calibration procedure as shown in FIGS. 2B and 2D to define a world coordinate system for the bottom view and the two side perspective views, comprising world coordinate plane 250 with world coordinate origin 251 and world coordinate axis X 252, Y 253 and Z 254 shown in FIG. 6A, and a pair of images 80 and 90 as shown in FIG. 3B, the processor computes a three dimensional location.
Now refer to FIG. 6B. The distance L1 is calculated by the processor as the difference between world point 258 defined by the intersection of ray 255 with the Z=0 world plane 250, and world point 260, defined by the intersection of ray 256 and the Z=0 world plane 250. The value Z is defined as the distance between world point 261 and 258 and is related to L1 as follows: ##EQU5##
ZFinal =Z-F
FIG. 7A shows one example of an image used in the grayscale blob method of the invention. The image processing method finds the location and dimensions of a ball 71 from a bottom image 80. From the expected position of a ball 71, a region of interest in image 80 is defined as (X1, Y1) by (X2,Y2). The width and height of the region of interest are large enough to allow for positioning tolerances of part 70 for inspection. Due to the design of the lighting for the bottom view, the spherical shape of balls 71 of part 70 present a donut shaped image where the region 281, including the perimeter of the ball 71, comprises camera pixels of higher grayscale values and where the central region 282 comprises camera pixels of lower grayscale values. The remainder 283 of the region of interest 280 comprises camera pixels of lower grayscale values.
The C language function "FindBlobCenter", as described below, is called to find the approximate center of the ball 71 by finding the average position of pixels that are greater than a known threshold value. Using the coordinates of the approximate center of the ball 71, the region 282 of lower grayscale pixel values can be converted to higher grayscale values by calling the C language function "FillBallCenter", as described below. The exact center of the ball 71 can be found by calling the C language function "FindBallCenter" which also returns an X world and Y world coordinate. The diameter of the ball 71 can be calculated by the C language function, "Radius=sqrt(Area/3.14)". The area used in the diameter calculation comprises the sum of pixels in region 281 and 282.
FIG. 7B shows one example of an image used with the method of the invention to perform a subpixel measurement of the ball reference point. The method of the invention finds a reference point on a ball 71 in an image 90 of a side perspective view as shown in FIG. 3B. From the expected position of a ball 71, a region of interest 290 in image 80 is defined as (X3, Y3) by (X4,Y4). The width and height of the region of interest are large enough to allow for positioning tolerances of part 70 for inspection. Due to the design of the lighting for a side perspective view, the spherical shape of balls 71 of part 70 present a crescent shaped image 291 comprising camera pixels of higher grayscale values and where the remainder 293 of the region of interest 290 comprises camera pixels of lower grayscale values.
The C language function "FindBlobCenter" is called to compute the approximate center of the crescent image 291 by finding the average position of pixels that are greater than a known threshold value. Using the coordinates of the approximate center of the crescent image 291, the C language function "FindCrescentTop" is called to determine the camera pixel, or seed pixel 292 representing the highest edge on the top of the crescent. The camera pixel coordinates of the seed pixel are used as the coordinates of a region of interest for determining the subpixel location of the side perspective ball reference point.
______________________________________/////////////////////////////////////////////////////////////////////////// FindBlobcenter - finds the X,Y center of the pixels thathave a value greater than THRESHOLD in the region (x1,y1) to(x2, y2)///////////////////////////////////////////////////////////////////////////////long FindBlobCenter (int x1, int y1, int x2, int y2,      double* pX, double* pY)int x,y;long Found = 0;long SumX = 0;long SumY = 0;for (x = x1; x &lt;= x2; x++){for (y = y1; y &lt;= y2; y++){if (Pixel [x] [y] &gt; THRESHOLD){   SumX += x;   SumY += y;   Found ++;}}}if (Found &gt; 0){*pX = (double)SumX / (double)Found;*pY = (double)SumY / (double)Found;}return Found;}/////////////////////////////////////////////////////////////////////////// FillBallCenter - fills the center of the BGA "donut"/////////////////////////////////////////////////////////////////////////void FillBallCenter (double CenterX, double CenterY, doubleDiameter){int x,y;int x1 = (int) (CenterX - Diamter / 4.0);int x2 = (int) (CenterX + Diamter / 4.0);int y1 = (int) (CenterY - Diamter / 4.0);int y2 = (int) (CenterY + Diamter / 4.0);for (x = x1; x &lt;= x2; x++){for (y = y1; y &lt;= y2 ; y++){Pixel[x] [y] = 255;}}}/////////////////////////////////////////////////////////////////////////// FindBallCenter - finds the X,Y center of the a BGA ball//         using the grayscale values/////////////////////////////////////////////////////////////////////////long FindBallCenter (int x1, int y1, int x2, int y2,    double* pX, double* pY, double* pRadius){int x,y;long Found = 0;long Total = 0;long SumX = 0;long SumY = 0;for (x = x1; x &lt;= x2; x++){for (y = y1; y &lt;= y2; y++){if (Pixel [x] [y] &gt; THRESHOLD){   SumX += x*Pixel [x] [y];   SumY += y*Pixel [x] [y];   Total += Pixel [x] [y];   Found ++;}}}if (Found &gt; 0){*pX        = (double)SumX / (double)Total;*pY        = (double)SumY / (double)Total;*pRadius   = sqrt ((double) Found / 3.14159279);}return Found;}/////////////////////////////////////////////////////////////////////////// FindCresentTop - finds the X,Y top position of a BGAcresent/////////////////////////////////////////////////////////////////////////void FindCresentTop(int CenterX, int CenterY, int Diameter,    int* pX, int* pY){int x,y,Edge,Max,TopX,TOPY;int x1 = CenterX - Diamter / 2;int x2 = CenterX + Diamter / 2;int y1 = CenterY - Diamter / 2;int y2 = CenterY;*pY = 9999;for (x = x1; x &lt;= x2; x++){Max = -9999;for (y = y1; y &lt;= y2; y++){Edge = Pixel [x] [y] - Pixel [x] [y - 1];if (Edge &gt; Max){   Max = Edge;   TopY = y;   TopX = x;}}if (TopY &lt; *pY){*pX = TopX;*pY = TopY;}}______________________________________
FIG. 8A shows a side perspective image of the calibration pattern magnified in cone dimension. FIG. 8A shows a side perspective image 30 of a reticle calibration pattern where the space 303 between dot 301 and dot 302 is magnified, increasing the number of lower value grayscale pixels when compared to a non magnified image.
Now refer to FIG. 10A. The processor locates a point 709 on the world plane 700 determined by a bottom view ray 705 passing through the center 718 of a ball 717. The processor locates a first side perspective view point 711 on the world plane 700 determined by a side view ray 706 intersecting a ball reference point 710 on ball 717 and intersecting the bottom view ray 705 at a virtual point 714. Ray 706 intersects the world plane 700 at an angle 715 determined by the reflection of ray 706 off of the back surface of prism 30. The value of angle 715 was determined during the calibration procedure. The processor locates a second side perspective view point 713 on the world plane 700 determined by a side view ray 707 intersecting a ball reference point 712 on ball 717 and intersecting the bottom view ray 705 at a virtual point 7L8. Ray 707 intersects the world plane 700 at an angle 716 determined by the reflection of ray 707 off of the back surface of prism 34. The value of angle 716 was determined during the calibration procedure.
Now refer to FIG. 10B. The distance L1 is calculated by the processor as the distance between world point 709 and world point 711. The distance L2 is calculated by the processor as the distance between world point 713 and world point 709. The value Z1 is defined as the distance between world point 714 ad 709 and is related to L1 as follows: ##EQU6##
The value Z2 is defined as the distance between world point 718 and 709 and is related to L2 as follows: ##EQU7##
FIG. 1B shows an example calibration pattern and example images of a calibration pattern acquired by the system, utilizing a single side perspective view, of the invention. FIG. 11B shows an example image 50 from camera 10 and an example image 64 from camera 15 acquired by the system. The image 50 showing dot, 52 acquired by camera 10 includes a bottom view of the dot pattern 22, containing precision dots 24 of known dimensions and spacing, located on the bottom surface of the calibration reticle 20. The image 64 shows a side perspective view of the dot pattern 22, containing precision dots 24 of known dimensions and spacing, located on the bottom surface of the calibration reticle 20. A side perspective view in image 64 contains images of dots 65 and is obtained by the reflection of the image of the calibration reticle cot pattern 22 off of fixed optical element 40, passing through nonlinear element 42 and into camera 15.
The determination of the state values for the side perspective view is identical to the method shown in FIG. 2D except the fixed optical elements may be different and there is only one side perspective view. The principles and is relationships shown in FIG. 2E and FIG. 2F apply.
FIG. 12B shows an example ball grid array and example images of the ball grid array for three dimensional inspection, utilizing a single side perspective view. FIG. 12B shows an example image 80 from camera 10 and an example image 94 from camera 15 acquired by the system. The image 80 shows the bottom view of the balls 71 located on the bottom surface of a part 70. The image 91 shows a side perspective view of the balls 71 located on past 70. The side perspective view in image 94 contains images of balls 95 and is obtained by the reflection of the image of the part 70 off of fixed optical element 40 and passing through the nonlinear fixed element 42 into camera 15.
FIG. 13 shows the apparatus of the invention for the three dimensional inspection of ball grid array devices, gullwing devices and J lead devices. The apparatus described in FIG. 13 allows the inspection of BGA, gullwing and J lead devices all on the same system. The apparatus includes a part 402 to be inspected located over the central area of a transparent reticle 400 with prisms 401 glued to the top surface to receive side perspective views of part 402. A gullwing and J lead inspection device 21 may be integrated into the ball grid array inspection device. One example embodiment of such a gullwing and J lead inspection device is the "UltraVim" scanner from Scanner Technologies of Minnetonka, Minn. The apparatus further includes a camera 10A with a lens 11A, located below the central area of part 402 and reticle 400 to receive a bottom view and side perspective views of part 402. The camera 10A is connected to a frame grabber board 12A to receive an image. The frame grabber board 12A provides an image data output to a processor 13A to perform a three dimensional inspection of part 402. The processor 13A may store an image in memory 14A. These components comprise the hardware of the gullwing and J lead inspection device 21 and are shared by the ball grid array inspection device as described herein.
The UltraVim is described in U.S. patent application Ser. No. 08/850,473 entitled THREE DIMENSIONAL INSPECTION SYSTEM by Beauty et al., filed May 5, 1997 which is incorporated in its entirely by reference thereto.
Refer now to FIG. 14. In still another embodiment of the invention, the system may use three cameras to image directly the bottom view and two side perspective views as shown in FIG. 14. FIG. 14 snows the apparatus of the invention for a three dimensional inspection of the balls of a BGA. The apparatus of the invention includes a part 70, with balls 71 to be inspected. The apparatus further includes a camera 10 with a lens 11, located below the central area of part 70, to receive a bottom image 80, described in conjunction with FIG. 12B, of part 70. The camera 10 is connected to a frame grabber board 12 to receive the image 80. The frame grabber board 12 provides an image data output to a processor 13 to perform a two dimensional inspection as described in conjunction with FIG. 12B. The processor 13 may store an image in memory 14. The apparatus for an image of a first side perspective view includes a camera 15 with a lens 19. The camera 15 is located to receive an image 94, comprising a single side perspective view, described in conjunction with FIG. 12B and utilizing fixed optical element 38, to magnify the side perspective view in one dimension. The camera 15 is connected to a frame grabber board 17 to receive the image 94. The frame grabber board 17 provides an image data output to a processor 13 to calculate the Z position of the balls, described in conjunction with FIG. 12B. The processor 13 may store an image in memory 14. The apparatus for an image of a second side perspective view includes a camera 15 with a lens 19. The camera 15 is located to receive an image similar to 94, comprising a single side perspective view, described in conjunction with FIG. 12B and utilizing fixed optical element 38, to magnify the side perspective view in one dimension. The camera 15 is connected to a frame grabber board 17 to receive the image similar to 94. The frame grabber board 17 provides an image data output to a processor 13 to calculate the Z position of the balls, described in conjunction with FIG. 12B. The processor 13 may store an image in memory 14. In another embodiment, the nonlinear fixed optical element 38 may be missing. In still another embodiment of the invention, only one side perspective view may be utilized.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4521807 *Jul 13, 1982Jun 4, 1985W. R. Grace & Co.Optical inspection systemUS4677473 *Nov 7, 1985Jun 30, 1987Matsushita Electric Works, Ltd.Soldering inspection system and method thereforUS4731855 *Apr 8, 1985Mar 15, 1988Hitachi, Ltd.Pattern defect inspection apparatusUS5058178 *Dec 21, 1989Oct 15, 1991At&T Bell LaboratoriesMethod and apparatus for inspection of specular, three-dimensional featuresUS5245671 *May 2, 1989Sep 14, 1993Omron CorporationApparatus for inspecting printed circuit boards and the like, and method of operating sameUS5355221 *Oct 25, 1993Oct 11, 1994Wyko CorporationRough surface profiler and methodUS5420689 *Mar 1, 1993May 30, 1995Siu; BernardHigh speed illumination system for microelectronics inspectionUS5465152 *Jun 3, 1994Nov 7, 1995Robotic Vision Systems, Inc.Method for coplanarity inspection of package or substrate warpage for ball grid arrays, column arrays, and similar structuresUS5546189 *May 19, 1994Aug 13, 1996View Engineering, Inc.Triangulation-based 3D imaging and processing method and systemUS5574668 *Feb 22, 1995Nov 12, 1996Beaty; Elwin M.Apparatus and method for measuring ball grid arraysUS5574801 *Aug 12, 1994Nov 12, 1996Collet-Beillon; OlivierMethod of inspecting an array of solder ball connections of an integrated circuit moduleUS5581632 *May 2, 1994Dec 3, 1996Cognex CorporationMethod and apparatus for ball bond inspection systemUS5592562 *Sep 2, 1994Jan 7, 1997International Business Machines CorporationInspection system for cross-sectional imagingUS5600150 *Jun 7, 1995Feb 4, 1997Robotic Vision Systems, Inc.Method for obtaining three-dimensional data from semiconductor devices in a row/column array and control of manufacturing of same with data to eliminate manufacturing errorsUS5617209 *Apr 27, 1995Apr 1, 1997View Engineering, Inc.Method and system for triangulation-based, 3-D imaging utilizing an angled scaning beam of radiant energyUS5621530 *Apr 26, 1995Apr 15, 1997Texas Instruments IncorporatedApparatus and method for verifying the coplanarity of a ball grid arrayUS5648853 *May 18, 1995Jul 15, 1997Robotic Vision Systems, Inc.System for inspecting pin grid arraysUS5652658 *Oct 19, 1993Jul 29, 1997View Engineering, Inc.Grid array inspection system and methodUS5654800 *Jul 29, 1996Aug 5, 1997General Scanning Inc,Triangulation-based 3D imaging and processing method and systemUS5734475 *Oct 15, 1996Mar 31, 1998Ceridian CorporationProcess of measuring coplanarity of circuit pads and/or grid arraysUS5801966 *Jul 24, 1995Sep 1, 1998Cognex CorporationMachine vision methods and articles of manufacture for determination of convex hull and convex hull angleUS5812268 *May 5, 1997Sep 22, 1998General Scanning Inc.Grid array inspection system and methodUS5812269 *May 9, 1997Sep 22, 1998General Scanning, Inc.Triangulation-based 3-D imaging and processing method and systemUS5815275 *Mar 27, 1997Sep 29, 1998General Scanning, Inc.Method and system for triangulation-based, 3-D imaging utilizing an angled scanning beam of radiant energyUS5818061 *Sep 25, 1995Oct 6, 1998Robotic Vision Systems, Inc.Apparatus and method for obtaining three-dimensional data from objects in a contiguous arrayUS5828449 *Jul 25, 1997Oct 27, 1998Acuity Imaging, LlcRing illumination reflective elements on a generally planar surfaceUS5859698 *May 7, 1997Jan 12, 1999Nikon CorporationMethod and apparatus for macro defect detection using scattered lightUS5859924 *Jul 12, 1996Jan 12, 1999Robotic Vision Systems, Inc.Method and system for measuring object featuresUS5870489 *Aug 13, 1997Feb 9, 1999Kabushiki Kaisha ShinkawaBall detection method and apparatus for wire-bonded partsUS5926557 *Mar 26, 1997Jul 20, 1999Acuity Imaging, LlcInspection methodUS5943125 *Feb 26, 1997Aug 24, 1999Acuity Imaging, LlcRing illumination apparatus for illuminating reflective elements on a generally planar surface* Cited by examinerNon-Patent CitationsReference1 *ICOS Vision Systems Product literature on BGA 3D InspectionSystem.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS6141040 *Dec 19, 1997Oct 31, 2000Agilent Technologies, Inc.Measurement and inspection of leads on integrated circuit packagesUS6751344 *May 24, 2000Jun 15, 2004Champion Orthotic Investments, Inc.Enhanced projector system for machine visionUS6778282 *Mar 1, 2000Aug 17, 2004Icos Vision Systems N.V.Measuring positions of coplanarity of contract elements of an electronic component with a flat illumination and two camerasUS6915006Apr 27, 2001Jul 5, 2005Elwin M. BeatyMethod and apparatus for three dimensional inspection of electronic componentsUS6915007 *Apr 27, 2001Jul 5, 2005Elwin M. BeatyMethod and apparatus for three dimensional inspection of electronic componentsUS7231081 *Dec 18, 2002Jun 12, 2007Applied Precision, LlcStereoscopic three-dimensional metrology system and methodUS7403648 *Nov 25, 2003Jul 22, 2008Mori Seiki Co., Ltd.Apparatus for generating three-dimensional model dataUS7423743Jan 2, 2002Sep 9, 2008Icos Vision Systems NvMethod and an apparatus for measuring positions of contact elements of an electronic componentUS7508974 *Feb 28, 2005Mar 24, 2009Scanner Technologies CorporationElectronic component products and method of manufacturing electronic component productsUS7634128 *May 21, 2007Dec 15, 2009Rudolph Technologies, Inc.Stereoscopic three-dimensional metrology system and methodUS7809158 *May 2, 2006Oct 5, 2010Siemens Industry, Inc.Method and apparatus for detecting doubles in a singulated stream of flat articlesUS8176624Nov 23, 2009May 15, 2012International Business Machines CorporationOptical alignment module utilizing transparent reticle to facilitate tool calibration during high temperature processUS8233041 *Jun 16, 2006Jul 31, 2012Omron CorporationImage processing device and image processing method for performing three dimensional measurementsUS8601677Mar 2, 2012Dec 10, 2013International Business Machines CorporationOptical alignment module utilizing transparent reticle to facilitate tool calibration during high temperature processUS8654352Aug 8, 2012Feb 18, 2014Asm Technology Singapore Pte LtdChromatic confocal scanning apparatusUS8885040 *Mar 30, 2007Nov 11, 2014Generic Power Pte Ltd.Method and apparatus for 3-dimensional vision and inspection of ball and like protrusions of electronic componentsUS9228957 *Aug 1, 2014Jan 5, 2016Gii Acquisition, LlcHigh speed method and system for inspecting a stream of partsUS20020037098 *Apr 27, 2001Mar 28, 2002Beaty Elwin M.Method and apparatus for three dimensional inspection of electronic componentsUS20030142862 *Dec 18, 2002Jul 31, 2003Snow Donald B.Stereoscopic three-dimensional metrology system and methodUS20040085549 *Jan 2, 2002May 6, 2004Carl SmetsMethod and an apparatus for measuring positions of contact elements of an electronic componentUS20040107018 *Nov 25, 2003Jun 3, 2004Mori Seiki Co., Ltd.Apparatus for generating three-dimensional model dataUS20050189657 *Feb 28, 2005Sep 1, 2005Beaty Elwin M.Electronic component products and method of manufacturing electronic component productsUS20060269102 *May 2, 2006Nov 30, 2006Carpenter Michael DMethod and apparatus for detecting doubles in a singulated stream of flat articlesUS20070023716 *Jul 26, 2005Feb 1, 2007Icos Vision Systems N.V.Apparatus for three dimensional measuring on an electronic componentUS20070269103 *May 21, 2007Nov 22, 2007Applied Precision, LlcStereoscopic three-dimensional metrology system and methodUS20090123060 *Jul 29, 2004May 14, 2009Agency For Science, Technology And Researchinspection systemUS20090128648 *Jun 16, 2006May 21, 2009Omron CorporationImage processing device and image processing method for performing three dimensional measurementsUS20100034452 *Feb 11, 2010Gines David LMethod And Apparatus For Reconstructing Solder Joints Using Constrained X-ray PhotogrammetryUS20100328435 *Mar 29, 2007Dec 30, 2010Yong Joo PuahMethod and apparatus for 3-dimensional vision and inspection of ball and like protrusions of electronic componentsUS20110119909 *Nov 23, 2009May 26, 2011International Business Machines CorporationAlignment module utilizing transparent reticle to facilitate tool calibration during the high temperature processUS20140346097 *Aug 1, 2014Nov 27, 2014Gii Acquisition, Llc Dba General Inspection, LlcHigh speed method and system for inspecting a stream of partsUS20150192528 *Jul 24, 2012Jul 9, 2015Generic Power Pte LtdMethod and apparatus for determining coplanarity in integrated circuit packagesEP1619623A1 *Jul 22, 2004Jan 25, 2006Icos Vision Systems N.V.Apparatus for three dimensional measuring on an electronic componentEP1923663A1 *Nov 8, 2007May 21, 2008Konica Minolta Sensing, Inc.Measuring device, positioning system and positioning method* Cited by examinerClassifications U.S. Classification382/146, 348/87, 382/151, 356/613, 702/152, 250/559.23, 382/154, 348/94, 250/559.31International ClassificationH05K13/08, G01N21/956, G06T1/00, G06T7/00, G01B11/245, G01B11/24Cooperative ClassificationG06T7/0004, G06T2207/30244, H05K13/08, G06T2207/30152, G06T7/0002, H01L2224/13, G06T7/004, G06T7/0018European ClassificationH05K13/08, G06T7/00B, G06T7/00P, G06T7/00C, G06T7/00B1Legal EventsDateCodeEventDescriptionAug 6, 2001ASAssignmentOwner name: SCANNER TECHNOLOGIES CORPORATION, MINNESOTAFree format text: LICENSE;ASSIGNORS:BEATY, ELWIN M.;BEATY, ELAINE E.;REEL/FRAME:012043/0617Effective date: 20010725Nov 13, 2001CCCertificate of correctionNov 12, 2003FPAYFee paymentYear of fee payment: 4Mar 17, 2006ASAssignmentOwner name: SCANNER TECHNOLOGIES CORPORATION, MINNESOTAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MORK, DAVID P.;REEL/FRAME:017344/0782Effective date: 20060317Owner name: SCANNER TECHNOLOGIES CORPORATION, MINNESOTAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BEATY, ELWIN M.;REEL/FRAME:017344/0751Effective date: 20060303Oct 29, 2007ASAssignmentOwner name: MORK, DAVID P., ARIZONAFree format text: SECURITY AGREEMENT;ASSIGNOR:SCANNER TECHNOLOGIES CORPORATION;REEL/FRAME:020031/0478Effective date: 20071029Owner name: BEATY, ELWIN M., MINNESOTAFree format text: SECURITY AGREEMENT;ASSIGNOR:SCANNER TECHNOLOGIES CORPORATION;REEL/FRAME:020031/0478Effective date: 20071029Nov 16, 2007FPAYFee paymentYear of fee payment: 8Nov 26, 2007REMIMaintenance fee reminder mailedDec 26, 2011REMIMaintenance fee reminder mailedMay 16, 2012LAPSLapse for failure to pay maintenance feesJul 3, 2012FPExpired due to failure to pay maintenance feeEffective date: 20120516RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services