Source: http://www.google.com/patents/US6915006?dq=7,094,863
Timestamp: 2015-05-24 19:23:15
Document Index: 270743015

Matched Legal Cases: ['art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 70', 'art 70']

Patent US6915006 - Method and apparatus for three dimensional inspection of electronic components - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA three dimensional inspection system for inspecting ball array devices having a plurality of balls, where the ball array device is positioned in an optical system. An illuminator is located to illuminate at least one ball on the ball array device. A first optical element is positioned to transmit light...http://www.google.com/patents/US6915006?utm_source=gb-gplus-sharePatent US6915006 - Method and apparatus for three dimensional inspection of electronic componentsAdvanced Patent SearchPublication numberUS6915006 B2Publication typeGrantApplication numberUS 09/844,232Publication dateJul 5, 2005Filing dateApr 27, 2001Priority dateJan 16, 1998Fee statusLapsedAlso published asUS7079678, US7085411, US20020037098, US20050190960, US20050190961Publication number09844232, 844232, US 6915006 B2, US 6915006B2, US-B2-6915006, US6915006 B2, US6915006B2InventorsElwin M. Beaty, David P. MorkOriginal AssigneeElwin M. Beaty, Elaine M. BeatyExport CitationBiBTeX, EndNote, RefManPatent Citations (14), Non-Patent Citations (2), Referenced by (14), Classifications (20), Legal Events (6) External Links: USPTO, USPTO Assignment, EspacenetMethod and apparatus for three dimensional inspection of electronic components
US 6915006 B2Abstract
A three dimensional inspection system for inspecting ball array devices having a plurality of balls, where the ball array device is positioned in an optical system. An illuminator is located to illuminate at least one ball on the ball array device. A first optical element is positioned to transmit light to the sensor. A second optical element is positioned to direct light from the at least one ball to the sensor, where the sensor, the first optical element and the second optical element cooperate to obtain at least two differing views of the at least one ball, the sensor providing an output representing the at least two differing views. A processor is coupled to receive the output, where the processor processes the output by using a triangulation method to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane.
Images(31) Claims(147)
1. A three dimensional inspection system for inspecting ball grid array devices having a plurality of balls, wherein the ball grid array device is positioned in an optical system, the inspection system comprising:
a) a fixed illuminator located to illuminate at least one ball on the ball grid array device; b) a single sensor; c) a first optical element positioned to transmit light to the sensor; d) a second optical element positioned to direct light from the at least one ball to the sensor, where the sensor, the first optical element and the second optical element cooperate to obtain at least two differing views of the at least one ball, the sensor providing an output representing the at least two differing views; and e) a processor, coupled to receive the output, where the processor processes the output by using a triangulation method to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane. 2. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.
3. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an XY measurement value is proportional to a Z measurement value.
4. The three dimensional inspection apparatus of claim 1 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein a Y measurement value is proportional to a Z measurement value.
5. The three dimensional inspection apparatus of claim 1 wherein the triangulation calculations are based on determining a center of the ball in a first view and determining a ball top location in a second view.
6. The three dimensional inspection apparatus of claim 1 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
7. The three dimensional inspection apparatus of claim 1 wherein the second optical element comprises a mirror.
8. The three dimensional inspection apparatus of claim 1 wherein the second optical element comprises a prism.
9. The three dimensional inspection apparatus of claim 1 wherein one of the at least two differing views is obtained at a low angle of view.
10. The three dimensional inspection apparatus of claim 1 wherein the sensor and the second optical element are positioned to receive light from different angles relative to the calibration plane.
11. The three dimensional inspection apparatus of claim 1 wherein the sensor comprises a charged coupled device array.
12. The three dimensional inspection apparatus of claim 1 wherein the sensor comprises a complementary metal oxide semiconductor device array.
13. The three dimensional inspection apparatus of claim 1 wherein the triangulation method comprises measurements derived from the at least two differing views include grayscale edge detection to locate ball positions.
14. The three dimensional inspection apparatus of claim 1 wherein the measurements include threshold analysis.
15. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a lens.
16. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a pin-hole lens.
17. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a plurality of lens elements.
18. The three dimensional inspection apparatus of claim 1 wherein the first optical element comprises a telecentric lens.
19. The three dimensional inspection apparatus of claim 1 wherein the ball grid array devices comprise bump on wafer devices.
20. The three dimensional inspection apparatus of claim 1 wherein the processor comprises a personal computer.
21. The three dimensional inspection apparatus of claim 1 wherein the sensor includes a solid state sensor array.
22. The three dimensional inspection apparatus of claim 1 wherein one of the views comprises a segment having a crescent shape.
23. The three dimensional inspection system of claim 1 further comprising a diffuser disposed to provide illumination for imaging of a perimeter of the ball grid array device.
24. The three dimensional inspection system of claim 1, wherein the at least one ball on the ball grid array device being inspected comprises a contact.
25. The three dimensional inspection system of claim 1, wherein the at least one ball on the ball grid array device being inspected comprises a pin.
26. The three dimensional inspection system of claim 1, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of: bump contact, ball contact, pad, and pedestal.
27. The three dimensional inspection system of claim 1, wherein the two different views of the at least one ball are obtained in a single image.
28. The three dimensional inspection system of claim 1, wherein the two different views of the at least one ball are each obtained in a separate image.
29. A three dimensional inspection apparatus for ball grid array devices having a plurality of balls, the apparatus comprising:
a) a fixed illuminator positioned to produce reflections from the ball gird array device; b) a single sensor disposed to receive light at a first angle relative to the ball array device; c) a first optical element positioned to transmit light to the sensor, where the sensor obtains a first view of the ball grid array device; d) a second optical element disposed to receive light at a second angle different from the first angle and to transmit a second view of the ball array device to the sensor; e) a frame grabber coupled to the sensor to transmit image information from the sensor; and f) a processor, coupled to receive the image information, where the processor applies triangulation calculations to measurements of the image information so as to calculate a three dimensional position of at least one ball with reference to a pre-calculated calibration plane. 30. The three dimensional inspection apparatus of claim 29 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.
31. The three dimensional inspection apparatus of claim 29 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an XY measurement value is proportional to a Z measurement value.
32. The three dimensional inspection apparatus of claim 29 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein a Y measurement value is proportional to a Z measurement value.
33. The three dimensional inspection apparatus of claim 27 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
34. The three dimensional inspection apparatus of claim 29 wherein the second optical element comprises a mirror.
35. The three dimensional inspection apparatus of claim 29 wherein the second optical element comprises a prism.
36. The three dimensional inspection apparatus of claim 29 wherein the illuminator comprises a ring light.
37. The three dimensional inspection apparatus of claim 29 wherein the illuminator comprises a plurality of light emitting diodes.
38. The three dimensional inspection apparatus of claim 37, wherein at least two of the plurality of light emitting diodes are spectrally diverse from one another.
39. The three dimensional inspection apparatus of claim 29 wherein the illuminator comprises reflected light.
40. The three dimensional inspection apparatus of claim 29 wherein the sensor comprises a charged coupled device array.
41. The three dimensional inspection apparatus of claim 29 wherein the sensor comprises a complementary metal oxide semiconductor device array.
42. The three dimensional inspection apparatus of claim 29 wherein the ball grid array devices comprise bump on wafer devices.
43. The three dimensional inspection apparatus of claim 29 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.
44. The three dimensional inspection apparatus of claim 29 wherein the measurements include threshold analysis.
45. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a lens.
46. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a pin-hole lens.
47. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a plurality of lens elements.
48. The three dimensional inspection apparatus of claim 29 wherein the first optical element comprises a telecentric lens.
49. The three dimensional inspection apparatus of claim 29 wherein the sensor includes a solid state sensor array.
50. The three dimensional inspection apparatus of claim 29 wherein the processor comprises a personal computer.
51. The three dimensional inspection apparatus of claim 29 wherein the second optical element reflects a view to the sensor where at least one ball of the ball grid array device exhibits a crescent shape.
52. The three dimensional inspection apparatus of claim 29 further comprising a diffuser disposed to provide illumination for imaging of a perimeter of the ball grid array device.
53. The three dimensional inspection apparatus of claim 29, wherein the at least one ball on the ball grid array device being inspected comprises a contact.
54. The three dimensional inspection apparatus of claim 29, wherein the at least one ball on the ball grid array device being inspected comprises a pin.
55. The three dimensional inspection apparatus of claim 29, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of: bump contact, ball contact, pad, and pedestal.
56. The three dimensional inspection apparatus of claim 29, wherein the first view of the ball grid array device and the second view of the ball grid array device are obtained in a single image.
57. The three dimensional inspection apparatus of claim 29, wherein the first view of the ball grid array device and the second view of the ball grid array device are each obtained in a separate image.
58. A three dimensional inspection apparatus for ball grid array devices having a plurality of balls, the apparatus comprising:
a) a fixed illuminator disposed to illuminate a ball grid array device; b) a single sensor disposed to receive light at a first angle relative to the ball grid array device, and wherein the sensor includes a solid state sensor array; c) a first optical element positioned to transmit light to the sensor, where the sensor obtains a first view of the ball grid array device; d) a second optical element disposed to receive light at a second angle different from the first angle and to transfer a second view of the ball array device to the sensor; e) an image acquisition apparatus coupled to the sensor to transmit image information representing the first view and the second view; and f) a processor, coupled to receive the image information, where the processor applies triangulation calculations to measurements of the image information so as to calculate a three dimensional position of at least one ball with reference to a pre-calculated calibration plane, wherein the calibration plane comprises a coordinate system having X, Y and Z axes, and wherein an X measurement value is proportional to a Z measurement value. 59. The three dimensional inspection apparatus of claim 58 wherein an XY measurement value is proportional to a Z measurement value.
60. The three dimensional inspection apparatus of claim 58 wherein a Y measurement value is proportional to a Z measurement value.
61. The three dimensional inspection apparatus of claim 58 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
62. The three dimensional inspection apparatus of claim 58 wherein the measurements include grayscale edge detection to locate ball positions.
63. The three dimensional inspection apparatus of claim 58 wherein the measurements include threshold analysis.
64. The three dimensional inspection apparatus of claim 58 wherein the illuminator comprises a plurality of light emitting diodes.
65. The three dimensional inspection apparatus of claim 64, wherein at least two of the plurality of light emitting diodes are spectrally diverse from one another.
66. The three dimensional inspection apparatus of claim 58 wherein the illuminator comprises reflected light.
67. The three dimensional inspection apparatus of claim 58 wherein the ball grid array devices comprise bump on wafer devices.
68. The three dimensional inspection apparatus of claim 58 wherein the solid state sensor array includes a charged coupled device array.
69. The three dimensional inspection apparatus of claim 58 wherein the solid state sensor array includes a complementary metal oxide semiconductor array.
70. The three dimensional inspection apparatus of claim 58 wherein the second optical element comprises a mirror.
71. The three dimensional inspection apparatus of claim 58 wherein the second optical element comprises a prism.
72. The three dimensional inspection apparatus of claim 58 wherein the second view comprises a segment having a crescent shape.
73. The three dimensional inspection apparatus of claim 58 wherein the image acquisition apparatus comprises a frame grabber.
74. The three dimensional inspection apparatus of claim 58 wherein the processor comprises a personal computer.
75. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a lens.
76. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a pin-hole lens.
77. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a plurality of lens elements.
78. The three dimensional inspection apparatus of claim 58 wherein the first optical element comprises a telecentric lens.
79. The three dimensional inspection apparatus of claim 58 further comprising a diffuser disposed to provide illumination for imaging of a perimeter of the ball grid array device.
80. The three dimensional inspection apparatus of claim 58, wherein the at least one ball on the ball grid array device being inspected comprises a contact.
81. The three dimensional inspection apparatus of claim 58, wherein the at least one ball on the ball grid array device being inspected comprises a pin.
82. The three dimensional inspection apparatus of claim 58, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of: bump contact, ball contact, pad, and pedestal.
83. The three dimensional inspection apparatus of claim 58, wherein the first view of the ball grid array device and the second view of the ball grid array device are obtained in a single image.
84. The three dimensional inspection apparatus of claim 58, wherein the first view of the ball grid array device and the second view of the ball grid array device are each obtained in a separate image.
85. A three dimensional inspection apparatus for ball grid array devices having a plurality of balls, wherein the ball grid array device is positioned in a fixed optical system, the apparatus comprising:
a) an illumination apparatus positioned for illuminating the ball grid array device; b) a camera disposed in a fixed focus position relative to the ball grid array device for taking a first image of the ball grid array device to obtain a circular doughnut shape image from at least one ball; c) an optical element disposed in a fixed focus position relative to the ball array device for transmitting a second image of the ball array device to the camera to obtain a side view image of the at least one ball; and d) a processor, coupled to receive the first image and the second image, that applies triangulation calculations on related measurements of the first image and the second image to calculate a three dimensional position of the at least one ball with reference to a pre-calculated calibration plane. 86. The three dimensional inspection apparatus of claim 85 wherein the second image comprises a segment having a crescent shape.
87. The three dimensional inspection apparatus of claim 85 wherein the calibration plane comprises a coordinate system having X, Y and Z axes and wherein an X measurement value is proportional to a Z measurement value.
88. The three dimensional inspection apparatus of claim 85 wherein the triangulation calculations are based on determining a center of the ball in the first image and determining a ball top location in the second image.
89. The three dimensional inspection apparatus of claim 85 wherein the pre-calculated calibration plane is defined by measuring a calibration pattern.
90. The three dimensional inspection apparatus of claim 85 wherein the optical element comprises a mirror that reflects light between the ball grid array device and the camera.
91. The three dimensional inspection apparatus of claim 85 wherein the second image is obtained at a low angle of view.
92. The three dimensional inspection apparatus of claim 85 wherein the camera and the optical element are fixed at different angles relative to the calibration plane.
93. The three dimensional inspection apparatus of claim 85 wherein the camera comprises a charged coupled device array.
94. The three dimensional inspection apparatus of claim 85 wherein the measurements from the first image and the second image include grayscale edge detection to locate ball positions.
95. The three dimensional inspection apparatus of claim 85 wherein the illumination apparatus further comprises a diffuser.
96. The three dimensional inspection apparatus of claim 85 wherein the ball grid array devices comprise bump on wafer devices.
97. The three dimensional inspection apparatus of claim 85 wherein the camera comprises a complementary metal oxide semiconductor device array.
98. The three dimensional inspection apparatus of claim 85 wherein the triangulation calculations include threshold analysis.
99. The three dimensional inspection apparatus of claim 85, is wherein the at least one ball on the ball grid array device being inspected comprises a contact.
100. The three dimensional inspection apparatus of claim 85, wherein the at least one ball on the ball grid array device being inspected comprises a pin.
101. The three dimensional inspection apparatus of claim 85, wherein the at least one ball on the ball grid array device being inspected is selected from the group consisting of: bump contact, ball contact, pad, and pedestal.
102. The three dimensional inspection apparatus of claim 85, wherein the circular doughnut shape image and the side view image of the at least one ball are obtained in a single image.
103. The three dimensional inspection apparatus of claim 85, wherein the circular doughnut shape image and the side view image of the at least one ball are each obtained in a separate image.
104. The three dimensional inspection apparatus of claim 85, wherein the illumination apparatus comprises a plurality of light emitting diodes.
105. The three dimensional inspection apparatus of claim 104, wherein at least two of the plurality of light emitting diodes are spectrally diverse from one another.
106. An apparatus for three dimensional inspection of a lead on a ball grid array device, the apparatus comprising:
one or more light sources providing illumination to the lead; fixed optical elements disposed so as to obtain both a bottom view of the lead and a side perspective view of the lead; a single camera disposed so as to receive from the fixed optical elements at least the bottom view and the side perspective view of the lead; a memory connected to receive from the camera as pixel values the bottom view and the side perspective view of the lead; a processor connected to the memory, the processor implementing software instructions adapted to cause the processor to execute the following actions: determining a first lead reference pixel position in the bottom view; determining a second lead reference pixel position in the side view; converting the first and second lead reference pixel positions into a world value by using pixel values and parameters determined during a calibration. 107. The apparatus of claim 106, wherein a single light source illuminates the lead.
108. The apparatus of claim 106, wherein more than one light source illuminates the lead.
109. The apparatus of claim 89, wherein at least two of the more than one light source are spectrally diverse from one another.
110. The apparatus of claim 106, wherein the bottom view of the lead and a side perspective view of the lead are obtained in a single image.
111. The apparatus of claim 106, wherein the bottom view of the lead and a side perspective view of the lead are obtained in more than one image.
112. The apparatus of claim 106, wherein the parameters determined during the calibration are selected from the group consisting of: pixel scale factors, an angle at a particular point in a view, and correspondence of one or more pixel values to world values.
113. The apparatus of claim 106, wherein the calibration includes resolving missing state values of an inspection system by imaging a precision pattern of known dimensions and spacing.
114. The apparatus of claim 106, wherein the calibration includes determining and storing pixel values of features of a precision pattern of known dimensions and spacing.
115. The apparatus of claim 106, wherein the calibration includes determining and storing deviations from ideal world locations of features of a precision pattern of known dimensions and spacing.
116. The apparatus of claim 106, wherein a z value is calculated by combining a deviation of the first lead reference pixel position from its ideal position with a deviation of the second lead reference pixel position from its ideal position.
117. The apparatus of claim 106, wherein the software instructions are further adapted to cause the processor to execute the further action of:
converting world values to Z deviations by calculating deviation values that represent the deviation of the lead from its ideal position. 118. The apparatus of claim 106, wherein the software instructions are further adapted to cause the processor to execute the further action of:
converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a reference plane. 119. The apparatus of claim 106, wherein the software instructions are further adapted to cause the processor to execute the further action of:
converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a seating plane. 120. The apparatus of claim 106, wherein the one or more light sources comprise a diffuse light.
121. The apparatus of claim 106, wherein the one or more light sources provide a diffuse light for the bottom view of the lead.
122. The apparatus of claim 106, wherein the one or more light sources provide a diffuse light for the side perspective view of the lead.
123. The apparatus of claim 106, wherein the one or more light sources comprise an overhead reflective diffuser to enhance an image of the outline of the ball grid array device.
124. The apparatus of claim 106, wherein the lead comprises a contact.
125. The apparatus of claim 106, wherein the lead comprises a pin.
126. The apparatus of claim 87, wherein the lead is selected from the group consisting of: bump contact, ball contact, pad, and pedestal.
127. An apparatus for three dimensional inspection of a lead on a ball grid array device, the method comprising:
one or more light sources providing illumination to the lead; fixed optical elements disposed so as to obtain both a bottom view of the lead and a side perspective view of the lead; a single camera disposed so as to receive from the fixed optical elements at least the bottom view and the side perspective view of the lead; a memory connected to receive from the camera as pixel values the bottom view and the side perspective view of the lead; a processor connected to the memory, the processor implementing software instructions adapted to cause the processor to execute the following actions: determining a first lead reference pixel position in the bottom view; determining a second lead reference pixel position in the side view; converting the first lead reference pixel position into a first world value and the second lead reference pixel position into a second world value by using pixel values and parameters determined during a calibration. 128. The apparatus of claim 127, wherein a single light source illuminates the lead.
129. The apparatus of claim 127, wherein more than one light source illuminates the lead.
130. The apparatus of claim 129, wherein at least two of the more than one light source are spectrally diverse from one another.
131. The apparatus of claim 127, wherein the bottom view of the lead and a side perspective view of the lead are obtained in a single image.
132. The apparatus of claim 127, wherein the bottom view of the lead and a side perspective view of the lead are obtained in more than one image.
133. The apparatus of claim 127, wherein the parameters determined during the calibration are selected from the group consisting of: pixel scale factors, an angle at a particular point in a view, and correspondence of one or more pixel values to world values.
134. The apparatus of claim 127, wherein the calibration includes resolving missing state values of an inspection system by imaging a precision pattern of known dimensions and spacing.
135. The apparatus of claim 127, wherein the calibration includes determining and storing pixel values of features of a precision pattern of known dimensions and spacing.
136. The apparatus of claim 127, wherein the calibration includes determining and storing deviations from ideal world locations of features of a precision pattern of known dimensions and spacing.
137. The apparatus of claim 127, wherein a Z value is calculated by combining a deviation of the first world value from its ideal position with a deviation of the second world value from its ideal position.
138. The apparatus of claim 127, wherein the software instructions are further adapted to cause the processor to execute the further action of:
converting world values to Z deviations by calculating deviation values that represent the deviation of the lead from its ideal position. 139. The apparatus of claim 127, wherein the software instructions are further adapted to cause the processor to execute the further action of:
converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a reference plane. 140. The apparatus of claim 127, wherein the software instructions are further adapted to cause the processor to execute the further action of:
converting world values to coplanarity values by calculating deviation values that represent the deviation of the lead from a seating plane. 141. The apparatus of claim 127, wherein the one or more light sources comprise a diffuse light.
142. The apparatus of claim 104, wherein the one or more light sources provide a diffuse light for the bottom view of the lead.
143. The apparatus of claim 127, wherein the one or more light sources provide a diffuse light for the side perspective view of the lead.
144. The apparatus of claim 127, wherein the one or more light sources comprise an overhead reflective diffuser to enhance an image of the outline of the ball grid array device.
145. The apparatus of claim 127, wherein the lead comprises a contact.
146. The apparatus of claim 127, wherein the lead comprises a pin.
147. The apparatus of claim 127, wherein the lead is selected from the group consisting of: bump contact, ball contact, pad, and pedestal.
This application is a continuation-in-part of pending U.S. application Ser. No. 09/351,892 filed Jul. 13, 1999, entitled “Method and Apparatus for Three Dimensional Inspection of Electronic Components,” incorporated by reference herein, that is a continuation-in-part of Ser. No. 09/008,243 filed Jan. 16, 1998 now U.S. Pat. No. 6,072,898 issued Jun. 6, 2000, entitled “Method and Apparatus for Three Dimensional Inspection of Electronic Components,” incorporated by reference herein.
FIG. 2E shows the relationship of a side perspective angle to the ratio of the perspective dimension to the non-perspective dimension. Ray 171, 172, and 173 defining point 181 is parallel to ray 174, 175 and 176 defining point 182. Point 181 and point 182 lie on a plane 170 parallel to a plane 180. The intersection of ray 175 and ray 176 define point 186. The intersection of ray 176 and ray 172 define point 184. The intersection of ray 173 and ray 172 define point 187. The intersection of ray 174 and ray 172 define point 183. The reflecting plane 179 intersecting plane 180 at an angle D is defined by ray 172 and ray 175 and the law of reflectance. Ray 172 and ray 175 intersect plane 170 at an angle 177. Referring to FIG. 2E it can be shown: tan θ = C D B C sin A = L sin A Therefore: C = L cos θ = D S L = D S C C = D S cos θ Substituting: tan θ = D S cos θ D B = D S D B cos θ ( tan θ ) ( cos θ ) = D S D B = sin θ θ = arcsin ( D S D B ) FIG. 2F shows a bottom view and a side perspective view of precision dots used in the method for determining a side perspective view angle 177 as shown in FIG. 2E of the system. A bottom view image 200 comprising precision dots 201, 202 and 203 of known spacing and dimensions from the calibration method described earlier can be used to provide a reference for determination of a side perspective view angle 177. The value DH and DB are known from the bottom view calibration. A side perspective view image 210 comprising precision dots 211, 212 and 213, corresponding to bottom view dots 201, 202 and 203 respectively, of known spacing and dimensions DS and Dh from the calibration method described earlier, can be used to determine the side view perspective angle. The ratio of (Dh/DH) from the bottom image 200 and the side perspective image 210 can be used in the bottom view to calibrate DB in the same units as the side perspective view as follows:
Substituting into the equation for the side perspective view angle 177 described earlier yields: θ = arcsin ( D S D B ) = arcsin ( D S D Bcal ) θ = arcsin ( D S D H D B D h ) FIG. 3A shows the apparatus of the invention for a three dimensional inspection of the balls of a ball grid array. The apparatus of the invention includes a part 70 to be inspected. The apparatus further includes a camera 10 with a lens 11, located below the central area of part 70, to receive a bottom image 80, described in conjunction with FIG. 3B, of part 70. The camera 10 is connected to a frame grabber board 12 to receive the image 80. The frame grabber board 12 provides an image data output to a processor 13 to perform a two dimensional inspection as described in conjunction with FIG. 3A. The processor 13 may store an image in memory 14. The apparatus of the invention obtains an image of a pair of side perspective views with a camera 15 and a lens 16. The camera 15 is located to receive an image 90, comprising a pair of side perspective views, described in conjunction with FIG. 3B and utilizing fixed optical elements 30, 32 and 38 for a first side perspective view and fixed optical elements 34, 36 and 38 for a second side perspective view. In one embodiment of the invention, the apparatus may contain a nonlinear optical element 39 to magnify the side perspective image 60 in one dimension as shown in FIG. 8B. In another embodiment of the invention optical element 38 may be the nonlinear element. The fixed optical elements 30, 32, 34, 36 and 38 may be mirrors or prisms. As will be appreciated by those skilled in the art additional optical elements may be incorporated without deviating from the spirit and scope of the invention. The camera 15 is connected to a frame grabber board 17 to receive the image 90. The frame grabber board 17 provides an image data output to a processor 13 to calculate the Z position of the balls, described in conjunction with FIG. 3B. The processor 13 may store an image in memory 14.
Now refer to FIG. 6B. The distance L1 is calculated by the processor as the difference between world point 258, defined by the intersection of ray 255 with the Z=0 world plane 250, and world point 260, defined by the intersection of ray 256 and the Z=0 world plane 250. The value Z is defined as the distance between world point 261 and 258 and is related to L1 as follows: tan θ 1 = Z L 1 Z = L 1 tan θ 1 Z can be computed by processor 13 since the angle 262 is known from calibration. The offset E 265 is the difference between the virtual point 261 defined by the intersection of ray 255 and ray 256 and the crown of ball 71 at point 264, defined by the intersection of ray 255 with the crown of ball 71, and can be calculated from the knowledge of the angle 262 and the ideal dimensions of the ball 71. The final value of Z for ball 71 is:
FIG. 7A shows one example of an image used in the grayscale blob method of the invention. The image processing method finds the location and dimensions of a ball 71 from a bottom image 80. From the expected position of a ball 71, a region of interest in image 80 is defined as (X1,Y1) by (X2,Y2). The width and height of the region of interest are large enough to allow for positioning tolerances of part 70 for inspection. Due to the design of the lighting for the bottom view, the spherical shape of balls 71 of part 70 present a donut shaped image where the region 281, including the perimeter of the ball 71, comprises camera pixels of higher grayscale values and where the central region 282 comprises camera pixels of lower grayscale values. The remainder 283 of the region of interest 280 comprises camera pixels of lower grayscale values.
The C language function “FindBlobCenter”, as described below, is called to find the approximate center of the ball 71 by finding the average position of pixels that are greater than a known threshold value. The exact center of the ball 71 can be found by calling the C language function “FindBallCenter” which also returns an X world and Y world coordinate. FIG. 7B shows one example of an image used with the method of the invention to perform a subpixel measurement of the ball reference point. The method of the invention finds a reference point on a ball 71 in an image 90 of a side perspective view as shown in FIG. 3B. From the expected position of a ball 71, a region of interest 290 in image 80 is defined as (X3, Y3) by (X4,Y4). The width and height of the region of interest are large enough to allow for positioning tolerances of part 70 for inspection. Due to the design of the lighting for a side perspective view, the spherical shape of balls 71 of part 70 present a crescent shaped image 291 comprising camera pixels of higher grayscale values and where the remainder 293 of the region of interest 290 comprises camera pixels of lower grayscale values.
/////////////////////////////////////////////////////////////////////////////////// // FindBlobCenter - finds the X,Y center of the pixels that have a value greater than THRESHOLD in the region (x1,y1) to (x2,y2) /////////////////////////////////////////////////////////////////////////////////// long FindBlobCenter (int x1, int y1, int x2, int y2, double* pX, double* pY) { int x, y; long Found = 0; long SumX = 0; long SumY = 0; for (x=x1; x<=x2; x++) { for (y=y1; y<=y2; y++) { if (Pixel [x] [y] > THRESHOLD) { SumX += x; SumY += y; Found ++; } } } if (Found > 0) { *pX = (double)SumX / (double)Found; *pY = (double)SumY / (double)Found; } return Found; } /////////////////////////////////////////////////////////////////////////////////// // FindBallCenter - finds the X,Y center of the a BGA ball // using the grayscale values /////////////////////////////////////////////////////////////////////////////////// long FindBallCenter (int x1, int y1, int x2, int y2, double* pX, double* pY) { int x,y; long Found = 0; long Total = 0; long SumX = 0; long SumY = 0; for (x=x1; x<=x2; x++) { for (y=y1; y<=y2; y++) { if (Pixel [x] [y] > THRESHOLD) { SumX += x*Pixel [x] [y]; SumY += y*Pixel [x] [y]; Total += Pixel [x] [y]; Found ++; } } } if (Found > 0) { *pX = (double)SumX / (double)Total; *pY = (double)SumY / (double)Total; } return Found; } /////////////////////////////////////////////////////////////////////////////////// // FindCrescentTop - finds the X,Y top position of a BGA crescent /////////////////////////////////////////////////////////////////////////////////// void FindCrescentTop (int CenterX, int CenterY, int Diameter, int* PX, int* pY) { int x, y, Edge, Max, TopX, TopY; int x1 = CenterX − Diamter / 2; int x2 = CenterX + Diamter / 2; int y1 = CenterY − Diamter / 2; int y2 = CenterY; *pY = 9999; for (x=x1; x<=x2; x++) } Max = −9999; for (y=y1 ; y<=y2; y++) } Edge = Pixel [x] [y] − Pixel [x] [y−1]; if (Edge > Max) { Max = Edge; TopY = y; TopX = x; } } if (TopY < *pY) { *pX = TopX; *pY = TopY; } } (c) 1997 Scanner Technologies Inc.
Now refer to FIG. 10B. The distance L1 is calculated by the processor as the distance between world point 709 and world point 711. The distance L2 is calculated by the processor as the distance between world point 713 and world point 709. The value Z1 is defined as the distance between world point 714 and 709 and is related to L1 as follows: tan θ 1 = Z 1 L 1 Z 1 = L 1 tan θ 1 The value Z2 is defined as the distance between world point 718 and 709 and is related to L2 as follows: tan θ 2 = Z 2 L 2 Z 2 = L 2 tan θ 2 The average of Z1 and Z2 are calculated and used as the value for Z of the ball. This method is more repeatable and accurate than methods that use only one perspective view per ball. In still another embodiment of the invention, the method and apparatus disclosed herein is a method and apparatus for calibrating the system by placing a pattern of calibration dots of known spacing and dimensions on the bottom plane of a calibration reticle and for providing a single side perspective view for the three dimensional inspection of parts. From the precision dots the missing state values of the system are determined allowing for three dimensional inspection of balls on BGA devices or balls on wafers or balls on die.
FIG. 11A shows the apparatus of the invention for system calibration, utilizing a single side perspective view. The method and apparatus for calibration of the bottom view is identical to the method and apparatus described earlier in FIG. 2A and 2B for the two side perspective views method. The apparatus for an image of a side perspective view includes a camera 15 with a lens 18 and a calibration reticle 20. The camera 15 is located to receive an image 64 of a side perspective view comprising dots 65, described in conjunction with FIG. 11B, and utilizing fixed optical elements 40 and 42. The fixed optical element 40 may be a mirror or prism. The fixed optical element 42 is a nonlinear element that magnifies the image in one direction. In another embodiment fixed optical element 40 may be this nonlinear element. As will be appreciated by those skilled in the art additional optical elements may be incorporated. The camera 15 is connected to a frame grabber board 17 to receive the image 64. The frame grabber board 17 provides an image data output to a processor 13 to perform a two dimensional inspection as described in conjunction with FIG. 2B. The processor 13 may store an image in memory 14.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5245671May 2, 1989Sep 14, 1993Omron CorporationApparatus for inspecting printed circuit boards and the like, and method of operating sameUS5440391Mar 22, 1991Aug 8, 1995Icos Vision Systems N.V.Method and device for determining a position of at least one lead of an electronic componentUS5617209 *Apr 27, 1995Apr 1, 1997View Engineering, Inc.Method and system for triangulation-based, 3-D imaging utilizing an angled scaning beam of radiant energyUS5859924 *Jul 12, 1996Jan 12, 1999Robotic Vision Systems, Inc.Method and system for measuring object featuresUS5909285Oct 21, 1997Jun 1, 1999Beaty; Elwin M.Three dimensional inspection systemUS6055054May 5, 1997Apr 25, 2000Beaty; Elwin M.Three dimensional inspection systemUS6055055 *Jan 23, 1998Apr 25, 2000Hewlett-Packard CompanyCross optical axis inspection system for integrated circuitsUS6064756May 28, 1999May 16, 2000Elwin M. BeatyApparatus for three dimensional inspection of electronic componentsUS6064757May 28, 1999May 16, 2000Elwin M. BeatyProcess for three dimensional inspection of electronic componentsUS6072898Jan 16, 1998Jun 6, 2000Beaty; Elwin M.Method and apparatus for three dimensional inspection of electronic componentsUS6096567 *Dec 1, 1997Aug 1, 2000Electroglas, Inc.Method and apparatus for direct probe sensingUS6118540 *Jul 11, 1997Sep 12, 2000Semiconductor Technologies & Instruments, Inc.Method and apparatus for inspecting a workpieceUS6141040 *Dec 19, 1997Oct 31, 2000Agilent Technologies, Inc.Measurement and inspection of leads on integrated circuit packagesUS6236747 *Aug 14, 1998May 22, 2001Acuity Imaging, LlcSystem and method for image subtraction for ball and bumped grid array inspection* Cited by examinerNon-Patent CitationsReference1John Lau, Ball Grid Array Technology, pp. 558, 561, 565, 591 (1995 McGraw-Hill).2 *Shenghua Ye et al. "Vision -based system calibration for dimensional inspection". Nov. 9, 1992, IEEE, pp. 731-734 , vol. 2.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7079678 *Feb 28, 2005Jul 18, 2006Scanner Technologies CorporationElectronic component products made according to a process that includes a method for three dimensional inspectionUS7085411 *Feb 28, 2005Aug 1, 2006Scanner Technologies CorporationMethod of manufacturing electronic components including a method for three dimensional inspectionUS7535560Feb 26, 2007May 19, 2009Aceris 3D Inspection Inc.Method and system for the inspection of integrated circuit devices having leadsUS7551272Nov 9, 2005Jun 23, 2009Aceris 3D Inspection Inc.Method and an apparatus for simultaneous 2D and 3D optical inspection and acquisition of optical inspection data of an objectUS7751611 *Dec 22, 2005Jul 6, 2010Saki CorporationApparatus for inspecting appearance of inspection pieceUS7808525 *Feb 9, 2005Oct 5, 2010Japan Aerospace Exploration AgencyTransparent camera calibration tool for camera calibration and calibration method thereofUS8144968Jan 12, 2009Mar 27, 2012Aceris 3D Inspection Inc.Method and apparatus for scanning substratesUS8212869 *Nov 18, 2009Jul 3, 2012Tokyo Electron LimitedSubstrate inspection method, substrate inspection system and storage mediumUS8233041 *Jun 16, 2006Jul 31, 2012Omron CorporationImage processing device and image processing method for performing three dimensional measurementsUS8437533 *Oct 8, 2009May 7, 2013Koh Young Technology Inc.Method of measuring a three-dimensional shapeUS8885040 *Mar 30, 2007Nov 11, 2014Generic Power Pte Ltd.Method and apparatus for 3-dimensional vision and inspection of ball and like protrusions of electronic componentsUS20100092041 *Oct 8, 2009Apr 15, 2010Koh Young Technology Inc.Method of measuring a three-dimensional shapeUS20100141755 *Nov 18, 2009Jun 10, 2010Tokyo Electron LimitedSubstrate inspection method, substrate inspection system and storage mediumUS20100328435 *Mar 29, 2007Dec 30, 2010Yong Joo PuahMethod and apparatus for 3-dimensional vision and inspection of ball and like protrusions of electronic components* Cited by examinerClassifications U.S. Classification382/145, 250/559.23, 382/151, 348/87International ClassificationH05K13/08, G06T7/00Cooperative ClassificationG06T7/004, G06T2207/30148, G06T7/0018, G06T7/0004, G06T2200/04, G06T2207/30152, G06T7/0002, G06T2207/30244, H05K13/08European ClassificationG06T7/00C, G06T7/00P, H05K13/08, G06T7/00B, G06T7/00B1Legal EventsDateCodeEventDescriptionAug 25, 2009FPExpired due to failure to pay maintenance feeEffective date: 20090705Jul 5, 2009LAPSLapse for failure to pay maintenance feesJan 13, 2009REMIMaintenance fee reminder mailedOct 29, 2007ASAssignmentOwner name: BEATY, ELWIN M., MINNESOTAFree format text: SECURITY AGREEMENT;ASSIGNOR:SCANNER TECHNOLOGIES CORPORATION;REEL/FRAME:020031/0478Effective date: 20071029Owner name: MORK, DAVID P., ARIZONAFree format text: SECURITY AGREEMENT;ASSIGNOR:SCANNER TECHNOLOGIES CORPORATION;REEL/FRAME:020031/0478Effective date: 20071029Mar 17, 2006ASAssignmentOwner name: SCANNER TECHNOLOGIES CORPORATION, MINNESOTAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BEATY, ELWIN M.;BEATY, ELAINE E.;REEL/FRAME:017344/0747Effective date: 20060303Oct 1, 2001ASAssignmentOwner name: BEATY, ELAINE E., MINNESOTAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MORK, DAVID P.;REEL/FRAME:012224/0544Effective date: 20010905Owner name: BEATY, ELWIN M., MINNESOTAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MORK, DAVID P.;REEL/FRAME:012224/0544Effective date: 20010905Owner name: BEATY, ELWIN M. 13529 ARTHUR STREETMINNETONKA, MINFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MORK, DAVID P. /AR;REEL/FRAME:012224/0544Owner name: BEATY, ELWIN M. 13529 ARTHUR STREETMINNETONKA, MINFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MORK, DAVID P. /AR;REEL/FRAME:012224/0544Effective date: 20010905RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services