Source: http://www.google.com/patents/US7813559?dq=5572193
Timestamp: 2017-10-24 01:34:46
Document Index: 736365283

Matched Legal Cases: ['application No. 200580033036', 'application No. 2003', 'application No. 10', 'application No. 2006', 'application No. 2006', 'application No. 102', 'application No. 2006', 'application No. 2003', 'application No. 2005', 'application No. 2005']

Patent US7813559 - Image analysis for pick and place machines with in situ component placement ... - Google Patents
The present invention includes a method of determining a location of a component on a workpiece. A before-placement standard image is acquired of an intended placement location on a standard workpiece. Then, a standard component is placed upon the standard workpiece and the placement is verified. An...http://www.google.com/patents/US7813559?utm_source=gb-gplus-sharePatent US7813559 - Image analysis for pick and place machines with in situ component placement inspection
Publication number US7813559 B2
Application number US 11/131,926
Also published as US20050276464
Publication number 11131926, 131926, US 7813559 B2, US 7813559B2, US-B2-7813559, US7813559 B2, US7813559B2
Inventors David W. Duquette, Eric P. Rudd, Thomas W. Bushman, Swaminathan Manickam, Timothy A. Skunes, Steven K. Case
Patent Citations (167), Non-Patent Citations (21), Referenced by (13), Classifications (14), Legal Events (4)
Image analysis for pick and place machines with in situ component placement inspection
US 7813559 B2
20. The method of claim 19 wherein the background feature is a workpiece feature.
One aspect of embodiments of the present invention provides an efficient computational method for determining good locations in one image for selecting a template for correlation in another image. The method is illustrated in FIG. 6 and begins at block 510 where a Sobel edge filter is computed on the before-placement image, computing both the magnitude and direction of the strongest edge around every pixel. (Alternatively, a simpler gradient operation could be used in a 3×3 neighborhood around every pixel). At block 512, the edge magnitudes are thresholded and the edge directions are rounded to one of eight degrees: 0, 45, 90, 135, 180, 225, 270 and 315 degrees.
FIG. 7 illustrates a pixel 514 and its eight neighbors. Each arrow indicates a normal edge direction. At block 516, the eight directions are encoded for each pixel into an eight-bit byte (edge-encoded pixel). The eight directions listed in this step can be described by their compass directions as follows: east-7, northeast-6, north-5, northwest-4, west-3, southwest-2, south-1 and southeast-0. At block 518, an OR'ing boxcar filter is performed on the encoded data. Unlike a normal boxcar filter, which computes an average like a low-pass filter, this OR'ing boxcar filter performs a bit-wise OR operation on the pixels within the aperture neighborhood. The aperture may be 5×5 pixels or 7×7 pixels (or some other size as appropriate). In the resulting “image” of edge-encoded pixels, each pixel indicates which edge directions are present in its 5×5 or 7×7 neighborhood. At block 520, a predefined look-up table is used to assign scores to every eight-bit edge-encoded pixel in the image. Only a small number of scores (e.g., 4) are generally needed. Since eight-bit codes index into the look-up table, the look-up table need only be 28 or 256 elements. The higher the score, the better the 5×5 or 7×7 neighborhood represented is for use in a correlation template. Of the 256 8-bit edge encodings, the vast majority are symmetrical, so only a few sample scores are shown in the following table.
Score Sample 8-bit codes
0 ( ), (E), (E, W), (E, SE), (E, NW, W, SE)
1 (E, NE, SE), (E, NE, NW, W, SW, SE), (E, NE, W, SE)
2 (E, NE, S, SE), (E, W, SW, S, SE), (E, NW, W, SW, S, SE)
3 (E, NE, N, NW, W, SW, S, SE), (E, N, NW, W, SW, S, SE),
At block 522, the size of the image is reduced by summing 4×4 neighborhoods of scores. (In other words, a pyramid image is built). Alternatively, an additive boxcar filter can be applied on the image with a 4×4 or larger aperture size. Finally, at block 524, the resulting image of scores is scanned for high scores. The high scores indicate good locations in the original image that can be used as correlation templates. Since the component will be placed somewhere in the central part of the image, the search for high scores should be restricted to be outside of the central part of the image. (In fact, all of the above processing can avoid the central area of the image for the sake of efficiency.)
FIG. 11 is a diagrammatic view of a method for analyzing images for “Value/type verification” [block 344 in FIG. 4], “X, Y, theta registration measurement” [block 346 in FIG. 4], and “Polarity determination” [block 348 in FIG. 4]. The method illustrated in FIG. 11 includes steps 540, 542, 544, 546, 548, 549 a and 549 b described with respect to FIG. 8. Accordingly, those steps are neither described nor shown in FIG. 11, which begins with step 582 that executes after step 549 b.
The sequence of steps 544 to 549 b and 582 to 584 produces a mask image. Steps 544 to 549 b were described above. Step 582 performs multiple (e.g., five) 3×3 binary dilations on the thresholded difference image. Step 584 is a large boxcar filter (typically 19×19 pixels) to blur the dilated difference image. In step 586, the blurred mask image produced by step 584 is multiplied times the “after image” acquired in step 540. This isolates the placed component in the image, erasing the non-changing parts of the image. Multiplying the blurred mask times the after image is better than simply ANDing the non-blurred mask image with the after image because ANDing the mask with the gray-scale image would produce artificial edges along the perimeter of the mask.
US4776088 Nov 12, 1987 Oct 11, 1988 The United States Of America As Represented By The United States Department Of Energy Placement accuracy gauge for electrical components and method of using same
US4805111 Sep 11, 1986 Feb 14, 1989 Moore Business Forms, Inc. Size independent modular web processing line and modules
US4989082 * Jul 26, 1989 Jan 29, 1991 Westinghouse Electric Corp. Image processing system for comparing a test article with a master article to determine that an object is correctly located on the test article
US5060366 Aug 21, 1990 Oct 29, 1991 Fuji Machine Mfg. Co., Ltd. Electronic component mounting apparatus
US5127061 Dec 3, 1990 Jun 30, 1992 At&T Bell Laboratories Real-time three-dimensional imaging technique
US5129725 Sep 17, 1991 Jul 14, 1992 Canon Kabushiki Kaisha Method of optically detecting position of object and position detecting apparatus using the method
US5194791 * Jul 19, 1990 Mar 16, 1993 Mcdonnell Douglas Corporation Compliant stereo vision target
US5237622 * Dec 4, 1991 Aug 17, 1993 Micron Technology, Inc. Semiconductor pick-and-place machine automatic calibration apparatus
US5392360 * Apr 28, 1993 Feb 21, 1995 International Business Machines Corporation Method and apparatus for inspection of matched substrate heatsink and hat assemblies
US5467186 * Nov 5, 1993 Nov 14, 1995 Yamaha Hatsudoki Kabushiki Kaisha Attracting nozzle control apparatus for a chip component mounting machine
US5541834 * Oct 14, 1994 Jul 30, 1996 Sanyo Electric Co., Ltd. Control system for component mounting apparatus
US5878151 Oct 31, 1996 Mar 2, 1999 Combustion Engineering, Inc. Moving object tracking
US5903353 * Mar 19, 1998 May 11, 1999 Teradyne, Inc. Method and apparatus for inspecting component placement and solder connection in printed circuit board manufacture
US5911001 Dec 17, 1997 Jun 8, 1999 Fuji Machine Mfg. Co., Ltd. Image processing apparatus
US5912984 * Dec 19, 1996 Jun 15, 1999 Cognex Corporation Method and apparatus for in-line solder paste inspection
US5949681 Mar 10, 1997 Sep 7, 1999 Fuji Machine Mfg. Co., Ltd. Electronic-component supplying apparatus
US6027019 Sep 10, 1997 Feb 22, 2000 Kou; Yuen-Foo Michael Component feeder configuration monitoring
US6047084 * Nov 18, 1997 Apr 4, 2000 Motorola, Inc. Method for determining accuracy of a circuit assembly process and machine employing the same
US6195165 Aug 3, 1999 Feb 27, 2001 Cyberoptics Corporation Enhanced sensor
US6223425 Nov 30, 1998 May 1, 2001 Fuji Machine Mfg. Co., Ltd. Circuit-component supplying system
US6240633 Aug 11, 1999 Jun 5, 2001 Motorola, Inc. Automatic defect detection and generation of control code for subsequent defect repair on an assembly line
US6317972 * May 14, 1999 Nov 20, 2001 Fuji Machine Mfg. Co., Ltd. Method for mounting and inspecting the mounting of electric components
US6332536 Nov 3, 1999 Dec 25, 2001 Solectron Corporation Component tape including a printed component count
US6334840 Jan 21, 1999 Jan 1, 2002 Fuji Machine Mfg. Co., Ltd. Electric-component transferring apparatus, and method and apparatus for exchanging component holders therein
US6404847 Sep 29, 2000 Jun 11, 2002 Rigaku Industrial Corporation Continuously scanning X-ray analyzer having improved readiness and accuracy
US6478533 Feb 7, 2001 Nov 12, 2002 Davis, Iii Maurice M. Method of using a tube feeder for circuit board components
US6506614 * Jan 29, 2002 Jan 14, 2003 Tyco Electronics Corporation Method of locating and placing eye point features of a semiconductor die on a substrate
US6583884 Apr 8, 2002 Jun 24, 2003 Cyberoptics Corporation Tomographic reconstruction of electronic components from shadow image sensor data
US6616263 Oct 31, 2001 Sep 9, 2003 Hewlett-Packard Development Company, L.P. Image forming apparatus having position monitor
US6718626 Sep 7, 2001 Apr 13, 2004 Fuji Machine Mfg. Co., Ltd. Apparatus for detecting positioning error of a component with respect to a suction nozzle
US6762847 Jan 22, 2001 Jul 13, 2004 Cyberoptics Corporation Laser align sensor with sequencing light sources
US6778878 Nov 27, 2000 Aug 17, 2004 Accu-Assembly Incorporated Monitoring electronic component holders
US6987530 May 29, 2001 Jan 17, 2006 Hewlett-Packard Development Company, L.P. Method for reducing motion blur in a digital image
US7029224 Mar 11, 2003 Apr 18, 2006 Matsushita Electric Industrial Co., Ltd. Method and apparatus for transferring a thin plate
US7043824 * May 3, 2002 May 16, 2006 Fuji Machine Mfg. Co., Ltd. Electric-component mounting system
US20010033386 Jan 5, 2001 Oct 25, 2001 Kranz David M Phase profilometry system with telecentric projector
US20010040117 Nov 3, 1999 Nov 15, 2001 Mark Easton Component tape including a printed component count
US20020053133 May 24, 2001 May 9, 2002 Fuji Machine Mfg. Co., Ltd. Electric-component mounting apparatus
US20020069395 Nov 29, 2001 Jun 6, 2002 Hiroaki Fujiwara Method and apparatus of checking mount quality of circuit board
US20020124391 Mar 4, 2002 Sep 12, 2002 Fuji Machine Mfg. Co., Ltd. Apparatus for assisting operator in performing manual operations in connection with component feeders
US20020133940 Mar 21, 2002 Sep 26, 2002 Fuji Machine Mfg. Co., Ltd Electric-component supplying method and device, and electric-component mounting method and system
US20020143423 Apr 3, 2001 Oct 3, 2002 Robert Huber Scheduling system for an electronics manufacturing plant
US20030027363 Jul 18, 2002 Feb 6, 2003 Fuji Machine Mfg. Co., Ltd. Circuit-substrate working system and electronic-circuit fabricating process
US20030111494 Oct 25, 2002 Jun 19, 2003 Sequenom, Inc. Method and apparatus for high-throughput sample handling process line
US20030125895 Oct 7, 2002 Jul 3, 2003 Fuji Machine Mfg. Co., Ltd. Apparatus and method for inspecting working operations on circuit substrate, and system and method for fabricating electric circuit
US20030219330 May 24, 2002 Nov 27, 2003 Lyndaker David W. Auto-loading component tape feeder
US20040135884 Oct 21, 2003 Jul 15, 2004 Kazuhito Saeki Image processing system and image processing method
US20040184653 Mar 20, 2003 Sep 23, 2004 Baer Richard L. Optical inspection system, illumination apparatus and method for use in imaging specular objects based on illumination gradients
US20040186616 Apr 2, 2004 Sep 23, 2004 Bowe Bell + Howell Postal Systems Company Flats bundle collator
US20050077340 Sep 29, 2004 Apr 14, 2005 Matsushita Electric Industrial Co., Ltd. Method, apparatus, system, method and device for data creating, and program for mounting electronic component
US20050097035 Nov 5, 2003 May 5, 2005 John Likas Master system of record
US20050115060 Mar 31, 2003 Jun 2, 2005 Toshihiro Kondo Working system for substrate
US20050137979 Dec 22, 2003 Jun 23, 2005 James Rekeweg Apparatus and method for amount verification of paper checks for electronic redeposit
US20050161498 Mar 6, 2003 Jul 28, 2005 Valor Denmark A/S Feeder verification with a camera
US20050210832 Apr 15, 2003 Sep 29, 2005 I.M.A. Industria Macchine Automatiche S.P.A. Method for carrying out a size change over in a packaging machine
US20050235913 Apr 23, 2004 Oct 27, 2005 Speedline Technologies, Inc. Imaging and inspection system for a dispenser and method for same
US20060174480 Jan 24, 2006 Aug 10, 2006 Fuji Machine Mfg. Co., Ltd. Inspection method and apparatus for mounted electronic components
US20070003126 May 18, 2006 Jan 4, 2007 Case Steven K Method and apparatus for evaluating a component pick action in an electronics assembly machine
US20070010969 Sep 13, 2006 Jan 11, 2007 Cyberoptics Corporation Pick and place machine with improved setup and operation procedure
CN1317926C Sep 3, 2004 May 23, 2007 艾默生网络能源有限公司 Inserted hanging door type cabinet
CN1438836A Jan 29, 2003 Aug 27, 2003 蒂科电子公司 Method for positioning on base and placing view-point characteristic of semiconductor piece
DE19801978A1 Jan 20, 1998 Jul 29, 1999 Sensor Instr Gmbh Unit for inspecting correct loading of spinning arm carrying SMD component to be mounted on circuit board
EP0932331A1 Jan 15, 1999 Jul 28, 1999 FUJI MACHINE Mfg. Co., Ltd. Electric-component mounter equipped with a damping apparatus
EP0948250B1 Dec 12, 1997 Feb 8, 2006 Matsushita Electric Industrial Co., Ltd. Electronic component and mounting method and device therefor
EP0959654A3 May 19, 1999 Aug 9, 2000 FUJI MACHINE Mfg. Co., Ltd. Method and system for mounting electric components
EP1014438A3 Dec 22, 1999 Aug 27, 2003 M.V. Research Limited A measurement system
GB2317496B Title not available
JP2000339456A Title not available
JP2001168594A Title not available
JP2001177836A Title not available
JP2002271099A Title not available
JP2003152392A Title not available
JP2003243899A Title not available
JP2005285840A Title not available
WO2001067831A2 Mar 9, 2001 Sep 13, 2001 Infotech Ag One camera system for component to substrate registration
WO2003043400A1 Nov 12, 2002 May 22, 2003 Cyberoptics Corporation Pick and place machine with component placement inspection
2 First Office Action from Chinese patent application No. 200580033036.9, filed Oct. 5, 2005.
3 First Rejection Notice for Japanese patent application No. 2003-545092, dated Jan. 23, 2008.
4 International Search Report and Written Opinion for application No. PCT/US2006/035954, dated Jan. 31, 2007.
5 International Search Report and Written Opinion for application No. PCT/US2006/042337, dated Mar. 8, 2007.
6 International Search Report and Written Opinion for application No. PCT/US2007/011819, dated Jan. 16, 2008.
7 International Search Report and Written Opinion from application No.: PCT/US05/017425, filed May 18, 2005; Notification of Transmittal of the International Preliminary Report from application No.: PCT/US05/017425, filed May 18, 2005.
8 International Search Report from Appln No.: PCT/US2004/036683, filed Nov. 3, 2004; International Preliminary Report on Patentability from Application No.: PCT/US2004/036683, filed Nov. 3, 2004.
9 Machine translation of JP 06201532.
10 Notice of Preliminary Rejection for Korean patent application No. 10-2004-7007281, dated Jul. 30, 2009.
11 Official Action for Japanese patent application No. 2006-539579, mailed Jul. 28, 2009.
12 Official Action for Japanese patent application No. 2006-539635, mailed Jul. 28, 2009.
13 Official Action from German patent application No. 102 97 444.6, dated Nov. 16, 2009.
14 Official Action from Japanese patent application No. 2006-538473, dated Nov. 10, 2009.
15 Second Rejection Notice for Japanese patent application No. 2003-545092, dated May 21, 2008.
16 The International Preliminary Report on Patentability for foreign application No. PCT/US2004/036238 filed Oct. 29, 2004; Notification of Transmittal of The International Search Report and Written Opinion of the International Searching Authority for application No. PCT/US2004/036238, filed Oct. 29, 2004.
17 The International Preliminary Report on Patentability from Appln No. PCT/US/2005/35985, filed Oct. 5, 2005; International Search Report and Written Opinion from Appln No. PCT/US/2005/35985, filed Oct. 5, 2005.
18 The International Preliminary Report on Patentability in foreign Appln No. PCT/US/2004/036704, filed Nov. 3, 2004; Notification of Transmitttal of the International Search Report and Written Opinion of the International Searching Authority in foreign Appln No. PCT/US/2004/036704, filed Nov. 3, 2004.
19 The International Search Report and Written Opinion from Appln No. PCT/US/2006/019281, filed May 18, 2006.
20 The International Search Report from Appln No. PCT/US/2002/36154, filed Nov. 11, 2002.
21 The Notification of Transmittal of International Preliminary Examination Report in foreign application No. 2005/025905, filed Jul. 21, 2005; International Search Report and Written Opinion from foreign application No. 2005/025905, filed Jul. 21, 2005.
US8363924 * Aug 11, 2005 Jan 29, 2013 Advantest Corporation Electronic device testing apparatus
US8644591 * Jan 17, 2012 Feb 4, 2014 Metal Industries Research & Development Centre Alignment method for assembling substrates without fiducial mark
US8768514 * Apr 26, 2010 Jul 1, 2014 Fuji Machine Mfg. Co., Ltd. Image taking system and electronic-circuit-component mounting machine
US9159130 * Oct 14, 2010 Oct 13, 2015 Koninklijke Philips N.V. Alignment of an ordered stack of images from a specimen
US9456536 * Nov 8, 2013 Sep 27, 2016 Industrial Technology Research Institute Method for pins detection and insertion of electronic component
US9697597 * Apr 9, 2014 Jul 4, 2017 Vit Method of correcting a three-dimensional image of an electronic circuit
US20090127068 * Aug 11, 2005 May 21, 2009 Advantest Corporation Electronic device testing apparatus
US20120207367 * Oct 14, 2010 Aug 16, 2012 Koninklijke Philips Electronics N.V. Alignment of an ordered stack of images from a specimen
US20130251238 * Nov 8, 2012 Sep 26, 2013 Samsung Electronics Co., Ltd. Methods of aligning objects and apparatuses for performing the same
US20140130341 * Nov 8, 2013 May 15, 2014 Industrial Technology Research Institute Method and system for pins detection and insertion of electronic component
US20150356718 * Jul 23, 2015 Dec 10, 2015 Koninklijke Philips N.V. Alignment of an ordered stack of images from a specimen
US20160267636 * Apr 9, 2014 Sep 15, 2016 Vit Method or correcting a three-dimensional image of an electronic circuit
U.S. Classification 382/219, 382/151, 382/153
International Classification G06K9/68, G06K9/00, G06T7/00
Cooperative Classification G06K9/00, G06T7/73, G06T7/0004, G06T7/337
European Classification G06T7/00P1, G06T7/00D1F3, G06K9/00, G06T7/00B1
Free format text: RE-RECORD TO CORRECT THE FIRST ASSIGNOR S NAME ON A DOCUMENT PREVIOUSLY RECORDED AT REEL 016911, FRAME 0282. (ASSIGNMENT OF ASSIGNOR S INTEREST);ASSIGNORS:DUQUETTE, DAVID W.;RUDD, ERIC P.;BUSHMAN, THOMAS W.;AND OTHERS;REEL/FRAME:017474/0156;SIGNING DATES FROM 20050812 TO 20050817
Free format text: RE-RECORD TO CORRECT THE FIRST ASSIGNOR S NAME ON A DOCUMENT PREVIOUSLY RECORDED AT REEL 016911, FRAME 0282. (ASSIGNMENT OF ASSIGNOR S INTEREST);ASSIGNORS:DUQUETTE, DAVID W.;RUDD, ERIC P.;BUSHMAN, THOMAS W.;AND OTHERS;SIGNING DATES FROM 20050812 TO 20050817;REEL/FRAME:017474/0156
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DUQUETE, DAVID W.;RUDD, ERIC P.;BUSHMAN, THOMAS W.;AND OTHERS;REEL/FRAME:016911/0282;SIGNING DATES FROM 20050812 TO 20050817
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DUQUETE, DAVID W.;RUDD, ERIC P.;BUSHMAN, THOMAS W.;AND OTHERS;SIGNING DATES FROM 20050812 TO 20050817;REEL/FRAME:016911/0282
Apr 24, 2014 SULP Surcharge for late payment