Source: http://www.google.com/patents/US4581762?dq=6668407
Timestamp: 2016-08-25 15:54:19
Document Index: 302943413

Matched Legal Cases: ['art 40', 'art 40', 'art 40', 'art 40', 'art 82', 'art 82']

Patent US4581762 - Vision inspection system - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA vision inspection system operable with foreground illumination provides user identification of selected regions of a known object for later comparison to an unknown object. A gray scale pixel array of each selected region is processed for edges and this processed data array is stored as a template...http://www.google.com/patents/US4581762?utm_source=gb-gplus-sharePatent US4581762 - Vision inspection systemAdvanced Patent SearchPublication numberUS4581762 APublication typeGrantApplication numberUS 06/572,570Publication dateApr 8, 1986Filing dateJan 19, 1984Priority dateJan 19, 1984Fee statusPaidPublication number06572570, 572570, US 4581762 A, US 4581762A, US-A-4581762, US4581762 A, US4581762AInventorsStanley N. Lapidus, Joseph J. Dziezanowski, Seymour A. Friedel, Michael P. GreenbergOriginal AssigneeItran CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (5), Referenced by (207), Classifications (10), Legal Events (12) External Links: USPTO, USPTO Assignment, EspacenetVision inspection system
US 4581762 AAbstract
A vision inspection system operable with foreground illumination provides user identification of selected regions of a known object for later comparison to an unknown object. A gray scale pixel array of each selected region is processed for edges and this processed data array is stored as a template for each region. Gray scale illumination data from larger corresponding areas of the unknown object are processed for edges to form gradient maps. The first template is iteratively compared to the first gradient map. A correlation value greater than a threshold value causes the system to examine the second and possibly third gradient maps on the unknown object. Distance and angular relationships of the regions are used to both identify and orient the object under test. Once the unknown object is identified and its orientation determined, various visual attributes and measurements of the object can be determined through use of visual tools.
1. A vision inspection system for inspecting unknown objects in relationship to a known object and for performing various visual tests on the unknown object if it is recognized as corresponding to the known object, comprising:A. vision sensing means for producing data representing the illumination values for viewed objects; B. means for receiving data from the vision sensing means and for storing the image of objects under test; C. means interconnected to the image storing means for viewing an imaged object; D. means interfaced with the viewing means for selecting at least three regions of the known imaged object; E. means interfaced with the image storing means for selecting regions on an unknown object, each region corresponding to one of the selected regions of the known object, each selected region on the unknown object having a size at least equal to the size of the corresponding region on the known object; F. means interfaced with the image storing means for determining and storing gradient values based upon the rate of change of illumination values for each selected region of the known and unknown objects, said gradient values for each selected region of the known object called a template and for each selected region of the unknown object called a gradient map; G. means interfaced with the gradient value determining means for determining the spatial and angular relationships between the templates; H. means interfaced with the gradient value determining means and spatial and angular relationship determining means for overlaying the templates with the gradient maps and for determining the correlation value between each template and the corresponding gradient map, with the separation of said templates based upon the previously determined spatial and angular relationships of said templates; I. means interfaced to the correlation value determining means for determining if each correlation value is greater than a predetermined value and if true, determining a composite correlation value of said correlation values; J. means interfaced with the gradient value determining and storing means for moving the templates over each corresponding gradient map so that the correlation value means and composite correlation value means determine their respective correlation and composite correlation values throughout the entire area of each gradient map; K. means interfaced with the composite correlation value determining means for determining the maximum composite correlation value, if any, and for determining the portions of each gradient map corresponding to this highest composite correlation value; and L. input/output (I/O) means interfaced with the maximum correlation value determining means for controlling the disposition of the test object based upon the presence or absence of a maximum composite correlation value;whereby the portions of each gradient map yielding the highest composite correlation value represent recognition of the object under test as corresponding to the known object. 2. A vision inspection system as defined in claim 1, wherein the means for determining gradient values uses a Sobel technique wherein for an array of illumination values denoted by ##STR9## the gradient is equal to Grad=|Gradx |+|Grady |; where Gradx =(c+2f+i)-(a+2d+f) and where Grady =(a+2b+c)-(g+2h+i). 3. A vision inspection system as defined in claim 2, wherein the correlation value determined by the correlation means is defined by the equation: Correlation=(Cross�Cross)/(AutoPatch�AutoTemp); where Cross equals a number obtained by summing the results of multiplying each element in the template with the overlaid element in the gradient map and summing the products, where AutoPatch equals a number obtained by summing the results of multiplying each element in the overlaid portion of the gradient map times itself, and where AutoTemp equals a number obtained by summing the results of multiplying each element in the template with itself. 4. A vision inspection system as defined in claim 3, wherein the means for determining a composite correlation value derives a number obtained by multiplying the correlation value of the first template-gradient map pair by the other correlation values for the remaining template-gradient map pairs.
The present invention relates to a vision inspection apparatus and in particular such apparatus which can automatically determine the similarity and orientation between a known object and an object under test and can also perform various visual tests on the unknown object once it has been identified and its orientation determined.
A number of prior art devices have been developed to visually inspect objects so as to determine if they meet predefined criteria. Two basic types of vision inspection systems have been developed: those which use backlighting to silhouette the object and those which use frontlighting to observe the object as a variation of illumination across its surface area. The latter type of lighting produces an image, in black and white, known in the art as a gray scale since the observed image can vary in illumination throughout all shades of gray, including zero albedo (black) and 100 percent albedo (total reflectance).
Another form of edge detection is known as the Sobel technique as described at page 337 of a book entitled Digital Image Processing by Gonzales and Wintz, published by Addison Wesley Company, 1977. For a 3�3 image defined by rows ##STR1## and the gradient at midpoint e is defined as
G=(Gx 2 +Gy 2)1/2,                  (EQ 1-1)
G=|Gx |+|Gy |;(EQ 1-2)
Gx =(c+2f+i)-(a+2d+g)                                 (EQ 1-3)
Gy =(a+2b+c)-(g+2h+i)                                 (EQ 1-4)
The actual corners are determined by the fixed relationship of the ob3ect since its actual shape is known. In the example with the integrated circuit chip, it is known that the corners comprise four 90� angles, that the chip has specified dimensions and an approximate orientation angle. These characteristics are combined into a global template which is matched against sets of possible corners. An algorithm then calculates a "goodness of fit" figure for each set and selects the best fitting set as the actual corners.
An automated vision inspection system is disclosed which allows the user to select points of a known object so as to be able to identify and orient objects under test. Once a test object is recognized and oriented with respect to the known object, desired visual measurements are automatically performed on the test object. The results of these measurements provide the means for making logical decisions regarding the ultimate disposition of the test object. The automated vision inspection system is therefore able to perform the tasks of a human inspector in a highly reliable and efficient manner.
The selection of each location by the user then causes the system to store a 9�9 pixel region about the selected location. Based upon the illumination values of the respective pixels, the system computes the illumination gradient at each pixel within a 7�7 region enclosed by the 9�9 pixel region. The gradient value for each pixel within the 7�7 region represents the differential change in illumination in both Cartesian directions. This gradient information is stored and used as a template by the system since it more accurately represents the visual characteristics of the location selected by the user. The actual center point selected by the user is defined as the centroid of the template and is used to compute distances between it and two other points selected by the user. In addition, the angle subtended between the first point (centroid) and second and third points (centroids) is determined for later use in examining unknown objects. This angle in combination with the two lengths defines the vertices of the three points, which by definition, define a triangle.
Therefore it is a principal object of the present invention to provide a vision inspection system that can emulate the decision-making processes and actions performable by a human inspector.
For a better understanding of the nature and objects of the present invention, reference should be made to the following detailed description taken in connection with the following drawings in which:
FIG. 2 is a view of the cathode ray tube (CRT) screen showing an imaged part thereon and three points which a user has selected via the light pen shown in FIG. 1. It also shows, in phantom, the 9�9 pixel arrays about the selected points, search areas associated with and used in the recognition of unknown objects, and selected areas where area counter visual tests are to be performed on recognized objects;
FIG. 3 is an enlarged diagrammatic representation of the 9�9 illumination intensity pixel array which surrounds a point selected by the user;
FIG. 4 is an enlarged diagrammatic view of the 7�7 template formed from the gradient computations made on the 9�9 pixel array shown in FIG. 3;
As best seen in FIG. 1, vision inspection system 20 according to the present invention comprises a number of modules which interact to produce the desired result of defining, in terms of position points and visual tests (tools), a known object to form the basis for testing other (unknown or test) objects, including inspecting unknown objects to determine if they correspond to the known object, and if they do correspond (i.e., are recognized), performing the selected visual tests to those recognized objects. The system also presents output information regarding the test results of each tested object for use by external devices including controlling the disposition of the tested object.
Operation of the Visual Inspection System
The programmer module 34 provides the user with the necessary tools to instruct and use the overall visual inspection system 20. In this regard the module 34 comprises a cathode ray tube (CRT) 38 which displays the image 40' of part 40 as viewed by cameras 36. Part 40 corresponds to either the known object to form the basis for visual tests, or an unknown test object, to be tested for recognition, and if recognized as corresponding to the known object, visually tested with one or more visual tools. Depending upon the particular application and need for resolution, one or more cameras may be used to view part 40, up to a maximum of eight cameras. If multiple cameras are used, each camera views a different portion of the part. A light pen 42 (shown in exaggerated form in FIG. 1) is physically manipulated by the user and positioned on CRT screen 38 so as to pick out various points on the CRT image 40' of part 40 which correspond to points which are of particular interest to the user. These points are typically where an edge is located. They can also be places where the user wishes some visual test to be performed on an object. The light pen is also used to select various options displayed on CRT screen 38 which are available during the setup and operation of a test procedure.
For each point selected, the visual inspection system captures the illumination values of an area 56, 56', or 56" surrounding that given point. As shown in FIG. 3 for the example corresponding to point A, a 9�9 pixel area 56 is stored by the visual inspection system about point A. This 9�9 pixel area 56 is also shown in phantom in FIG. 2. The camera which images the area about point A, as well as all the other cameras, if any, produces illumination values which in their smallest composite size are known as pel or pixels. Each pixel in the present invention can have a digital illumination value ranging from zero (corresponding to no reflectance or brightness), to 63 (corresponding to maximum reflectance or brightness). The visual inspection system thus stores an array of whole numbers in the form of a 9�9 matrix where each number represents the illumination of the corresponding pixel imaged within area 56 about point A.
Therefore, the present invention uses a gradient value based upon a 3�3 pixel area about each point for which the gradient value is to be computed. This can best be seen in FIG. 5 which shows the nine illumination values whose center element is at the second row and fifth column FIG. 3. By use of a technique known as the Sobel process, described at page 337 of a book entitled Digital Image Processing (by Gonzales and Wintz published by Addison Wesley Company, 1977), the gradient of a midpoint of a 3�3 matrix of values is defined as follows:
If the 3�3 matrix values are denoted by, ##STR4## the gradient at midpoint e is defined as
G=(Gx 2 +Gy 2)1/2                   (EQ 1-1)
G=|Gx |+|Gy |(EQ 1-2)
Gy =(a+2b+c)-(g+2h+i).                                (EQ 1-4)
Gx =(63+2�63+63)-(0+2�0+0)
Gx =(63+126+63)-(0+0+0)=252-0=252
Gy =(0+2�32+63)-(0+2�32+63)
Gy =(0+64+63)-(0+64+63)=127-127=0
G=|Gx |+|Gy |=|252|+|0|=252+0=252
This gradient value is then inserted in a 7�7 gradient array as shown in FIG. 4. The value computed for the example shown in FIG. 5 is entered in the first row and fourth column of this gradient array. This process is repeated for all of the illumination values within the 9�9 array shown in FIG. 3 for which the process can be computed; that is, for all values which have neighbors on all sides. This eliminates the perimeter illumination values from having corresponding gradient values.
Thus, the 7�7 gradient array 60 shown in FIG. 4 represents the change in illumination values for all interior points of the 9�9 pixel area corresponding to selected point A. In this example, the gradient values represent a vertical edge (see FIG. 2). This 7�7 gradient array is referred to as a template since it forms the basis for determining if an unknown object has a region which corresponds to this template within some measure of probability. Other examples of illumination values and the computation of corresponding gradient arrays for both a sharp edge and a gradual sharp edge are presented in Table 1.
TABLE 1__________________________________________________________________________CALCULATIONS OF SOBEL TYPE GRADIENT VALUES__________________________________________________________________________(A) 9 � 9 matrix of illumination values for a sharp 45degree edge starting at lower left-hand corner (row 9,column 1); extending up to row 4, column 5, and from thereforming a horizontal edge.   0  0  0  0  0  0  0    0   0   0  0  0  0  0  0  0    0   0   0  0  0  0  0  0  0    0   0   0  0  0  0  5  5  5    5   5   0  0  0  5  10 10 10   10  10   0  0  0  10 10 10 10   10  10   0  5  10 10 10 10 10   10  10   5  10 10 10 10 10 10   10  10   10 10 10 10 10 10 10   10  10Result of   0  0  0  0  0  0  0  SOBEL GradxSOBEL Gradx   0  0  -5 -5 0  0  0   0  -5 -20            -15               0  0  0   -5 -20         -30            -15               0  0  0   -20      -30         -20            -5 0  0  0   -30      -20         -5 0  0  0  0   -20      -5 0  0  0  0  0Result of   0  0  0  0  0  0  0  SOBEL GradySOBEL Grady   0  0  -5 -15               -20                  -20                     -20   0  -5 -20            -35               -40                  -40                     -40   -5 -20         -30            -25               -20                  -20                     -20   -20      -30         -20            -5 0  0  0   -30      -20         -5 0  0  0  0   -20      -5 0  0  0  0  0Gradient   0  0  0  0  0  0  0  Result:Results 0  0  10 20 20 20 20 Grad = |Gradx |   0  10 40 50 40 40 40 +   10 40 60 40 20 20 20 |Grady |   40 60 40 10 0  0  0  7 �  7 array   60 40 10 0  0  0  0   40 10 0  0  0  0  0__________________________________________________________________________(B) 9 � 9 matrix of illumination values for a moregradual edge than that shown for (A) above, but having anedge outline corresponding to that in (A) above.   0  0  0  0  0  0  0    0   0   0  0  0  0  0  0  0    0   0   0  0  0  0  0  0  0    0   0   0  0  0  0  5  5  5    5   5   0  0  0  5  8  8  8    8   8   0  0  5  8  10 10 10   10  10   0  5  8  10 10 10 10   10  10   5  8  10 10 10 10 10   10  10   8  10 10 10 10 10 10   10  10Result of   0  0  0  0  0  0  0  GradxSOBEL   0  0  -5 -5 0  0  0Gradx   0  -5 -18            -13               0  0  0   -5 -18         -26            -13               0  0  0   -18      -26         -20            -7 0  0  0   -26      -20         -9 -2 0  0  0   -20      -9 -2 0  0  0  0Result of   0  0  0  0  0  0  0  GradySOBEL   0  0  -5 -15               -20                  -20                     -20Grady   0  -5 -18            -29               -32                  -32                     -32   -5 -18         -26            -23               -20                  -20                     -20   -18      -26         -20            -11               -8 -8 -8   -26      -20         -9 -2 0  0  0   -20      -9 -2 0  0  0  0Gradient   0  0  0  0  0  0  0  Result:Results 0  0  10 20 20 20 20 Grad = |Grad.sub. x | +   0  10 36 42 32 32 32 |Grady |   10 36 52 36 20 20 20 7 �  7 array   36 52 40 18 8  8  8   52 40 18 4  0  0  0   40 18 4  0  0  0  0__________________________________________________________________________
When the unknown object is positioned for viewing by cameras 36, a recognition test can be performed. Its image can, if desired, be projected on CRT screen 38 or color monitor 38'. The recognition test is performed by selecting a search area 62 on the unknown object in which an area corresponding to the 9�9 pixel area 56 on the known object may exist. The center of the search area corresponds to the template center of the known object. Since the unknown object is normally positioned with respect to cameras 38 and light sources 41 in a manner similar to that of the known object, the visual inspection system need not inspect the entire surface of the unknown object in order to determine if a region corresponds to a region on the known object. Within the alignment accuracy of the mechanism for viewing the unknown object 40, (e.g. turntable 120 shown in FIG. 1) the visual inspection system views an area on the unknown object with an area larger than the 9�9 pixel area (the illumination values for the original template) but whose overall size is inversely proportional to the placement accuracy of the unknown object for viewing by cameras 38.
In a typical situation, the visual inspection system analyzes a search area 62 (shown in phantom in FIG. 2) on the unknown object within which a region corresponding to region 56 for point A of the known object may reside. The size of this search area is typically 40�32 pixels, but it can be adjusted by the system operator. The gradient values for each interior pixel of this matrix is determined using the Sobel process with the numerical gradient values producing an array of pixel gradient values known as gradient map 64 as shown in FIG. 6B. Thus for a 40�32 search area, a 38�30 gradient map is produced. The computation of the gradients from the search area 62 to the search area gradient map 64 corresponds to the generation of the 7�7 template shown in FIG. 4 based upon the illumination values in the 9�9 pixel area 56 shown in FIG. 3. Similar search areas 62' and 62" are also produced by the visual inspection system with corresponding search area gradient maps 64' and 64" for regions centered about points B and C as those points are located on the known object in relationship to line segments 57 and 58 and angle theta. Each gradient map 64, 64' and 64" as well as templates 60, 60' and 60" and their angular and distance relationships are stored in the system's random access memory 70 in the CPU module 26 (see FIG. 1).
Once the gradient maps have been determined, the recognition procedure then begins a systematic search for a correspondence between a region in gradient map 64 to template 60 (corresponding to the region about point A) in the known object. As seen in FIG. 6B, the recognition test procedure begins with the placement of template 60 at the upper left-hand corner of the search area gradient map. The 7�7 template thus overlies a 7�7 portion of the search area gradient map. In order to determine how well the overlaid portion of the gradient map corresponds to the template, a normalized form of a mathematical correlation is determined by use of the following equation:
where Cross is defined as a number obtained by summing the results of multiplying (this summing of the products is called "convolving") each element in the 7�7 template with the overlaid element in the 7�7 portion of the gradient map, where AutoPatch is defined as convolving the 7�7 overlaid portion of the gradient map with itself, and where AutoTemp is defined as convolving the 7�7 template with itself.
Based upon this degree of correspondence (or goodness), the visual inspection system either accepts or rejects the value as representing a region potentially corresponding to that about point A of the known object. In the preferred embodiment of the present invention, if the normalized correlation exceeds a value of 0.85, the system accepts that portion of the gradient map as potentially corresponding to a region 56 about point A of the known object (see FIG. 1). If the value does not exceed 0.85, then the template is moved one position with respect to the gradient map as shown by the 7�7 dotted area defined by line segments 67 shown in FIG. 6B. At this time a new correlation value is computed. Thus, the letter "n" for the variable C(n) corresponds to the particular location of the template with respect to the gradient map. This can be defined as C(1) when the template is over the upper lefthand portion of the gradient map (as shown in FIG. 6B), C(2) when the template is at the location defined by line segments 67, C(3) when the template is defined by line segments 68 as shown in FIG. 6B, and so on, for 32 movements of the template across the upper portion of the gradient map. As will be explained further below, the template is then shifted down one row as shown by line segments 69 and the process repeated for 32 more locations corresponding to this row. The template can be moved down equal to the height of the gradient map (namely 30 pixels) minus the height of the template plus one. Thus if the gradient map is 30 pixels high and the template 7 pixels high, 24 rows are searched. Therefore, for such a gradient map, there are 30 times 24 or 720 locations on the gradient map that the template can overlie.
A simple example of the calculation of the normalized correlation for a template having a size of 2�2 is shown in the following example: ##STR5## The overlaid portion of the gradient map may for instance be: ##STR6## The value of the Cross term is thus:
Cross=(4�3)+(20�15)+(6�2)+(10�20)
The AutoPatch term is the 2�2 area of the overlaid portion of the gradient map times itself or,
AutoPatch=3�3+15�15+2�2+20�20
AutoTemp=4�4+20�20+6�6+10�10
Thus, correlation C(n)=(524�524)/(638�552)
Cross=(4�4)+(20�18)+(6�7)+(10�12)
AutoPatch=(4�4)+(18�18)+(7�7)+(12�12)
The AutoTemp value is again 552. It is therefore seen that once the AutoTemp value for the 7�7 template is computed, the visual inspection system need no longer compute its value for that template. This consequently saves in the computation time for the inspection system. C(n) for this second overlaid portion of the gradient map equals:
C(n)=(538�538)/(533�552)
As shown in FIG. 6C, once a candidate location 72 has been determined on the gradient map, that is, a location with a correlation exceeding 0.85, the system proceeds to determine if a candidate region exists for the second template (template 60' representing the region about point B) having a distance from the center of the first candidate location equal to the length of line segment 57. In essence, the line segment is used as a radius where an arc 74 is transversed along the second gradient map with the center of this arc corresponding to the orientation of second known point B with respect to first known point A as referenced to background 51 (see FIG. 2). Typically, the second template 60' is moved along this arc in a clockwise direction, although other directions could be used if desired (e.g., counterclockwise or starting from the midpoint of the arc and moving first in one direction to the end of the arc, returning to the midpoint and then moving in the other direction). The arc usually subtends plus and minus five degrees about the location where point B is oriented on the known object; although smaller or larger arcs can be selected if desired. The maximum orientation angle that can be tolerated is related to the illumination size or each selected location. For a 9�9 pixel array the maximum rotation of the object under test with respect to the known object must be less than plus/minus the angle whose tangent is 1/9, or approximately �6�. The tangent value is based upon the observation that if a greater angle is allowed, the value read for the last column can be in a different row than the value in the first column.
ComCor=C(candidate region 72)�C(candidate region 76)�C(candidate region 78)                          (EQ 2-2)
Visual Tools of the Vision Inspection System
Each visual tool provides the means for allowing the user to define a type of measurement to be made at a certain location on an unknown object based upon the use of the corresponding tool on the known object. Thus, when the user first defines the three points on the known object which are to be used in the recognition procedure on unknown objects, he or she can also define the various tools and the placement of those tools and the types of measurements which are to be made by those tools on the unknown object based upon the use of those tools on corresponding places on the known object. The particular test sequences are stored by the visual inspection system shown in FIG. 1 based upon setups of those tests on the recognized known object. A description of these tools is presented below. Program listings for these tools are presented in the attached microfiche Appendix A. The listings are in the Pascal language for use on the Hewlett-Packard (Fort Collins, Colo.) HP computer, and incorporate use of the Hewlett-Packard global variable module.
The caliper is a visual tool whose function is analogous to a mechanical caliper; that is, it provides a technique for measuring the inner or outer transverse distances between two surfaces on the object under test. A typical example is shown in FIG. 7 where a machine part 82 includes a disk 83 having a diameter of dimension X, plus or minus some tolerance as identified by line segment 84. In the given example, the diameter is 1.000 in. (2.54 cm.) with a tolerance of �0.001 in. (�0.00254 cm.). The caliper tool, when the known object is shown on CRT screen 38 (see FIG. 1), is selected by the user and positioned onto the imaged part 82 so as to "clamp" over the periphery of disk portion 83. This is shown diagrammatically in FIG. 7 by the illustrated visual tool 86 corresponding to a mechanical caliper. Such a visual representation of a caliper is actually displayed on the CRT screen, and the particular surfaces or points across which the caliper is to measure via its jaws 87 and 88 is determined by placement of light pen 68 at each end of disk portion 83.
Another visual tool implemented by the present invention is an area counter. A complete program listing appears in the microfiche appendix under the module name "AREA." The area visual tool is sometimes referred to as a defect finder, and in operation it defines a rectangle on the known object, such as shown by areas 46, 46', 46" in FIG. 2. It then determines the total number of edge pixels within each area for the known object. An edge pixel is defined as a gradient pixel, such as computed with the Sobel matrix as discussed above, where the magnitude of the gradient pixel is greater than some value, such as 32. Basically, this value represents a gradient pixel in which the magnitude of the illumination varies rapidly between adjacent illumination pixels. Thus the user specifies the size of each rectangle on which edge gradient pixels are to be counted and the system for the known object then determines the number of edge gradient pixels within each area and maintains this information in the system's memory.
Segment Reader
The segment reader visual tool provides the vision system with the capability of reading segmented fonts typically comprised of seven line segments. Although there are a number of seven-segment fonts for presenting alphanumeric information, all of these fonts are similar except to the extent that they may differ in size, tilt, aspect ratio (that is, ratio of height to width), and orientation (which direction the character points), but do not vary in basic form. In operation the segment reader is selected by the user on the CRT screen through use of light pen 42 which is then pointed to the character on the known object. The peripheral segment reader jig 105 is then adjusted in height, aspect ratio, tilt, and orientation until it overlies the area of the seven-segment digit.
As best seen in FIG. 1, the vision inspection system 20 comprises a number of modules which combine to perform the functions of the inspection system as described above and as submitted in the attached microfiche appendix. Each module of the vision inspection system comprises components or submodules which are well known in the art. A description of each of these modules as well as the peripheral equipment is presented below.
Cameras 36
Cameras 36 acquire the image or images of the objects under test. Standard commercially available television cameras are typically used such as the Model KP120 320�240 resolution camera manufactured by the Hitachi Corp. of Tokyo, Japan. Such cameras provide 320 horizontal by 240 vertical pixel resolution which for most applications of the visual inspection system is adequate. If greater resolution is required, 1024�1024 pixel resolution cameras such as those manufactured by MTI-DAGE Corp. may be used. As explained above, a plurality of cameras can be used to view different portions of an object where the resolution required to perform the various tests are not satisfied by a single camera. When multiple cameras are used, each image is sequentially stored and used to perform various portions of the vision inspection process. As shown in FIG. 1, analog video signals 43 are first transferred to the frame grabber module 22.
Frame Grabber Module 22
The frame grabber module 22 performs the task of writing into its memory 23 video digital information as received via analog to digital (A/D) converter 25. The A/D converter in turn receives the analog video signals 43 via a multiplexer 29 which can selectively choose any of the camera video signals when multiple cameras are used.
The frame grabber thus stores the digital video high speed information in a 320�240�8 six bit memory arrays if 320�240 pixel resolution cameras are used or in a 1024�1024�8 six bit arrays if high resolution cameras are used. Each pixel may have one of 64 illumination values associated with it and thus 6 bits (26)are required to store this information. The memory module 23 maintains a 320�240 six bit array for each of the up to eight camera images that may be in use.
Video Output Module 28
The video output module 28 contains a refresh memory 37 which receives data via the high speed private port 33. This memory is typically 320�240�6 bits and thus can store the illumination values associated with one of the cameras. A 1024�1024�6 bit image memory may also be used when high resolution cameras are in use.
This memory also receives color overlay information via lines 35 from the CPU module 26 for visual annunciation of the tools and menu information to the operator. Since this information is normally in a color format, a color map memory 37 is used to generate the desired colors from the overlay data. This memory preferably comprises 4K�12 bits of information for converting the overlay data stored in the image memory into output data which is representative of color information. This color map RAM outputs its information to three 4-bit digital to analog (D/A) converters 65 which convert the digital information into analog information for presention to color monitor 38'.
Central Processing Unit Module 26
The central processing unit (CPU) module 26 comprises a central processing unit (CPU) 100, from 512K bytes to 2 megabytes of random access memory (RAM) 70, a read only memory (ROM) 104, a direct memory access module (DMA) 106, and a tape interface 108. The CPU is preferably a Motorola type 68000 CPU. The amount of random access memory is determined by the amount of vision instructions needed by the system for any given application. Thus if a plurality of different parts are to be imaged with a plurality of cameras, then a larger memory size is needed than if only a single known part is to be imaged. The memory size can also vary depending upon the types of visual tools used and the particular tests implemented that use those tools. In any event, the minimum memory size in the preferred embodiment of the present invention is 512K bytes for program and data storage.
Array Processor Module 24
The array processor module 24 provides the special purpose hardware required to perform the various array operations discussed above for performing Sobel edge detection and correlation at a high speed. Due to the great number of calculations which need to be performed in both the Sobel edge detection process and the correlation processes, a general purpose CPU module would not be able to provide real time output with the current state of the art of such microprocessor based central processing units. The array processor module overcomes this problem by being a dedicated hardware device for performing certain types of mathematical functions.
As best seen in FIGS. 1, 9A, and 9B the array processor performs the majority of the mathematical calculations needed to determine the Sobel gradient values and the correlation values. FIG. 9A shows a detailed block diagram of the portion of the array processor which produces Sobel type gradient information based upon illumination data. The illumination data is received from the frame grabber module (see FIG. 1) for each pixel or pel address about the selected pel address. The gradient value for a selected address is thus based upon the eight neighboring illumination values, forming a "window" of values. FIG. 3 shows a 9�9 sample array of illumination values. The fifth row, fifth column entry denoted by the letter "J" is seen to be the number 10 which corresponds to the illumination value at this particular pixel or pel location. The surrounding numbers are thus: ##STR8## This is also shown in the scene patch window block 132 in FIG. 6A. As noted above (see equation 1-3), the gradient in the X direction is defined by the equation
Gradx =(c+2f+i)-(a+2d+g).
This is shown in FIG. 9A by the Gradx filter array 134. Similarly, the gradient in the y direction is defined by the equation
Grady =(a+2b+c)-(g+2h+i).
This is shown in FIG. 9A by the Grady filter array 136.
The array processor as shown in FIG. 9B also performs the correlation process between the overlaid portion of the gradient map 64, 64' or 64" as denoted by patch 160 and the template 60, 60' or 60". The patch 160 therefore corresponds to a 7�7 array from the corresponding gradient map 64, 64' or 64". The template similarly comprises an array of 7�7 elements, and the output from patch 160 is transferred to multiplying module 162 with associated summing module 163 so as to compute the Autopatch term for the correlation value C(n) (see equation 2-1 above). The Cross term between the patch and the template is performed by multiplying module 164 and summing module 165, generating output 167. The output 166 from Autopatch summing module 163 is transferred to a division node 168 where its value is divided by the output value 167 of the Cross term so as to yield Cross/Autopatch (A/B as shown in FIG. 9B). Similarly, the value of Cross divided by the Autotemp is obtained by division node 169 where C has been previously calculated. As noted earlier, the autotemplate (Autotemp) value is a constant for the entire correlation process for each template and thus once computed, it need no longer be re-evaluated. In FIG. 9B, this is shown by the numeral C which is equal to convolving the template with itself.
Outputs 170 and 171 from the division nodes 168 and 169 are in turn multiplied at node 172 so as to yield the correlation value; that is, Cross�Cross/(Autopatch�Autotemp). The output 174 is transferred to a score module 176. This autocorrelation score represents the degree of fit between the template and the overlaid portion of the gradient map.
Bus 30 provides the data path for interconnecting the frame grabber module 22, the array processor 24, the CPU module 26, and the video output module 28.
This bus incorporates the VME™ bus manufactured by the Motorola Semiconductor Products Corporation of Phoenix, Ariz. in association with the Mostek Corporation of Carrolton, Tex. and the Signetics Corporation of Sunnyvale, Calif. It is described in a Motorola Corporation document entitled VME™ Bus Specification Manual, No. M68KVMEB/D1, Revision A, October, 1981.
Input/Output Module 32
The input/output (I/O) module 32 provides the interconnection of the inspection system to the outside world. Typically this module comprises a Series 500 input/output module as manufactured by the Programmable Control Division of Gould Inc., Andover, Mass. The output portion of this input/output module provides the necessary isolation and power amplification needed to drive external motors, solenoids and other devices used to control loading and unloading of parts to be tested in work area 116. Input information is also received from the work area to confirm proper positioning of the various mechanisms associated with the work area. This information in turn is transferred to the vision inspection system through an input/output interface module 55 (see FIG. 1).
Programmer Module 34
The programmer module 34 includes a CRT module 38, a light pen 42, and a tape drive 102. The CRT can be any standard black and white type cathode ray tube for imaging parts under test. A color monitor may also be used. In the preferred embodiment of the present invention a type CU 901DA monitor manufactured by Nippon Electric Corp. (NEC) of Tokyo, Japan is used. The tape drive is also of conventional design. The preferred embodiment of the vision system uses a Model 3214 tape drive manufactured by the Tandberg of America, Inc. of Armonk, N.Y.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS3636513 *Oct 17, 1969Jan 18, 1972Westinghouse Electric CorpPreprocessing method and apparatus for pattern recognitionUS3638188 *Oct 17, 1969Jan 25, 1972Westinghouse Electric CorpClassification method and apparatus for pattern recognition systemsUS3748644 *Dec 31, 1969Jul 24, 1973Westinghouse Electric CorpAutomatic registration of points in two separate imagesUS4131879 *Apr 25, 1977Dec 26, 1978Gretag AktiengesellschaftMethod and apparatus for determining the relative positions of corresponding points or zones of a sample and an orginalUS4433438 *Nov 25, 1981Feb 21, 1984The United States Of America As Represented By The Secretary Of The Air ForceSobel edge extraction circuit for image processing* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS4701859 *Oct 10, 1984Oct 20, 1987Fujitsu LimitedInspecting method and apparatus for a photomask patternUS4706120 *Aug 30, 1985Nov 10, 1987Texas Instruments IncorporatedModular, vision system for automation of inspection and process controlUS4759074 *Oct 28, 1986Jul 19, 1988General Motors CorporationMethod for automatically inspecting parts utilizing machine vision and system utilizing sameUS4794646 *Aug 14, 1986Dec 27, 1988Mitsubishi Denki Kabushiki KaishaCharged beam pattern defect inspection apparatusUS4807163 *Jul 30, 1985Feb 21, 1989Gibbons Robert DMethod and apparatus for digital analysis of multiple component visible fieldsUS4825394 *May 7, 1985Apr 25, 1989General Dynamics CorporationVision metrology systemUS4837616 *Jun 17, 1988Jun 6, 1989Kabushiki Kaisha ToshibaImage processing apparatus suitable for measuring depth of given point with respect to reference point using two stereoscopic imagesUS4841247 *May 31, 1988Jun 20, 1989Mitsubishi Denki Kabushiki KaishaMethod for interpolative extension of nuclear magnetic resonance imageUS4876600 *Jan 26, 1988Oct 24, 1989Ibp Pietzsch GmbhMethod and device for representing a composite image on a screen of a screen deviceUS4899290 *Oct 16, 1987Feb 6, 1990Digital Equipment CorporationSystem for specifying and executing protocols for using iterative analogy and comparative induction in a model-based computation systemUS4918442 *Oct 3, 1988Apr 17, 1990Bogart Jr Donald WAirplane collision avoidance systemUS4924307 *Jun 3, 1988May 8, 1990Allen-Bradley Company, Inc.Image processorUS4942539 *Dec 21, 1988Jul 17, 1990Gmf Robotics CorporationMethod and system for automatically determining the position and orientation of an object in 3-D spaceUS4972359 *Apr 3, 1987Nov 20, 1990Cognex CorporationDigital image processing systemUS4972495 *Dec 21, 1988Nov 20, 1990General Electric CompanyFeature extraction processorUS5012524 *Feb 27, 1989Apr 30, 1991Motorola, Inc.Automatic inspection methodUS5013154 *Oct 16, 1989May 7, 1991K And M Electronics, Inc.System for remotely reading an analog meterUS5018218 *Aug 29, 1988May 21, 1991Raytheon CompanyConfirmed boundary pattern matchingUS5033101 *Jun 17, 1988Jul 16, 1991Sood Ralf AMethod for character-and/or-object recognitionUS5054102 *Jun 11, 1990Oct 1, 1991Eastman Kodak CompanySelf-centering character stroke thickening for optical character recognitionUS5068908 *Nov 28, 1989Nov 26, 1991Kabushiki Kaisha ToshibaMethod and apparatus for detecting mark from image dataUS5073857 *Aug 28, 1989Dec 17, 1991Accuron CorporationMethod and apparatus for cell analysisUS5077806 *Jun 1, 1989Dec 31, 1991Accuron CorporationMachine vision analysis apparatusUS5111411 *Aug 1, 1988May 5, 1992U.S. Philips CorporationObject sorting systemUS5226095 *May 26, 1992Jul 6, 1993Matsushita Electric Industrial Co., Ltd.Method of detecting the position of an object pattern in an imageUS5231675 *Aug 31, 1990Jul 27, 1993The Boeing CompanySheet metal inspection system and apparatusUS5231677 *Feb 21, 1991Jul 27, 1993Canon Kabushiki KaishaImage processing method and apparatusUS5260780 *Nov 15, 1991Nov 9, 1993Threadmasters, Inc.Visual inspection device and processUS5313286 *Jun 18, 1990May 17, 1994Canon Kabushiki KaishaImage data processing apparatus and method thereofUS5318173 *May 29, 1992Jun 7, 1994Simco/Ramic CorporationHole sorting system and methodUS5365596 *Dec 17, 1992Nov 15, 1994Philip Morris IncorporatedMethods and apparatus for automatic image inspection of continuously moving objectsUS5412569 *Mar 29, 1994May 2, 1995General Electric CompanyAugmented reality maintenance system with archive and comparison deviceUS5416308 *Aug 29, 1991May 16, 1995Video Lottery Technologies, Inc.Transaction document readerUS5471312 *Jul 6, 1992Nov 28, 1995Fanuc Ltd.Automatic calibration methodUS5487011 *Mar 17, 1994Jan 23, 1996Gerber Garment Technology, Inc.Garment marker system having computer assisted alignment of variable contrast cloth designsUS5537615 *Jun 10, 1993Jul 16, 1996The Governor And Company Of The Bank Of EnglandProduction of an image model and inspection of a pixel representation of an imageUS5542280 *Jan 9, 1995Aug 6, 1996Chrysler CorporationAutomated gauge, assessment system and methodUS5563964 *Feb 13, 1995Oct 8, 1996Canon Kabushiki KaishaMethod and apparatus for processing a plurality of designated areas of an imageUS5579415 *Aug 29, 1994Nov 26, 1996Matsushita Electric Industrial Co., Ltd.Computer implemented method of recognizing a position of an objectUS5583954 *Mar 1, 1994Dec 10, 1996Cognex CorporationMethods and apparatus for fast correlationUS5621825 *Mar 8, 1993Apr 15, 1997Omron CorporationImage processor, image processing method and apparatus applying sameUS5631981 *Mar 12, 1996May 20, 1997Eastman Kodak CompanyBitmap registration by gradient descentUS5662034 *Mar 8, 1996Sep 2, 1997Utz Quality Foods, Inc.Potato peeling systemUS5706195 *Sep 5, 1995Jan 6, 1998General Electric CompanyAugmented reality maintenance system for multiple rovsUS5745387 *Sep 28, 1995Apr 28, 1998General Electric CompanyAugmented reality maintenance system employing manipulator arm with archive and comparison deviceUS5752436 *Oct 24, 1996May 19, 1998Utz Quality Foods, Inc.Potato peeling apparatusUS5774177 *Sep 11, 1996Jun 30, 1998Milliken Research CorporationTextile fabric inspection systemUS5778107 *Oct 17, 1997Jul 7, 1998Kabushiki Kaisha Komatsu SeisakushoPosition recognition methodUS5843508 *Jan 31, 1997Dec 1, 1998Utz Quality Foods, Inc.Potato peeling systemUS5859930 *Dec 6, 1995Jan 12, 1999Fpr CorporationFast pattern recognizer utilizing dispersive delay lineUS5872870 *Feb 16, 1996Feb 16, 1999Cognex CorporationMachine vision methods for identifying extrema of objects in rotated reference framesUS5878152 *May 21, 1997Mar 2, 1999Cognex CorporationDepth from focal gradient analysis using object texture removal by albedo normalizationUS5881167 *Jan 8, 1997Mar 9, 1999Matsushita Electric Industrial Co., Ltd.Method for position recognitionUS5909504 *Mar 15, 1996Jun 1, 1999Cognex CorporationMethod of testing a machine vision inspection systemUS5953130 *Jan 6, 1997Sep 14, 1999Cognex CorporationMachine vision methods and apparatus for machine vision illumination of an objectUS5956417 *Feb 28, 1994Sep 21, 1999Sensor Adaptive Machines, Inc.Robot vision using target holes, corners and other object featuresUS5960125 *Nov 21, 1996Sep 28, 1999Cognex CorporationNonfeedback-based machine vision method for determining a calibration relationship between a camera and a moveable objectUS5974169 *Mar 20, 1997Oct 26, 1999Cognex CorporationMachine vision methods for determining characteristics of an object using boundary points and bounding regionsUS5978080 *Sep 25, 1997Nov 2, 1999Cognex CorporationMachine vision methods using feedback to determine an orientation, pixel width and pixel height of a field of viewUS5978502 *Apr 1, 1996Nov 2, 1999Cognex CorporationMachine vision methods for determining characteristics of three-dimensional objectsUS6025854 *Dec 31, 1997Feb 15, 2000Cognex CorporationMethod and apparatus for high speed image acquisitionUS6026176 *Aug 31, 1995Feb 15, 2000Cognex CorporationMachine vision methods and articles of manufacture for ball grid array inspectionUS6044183 *Jun 6, 1995Mar 28, 2000Laser Measurement International Inc.Robot vision using target holes, corners and other object featuresUS6067379 *Nov 12, 1996May 23, 2000Cognex CorporationMethod and apparatus for locating patterns in an optical imageUS6075881 *Mar 18, 1997Jun 13, 2000Cognex CorporationMachine vision methods for identifying collinear sets of points from an imageUS6137893 *Oct 7, 1996Oct 24, 2000Cognex CorporationMachine vision calibration targets and methods of determining their location and orientation in an imageUS6141033 *May 15, 1997Oct 31, 2000Cognex CorporationBandwidth reduction of multichannel images for machine visionUS6178262 *Mar 11, 1994Jan 23, 2001Cognex CorporationCircle locationUS6208758 *Oct 9, 1997Mar 27, 2001Fuji Photo Film Co., Ltd.Method for learning by a neural network including extracting a target object image for which learning operations are to be carried outUS6208772Oct 17, 1997Mar 27, 2001Acuity Imaging, LlcData processing system for logically adjacent data samples such as image data in a machine vision systemUS6215915Feb 20, 1998Apr 10, 2001Cognex CorporationImage processing methods and apparatus for separable, general affine transformation of an imageUS6219442 *Oct 8, 1996Apr 17, 2001International Business Machines CorporationApparatus and method for measuring distortion of a visible pattern on a substrate by viewing predetermined portions thereofUS6236769Jan 28, 1998May 22, 2001Cognex CorporationMachine vision systems and methods for morphological transformation of an image with zero or other uniform offsetsUS6259827Mar 21, 1996Jul 10, 2001Cognex CorporationMachine vision methods for enhancing the contrast between an object and its background using multiple on-axis imagesUS6282328Jan 28, 1998Aug 28, 2001Cognex CorporationMachine vision systems and methods for morphological transformation of an image with non-uniform offsetsUS6298149Aug 25, 1998Oct 2, 2001Cognex CorporationSemiconductor device image inspection with contrast enhancementUS6301396Dec 31, 1998Oct 9, 2001Cognex CorporationNonfeedback-based machine vision methods for determining a calibration relationship between a camera and a moveable objectUS6381366Dec 18, 1998Apr 30, 2002Cognex CorporationMachine vision methods and system for boundary point-based comparison of patterns and imagesUS6381375Apr 6, 1998Apr 30, 2002Cognex CorporationMethods and apparatus for generating a projection of an imageUS6396949Jun 15, 2000May 28, 2002Cognex CorporationMachine vision methods for image segmentation using multiple imagesUS6400996Feb 1, 1999Jun 4, 2002Steven M. HoffbergAdaptive pattern recognition based control system and methodUS6418424May 4, 1999Jul 9, 2002Steven M. HoffbergErgonomic man-machine interface incorporating adaptive pattern recognition based control systemUS6442291Dec 31, 1998Aug 27, 2002Cognex CorporationMachine vision methods and articles of manufacture for ball grid arrayUS6587582Aug 8, 2001Jul 1, 2003Cognex CorporationSemiconductor device image inspection with contrast enhancementUS6608647May 29, 1998Aug 19, 2003Cognex CorporationMethods and apparatus for charge coupled device image acquisition with independent integration and readoutUS6640002 *May 25, 1999Oct 28, 2003Fuji Machine Mfg. Co., Ltd.Image processing apparatusUS6640145Jun 3, 2002Oct 28, 2003Steven HoffbergMedia recording device with packet data interfaceUS6658145Dec 22, 2000Dec 2, 2003Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS6684402Dec 1, 1999Jan 27, 2004Cognex Technology And Investment CorporationControl methods and apparatus for coupling multiple image acquisition devices to a digital data processorUS6687402Oct 23, 2001Feb 3, 2004Cognex CorporationMachine vision methods and systems for boundary feature comparison of patterns and imagesUS6701001 *Jun 20, 2000Mar 2, 2004Dunkley International, Inc.Automated part sorting systemUS6728404Nov 10, 1999Apr 27, 2004Fuji Photo Film Co., Ltd.Method for recognizing object images and learning method for neural networksUS6748104Mar 24, 2000Jun 8, 2004Cognex CorporationMethods and apparatus for machine vision inspection using single and multiple templates or patternsUS6763130 *Jul 21, 1999Jul 13, 2004Applied Materials, Inc.Real time defect source identificationUS6763132 *Aug 16, 2002Jul 13, 2004Daniel FreifeldRepetitive inspection system with intelligent toolsUS6784973 *Aug 31, 2000Aug 31, 2004Eastman Kodak CompanyQuality assurance system for retail photofinishingUS6813376 *Oct 27, 2000Nov 2, 2004Rudolph Technologies, Inc.System and method for detecting defects on a structure-bearing surface using optical inspectionUS6816625Aug 15, 2001Nov 9, 2004Lewis Jr Clarence ADistortion free image capture system and methodUS6836567Nov 10, 2003Dec 28, 2004Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS6850646Nov 10, 2003Feb 1, 2005Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS6856698 *Oct 18, 2002Feb 15, 2005Cognex CorporationFast high-accuracy multi-dimensional pattern localizationUS6942151May 10, 2002Sep 13, 2005Welch Allyn Data Collection, Inc.Optical reader having decoding and image capturing functionalityUS6975747Aug 14, 2001Dec 13, 2005Acuity Cimatrix, Inc.Method and system for monitoring and controlling workpiecesUS6975764Sep 8, 2003Dec 13, 2005Cognex Technology And Investment CorporationFast high-accuracy multi-dimensional pattern inspectionUS6985625Nov 10, 2003Jan 10, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS6993192Nov 10, 2003Jan 31, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7006669Dec 31, 2000Feb 28, 2006Cognex CorporationMachine vision method and apparatus for thresholding images of non-uniform materialsUS7006712Nov 10, 2003Feb 28, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7043081Nov 10, 2003May 9, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7058225Nov 10, 2003Jun 6, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7065262Nov 10, 2003Jun 20, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7088862Nov 10, 2003Aug 8, 2006Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7111787May 15, 2001Sep 26, 2006Hand Held Products, Inc.Multimode image capturing and decoding optical readerUS7164796Nov 11, 2003Jan 16, 2007Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7222789Apr 1, 2005May 29, 2007Hand Held Products, Inc.Bar code reading device having image processing modeUS7242806 *Jan 23, 2002Jul 10, 2007Honeywell International Inc.Methods, functional Data, and Systems for image feature translationUS7242988Mar 2, 1999Jul 10, 2007Linda Irene HoffbergAdaptive pattern recognition based controller apparatus and method and human-factored interface thereforeUS7251366Nov 10, 2003Jul 31, 2007Cognex CorporationFast high-accuracy multi-dimensional pattern inspectionUS7287697Jan 26, 2004Oct 30, 2007Hand Held Products, Inc.Optical reader having a color imagerUS7293712Oct 5, 2004Nov 13, 2007Hand Held Products, Inc.System and method to automatically discriminate between a signature and a dataformUS7413127Nov 3, 2006Aug 19, 2008Hand Held Products, Inc.Optical reader for classifying an imageUS7523866Oct 30, 2006Apr 28, 2009Hand Held Products, Inc.Bar code reading device having image processing modeUS7543747May 25, 2006Jun 9, 2009Hand Held Products, Inc.Image capture apparatus and methodUS7551783May 30, 2007Jun 23, 2009Honeywell International Inc.Methods, functional data, and systems for image feature translationUS7637430Dec 29, 2009Hand Held Products, Inc.Picture taking optical readerUS7639861Dec 29, 2009Cognex Technology And Investment CorporationMethod and apparatus for backlighting a wafer during alignmentUS7686222Aug 8, 2008Mar 30, 2010Hand Held Products, Inc.Optical reader having a color imagerUS7841532Nov 30, 2010Hand Held Products, Inc.Bar code reading device having image processing modeUS7922088Sep 17, 2010Apr 12, 2011Hand Held Products, Inc.System and method to automatically discriminate between different data typesUS7974714Aug 29, 2006Jul 5, 2011Steven Mark HoffbergIntelligent electronic appliance system and methodUS8046313Nov 13, 2006Oct 25, 2011Hoffberg Steven MErgonomic man-machine interface incorporating adaptive pattern recognition based control systemUS8081820Jul 22, 2003Dec 20, 2011Cognex Technology And Investment CorporationMethod for partitioning a pattern into optimized sub-patternsUS8103085Jan 24, 2012Cognex CorporationSystem and method for detecting flaws in objects using machine visionUS8104686Dec 22, 2009Jan 31, 2012Hand Held Products, Inc.Apparatus comprising image sensorUS8111904Oct 7, 2005Feb 7, 2012Cognex Technology And Investment Corp.Methods and apparatus for practical 3D vision systemUS8126259 *Apr 7, 2010Feb 28, 2012Hitachi High-Technologies CorporationMethod and apparatus for visual inspectionUS8131058Jul 20, 2006Mar 6, 2012Hitachi High-Technologies CorporationMethod and apparatus for visual inspectionUS8162584Apr 24, 2012Cognex CorporationMethod and apparatus for semiconductor wafer alignmentUS8196842Apr 11, 2011Jun 12, 2012Hand Held Products, Inc.System and method to automatically discriminate between different data typesUS8229222Jul 24, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8244041Aug 14, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8249362Aug 21, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8254695Aug 28, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8264377Mar 2, 2009Sep 11, 2012Griffith Gregory MAircraft collision avoidance systemUS8265395Sep 11, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8270748Sep 18, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8282006Oct 9, 2012Hand Held Products, Inc.Imaging device operative for image processingUS8292180Aug 1, 2011Oct 23, 2012Hand Held Products, Inc.Optical reader having an imagerUS8295613Oct 23, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8320675Nov 27, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8331673Dec 11, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8335380Dec 18, 2012Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8345979Feb 1, 2007Jan 1, 2013Cognex Technology And Investment CorporationMethods for finding and characterizing a deformed pattern in an imageUS8363942Jan 29, 2013Cognex Technology And Investment CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8363956Jan 29, 2013Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8363972Jan 29, 2013Cognex CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8364136Jan 29, 2013Steven M HoffbergMobile system, a method of operating mobile system and a non-transitory computer readable medium for a programmable control of a mobile systemUS8369967Mar 7, 2011Feb 5, 2013Hoffberg Steven MAlarm system controller and a method for controlling an alarm systemUS8437502May 7, 2013Cognex Technology And Investment CorporationGeneral pose refinement and tracking toolUS8439262May 14, 2013Hand Held Products, Inc.Image capture apparatus and methodUS8472697Jan 19, 2012Jun 25, 2013Hitachi High-Technologies CorporationMethod and apparatus for visual inspectionUS8498489 *Apr 16, 2009Jul 30, 2013Hitachi High-Technologies CorporationImage inspection apparatusUS8528818Mar 26, 2010Sep 10, 2013Hand Held Products, Inc.Optical reader having an imagerUS8543456Dec 13, 2004Sep 24, 2013Ip Mining CorporationMedia targeting system and methodUS8583263Mar 8, 2011Nov 12, 2013Steven M. HoffbergInternet appliance system and methodUS8636224Jun 11, 2012Jan 28, 2014Hand Held Products, Inc.System and method to automatically discriminate between different data typesUS8657200Jun 20, 2011Feb 25, 2014Metrologic Instruments, Inc.Indicia reading terminal with color frame processingUS8789758Sep 14, 2012Jul 29, 2014Hand Held Products, Inc.Picture taking reading apparatusUS8794522May 13, 2013Aug 5, 2014Hand Held Products, Inc.Image capture apparatus and methodUS8803710Sep 10, 2012Aug 12, 2014Gregory M. GriffithAircraft collision avoidance systemUS8867847Oct 19, 2012Oct 21, 2014Cognex Technology And Investment CorporationMethod for fast, robust, multi-dimensional pattern recognitionUS8892495Jan 8, 2013Nov 18, 2014Blanding Hovenweep, LlcAdaptive pattern recognition based controller apparatus and method and human-interface thereforeUS8910875Feb 24, 2014Dec 16, 2014Metrologic Instruments, Inc.Indicia reading terminal with color frame processingUS9147252Dec 19, 2011Sep 29, 2015Cognex Technology And Investment LlcMethod for partitioning a pattern into optimized sub-patternsUS9317763Jan 23, 2014Apr 19, 2016Hand Held Products, Inc.System and method to automatically discriminate between different data typesUS20010048766 *Dec 29, 2000Dec 6, 2001Young,Jr Robert S.Time invariant feature location method and systemUS20020170970 *May 10, 2002Nov 21, 2002Welch Allyn Data Collection, Inc.Optical reader having decoding and image capturing functionalityUS20030138146 *Jan 23, 2002Jul 24, 2003Honeywell Inc.Methods, functional data, and systems for image feature translationUS20040151364 *Dec 31, 2003Aug 5, 2004Kenneway Ernest K.Automated part sorting systemUS20040155110 *Jan 26, 2004Aug 12, 2004Michael EhrhartOptical reader having a color imagerUS20050001035 *May 11, 2004Jan 6, 2005Thomas HawleyPicture taking optical readerUS20050036144 *Jul 7, 2004Feb 17, 2005Takehiko SuzukiPosition detecting methodUS20050167504 *Apr 1, 2005Aug 4, 2005Meier Timothy P.Bar code reading device having image processing modeUS20060122886 *Jan 31, 2006Jun 8, 2006Mckay BrentMedia targeting system and methodUS20060255143 *May 25, 2006Nov 16, 2006Michael EhrhartImage capture apparatus and methodUS20060280358 *Jun 29, 2004Dec 14, 2006Akio IshikawaPattern comparison inspection method and pattern comparison inspection deviceUS20070019858 *Jul 20, 2006Jan 25, 2007Hitachi High-Technologies CorporationMethod and apparatus for visual inspectionUS20070081714 *Oct 7, 2005Apr 12, 2007Wallack Aaron SMethods and apparatus for practical 3D vision systemUS20070183668 *Feb 1, 2007Aug 9, 2007Jason DavisMethods for finding and characterizing a deformed pattern in an imageUS20070237397 *May 30, 2007Oct 11, 2007Honeywell Inc.Methods, functional data, and systems for image feature translationUS20080050006 *Aug 23, 2006Feb 28, 2008Cognex CorporationMethod and apparatus for semiconductor wafer alignmentUS20090045259 *Aug 8, 2008Feb 19, 2009Hand Held Products, Inc.Optical reader having a color imagerUS20090200380 *Apr 27, 2009Aug 13, 2009Hand Held Products, Inc.Bar code reading device having image processing modeUS20100001073 *Jun 8, 2009Jan 7, 2010Hand Held Products, Inc.Image capture apparatus and methodUS20100039682 *Feb 18, 2010Waterloo Industries, Inc.Systems And Arrangements For Object IdentificationUS20100140356 *Dec 22, 2009Jun 10, 2010Hand Held Products, Inc.Apparatus comprising image sensorUS20100194876 *Aug 5, 2010Hitachi High-Technologies CorporationMethod and apparatus for visual inspectionUS20100219988 *Mar 2, 2009Sep 2, 2010Griffith Gregory MAircraft collision avoidance systemUS20110110597 *Apr 16, 2009May 12, 2011Yuichi AbeImage inspection apparatusUS20110188752 *Aug 4, 2011Hand Held Products, Inc.System and method to automatically discriminate between different data typesUSRE38559 *Jun 30, 2000Jul 27, 2004Orbotech LtdAutomatic visual inspection systemDE3939621A1 *Nov 30, 1989May 31, 1990Toshiba Kawasaki KkVerfahren und geraet zur erfassung von markendaten in bilddatenEP0367295A2 *Nov 6, 1989May 9, 1990Matsushita Electric Industrial Co., Ltd.A method of detecting the position of an object pattern in an imageEP0443062A1 *Feb 22, 1990Aug 28, 1991Komori CorporationDevice for inspecting quality of printed matter and method thereofEP0503179A1 *Mar 15, 1991Sep 16, 1992Raytheon CompanyConfirmed boundary pattern matchingEP0586708A1 *Mar 8, 1993Mar 16, 1994Omron CorporationImage processor, method therefor and apparatus using the image processorWO2006113646A2 *Apr 13, 2006Oct 26, 2006Eastman Kodak CompanySystem for digital radiographic image quality verification* Cited by examinerClassifications U.S. Classification382/263, 382/199, 382/209, 348/130, 382/141, 356/615International ClassificationG06T7/00Cooperative ClassificationG06T2207/30164, G06T7/001European ClassificationG06T7/00B1RLegal EventsDateCodeEventDescriptionJan 19, 1985ASAssignmentOwner name: ITRAN CORPORATION, 670 NORTH COMMERICAL STREET ,P.Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNORS:TRUHAN, ANDREW;DZIEZANOWSKI, JOSEPH J.;FRIEDEL, SEYMOURA.;AND OTHERS;REEL/FRAME:004385/0526;SIGNING DATES FROM 19840118 TO 19841031Oct 4, 1989FPAYFee paymentYear of fee payment: 4Nov 16, 1993REMIMaintenance fee reminder mailedMar 25, 1994ASAssignmentOwner name: SILICON VALLEY BANK A CORPORATION OF CA, MASSACHUFree format text: SECURITY INTEREST;ASSIGNOR:ACUITY IMAGING, INC.;REEL/FRAME:006918/0050Effective date: 19940321Owner name: ACUITY IMAGING, INC., MASSACHUSETTSFree format text: MERGER;ASSIGNOR:ITRAN CORP. A CORPORATION OF DE;REEL/FRAME:006918/0038Effective date: 19940126Apr 8, 1994FPAYFee paymentYear of fee payment: 8Apr 8, 1994SULPSurcharge for late paymentAug 15, 1997FPAYFee paymentYear of fee payment: 12Apr 1, 1998ASAssignmentOwner name: ACUITY IMAGING LLC, NEW HAMPSHIREFree format text: MERGER;ASSIGNOR:ACUITY IMAGING, INC;REEL/FRAME:009064/0827Effective date: 19970930Owner name: BANK OF NEW YORK, THE, NEW YORKFree format text: SECURITY INTEREST;ASSIGNOR:ACUITY IMAGING LLC;REEL/FRAME:009052/0790Effective date: 19980318Owner name: ACUITY IMAGING, INC., NEW HAMPSHIREFree format text: RELEASE OF PATENT SECURITY AGREEMENT;ASSIGNOR:SILICON VALLEY BANK;REEL/FRAME:009064/0824Effective date: 19980313Apr 17, 2000ASAssignmentOwner name: ACUITY IMAGING LLC, NEW HAMPSHIREFree format text: RELEASE OF SECURITY INTEREST;ASSIGNOR:BANK OF NEW YORK, THE;REEL/FRAME:010756/0366Effective date: 20000329May 12, 2000ASAssignmentOwner name: PNC BANK, NATIONAL ASSOCIATION, NEW JERSEYFree format text: SECURITY INTEREST;ASSIGNOR:ACUITY IMAGING LLC;REEL/FRAME:010804/0026Effective date: 20000428Owner name: PNC BANK, NATIONAL ASSOCIATION, NEW JERSEYFree format text: SECURITY INTEREST;ASSIGNOR:ROBOTIC VISION SYSTEMS, INC.;REEL/FRAME:010804/0380Effective date: 20000428Jun 14, 2001ASAssignmentOwner name: ACUITY IMAGING LLC, MASSACHUSETTSFree format text: RELEASE OF SECURITY AGREEMENT;ASSIGNOR:BANK OF NEW YORK, THE;REEL/FRAME:012166/0392Effective date: 20010329Oct 27, 2008ASAssignmentOwner name: ACUITY CIMATRIX, INC. (FORMALLY KNOWN AS ROBOTIC VFree format text: RELEASE OF ALL LIENS, CLAIMS, INTERESTS AND ENCUMBRANCES BY BANKRUPTCY COURT;ASSIGNOR:UNITED STATESBANKRUPTCY COURT, DISTRICT OF NEW HAMPSHIRE, CASE 04-14151 AND 04-14152, JUDGE J. MICHAEL DEASY;REEL/FRAME:021744/0081Effective date: 20050927RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services