Source: https://patents.google.com/patent/US6731777?oq=flatulence
Timestamp: 2018-03-19 17:01:59
Document Index: 472287442

Matched Legal Cases: ['art 7', 'art 40', 'art 48', 'art 49', 'art 36', 'art 41', 'art 42', 'art 43', 'art 44', 'art 36', 'art 48', 'art 36', 'art 48', 'art 36', 'art 49', 'art 42', 'art 43', 'art 36', 'art 41', 'art 42', 'art 42', 'art 44']

US6731777B1 - Object recognition system - Google Patents
US6731777B1
US6731777B1 US09567734 US56773400A US6731777B1 US 6731777 B1 US6731777 B1 US 6731777B1 US 09567734 US09567734 US 09567734 US 56773400 A US56773400 A US 56773400A US 6731777 B1 US6731777 B1 US 6731777B1
US09567734
The distance calculating part 7 determines the distance al, to the object in the window W11 using the aforementioned formula: a=B·f/(X1+X2). The distance a11 thus determined is stored in a distance memory 8. A similar calculation process is successively performed for respective windows, and the resulting distances a11, a12, . . . are stored in the distance memory 8. The distance to a captured object calculated for each window is referred to as the measured distance of the window.
As an example, windows in which an image of the characters “60” on the road surface shown is captured as shown in FIG. 3(b) will be described. An image of the characters “60” is captured in an area (i.e., the shaded area in FIG. 3(b)) surrounded by the windows W75, W7B, WA5 and WAB. Examples of the distances measured for the respective windows included in this area are shown in FIG. 4(a). The units of the numerals are meters. Here, windows for which no measured distance is indicated are windows for which the distance could not be calculated because of lack of contrast. In FIG. 4(a), the measured distances of adjacent windows are compared, and if the difference in the measured distances is within a specified range (for example, a difference in measured distances of 0.5 meters or less may be designated as being within this specified range), then the same label is assigned to the two adjacent windows. This process is performed for all of the windows that have measured distances.
For example, the difference between the measured distance 5.8 m of the window W76 and the measured distance 6.0 m of the window W77 is 0.2 m. Accordingly, a label “1” is assigned to the respective windows. When similar process is performed for adjacent windows in the left-side portion of the image, a label “1” is assigned to each of the windows in the left-side portion of FIG. 4(b). In the right-side portion of the image shown in FIG. 4(a), the difference between the measured distance 5.5 m of the window W89 and the measured distance 5.6 m of the window W8A (for example) is 0.1 m. Accordingly, a label “2” is assigned to the respective windows. Here, neither the window W89 nor the window W8A is adjacent to a window to which the label “1” has been assigned. Consequently, different labels are assigned. The labels do not have to be numerals. Any symbols that can be distinguished, such as letters of the alphabet, etc., may be used.
When labels are thus assigned to the respective windows that have measured distance values, an area 51 which is united by the label “1” and an area 52 which is united by the label “2” are determined as shown in FIG. 4(b). These united areas are called “clusters”.
Referring to FIG. 1, three-dimension converter 35 generates three-dimensional data of the clusters. As shown in FIG. 5, the three-dimensional information includes three coordinates in the present embodiment, i.e., horizontal position (x), vertical position (y) and road surface distance (z). The “x” coordinate expressing the horizontal position corresponds to the direction in which the columns of the windows are lined up (see FIG. 3(b)). The “y” coordinates that expresses the vertical position corresponds to the direction of height from the road surface. The z coordinate indicating the distance of the road surface corresponds to the direction in which the rows of the windows are lined up (see FIG. 3(b)). The “z” coordinate is proportional to the measured distance “d”.
The origin O indicates that point of the road surface where the vehicle is located. The “x”, “y” and “z” axes intersect at right angles at the origin O. The “x” axis extends to the left and right as seen from the vehicle. The “y” axis extends in the direction perpendicular to the road surface and the “z” axis in the direction of advance of the vehicle. The imaging camera 53 is located at a height “H” in the direction of the “y” axis from the origin O. The physical object 54 has a height “h” and a width of “g”, and is located at a distance “i” in the direction of the “z” axis.
If the physical object 54 is not present, then the point 55 on the road surface is included in the image captured by the imaging camera 53. If the physical object 54 is present on the road, the window that would include the image of point 55 will include a point 56 of the physical object instead of the image of point 55 of the road surface. The estimated distance “D” is the distance between the imaging camera 53 and the point 55 on the road surface.
When no physical object 54 is present, this estimated distance “D” is equal to the measured distance to the captured point 55. In FIG. 5, the measured distance “d” is the distance from the imaging camera 53 to the point 56 of the physical object 54, which is calculated by the method described above with reference to FIG. 2. In the (x, y, z) coordinate system, the position of the imaging camera 53 is (0, H, 0) and the position of point 56 is (g, h, i).
Since the estimated distance “D” for each window and the height “H” of the imaging camera from the estimated road surface are fixed values, they can be calculated beforehand and stored. As is clear from FIG. 5, the height “h” of the object can be determined from the following equation (1), and distance “i” to the object 54 can be determined from the following equation (2).
The horizontal distance from the vehicle that is the distance in the “x” axis in FIG. 5 is determined beforehand for each column of windows based on the position of the imaging camera. For example, the third column of windows indicates positions 1 meter to the left from the center of the vehicle. Accordingly, the value of the “x” coordinate of point 56 (in the present example, this is g, and is equal to the value of the width of the object of imaging) can be determined based on the position of the window that includes point 56. Thus, the respective windows forming clusters can be expressed in terms of x, y and z coordinates. In another embodiment, it would also be possible to use (for example) the measured distance “d” instead of the “z” coordinate indicating the road surface distance, and windows could also be expressed using a different coordinate system from the coordinate system described above.
Distance of two clusters=(d 1 w 1+d 2×w 2)/(w 1+w 2) (3)
The differences dx and dy in the horizontal positions and vertical positions of the two clusters are expressed as the spacing of the two clusters, and the difference in distance dz is expressed as the difference in the distances of the respective clusters (d1 and d2 in the above description). For example, FIG. 6(a) shows a plurality of clusters as seen from the x-y plane, and FIG. 6(b) shows the same clusters as those in FIG. 6(a), as seen from the x-z plane. The difference in the horizontal positions of the clusters C4 and C6 is expressed by dx in the direction of the “x” axis, and the difference in the vertical positions is expressed by dy in the direction of the “y” axis. If the distances of the clusters C4 and C6 from the vehicle are respectively d4 and d6, then the difference in distance is expressed by dz in the direction of the z axis.
30.0˜ 8.0
Distance of two clusters (meters) (dx and dy) (meters)
The physical object inference part 40 reads out the positions and relative speeds of the previously recognized physical objects 65 and 66 from the physical object memory 39, and calculates the current positions of the physical objects 65 and 66. This calculation can be performed using the calculation formula: (position of previous physical object+relative speed×detection time interval)
Since the relative speed often varies according to the time, the positions of estimated physical objects can be specified as ranges. For example, instead of specifying the position of an estimated physical object calculated at a certain relative speed as a single point expressed as (x, y, z), it would also be possible to specify this position as a range such as (x−1, y−1, z−1) (x+1, y+1, z+1). Or, instead of specifying the position by means of a relative distance calculated at a certain relative speed of s kilometers/hour, it would also be possible to specify the position by means of a range of relative distances calculated at relative speeds of (s−5)˜(s+5) kilometers per hour. By thus specifying the positions of physical objects as ranges, it is possible to estimate the positions of physical objects more accurately even in cases where the relative speed varies to a certain extent.
E 1={square root over ((Xc−Xt)2+(Yc−Yt)2+(Zc−Zt)2 /C·Zt)}+|Wc−Wt|+|Hc−Ht| (4)
The clusters C22 through C26 recognized as a physical object and the corresponding inferred physical object 75 are stored in the cluster memory part 48 and inferred physical object memory part 49 respectively with “processing completed” flags set in order to indicate that the physical object recognition process has been performed.
All the processes performed by the cluster group determining part 36, cluster selection part 41, physical object candidate extraction part 42 and first physical object recognition part 43 (or second physical object recognition part 44) is repeated until processing is completed for all of the clusters (in this example, until “processing completed” flags are set for all of the clusters). In other words, the cluster group determining part 36 checks the “processing completed” flags of the clusters stored in the cluster memory part 48, and when no clusters exist for which “processing completed” flags are yet to be set, the repetition ends.
FIG. 8 shows the process that follows the process of FIG. 7(f). In order to facilitate understanding, the clusters C22 through C26 recognized as a physical object and the corresponding inferred physical object are removed. The cluster group determining part 36 checks the “processing completed” flags of the clusters stored in the cluster memory part 48, and fetches the clusters C21 and C27 through C31 for which “processing completed” flags have not been set. The cluster group determining part 36 also checks the “processing completed” flags of the infrared physical objects stored in the inferred physical object memory part 49, and fetches the inferred physical object 76 for which a “processing completed” flag has not been set.
The candidate generating part 42 determines combined clusters from combinations of the clusters C27 through C31. The first recognition part 43 compares the attributes of the respective combined clusters with the attributes of the inferred physical object 76. As a result, the combined cluster consisting of the clusters C27 through C31 is determined to have attributes that are the closest to those of the inferred physical object 76 so that the combined cluster consisting of the clusters C27 through C31 is recognized as a physical object 79 (FIG. 7(f)). The clusters C27 through C31 recognized as a physical object and the corresponding inferred physical object 76 are stored with “processing completed” flags in the cluster memory 48 and inferred physical object memory 49 respectively.
Next, the cluster grouping part 36 fetches from the cluster memory 48 the cluster C21 for which no “processing completed” flag has been set. Since this is a single cluster, the cluster C21 is treated as a cluster group. In this example, all the inferred physical objects have been processed so that there is no corresponding inferred physical object to be compared. Accordingly, the cluster selection part 41 selects the cluster C21 and transfers it to the candidate generating part 42. The candidate generating part 42 determines combined clusters from combinations of all of the clusters contained in a cluster group. Since the cluster C21 is a single cluster, C21 is treated as a combined cluster. The combined cluster consisting of cluster C21 is processed by the second recognition part 44.
(current distance—previous distance)/detection time interval
a controller that is adapted for measuring the distance from the system to objects with respect to respective windows of an image captured by the sensors, wherein said controller forms clusters by uniting adjacent windows that have similar measured distances; and
a memory for storing data on a previous recognized physical object;
wherein said controller infers a physical object using the data on the previously recognized physical object and the speed of a vehicle relative to the previously recognized physical object and wherein said controller determines a combination of clusters that best matches the inferred physical object.
2. The system of claim 1, wherein said controller groups the clusters into one or more cluster groups according to the distance from the vehicle carrying the system.
E 1={square root over ((Xc−Xt)2+(Yc−Yt)2+(Zc−Zt)2 /C·Zt)}
where Xc is x coordinate of horizontal center of combined clusters, Yc is y coordinate of vertical center of combined clusters, Zc is z coordinate indicating distance of combined clusters from the vehicle, Xt is x coordinate of horizontal center of an inferred physical object, Yt is y coordinate of vertical center of the inferred physical object, Zt is z coordinate indicating distance of the inferred physical object from the vehicle, and C is a constant.
6. The system of claim 5, wherein Zt is determined by an equation:
previous Zt+relative speed×detection time interval.
7. The system of claim 5, wherein the following value is added to E1:
|Wc−Wt|+|Hc−Ht|
where Wc and Wt are width of combined clusters and the inferred physical object respectively, and Hc and Ht are height of combined clusters and the inferred physical object respectively.
8. The system of claim 6, wherein the relative speed is determined by dividing by the detection time interval the difference between a distance from the vehicle to the physical object that has been measured in the present process cycle and a distance from the vehicle to said physical object that was measured in the preceding process cycle.
means for measuring distance from the system to a physical object with respect to each window of an image captured by the sensors;
means for clustering adjacent windows that are in a predetermined distance range;
means for inferring a present position of the physical object that was recognized in the previous recognition cycle, based on previous position of the physical object and the speed of a vehicle relative to the object;
selection means for selecting clusters whose distance from the vehicle is within a predetermined tolerance relative to the distance of the inferred physical object and which overlaps with the inferred physical object;
means for recognizing, as representing the physical object, the combined clusters that comprise one or more clusters selected by said selection and that have closest attributes to the attributes of the inferred physical object.
a memory for storing attributes of at least one sample physical object; and
means for comparing attributes of combined clusters that were not selected by said selection means with attributes of said at least one sample physical object to recognize the sample physical object having closest attributes to be the physical object corresponding to the combined clusters.
11. A method for recognizing a physical object, comprising the steps of:
measuring distance from a vehicle to the physical object with respect to respective windows of an image captured by at least one image sensor;
inferring a present position of the physical object using data on a previously recognized physical object and the speed of the vehicle relative to the previously recognized physical object; and
determining a combination of clusters that best matches the inferred physical object.
12. The method of claim 11, wherein said step of determining includes the step of selecting those clusters that overlaps with a physical object inferred by the controller.
15. The method of claim 14, wherein Zt is determined by an equation: previous Zt+relative speed×detection time interval.
17. The method of claim 15, wherein the relative speed is determined by dividing by the detection time interval the difference between a distance from the vehicle to the physical object that has been measured in the present process cycle and a distance from the vehicle to said physical object that was measured in the preceding process cycle.
measuring distance from a vehicle to the physical object with respect to each window of an image captured one or more image sensors;
clustering adjacent windows that are in a predetermined distance range;
inferring a present position of the physical object that was recognized in the previous recognition cycle, based on previous position of the physical object and the speed of the vehicle relative to the object;
selecting clusters whose distance from the vehicle is within a predetermined tolerance relative to the distance of the inferred physical object and which overlaps with the inferred physical object;
recognizing, as representing the physical object, the combined clusters that comprise one or more clusters selected by the step of selecting and that have closest attributes to the attributes of the inferred physical object.
storing attributes of at least one sample physical object; and
comparing attributes of combined clusters that were not selected in said step of selecting with attributes of said at least one sample physical object to recognize the sample physical object having closest attributes to be the physical object corresponding to the combined clusters.
US09567734 1999-06-16 2000-05-10 Object recognition system Expired - Fee Related US6731777B1 (en)
JP11-169568 1999-06-16
JP16956899A JP4118452B2 (en) 1999-06-16 1999-06-16 Object recognition device
US6731777B1 true US6731777B1 (en) 2004-05-04
ID=15888893
US09567734 Expired - Fee Related US6731777B1 (en) 1999-06-16 2000-05-10 Object recognition system
US (1) US6731777B1 (en)
JP (1) JP4118452B2 (en)
DE (1) DE10029866B4 (en)
EP2858869A1 (en) * 2012-06-06 2015-04-15 Google, Inc. Obstacle evaluation technique
US7284012B2 (en) 2003-01-24 2007-10-16 International Business Machines Corporation Multiple attribute object comparison based on quantitative distance measurement
DE102006050850B4 (en) * 2006-10-27 2009-01-02 Locanis Ag Method and apparatus for distance measurement
KR101340014B1 (en) 2011-12-09 2013-12-10 에스엘 주식회사 Apparatus and method for providing location information
US7474765B2 (en) 2001-11-09 2009-01-06 Honda Giken Kogyo Kabushiki Kaisha Image recognition apparatus
EP2858869A4 (en) * 2012-06-06 2016-05-11 Google Inc Obstacle evaluation technique
DE10029866A1 (en) 2001-02-15 application
JP4118452B2 (en) 2008-07-16 grant
JP2000357233A (en) 2000-12-26 application
DE10029866B4 (en) 2006-06-08 grant
JP2003284057A (en) 2003-10-03 Vehicle periphery monitoring unit
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NISHIGAKI, MORIMICHI;SAKA, MASAKAZU;REEL/FRAME:010988/0104