THREE-DIMENSIONAL VISION SYSTEM

Abstract: The present invention consists in a three-dimensional vision system comprising a three-dimensional vision sensor (II) which projects slit-like light on an object (15) to-be-han­ dled and which reads out a resulting slit image, an image input portion (41) which controls the slit-like light to be lit up and put out and also controls the read-out of the slit image and which performs extraction processing of differential sig­ nals as to image signals read out, and an image processor (12; 42) which performs required controls concerning said image input portion and which processes image data of said image input portion; an image with the projected slit-like light and an image without it being sequentially sampled on an identical scanning line of the read-out image so as to obtain the differential image between them, thereby to extract a light segmentation line based on the projected slit-like light.

Description: The present invention relates to a three-dimensional vision system which recognizes the three-dimensional shape of an object using planar light beams projected onto the object when subjected to operations of assembly etc. in a robot, an automaton etc. Examples for vision sensors employing planar light beams have been developed by General Motors USA (CONSIGHT, described in Robotics Today, pp. 26-32 (Summer 1979) by M. R. Ward et al.) and by Hitachi Ltd. (a compact robot vision, described in Proc. of 1st Int. Symp. of Robots Res. Bretton Woods, Aug. 28 - Sept. 2, 83 by Nakagawa et al.). If a surface of an object to be seen is liable to irregularly reflect light and if the external light conditions are stable, such a vision sensor based on a light segmentation system can receive a clear image and is capable of producing a three-dimensional image. If, however, the object to be handled has a blackish surface liable to absorb light or if it has a reflectivity close to total reflection, the quantity of light entering the image sensor is comparatively small, and the extraction of a light segmentation line becomes difficult due to the occurrence of a background image which may be brighter than the reflected light. Moreover, fluctuations of the external light leads to instabilities in extracting the desired image. Further, US-A-4 343 553 discloses a shape detecting apparatus which comprises a slit projecting means for projecting a slit image on a three-dimensional object, a positioning means for positioning said three-dimensional object relative to said slit projecting means, an image pickup means for two-dimensionally scanning the slit image projected by said slit projecting means to pickup the image, a light segment extracting circuit including a center position extracting means for extracting a mean position of two position signals at which a video signal derived by transversely scanning said slit image by said image pickup means corresponds to a first higher reference when a maximum value of said video signal exceeds said first higher reference, a maximum value position extracting means for extracting a position corresponding to the maximum value of said video signal when the maximum value of said video signal is not higher than said first higher reference and exceeds a second lower reference, an erasing means for erasing the position signal of said video signal when the maximum value of said video signal is not higher than said second lower reference and a detecting means for detecting a shape of said three-dimensional object by analyzing the light segment position information derived from said light segment extracting circuit. Finally, US-A-4 128 830 discloses an apparatus for providing a compensation for dark scan noise level for each of a plurality of individual light sensors arranged in a predetermined relation. That apparatus includes means to produce a separate output signal from each of the individual light sensors in accordance with the level of light received by each of the individual light sensors and to produce separate compensation signals for dark scan noise level from each of the individual light sensors when the individual light sensors are prevented from being subjected to light, storage means for receiving and storing the separate compensation signal from said producing means for each of the light sensors to compensate for the dark scan noise level affecting the accuracy of the separate output signal of the same light sensor from said producing means, means to connect said producing means to said storage means only when the individual light sensors are prevented from being subjected to light, subtracting means to subtract the separate compensation signal from said storage means from the output signal of the same light sensor from said producing means to produce a signal compensated for the dark scan noise level and means to supply all of the separate output signals from said producing means to said subtracting means. Summary of the Invention It is an object of the present invention to provide a three-dimensional vision system which can extract a light segmentation line stably without being affected by the surface condition of a object to be handled and by the fluctuation of external light and which can precisely recognize a three-dimensional shape. The said object is achieved according to the present invention by a vision system as defined in the appended claims 1. Further advantageous developments of the present invention can be derived from the subclaims. The basic idea of the present invention lies in produce a difference image between a first image obtained when a planar light beam is switched on and second image obtained when said planar light beam is switched off. Due to the said basic feature the vision system according to the present invention is capable of extracting a light segmentation line stably without being affected through environmental conditions such as a surface condition of an object to be handled or fluctuations of external light and thus can precisely recognize a three-dimensional shape. Brief Description of the Drawings Fig. 1 is an arrangement diagram for a vision system according to an embodiment of the present invention; Fig. 2A and 2B are a detailed block diagrams for the embodiment shown in Fig. 1; Fig 3(a) to 3(d) are diagrams for explaing the basic principle of a light segmentation extraction circuit according to the present invention and Fig. 4 is a conceptual diagram showing the three-dimensional positions of three points on a two-dimensional image which is detected by a three-dimensional vision sensor. Detailed Description of the Preferred Embodiment Now, the present invention will be described in conjunction with an embodiment shown in the drawings. Fig. 1 is a view which shows the schematic arrangement of a three-dimensional vision system according to the present invention. In the arrangement of the system, an image input from a three-dimensional vision sensor (range finder) 11 mounted on the hand of a robot 13 is fed into an image processing apparatus 12, in which the position and attitude of an object to-be-handled 15 are calculated by image processing. The results are transferred to a robot control device 14, and the robot 13 performs, e. g., the operation of gripping the object under the control of the control device. Fig. 2 is a detailed block diagram corresponding to Fig. 1. The three-dimensional vision sensor 11 comprises a projector portion for slit-like or planar light beams 28, which is composed of laser luminous diodes 21, slit-like light patterns 22, and projective lenses 23; and an image detector portion in which a linear image sensor 26 is combined with a rotating mirror 25, an imaging lens 24, and a sensor signal reading circuit 27 for the image sensor 26. Such arrangement affords a three-dimensional image sensor which can receive a slit image when the luminous diodes 21 are lit up and a background image when they are put out. The internal structure of the image processing apparatus 12 is such that an image input portion 41 which is composed of various circuits 30 - 40 to be described later, an image processor 42 which is constructed of a microcomputer by way of example, and a data input/output interface circuit 43 which serves to communicate with the robot control device 14 are connected by a bus line 44. Here, the image input portion 41 is generally controlled by the scanning control circuit 30 thereof. The scanning of one line of the image is performed as stated below. First, the scanning control circuit 30 supplies the rotating mirror control circuit 32 with a positioning signal, by which the rotating mirror 25 is positioned. Thereafter, the laser luminous diode driving circuit 31 is turned "on" (brought into a closed state) to light up the luminous diodes 21. The slit image is applied from the linear image sensor 26 by the sensor signal reading circuit 27, and it is digitized by the A/D converter 33. While the slit image is input, the signal distribution circuit 34 is sending a video signal to the shift register 35. This shift register 35 can store data of the same number of pixels as that of the linear image sensor 26, and can hold the data until the next scanning cycle. When the slit image has been input by one scanning line, the scanning control circuit 30 puts out the luminous diodes 21 and starts the sensor signal reading circuit 27 again so as to receive the background image. While the background image is input, the signal distribution circuit 34 sends a video signal to the differential signal extracting circuit 36 in accordance with a signal from the scanning control circuit 30. The differential signal extracting circuit 36 is simultaneously supplied with the slit-like light segmentation image data from the shift register 35. It finds the differences of the background image signals from the slit image signals for the same pixels, and stores the results into the differential signal memory 39. At the same time, the position P max of the maximum signal level within the scanning operation is detected by the peak signal position detecting circuit 37 (which is concretely disclosed in U. S. Patent No. 4343553), and it is recorded in the peak signal position memory 40 after the end of the scanning operation. Further, the differential signals f i in the vicinity of the position of the maximum signal level [the range of (P max - n) - (P max + n)] are read out from the differential signal memory 39, and the operation of is executed. Then, the position P max ' of the maximum signal level can be precisely obtained. This position P max ' of the maximum signal level is stored in the peak signal position memory 40. When the image input operation corresponding to one line has ended in the above way, the scanning control circuit 30 positions the rotating mirror 25 to the next scanning line so as to repeat the image input operation of the next line. When one image frame has been input in the above-stated manner, image inputing ends. When the processing of one image frame has thus ended, a gradational image free from the slit-like light is stored in the image memory 38, and the positions of the brightest points found on the scanning lines struck by the slit-like light are stored in the peak signal position memory 40. When the contents of the peak signal position memory 40 are utilized, a distance from the three-dimensional vision sensor 11 to the peak position of the slit-like light can be detected on the basis of the principle of a light segmentation type range finder which is described in, for example, U. S. Patent Application No. 482730 (EP Application No. 83103402.0), U. S. Patent Application No. 537095 (EP Application No. 83109972.6), or Nakagawa et al, Structured light method for inspection of solder joints and assembly robot vision system: Proceeding of 1st International Symposium of Robotics Research Bretton Woods, NH, U. S. A., Aug. 28 - Sept. 2, 1983. When the detecting operations are performed as to the right and left slit-like light beams, the spatial position of the surface of the object 15 struck by the slit-like light beams can be determined. Here, as disclosed in U. S. Patent Application No. 582555 (corresponding to Japanese Patent Application Laying-open Nos. 59-154574 and 59-154578), two-dimensional image processing is performed on the gradational image memory 38 to extract the contour shape of the two-dimensional surface of the object 15, and the extracted contour shape of the two-dimensional surface is coordinate-transformed on the basis of the spatial position data of the three-dimensional surface extracted by utilizing the slit-like light as described before, whereby a three-dimensional surface shape can be obtained. More specifically, assuming that the positions (i₁, j₁), (i₂, j₂) and (i₃, j₃) of respective points P1, P2 and P3 on an image screen and the corresponding coordinates (x₁, y₁, z₁), (x₂, y₂, z₂) and (x₃, y₃, z₃) viewed from the three-dimensional vision sensor 11 have been known from the slit-like light segmentation, the coordinates (x, y, z) of a contour point P (i, j) are given by the following equations: x = x₁ + {(i - i₁) a i + (j - j₁) a j } z/z₁ (1) y = y₁ + {(i - i₁) b i + (j - j₁) b j } z/z₁ (2) z = z₁ + {(i - i₁) c i + (j - j₁) c j } z/z₁ (3) Here, b i , b j and c i , c j are respectively obtained by replacing x in Eqs. (4) and (5) with y and z . In the above way, the three-dimensional coordinates of the contour line can be found. Thus, the center and attitude of the surface can be found by utilizing diagrammatical primary and secondary moments, and the position and attitude of the object can be determined. When the data are sent to the robot control device 14 through the input/output interface circuit 43 and then given to the robot 13 as position data, the robot 13 can perform an operation such as gripping for the object 15. Referring now to Figs. 3(a) - 3(d), the basic principle of the slit-like light segmentation extraction according to the present invention will be described. In an image in Fig. 3(a), the sensor output of one scanning line 3 is schematically graphed as shown in Fig. 3(b). On the other hand, the sensor output of the scanning line produced without projecting slit-like light is as shown in Fig. 3(c). As illustrated in Fig. 3(d), when the difference is obtained between the two image outputs, a background image part almost consists of only noise components, and only a signal component based on the slit-like light segmentation can be clearly taken out. Although, in the above embodiment, the shift register 35 has a storage capacity corresponding to one raster, it may well be constructed so as to store signals of one image frame and to be fed with the signals of one image frame by revolving the rotating mirror 25 with the luminous diodes 21 put out. In this case, the signal distribution circuit 34 is changed-over, and the luminous diodes 21 are lit up, whereupon the slit-like light segmentation image corresponding to one image frame is input. Then, the background image read out from the shift register 35 is subtracted from the input light segmentation image by the differential signal extracting circuit 36. Thus, the differential signals can be extracted. As described above in detail, in an image input apparatus which applies the slit-like light segmentation system, components possessed by a background image can be removed from an image of slit-like light segmentation, so that segmentation extraction without the influence of the background image can be performed. In addition, even when the brightness of external light has fluctuated, the fluctuant component can be eliminated, so that the threshold value of the slit-like light segmentation extraction, etc. need not be changed, and stable extraction is possible. Accordingly, remarkable effects are attained for enhancing the precision, efficiency and stability of the image recognition processing of a vision sensor in a robot or a automaton.

Claims: A three-dimensional vision system, comprising a vision sensor (11) including means (21, 22) for projecting planar light beams (28) onto an object (15) to be handled and means (25, 26) for extracting two-dimensional images of said object both when said planar light beams are switched on and when they are off, an image input portion (41) including control means (30) for timely controlling the switching on and off of said planar light beams (28) and the sequential extraction of said two-dimensional images and means (34 - 36) for processing image signals representing said two-dimensional images when said planar light beams are switched on and when they are off in order to obtain a difference image and an image processor (42) for recognizing the three-dimensional shape of said object (15) through calculations performed on the basis of positions corresponding to the maximum intensity values in said difference image. A vision system according to claim 1, wherein said image input portion (41) comprises an image memory (38) for storing said two-dimensional image obtained without projecting said planar light beams (28) onto said object (15). A vision system according to claim 1, wherein said image input portion (41) comprises peak position detection means (37, 40) for analysing difference signals corresponding to a line of data in said difference image and for calculating the position at which the intensity values in said line peak.

Inventors: ARIGA, MAKOTO, HATA, SEIJI, OKABE, TAKAFUMI, OKADA, TAKUSHI
Applicants: HITACHI, LTD.
Application year: 1985
Main classification: G06K9
IPC codes: G01B11, G06K9