Source: https://patents.justia.com/patent/7576845
Timestamp: 2019-09-19 02:25:48
Document Index: 128321779

Matched Legal Cases: ['Application No. 2006', 'art 13', 'art 14', 'art 15', 'art 15', 'art 15', 'art 15', 'art 13', 'art 13', 'art 13', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 13', 'art 37', 'art 37', 'art 37', 'art 66', 'art 66', 'art 13', 'art 13', 'art 15', 'art 15', 'art 15', 'art 15', 'art 37', 'art 37', 'art 37', 'art 13', 'art 37', 'art 37', 'art 37', 'art 13', 'art 14', 'art 37', 'art 15', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 37', 'art 13', 'art 13', 'art 13', 'art 13', 'art 13', 'art 13', 'art 13', 'art 13', 'art 78', 'art 78', 'art 78', 'art 15', 'art 78', 'arts 78', 'art 15', 'art 15', 'art 78', 'art 15', 'art 37', 'art 78', 'art 15', 'art 78', 'art 15', 'art 15', 'art 86', 'art 15', 'art 86', 'art 15', 'art 15', 'art 15', 'art 37', 'art 15', 'art 15', 'art 15', 'art 15', 'art 15', 'art 15', 'art 37', 'art 15', 'art 37', 'art 15', 'art 13', 'art 14', 'art 37', 'art 15', 'art 37', 'art 37', 'art 15', 'art 15', 'art 15', 'art 14', 'art 13', 'art 37', 'art 15', 'art 15', 'art 15', 'art 37', 'art 37', 'art.\n6927862']

US Patent for Three-dimensional color and shape measuring device Patent (Patent # 7,576,845 issued August 18, 2009) - Justia Patents Search
Justia Patents Plural TestUS Patent for Three-dimensional color and shape measuring device Patent (Patent # 7,576,845)
Three-dimensional color and shape measuring device
Sep 30, 2008 - Brother Kogyo Kabushiki Kaisha
A three-dimensional color and shape measuring device is provided which measures a color and a three-dimensional shape of an object based on an image signal acquired by picking up an image of the object to be measured by the same image pick-up part. The device includes a correction part configured to convert the image signal into a color measuring image signal by a first correction in accordance with a first gray scale characteristic and to convert the image signal into a shape-measuring image signal by a second correction in accordance with a second gray scale characteristic; and a color and shape extracting part which is configured to restore a three dimensional image of the object by using a three-dimensional model on which a three dimensional shape of the object is restored based on the shape-measuring image signal and a color of the object is restored based on the color-measuring image signal.
The present application is a Continuation-in-Part of International Application PCT/JP2007/056720 filed on Mar. 28, 2007, which claims the benefits of Japanese Patent Application No. 2006-096096 filed on Mar. 30, 2006.
Generally, in such a three-dimensional color and shape measuring device, color tone of the three-dimensional color image of the object to be measured displayed on the display part is made to approximate the more real color tone by applying the gray scale correction which conforms to the display characteristic of a display part (herein after referred to as “gamma correction”) to an image signal acquired by picking up an image of the object to be measured by the pickup part.
In the above-mentioned conventional three-dimensional color and shape measuring device, since gamma correction is applied to the image signal acquired by picking up an image of the object to be measured, the color tone of the object to be measured is faithfully reproduced. On the other hand, however, with respect to the three-dimensional shape of the object to be measured, there exists a possibility that the shape may not be faithfully reproduced due to lowering of the measuring accuracy.
FIG. 1 is a perspective view showing an appearance of a three-dimensional color and shape measuring device according to a first embodiment;
FIG. 16 is a flow chart conceptually showing the processing as three-dimensional measuring processing sub routine executed in step S1220
FIG. 17 is a flow chart conceptually showing the processing as a coded image forming program 36d executed in step S1222 in FIG. 16;
Preferred embodiments of the present invention are explained in detail in conjunction with attached drawings.
FIG. 1 is a perspective view showing the appearance of a three-dimensional color and shape measuring device 1 according to the first embodiment of the present invention. The three-dimensional color and shape measuring device 1 is designed to perform the projection processing of image light (also referred to as “image signal light”) indicative of an image on a projection surface (for example, a planar surface, a screen, a desk surface or the like) (usual projection) and the projection processing of stripe-shaped patterned light on an object to be measured (for acquiring three-dimensional information), the image pick-up processing of the object to be measured, and the acquisition processing of three-dimensional information (containing three-dimensional positional information, three-dimensional shape information, color information) of the object to be measured based on an image-pick-up result using a computer. Accordingly, the three-dimensional color and shape measuring device 1 includes, as shown in FIG. 2, a projection part 13, an image pick-up part 14 and a processing part 15.
The release button switch 8 is operated by a user for operating the three-dimensional color and shape measuring device 1. The release button switch 8 is constituted of a two-stage push-button-type switch which can generate an instruction which differs between when the user operation state (push-down state) is a “half-pushed state” in which the user pushes the button switch by half and when the user operation state is a “full-pushed state” in which the user pushes the button switch fully. The operation state of the release button switch 8 is monitored by the processing part 15. When the processing part 15 detects the “half-pushed state”, a well-known auto focusing function (AF) and an automatic exposure function (AE) are generated and hence, focusing, stop and a shutter speed are automatically adjusted. On the other hand, when the processing part 15 detects the “full-pushed state”, an image pick-up operation or the like is performed.
The projection part 13 is a unit for projecting an arbitrary image light (projection image) or a patterned light on a projecting surface or an object to be measured. The projection part 13 includes, as shown in FIG. 2, the substrate 16, a plurality of LEDs (Light Emitting Diodes) 17 (herein after, an array thereof is referred to as “LED array 17A”), the light source lens 18, the projection LCD 19 and the projection optical system 20 which are arranged in series along the projecting direction. The detail of the projection part 13 is explained later in conjunction with FIG. 3.
As shown in FIG. 3A, the light source lens 18 includes a plurality of convex lens portions 18a, a base portion 18b which supports these lens portions 18a, an epoxy sealing material 18c and a plurality of positioning pins 18d.
As shown in FIG. 3A, the respective lens portions 18a are formed in a protruding manner toward the projection LCD 19 from the base portion 18b at positions of the base portion 18b where the lens portions 18a face the respective LEDs 17 of the LED array 17A. The epoxy sealing material 18c is filled in a concave portion 18e formed in the base portion 18b where the LED array 17A is to be hermetically stored. Due to such a filling operation, the LED array 17A is sealed in the concave portion 18e. The epoxy sealing material 18c has a function of sealing the LED array 17A and also has a function of adhering the substrate 16 and the light source lens 18 to each other.
As shown in FIG. 3A, the plurality of positioning pins 18d is formed on the light source lens 18 in a protruding manner toward the substrate 16 from the light source lens 18 for positioning the light source lens 18 and the substrate 16 relative to each other. As shown in FIG. 3B, some of the plurality of positioning pins 18d is inserted into elongated holes 16a formed in the substrate 16 while the remaining positioning pins 18d are inserted in true circular holes 16b formed in the substrate 16. Accordingly, the light source lens 18 is fixed to the substrate 16 without a play at a prescribed position.
Further, in this embodiment, the respective lens portions 18a are, as shown in FIG. 3A, arranged in a state that the respective lens portions 18a opposedly face the respective LEDs 17 of the LED array 17A under the 1 to 1 relationship. Accordingly, the radially extending light emitted from the respective LEDs 17 is efficiently collected by the respective lens portions 18 opposedly facing the respective LEDs 17 and, as shown in FIG. 3A, and is radiated to the projection LCD 19 as radiation light having high directivity.
The projection optical system 20 has telecentric property as described above and the incident NA is approximately 0.1 and hence, an optical path of the projection optical system 20 is regulated so as to allow only light at an angle within ±5° perpendicular to the projection optical system 20 to pass through the stop in the inside of the projection optical system 20.
Accordingly, in this embodiment, in combination with the constitution which allows only light which passes through the projection LCD 19 at an angle ±5° perpendicular to the projection LCD 19 to be projected on the projection optical system 20 due to the telecentric property of the projection optical system 20, it is possible to easily realize the enhancement of image quality.
Accordingly, in this embodiment, for enhancing the image quality, it is important to align a radiation angle of light radiated from each LED 17 such that the radiated light from the each LED 17 is incident on the projection LCD 19 approximately perpendicular to the projection lens 19 and, at the same time, most of the light radiated from each LED 17 is allowed to be incident on the projection optical system 20 within an incident angle range of ±5° perpendicular to the projection optical system 20.
In this manner, in this embodiment, since the plurality of pixels constituting the projection LCD 19 is arranged in a staggered array, the light to which space modulation is applied by the projection LCD 19 is controlled at an interval which is ½ of the pixel interval in the longitudinal direction of the projection LCD 19. In this manner, according to this embodiment, the projected patterned light can be controlled at a small interval and hence, it is possible to detect a three-dimensional shape of the object to be measured with high accuracy and high-resolution.
Further, in the stereoscopic image mode and the planation image mode described later for detecting the three-dimensional shape of the object to be measured, as shown in FIG. 1, stripe-shaped patterned light formed by alternatively arranging a bright portion and a dark portion is projected toward the object to be measured. In this embodiment, the patterned light is preliminarily defined in a state that the direction along which a plurality of stripes (bright portion or dark portion) (width direction of each stripe) is arranged in the patterned light agrees with the longitudinal direction of the projection LCD 19. Accordingly, in the patterned light, a boundary between the bright portion and the dark portion can be controlled at ½ pixel interval and hence, it is possible to detect the three dimensional shape of the object to be measured with high accuracy in the same manner.
Further, the CCD 22 is arranged on the longitudinal direction side of the projection LCD 19 (side in the extension direction of the pixel row). Accordingly, particularly, in the stereoscopic image mode or the planation image mode, when the three-dimensional shape of the object to be measured is detected by making use of the principle of triangulation, an inclination defined by the CCD 22 and the object to be measured can be controlled at ½ pixel interval and hence, it is possible to detect the three-dimensional shape of the object to be measured with high accuracy in the same manner.
Hereinafter, the arrangement of the LED array 17A is explained in further detail in conjunction with FIG. 4. FIG. 4A is a side view showing a three-dimensional shape of light radiated from the light source lens 18. FIG. 4B is a graph showing illuminance distribution of the light incident on an incident surface 19a of the projection LCD 19 from one LED 17. FIG. 4C is a front view showing the arrangement of the LED array 17A in a partially enlarged manner. FIG. 4D is a graph showing combined illuminance distribution of a plurality of lights incident on the incident surface 19a of the projection LCD 19 from the plurality of LEDs 17.
As shown in FIG. 4A, the light source lens 18 is designed so that, in a state that a half-maximum spreading half angle θ is approximately 5°, the light radiated from the light source lens 18 reaches the incident surface 19a of the projection LCD 19 as light having the illuminance distribution as shown in FIG. 4B.
Accordingly, in this embodiment, the combined illuminance distribution of light which passes through the light source lens 18 and reaches the incident surface 19a of the projection LCD 19 is expressed by an approximately straight line graph having small ripples in FIG. 4D. As a result, light is radiated to the whole incident surface 19a of the projection LCD 19 approximately uniformly. According to this embodiment, the illumination irregularities in the projection LCD 19 can be suppressed and, as a result, the image light having high quality is projected on the projection surface and, further, the patterned light having high quality is projected on the object to be measured.
The ROM 36 stores a camera control program 36a, a patterned light imaging program 36b, a brightness image forming program 36c, a coded image forming program 36d, a code boundary extraction program 36e, a lens aberration correction program 36f, a triangulation calculation program 36g, an document Posture calculation program 36h, and a plane conversion program 36i.
The camera control program 36a is executed for performing a control of the whole three-dimensional color and shape measuring device 1, and the control includes main processing which is conceptually described in a flow chart shown in FIG. 8.
The patterned light imaging program 36b is executed for picking up an image of the object to be measured to which patterned light is projected for detecting a three-dimensional shape of the document P thus acquiring a patterned light illuminated image and, further, for picking up an image of the object to be measured to which a patterned light is not projected thus acquiring a patterned light non-illuminated image.
The brightness image forming program 36c is executed for acquiring difference between the patterned light illuminated image and the patterned light non-illuminated image acquired with respect to the same object to be measured by the execution of the patterned light imaging program 36b and forming a brightness image indicative of the object to be measured on which the patterned light is projected.
The coded image forming program 36d is executed for forming a coded image in which a space code is allocated to every pixel based on a binary images formed by applying threshold processing of the plurality of respective brightness images formed by executing the brightness image forming program 36c.
To schematically explain the coded image forming program 36d, when the coded image forming program 36d is executed, the distance between the pattern lines in the brightness image of the object to be measured on which the patterned light having the smallest distance between the pattern lines out of the plural kinds of patterned lights is projected is acquired as a period, and the distribution of the period in the whole brightness image is acquired as the periodic distribution.
When the coded image forming program 36d is executed, further, a variable window which changes a size thereof is accordance with to the acquired periodic distribution is locally set to the brightness image for every patterned light and hence, a threshold value is locally calculated and set with respect to the whole brightness image by filtering processing using the variable window.
The code boundary extraction program 36e is executed for acquiring boundary coordinates of the code with accuracy of sub pixel order by making use of the coded image formed due to the execution of the coded image forming program 36d and the brightness image formed due to the execution of the brightness image forming program 36c.
The lens aberration correction program 36f is executed for performing the aberration correction of the image pick-up optical system 21 with respect to the boundary coordinates of the code acquired with accuracy of sub pixel order due to the execution of the code boundary extraction program 36e.
The triangulation calculation program 36g is executed for calculating the three-dimensional coordinates in a real space relating to the boundary coordinates from the boundary coordinates of the code to which the aberration correction is applied due to the execution of the lens aberration correction program 36f.
The document posture calculation program 36h is executed for estimating and acquiring a three-dimensional shape of the document P from the three-dimensional coordinates calculated due to the execution of the triangulation calculation program 36g.
The plane conversion program 36i is, based on the three-dimensional shape of the document P calculated due to the execution of the document Posture calculation program 36h, executed for forming a planation image such as an image formed when the document P is picked up from a front side.
Further, as shown in FIG. 5, to the RAM 37, a patterned light illuminated image storing part 37a, a patterned light non-illuminated image storing part 37b, a brightness image storing part 37c, a coded image storing part 37d, a code boundary coordinate storing part 37e, an aberration correction coordinate storing part 37g, a three-dimensional coordinate storing part 37h, an document Posture arithmetic operation result storing part 37i, a plane conversion result storing part 37j, a projection image storing part 37k, a working area 37l, a periodic distribution storing part 37p, a threshold value image storing part 37q and a binary image storing part 37r are respectively allocated as memory regions.
The patterned light illuminated image storing part 37a stores patterned light illuminated image data indicative of the patterned light illuminated image which is picked up due to the execution of the patterned light imaging program 36b. The patterned light non-illuminated image storing part 37b stores patterned light non-illuminated image data indicative of the patterned light non-illuminated image which is picked up due to the execution of the patterned light imaging program 36b.
The brightness image storing part 37c stores data indicative of the brightness image formed due to the execution of the brightness image forming program 36c. The coded image storing part 37d stores data indicative of the coded image formed due to the execution of the coded image forming program 36d. The code boundary coordinate storing part 37e stores data indicative of boundary coordinates of the respective code extracted with accuracy of subpixel order due to the execution of the code boundary extraction program 36e.
The aberration correction coordinate storing part 37g stores data indicative of the code boundary coordinates to which the aberration correction is applied due to the execution of the lens aberration correction program 36f. The three-dimensional coordinate storing part 37h stores data indicative of three-dimensional coordinates in a real space calculated due to the execution of the triangulation calculation program 36g.
The document Posture arithmetic operation result storing part 37i stores a parameter relating to the three-dimensional shape of the document P calculated due to the execution of the document Posture calculation program 36h. The plane conversion result storing part 37j stores data indicative of a plane conversion result acquired due to the execution of the plane conversion program 36i. The projection image storing part 37k stores information on the projection image which the projection part 13 projects to the object to be measured, that is, the patterned light. A working area 37l stores data which the CPU 35 temporarily uses for operating the CPU 35.
The periodic distribution storing part 37p, the threshold value image storing part 37q and the binary image storing part 37r store the data indicative of the periodic distribution, the threshold value image and the binary image acquired due to the execution of the coded image forming program 36d respectively.
The correction part 66 includes a first correction part which is constituted of a first correction circuit 64 for converting the image signal inputted from the A/D converter 63 into the color measuring image signal by first correction in accordance with a first gray scale characteristic, and an amplifier 64a which amplifies the color measuring image signal outputted by the first correction circuit 64 and outputs the amplified color measuring image signal to the main device. The correction part 66 also includes a second correction part which is constituted of a second correction circuit 65 for converting the image signal inputted from the A/D converter 63 into the shape measuring image signal by second correction in accordance with a second gray scale characteristic, and an amplifier 65a which amplifies the shape measuring image signal outputted by the second correction circuit 65 and outputs the amplified shape measuring image signal to the main device.
In the first correction part, the first correction circuit 64 and the amplifier 64a are operated to convert the image signal into the color measuring image signal and to output the color measuring image signal. When the patterned light is not projected by the projection part 13 which functions as a patterned light projection part, the first correction part applies the first correction to the image signal which the CCD 22 constituting the image pick-up part forms by picking up an image of the object to be measured and outputs in accordance with the first gray scale characteristic so as to convert the image signal into the color measuring image signal.
Further, in the second correction part, the second correction circuit 65 and the amplifier 65a are operated to output the shape measuring image signal. When the patterned light is projected by the projection part 13 which functions as a patterned light projection part, the first correction part applies the second correction to the image signal which the CCD 22 constituting the image pick-up part forms by picking up an image of the object to be measured and outputs in accordance with the second gray scale characteristic so as to convert the image signal into the shape measuring signal.
Further, the main device includes a selection part which selects either one of the color and shape measuring image signal and the shape measuring image signal outputted from the image pick-up unit 60 and outputs the selected image signal to the color and shape extracting part. Here, with respect to the selection of the image signal by the selection part, when the patterned light pickup program 36b is executed by the processing part 15, either one of the color and shape measuring image signal and the shape measuring image signal is selected based on a control performed by the CPU 35. That is, in the main device, the processing part 15 performs a function of the selection part.
To be more specific, in the main device, when the patterned light imaging program 36b is executed by the processing part 15 and the patterned light non-illuminated image is acquired, the first correction part and the main device are connected to each other based on a control performed by the CPU 35 whereby the color measuring image signal is selected.
On the other hand, in the main device, when the patterned light imaging program 36b is executed by the processing part 15 and the patterned light illuminated image is acquired, the second correction part and the main device are connected based on a control performed by the CPU 35 whereby the shape measuring image signal is selected.
Here, the camera control program 36a is explained in conjunction with FIG. 8. Due to the execution of the camera control program 36a by the computer, the main processing is executed.
Subsequently, in step S703a, the operation state of the release button switch 8 is scanned and, thereafter, in step S703b, based on the scanning result, it is determined whether or not the release button switch 8 assumes a half-pushed state. When the release button switch 8 assumes the half-pushed state, the determination is affirmative so that, in step S703c, an auto-focus (AF) function and an automatic exposure (AE) function are started and hence, focusing, stop and a shutter speed are adjusted. In step S703b, it is determined whether or not the release button switch 8 is moved to the half-pushed state. When the release button switch 8 is not shifted to the half-pushed state, the determination in step S703b is negative so that, the processing returns to step S703a.
After step S703c is executed, in step S703d, the operation state of the release button switch 8 is scanned again and, thereafter, in step S703e, based on the scanning result, it is determined whether or not the release button switch 8 assumes a fully-pushed state. In step S703e, it is determined whether or not the release button switch 8 is moved to the fully-pushed state. When the release button switch 8 is not moved to the fully-pushed state, the determination in step S703e is negative so that, the processing returns to step S703a.
When the release button switch 8 is shifted to the fully-pushed state from the half-pushed state, the determination in step S703e is affirmative so that, in step S704, it is determined whether or not a flash mode is selected. When the flash mode is selected, the determination is affirmative so that, in step S705, the flash 7 is allowed to emit light, while when the flash mode is not selected, the determination in step S704 is negative so that, step S705 is skipped.
Thereafter, in step S806, the pick-up image is stored in the projection image storing part 37k. Subsequently, in step S807, the projection processing described later is executed so as to project the image stored in the projection image storing part 37k on the projection surface.
FIG. 11 conceptually shows step S807 in FIG. 10 as a projection processing routine using a flow chart. Due to the execution of the projection processing routine, the projection processing which projects the image stored in the projection image storing part 37k on the projection surface from the projection part 13 is executed.
In this projection processing, first of all, in step S901, it is determined whether or not the image is stored in the projection image storing part 37k. When the image is not stored, the determination is negative so that, the projection processing this time is immediately finished. On the other hand, when the image is stored, the determination is affirmative so that, in step S902, the image stored in the projection image storing part 37k is transmitted to the projection LCD driver 30. Subsequently, in step S903, an image signal corresponding to the stored image is transmitted to the projection LCD 19 from the projection LCD driver 30 and hence, the image is displayed on the projection LCD 19.
In the stereoscopic image processing, first of all, in step S1001, a high resolution setting signal is transmitted to the CCD 22. Next, steps S1002 to S1003h are executed in the same manner as steps S702 to S706 shown in FIG. 9.
To be more specific, in step S1002, a finder image is displayed on the monitor LCD 10. Subsequently, in step S1003a, an operation state of the release button switch 8 is scanned and, thereafter, in step S1003b, based on the scanned result, it is determined whether or not the release button switch 8 assumes a half-pushed state. When the release button switch 8 assumes the half-pushed state, the determination is affirmative so that, in step S1003c, an auto-focus (AF) function and an automatic exposure (AE) function are started.
After processing in step S1003c is executed, in step S1003d, the operation state of the release button switch 8 is scanned again and, thereafter, in step S1003e, based on the scanned result, it is determined whether or not the release button switch 8 assumes a fully-pushed state.
When the release button switch 8 is shifted to the fully-pushed state from the half-pushed state, the determination in step S1003e is affirmative so that, in step S1003f, it is determined whether or not a flash mode is selected. When the flash mode is selected, the determination is affirmative so that, in step S1003g, the flash 7 is allowed to emit light, while when the flash mode is not selected, the determination in step S1003f is negative so that, step S1003g is skipped. In any case, thereafter, in step S1003h, an image of the object to be measured is picked up.
Thereafter, in step S1009, a polygonal figure which passes a plurality of measured vertexes which constitute the three-dimensional shape detection result is assumed, and a three-dimensional shape detection result image which is a stereoscopic image (three-dimensional computer graphic image) which expresses surfaces of the assumed polygonal figure is stored in the projection image storing part 37k.
Subsequently, in step S1010, the projection processing similar to the projection processing in step S807 shown in FIG. 10 is executed.
When the patterned light generated using each masks A, B, C is projected on the object to be measured, each one of eight fan-shaped regions is coded to either a bright region “1” or a dark region “0”. When the light which passes through three masks A, B, C is projected on the object to be measured sequentially in order of A, B, C, a code of three bits is allocated to each fan-shaped region. The three bits are sequentially arranged from the most significant bit MSB which corresponds to the first mask A to the least significant bit LSM which corresponds to the last mask C. For example, in the example shown in FIG. 13A, in the fan-shaped region to which the point P belongs, the light is blocked by the masks A and B, while the light is allowed to pass through only the mask C to form the fan-shaped region into a bright region and hence, the fan-shaped region is coded to a code as “001(A□0, B□0, C□1)”.
The detail of the space coding method is disclosed in, for example, “Kukankodoka ni yoru Kyorigazo Nyuryoku (Inputting of Distance Image using Spacing Coding)” in Denshi Tsushin Gakkai RonbunShi (Journal of Institute of Electronics and Communication Engineers of Japan), 85/3 Vol□J 68-D No 3 p 369 to 375, by Kohsuke Satoh and one other.
Next, in step S1212, due to the execution of the patterned light imaging program 36b, without projecting the patterned light on the object to be measured from the projection part 13, one patterned light non-illuminated image is acquired by picking up an image of the object to be measured by the image pick-up part 14.
The patterned light non-illuminated image acquired here is formed based on a color measuring image signal which is formed by amplifying the color measuring image signal to which the first correction is applied by the image pick-up unit 60 using the amplifier 64a. The patterned light non-illuminated image is stored in the patterned light non-illuminated image storing part 37b.
Next, in step S1213, the processing for changing over the selection part is executed. That is, in step S1213, the changeover of the selection part is performed based on a control performed by the CPU 35 of the processing part 15, and out of the two kinds of image signals (color measuring image signal and shape measuring image signal) outputted by the image pick-up unit 60, the shape measuring image signal is selected.
The patterned light illuminated image which is acquired this time is formed based on a shape measuring image signal formed by amplifying the shape measuring image signal to which the second correction is applied by the image pick-up unit 60 using the amplifier 65a. The acquired patterned light illuminated image is stored in the patterned light illuminated image storing part 37a in association with the corresponding pattern number PN.
In the three-dimensional measuring processing sub routine, first of all, in step S1221, due to the execution of the brightness image forming program 36c, a brightness image is formed. In step S1221, a brightness value is defined as a Y value in a YCbCr space and is calculated using a formula of Y=0.2989·R+0.5866·G+0.1145·B based on RGB values of each pixel. By acquiring the Y value with respect to each pixel, a plurality of brightness images in association with the plurality of patterned light illuminated images and one patterned light non-illuminated image is formed. The formed brightness images are stored in the brightness image storing part 37c in association with the pattern number PN. However, a formula used for calculating a brightness value is not limited to the above-mentioned formula and can be suitably exchanged with other formula.
Next, in step S1222, the coded image forming program 36d is executed. When the coded image forming program 36d is executed, by combining the plurality of formed brightness images using the above-mentioned space coding method, a coded image in which a space code is allocated to every pixel is formed. The coded image is formed using binary image processing which compares the brightness images in association with the plural kinds of patterned light illuminated images stored in the brightness image storing part 37c and a threshold image in which a brightness threshold value is allocated to every pixel. The formed coded image is stored in the coded image storing part 37d.
FIG. 17 conceptually shows the detail of the coded image forming program 36d using a flow chart. Hereinafter, the coded image forming program 36d is explained time-sequentially in conjunction with FIG. 17.
In the coded image forming program 36d, first of all, in step S101, the brightness image formed by picking up an image of the object to be measured on which the patterned light whose pattern number PN is 0 is projected is read as a representative patterned image from the brightness image storing part 37c.
Next, in step S102, with respect to the representative patterned image, based on the above-mentioned read brightness image, a pattern line period is calculated for each one of pixels which are continuously arranged in the columnar direction in the representative patterned image by the above-mentioned approach based on FFT conversion. The plurality of calculated pattern line periods is stored in the periodic distribution storing part 37p in association with the respective pixels (respective pixel positions in the columnar direction).
Thereafter, in step S104, with respect to the representative patterned image, the variable window VW is set in plane along the line direction and the columnar direction and in association with the respective pixels. Accordingly, an average value of the brightness values of the plurality of pixels which exist in the inside of the variable window VW is calculated as a local threshold value for every pixel. In step S105, further, a threshold image in which the calculated threshold values are allocated to the respective pixels is formed. The formed threshold value image is stored in the threshold value image storing part 37q.
Subsequently, in step S105, the pattern number PN is initialized to 0 and, thereafter, in step S106, it is determined whether or not the present value of the pattern number PN is smaller than the maximum value PNmax. Since the present value of the pattern number PN is 0 this time, the determination is affirmative so that the processing advances to step S107.
In step S109, for every pixel, a pixel value (“1” or “0”) is extracted from the binary images which is equal to the maximum value PNmax in number in order from the binary image corresponding to the brightness image whose pattern number PN is 0 to the binary image corresponding to the brightness image whose pattern number PN is (PNmax−1), and space codes which are arranged in order from the least significant bit LSM to the most significant bit MSB are generated. The number of bits of the space code for every pixel is equal to the number of the maximum value PNmax. The space code is generated for every pixel and hence, the space coded image corresponding to the object to be measured of this time is formed. The generated space codes are stored in the coded image storing part 37d in association with the respective pixel positions.
In the binary image forming sub routine, first of all, in step S111, an image is read from the brightness image storing part 37c and, subsequently, in step S112, the binarization processing is performed.
To be more specific, a brightness value of the brightness image to which the pattern number PN equal to the present value of the pattern number PN is allocated and the threshold value of the above-mentioned formed threshold image are compared to each other for every pixel. A comparison result is reflected to the binary image for every pixel. To be more specific, when the brightness value of the brightness image is larger than the threshold value, data indicative of “1” is stored in the binary image storing part 37r in association with the corresponding pixel position in the binary image and, while when the brightness value of the brightness image is not larger than the threshold value, data indicative of “0” is stored in the binary image storing part 37r in association with the corresponding pixel position in the binary image.
With the above-mentioned steps, the execution of one-round coded image forming program 36d is finished.
Thereafter, in step S1223 in FIG. 16, due to the execution of the code boundary extraction program 36e, the code boundary coordinate detection processing is performed. The coding using the above-mentioned space coding method is performed for every pixel unit and hence, there arises an error in accuracy of sub pixel order between the boundary line between a bright portion and a dark portion in actual patterned light and a boundary line of space codes in the above-mentioned formed coded image (boundary line between a region to which one space code is allocated and a region to which another space code is allocated). Accordingly, the code boundary coordinate detection processing is provided for detecting the boundary coordinate values of the space codes with accuracy of sub pixel order.
The detected code boundary coordinate value is stored in the code boundary coordinate storing part 37e. The code boundary coordinate value is defined by CCD coordinates ccdx-ccdy which is a two-dimensional coordinate system set on an imaging surface of the CCD 22.
Subsequently, in step S1224, due to the execution of the lens aberration correction program 36f, the lens aberration correction processing is performed. The lens aberration correction processing is executed for correcting an actual image-forming position of the optical flux incident on the image pick-up optical system 21 which is influenced by aberration of the image pick-up optical system 21 in a state that the actual image-forming position approaches an ideal image-forming position where the image is expected to be formed when the image pick-up optical system 21 is an ideal lens.
Due to the lens aberration correction processing, the code boundary coordinate value detected in step S1223 is corrected so that an error attributed to distortion of the image pick-up optical system 21 or the like is corrected. The code boundary coordinates corrected in this manner are stored in the aberration correction coordinate storing part 37g.
Both of the code boundary coordinate detection processing and the lens aberration correction processing are not prerequisite for understanding the present invention and are disclosed in detail in the specification of JP-A-2004-105426 by the applicant of the present invention. Accordingly, the detailed explanation of the code boundary coordinate detection processing and the lens aberration correction processing is omitted here by reference to JP-A-2004-105426.
Thereafter, in step S1225, due to the execution of the triangulation calculation program 36g, the real space conversion processing using the principle of triangulation is performed. When the real space conversion processing is performed, using the principle of triangulation, the above-mentioned code boundary coordinate value on the CCD coordinate system ccdx-ccdy to which the aberration correction is applied is converted into the three-dimensional coordinate value on the real space coordinate system X-Y-Z which is a three-dimensional coordinate system set in the real space and, as a result, the three-dimensional coordinate value is acquired as a three-dimensional shape detection result. The acquired three-dimensional coordinate value is stored in the three-dimensional coordinate storing part 37h, and the three-dimensional measuring processing is finished and, thereafter, the processing in step S1230 shown in FIG. 14 is performed.
In the three-dimensional-color-shape detection result generating processing sub routine, first of all, in step S5501, a plurality of three-dimensional coordinate values is loaded from the three-dimensional coordinate storing part 37h. In this embodiment, the whole external surface of the object to be measured is divided into four partial surfaces (front surface, right surface, left surface and back surface), and a stereoscopic image is formed for each partial surface. In step S5501, with respect to all four partial surfaces, a plurality of three-dimensional coordinate values which belongs to each partial surface are loaded from the three-dimensional coordinate storing part 37h.
Next, in step S5502, based on the plurality of loaded three-dimensional coordinate values (vertex coordinate values), a plurality of three-dimensional coordinate values which belongs to the four partial surfaces are combined to each other. As a result, the four partial surfaces which are three-dimensionally expressed using the plurality of three-dimensional coordinate values are integrated whereby an image expressing the whole outer surface of the object to be measured is synthesized.
Subsequently, in step S6002, for every vertex, the corresponding real space coordinate value and the RGB values are combined to form color-shape information. Further, the generated color-shape information is locally stored in a working area 37l directly or indirectly in association with the corresponding vertex.
Thereafter, in step S6004, to form the polygon for each polygon, the combination of three vertexes to be connected with each other is locally stored in the working area 37l as polygon information directly or indirectly in association with each polygon.
In the real space coordinate system X-Y-Z, a projection angle from the projection part 13 to the document P is indicated by “θp”, and a distance between the optical axis of the image pick-up optical system 21 and the optical axis of the projection part 13 is indicated by “D”. The projection angle θp is univocally specified by a space code allocated for every pixel.
Further, in the real space coordinate system X-Y-Z, a Y coordinate value and an X coordinate value of an intersection between a straight line which is an extension opposite to an optical path through which reflection light from an object point target on the document P is incident on the CCD 22 and an X-Y plane are respectively indicated by “Ytarget” and “Xtarget”. In the real space coordinate system X-Y-Z, further, a viewing field of the image pick-up optical system 21 in the Y direction is defined as a region ranging from a point indicated by “Yftop” to a point indicated by “Yfbottom”, and a viewing field of the image pick-up optical system 21 in the X direction is defined as a region ranging from a point indicated by “Xfstart” to a point indicated by “Xfend”. Further, a length (height) of the CCD 22 in the Y-axis direction is indicated by “Hc”, and a length (width) of the CCD 22 in the X-axis direction is indicated by “Wc”.
(a) an object point target (X, Y, Z) on the document P (indicated by a leader line as “(a)” in FIG. 20)
(b) an input pupil position of the image pick-up optical system 21 (indicated by a leader line as “(b)” in FIG. 20)
(c) an output pupil position of the projection optical system 20 (indicated by a leader line as “(c)” in FIG. 20)
(d) an intersection (Xtarget□Ytarget) between a straight line which passes the input pupil position of the image pick-up optical system 21 and the object point on the document P and the X-Y plane (indicated by a leader line as “(d)” in FIG. 20)
(e) an intersection between a straight line which passes the output pupil position of the projection optical system 20 and the object point on the document P and the X-Y plane (indicated by a leader line as “(e)” in FIG. 20)
Y=(PPZ−Z)·tan θp−D+cmp(Xtarget) (1)
Y=−(Ytarget/VPZ)Z+Ytarget (2)
X=−(Xtarget/VPZ)Z+Xtarget (3)
Ytarget=Yftop−(ccdcy/Hc)×(Yftop−Yfbottom) (4)
Xtarget=Xfstart+(ccdcx/Wc)×(Xfend−Xfstart) (5)
Here, “cmp(Xtarget)” in formula (1) is a function of correcting the displacement between the image pick-up optical system 21 and the projection part 13, and in an ideal case, that is, when there is no displacement between the image pick-up optical system 21 and the projection part 13, “cmp(Xtarget)” assumes 0.
ccdcx=(ccdx−Centx)/(1+dist/100)+Centx (6)
ccdcy=(ccdy−Centy)/(1+dist/100)+Centy (7)
hfa=arctan [(((ccdx−Centx)2+(ccdy−Centy)2)0.5)×pixel length/focal length] (8)
Here, an aberration quantity dist (%) is described as “dist=f(hfa)” using a function f of half angle of view hfa (deg). Further, a focal length of the image pick-up optical system 21 is indicated by “focal length(mm)”, a ccd pixel length is indicated by “pixel length(mm)”, and a coordinate value of the center of the lens in the CCD 22 is defined as “(Centx, Centy)”.
Y=−(Yptarget/PPZ)Z+Yptarget (9)
X=−(Xptarget/PPZ)Z+Xptarget (10)
Yptarget=Ypftop−(lcdcy/Hp)×(Xpftop−Xpfbottom) (11)
Xptarget=Xpfstart+(lcdcx/Wp)×(Xpfend−Xpfstart) (12)
Here, in the real space coordinate system X-Y-Z, as shown in FIG. 20, a Y coordinate value and an X coordinate value of an intersection between a straight line which is an extension in the same direction as a optical path through which an optical flux is incident on the object point target on the document P from the projection part 13 and an X-Y plane are respectively indicated by “Yptarget” and “Xptarget”. Further, an output pupil position of the projection part 13 is defined as (0, 0, PPZ). Further, a viewing field of the projection part 13 in the Y direction is defined as a region ranging from a point indicated by “Ypftop” to a point indicated by “Ypfbottom”, and a viewing field of the projection part 13 in the X direction is defined as a region ranging from a point indicated by “Xpfstart” to a point indicated by “Xpfend”. Further, a length (height) of the projection LCD 19 in the Y axis direction is indicated by “Hp”, and a length (width) of the projection LCD 19 in the X axis direction is indicated by “Wp”.
Next, a second embodiment of the present invention is explained. The three-dimensional color and shape measuring device according to the second embodiment has the basic structure which is substantially equal to the basic structure of the three-dimensional color and shape measuring device 1 of the first embodiment shown in FIG. 1 to FIG. 4, but differs from the three-dimensional color and shape measuring device 1 of the first embodiment only in the constitution of the image pick-up unit and the processing by the processing part associated with the constitution of the image pick-up unit.
Further, the image pick-up unit 70 includes a preceding-stage selection part 78a which selects either one of the first correction part and the second correction part and outputs the image signal to the selected correction part, and a succeeding-stage selection part 78b which selects either one of the color measuring image signal which the first correction part outputs and the shape measuring image signal which the second correction part outputs and outputs the selected signal to the main device.
The preceding-stage selection part 78a, when a patterned light imaging program 36b is executed by the processing part 15 of the main device so that the patterned light non-illuminated image is acquired, selects the first correction part based on a control performed by the CPU 35 so as to connect the A/D converter 73 and the first correction part, while the succeeding-stage selection part 78b selects the color measuring image signal based on a control performed by the CPU 35 and outputs the color measuring image signal to the main device.
In this manner, by providing the correction part which includes the first correction part, the second correction part and the selection parts 78a, 78b to the image pick-up unit 70 of the second embodiment, the color measuring image signal and the shape measuring image signal are outputted to the main device by alternately operating the first correction part and the second correction part and hence, different from the constitution shown in FIG. 5, it is unnecessary to connect the image pick-up unit 60 and the processing part 15 using two signal lines and hence, the circuit constitution can be simplified.
FIG. 22 is a flow chart showing image pick-up processing executed in the processing part 15 of the second embodiment. As shown in FIG. 22, in this image pick-up processing, first of all, in step S1211a, the setting processing by the selection part in the preceding-stage side and the succeeding-stage side are executed. That is, in step S1211a, the selection part 78a in the preceding-stage side in the inside of the image-pickup unit 70 shown in FIG. 21 selects the first correction part and connects the A/D converter 73 and the first correction circuit 74 based on a control performed by the CPU 35 of the processing part 15.
The patterned light non-illuminated image acquired here is formed based on a color measuring image signal to which the first correction is applied by the image pick-up unit 70. The patterned light non-illuminated image is stored in the patterned light non-illuminated image storing part 37b.
Next, in step S1213a, the processing for changing over the selection part in the preceding-stage side and the selection part in the succeeding-stage side is executed. That is, in this step S1213a, the selection part 78a in the preceding-stage side in the inside of the image-pickup unit 70 shown in FIG. 21 selects the second correction part and connects the A/D converter 73 and the amplifier 77 based on a control performed by the CPU 35 of the processing part 15.
Simultaneously with such processing, the succeeding-stage selection part 78b in the image pick-up unit 70 shown in FIG. 21 selects the amplifier 77 of the second correction part and the shape measuring image signal outputted from the amplifier 77 is outputted to the main device.
After the processing in step S1213a is finished, in the same manner as the first embodiment, the processing in step S1214 to S1219 are sequentially executed thus finishing the image pick-up processing.
In this manner, according to the processing part 15 which the three-dimensional color and shape measuring device 1 of the second embodiment possesses, by merely changing the processing executed in steps S1211 and S1213 in the image pick-up processing sub routine of the first embodiment shown in FIG. 15 with the processing executed in steps S1211a and S1213a in the image pick-up processing sub routine shown in FIG. 22, it is possible to operate the image pick-up unit 70 shown in FIG. 21 without largely changing the whole program executed by the processing part 15.
Next, a third embodiment of the present invention is explained. The three-dimensional color and shape measuring device according to the third embodiment has the basic structure which is substantially equal to the basic structure of the three-dimensional color and shape measuring device 1 of the first embodiment shown in FIG. 1 to FIG. 4, but differs from the three-dimensional color and shape measuring device 1 of the first embodiment only in the constitution of the image pick-up unit and the processing by the processing part associated with the constitution of the image pick-up unit. (0246)
Further, in this correction part 86, when a patterned light imaging program 36b is executed by the processing part 15 of the main device so that the patterned light non-illuminated image is acquired, the correction circuit 84 is operated based on a control performed by the CPU 35 so that the first correction is applied to the image signal using the first correction LUT provided to the correction circuit 84 and, the color measuring image signal generated as a result of such an operation is outputted to the main device.
On the other hand, in this correction part 86, when a patterned light imaging program 36b is executed by the processing part 15 of the main device so that the patterned light illuminated image is acquired, the correction circuit 84 is operated based on a control performed by the CPU 35 so that the second correction is applied to the image signal using the second LUT provided to the correction circuit 84 and, the shape measuring image signal generated as a result of such an operation is outputted to the main device.
FIG. 24 is a flow chart showing image pick-up processing executed in the processing part 15 of the third embodiment. As shown in FIG. 24, in this image pick-up processing, first of all, in step S1211b, the setting processing of the LUT is executed. That is, in step S1211b, the first correction LUT to be referenced in an image pick-up unit 80 shown in FIG. 23 is set based on a control performed by the CPU 35 of the processing part 15.
The patterned light non-illuminated image acquired here is formed based on a color measuring image signal to which the first correction is applied by the image pick-up unit 80. The patterned light non-illuminated image is stored in the patterned light non-illuminated image storing part 37b.
Next, in step S1213b, the replacement processing of the LUT is executed. That is, in step S1213b, the LUT referenced in the image-pickup unit 80 shown in FIG. 23 is replaced from the first correction LUT to the second correction LUT based on a control performed by the CPU 35 of the processing part 15.
After the processing in step S1213b is finished, in the same manner as the first embodiment, the processing in step S1214 to S1219 are sequentially executed thus finishing the image pick-up processing.
In this manner, according to the processing part 15 which the three-dimensional color and shape measuring device 1 of the third embodiment possesses, by merely replacing the processing executed in steps S1211 and S1213 in the image pick-up processing sub routine of the first embodiment shown in FIG. 15 with the processing executed in steps S1211b and S1213b in the image pick-up processing sub routine shown in FIG. 24, it is possible to operate the image pick-up unit 80 shown in FIG. 23 without largely changing the whole program executed by the processing part 15.
Next, a fourth embodiment of the present invention is explained. The three-dimensional color and shape measuring device according to the fourth embodiment has the basic structure which is substantially equal to the basic structure of the three-dimensional color and shape measuring device 1 of the first embodiment shown in FIG. 1 to FIG. 4, but differs from the three-dimensional color and shape measuring device 1 of the first embodiment only in the constitution of the image pick-up unit, a program stored in the ROM of the processing part and the processing by the processing part associated with the constitution of the image pick-up unit.
Further, in the main device of the fourth embodiment, for applying the third correction to the color measuring image signal inputted from the image pick-up unit 90 as described above, as shown in FIG. 26, a gray scale correction program 36j and a gray scale correction LUT 36k are stored in the ROM 36 provided to the processing part 15 in addition to the various programs 36a to 36i stored in the ROM 36 of the first embodiment.
This gray scale correction program 36j is executed for applying the third correction in accordance with the third gray scale characteristic to the color measuring image signal outputted from the image pick-up unit 90 and, thereafter, is executed for converting the color measuring image signal into the shape measuring image signal after amplification of the signal.
In this manner, since the gray scale correction program 36j is stored in the ROM 36, even when the image pick-up unit 90 which is preliminarily provided with only the correction circuit which performs the first correction as the correction part is mounted on the three-dimensional color and shape measuring device, due to the execution of the gray scale correction program 36j, the third correction can be applied to the color measuring image signal outputted from the image pick-up unit 90 thus generating the shape measuring image signal capable of measuring the three-dimensional shape of the object to be measured with high accuracy.
Further, the gray scale correction LUT 36k is a look-up table which is stored in a state that the color measuring image signal before the third correction and the shape measuring image signal acquired after the third correction are made to correspond to each other, and the gray scale correction LUT 36k is referenced by the CPU 35 when the gray scale correction program 36j is executed by the processing part 15. That is, the above-mentioned third correction is performed by making use of this third correction LUT.
In the three-dimensional color and shape measuring device 1 of the fourth embodiment having such a constitution, when a patterned light imaging program 36 bis executed by the processing part 15 of the main device so that the patterned light non-illuminated image is acquired, the patterned light non-illuminated image is formed based on the color measuring image signal outputted from the image pick-up unit 90 and the patterned light non-illuminated image is stored in the patterned light non-illuminated image storing part 37b based on a control performed by the CPU 35.
On the other hand, when a patterned light imaging program 36b is executed by the processing part 15 of the main device so that the patterned light illuminated image is acquired, the gray scale correction program 36j is executed and the third correction circuit 96 is operated based on a control performed by the CPU 35 so that the third correction is applied to the color measuring image signal outputted by the image pick-up unit 90, and the patterned light illuminated image is formed based on the shape measuring image signal acquired as a result of the processing of the gray scale correction program 36j, and the patterned light illuminated image is stored in the patterned light illuminated image storing part 37a.
That is, the color extracting part is configured to restore the color of the object to be measured directly using the color measuring image signal outputted from the image pick-up unit 90 in restoring the color of the object to be measured, and to measure and restore the three-dimensional shape of the object to be measured using the shape measuring image signal generated by applying the inverse gamma correction to the color measuring image signal outputted from the image pick-up unit 90 in measuring and restoring the three-dimensional shape of the object to be measured.
FIG. 27 is a flow chart showing an image pick-up processing executed in the processing part 15 of the fourth embodiment. As shown in FIG. 27, in this image pick-up processing, first of all, in step S1212, due to the execution of the patterned light imaging program 36b, without projecting the patterned light on the object to be measured from the projection part 13, one patterned light non-illuminated image is acquired by picking up an image of the object to be measured by the image pick-up part 14.
The patterned light non-illuminated image acquired here is formed based on a color measuring image signal which is acquired by amplifying a color measuring image signal to which the first correction is applied in the image pick-up unit 90. The acquired patterned light non-illuminated image is stored in the patterned light non-illuminated image storing part 37b.
Next, in step S1213c, the third correction processing is executed. That is, in step S1213c, due to the execution of the gray-scale correction program 36j, gray scale correction LUT 36k is referenced by the CPU 35, the third correction is applied to the color measuring image signal inputted from the image pick-up unit 90 and, thereafter, the color measuring image signal is amplified and is converted into the shape measuring image signal.
After the processing in step S1213c is finished, in the same manner as the first embodiment, the processing in steps S1214 to S1219 are sequentially executed thus finishing the image pick-up processing.
As a modification of this fourth embodiment, the processing executed by the processing part 15 in performing the third correction in the coded image forming processing is explained. In this case, the third correction processing in step S1213c shown in FIG. 27 is deleted, and the third correction processing is executed in the coded image forming processing shown in FIG. 28 and in the binary image forming processing shown in FIG. 29.
In the coded image forming processing, by executing the coded image forming program 36d as shown in FIG. 28, first of all, in step S101, the brightness image formed by picking up the object to be measured on which the patterned light whose pattern number PN is 0 is projected is read as a representative patterned image from the brightness image storing part 37c.
Next, in step S101a, the third correction processing is executed. That is, by executing the gray scale correction program 36j, the gray scale correction LUT 36k is referenced by the CPU 35 and the processing for applying the third correction to the image signal (color measuring image signal) indicative of the representative pattered image is executed.
After finishing of the processing in step S101a, in the same manner as the first embodiment, processing in steps S102 to S106 are sequentially executed. When it is determined that the present value of the pattern number PN is smaller than the maximum value PNmax in step S106, the third correction processing is performed in step S107 (processing for forming a binary image). On the other hand, if it is determined that the present value of the pattern number PN is larger than the maximum value PNmax in step S106, the processing in step S109 (processing for forming coded image) is performed in the same manner as the first embodiment, and the coded image forming processing is finished.
In the binary image forming processing, as shown in FIG. 29, an image is read from the brightness image storing part 37c in step S111 and, thereafter, the third correction processing is performed in step S111a. That is, by executing the gray scale correction program 36j, the gray scale correction LUT 36k is referenced by the CPU 35, and the processing for applying the third correction to the image signal (color measuring image signal) indicative of the image read in step S111 is performed.
After finishing such processing in step S111a, the binary processing S112 similar to the binary processing in the first embodiment is executed thus finishing the binary processing.
Next, a fifth embodiment of the present invention is explained. The three-dimensional color and shape measuring device according to the fifth embodiment has the basic structure which is substantially equal to the basic structure of the three-dimensional color and shape measuring device 1 of the first embodiment shown in FIG. 1 to FIG. 4, but differs from the three-dimensional color and shape measuring device 1 of the first embodiment only with respect to the constitution of the image pick-up unit, a program stored in the ROM of the processing part and the processing by the processing part.
Further, in the main device of the fifth embodiment, for applying the first correction and the second correction to the image signal inputted from the image pick-up unit 50 as mentioned above, in the same manner as the processing part 15 of the fourth embodiment shown in FIG. 26, a gray scale correction program 36j and a gray scale correction LUT 36k are stored in the ROM 36 in addition to the respective programs 36a to 36i stored in the ROM 36 of the first embodiment.
Here, the gray scale correction program 36j and the gray scale correction LUT 36k in the fifth embodiment have contents different from the contents of the gray scale correction program 36j and the gray scale correction LUT 36 kin the fourth embodiment.
That is, the gray scale correction program 36j in the fifth embodiment is constituted of a first correction program executed at the time of applying the first correction to the image signal and a second correction program executed at the time of applying the second correction to the image signal.
Further, the gray scale correction LUT 36k in the fifth embodiment is constituted of a first correction LUT and a second correction LUT.
Further, the second correction LUT is a look-up table which is stored in a state that the image signal before the second correction and the shape measuring image signal acquired after the second correction are made to correspond to each other, and is referenced by the CPU 35 when the second correction program 36k is executed by the processing part 15. That is, the above-mentioned second correction is performed by making use of this second correction LUT.
FIG. 31 is a flow chart showing image pick-up processing executed in the processing part 15 of the fifth embodiment. As shown in FIG. 31, in this image pick-up processing, first of all, the first correction processing is executed in step S1211d. That is, in step S1211d, the selection part selects, upon execution of the first correction program, the first correction part and, at the same time, the first correction LUT is referenced by the CPU 35, and by applying the first correction to the image signal outputted from the image pick-up unit 50, a color measuring image signal is generated.
Next, in step S1212, upon execution of the patterned light imaging program 36b, one patterned light non-illuminated image is acquired by picking up an image of the object to be measured by the image pick-up part 14 without projecting the patterned light on the object to be measured from the projection part 13.
The patterned light non-illuminated image acquired here is formed based on the color measuring image signal to which the first correction is applied by the first correction circuit. The patterned light non-illuminated image is stored in the patterned light non-illuminated image storing part 37b.
Next, in step S1213d, the second correction processing is executed. That is, in step S1213d, upon execution of the second correction program, the selection part selects the second correction part and, at the same time, the second correction LUT is referenced by the CPU 35, and by applying the second correction to the image signal outputted by the image pick-up unit 50, a shape measuring image signal is generated.
After processing in step S1213d is finished, in the same manner as the first embodiment, the processing in step S1214 to S1219 are sequentially executed thus finishing the image pick-up processing.
In this manner, according to the processing part 15 which the three-dimensional color and shape measuring device 1 of the fifth embodiment possesses, by merely replacing the processing executed in steps S1211 and S1213 in the image pick-up processing sub routine of the first embodiment shown in FIG. 15 with the processing executed in steps S1211d and S1213d in image pick-up processing subroutine shown in FIG. 31, it is possible to operate the image pick-up unit 50 shown in FIG. 30 without largely changing the whole program executed by the processing part 15.
Here, as a modification of the fifth embodiment, the explanation is made with respect to processing executed by the processing part 15 in performing the second correction in the coded image forming processing and performing the first correction in the three-dimensional color shape detection result generating processing. In such a case, the first correction processing in step S1211d shown in FIG. 31 is deleted, the first correction is performed in the three-dimensional color shape detection result generating processing shown in FIG. 34, the second correction processing in step S1213d shown in FIG. 31 is deleted, and the second correction processing is performed in the coded image forming processing shown in FIG. 32 and in the binary image forming processing shown in FIG. 33.
In the coded image forming processing, by executing the coded image forming program 36d as shown in FIG. 32, first of all, in step S101, the brightness image formed by picking up the object to be measured on which the patterned light whose pattern number PN is 0 is projected is read as a representative patterned image from the brightness image storing part 37c.
Next, in step S101b, the second correction processing is executed. That is, by executing the second correction program in step 101b, the second correction LUT is referenced by the CPU 35 and the processing for applying the second correction to the image signal indicative of the representative pattered image is executed.
After finishing the processing in step S101b, in the same manner as the first embodiment, processing in steps S102 to S106 are sequentially executed. When it is determined that the present value of the pattern number PN is smaller than the maximum value PNmax, the second correction processing is performed in step S107 (processing for forming a binary image). On the other hand, if it is determined that the present value of the pattern number PN is larger than the maximum value PNmax, the processing in step S109 (processing for forming coded image) is performed in the same manner as the first embodiment, and the coded image forming processing is finished.
In the binary image forming processing, as shown in FIG. 33, an image is read from the brightness image storing part 37c in step S111 and, thereafter, the second correction processing is performed in step S111b. That is, by executing the second correction program in step S111b, the second correction LUT is referenced by the CPU 35, and the processing for applying the second correction to the image signal indicative of the image read in step S111 is performed.
After finishing such processing in step S111b, the binary processing S112 similar to the binary processing in the first embodiment is executed thus finishing the binary processing.
Further, in the three-dimensional color shape detection result generating processing, as shown in FIG. 34, in the same manner as the three-dimensional color shape detection result generating processing of the first embodiment, processing in steps S5501 to S6001 are sequentially performed and, thereafter, the first correction processing is performed in step S6001a.
That is, by executing the first correction program in step S6001a, the first correction LUT is referenced by the CPU 35, and the processing for applying the first correction to the RGB values of the patterned light non-illuminated image extracted in step S6001 is performed.
After finishing such processing in step S6001a, the processing in steps S6002 to S6004 are sequentially executed thus finishing the three-dimensional color and shape detection result generation processing.
Although some embodiments of the present invention have been explained in detail in conjunction with the drawings, these embodiments are merely examples and the present invention may be performed in other modes including various modifications and improvements as well as the mode described in “Summary of the Invention” based on knowledge of those who are skilled in the art.
6927862 August 9, 2005 Iwasaki
20030107747 June 12, 2003 Iwasaki
20070031029 February 8, 2007 Sasaki
1151778 July 2001 EP
H05-137011 June 1993 JP
2001-166809 June 2001 JP
2003-172610 June 2003 JP
2005043233 February 2005 JP
2005-258622 September 2005 JP
2005293075 October 2005 JP
0048699 August 2000 WO
2005095886 October 2005 WO
2005096126 October 2005 WO
2007114189 October 2007 WO
Patent number: 7576845
Patent Publication Number: 20090046301
Inventors: Hirotaka Asakura (Nagoya), Kenji Natsuhara (Nagoya)
Application Number: 12/242,778
Current U.S. Class: Plural Test (356/73); Color Image Processing (382/162); Pattern Recognition Or Classification Using Color (382/165)