Abstract:
A structured light pattern is projected onto a scene using pulses of light emitted by light emitting diodes (LEDs) of the projector, wherein during the pulses of the LEDs, the LEDs are driven at an overdrive current when the LEDS are ON. A sensor of a camera only integrates light from the scene during the pulses to acquire images of the scene.

Description:
FIELD OF THE INVENTION 
     This invention relates generally to determining shapes of objects using cameras, and more particularly to using structured light to determine 3D shapes in scenes. 
     BACKGROUND OF THE INVENTION 
     3D Scanning 
     Three-dimensional (3D) points in a scene can be determined using stereoscopic techniques, e.g., from stereo images acquired by two cameras. That technique depends on relating a pixel in one camera to the pixel in the other camera that corresponds to the same point in the 3D scene. However, it may be difficult to establish correspondences between the two cameras if the points are on a uniform surface. This makes it unclear exactly which of the many nearly identical pixels in one image correspond to a specific pixel in the other image. 
     SLS 
     For this reason, some 3D scanning techniques use a camera and a projector as a structured light scanner (SLS). The projector operates analogously to a camera, in the sense that 3D points in the scene can be associated with pixels on the projector image plane. Because the projector patterns are controlled by the SLS, it is easier to determine corresponding projector and camera pixels. The SLS becomes more compact as cameras and projectors decrease in size, e.g., the SLS can be arranged on a robot arm to assist automated machine operations, such as parts picking and assembly. 
     DLP 
     Many projectors use digital light projection (DLP). A DLP includes an array of digital micro mirrors (DMD) that can be switched independently to ON and OFF positions. Because the mirrors can switch rapidly, it is possible to use pulse width modulation (PWM) to project apparently intermediate light levels using different proportions of the “ON” and “OFF” times. 
     A pico-projector can enable a very small SLS. Pico-projectors typically use LEDs, which are not as bright as conventional projector bulbs. Lower brightness makes intensity (color) distinction difficult in many SLS applications, in particular when the scanned scene is dominated by ambient light because the ambient light makes the contrast in the structured light pattern low. 
     Strobing 
     The rapid switching ON and OFF is similar to strobing, which appears to stop motion. Strobing produces a very brief pulse of bright light. 
     SUMMARY OF THE INVENTION 
     A projector projects a structured light pattern onto a scene by using pulses of light emitted by light emitting diodes (LEDs) of the projector. The pulses of the LEDs are driven at an overdrive current when the LEDS are ON. A camera to integrate light from the scene by a sensor of the camera only during the pulses to acquired images of the scene and the structured light pulses. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWING 
         FIG. 1  is a schematic of prior art structured light projection and acquisition; 
         FIG. 2  is a schematic of structured light projection and acquisition according to embodiments of the invention; and 
         FIG. 3  shows tables of overdrive factors that can be used by embodiments of the invention. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     The embodiments of my invention provide an apparatus and method for generating and acquiring structured light patterns. The embodiments increase the contrast of the structured light in the images acquired by the camera in the presence of ambient light. 
       FIG. 1  shows a prior art process schematically for two time intervals  101 - 102  with projector structured light  110 , and ambient light  120 , two patterns  131 - 132  (pattern 1, pattern 2). Two images are captured during the two acquire intervals  141 - 142  at a rate of, e.g., 30 frames per second (fps). The sensor is readout  150  at the end of each interval. Generally, the patterns are a set of binary or gray-scale images. A set of color images can be also used as the patterns. 
     Because the ambient light and the projector  105  are ON continuously, the light in the scene is a superposition of the structured and ambient light. 
     The quality of structured light images can be indexed by image contrast
 
a.  C =(max−min)/(max+min)
 
b. =( p+a−a )/( p+a+a )
 
c. = p /( p+ 2 a ),  (1)
 
where a is the luminance of the ambient light, and p is the peak luminance of the projector light, i.e., the luminance in a region where “white” light is projected. The projector contributes only a small amount of light to “min,” because the projected light in “black” regions is much smaller than a.
 
     Overdriving 
       FIG. 2  shows the novel method of operation of embodiments of my invention. The invention exploits both the discreteness of the camera image acquisition process, and LED “overdriving.” LED overdriving refers to briefly operating the LED at higher than rated currents. 
     An overdriven LED can produce a light pulse an order of magnitude brighter than the level the LED can continuously produce. Since the total light intensity is directly proportional to the time the light is ON, and the amount of light emitted per unit time, the same total can be created by (e.g.) overdriving by 10 times for 1/10 the amount of time. Overdriving enables shorter pulses and therefore permits camera image integration in a briefer interval. During each image frame, the sensor of the camera acquires light in a discrete interval  141 - 142  during which the sensor pixels integrate the ambient and projected light. By using overdriving, the acquisition interval can be made briefer  241 - 242 , thereby reducing the amount of ambient light integrated, while overdriving maintains the total amount of light integrated from the projector. 
     LEDs convert current into light roughly linearly, i.e., twice the current produces about twice the light, up to a point where thermal failure is possible. If the LED can operate at some current i, with an average (normalized) light output of l, then the LED can typically operate briefly  201 - 202  at a current 10i, e.g., for milliseconds without thermal damage, and produce 10l peak light output  210  at a 10% duty cycle, yet the average light and heat output over time remains the same. If the LED is OFF 90% of the time, excessive heat can readily be dissipated. 
     By overdriving, the projected light p in equation (1) can remain constant, while the ambient light a is reduced by r, the ratio of the duration of the original capture interval  141 - 142  to that of the reduced integration interval  241 - 242 . 
     Substituting a′=a*r into equation (1) shows that the contrast is substantially increased. This increased contrast makes the SLS output more accurate and produces depth images with fewer artifacts and noise. 
     Using overdriving, the contrast of the structured light  210  can overwhelm the ambient light  220  for a brief interval in time. If the sensor acquisition time is reduced to the interval of time when the LED is ON, and a minimal amount of additional time, then the contrast of the structure light pattern captured by the camera can be greatly increased. 
     For example, suppose the sun illuminance on the object being scanned is 50 lux and the peak projector illuminance on that object is only 10 lux. Furthermore assume that when the projector pattern is a typical one with ½ white bars and ½ black bars that the ratio of the light in the white areas to the black bars is 100:1. Then, if there were no ambient light, then the projector contrast is:
 
a.  C   projector =(max−min)/(max+min)
 
b. =(100−1)/(100+1)
 
c. =99/101˜98%,
 
which is near optimal.
 
     If the ambient light is added, then the contrast is drastically reduced:
 
a.  C   projector+ambient =(max−min)/(max+min)
 
b. =(50+10−50−10/100)/(50+10+50−10/100)
 
c. =(9.9)/(110.1)
 
d. ˜9%,
 
which is poor.
 
     However if the LED is overdriven by 20 time, and the acquisition time is reduced by 20 times, thereby reducing the ambient light capture by to 50/20, while preserving the same amount of light acquired from the projector, then the contrast is:
 
a.  C   overdriving+ambient =(max−min)/(max+min)
 
b. =(50/20+10−50/20−10/100)/(50/20+10+50/20+10/100)
 
c. =(9.9)/(15.1)
 
d. ˜65%,
 
which is very good.
 
     Although the invention has been described with reference to certain preferred embodiments, it is to be understood that various other adaptations and modifications can be made within the spirit and scope of the invention. Therefore, it is the object of the append claims to cover all such variations and modifications as come within the true spirit and scope of the invention.