Patent ID: 12206832

DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The present invention is illustrated in further details by the following non-limiting examples.

A system for band-limited illumination profilometry (BLIP) with temporally interlaced acquisition (TIA) according to an embodiment of an aspect of the present invention generally comprises a projection unit to project pre-defined fringe patterns onto the surface of the measured object, the fringe patterns being distorted and reflected by the object surface, point by point, and cameras capturing the distorted fringes images, point by point.

InFIG.1A, the projecting unit comprises a high-coherent light source10, a spatial light modulator18, a pattern conversion unit21, a projecting optics30.

After expansion and collimation by a beam expander12, the laser beam from a 10 lns a 200-mW continuous-wave laser source10, of wavelength λ=671 nm (MRL-III-671, CNI Lasers), is directed by mirrors14and16to a 0.45″ Digital Micromirror Device (or DMD)18(AJD-4500, Ajile Light Industries) at an incident angle of about 24° to the normal of the surface of the Digital Micromirror Device18, for sinusoidal fringes generation; using four phase-shifting binary patterns I0, I1, I2, I3, generated by an error diffusion algorithm from their corresponding grayscale sinusoidal patterns, loaded onto the Digital Micromirror Device18. The pattern conversion unit21, comprising a 4 f imaging system25with a low pass filter24such as a pinhole, converts the binary patterns to grayscale fringes at the intermediate image plane28.

The minimal pinhole diameter D for all spatial frequency content of the sinusoidal fringe pattern to pass through the system is determined by the system bandwidth as follows:

D=λ⁢f1pf(1)

where pf=324 μm is the period of the fringes composed by the digital micromirror device pixels, f1being the focal length of lens22. With lenses22and26of the 4 f imaging system25having focal lengths f1=120 mm and f2=175 mm respectively, the minimal pinhole diameter is D=248.52 μm. In an experiment, a 300 μm-diameter pinhole was selected.

A projector lens/projector30(AF-P DX NIKKOR, Nikon) projects the output fringe patterns31on a 3D object32.

Deformed structured images are captured alternately by two high-speed CMOS cameras34,36(CP70-1HS-M-1900, Optronis) placed side by side, i.e on a same side of the projector. Depending on their roles in image reconstruction, the cameras are referred to as the main camera34and the auxiliary camera36respectively, as will be described hereinbelow. Synchronized by the trigger signal of the digital micromirror device, each camera34,36captures half of the sequence (FIG.1B: Capture I0of fringe pattern intensity I0and Capture I1of fringe pattern intensity I1: Capture I2of fringe pattern intensity I2and Capture I3of fringe pattern intensity I3respectively, etc., for exposure time te). The acquired images from each camera are transferred to a computer38via a CoaxPress cable connected to a frame grabber (Cyton-CXP, Bitflow).

The light source10is a high coherent light source of power at least 50 mW, selected depending on the sensitivity of cameras34and36. with a laser wavelength is comprised in the range between about 380 and about 750 nm in case of visible light cameras, and in the range between about 800 and about 1100 nm in case of near infrared (NIR) cameras.

The high-speed cameras34,36may be cameras with global shutter, of imaging speed of at least about 2 k frames/second, with image resolution at least about 1000×800 pixels.

The spatial light modulator18has a refreshing rate of at least about 4 kHz, on board memory of at least about 1 Mb, and is selected to work at the corresponding wavelength of the light source. It may be a liquid crystal display or a binary fringe mask with a motorized translation stage for example.

The pattern conversion unit21may comprise a 4 f imaging system25with lenses of different focal lengths and the low-pass filter24may be a slit. The focal lengths of the two lenses are selected with a ratio (focal length of the first lens/focal length of the second length) comprised in the range between about 0.75 and about 1.5. The diameter of the low pass filter is selected in the range between about 150 μm and about 300 μm.

The projecting optics/projector30is selected with a focal length in the range between about 18 and about 55 mm, a F number in the range between about 3.5 and about 5.6, and a magnification ratio in a range between about 5 and about 10 times.

The imaging speed and field of view may be further improved by using more than two cameras, in such a way to separate the workload to an array of cameras, for example to trace and recognize hand gesture in 3D space to provide information for human-computer interaction.

The system thus projects sinusoidal fringe patterns31onto the object32and captures the corresponding deformed patterns37modulated by the surfaces of the object32. The depth information is encoded into the phase of the distorted fringe images38. For phase demodulation and reconstruction of the 3D object, the retrieved phase distribution corresponding to the object height is mathematically wrapped to principle values of arctangent function ranging between −π and π, and consequently, the phase discontinuities occur at the limits every time when the unknown true phase changes by 2π, which is referred to as the phase ambiguity problem, resulting from the periodical nature of the sinusoidal signal. A unique pixel correspondence between the cameras34,36and the projector30is obtained by phase unwrapping.

According to an aspect of the present disclosure, a method to recover the 3D image of the object32pixel by pixel from the mutually incomplete images provided by the cameras34,36generally comprises locating a point (u′a, v′a) in the images of the auxiliary camera36that matches a selected pixel (um, vm) of the main camera34; determining estimated 3D coordinates and wrapped phase from knowledge of the cameras calibration, determining the horizontal coordinate on the plane of the projector from knowledge of the projector calibration, and using the wrapped phase value to recover the 3D point of 3D coordinates (x, y, z) with the coordinate-based method.

System Calibration

To recover the object's 3D information, the method relies on a coordinate-based understanding of the spatial relationship between the projector30and the cameras34,36in image formation. The projection of the 3D coordinates (x, y, z) of the 3D point onto the camera coordinates (u, v) is described in a pinhole model using extrinsic parameters R and T describing the rotation and translation of coordinates, respectively, and intrinsic parameters characterizing the properties of the cameras in image formation, with fuand fvthe effective focal lengths along each axis of the sensor of the cameras; uppand vppthe coordinates of the principal point of the cameras; and a accounting for pixel skewness, as follows:

s⁢[uv1]=[fuαup⁢p0fvvp⁢p001]⁡[RT]⁡[xyz1],(2)

Column vectors [u, v, 1]Tand [x, y, z, 1]Trepresent the camera coordinates (u, v) and the 3D coordinates (x, y, z) in homogeneous coordinates, which allow for the numerical extraction of the camera coordinates (u, v) from Relation (2) through a scalar factor s.

The cameras34,36and the projector30are calibrated to determine the values of the extrinsic and intrinsic parameters using a checkerboard (not shown). Since the direct image acquisition is not possible for a projector30, projector-centered images of the calibration object obtained by the phase-based mapping method are sent to a toolbox (also not shown) with calibration in the same manner as for the cameras34,36.

Coordinate-Based 3D Point Determination

3D information is recovered from the calibrated imaging system using a coordinate-based method. To a point on the 3D object with the 3D coordinates (x, y, z) correspond two independent coordinates, (u, v) for the cameras24,36and (u″, v″) for the projector30.

In a calibrated phase-shifting fringe projection profilometry system, any three of these coordinates {u, v, u″, v″} can be determined and a linear system of the form E=M[x, y, z]Tis derived. The elements of E and M are obtained by using the calibration parameters of each device, the scalar factors and the three determined coordinates among u, v, u″ and v″. Thus, 3D information of an object point can be extracted via matrix inversion.

Returning to the system discussed in relation toFIG.1A, first, images from the calibrated main camera34are used to provide the camera coordinates (u, v) of a point on the 3D object. Along with the calibration parameters of the system, an epipolar line39is determined on the calibrated auxiliary camera36. The horizontal coordinate in the images of the auxiliary camera36is recovered using search-based algorithms along the epipolar line39, in stereo vision. Second, by substituting the calibrated projector in place of the auxiliary camera36, the intensity values of the pixel (u, v) of the auxiliary camera36across a sequence of images is used by structured light methods to recover information about a coordinate of the calibrated projector. The object's 3D information is extracted pixel by pixel based on interlaced image acquisition by incorporating the camera coordinates (u, v) of a point on the 3D object and its corresponding projector coordinates, using a triangulation method to solve the point in 3D space.

Data Acquisition

For data acquisition, four fringe patterns with phases equally shifted by π/2 illuminate the 3D object. The intensity value Ik(u, v) for the pixel (u, v) in the kth image acquired by the calibrated main camera34is obtained as follows:

Ik⁡(u,v)=Ib⁡(u,v)+Iv⁢a⁡(u,v)⁢cos⁡[φ⁡(u,v)-π⁢k2],(3)

where k∈[0,3]. Ib(u, v) is the background intensity, Iva(u, v) is the variation of intensity and φ(u, v) is the depth-dependent phase.

Relation (3) allows analyzing two types of intensity matching conditions for the order of pattern projection shown inFIG.1B. For a pixel (u′a, v′a) in the images of the auxiliary camera36that perfectly corresponds with the coordinates of a selected pixel (um, vm) in the images of the main camera34, Relation (3) yields:
I0(um,vm)+I2(u′a,v′a)=I1(um,vm)+I3(u′a,v′a).  (4)

Rearrangement of Relation (4) leads to the equivalent relation, selected as the intensity matching condition:
I0(um,vm)−I1(um,vm)=I3(u′a,v′a)−I2(u′a,v′a).  (5)

Each side of Relation (5) contains images captured by the same camera and represent a residual fringe component of sinusoidal characteristics, which allows to increase the efficiency of line-constrained searches by regularizing local maxima and minima in the patterns and by including additional phase information. Moreover, by considering the right-hand side as a continuously varying function along the epipolar line (39) determined on the calibrated auxiliary camera36, Relation (5) and bi-linear interpolation allows for the selection range41of discrete candidates37with sub-pixel accuracy.

FIG.2is a flowchart of a method for coordinate-based 3D point determination according to an aspect of the present disclosure, with coordinates of the point to be matched for the main camera4(um, vm); coordinates of the estimated corresponding point for the auxiliary camera (u′a, v′e)42; recovered 3D coordinates (x, y, z); horizontal distance between the candidates and the estimated corresponding point ri; phase value of the selected point in the main camera obtained by the Fourier transform profilometry method ωm; phase value of the candidate points in the auxiliary camera obtained by the Fourier transform profilometry method; phase value obtained by the phase-shifting method φ′ai; phase value determined on the projector's plane φ″pi; 3D points determined by candidates Pi; principal point of the main camera Pm; principal point of the auxiliary camera Pa; ΔIm=I0(um, vm)−I1(um, vm); and intensity profile of I3−I2along the epipolar line ΔIep.

In a quality map determination step (see Step I40inFIG.2), (I0+I1)/242and (I2+I3)/244are obtained from images by the main camera34. Then, a threshold intensity, obtained from a selected background region, is used to eliminate pixels with low intensities and obtain a binary quality map. Subsequently, after such thresholding of the intensity map, only pixels (um, vm)46that fall within the quality map of the main camera34are considered for 3D information recovery.

In a candidate discovery step (see Step II48inFIG.2), the selected pixel (um, vm) of the main camera34determines an epipolar line50containing the matching point within the images of the auxiliary camera36. Then, the candidates (u′ai, v'ai) for the matching point in the auxiliary camera36images are extracted, the subscript “i” denoting the ithcandidate, that satisfy the intensity matching condition determined by Relation (5) above in addition to a quality map constraint52, a transformation constraint54and a phase sign constraint56.

The quality map constraint52requires that the candidates (u′ai, v′ai) for the matching point in the auxiliary images fall within the quality map of the auxiliary camera36.

The transformation constraint54requires that candidates occur within a segment of the epipolar line50determined by a fixed two-dimensional projective transformation or homography that approximates the location of the matching point (u′e, v′e)58within the images of the auxiliary camera36as follows:
s′[u′e,v′e,1]T=H[um,vm,1]T,  (6)

where s′ is a scalar factor representing extraction of the pair of coordinates of the estimated corresponding point (u′e, v′e)58from its homogeneous coordinates [x, y, z, 1]T. H is obtained by applying Relation (6) to four points chosen as the corners of a flat rectangular plane when imaged by both cameras34,36at the approximate center of the measurement volume. [um, vm, 1]Tare the homogeneous coordinates of the selected pixel (um, vm) of the main camera34. Once the coordinates of the estimated corresponding point (u′e, v′e)58are determined, the search along the epipolar line is confined to the segment occurring over the horizontal interval [u′e−r0, u′e+r0], where r0is an experiment-dependent constant. In general, r0is selected as small as possible while still covering the targeted depth range. For the presently described experiments, the value of r0was set to 40 pixels.

The phase sign constraint56requires that the selected point (um, vm)60of the main camera34and candidates (u′ai, v′ai)62have the same sign of their wrapped phases64ωmand ω′airespectively. Estimates of the wrapped phases64are obtained using Fourier transform profilometry. In particular, the intensity If(um, vm) of the selected pixel (um, vm) of the main camera pixel in the filtered image is obtained by band-pass filtering the left-hand side of Relation (5) I0−I1, as follows:

If⁡(um,vm)=22⁢Iv⁢a⁡(um,vm)⁢exp⁡[j⁡(φ⁡(um,vm)+π4)].(7)

The wrapped phase estimation ωmof the selected point (um, vm) is obtained as follows:

ωm=tan-1⁢{⁡[If⁡(um,vm)]⁡[If⁡(um,vm)]},(8)
where ℑ[·] and[·] denote the imaginary and real part of a complex variable respectively. The same band-pass filtering applied to the right-hand side of Relation (5) I3−I2yields the estimate of its wrapped phase64ω′aiof the candidate (u′ai, v′ai), as follows:

ωa⁢i′=tan-1⁢{⁡[If′⁡(uai′,va⁢i′)]⁡[If′⁡(uai′,va⁢i′)]}.(9)

The phase sign constraint requires that the wrapped phase estimation ωmof the selected point (um, vm) and the wrapped phase estimation ω′aiof the candidate (u′ai, v′ai) have the same sign in the interval (−π, π].

Other Fourier transform profilometry methods for wrapped phase value extraction.

The output of the candidate discovery step48is a pool of candidates for further evaluation and the method proceeds to matching point selection66. If no candidate is found, the candidate discovery step48is re-initiated for the next pixel in the main camera34, until a candidate62is obtained, and the method proceeds to the matching point selection66.

In the matching point selection step (see Step III66inFIG.2), penalty scores68for each candidate62obtained from the candidate discovery step48are determined. A first and primary criterion compares the phase values of the candidates using two methods. First, the phase of the candidate62is obtained from the intensities of the candidate (u′ai, v′ai) and of the pixel (um, vm) of the selected point60as follows:

φa⁢i′=tan-1⁡[I1⁡(um,vm)-I3⁡(uai′,va⁢i′)I0⁡(um,vm)-I2⁡(uai′,va⁢i′)].(10)

Meanwhile, for each candidate (u′ai, v′ai)62, the coordinate triple (um, vm, u′ai) and knowledge of camera calibration allows determining an estimated 3D point Piby using the stereo vision method. In addition, with the knowledge of the projector calibration, a point with coordinates (u″pi, v″pi) on the plane of the projector30is determined for each candidate. Then, an unwrapped phase value φ″piis obtained by:

φp⁢i″=2⁢πp⁢(up⁢i″-ud″),(11)

where u″dis a horizontal datum coordinate on the plane of the projector associated with the zero phase, and p is the fringe period in units of projector pixels. Since these independently obtained phase values must agree if the candidate correctly matches (um, vm), a penalty score Ai70, as a normalized difference of these two phase values, is obtained as follows:

Ai=R⁡(φa⁢i′-φp⁢i″)π,(12)

where the rewrapping function R(·) computes the subtracted difference between wrapped and unwrapped phase values.

To improve the robustness of the method, two additional criteria are implemented using data available from the candidate discovery step. Bi72is a normalized distance score favoring candidates located closer to the estimated matching point (u′e, v′e), which is obtained by:

Bi=ue′-ua⁢i′r0.(13)

Moreover, Ci74is a normalized difference of wrapped phase values obtained by using the wrapped phases ωmand ω′ai, as follows:

Ci=R⁡(ωm-ωa⁢i′)π.(14)

A total penalty score Si76for each candidate is then determined as a weighted linear combination of three individual scores as follows:
Si=η1Ai+η2Bi+η3Ci,  (15)

where the normalized weights [η1, η2, η3]=[0.73, 0.09, 0.18] are empirically selected to lead to the results that are most consistent with physical reality. Finally, the candidate with the minimum total penalty score Siis selected as the matching point (u′a, v′a), and its phase values are obtained by using relations. (10) and (11) are denoted as φ′aand of φ″p, respectively.

In a final step of 3D point recovery (see Step IV78inFIG.2), the method determines the final 3D coordinates 80. First, the phase of the candidate φ′ais unwrapped as φ′a+2πq, where q is an integer such that φ″p−(φ′a+2πq)∈(−π, π]. Then, the coordinate on the plane of the projector u″pis obtained with sub-pixel resolution as follows:
u″p=u″d+P(φ′a/2π+q),  (16)
from which the final 3D coordinates (x, y, z)80are obtained using calibration information associated with the coordinate triple (um, vm, u″p).

Results

FIGS.3A and3Bshow quantification of the depth resolution of the method. To quantify the depth resolution with different exposure times, two stacked planar surfaces offset by about 9° were imaged. Reconstructed results at four representative exposure times te(illustratively 950 μs, 250 μs, 205 μs and 100 μs) are shown inFIG.3A. One area on each surface82, marked as white boxes84in full lines inFIG.3A, was selected in the reconstructed image. The depth information on the x axis was obtained by averaging the depth values along the y axis. The difference in depths between these two surfaces is denoted by zd. In addition, the noise86is defined as the averaged values of the standard deviation in depth from both surfaces. The depth resolution88is defined as when zdequals to two times the noise level of the system. As shown in the four plots ofFIG.3A, the reconstruction results deteriorate with shorter exposure times, manifested by increased noise levels and more points incapable of retrieving 3D information. As a result, the depth resolution degrades from 0.06 mm at te=950 μs to 0.45 mm at te=150 μs (FIG.3B). At exposure time te=100 μs, the method fails in 3D measurements. The region of unsuccessful reconstruction prevails across most of the planar surfaces. The noise86dominates the obtained depth difference, which is attributed to the low signal-to-noise ratio in the captured images.

To examine the feasibility of the method, various static 3D objects were imaged. First, two sets of 3D distributed letter toys90that composed the words of “LACI”92and “INRS”94were imaged.FIGS.4A and4Bshow static 3D objects. Shown inFIG.4A, the two perspective views of the reconstructed results reveal the 3D position of each letter toy90. The detailed surface structures are illustrated by the selected depth profiles (white dashed lines inFIG.4A). A proof-of-concept experiment was also conducted on three cube toys96with fine structures, with a depth of about 4 mm, on the surfaces. As can be seen inFIG.4B, the detailed structural information of these cube toys96is recovered.

Imaging of Dynamic 3D Objects

To verify high-speed 3D surface profilometry, the method was used to image two dynamic scenes: a moving hand (FIG.5A) and three bouncing balls (FIG.5B). The fringe patterns were projected at 4 kHz. The exposure times of both cameras were te=250 μs. Under these experimental conditions, a 3D imaging speed of 1 thousand frames per second (kfps), a field of view (FOV) of 150 mm×130 mm, corresponding to 1180×860 pixels in captured images, and a depth resolution of 0.24 mm were achieved.

FIG.5Ashows the reconstructed 3D images98of the moving hand at five time points from 0 ms to 60 ms with a time interval of 15 ms. The high-speed 3D imaging allowed tracking the movements of four fingertips100. As shown inFIG.5B, all the four fingers have apparent movement in both the x axis and the z axis but stay relatively stationary in the y axis, which agrees with the experimental condition.

In the second experiment, three white balls102, each of which was marked by a different letter on its surface, bounced in an inclined transparent container.FIG.5Cshows five representative reconstructed images104from 8 ms to 28 ms with a time interval of 5 ms. The changes of the letter “C”106on B1and the letter “L”108on B2, marked in the third panel110ofFIG.5C, clearly show the rotation of the two balls. The method enabled tracking the 3D centroids of each ball102over time. As shown inFIG.5D, B1collides with B2at 16 ms, resulting in a sudden change in the moving directions. This collision temporarily interrupted the free fall of B1, represented by the two turning points in the curve of evolution along the y-axis (second panel112ofFIG.5D). The collision also changed the moving direction of B2, making it touch the base at 27 ms and then bounce up. In this scene, B3maintained its movement in a single direction in both the x axis and the z axis. It fell onto the based and bounced back at 16 ms, resulting in a turning point in its y-t curve. Because of the inclined bottom plane, the y-value of B3at 16 ms was smaller than that of B2at 27 ms.

Application to the Study of Sound-Induced Vibration on Glass

To highlight the broad utility of the method, sound-induced vibration on glass cup114was imaged. In an experiment (FIG.6A), the glass cup114was fixed on a table116. A function generator drove a speaker118to produce single-frequency sound signals, from 450 Hz to 550 Hz with a step of 10 Hz through a sound channel120placed close to the-wall of the cup114. To image the vibration dynamics, fringe patterns were projected at 4.8 kHz. The cameras had an exposure time of te=205 μs. This configuration enabled a 3D imaging speed of 1.2 kfps, a field of view (FOV) of 120 mm×110 mm, corresponding to 960×800 pixels in captured images, and a depth resolution of 0.31 mm.FIG.6Bshows four representative 3D images122of the instantaneous shapes of the glass cup114driven by the 500-Hz sound signal, showing the dynamic of structural deformation of the glass cup114. The evolution of depth changes was analyzed using five selected points, marked by PAto PEin the first panel124ofFIG.6B. A shown inFIG.6C, the depth changes of the five points PAto PEare in accordance, which is attributed to the rigidness of the glass cup114.

Time histories of averaged depth displacements under different sound frequencies were further analyzed.FIG.6Dshows the results at the driving frequencies of 490 Hz, 500 Hz, and 510 Hz. Each result was fitted by a sinusoidal function with a frequency of 490.0 Hz, 499.4 Hz, and 508.6 Hz, respectively. These results show that the rigid glass cup114vibrated in compliance with the driving frequency. Moreover, the amplitudes of fitted results, Δzfit, were used to determine the relationship between the depth displacement and the sound frequency (FIG.6E). This result was fitted by the Lorentz function, which determined the resonant frequency of this glass cup114to be 499.0 Hz.

Application to the Study of Glass Breakage

To further apply the method to recording non-repeatable 3D dynamics, the process of a glass cup114breaking by a hammer was imaged. As displayed inFIG.7A, the growth of cracks126and the burst of fragments128with different shapes and sizes were clearly shown in the reconstructed 3D images. The time courses of velocities of four fragments128, marked by FAto FDinFIG.7A, are plotted inFIG.7B. The velocities in the y axis are considerably small compared to the other two directions, which indicates the impact of the hammer force was exerted on the x-z plane. vyof fragments FAand FCshows that they moved upward until 15 ms and fell afterward. vyof fragments FBand FDreveals that they fell onto the remaining base of the cup at 15 ms and kept sliding down on the surface. The data of vzillustrates that FAand FCmoved closer to the cameras, which were directly driven by the hammer's force. However, FBand FD, which collided with other pieces, maintaining their positive directions in vzto move away from the cameras. The corresponding accelerations are displayed as inFIG.7C, which indicates the influence of both the main strike and the ensuing collision among different fragments. At 14 ms, the collision with other fragments, which applied an impact along the +x direction, dominated the acceleration direction for all four tracked fragments. In contrast, at 15 ms, another collision produced an impact in the −x direction, causing a sharp decrease in the acceleration for FAand FC. In addition, the direction of acceleration for FDalong the y-axis changed several times, which is attributed in several collisions of FDwith the base of the glass cup114while sliding down.

Referring back toFIG.1A, there is thus presented a method with a kfps-level 3D imaging speed over a field of view of up to 150 mm×130 mm. The method implements temporally interlaced acquisition in multi-view 3D phase-shifting fringe projection profilometry systems, which allows each camera34,36capturing half of the sequence of phase-shifting fringes. Leveraging the characteristics indicated in the intensity matching condition [Relation (5)], the method applies constraints in geometry and phase to find the matching pair of points in the main34and auxiliary36cameras and guides phase unwrapping to extract the depth information. The method was shown to allow the 3D visualization of glass vibration induced by sound and the glass cup breakage by a hammer.

Still referring toFIG.1Athere is thus presented a system and a method for high-speed dual-view band-limited illumination profilometry using temporally interlaced acquisition.

As people in the art will now be in a position to appreciate, temporally interlaced acquisition eliminates the redundant capture of fringe patterns in data acquisition. The roles of the main camera34and the auxiliary camera36are interchangeable and the present method may be adapted to a range of multi-view phase-shifting fringe projection profilometry systems. Moreover, temporally interlaced acquisition reduces the workload for both cameras by half. For the given bandwidth of the camera's interface, this more efficient use of cameras can either increase the 3D imaging speed for a fixed field of view or enlarge the field of view with a maintained 3D imaging speed. Both advantages shed light on implementing the present method with an array of cameras to simultaneously accomplishing high accuracy and high speed 3D imaging over a larger fields of view. Also, the two cameras34,36deployed in the present method are placed on a same side relative to the projector30, which circumvents the intensity difference induced by the directional scattering light from the 3D object and reduces shadow effect by occlusion occurring when placing the cameras on different sides of the projector. As a result, robust pixel matching in the image reconstruction algorithm allows to recover 3D information on non-Lambertian surfaces.

In an alternative embodiment, the imaging speed and field of view may be optimized by separating the workload to four cameras, by using a faster digital micromirror device18, and by using a more powerful laser10. The image reconstruction toward real-time operation may be increased by further adapting the 3D point recovery method to four cameras and by using parallel computing to accelerate the calculation.

Still referring toFIG.1A, the present method may be integrated in structured illumination microscopy and frequency-resolved multi-dimensional imaging. The present method may also be implemented in the study of the dynamic characterization of glass in its interaction with the external forces in non-repeatable safety test analysis. As another example, the present method may be used to trace and recognize the hand gesture in 3D space to provide information for human-computer interaction. Furthermore, in robotics, the present method may provide a dual-view 3D vision for object tracking and reaction guidance. Finally, the present method can be used as an imaging accelerometer for vibration monitoring in rotating machinery and for behavior quantification in biological science.

Still referring toFIG.1A, temporally interlaced acquisition thus integrated in a dual-view phase-shifting fringe projection profilometry system allows each camera34,36capturing half of the sequence of phase-shifting fringes31. Leveraging the characteristics indicated in the intensity matching condition, the method applies constraints in geometry and phase to find the matching pair of points in the main and auxiliary cameras and guides phase unwrapping to extract the depth information.

Still referring toFIG.1A, the present method and system eliminate the redundant capture of fringe patterns in data acquisition, which lifts the long-standing limitation in imaging speed for multi-view phase-shifting fringe projection profilometry, and allows reducing the workload of cameras, which enables the enhancement of either the 3D imaging speed or the imaging field of view. Dynamic 3D imaging of over 1 thousand frames per second on a field of view of up to 150×130 mm2, corresponding to 1180×860 pixels in captured images, was demonstrated. Moreover, by putting the two cameras34,36side by side on a same did-side of the projector30, the present method and system circumvent the influence of directional scattering light and occlusion effect for more robust reconstruction, thereby expanding the application range of multi-view phase-shifting fringe projection profilometry to non-Lambertian surfaces.

Still referring toFIG.1A, the present method and system may be adapted into other multi-view 3D profilometers, thus opening new opportunities to blur-free 3D optical inspection and characterization with high speeds, large fields of view, and high accuracy. The present method and system provide a versatile tool for dynamic 3D metrology with potential applications in advanced manufacturing, such as characterization of glass in non-repeatable safety test and high-speed vibration monitoring in rotating machinery. The present compact and symmetric system may be embedded in the vision system of robots to track objects, to recognize the gesture for human-computer interaction, and to guide reactions.

The scope of the claims should not be limited by the embodiments set forth in the examples, but should be given the broadest interpretation consistent with the description as a whole.