Abstract:
A method, system and computer program product are provided for displaying three-dimensional measurement points on a two-dimensional plane of a display screen having a plurality of pixels. The method includes projecting the measurement points onto the plane. Each of the measurement points is assigned to one of the pixels. A depth value is assigned to each of the pixels. A first pixel is selected having a first measurement point and a first depth value. A first side is searched for a second pixel having a second measurement point and a second depth value. A second side is searched for a third pixel having a third measurement point and a third depth value. It is determined whether the second and third measurement points are on a same plane. The first depth value of the first pixel is changed when the second and third measurement points are on the same plane.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    The present application is a continuation application of U.S. application Ser. No. 15/140,909 filed on Apr. 28, 2016, which is a continuation application of U.S. application Ser. No. 13/697,031 filed on Apr. 29, 2013, now U.S. Pat. No. 9,329,271, which is a National Stage Application of PCT Application No. PCT/EP2011/001662, filed on Apr. 1, 2011, which claims the benefit of U.S. Provisional Patent Application No. 61/362,810, filed on Jul. 9, 2010, and of German Patent Application No. DE 10 2010 020 925.2, filed on May 10, 2010, and which are hereby incorporated by reference. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    The invention relates to a method for optically scanning and measuring an environment. 
         [0003]    Through use of a known method of this kind, a three-dimensional scan is recorded which is then displayed two-dimensionally. Provided that density and extension of the measurement points are smaller than the pixels of the display, a relatively better visual impression is generated if a gap-filling takes place between the measurement points, i.e., if surfaces are generated from the single measurement points. All measurement points can thus be projected onto one plane and be assigned to single pixels. The intermediate pixels of the plane are then filled, for example, by interpolation. 
       SUMMARY OF THE INVENTION 
       [0004]    According to an embodiment of the present invention, a method, system and computer program product are provided for displaying a plurality of measurement points in three-dimensional space on a two-dimensional plane of a display screen. The method includes projecting the plurality of measurement points onto the two-dimensional plane, the display screen has a plurality of pixels. Each of the measurement points of the plurality of measurement points is assigned to one of the pixels in the plurality of pixels. A depth value is assigned to each of the plurality of pixels that are assigned one of the measurement points of the plurality of measurement points. A first pixel is selected, the first pixel having a first measurement point of the plurality of measurement points assigned to the first pixel, the first pixel having a first depth value assigned to the first pixel. A first side of the first pixel is searched for a second pixel having a second measurement point of the plurality of measurement points assigned to the second pixel, the second pixel having a second depth value assigned to the second pixel. A second side of the first pixel is searched for a third pixel having a third measurement point of the plurality of measurement points assigned to the third pixel, the second side being opposite the first side, the third pixel having a third depth value assigned to the third pixel. It is determined whether the second measurement point and the third measurement point are on a same object plane based at least in part on the second depth value and the third depth value. The first depth value assigned to the first pixel is changed based on determining the second measurement point and the third measurement point are on the same object plane. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0005]    Embodiments of the present invention are explained in more detail below on the basis of exemplary embodiments illustrated in the drawings, in which 
           [0006]      FIG. 1  is a schematic illustration of the assignment and filling of the pixels with a view onto the plane, wherein the adjacent pixels are on the same surface; 
           [0007]      FIG. 2  is a schematic illustration of the assignment and filling of the pixels, according to  FIG. 1 , with a view onto the plane; 
           [0008]      FIG. 3  is a schematic illustration of the assignment and filling of the pixels with a view onto the plane, wherein the adjacent pixels are located on different surfaces; 
           [0009]      FIG. 4  is a schematic illustration of the assignment and filling of the pixels, according to  FIG. 3 , with a view onto the plane; 
           [0010]      FIG. 5  is a schematic illustration of a laser scanner in the environment including the display device, and 
           [0011]      FIG. 6  is a partial sectional illustration of the laser scanner. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0012]    Referring to the Figures, a laser scanner  10  is provided as a device for optically scanning and measuring the environment of the laser scanner  10 . The laser scanner  10  has a measuring head  12  and a base  14 . The measuring head  12  is mounted on the base  14  as a unit that can be rotated about a vertical axis. The measuring head  12  has a rotary mirror  16  that can be rotated about a horizontal axis. The point of intersection between the two axes of rotation is designated as the center C 10  of the laser scanner  10 . 
         [0013]    The measuring head  12  also has a light emitter  17  for emitting an emission light beam  18 . The emission light beam  18  may be a laser beam in the range of wave length of approximately 300 to 1600 nm, for example, 790 nm, 905 nm or less than 400 nm, but other electro-magnetic waves having, for example, a greater wave length can be used. The emission light beam  18  is amplitude-modulated with, for example, a sinusoidal or rectangular-waveform modulation signal. The emission light beam  18  is passed from the light emitter  17  onto the rotary mirror  16  where it is deflected and then emitted into the environment. A reception light beam  20 , which is reflected by or otherwise scattered from an object O, is captured again by the rotary mirror  16 , deflected and passed onto a light receiver  21 . The direction of the emission light beam  18  and of the reception light beam  20  results from the angular positions of the rotary mirror  16  and the measuring head  12 , which depend on the positions of their respective rotary drives which are, in turn, detected by a respective encoder. 
         [0014]    A control and evaluation device  22  has a data link connection to the light emitter  17  and to the light receiver  21  in the measuring head  12 , parts thereof being arranged also outside the measuring head  12 , for example as a computer connected to the base  14 . The control and evaluation device  22  determines, for a multiplicity of measurement points X, the distance d of the laser scanner  10  from the illuminated point on the object O, and from the propagation times of emission light beam  18  and reception light beam  20 . For this purpose, the phase shift between the two light beams  18  and  20  can be determined and evaluated. 
         [0015]    Through use of the relatively rapid rotation of the mirror  16 , scanning takes place along a circular line. Also, through use of the relatively slow rotation of the measuring head  12  relative to the base  14 , the entire space is gradually scanned with the circular lines. The totality of the measurement points X of such a measurement shall be designated as a scan. The center C 10  of the laser scanner  10  defines for such a scan the origin of the local stationary reference system. The base  14  is stationary in this local stationary reference system. 
         [0016]    In addition to the distance d to the center C 10  of the laser scanner  10 , each measurement point X comprises a brightness value which may also be determined by the control and evaluation device  22 . The brightness is a gray-tone value which is determined, for example, by integration of the bandpass-filtered and amplified signal of the light receiver  21  over a measuring period which is assigned to the measurement point X. Through use of a color camera, it is optionally possible to generate pictures, by which colors (R, G, B) can be assigned as a value to the measurement points X in addition to the brightness or comprising the brightness. 
         [0017]    A display device  30  is connected to the control and evaluation device  22 . The display device  30  can be integrated into the laser scanner  10 , for example into the measuring head  12  or into the base  14 , or it can be an external unit, for example part of a computer which is connected to the base  14 . The display device  30  has a graphic card  32  and a screen  34  which can be arranged separately from one another or as a structural unit. The control and evaluation device  22  provides 3D data of the scan. 
         [0018]    Referring also to  FIGS. 1-4  as well as  FIGS. 5 and 6 , the graphic card  32  converts the 3-D data into 2-D data (e.g., rendering data), which are displayed on the screen  34 . The 3-D data are the measurement points X, wherein several scans from different positions of the laser scanner  10  can be assembled into one scene. For representing the 2-D data, there are pixels P, i.e., adjacent, small polygonal surfaces (e.g. squares or hexagons), which are arranged in a two-dimensional plane E which corresponds to the screen  34 . The starting point is the projection of the measurement points X onto the plane E with a viewer (e.g., eye, camera), at a certain viewpoint V. The projection appears to be in perspective (i.e., the viewpoint V is in the finite) or orthographical (i.e., the viewpoint V in is the infinite). The projected measurement points X are assigned to single pixels P. A Z-buffer serves for representing the  2 -D data, i.e., a two-dimensional auxiliary field for the pixels P. In this Z-buffer, a field element (e.g., depth z) is assigned to each pixel P. The depth z of each projected measurement point X corresponds to the distance of the measurement point X to the plane E with respect to the viewpoint V. The field of the pixels P and the Z-buffer may be treated in the same way as the images. 
         [0019]    The viewpoint V may be arbitrary per se and is usually changed several times when regarding the scan and/or the scene. 
         [0020]    Since the measurement points X are punctiform, with gaps in between, and the pixels P usually, in the case of nearby objects O, have a higher density in the plane E than the projections of the measurement points X, a gap-filling is carried out to fill as many pixels P as possible for an improved representation. The graphic card  32  carries this out in parallel using the 3-D data and the indication of the viewpoint V and of the plane E. 
         [0021]    Initially only those pixels P are filled to which the projection of a measurement point X is assigned, i.e., which exactly cover one measurement point X. These pixels P are filled with the value of the assigned measurement point X, i.e., brightness and, where applicable, color. All other pixels P, which do not exactly correspond with a projection of a measurement point X, i.e., which are “in between” are empty at first, for example are set to zero. Each of the depths z, i.e., the field elements of the Z-buffer, which are assigned to the initially filled pixels P, is set to that depth z 0 , z 1 , z 2 , which corresponds to the distance of the assigned measurement point X to the plane E. All other field elements of the Z-buffer (e.g., depths z) are set to an extreme value, for example, to infinite. If, when the projection of the measurement points X is made, it turns out that two measurement points X are available for one pixel P, the measurement point having the smaller depth z is selected and the other one is rejected, so that covered surfaces and covered edges are not visible. 
         [0022]    According to embodiments of the present invention, gap-filling takes place in dependence on the depth z 0 , z 1 , z 2 , i.e., on the distance to the plane E. The graphic card  32  selects all pixels P in parallel with respect to time. By way of example, one selected pixel P 0  is regarded now. The assigned depth z, i.e., field element of the Z-buffer, contains the depth z 0 . For each selected pixel P o  the adjacent pixels P 1 , P 2 , are searched consecutively, i.e., to the left and to the right and above and below. If the adjacent pixel P 1  is not yet filled or if its depth z is bigger than the depth z 0  of the selected pixel P 0 , it is skipped and the second next pixel P is taken as adjacent pixel P i , if necessary iteratively. If an adjacent pixel P 1 , the depth z 1  of which is smaller than the depth z 0  of the selected pixel P 0 , is found in one of the directions, a change to the next direction takes place, and it is looked for the adjacent pixel P 2  (e.g., the depth z 2  of which is smaller than the depth z 0  of the selected pixel P 0 ). It is possible to define a maximum number of skipped pixels, i.e., if the adjacent pixel P 1  or P 2  is not yet found after skipping the maximum number of skipped pixels, the search for P 1  or P 2  is aborted. 
         [0023]    If the adjacent pixels P 1  and P 2  to the selected pixel P 0  have been found in opposite directions, with the depths z 1  and z 2  of the adjacent pixels P 1  and P 2  being smaller than the depth z 0 , it is checked whether P 1  and P 2  are on the same plane, i.e., whether the amount of the difference of z 2  and z 1  is below a threshold value for the depth z crit , i.e., 
         [0000]      | z   2   −z   1   |&lt;z   crit    
         [0000]    applies. In such a case, the selected pixel P 0  is filled with the value which is interpolated between P 1  and P 2 , i.e., brightness and, if applicable color. The assigned field element of the Z-buffer is likewise set to the interpolated depth between z 1  and z 2 . Interpolation depends on the distance of the selected pixel P 0  from P 1  and P 2  in plane E. 
         [0024]    If the difference of the depths is too big, i.e., the condition 
         [0000]      | z   2   −z   1   |&gt;z   crit    
         [0000]    applies, it is assumed that P 1  and P 2  are located on different planes. The selected pixel P 0  is then filled with the value, i.e., brightnesses and, if applicable color, of, for example, the more remote pixel P 1  or P 2 , and the assigned field element of the Z-buffer with the bigger depth z 1  or z 2 . Alternatively, the value and the depth of pixel P 1  or P 2  having the smaller depth z 1  or z 2  is transferred. In the case of more than two adjacent pixels P 1 , P 2 , the average value of the majority, i.e., of the adjacent pixels P 1 , P 2 , which are located on the same surface, can be transferred. 
         [0025]    Selected pixels P 0 , which are already filled with values of the measurement points, are overwritten by the interpolation of the values of the adjacent pixels P 1  and P 2 . Alternatively, a selected pixel P 0 , which is already filled, remains unvaried. 
         [0026]    If pixels P have been skipped when finding the pixels P 1  and P 2 , because they were not filled or because their depth z was too big, their adjacent pixels P 1 , P 2  are the same as with the selected pixel P 0 , so that the skipped pixels P and the assigned field elements of the Z-buffer, within the framework of the selection taking place in parallel, are likewise filled either with a value which is interpolated between the pixels P 1  and P 2  and/or the depths z 1  and z 2  (depending on the distance of the selected pixel P 0  from P 1  and P 2  in plane E) or with the value and/or the depth z 1  or z 2  of the more remote one among pixels P 1  or P 2  (or the average value of the majority). 
         [0027]    Due to the selection taking place in parallel, filling with the value and/or the depth z 1  or z 2  of the more remote among the pixels P 1  or P 2  on account of a difference of depths which is too big, leads to the closer-by pixel P 1  or P 2  forming an edge. Even if no adjacent pixel P 1  or P 2  is found, the depth z 1  or z 2  of which is smaller than the depth z 0  of the selected pixel P 0 , since the selected pixel P 0  is at the side of the screen  34 , an edge is generated, since these selected pixels P 0  at the edge are not filled then. 
         [0028]    Gap-filling may take place once again to fill further pixels, i.e., to improve the representation once again. 
         [0029]    Gap-filling may take place in the control and evaluation device  22  or by software running on an external computer. Due to the savings in time by a parallel selection, the hardware-based gap-filling on the graphic card  32  may be used together with the programming interface of the latter.