# Patent

## Title
Apparatus for projecting a picture element

## Abstract

An apparatus for projecting a picture element based on at least a first and a second sample comprises a mirror configured to pivot about two axes, at least a first and a second light generation unit, and a processor being configured to output the first sample received at a first point in time at a second point in time to the first light generation unit and the second sample received at a first point in time at a third point in time to the second light generation unit. At least one of the second and third points in time is determined such that the distance between the actual first and second positions is reduced.

## Background
CROSS-REFERENCE TO RELATED APPLICATION(S)
This application claims priority to European Patent Application No. 15 154 275.0, filed Feb. 9, 2015, the disclosure of which is herein incorporated by reference in its entirety.
FIELD OF THE INVENTION
The present invention relates to an apparatus for projecting a picture element composed of at least a first component based on a first sample of a picture signal and a second component based on a second sample of the picture signal.
BACKGROUND
Mirror systems in the form of micro-electro-mechanical systems (MEMS) are known in the art as devices capable of pivoting a mirror around one or two axes at a high frequency, e.g., in order to deflect a light beam in a specific direction. This can be used for projectors which project light onto, e.g., a projection screen or directly into the human eye.
To project light in varying colors, it is known in the state of the art to collimate three light beams of different primary colors such as red, green, and blue, e.g., by means of beam combiners, in order to deflect the coaxial light beams which can then be regarded as a single light beam of varying color.
However, the light generation units, collimation optics, and beam combiners have to be aligned very carefully and precisely with respect to each other to achieve a perfect coaxial match of the beams since an adjustment after the assembly of the components is not possible. Thus, such multi-color projectors are complicated to manufacture, and a re-adjustment or calibration of the individual light generation units in case of mismatch is not possible.
SUMMARY
It is an object of the invention to provide for an improved apparatus for projecting a picture element which overcomes the above-mentioned drawbacks of the state of the art.
To this end, the invention provides for an apparatus for projecting a picture element composed of at least a first component based on a first sample of a picture signal and a second component based on a second sample of the picture signal, the apparatus comprising: a carrier, a mirror mounted on the carrier, being configured to pivot about two axes, and having a mirror period, at least a first and a second light generation unit, each light generation unit being mounted on the carrier, directed at the mirror, and having an input via which it can be controlled, and a processor having an input and a plurality of outputs, wherein each output is connected to one of the inputs of the light generation units, the processor being configured to receive, at its input, the first sample and the second sample at a first point in time within the mirror period, output the first sample at a second point in time within the same mirror period to the first light generation unit, at which second point in time the first light generation unit projects, via the pivoting mirror, the first component of the picture element onto an actual first position on a projection area, output the second sample at a third point in time within the same mirror period to the second light generation unit, at which third point in time the second light generation unit projects, via the pivoting mirror, the second component of the picture element onto an actual second position on the projection area, wherein at least one of the second and third points in time is determined such that the distance between the actual first and second positions is reduced.
Due to the movement of the mirror, each light generation unit projects a trajectory of its components of the picture elements onto the projection area, e.g., a projection plane. If the light generation units are mutually displaced and/or not calibrated, they each project trajectories which are mutually displaced, which leads to a picture of picture elements whose components, e.g., the color components, are offset from one another, i.e., un- or decalibrated. As an example, the “red” component can be at a small spatial distance on the projection screen from the “green” component of one and the same picture element, yielding an unclear, blurred picture. According to the invention, the components of at least one light generation unit are thus (re-)calibrated or shifted towards the components of another light generation unit.
Time-shifting all picture element components of at least one of the light generation units in that way, e.g., the “red”, “green”, and “blue” components towards each other by altering their output times, leads to a picture with locally better matching (color) components.
The invention thus provides for a projection apparatus which can be produced cheaply and efficiently since the light generation units do not have to be aligned in a hardware-based manner but can be calibrated by means of a software function or table providing said time-shifting of samples of the picture signal fed to the light generation units.
Optionally, said actual first and second positions are derived from a reference position lying on a reference trajectory. According to this embodiment of the invention, various ways of choosing a reference position or reference trajectory for said time-shifting of the samples are possible.
In a first embodiment, the reference trajectory, i.e., the reference positions, are predetermined. Thus, an arbitrary reference trajectory, e.g., an idealized reference trajectory of straight lines as a grid over the projection area, can be chosen.
In a second embodiment of the invention, the third point in time corresponds to the first point in time, and said actual first position is derived from the actual second position, i.e., the actual second position is the reference position. Thus, for example, the trajectory projected by one light generation unit, e.g., the “green” trajectory, is taken as a reference trajectory, and the “red” light generation unit is calibrated towards this “green” trajectory. This has the advantage that the samples fed to one of the light generation units need not be time-shifted since its trajectory serves as the reference trajectory.
In a third embodiment of the invention, said actual first and second positions are derived from both an estimated second position onto which the second light generation unit would project the second component of the picture element upon receiving the second sample at the first point in time at its input and from an estimated first position onto which the first light generation unit would project the first component of the picture element upon receiving the first sample at the first point in time at its input, i.e., the reference position can be, for example, a (weighted) mean position based on the estimated first and second position. Thus, a (optionally weighted median) reference trajectory, for example, can be chosen based on the trajectories of the first and the second light generation units, which means that, for example, the components of both the “red” and the “green” light generation units are time-shifted such that they meet “in the middle”. This is especially favorable if there are three or more light generation units, such that their trajectories lie around the reference trajectory.
Optionally, the apparatus of the invention is capable of projecting full-color picture elements composed of three components of different primary colors, i.e., the picture element is composed of said first component, said second component, and a third component based on a third sample of the picture signal, and the apparatus comprises three light generation units corresponding to such different colors. Furthermore, the processor is further configured to output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area, wherein the fourth point in time is determined such that the distance between the actual first and third positions and the distance between the actual second and third positions are reduced.
Also in this three-color-component scenario, the reference trajectory can be chosen in different ways. In analogy to the first embodiment explained above for two colors, the first, second, and third positions can be derived from a reference position lying on a reference trajectory, e.g., an arbitrary chosen, predetermined reference trajectory.
In analogy to the second embodiment outlined above for two colors, the processor can alternatively be further configured to output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area, said actual third position being derived from the actual second position. In other words, for example, the “green” trajectory is considered as the reference trajectory and the “red” and “blue” trajectories are shifted towards the “green” trajectory.
And in analogy to the third embodiment explained above for two colors, said actual first position and said actual second position can be further derived from an estimated third position onto which the third light generation unit would project the third component of the picture element upon receiving the third sample at the first point in time at its input, and the processor can be further configured to output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area, said actual third position being derived from the estimated first position, the estimated second position, and the estimated third position. By way of example, a reference trajectory (or simply a reference point) is chosen by calculating a “mean” position towards which the trajectories are shifted, i.e., towards the middle of the “red”, “green”, and “blue” estimated position.
In an embodiment, at least one of the second and third points in time is determined such that the distance between the actual first and second positions is minimized, i.e., the actual first position and the actual second position are derived such that they lie as close together as possible. Since, as mentioned above, the trajectories of the first and second light generation unit do not generally coincide, also the actual first and second positions do not coincide. Thus, by matching the first and second positions as closely as possible, the picture elements can be displayed as sharp as possible.
In a further embodiment, the actual first position and the actual second position are derived such that they lie on a line which is substantially parallel to one of the axes about which the mirror pivots. Thus, for example, the components can be projected on a (virtual) vertical grid pattern on the projection area. Even though, in general, the first and second actual position will still not match globally, at least their horizontal distances can be regular.
To compensate for the sine-like movement of the mirror and the non-linear behavior of the trajectory on the projection screen, the second point in time can be shifted by an amount corresponding to an angular velocity of the mirror at the second point in time.
Because the mirror has an especially low velocity on its turning points (extrema of the trajectory), the picture to be displayed will usually become distorted on the “edges” of the picture. To avoid this problem, the first sample is not projected by the first light generation unit if the angular velocity of the mirror at the second point in time is below a threshold. This results in the edges of the projected picture being “cut off”, i.e., the projected picture is not blurred on its edges.
To achieve an especially dense picture on the projection area, the mirror is optionally a MEMS mirror. MEMS mirrors can be driven to pivot about two axes, e.g., resonant about a first axis and non-resonant about a second axis, to “sweep” the projection area in a line-like manner. This simplifies the control and calibration as described considerably.
The concept of the invention works for all kinds of light projection devices. Two embodiments are described in the following. In a first variant of the invention, the light generation units are configured to emit light beams which are substantially parallel to each other and strike the mirror at mutual distances. This is especially advantageous since the mutual displacement of the components on the projection area can be compensated by the time-shifting scheme according to the invention. Furthermore, beam combiners between the light generation units and the mirror can be avoided.
In a second, alternative variant of the invention, the light generation units are configured to emit light beams which substantially cross at the mirror. This is especially advantageous since the size of the pivoting reflective surface can be minimized, which increases the resonance frequencies of the mirror and hence the resolution of the projected image.
In a further embodiment of the invention, the processor can be configured to interpolate an intensity of the first sample based on the estimated second position between the original intensity of the first sample and an intensity of the further sample. This constitutes a further calibration technique and accounts for the issue that generally the trajectory of neither light generation unit exactly matches the target reference trajectory. By way of example, the intensity of the “red” component should be high on the reference position and low at a neighboring reference position. However, the “red” component actually strikes the projection screen between these reference positions and is thus interpolated between the intense and the light intensity values. Again, the reference position is dependent on the (estimated or actual) position of, e.g., the “green” component.
Said interpolated intensity can in particular be given by




  
    
      I
      
        i
        ,
        new
      
    
    =
    
      
        I
        i
      
      =
      
        
          
            d
            j
          
          
            
              d
              i
            
            +
            
              d
              j
            
          
        
        +
        
          
            I
            j
          
          ⁢
          
            
              d
              i
            
            
              
                d
                i
              
              +
              
                d
                j
              
            
          
        
      
    
  
  ,




wherein
Ii is the original intensity of the first sample,
Ij is an intensity of the further sample,
di is a distance between the first actual position and a reference position derived from the estimated second position, and
dj is a distance between the first actual position and a further reference position corresponding to the further sample.
This linear interpolation is especially useful when the actual first positions and the actual second position lie on a vertical grid pattern since no horizontal displacement has to be corrected. Also, even in general cases it gives a quick interpolation between the sample's own intensity and the intensity of, e.g., the nearest sample. The above-stated formula thus interpolates the intensity of the first sample based on its distance to the reference point above it and below it. One of these reference points corresponds to the position where the component should actually be, and the other reference point corresponds to a position where another (neighboring) component should be.
In an alternative embodiment, the interpolated intensity is obtained by means of an irregular bilinear interpolation using the original intensity of the first sample and intensities corresponding to three further samples. This interpolation accounts for cases in which the component is projected generally between two reference trajectories and thus adjusts the intensity of the sample based on its original intensity and on the intensity of three further (neighboring) samples.
All above-mentioned mappings between a point in time at which a sample is received by the processor and a point in time at which a sample is output by the processor can be stored in a memory connected to the processor, e.g., in form of functions or tables, and the processor uses said mappings, i.e., accesses said memory, when outputting said samples.
In a further aspect of the invention, a method for setting-up said mapping function or table by means of a calibration run is disclosed. In this calibration run, an auxiliary table of points in time and corresponding estimated positions is created on the basis of which the above-mentioned function or table mapping first points in time to second points in time (or third or fourth points in time) can be set up.
To this end, the processor is configured to, before it starts its actual operation as defined above, output—as said “calibration run”—several first and second samples at points in time which are distributed over one whole mirror period to the first and second light generation unit, respectively, and store the positions onto which the light generation units project the corresponding components as said estimated first and second positions, together with each corresponding output time with respect to the mirror period, in the memory, yielding said auxiliary table. The auxiliary table can then be used to find the nearest matching positions of different trajectories, or a trajectory and the predetermined reference trajectory, respectively, from the corresponding times, of which nearest matching positions the final time mapping function or table can then be set-up and stored in the memory for access by the processor during operation.
The invention also provides for a projection device which can re-calibrate said auxiliary table. This is especially useful since the calibration can be done, e.g., on a calibration screen at a close distance to the projection device, whereas the actual projection would be performed onto a different screen in a different distance to the projection device, without having to re-perform said calibration run to set up the auxiliary table.
To this end, the processor is further configured to calculate new estimated first and second positions according to



  
    
      x
      
        d
        ⁢
        
          
        
        ⁢
        2
      
    
    =
    
      
        
          x
          
            d
            ⁢
            
              
            
            ⁢
            1
          
        
        ⁢
        
          
            d
            2
          
          
            d
            1
          
        
      
      +
      
        
          x
          ~
        
        ⁡
        
          (
          
            1
            -
            
              
                d
                2
              
              
                d
                1
              
            
          
          )
        
      
    
  




  and




  
    
      
        y
        
          d
          ⁢
          
            
          
          ⁢
          2
        
      
      =
      
        
          
            y
            
              d
              ⁢
              
                
              
              ⁢
              1
            
          
          ⁢
          
            
              d
              2
            
            
              d
              1
            
          
        
        +
        
          
            y
            ~
          
          ⁡
          
            (
            
              1
              -
              
                
                  d
                  2
                
                
                  d
                  1
                
              
            
            )
          
        
      
    
    ,
  



wherein
xd1 is the horizontal component of the stored estimated first or second position;
yd1 is the vertical component of the stored estimated first or second position;
xd2 is the horizontal component of the new estimated first or second position;
yd2 is the vertical component of the new estimated first or second position;
{tilde over (x)} is a horizontal offset, e.g., between the light generation units;
{tilde over (y)} is a vertical offset, e.g., between the light generation units;
d1 is the distance at which the projection area was during obtaining the stored estimated first or second position; and
d2 is the distance of the projection area at which the new estimated first or second positions are to be projected.
The invention shall now be explained in more detail below on the basis of exemplary embodiments thereof with reference to the accompanying drawings, in which:



BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a schematic side view and beam plot of a projection apparatus with three light generation units according to the invention;
FIG. 2 shows a schematic side view and beam plot of a variant of the projection apparatus of FIG. 1 according to the invention;
FIG. 3 shows a combined schematic side view, beam plot and block diagram of the projection apparatus according to the invention;
FIG. 4 shows a schematic perspective view of a MEMS mirror pivotable about two axes to be used in the projection apparatus of FIGS. 1-3;
FIG. 5 shows an idealized projection trajectory on a projection area in a schematic view;
FIG. 6 shows an idealized projection trajectory as projected by the projection apparatus of FIGS. 1-2;
FIG. 7 shows a projection trajectory as projected by two exemplary light generation units of the projection apparatus of FIGS. 1-3;
FIGS. 8-12 show calibration schemes for the components of a picture element performed in the apparatus according to the invention;
FIGS. 13a and 13b show calibrated projection trajectories on a projection area in a schematic view;
FIGS. 14a, 14b, 15a, 15b show signal timing sequences for the samples before (FIGS. 14a, 14b) and after (FIGS. 15a, 15b) the time-shift calibration, respectively; and
FIGS. 16 and 17 show an intensity interpolation scheme performed in the apparatus according to the invention.



DETAILED DESCRIPTION
FIGS. 1 and 2 show a projection device 1 for projecting a multi-colored light spot, i.e., picture element 2, via a mirror 3 onto a projection area 4, e.g., a screen (FIG. 1) or the human eye (FIG. 2), in a distance d1. The picture element 2 is composed of three individual components 51, 52, 53. To this end, the projection device 1 comprises three light generation units 61, 62, 63—each, e.g., in the form of one and more light emitting diodes (LEDs) or laser diodes—which emit light beams 71, 72, 73 according to the primary colors red (R), green (G), and blue (B). However, the projection device 1 could also only contain two (or more than three) light generation units 61, 62 for purposes other than mixing colors according to the RGB color model, thereby emitting a picture element 2 composed of two (or more) components 51, 52.
The projection device 1 further comprises a carrier 8, onto which the mirror 3 and the light generation units 61, 62, 63 are mounted. Furthermore, an optical element such as a fast axis collimation (FAC) and/or a slow axis collimation (SAC) lens 11 can be mounted on the carrier 8. The carrier 8 can be a single piece or composed of separate elements and can, e.g., include a subcarrier or substrate for mounting the light generation units 61, 62, 63.
According to the embodiment of FIG. 1, the light generation units are configured to emit light beams 71, 72, 73 which are substantially parallel to each other and strike the mirror 3 at mutual distances. If the light beams 71, 72, 73 were to be emitted at same times, this would also cause the components 51, 52, 53 of a single picture element 2 to be projected onto mutually displaced positions on the projection area 4. According to the embodiment of FIG. 2, the light generation units 61, 62, 63 are configured to emit light beams 71, 72, 73 which substantially cross at the mirror 3. Thus, a smaller reflective surface 22 of the mirror 3 can be used, but the components 51, 52, 53 of a single picture element 2 would still be projected onto mutually displaced positions on the projection area 4. Either of the embodiments of FIGS. 1 and 2 can be used to project the picture element 2 onto a projection area 4 in the form of a screen or a human eye.
FIG. 3 shows the controlling of the light generation units 61, 62, 63 by means of a processor 9. The processor 9 receives a picture signal 10 at its input 12. The picture signal 10 is composed of a “red” sequence Rin of first samples s1,i, a “green” sequence Gin of second samples s2,i, and a “blue” sequence Bin of third samples s3,i, wherein i=1, 2, . . . denotes the index of time (sample index) within the picture signal 10. Each sample s1,i, s2,i, s3,i corresponds to one of the components 51, 52, 53 of the picture element 2 at the time ti, called 2i, and will thus be denoted by 51,i, 52,i, 53,i from hereon. As will be explained later in detail, the processor 9 outputs the samples s1,i, s2,i, s3,i received at its input 12 to outputs 131, 132, 133 connected to corresponding inputs 141, 142, 143 of the three light generation units 61, 62, 63.
Via the mirror 3, the picture elements 2i are projected across the whole projection area 4. To this end, the mirror 3 is capable of pivoting about two axes 17, 18. FIG. 4 shows such a micro-electro-mechanical system (MEMS) mirror 3. There are various embodiments of the actuation principle of the MEMS mirror 3, e.g., electrostatic, electromagnetic, piezoelectric, electrothermal, or magnetostrictive. In the case of an electromagnetic actuation, by feeding a current to a coil 16 of the mirror 3 lying in a magnetic field B, a movement about the two axes 17, 18 is induced to pivot the mirror plate 22. Thus, the light beam 71 is deflected in varying directions.
Using such a MEMS mirrors 3 allows a fast pivoting movement about the axis 17 if the mirror 3 is driven at a resonance frequency corresponding to the axis 17. Optionally, the mirror 3 is driven at a lower (non-resonant) frequency about the other axis 18 to allow a line-like “scanning” or “sweeping” movement of the light beam 71 over the projection area 4, as will be described below in detail by means of FIG. 5. However, it is also possible to drive the mirror 3 at, e.g., two different resonance frequencies, each corresponding to one of the axes 17, 18. This deflects each of the light beams 71, 72, 73 (71 being depicted as an example in FIG. 4) to sweep the projection area 4 according to a Lissajous curve. In any case, the least common multiple of the two pivoting periods around the two axes 17, 18 constitutes a mirror period pm, after which the mirror 3 resumes the same alignment as before.
FIG. 5 shows an idealized scanning pattern of congruent trajectories 191, 192, 193 on the projection area 4 caused by deflecting each one of the light beams 71, 72, 73 according to the slow and fast axis scheme described above in perfect alignment. Since the processor 9 outputs the samples s1,i, s2,i, s3,i at successive times i, picture elements 2i of perfectly spatially aligned components 51,i, 52,i, 53,i are projected at discrete locations along the trajectory 191=192=193.
In contrast thereto, FIG. 6 shows a realistic scanning pattern of three non-congruent trajectories 191, 192, 193 on the projection area 4 caused by deflecting the three light beams 71, 72, 73 onto the projection area 4. Due to the mutual displacement of the light generation units 61, 62, 63, also the trajectories 191, 192, 193 are mutually displaced. The components 51,i, 52,i, 53,i of a single picture element 2i are thus spatially dispersed over different positions 201,i, 202,i, 203,i (depicted by squares, circles, and triangles, respectively, in the figures), yielding a picture on the projection area 4 which is misaligned or blurred in its color components.
To simplify the matter at hand, FIG. 7 shows only two trajectories 191, 192 which are mutually displaced both in a horizontal and in a vertical direction; it goes without saying that all explanations given herein for two trajectories and two colors apply mutatis mutandis to the case of three trajectories and three colors.
To compensate for the spatial misalignment of the components 51,i, 52,i, 53,i within a picture element 2i, the processor 9 performs a “time-shifting” or “time-mapping” function on the samples s1,i, s2,i, s3,i received at its input 12 for outputting them at different instances of time within a mirror period pm such that the components 51,i, 52,i, 53,i of a picture element 2i are projected in closer spatial proximity to each other.
Thus, according to the calibration technique described herein, a “theoretical” or “estimated” reference trajectory 19r is introduced, see FIGS. 8 and 9. In the simplest case, one of the two trajectories 191, 192 is declared as the reference trajectory 19r (trajectory 192 in this example).
In the course of the reference trajectory 19r, e.g., a sample s2,i is output to the light generation unit 62 by the processor 9 at the time ti (also called “first” point in time in the following), whereupon the component 52,i is actually projected onto the position p2,a, (also called actual “second” position in the following and serves as a reference position pr in this case).
On the other hand, in the course of the first trajectory 191, if the processor 9 would output the sample s1,i at the time ti, this would lead to a projection of the component 51,i of the same picture element 2i onto the (misaligned) “theoretical” position p1,e, also called “estimated” first position in the following.
Now, to project the first component 51,i in a close proximity to the actual position p2,a, of the second component 52,i in order to achieve a well-defined picture element 2i, the processor 9 actually outputs the sample s1,i—according to its time-shifting function—not at the first time ti, but at another point in time tn in the course of the trajectory 191, i.e., within the mirror period pm, called “second” point in time tn in the following, so that the sample s1,i is projected onto the actual first position p1,a. Thus, the distance between the actual first and second positions (p1,a, p2,a) is reduced with respect to the distance between the estimated first and second positions (p1,e, p2,e).
In the present case where the trajectory 192 is considered as the reference trajectory 19r, it can thus be said that the second point in time tn and therefore the first actual position p1,a are “derived” from the actual second position p2,a, (the reference position pr). For clarification, it is noted that in the case where the reference trajectory 19r corresponds to the trajectory 192, the actual second position p2,a, is equal to an “estimated” second position p2,e.
For performing the time-shifting function s1,i(ti)→s1,i(tn), the processor 9 can access a corresponding function or table, e.g., when outputting said sample s1,i, s2,i, s3,i, the function or table mapping first points in time ti to second points in time tn which can be stored in a memory 21 connected to the processor 9. More precisely, this function or table is a mapping between a point in time ti at which a sample is received by the processor 9 and a point in time tn, tm, tk at which a sample is output by the processor 9.
Said function or table can either be generated analytically by means of known mutual displacements or known aging effects or can be generated by means of a calibration run as follows.
In a first step, the processor outputs several first and second samples s1,i, s1,i+1, s2,i, s2,i+1 at points in time ti, ti+1, . . . distributed over one whole mirror period pm to the first and second light generation unit 61, 62, respectively. Said points in time ti, ti+1, . . . are optionally spaced at regular intervals which are short enough to obtain a large number of samples over the mirror period. Each output sample will then yield a component 51,i, 52,i on the projection area 4 at the distance d1 from the projection device 1. The sum of the projected components thus draws the trajectories 191, 192 on the projection area 4.
In a second step, the processor, or a different entity, stores said positions drawn on the projection area 4, i.e., onto which the light generation units 61, 62 have projected the components 51,i, 52,i, as estimated first and second positions p1,e, p2,e, together with each corresponding output time ti, ti+1 with respect to the mirror period pm, in the memory 21 in form of an auxiliary function or table. Thus, for each trajectory 191, 192, or light generation unit 61, 62, respectively, said auxiliary table relates estimated positions p1,e, p2,e to their corresponding output times ti.
In a third step, the final function or table mapping first points in time ti to second points in time tn is created and stored in the memory 21 by the processor 9 using said auxiliary tables. To this end, for example, each estimated first position p1,e of time ti, as retrieved from the auxiliary table, is positionally matched to the nearest estimated second position p2,e (generally: matched to a nearest reference position pr) of time tn, as retrieved from the auxiliary table (generally: as retrieved from a predetermined auxiliary table), which results in a mapping of time t1 to time tn. All time mappings obtained in this way yield a final mapping table or function to be stored in memory 21. It is also possible to interpolate a more precise second point in time tn, e.g., by retrieving two nearest estimated second postions p2,e of times tn, tn+1 and taking a (weighted) median of these two times as a new second point in time tn.
Furthermore, it is possible to calibrate the projection device 1 by means of the projection area 4 for a distance d1 and then project a calibrated image onto a projection area 4 at a different distance d2. To this end, the positions in the preliminary table can be re-calculated as “new” estimated first and second positions p1,e, p2,e according to



  
    
      x
      
        d
        ⁢
        
          
        
        ⁢
        2
      
    
    =
    
      
        
          x
          
            d
            ⁢
            
              
            
            ⁢
            1
          
        
        ⁢
        
          
            d
            2
          
          
            d
            1
          
        
      
      +
      
        
          x
          ~
        
        ⁡
        
          (
          
            1
            -
            
              
                d
                2
              
              
                d
                1
              
            
          
          )
        
      
    
  




  
    
      y
      
        d
        ⁢
        
          
        
        ⁢
        2
      
    
    =
    
      
        
          y
          
            d
            ⁢
            
              
            
            ⁢
            1
          
        
        ⁢
        
          
            d
            2
          
          
            d
            1
          
        
      
      +
      
        
          y
          ~
        
        ⁡
        
          (
          
            1
            -
            
              
                d
                2
              
              
                d
                1
              
            
          
          )
        
      
    
  



wherein
xd1 is the horizontal component of the stored estimated first or second position p1,e, p2,e;
yd1 is the vertical component of the stored estimated first or second position p1,e, p2,e;
xd2 is the horizontal component of the new estimated first or second position p1,e, p2,e;
yd2 is the vertical component of the new estimated first or second position p1,e, p2,e;
{tilde over (x)} is a horizontal offset, e.g., between the light generation units 61, 62;
{tilde over (y)} is a vertical offset, e.g., between the light generation units 61, 62.
After determining the new estimated first and second positions p1,e, p2,e for the distance d2, the final function or table mapping the first points in time ti to the second points in time tn can then be freshly created by determining mapping each first estimated position p1,e towards the respective estimated second position p2,e.
It is understood that the described calibration techniques can be performed for the third light generation unit 63 and by means of the reference trajectory 19r technique, too.
In a further embodiment, depicted in FIG. 10, the reference trajectory 19r can be chosen as a (e.g., weighted) mean trajectory lying between the trajectories 191 and 192, and the reference position pr is chosed as a (e.g., weighted) mean position lying between the estimated first and second positions p1,e and p2,e. In this case, from the viewpoint of the first light generation unit 61, each “second” (shifted) time tn is dependent on both the estimated second position p2,e and the estimated first position p1,e, i.e., on the (mean) reference trajectory 19r.
Much in the same way, from the viewpoint of the second light generation unit 62, the above time-shifting can be performed by the processor 9 mutatis mutandis. That is, the processor 9 outputs the second sample s2,i at a “third” point in time tm within the same mirror period pm to the second light generation unit 62, at which third point in time tm the second light generation unit 62 projects, via the pivoting mirror 3, its second component 52,i of the picture element 2i onto the “actual” second position p2,a on the projection area 4. In other words, it can be said that the actual second position p2,a, is “derived” from the estimated second position p2,e and the estimated first position p1,e, i.e., from the (mean) reference trajectory 19r.
In a more general case, depicted in FIG. 11, the reference trajectory 19r can be predetermined, i.e., not dependent on the estimated first and second positions p1,e, p2,e. The reference trajectories 19r can, for example, be chosen to be formed by straight parallel lines, such that the reference positions pr cover the projection area 4 in a regular pattern.
FIG. 12 shows the application of the described calibration scheme to all three light generation units 61, 62, 63. Again, either one of the trajectories 191, 192, 193 can be considered as the reference trajectory 19r or, e.g., a mean reference trajectory 19r is taken, whereupon the actual positions p1,a, p2,a, p3,a of the components 51,i, 52,i, 53,i stemming from the samples s1,i, s2,i, s3,i at output times tn, tm, tk are each derived from the estimated positions p1,e, p2,e, p3,e at their common input time ti, i.e., from the (mean) reference trajectory 19r. Again, the corresponding function/s or table/s can be stored in the memory 21.
FIG. 13a shows the projection screen 4, onto which the first components 51,i have been projected in a calibrated manner as described, wherein the second components 52,i have been declared reference points on the reference trajectory 19r. FIG. 14a shows the corresponding samples s1,i, s2,i of the input picture signal 10 in the two component channels Gin and Rin for the color components “green” and “red”. To emphasize the mapping, successive incoming samples s1,i, s2,i have different intensities I1,i, I2,i. For example, this could be used for a test signal used for the calibration technique described above.
FIG. 15a shows the mapped samples s1,i, s2,i appearing at the outputs 131, 132 of the processor 9, i.e., after the time-shifting function, for the two component channels Gout, (“green”) and Rout (“red”). It can be seen that, e.g., sample s1,i at input time ti has been shifted to the output time tn and there spatially forms the picture element 2i with closer proximity of the components 51,i and 52,i.
Furthermore, from FIG. 15a it can be seen that samples s1,i, s2,i at the “edge” of the projection area 4 can be deleted, see the exemplary gap 24 in the upper diagram of FIG. 15a. The first reason for this is that the angular velocity of the mirror 3 becomes too slow at its turning-points, such that the components 51,i, 52,1 cannot be projected accurately enough. The second reason is that due to the mutual displacement there may be no overlap at all of the trajectories 191, 192 in the edge area. To account for both cases, a first sample s1,i and/or a second sample s2,i is not projected by the respective light generation unit 61, 62 if the angular velocity of the mirror 3 at the second (output) points in time tn, tm is below a given threshold. Only components 51,i, 52,i within the frame 23 on the projection area 4 are thus actually projected.
From FIGS. 14a and 15a it can be seen that the times ti, tn have been considered as “regularly discrete”, i.e., at regular intervals, so far. This results in actual first positions p1,a and actual second position p2,a as shown in FIG. 13a. Contrary to that, FIGS. 13b-15b show a variant of the described calibration scheme wherein the actual first positions p1,a and the actual second positions p2,a were derived such that they lie on lines 25 which are substantially parallel to one of the axes 17, 18 about which the mirror 3 pivots. To this end, the intervals between the output times tn can be chosen to be irregular. In particular, second points in time tn of a sample s1,i are shifted by an amount corresponding to an angular velocity of the mirror 3 at the second point in time tn. By means of this, the non-linear, sine-like movement of the mirror 3 can be compensated and a perfect alignment along regularly spaced lines 25 (FIG. 13b) can be achieved.
Again, components 51,i, 52,i could be skipped to account for the edge of the projection area 4. Alternatively, all components 51,i, 52,i could be “squeezed” such that they all fit within the defined frame 23 on the projection area 4. The frame 23 can be chosen so that the angular velocity of the mirror 3 within the frame 23 is always above a given threshold.
FIGS. 16 and 17 show a further calibration technique performed with time-shifted components 51,i as described above. As can be seen from FIG. 13a, even though the first components 51,i have been adjusted towards the second components 52,i serving as reference points, there may still be an offset between them. In other words, a component 51,i should be at its reference position pr,i (in this case the actual second position p2,a, in general a position derived from at least the estimated second position p2,e). Thus, the intensity Ii carried by the sample s1,i should actually occur at the reference position pr,i. To achieve this aim, the intensity Ii,new of the sample s1,i is interpolated between its original intensity Ii corresponding to its reference position pr,i and an intensity Ij which should occur at a neighboring reference position pr,j. The latter intensity Ij is carried by a sample s1,j (j≠i) which should be at the reference position pr,j. The intensities Ii, Ij should be weighted by the corresponding distances, whereupon the intensity Ii,new of the sample s1,i can be calculated by means of a linear interpolation, e.g., by




  
    I
    
      i
      ,
      new
    
  
  =
  
    
      I
      i
    
    =
    
      
        
          d
          j
        
        
          
            d
            i
          
          +
          
            d
            j
          
        
      
      +
      
        
          I
          j
        
        ⁢
        
          
            d
            i
          
          
            
              d
              i
            
            +
            
              d
              j
            
          
        
      
    
  




wherein
Ii is the original intensity of the first sample s1,i,
Ij is an intensity of the further sample si,j,
di is a distance between the first actual position p1,a and a reference position pr,i derived from the estimated second position p2,e, and
dj is a distance between the first actual position p1,a and a further reference position pr,j corresponding to the further sample s1,j.
FIG. 17 corresponds to the calibration scheme of FIG. 13a. The same principles as described above apply, but this time there is also an offset in the horizontal direction. To this end, two more intensities Io, Ip can be considered to perform a so-called irregular bilinear interpolation, which accounts for components 51,i, 52,i forming a non-regular pattern on the projection area 4. To this end, a weighted average is determined by means of a distance di between the first actual position p1,a and a reference position pr,i derived from the estimated second position p2,e, and distances dj, do, dp between the first actual position p1,a and three further reference positions pr,j, pr,o, pr,p corresponding to the further samples s1,j, s1,o, s1,p. To simplify the matter, in a first approximation a bilinear interpolation or any other interpolation whose mathematical outline is known in the state of the art can be performed, possibly also using the weighted average of more than four intensities.
The invention is not restricted to the specific embodiments described in detail herein, but encompasses all variants, combinations and modifications thereof that fall within the framework of the appended claims.

## Claims

1. An apparatus for projecting a picture element composed of at least a first component based on a first sample of a picture signal and a second component based on a second sample of the picture signal, the apparatus comprising:
a carrier;
a mirror mounted on the carrier, being configured to pivot about two axes, and having a mirror period;
at least a first and a second light generation unit, each light generation unit being mounted on the carrier, directed at the mirror, and having an input via which it can be controlled; and
a processor having an input and a plurality of outputs, wherein each output is connected to one of the inputs of the light generation units;
the processor being configured to
receive, at its input, the first sample and the second sample at a first point in time within the mirror period,
output the first sample at a second point in time within the same mirror period to the first light generation unit, at which second point in time the first light generation unit projects, via the pivoting mirror, the first component of the picture element onto an actual first position on a projection area,
output the second sample at a third point in time within the same mirror period to the second light generation unit, at which third point in time the second light generation unit projects, via the pivoting mirror, the second component of the picture element onto an actual second position on the projection area;
wherein at least one of the second and third points in time is determined such that the distance between the actual first and second positions is reduced; and
wherein said actual first and second positions are derived from both an estimated second position onto which the second light generation unit would project the second component of the picture element upon receiving the second sample at the first point in time at its input and from an estimated first position onto which the first light generation unit would project the first component of the picture element upon receiving the first sample at the first point in time at its input.

2. The apparatus according to claim 1, wherein said actual first and second positions are derived from a reference position lying on a reference trajectory.

3. The apparatus according to claim 1, wherein the third point in time corresponds to the first point in time; and
wherein said actual first position is derived from the actual second position.

4. The apparatus according to claim 1 for projecting a picture element composed of the first component, the second component, and a third component based on a third sample of the picture signal, the apparatus comprising three light generation units corresponding to different colors, wherein the processor is further configured to
output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area;
wherein the fourth point in time is determined such that the distance between the actual first and third positions and the distance between the actual second and third positions are reduced.

5. The apparatus according to claim 1, wherein at least one of the second and third points in time is determined such that the distance between the actual first and second positions is minimized.

6. The apparatus according to claim 2, wherein the actual first position and the actual second position are derived such that they lie on a line which is substantially parallel to one of the axes about which the mirror pivots.

7. The apparatus according to claim 1, wherein the second point in time is shifted by an amount corresponding to an angular velocity of the mirror at the second point in time.

8. An apparatus for projecting a picture element composed of at least a first component based on a first sample of a picture signal and a second component based on a second sample of the picture signal, the apparatus comprising:
a carrier;
a mirror mounted on the carrier, being configured to pivot about two axes, and having a mirror period;
at least a first and a second light generation unit, each light generation unit being mounted on the carrier, directed at the mirror, and having an input via which it can be controlled; and
a processor having an input and a plurality of outputs, wherein each output is connected to one of the inputs of the light generation units;
the processor being configured to
receive, at its input, the first sample and the second sample at a first point in time within the mirror period,
output the first sample at a second point in time within the same mirror period to the first light generation unit, at which second point in time the first light generation unit projects, via the pivoting mirror, the first component of the picture element onto an actual first position on a projection area,
output the second sample at a third point in time within the same mirror period to the second light generation unit, at which third point in time the second light generation unit projects, via the pivoting mirror, the second component of the picture element onto an actual second position on the projection area;
wherein at least one of the second and third points in time is determined such that the distance between the actual first and second positions is reduced; and
wherein the first sample is not projected by the first light generation unit if the angular velocity of the mirror at the second point in time is below a threshold.

9. The apparatus according to claim 1, wherein the processor is further configured to interpolate an intensity of the first sample based on the estimated second position between the original intensity of the first sample and an intensity of a further sample.

10. The apparatus according to claim 9, wherein the interpolated intensity is given by




  
    
      I
      
        i
        ,
        new
      
    
    =
    
      
        I
        i
      
      =
      
        
          
            d
            j
          
          
            
              d
              i
            
            +
            
              d
              j
            
          
        
        +
        
          
            I
            j
          
          ⁢
          
            
              d
              i
            
            
              
                d
                i
              
              +
              
                d
                j
              
            
          
        
      
    
  
  ,



wherein
Ii is the original intensity of the first sample,
Ij is an intensity of the further sample,
di is a distance between the first actual position and a reference position derived from the estimated second position, and
dj is a distance between the first actual position and a further reference position corresponding to the further sample.

11. The apparatus according to claim 9, wherein the interpolated intensity is obtained by means of an irregular bilinear interpolation using the original intensity of the first sample and intensities corresponding to three further samples.

12. The apparatus according to claim 1, wherein a mapping between a point in time at which a sample is received by the processor and a point in time at which a sample is output by the processor, is stored in a memory connected to the processor, and wherein the processor uses said mapping when outputting said sample.

13. The apparatus according to claim 1, wherein the processor is configured to,
before the processor receives the first sample at the first point in time,
output two or more first and second samples at points in time distributed over one whole mirror period to the first and second light generation unit, respectively, and
store the positions onto which the light generation units project the corresponding components as said estimated first and second positions, together with each corresponding output time with respect to the mirror period, in the memory.

14. An apparatus for projecting a picture element composed of at least a first component based on a first sample of a picture signal and a second component based on a second sample of the picture signal, the apparatus comprising:
a carrier;
a mirror mounted on the carrier, being configured to pivot about two axes, and having a mirror period;
at least a first and a second light generation unit, each light generation unit being mounted on the carrier, directed at the mirror, and having an input via which it can be controlled; and
a processor having an input and a plurality of outputs, wherein each output is connected to one of the inputs of the light generation units;
the processor being configured to
receive, at its input, the first sample and the second sample at a first point in time within the mirror period,
output the first sample at a second point in time within the same mirror period to the first light generation unit, at which second point in time the first light generation unit projects, via the pivoting mirror, the first component of the picture element onto an actual first position on a projection area,
output the second sample at a third point in time within the same mirror period to the second light generation unit, at which third point in time the second light generation unit projects, via the pivoting mirror, the second component of the picture element onto an actual second position on the projection area;
wherein at least one of the second and third points in time is determined such that the distance between the actual first and second positions is reduced;
wherein the processor is configured to,
before the processor receives the first sample at the first point in time,
output two or more first and second samples at points in time distributed over one whole mirror period to the first and second light generation unit, respectively, and
store the positions onto which the light generation units project the corresponding components as said estimated first and second positions, together with each corresponding output time with respect to the mirror period, in the memory; and
wherein the processor is configured to calculate new estimated first and second positions according to




  
    
      x
      
        d
        ⁢
        
          
        
        ⁢
        2
      
    
    =
    
      
        
          x
          
            d
            ⁢
            
              
            
            ⁢
            1
          
        
        ⁢
        
          
            d
            2
          
          
            d
            1
          
        
      
      +
      
        
          x
          ~
        
        ⁡
        
          (
          
            1
            -
            
              
                d
                2
              
              
                d
                1
              
            
          
          )
        
      
    
  




  and




  
    
      
        y
        
          d
          ⁢
          
            
          
          ⁢
          2
        
      
      =
      
        
          
            y
            
              d
              ⁢
              
                
              
              ⁢
              1
            
          
          ⁢
          
            
              d
              2
            
            
              d
              1
            
          
        
        +
        
          
            y
            ~
          
          ⁡
          
            (
            
              1
              -
              
                
                  d
                  2
                
                
                  d
                  1
                
              
            
            )
          
        
      
    
    ,
  


wherein
xd1 is the horizontal component of the stored estimated first or second position;
yd1 is the vertical component of the stored estimated first or second position;
xd2 is the horizontal component of the new estimated first or second position;
yd2 is the vertical component of the new estimated first or second position;
{tilde over (x)} is a horizontal offset between the light generation units;
{tilde over (y)} is a vertical offset between the light generation units;
d1 is the distance at which the projection area was during obtaining the stored estimated first or second position; and
d2 is the distance of the projection area at which the new estimated first or second positions are to be projected.

15. The apparatus according to claim 8, wherein said actual first and second positions are derived from a reference position lying on a reference trajectory.

16. The apparatus according to claim 8, wherein the third point in time corresponds to the first point in time; and
wherein said actual first position is derived from the actual second position.

17. The apparatus according to claim 8, wherein said actual first and second positions are derived from both an estimated second position onto which the second light generation unit would project the second component of the picture element upon receiving the second sample at the first point in time at its input and from an estimated first position onto which the first light generation unit would project the first component of the picture element upon receiving the first sample at the first point in time at its input.

18. The apparatus according to claim 8 for projecting a picture element composed of the first component, the second component, and a third component based on a third sample of the picture signal, the apparatus comprising three light generation units corresponding to different colors, wherein the processor is further configured to
output the third sample at a fourth point in time within the same mirror period to the third light generation unit, at which fourth point in time the third light generation unit projects, via the pivoting mirror, the third component of the picture element onto an actual third position on the projection area;

wherein the fourth point in time is determined such that the distance between the actual first and third positions and the distance between the actual second and third positions are reduced.

19. The apparatus according to claim 8, wherein at least one of the second and third points in time is determined such that the distance between the actual first and second positions is minimized.

20. The apparatus according to claim 8, wherein the second point in time is shifted by an amount corresponding to an angular velocity of the mirror at the second point in time.

21. The apparatus according to claim 8, wherein the processor is further configured to interpolate an intensity of the first sample based on the estimated second position between the original intensity of the first sample and an intensity of a further sample.

22. The apparatus according to claim 8, wherein the processor is configured to,
before the processor receives the first sample at the first point in time,
output two or more first and second samples at points in time distributed over one whole mirror period to the first and second light generation unit, respectively, and

store the positions onto which the light generation units project the corresponding components as said estimated first and second positions, together with each corresponding output time with respect to the mirror period, in the memory.