Abstract:
In an exemplary display system adapted to display an image to an observer using a display where at least one of the display and observer is subjected to physical vibration referred to as jitter, accelerometers are mounted to elements subject to jitter and associated with the observer. A processing system uses acceleration measurements from the accelerometers to calculate an x-axis and y-axis jitter correction factors based on a comparison of the acceleration measurements, and generates a video output in which the visual information is displaced along the x-axis and y-axis based on the x-axis and y-axis jitter correction factors so that an image corresponding to the video output shown on the display appears to the observer as not having jitter.

Description:
BACKGROUND 
     This invention relates to the electronic display of an image and more specifically relates to compensating for undesired external physical movement (jitter) associated with a displayed image. 
     Environments exist in which significant jitter becomes associated with a displayed image. For example, a display system in a helicopter may display an image on a screen for the pilot where the image may include video information from a video camera, night vision viewing device, infrared viewing device, etc. mounted to the helicopter. In this relatively high vibration environment, the image displayed on the screen will be perceived by the pilot as having a substantial amount of jitter. There exists a need to minimize the perceived jitter of images displayed in a high vibration environment. 
     SUMMARY 
     In an embodiment of the invention, a display system is adapted to display an image to an observer using a display where at least one of the display and observer is subjected to physical vibration referred to as jitter. A first set of accelerometers is mounted to the display and a second set of accelerometers associated with the observer, such as mounted to a helmet worn by the observer. Each set of accelerometers comprises one or more accelerometers providing sensing of acceleration in one or more axes as appropriate. A processing system receives first and second sets of acceleration measurements from the first and second sets of accelerometers, respectively. A source of visual information provides the visual information to the processing system. The processing system calculates an x-axis and y-axis jitter correction factor based on a comparison of the first and second sets of acceleration measurements, generates a video output in which the visual information is displaced along the x-axis and y-axis based on the x-axis and y-axis jitter correction factors, respectively, and transmits the video output to the display so that an image corresponding to the video output shown on the display does not appear to the observer to be subject to jitter. 
     In another embodiment, the jitter correction factors are based on three sets of accelerometer measurements associated with the display, source of visual information and the observer where all are in an environment subject to jitter. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram illustrating an embodiment of a display system in accordance with the present invention in which jitter is minimized. 
         FIG. 2  is a flow diagram showing steps in an embodiment of an exemplary method in accordance with the present invention for minimizing jitter in a display system. 
     
    
    
     DETAILED DESCRIPTION 
     One of the aspects of the present invention resides in the recognition of the causes associated with jitter in display systems where multiple elements of the system are experiencing jitter. The source of jitter may come from jitter motion associated with the image capturing device, jitter motion associated with the screen itself on which the image is displayed, jitter motion associated with the head of the observer, or a combination thereof. For example, only compensating for jitter motion associated with the image capturing device relative to a fixed point of reference may be effective for some applications such as stabilizing images captured by a television camera undergoing jitter motion. Such compensation can be effective where the screen on which the image is displayed and the observer are not undergoing significant jitter motion. However, such compensation is not sufficient to present a satisfactory image to an observer where two or more elements in the display system (information source, display screen, observer) are each undergoing independent or semi-independent jitter motion. As explained with regard to the following description of an embodiment of the invention, compensation for jitter motion being experienced by multiple elements of the display system is provided in order to provide an image with minimized jitter from the perspective of the observer. 
       FIG. 1  illustrates a processing system  10  that provides a video signal to an electronic display  12  and receives information to be shown on the display from sensor  14 . Display  12  can comprise any type of electronic video display and preferably is a video display that can accommodate screen rewrite rates of 30 Hz or higher, i.e. the ability to rewrite the screen at a rate at least faster than the ability of the human eye to follow each rewrite but preferably at a much higher rate, such as 300 Hz or higher. For example, an organic light emitting diode (OLED) display with rewrite times in the tens of microseconds would be suitable. The sensor  14  may comprise a video camera, other types of light sensors, or a sensor of other information where the visual presentation of the information can be adversely impacted by jitter motion of the sensor. The visually depicted information on display  12  is presented to an observer  16 . In this illustrative example, the observer  16  may be a helicopter pilot, the monitor  12  may represent a display screen mounted to a console in the helicopter, and the sensor  14  may be a video camera mounted to the external fuselage of the helicopter for reconnaissance. 
     Accelerometers (or other inertial measurement units potentially also including gyroscopes)  18 ,  20  and  22  are mounted respectively to the display  12 , sensor  14  and observer  16 . The accelerometer  18  is adapted to provide two-dimensional acceleration measurements along the x-axis and y-axis as indicated on the screen of display  12 . That is, the acceleration measurements correspond to acceleration in the plane of the screen of the display. Accelerometers  20  and  22  provide similar two-dimensional acceleration measurements, preferably along the same x-axis and y-axis as defined for accelerometer  18 . The accelerometers  22  serves to monitor jitter motion associated with the observer, and in the illustrative example where the observer is a helicopter pilot, accelerometers  22  may be preferably mounted to the pilot&#39;s helmet or headset. It will also be appreciated that accelerometers  22  could be mounted to the pilot&#39;s seat, but would not provide acceleration measurements that would be as accurate as those provided by the accelerometers being, mounted as close as possible to track the motion of the head of the pilot. Each accelerometer set supplies an output containing two-dimensional acceleration information. In more sophisticated embodiments, a full inertial measurement unit comprising three axes of acceleration and angular rate sensing may be used in place of accelerometers  18 ,  20 , and  22  to provide a 3 dimensional measurement of displacement at each of the three locations. These displacement measurements can then be projected in the appropriate plane for stabilization of the image. 
     The exemplary processing system  10  includes a microprocessor  24  that is supported by read-only memory (ROM)  26 , random access memory (RAM)  28 , and a nonvolatile data storage device  30  such as a hard drive. The microprocessor  24  is connected to a video output card  32  that supplies a video output signal to display  12 . An input/output (I/O) interface device  34  is coupled to microprocessor  24  and receives acceleration measurements from accelerometers  18 ,  20  and  22 . The interface device  34  also receives digital information from sensor  14  where the digital information is sent to the microprocessor  24  to be processed and forwarded to the video output device  32  for transmission to the display  12 . The digital information may comprise digitized output from a video camera or other sensor. 
     The microprocessor  24  operates under stored program control instructions that may be stored in ROM  26  and/or storage device  30 . As will be understood by those skilled in the art, microprocessor  24  performs a variety of conventional functions and tasks. In accordance with the exemplary embodiment, two-dimensional acceleration measurements from each of the accelerometers are periodically read and stored for use in jitter compensation calculations that are described in more detail in regard to  FIG. 2 . The microprocessor  24  processes the digital information received from sensor  14  and generates a modified video signal transmitted to video output card  32  that is based on the digital information received from sensor  14  and jitter compensation calculations. 
       FIG. 2  illustrates a flow diagram showing steps in accordance with an exemplary method that may be practiced by the embodiment as shown in  FIG. 1 . In step  50  the acceleration measurements are periodically retrieved from each of the accelerometers. The rate at which the acceleration data is read is preferably equal to or greater than the rate at which the screen of the display  12  is to be refreshed. The total jitter as measured by all of the accelerometers, except for the accelerometer associated with the display, is computed in step  52 . In the illustrative example, the total jitter is defined by adding the respective x-axis and y-axis acceleration measurements by accelerometers  20  and  22 . 
     In step  54  x-axis and y-axis jitter correction factors are calculated to be applied in modifying the video to be displayed to the observer. The x-axis correction factor is calculated by comparing the total x-axis jitter with the x-axis acceleration measurements from accelerometer  18  representing the x-axis jitter associated with display  12 . The difference between the total x-axis jitter and the x-axis jitter associated with display  12  is utilized to modify the placement of the image on the screen with regard to the x-axis. Linear displacement can be calculated using the well-known techniques such as double integration of the acceleration. An approximation for linear displacement may also comprise an equation in which displacement is proportional to acceleration multiplied by time squared (assuming the velocity component is zero or small enough to be ignored as in the illustrative example). Thus, the distance to shift the image to be displayed along the x-axis to stabilize the image from the perspective of the observer can be computed based on the x-axis correction factor processed to yield an image displacement value. This will result in the projection of the image at a location along the x-axis so as to appear stationary or not having moved due to jitter from the perspective of the observer. The y-axis correction factor is calculated similarly in order to determine the amount, if any, that the image should be shifted in the y-axis so as to appear stabilized from the perspective of the observer. Filters (in particular high-pass or band-pass filters) may be applied to the either the acceleration measurements or to the displacement value in order to ensure that corrections are only applied for high frequencies and not for motions resulting from actual maneuvers of the vehicle. In this way the correction accounts for jitter but does not negate a change in position of an object shown in the image where the observer is deliberately moving with respect to the object. The filter characteristics are to be chosen depending on the characteristics of the vehicle, the frequency bands of the jitter motion, and the frequency bands where image jitter suppression is desirable. 
     In step  56  modified video is generated by the microprocessor based on the video information received from the sensor  14  and the x-axis and y-axis correction factors. The video information to be transmitted to the video output card  32  is modified so that the video information output from the video output card  32  to display  12  will be shifted on the screen based on the computed x-axis and y-axis correction factors. In step  58  display  12  is updated with a new frame of video information that has been modified with appropriate x-axis and y-axis image shifting so that undesired external jitter is neutralized from the perspective of observer  16 . This process terminates at END  60 . It will be understood that x-axis and y-axis correction factors are preferably computed for each frame of video (or more frequently if acceleration information is updated more often than for each video frame) and are utilized to generate a shifted image during each frame. 
     Although an embodiment of the present invention has been described, it will be apparent to those skilled in the art that various changes and alterations can be made to the embodiment without departing from the present invention. Depending upon the specific environment, it may be possible to provide adequate image stabilization based on only two different sources of acceleration measurements. For example, assuming a helicopter environment in which the observer is the pilot and the display is mounted to a console in the helicopter that is in motion, a land-based video camera (having no substantial jitter) may be wirelessly transmitted to the helicopter to be displayed. In this situation acceleration measurements would only be required for display  12  and the observer  16  since no significant jitter would be introduced by the sensor  14 , the land-based video camera. Video, in addition to being real-time image information, may comprise a stored image such as a chart, graph, map, or picture. Although the processing system  10  is shown as a separate device such as a computer or work station, the steps of the exemplary method could be performed in a computing environment that may already exist to perform other functions including integration of the method into the display itself. The acceleration measurement data could be integrated for transmission with other information. These modifications are merely intended to suggest some of the possible modifications. The scope of the invention is defined by the following claims.