Abstract:
Proposed is a system for the calculation of gamma correction, which enables an improved display for images, taken under various conditions. Further, the invention accepts the purpose of making available an image processing apparatus, which makes use of this method for the calculation of gamma correction values. The principal concept of the present invention is to be found therein, in that instead of a standardized, fixed gamma correction, a dynamic gamma correction for individual images is created on the basis of image-specific statistical data pertaining to these individual images. By these means, the user of the image is placed in a situation, wherein the user is able to exploit the brightness dynamic realm better than previously.

Description:
CROSS-REFERENCE TO RELATED APPLICATION  
       [0001]     This application claims priority from German Patent Application No. 102006036305.1 filed Aug. 3, 2006.  
       FIELD OF THE INVENTION  
       [0002]     This invention is directed to a system for providing gamma corrected electronic images for internal display in a vehicle capture from an external image capture device so that the operator of a vehicle can be provided with an image of objects exterior to the vehicle.  
       BACKGROUND  
       [0003]     Image capture and display devices are used in many applications, including motor vehicles for capturing electronic images of external objects for display in the interior of a vehicle. These images are particularly helpful when maneuvering a vehicle newer or around objects. However, the environmental conditions under which the images are captured can vary greatly from full sunlight to dark shadows. Therefore it is advantageous to have a image display system that displays images with sufficient detail to assist the operator of the vehicle in maneuvering the vehicle.  
         [0004]     In modern industrial image processing, although also appearing in many regions of motor vehicle activities, image processing systems are desired, which in different applications, provide high quality presentations and images. A gamma-correction offers possibilities for improvement in this regard. Image projection systems, such as RGB and LCD monitors have different brightness gradations and require correction measures frequently, in order to present an image in its best sense. In an ideal manner, a display apparatus would rate the decree of brightness for black as 0, and ascribe to a white reproduction a 1, with interposed values being evaluated as successively increasing or decreasing values of gray. Such a rated scale is representative, respectively, of a gamma rating in the same range. Actual cameras and image reproductive systems possess no such linear function. Most such instruments burden the non-linear characteristics of a display means, for example, an LCD screen, or CCDs of a camera. The results of these non-linear characteristics are that displayed images lack detail, be too dark or too light so that detail of the image is lost. So that, in a case where the capture of an image and the projection of the same is concerned, the least possible brightness data might be lost, or be over emphasized, apparatuses having non-linear input characteristics generally offer a possibility for gamma correction to acquire a linearization of displayed imaging.  
         [0005]     Traditionally, image display systems possess an adjustable, but nevertheless a standard, fixed gamma corrective means. This is used to display the image with certain brightness so that reproduction is capable by the display equipment in use. For example, the non-linearity of a television tube which has a gamma value of 2.2 has to be corrected by a value of gamma=0.45. When the image capture device has the same gamma of the display device, acceptable display of image can be reached with our significant gamma correction. When pictures are taken, in cases where the brightness dynamics are greater than have been assumed, such an arrangement can show considerable disadvantages. Considering image quality in cases troubled by high brightness dynamics and reflective interferences, the resulting displays can evidence lighting qualities which are over or under optimal levels. In such instances, the display equipment shows surfaces which tend toward either white or black. In many cases, this effect is notably emphasized by the commonly used statistical gamma correction, even though the image taking system, i.e. a camera, has acquired image data of considerable importance which will not be used in display.  
         [0006]     Therefore, it is an objective of the present invention to make available a method for the calculation of gamma correction values, which makes possible a better image reproduction.  
       SUMMARY OF THE INVENTION  
       [0007]     The present invention is provided by providing a gamma corrected exterior viewing apparatus comprising: an image capturing device for mounting on a vehicle for capturing an electronic image exterior to the vehicle; a computer readable medium in communication with the image capturing device; an interior display device in communication with the computer readable medium for displaying an electronic image captured by the image capturing device; and, a set of computer readable instructions embodied in the computer readable medium for receiving electronic image information representing the electronic image, performing an image specific statistical analysis on the electronic image information, determining a gamma correction value according to the image specific statistical analysis so that the electronic image can be displayed on the display device according to the gamma correction value.  
         [0008]     The invention can include a set of computer readable instructions for displaying an electronic image according to the electronic image information and the gamma correction value on the display device, displaying a second electronic image according to a second electronic image information and the gamma correction value on the display device; receiving a plurality of electronic image information from the image capturing device, performing image specific statistical analysis on each electronic image information, and determining a gamma correction value according to the image specific statistical analysis for each electronic image information; determining a total gamma correction value according to each of the image specific statistical analysis results; performing an average, median and gradation luminance dispersion analysis for each electronic image information; calculating, average, median and gradation analysis results from the average, median and gradation luminance dispersion analysis, summing the gamma correction values, and calculating a total gamma correction value from the average, median and gradation analysis results and the summed gamma correction values; determining a brightness value of the electronic image information; performing luminance dispersion analysis; and performing an average, median, and gradation luminance dispersion analysis.  
         [0009]     The present invention can include a vehicle carrying the image capturing device, the interior display device, and the computer readable instructions. The invention can include a mirror assembly wherein the image capturing device is carried by the mirror assembly. 
     
    
     DESCRIPTION OF THE DRAWINGS  
       [0010]     The invention will be more readily understood by referring to the specification which includes the following figures:  
         [0011]      FIG. 1  is a diagram of the invention; and,  
         [0012]      FIG. 2  is a flow chart of the invention. 
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0013]     The detailed description that follows may be presented in terms of program procedures executed on a computer or network of computers. These procedural descriptions are representations used by those skilled in the art to most effectively convey the substance of their work to others skilled in the art. These procedures herein described are generally a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities such as electrical or magnetic signals capable of being stored, transferred, combined, compared, or otherwise manipulated readable medium that is designed to perform a specific task or tasks. Actual computer or executable code or computer readable code may not be contained within one file or one storage medium but may span several computers or storage mediums.  
         [0014]     The terms “host” and “server” may be hardware, software, or combination of hardware and software that provides the functionality described herein. This invention thereby allows multiple users, being geographically dispersed, to interact with data relating to physical characteristics of manufactured products using a system that ensures the precise and accurate conveyance of such information (data).  
         [0015]     The present invention is described below with reference to flowchart illustrations of methods, apparatus (“systems”) and computer program products according to the invention. It will be understood that each block of a flowchart illustration can be implemented by a set of computer readable instructions or code. These computer readable instructions may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions will execute on a computer or other data processing apparatus to create a means for implementing the functions specified in the flowchart block or blocks.  
         [0016]     These computer readable instructions may also be stored in a computer readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in a computer readable medium produce an article of manufacture including instruction means to implement the functions specified in the flowchart block or blocks. Computer program instructions may also be loaded onto a computer or other programmable apparatus to produce a computer executed process such that the instructions are executed on the computer or other programmable apparatus providing steps for implementing the functions specified in the flowchart block or blocks. Accordingly, elements of the flowchart support combinations of means for performing the special functions, combination of steps for performing the specified functions and program instruction means for performing the specified functions. It will be understood that each block of the flowchart illustrations can be implemented by special purpose hardware based computer systems that perform the specified functions, or steps, or combinations of special purpose hardware or computer instructions.  
         [0017]     The present invention is now described more fully herein with reference to the drawings in which the preferred embodiment of the invention is shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art.  
         [0018]     The invention concerns a system for displaying images using gamma correction values as well as an image processing system, which is required for the execution of the said method for the calculation of gamma correction values.  
         [0019]     Especially present in modern industrial image processing, although also appearing in many regions of motor vehicle activities, image processing systems are desired, which in different applications, provide high quality presentations. A gamma-correction offers possibilities for improvement in this regard. Image projection systems, such as RGB and LCD monitors have different brightness gradations and require correction measures frequently, in order to present an image in its best sense. In an ideal manner, a display apparatus would rate the decree of brightness for black as 0, and ascribe to a white reproduction a 1, with interposed values being evaluated as successively increasing or decreasing values of gray. Such a rated scale is representative, respectively, of a gamma rating in the same range. Actual cameras and image reproductive systems possess no such functional linearity. Most such instruments burden the non-linear characteristics of a display means, these being, for example, an LCD screen, or CCD chips of a camera, with the role of causing the disturbing non-linearity. So that, in a case where the capture of an image and the projection of the same is concerned, the least possible brightness data might be lost, or be over emphasized, apparatuses having non-linear input characteristics generally offer a possibility for gamma correction to acquire a linearization of displayed imaging. In the case of apparatuses, which possess image converter facilities for different colors, the normal practice possibly calls for a gamma correction for each single color channel.  
         [0020]     Up to this time and on this account, applied image pickup systems possess an adjustable, but nevertheless a standard, fixed gamma corrective means. This is used, as a rule, to bring the captured image into such brightness reproduction capabilities as are possesses by the display equipment in use. Classically, at the time of the television tube, the physically conditioned non-linearity of the image tube with a gamma of 2.2 has been corrected by a value of 0.45. Using a camera, which approaches the same area of brightness dynamics that the display apparatus has, can lead to acceptable results. When pictures are taken, in cases where the brightness dynamics are greater than have been assumed, such an arrangement can show considerable disadvantages. Considering image quality in cases troubled by high brightness dynamics and reflective interferences, the resulting displays can evidence lighting qualities which are over or under optimal levels. In such instances, the display equipment shows surfaces which tend toward either white or black. In many cases, this effect is notably emphasized by the commonly used statistical gamma correction, even though the image taking system, i.e. a camera, has acquired image data of considerable importance which will not be used in display.  
         [0021]     On this account, it is the purpose of the present invention to make available a method for the calculation of gamma correction values, which makes possible a better image reproduction from various display equipment. Further, it is another purpose of the present invention to propose an image processing system, which this said method requires for the application of the calculations for the dynamic gamma correction values.  
         [0022]     The principal concept of the invention is to be found therein, in that instead of a standard, unchangeable gamma correction, a dynamic gamma correction value for individual images is to be made on the basis of specific statistical data relating to these specific images. If this be done, then the user is in the position of being able to employ the brightness dynamics of the image taking system better than previously. Thus it may be assured, that the data retained in the image which is captured and is to be shown can be completely exploited to its fullest extent. The results of this statistical evaluation of individual images are put to use in order that by means of specific obtaining of gamma corrective values, relevant image features may be emphasized.  
         [0023]     In the case of conventional cameras, the lighting regulations are based, as a rule, on data, which is computed from the corresponding brightness histogram and the relevant values, such as lighting which may be average, medium or dispersed, in accord with the current conditions. These statistic image data, which are necessary for the regulation of lighting, or more exactly expressed, the effects of the dispersion of the brightness, i.e. the luminescence dispersion, of the individual images are used, in order that custom-fit, dynamic gamma correction values become available for the respectively individual images.  
         [0024]     In accord with an advantageous embodiment of the invention it is possible that an underexposed image can be brightened by a diminution of the gamma value and an overexposed image can be darkened by increasing the gamma value.  
         [0025]     In the course of definite applications, it can be advantageous to compute the statistical values on the basis of a majority of individual images. In the case of specific applications, it can be sufficient to employ the same adapted gamma correction value for a majority of successively following pictures. In this way the time and expense of calculation time is reduced.  
         [0026]     The following areas of application are available for the invention for: inspection and positioning systems, which require reliable object identification and classification based on reliable image information, supervisory (monitoring) tasks in the automotive industry, such as, for example, the inspection of the interior space of motor vehicles, and/or traveled road inspection, pre-crash sensors and cameras for maneuvering in reverse, and general supervision tasks in the area of the video based monitoring technology, for example, CCTV.  
         [0027]      FIG. 1  shows a vehicle  10  having a display device  12  in communications with a computer readable medium  14 . An image capture device  16  is in communications with the computer readable medium. The field of view of the image capture device  18  can capture am image of an object  20 . The environment  22  of the object can be light, dark, in full sun, in shadows, or other lighting conditions that affect the way that the image capture device captures the visual information of the object. Once the image capture device captures an image, the image information is transmitted to the computer readable medium and computer readable instructions embodied in the computer readable medium can calculate the gamma correction from an image specific statistical analysis of the image and can then display the object with gamma correction.  
         [0028]      FIG. 2  shows that at  24  electronic information is received by the computer readable instructions. An image specific analysis if preformed at  26  and a gamma correction value is determined at  28 . The image is displayed using the gamma correction value at  30 .  
         [0029]     A plurality of image information is received at  32  and image specific statistical analysis is preformed for each image information at  34 . A gamma correction value is calculated for each image information at  36 . The total gamma correction value from each image information is calculated at  38 . The image is displayed using each image gamma correction value at  40 . Further, each image can be displayed at  42  using the total gamma value. It should be noted that images can be displayed using each images specific gamma correction value or that a specific gamma value can be used to display each image. Further, the gamma value calculated for a first image can be used to display a second image.  
         [0030]     Image receiving device  16  has an image sensor and image-data for capturing a multiplicity of individual images. The image information can be a identified on a pixel-wide basis, namely, for instance, as P(x, y), having a column location x and a row location y. The image data can be designated as the so-called RGB-data (i.e., red, green and blue components). Alternately these data can be known as luminescence-color components (for example, YUV or YCrCb and similar formats). Since the differing forms of acquired data, due to intermixing, cannot be used as a computation base, in the following text, terms relating to luminescent color components will be used. The brightness, that is, the luminescence of each individual image point, typically, Y(x, y), is directly proportional to the duration period of the adjusted illumination, namely t int  and the chosen reinforcement would be designated v int , as long as there is no under or over exposure at the image point.  
         [0031]     The image data acquired by the image receiving apparatus, in regard to individual images, are, with consideration given to their luminescence-color components, evaluated in computer readable instructions for performing statistical analysis. The computer readable instructions calculate from the luminescence-color components of the image data of an image with n columns and m lines, therewith mn pixels, the corresponding probability characteristics such as average value, the median quantity and the dispersion effect of the individual components. These characteristics now computed in the computer readable instructions can also be manipulated for lighting control based upon the duration of the image receiving apparatus. As a second, simultaneous step, the calculated characteristics are also be used to calculate a gamma correction value. In this gamma correction value can further be based upon calculations are made on the basis of the thereto transferred characteristics to obtain a gamma correction value for the current individual images. The probability characteristic of the luminescence is:  
         [0032]     1. Average Value  
               Y   _     =       1     n   ·   m       ·     (       ∑   x   n     ⁢     (       ∑   y   m     ⁢     Y   ⁡     (     x   ,   y     )         )       )               (   1   )             
 
         [0033]     2. Median  
               Y   ~     =     {             Y   Sort     ⁡     (   k   )             fur   ¨           k   =             n   ·   m     +   1     2     ⁢   Λ   ⁢           ⁢   k     ∈   ℕ                       Y   Sort     ⁡     (   k   )       +       Y   Sort     ⁡     (     k   +   1     )         2           fur   ¨           k   =           n   ·   m     2     ⁢   Λ   ⁢           ⁢   k     ∈   ℕ                       (   2   )             
 
         [0034]     The value of Y is the total of all the luminance values. 
 
 Y ( x,y )ε YxεNyεNx≦ny≦m   (3) 
 
         [0035]     These values can be sorted through the function Y sort(l)  and so presented. 
 
 Y   Sort (1)=min( Y )  (4) 
 
 Y   Sort ( l )≦ Y   Sort ( l+ 1)           Y   Sort ( l )ε Yl≦n·mlεN   (5) 
 
         [0036]     3. Dispersion  
               s     Y   ~     2     =       I       n   ·   m     -   1       ·     (       ∑   x   n     ⁢     (       ∑   y   m     ⁢       (       Y   ⁡     (     x   ,   y     )       -     Y   _       )     2       )       )               (   6   )             
 
         [0037]     The computer readable instructions calculate the gamma value for the corresponding optimal value using the general formulation:  
             γ   =       ∑     r   ≥   0     ∞     ⁢     (       γ       Y   _     ,   r       +     γ       Y   ~     ,   r       +     γ       s     Y   _     2     ,   r       +     γ     int   ,   r         )               (   7   )             
 
         [0038]     The running variables r are complete numbers and the individual sums are as follows:  
               γ       Y   _     ,   r       =         rect   1     ⁡     (           α       Y   _     r       ·     Y   _       -       β       Y   _     r       ·       Y   _     Soll       -     δ   ⁢       Y   _     r           Δ   ⁢           ⁢       Y   _     r         )       ·     [         ∑     u   ≥   0     ∞     ⁢       a     u   ,   r       ·       (         α       Y   _     r       ·     Y   _       -       β       Y   _     r       ·       Y   _     Soll         )     u         +         log   q     ⁡     (       ∑     u   ≥   0     ∞     ⁢       b     u   ,   r       ·       Y   _     u         )       ·       (       log   q     ⁡     (       ∑     u   ≥   0     ∞     ⁢       c     u   ,   r       ·       Y   _     Soll   u         )       )       -   1         +     Γ       Y   _     r         ]               (   8   )                 γ       Y   ~     ,   r       =         rect   1     ⁡     (           α       Y   ~     r       ·     Y   ~       -       β       Y   ~     r       ·       Y   ~     Soll       -     δ   ⁢       Y   ~     r           Δ   ⁢           ⁢       Y   ~     r         )       ·     [         ∑     u   ≥   0     ∞     ⁢       d     u   ,   r       ·       (         α       Y   ~     r       ·     Y   ~       -       β       Y   ~     r       ·       Y   ~     Soll         )     u         +         log   q     ⁡     (       ∑     u   ≥   0     ∞     ⁢       e     u   ,   r       ·       Y   ~     u         )       ·       (       log   q     ⁡     (       ∑     u   ≥   0     ∞     ⁢       f     u   ,   r       ·       Y   ~     Soll   u         )       )       -   1         +     Γ       Y   ~     r         ]               (   9   )                 γ       s     Y   _     2     ,   r       =         rect   1     ⁡     (           α     s       Y   _     ⁢   r     2       ·     s     Y   _     2       -       β     s       Y   _     ⁢   r     2       ·     s       Y   _     ⁢   Soll     2       -     δ   ⁢           ⁢     s       Y   _     ⁢   r     2           Δ   ⁢           ⁢     s       Y   _     ⁢   r     2         )       ·           ⁢     [           ⁢                   ∑       u   ≥   0     ∞     ⁢           ⁢       g     u   ,   r       ·       (         α     s       Y   _     ⁢   r     2       ·     s     Y   _     2       -       β     s       Y   _     ⁢   r     2       ·     s       Y   _     Soll     2         )     u         +     Γ     s       Y   _     r     2         ]               (   10   )                 γ     int   ,   r       =           rect   1     ⁡     (         t   int     -     δ     t     int   ,   r             Δ   ⁢           ⁢     t   intr         )       ·     [         ∑     u   ≥   0     ∞     ⁢       h     u   ,   r       ·     t   int   ur         +     Γ     t     int   ⁢           ⁢   r           ]       +         rect   1     ⁡     (         v   int     -     δ   ⁢           ⁢     v     int   ⁢           ⁢   r             Δ   ⁢           ⁢     v     int   ⁢           ⁢   r           )       ·     [         ∑     u   ≥   0     ∞     ⁢       i     u   ,   r       ·     v   int   u         +     Γ     v     int   ⁢           ⁢   r           ]                 (   11   )             
 
         [0039]     The rect 1  function is defined as:  
                 rect   1     ⁡     (   z   )       =     {         1         fur   ¨           0   ≤   z   ≤   1             0         fur   ¨             z   &gt;   1     ⁢     ⩔   z     ⁢     &lt;   0                       (   12   )             
 
         [0040]     The coefficients and variables used in these functions have the following meaning:  
         [0041]     Gamma adaption basis summation:  
         [0042]     The gamma adaption basis summation λ  y ,r encompasses the functional dependency of the gamma value, relative to the true average value  Y  and the set average value of the luminescence  Y soll. Correspondingly, the gamma adaption basis summation γ  y ,r encompasses the functional dependency of the gamma value, relative to the true median value {tilde over (Y)} and the set value of the luminance {tilde over (Y)}soll. The functional dependency of the dispersion is obtained through the gamma adaption basis summation γs r   2 , relative to the true dispersion s y   2  and the set dispersion of the luminance s Y   2 soll. Internal sensor parameters, such as the duration of illumination t int  and the internal reinforcement v int  are arrived at through the gamma adaption basis summation γint, y.  
         [0043]     The designated value for the average, median and dispersion characteristics is to be calculated from the current application.  
         [0044]     Weight coefficients:  
         [0000]     The weight of a respective real value is found in accord with the coefficients αYr, α{tilde over (Y)}r and αs Yr   2 . The set value appropriate thereto shows coefficients for weight as βYr, γ{tilde over (Y)}r, and βs Yr   2 .  
         [0045]     Offsets:  
         [0046]     Each gamma adaption basis summand governs an adjustable offset within the respective series of potentials. These are the summands: Γ  Y r, Γ{tilde over (Y)}r, ΓS Yr   2 , Γt int r and Γv int r,  
         [0047]     Potential series development coefficients:  
         [0048]     The coefficients a uY , b uY , c uY , d uY , f uY , g uY , h uY  and i uY  are the development coefficients of the current potential series, which are required in order to be able to determine application-specific functional dependency,  
         [0049]     Window of evaluation:  
         [0050]     Each of the employed potential series can be brought into and out of the excess reflective situation by means of the evaluation function. The current widths of the evaluation window determine the parameters: ΔΓ  Y r, ΔΓ{tilde over (Y)}r, ΔS Yr   2 , Δt int r and Δv int r. The planar displacement of the window of evaluation is achieved from δ  Y r, δ{tilde over (Y)}r, δs   r     2 , δt int r and δv int r.  
         [0051]     The presentation of the gamma adaption summation, respectively in connection with various r-values and average values as well as median and dispersion characteristics results from a sectional definition of the functional dependency of the gamma adaption summation of the respective average value {tilde over (Y)} and the respective dispersion value of s Y   2  in respect to an individual single image. The value for “r” releases the information, as to how many sections are possessed by the function “gamma adaption summations in connection with the average value, the median value and the dispersal value”. By means of the rect 1  function, respectively only that section which is “activated” becomes valid for the values of the average, median and dispersal characteristics. The summation in accord with the equation 7 contains also, independently of r, at the most, a gamma adaption summation for the dependency of the values of the average, median and dispersal characteristics as well as for the intensity of the illumination.  
         [0052]     The running variable “u” is the running variable of the potential series development in the equations (8) to (11).  
         [0053]     That is to say, in accord with the equations (1) to (6) the following values of the respective single images can be calculated: average, median and dispersion of the brightness or luminance. By the equations (7) to (12) the gamma correction value, namely y, can be obtained for the individual images. These gamma correction values are transferred to a gamma correction unit  10 , to which also the image data acquired from the image pickup apparatus  2  has been transmitted. The gamma correction unit  10  carries through, in a familiar fashion, the gamma correction of the submitted images. The gamma correction measures used here are based on the known gamma correction values found in the literature. As a rule, the gamma correction with the luminance component, namely Y(i,j), is present. For this purpose, on the side of the acquiring of the image, that is, the assembly of the image data, the following standardized functional equation is made use of:  
                 Y   γ     ⁡     (     i   ,   j     )       =       Y   ⁡     (     i   ,   j     )         1   γ               (   13   )             
 
         [0054]     The luminescence component Yλ(i,j) is required, in this respect, to contain the following condition, namely 0≦Y γ (i,j)≦1. If necessary, the luminescence components must be standardized before the gamma correction and again removed after the gamma correction. In regard to the details of the above gamma function (13), reference should be made to the text book  Fernseh Technology , vol. 2 1991, Hütig Buchverlag Heidelberg, pages 41ff.  
         [0055]     The calculation of the equation (13) can be done directly, by means of a potential series development, or by a table or yet by point to point interpolation between the said known points.  
         [0056]     Counter to the state of the technology, in the gamma correction unit  10 , what is used is not a firmed up gamma correction value, but rather a dynamic gamma correction value which is appropriate for the individual images from the gamma adaption unit  8 . The gamma correction image data are then conducted through an output shifting  12  of a display unit  14 , for example by an LCD monitor screen. In this way, it becomes possible to give consideration to the gamma correction for the respective unit in an intelligent manner.