Abstract:
A monitoring unit for the exterior in the direction of travel of a motor vehicle includes at least one camera system with an image recording sensor. The sensor has color encoding in some regions and, otherwise, monochrome encoding. An assist system is also provided with such a monitoring unit, in particular a traffic sign and/or lane recognition assist system.

Description:
BACKGROUND OF THE INVENTION 
     Field of the Invention 
     The present invention relates to a monitoring unit for the outside in a motor vehicle&#39;s direction of travel, which unit includes at least one camera system having an image-recording sensor. 
     The present invention further relates to an assistance system for motor vehicles, in particular for traffic-sign and/or traffic-lane registering, having at least one monitoring unit registering the outside in the motor vehicle&#39;s direction of travel, with said monitoring unit including at least one camera system having an image-recording sensor. 
     Background Information 
     Intelligent Advanced Driver Assistance Systems (ADAS) will play an increasingly important role in up-to-date motor vehicles of the future. Said vehicles will contain, for example, monitoring units such as camera systems having, for instance, digital CMOS (Complementary Metal-Oxide Semiconductor) or CCD (Charge-Coupled Device) image sensors as aids that register the outside, for example in the motor vehicle&#39;s direction of travel. 
     Pure black/white (B/W)—that is to say monochrome—image recording is for most tasks more favorable than color image recording in association with image-processing systems of said type employed in the automobile sector for registering travel environments. However, applications do exist where color information obtained from, for example, the three RGB primary colors red (R), green (G), and blue (B), and/or other colors such as, for instance, yellow (Y) etc. can be important for attaining a higher confidence level for the output vector generated by an image-processing means. 
     An instance thereof is traffic-sign recognition, with its being possible to recognize by way of the, where applicable, individual color information (R and/or G and/or B, and/or other colors such as, for instance, Y etc.) whether the sign concerned is a prohibition sign or a sign giving orders, or one that purely provides information. 
     Another function is recognizing colored lane markings associated with, for example, roadworks. Color information is helpful here, too, and necessary to be able, for example, to distinguish between the normal white markings that are no longer valid and the additional yellow lane markings. 
     Conversely, purely monochrome (B/W) image recording will suffice for the purpose of recognizing objects such as obstacles, other vehicles, and cyclists or other persons because color information (R, G, B, Y etc.) will as a rule not provide better recognition quality. 
     SUMMARY OF THE INVENTION 
     An object of an embodiment of the invention is to provide an improved monitoring unit for the outside in a motor vehicle&#39;s direction of travel, which unit includes at least one camera system having an image-recording sensor. An object of another embodiment of the invention is to provide an improved assistance system for motor vehicles, in particular for traffic-sign and/or traffic-lane registering. The above objects can be achieved by one or more embodiments of the invention disclosed herein. 
     The invention builds on generic monitoring units for the outside in a motor vehicle&#39;s direction of travel in that the monitoring unit includes at least one camera system having an image-recording sensor having color coding (R, G, B, Y, . . . ) in partial areas but otherwise monochrome coding (B/W). The present invention thus proceeds from an image-recording sensor that is substantially embodied as monochrome (B/W) or, as the case may be, coded and inventively has color coding (R and/or G and/or B and/or other colors such as, for instance, Y etc.) in partial areas. 
     For the purpose of recognizing or, as the case may be, assigning specific colors of traffic signs it is proposed providing a color coding (R and/or G and/or B and/or other colors such as, for instance, Y etc.) of vertical stripes and/or areas on the right-hand and/or left-hand image edge. Because traffic signs move from the center of the image outward from the perspective of a camera mounted in the front area of a motor vehicle traveling in a straight line, the color of the sign and the basic information associated therewith (prohibition, orders, other information) can be advantageously determined when the sign is located within the color-coded stripes or, as the case may be, areas. 
     For the purpose of recognizing a lane marking&#39;s for instance yellow or white color it is proposed providing a color coding (R and/or G and/or B and/or other colors such as, for instance, Y etc.) of horizontal stripes and/or areas on the sensor&#39;s bottom image edge, preferably in the area where the camera has a view onto the road directly over the hood, particularly in the case of a customary automobile. The camera can, of course, be arranged analogously when the inventive monitoring unit is employed in a truck or van etc. As said area is not absolutely essential for image evaluating in terms of object detecting or lane registering it can advantageously be used for color-recognition purposes. Because traffic lanes can be seen from the center/top of the central image area down to the bottom left-hand or, as the case may be, right-hand image area when the camera is mounted on the front of a vehicle traveling in a straight line, then in addition to the traffic lane&#39;s position the color can advantageously be determined and made available to an image-processing system. 
     In order to obtain the desired color information (R and/or G and/or B and/or other colors such as, for instance, Y etc.) it is proposed attaching a tiny color filter, where applicable specifically accommodated to the application, in front of each individual cell (pixel) of the stripes or, as the case may be, areas concerned. 
     The color-coded stripes and/or areas are in a first embodiment embodied as, for example, a single color (R; Y; etc. . . . ) The vertical stripes and/or areas expediently have, for example, a red (R) color coding and the horizontal stripes and/or areas preferably a yellow (Y) color coding. 
     The color-coded horizontal and vertical stripes and/or areas can alternatively or additionally be embodied as a combination of two colors (R, G). In particular vertical stripes and/or areas having red (R) and green (G) color coding have proved useful for increasing the contrast of signs that give orders and are placed in front of trees. 
     For obtaining the desired color information (R, G, B) it is proposed in a further embodiment to arrange a tiny color filter in one of the three RGB primary colors red (R), green (G), and blue (B) in front of each individual cell (pixel) of the stripes or, as the case may be, areas concerned, with said filters being arranged preferably in what is termed the “Bayer pattern”. 
     The invention further comprises an assistance system having a monitoring unit of the aforementioned type. The monitoring unit&#39;s advantages will in this way also come to bear within the scope of an overall system, in particular for traffic-sign and/or traffic-lane registering. The ratio of monochrome coding (B/W) to partial color coding is therein preferably 80:20%. Depending on the focus of the specific application the partially color-coded areas can also occupy 25% or up to 40% of the sensor surface. 
     The main advantage of an inventive monitoring unit for the outside in a motor vehicle&#39;s direction of travel or, as the case may be, of an assistance system for motor vehicles including said type of monitoring unit having a partially color-coded (R and/or G and/or B and/or other colors such as, for instance, Y etc.) camera is that all relevant data for registering travel environments can for the first time be obtained using just one camera. Owing to the camera&#39;s substantially monochrome (B/W) image recorder there will be no constraints on sensitivity so that reliable evaluating will be ensured even in poor light conditions; the color coding (R and/or G and/or B and/or other colors such as, for instance, Y etc.) in the sensor&#39;s edge area will not compromise applications for which the purely monochrome (B/W) image is more favorable. 
     By contrast, color coding (R and/or G and/or B and/or other colors such as, for instance, Y etc.) having vertical stripes and/or areas on the right-hand and/or left-hand edge of the sensor&#39;s image field will provide reliable information about the color (R, G, B, Y, . . . ) of traffic signs; the color coding (R, G, B, Y, . . . ) in the bottom image area will provide reliable information about the color of lane markings. 
     A single camera can thus be used for all relevant applications, and that will advantageously save costs and mounting space. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention is described below with reference to the accompanying drawing and with the aid of a preferred exemplary embodiment. 
         FIG. 1  is the exemplary image, shown in monochrome (B/W), of a scene outside a motor vehicle; 
         FIG. 2  is a schematic sketch of a possible color-filter distribution (R, G, B) of the image recorder of a monitoring unit&#39;s camera; 
         FIG. 3  shows an enlarged section of the color-filter distribution according to  FIG. 2 ; 
         FIG. 4  is the exemplary image according to  FIG. 1  showing a scene outside that has been partially color-coded according to  FIG. 2 ; and 
         FIGS. 5 and 6  are further instances of a color-filter distribution of a monitoring unit&#39;s image recorder. 
     
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
       FIG. 1  is the exemplary image, shown in monochrome, of a scene outside a motor vehicle. What is disadvantageous about black/white (B/W) image recording of said type is the lack of color information that could be obtained from, for example, the three primary colors red (R) and/or green (G) and/or blue (B) and/or other colors such as, for instance, Y etc. 
     Color information (R, G, B, Y, . . . ) of said type can be important for attaining a higher confidence level for the output vector generated by an image-processing means, in particular for recognizing traffic signs, with its being possible to recognize by way of the color information (R and/or G and/or B and/or other colors such as, for instance, Y etc.) whether the sign concerned is a prohibition sign or a sign giving orders—as in the case of the speed-limit sign on the right of the image section shown  FIG. 1 —or one that purely provides information (not shown). 
     A further function is recognizing colored lane markings associated with roadworks. It is helpful here and necessary to distinguish between the normal white markings that are no longer valid and the additional yellow lanes. 
     Conceivable solutions featuring an exclusively color-coded (R, G, B, Y, . . . ) image recorder are not only more compute-bound and hence more expensive; they also have the disadvantage that monochrome (B/W) images are more favorable for a number of outside applications, in particular for the purpose of detecting objects such as obstacles, other vehicles, and cyclists or other persons and the like, or night-time applications. 
     To resolve this conflict of requirements the present invention proposes the use in a generic monitoring unit of a, for example, specially embodied CCD sensor  10  having color coding (R, G, B, Y, . . . ) in partial areas  11 ,  12 ,  13  but otherwise monochrome coding (B/W). The present invention in other words proceeds from a sensor  10  that is embodied as substantially monochrome (B/W) or, as the case may be, coded and has color coding (R, G, B, Y, . . . ) in partial areas. That could be achieved by means of, for instance, a color coding (R, G, B, Y, . . . ) of vertical stripes (not shown) or areas on the right-hand  11  and left-hand  12  image edge, and would be helpful, for example, for assigning traffic signs the correct color (R, G, B, Y, . . . ). Because the traffic signs move from the center of the image outward from the perspective of a camera mounted in the front area of a motor vehicle traveling in a straight line, the color (R, G, B, Y, . . . ) of the sign can be determined when the sign is located within the color-coded (R, G, B, Y, . . . ) stripes or, as the case may be, areas  11 ,  12 . 
     The expedient approach for recognizing the color of the traffic lane (yellow or white, for example) is also to provide color coding, in particular yellow (Y) coding, in the bottom area  13  of the sensor  10 , preferably in the area where the camera has a view onto the road directly over the hood in the case of a customary automobile. As said area is not absolutely essential for image evaluating in terms of object detecting or lane registering it can be used for color-recognition (Y) purposes. Because traffic lanes can be seen from the center/top of the central image area down to the bottom left-hand and right-hand image area when the camera is mounted on the front of a vehicle traveling in a straight line, then in addition to the traffic lane&#39;s position the color can be determined and made available to an image-processing system. This feature is not, of course, restricted to motor vehicles having a hood but can be realized analogously when the inventive monitoring unit is employed in particular in a truck or van etc. 
     Especially in the case also of night-vision applications the best possible sensitivity is necessary across the entire, which is to say unfiltered, wavelength spectrum, including the near infrared. Since, though, a much smaller angular range generally has to be detected here, it suffices to provide a smaller central area with no color coding. 
       FIG. 2  is a schematic sketch of a possible color-filter distribution (R, G, B) of the image recorder  10  of a monitoring unit&#39;s camera. The light-sensitive cells of a non-color-coded (N) CCD sensor  10  in the monitoring unit&#39;s digital camera normally only register brightness values. In order to obtain color information (R, G, B, Y, . . . ) a tiny color filter is to be attached in front of each individual cell for example in one of the three RGB primary colors red (R), green (G), and/or blue (B), and/or another color such as, for instance, yellow (Y) etc. Filters constructed from primary colors are usually attached in the arrangement R-G-R-G and, in the next line, G-B-G-B, and in this arrangement produce what is termed the “Bayer pattern”. 
       FIG. 3  shows an enlarged section of the color-filter distribution (R, G, B) according to  FIG. 2  arranged in the Bayer pattern, which is a known repeating pattern of 2×2 pixel groups that each include two green pixels diagonally opposite one another as well as a red pixel and a blue pixel diagonally opposite one another in a square dimensioned 2 pixels by 2 pixels as shown in  FIGS. 2 and 3 . It can clearly be seen how each CCD element only supplies the information for a single color component so that the adjacent pixels accordingly have to be used for calculating the actual color. This process is referred to as “color interpolating”. 
     Color information useful in the context of road traffic can be obtained within the Bayer-pattern coded stripes or, as the case may be, areas  11 ,  12 ,  13  of the sensor  10  by means of color interpolating of said type and made available to an image-processing system. 
     The corresponding exemplary image according to  FIG. 1  showing a scene outside that has been partially color-coded according to  FIG. 2  is shown in  FIG. 4 . 
     The color information does not have to consist of the three primary colors: It is also possible only to use single color filters such as, for instance, red (R) for signs giving orders or yellow (Y) for lane markings associated with, for example, roadworks. Combinations of red (R) and green (G) color filters or others specifically accommodated to the application have also proved useful for increasing the contrast of signs that give orders and are placed in front of trees. Color filters of said type can advantageously also be arranged spaced apart, being located, for example, on every other pixel in a row and/or column; that means in any combination with non-coded (N) pixels or pixels coded in another color or, as the case may be, where applicable specifically accommodated color filters. 
       FIGS. 5 and 6  show arrangements of pixels in various different repeating patterns of 2×2 pixel groups and further instances of color coding that offer the above advantages and can be used instead of the known “Bayer pattern”. 
       FIG. 5  shows clearly how 2×2 pixel groups that each include one or two red (R) pixels and three or two monochrome (N) pixels can be configured and arranged so that only every other pixel in a row and/or every other column has been provided with red color filters (R) for recognizing, for instance, signs giving orders. 
       FIG. 6  shows the advantageous embodiment of a combination of two colors. Instead of R-G-R-G and in the next row G-B-G-B as in the case of the “Bayer pattern”, color filters are used in the pattern R-N-R-N and in the next row N-G-N-G, with N signifying no color coding. In this case, each 2×2 pixel group includes two N pixels diagonally opposite one another, as well as an R pixel and a G pixel diagonally opposite one another as shown in  FIG. 6 . 
     The main advantage of a partially color-coded camera is that all relevant data for registering travel environments can be obtained using just one camera. Owing to the camera&#39;s substantially monochrome (B/W) image recorder there will be no constraints on sensitivity so that reliable evaluating will be ensured even in poor light conditions; the color coding (R, G, B, Y, . . . ) in defined edge areas  11 ,  12 ,  13  of the sensor  10  will not compromise applications for which the purely monochrome (B/W) image is more favorable. By contrast, color coding (R, G, B, Y, . . . ) having vertical stripes or areas  12  on the left-hand edge and vertical stripes or areas  11  on the right-hand edge of the sensor&#39;s image field will provide reliable information about the color (R, G, B, Y, . . . ) of traffic signs; the color coding (R, G, B, Y, . . . ) in the bottom image area  13  will provide reliable information about the color of traffic lanes. 
     A single camera can thus be used for all applications, and that will advantageously save costs and mounting space. 
     The present invention is thus especially suitable for implementation in an assistance system for motor vehicles, in particular for traffic-sign and/or traffic-lane registering. It will advantageously increase road-traffic safety not only in combination with existing assistance systems for motor vehicles such as blind-spot detection, LDW (Lane Departure Warning) lane monitoring, night vision etc.