Vehicle camera system

A camera system for a vehicle. The system includes a camera configured to capture an image of an area about the vehicle, and a control module. The control module compares the captured image to a plurality of previously captured training images. The control module also determines which one of the plurality of training images is most similar to the captured image. Furthermore, the control module modifies settings of the camera to match camera settings used to capture the one or more of the plurality of training images that is most similar to the captured image.

FIELD

The present disclosure relates to a vehicle camera system.

BACKGROUND

More and more vehicles are being outfitted with cameras to detect lane markers, obstacles, signage, infrastructure, other vehicles, pedestrians, etc. The cameras can be used, for example, to enhance safe vehicle operation and/or to guide the vehicle during autonomous driving. While current cameras are suitable for their intended use, they are subject to improvement. Although there are various image processing technologies applied in imaging, no single technique or combination of techniques addresses the robustness issues experienced with automotive applications.

The present teachings provide for camera systems and methods that advantageously enhance the object detection capabilities of vehicle cameras, for example. One skilled in the art will appreciate that the present teachings provide numerous additional advantages and unexpected results in addition to those set forth herein.

SUMMARY

The present teachings include a camera system for a vehicle. The system includes a camera configured to capture an image of an area about the vehicle, and a control module. The control module compares the captured image to a plurality of previously captured training images. The control module also determines which one of the plurality of training images is most similar to the captured image. The control module then modifies settings of the camera to match camera settings used to capture the one or more of the plurality of training images that is most similar to the captured image.

DETAILED DESCRIPTION

With initial reference toFIG. 1, a camera system in accordance with the present teachings is illustrated at reference numeral10. The camera system10generally includes a camera20and a control module30. Although the camera system10is illustrated as included with a passenger vehicle40, the system10can be included with any suitable type of vehicle. For example, the camera system10can be included with any suitable recreational vehicle, mass transit vehicle, construction vehicle, military vehicle, motorcycle, construction equipment, mining equipment, watercraft, aircraft, etc. Further, the camera system10can be used with any suitable non-vehicular applications to enhance the ability of the camera20to detect objects of interest.

The camera20can be any suitable camera or other sensor capable of detecting objects of interest. For example, the camera20can be any suitable visual light, extended spectrum, multi-spectral imaging, or fused imaging system camera and/or sensor. The camera20can be mounted at any suitable position about the vehicle40, such as on a roof of the vehicle40, at a front of the vehicle40, on a windshield of the vehicle40, etc. The camera system10can include any suitable number of cameras20, although the exemplary system described herein includes a single camera20.

As explained further herein, the control module30receives an image taken by the camera20including an object of interest, and adjusts the settings of the camera20, such as gain, exposure, and shutter speed to the settings that are optimal based on the current environmental conditions for detecting the particular object of interest. In this application, including the definitions below, the term “module” may be replaced with the term “circuit.” The term “module” may refer to, be part of, or include processor hardware (shared, dedicated, or group) that executes code and memory hardware (shared, dedicated, or group) that stores code executed by the processor hardware. The code is configured to provide the features of the control module30described herein.

The present teachings advantageously adjust the settings of the camera20, such as gain, exposure, and shutter speed to the settings that are optimal based on the current environmental conditions for detecting particular objects. As described herein, the camera system10can be configured to adjust the settings of the camera20to optimal settings for identifying vehicle lane lines painted or printed on a road. However, the system10can be configured to set the settings of the camera20for optimal detection of any other suitable object as well, such as road signage, other vehicles, pedestrians, infrastructure, etc.

Any suitable portion of an image captured by the camera20can be used to identify the optimal camera settings based on current environmental conditions. For example and as illustrated inFIG. 2, the control module30can be configured to adjust the camera settings based on environmental conditions above a horizon line. To detect the horizon line, the control module30first identifies in an image captured by the camera20a vanishing point V where lines L1and L2, which are drawn along left and right lane markers of a lane that the vehicle40is traveling in, appear to meet and/or cross in the distance. Line H is arranged by the control module30to extend through the vanishing point V in a direction perpendicular to a direction that the vehicle40is traveling in, and generally parallel to a surface of the road. Image data from the area above line H has been determined to be the most relevant to setting the camera20, and thus it is data from above line H of each image captured by the camera20, and the training images described herein, which is used to set the camera20.

With continued reference toFIGS. 1 and 2, and additional reference toFIG. 3, a method according to the present teachings for creating a training model for optimally setting the camera20is illustrated at reference numeral110and will now be described in detail. The method110can be performed by the control module30, or with any other suitable control module or system. With initial reference to block112ofFIG. 3, multiple training images are obtained for training the camera20. The training images can be obtained in any suitable manner, such as from a developer, manufacturer, and/or provider of the camera system10. Any suitable number of training images can be obtained and used. For example, 5,000 training images of different environmental conditions for each one of a plurality of different scenes typically encountered by the camera20can be obtained. For example, 5,000 training images for each of the following typical scenes can be obtained: normal scene; rainy scene; snowy scene; sunny scene; cloudy scene; tunnel-enter scene; and tunnel-exit scene.

At block114, the camera settings for each one of the training images obtained is identified. For example, the gain, exposure, and shutter speed settings for each training image obtained is identified. At block116, each training image is classified according to the scene captured therein. Any suitable classifications can be used. For example, the training images can be classified into one of the following scenes: normal, rainy, snowy, sunny, cloudy, tunnel-enter, and tunnel-exit.

At block118, each one of the training images is prepared for the extraction of features therefrom that can be used to distinguish the different training images from one another. The different training images can be distinguished based on any relevant features, such as, but not limited to, one or more of the following:

TABLE AMean RGBThe mean value of red, green, blueplaneMean RedThe mean value of red planeMean GreenThe mean value of green planeMean BlueThe mean value of blue planeStandard Deviation RGBThe standard deviation value ofred, green, blue planeStandard Deviation RedThe standard deviation value ofred planeStandard Deviation GreenThe standard deviation value ofgreen planeStandard Deviation BlueThe standard deviation value ofblue planeMean HSVThe RGB image converted to HSV,the mean value of the hue,saturation, value planeMean HueThe RGB image converted to HSV,the mean value of the hue planeMean SaturationThe RGB image converted to HSV,the mean value of the saturationplaneMean ValueThe RGB image converted to HSV,the mean value of the value planeStandard Deviation HSVThe RGB image converted to HSV,the standard deviation value of thehue, saturation, value planeStandard Deviation HueThe RGB image converted to HSV,the standard deviation value of thehue planeStandard Deviation SaturationThe RGB image converted to HSV,the standard deviation value of thesaturation planeStandard Deviation ValueThe RGB image converted to HSV,the standard deviation value of thevalue planeMean Gaussian Blurs (10)The input converted to grayscalethen a Gaussian blur run (tendifferent times with different valuesof sigma) then the mean valuetakenStandard Deviation Gaussian BlursThe input converted to grayscale(10)then a Gaussian blur run (tendifferent times with different valuesof sigma) then the standarddeviation value takenMean Difference of Gaussian (10)The input converted to grayscalethen two Gaussian blurs run,followed by an image subtraction(difference of Gaussian) then themean value takenStandard Deviation Gaussian BlursThe input converted to grayscale(10)then two Gaussian blurs run,followed by an image subtraction(difference of Gaussian) then thestandard deviation value taken

Each one of the training images can be prepared for extraction of features therefrom at block118in any suitable manner. For example and with reference to block120, each color (red, green, blue) training image can be transformed to an HSV (hue, saturation, and value) image, from which various features listed above in Table A can be extracted. At block122, color (red, green, blue) training images are converted to grayscale images, and at block124a Gaussian blur of each grayscale image is performed. Multiple Gaussian blurs of each grayscale image can be performed, and the difference of the multiple Gaussian blurs is taken at block126.

With reference to block130, after each one of the training images has been prepared, such as set forth at blocks120,122,124, and126, features relevant to distinguishing each training image from one another are extracted at bock130. The features extracted at block130can be those set forth above at Table A, or any other suitable features. With reference to block132, the extracted features are used to build a model, data set, or file of images. The model can be trained in any suitable manner, such as with any suitable algorithm. One example of a suitable algorithm that may be used is a random forest algorithm, but any other suitable algorithm can be used as well.

With additional reference toFIG. 4, a method210according to the present teachings for setting the camera20will now be described. The method210can be performed by the control module30of the system10, or in any other suitable manner, such as with any other suitable control module. With initial reference to block212, the trained model of training image data obtained by performing the method110, or in any other suitable manner, is accessed by the control module30. The control module30can access the trained model of training image data in any suitable manner, such as by accessing data previously loaded to the control module30, or accessing the trained model of training image data from a remote source, such as by way of any suitable remote connection (e.g., internet connection).

At block214, the control module30retrieves a live image captured by the camera20, such as of an area about the vehicle40. At block216, any suitable image features are extracted from the live image captured by the camera20, such as the features listed above in Table A. To extract the features from the live image, the live image may be prepared in any suitable manner, such as set forth inFIG. 3at blocks120,122,124, and126with respect to the training images. At block218, the live image is classified according to the scene captured therein. For example, the live image can be classified into any one of the following classifications: normal, rainy, snowy, sunny, cloudy, tunnel-enter, tunnel-exit.

At block220, the control module30compares the extracted features of the classified live image with the features extracted from each training image at block130ofFIG. 3. At block222, the control module30identifies the training image with features most similar to the live image captured by the camera20. At block224, the control module30configures the settings of the camera20to correspond with the camera settings used to capture the training image identified as being most similar to the live image captured by the camera20. The control module30can configure any suitable settings of the camera20, such as the gain, exposure, shutter speed, etc. of the camera20.

The present teachings thus advantageously provide for methods and systems for running a computer vision algorithm automatically and dynamically to change camera settings in order to match the camera settings used to capture a reference image, the reference image previously having been found to be of a quality that facilitates identification of road lane lines, or any other suitable object of interest.