Patent Description:
It is known to provide a radar (radio detection and ranging) system (such as a <NUM> radar or other suitable frequency radar) on a vehicle for sensing the area forward of a vehicle, such as for an adaptive cruise control (ACC) system or an ACC stop and go system or the like. It is also known to provide a lidar (laser imaging detection and ranging) system for sensing the area forward of a vehicle for similar applications. Typically, the radar system is preferred for such vehicle applications because of its ability to detect better than the lidar system in fog or other inclement weather conditions.

Typically, such radar sensor devices are often located at the front grille of the vehicle and thus may be Intrusive to the underhood packaging of the vehicle and the exterior styling of the vehicle. Although it is known to provide a lidar sensing device or system at the windshield for scanning/detecting through the windshield, radar systems are typically not suitable for such applications, since they typically are not suitable for viewing through glass, such as through the vehicle windshield (because the glass windshield may substantially attenuate the radar performance or ability to detect objects forward of the vehicle). It is also known to augment such a radar or lidar system with a forward facing camera or image sensor.

<CIT> discloses a system which includes both radar and image capture devices, in which a correction factor is stored and applied to both the radar and image data to ensure that they are spatially correlated with one another.

There is provided a forward-facing sensing system for a vehicle according to claim <NUM> and the dependent claims.

There is disclosed herein a forward facing sensing system for detecting objects forward of the vehicle (such as for use with or in conjunction with an adaptive cruise control system or other object detection system or the like), with a radar sensor device being located behind, and transmitting through [typically, transmitting at at least about <NUM> frequency (such as <NUM>) and more preferably at least about <NUM> frequency (such as <NUM> or <NUM> or <NUM> or thereabouts)], a radar transmitting portion established at the upper windshield area of the vehicle. The radar sensor device is positioned at a recess or pocket or opening formed at and along the upper edge of the windshield so as to have a forward transmitting and receiving direction for radar electromagnetic waves that is not through the glass panels of the windshield. The vehicle or sensing system preferably includes a sealing or cover element, such as a plastic cover element at the sensing device to seal/environmentally protect the radar sensor device within the cabin of the vehicle while allowing for transmission of and receipt of radar frequency electromagnetic radiation waves to and from the exterior of the vehicle.

According to an aspect of the present disclosure, a forward facing sensing system or radar sensing system for a vehicle includes a radar sensor device disposed at a pocket or recess or opening established at an upper edge of the vehicle windshield and having a forward transmitting and receiving direction that is not through the windshield. A cover panel is disposed at the radar sensor device and is substantially sealed at the vehicle windshield at or near the pocket at the upper edge of the vehicle windshield. The cover panel comprises a material that is substantially transmissive to radar frequency electromagnetic radiation waves. The radar sensor device transmits and receives radar frequency electromagnetic radiation waves that transmit through the cover panel. The system includes a control that is responsive to an output of the radar sensor device.

According to another aspect of the present disclosure, a forward facing sensing system for a vehicle includes a radar sensor device operable to detect an object ahead of the vehicle, a forward facing image sensor having a forward field of view, and a control responsive to an output of the radar sensor device and responsive to an output of the forward facing image sensor. The control is operable to control sensing by the radar sensor device and the control is operable to control a focused or enhanced interrogation of a detected object (or area at which a detected object is detected) in response to a detection of an object forward of the vehicle by the radar sensor device. The control may be operable to at least one of (a) control enhanced interrogation of a detected object by the radar sensor device in response to the forward facing image sensor detecting an object (such as by enhancing the interrogation via a beam aiming or beam selection technique, such as by digital beam forming in a phased array antenna system or such as by digital beam steering or the like), and (b) control enhanced interrogation of a detected object by the forward facing image sensor in response to the radar sensor device detecting an object (such as by enhancing the interrogation via enhanced or intensified algorithmic processing of a portion of the image plane of the image sensor that is spatially related to the location of the detected object in the forward field of view of the image sensor). The control thus may be responsive to the forward facing image sensor to guide or control the focused interrogation of the detected object by the radar sensor device, or the control may be responsive to the radar sensor device to guide or control the focused or enhanced interrogation of the detected object by the forward facing image sensor (such as via directing or controlling the image sensor and/or its field of view or zoom function or via image processing of the captured image data, such as by providing enhanced processing of the area at which the object is detected).

Optionally, and desirably, the forward facing image sensor and the radar sensor device may be commonly established on a semiconductor substrate. Optionally, the semiconductor substrate may comprise one of (i) a germanium substrate, (ii) a gallium arsenide substrate, and (iii) a silicon germanium substrate.

These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.

Referring now to the drawings and the illustrative embodiments depicted therein, a sensing system or forward facing sensing system or radar sensing system <NUM> for a vehicle <NUM> includes a radar sensor device <NUM> at an upper region of the vehicle windshield 12a and with a forward transmitting and sensing direction forward of the vehicle and in the forward direction of travel of the vehicle (<FIG>). The windshield glass 12a may be formed with a cutout or pocket 12b at the upper edge. The pocket may be cut from the glass (so as to provide a cut opening at the upper edge of the glass windshield) or the glass may be formed with an inward bulge or pocket that provides an opening for the sensing device. The radar sensor device <NUM> thus may be disposed at the pocket 12b and may have a clear or unobstructed view or sensing direction forward of the vehicle that does not pass through glass (and whereby the glass windshield will not attenuate the performance of the radar sensor device). Because the upper region of the vehicle windshield is typically not used, the radar sensor device <NUM> may be disposed thereat without being intrusive of other systems or elements and without adversely affecting the vehicle design and/or layout. The sensing system <NUM> is operable to detect objects or vehicles or the like in front of the vehicle as the vehicle is traveling along a road, such as in conjunction with an adaptive cruise control system or the like. Although shown and described as being a forward facing sensing system, aspects of the present disclosure may be suitable for other sensing systems, such as a rearward facing sensing system or the like.

Radar sensor device <NUM> thus may be disposed within a windshield electronics module <NUM> or accessory module or overhead console of the vehicle, and within the vehicle cabin, without experiencing the adverse performance caused by the attenuation of radio or radar frequency electromagnetic radiation wave transmission through the windshield glass. Optionally, the vehicle sheet metal may be adapted to receive and/or support the radar sensor device at the upper edge of the windshield, or to accommodate the radar sensor device as disposed in and/or supported by the windshield electronics module or the like.

In order to seal the upper edge of the windshield at the pocket 12b, a cover element or plate <NUM> may be provided that substantially or entirely spans the opening at the pocket and that is sealed at the glass windshield and vehicle around the perimeter of the pocket, so as to limit or substantially preclude water intrusion or the like into the vehicle at the radar sensor device. The cover element <NUM> preferably comprises a plastic or polymeric or polycarbonate material that is transmissive to radar waves so as to limit or substantially preclude an adverse effect on the performance of the radar sensor device and system. Optionally, and desirably, the cover element may be colored to match or substantially match the shade band along the upper region of the windshield or to match or substantially match the windshield electronics module or other interior or exterior component of the vehicle. Because the radar sensor device does not require a transparent cover, the cover element may be opaque or substantially opaque and/or may function to substantially camouflage or render covert the sensor device and/or the windshield electronics module or the like.

The radar sensor device may utilize known transmitting and receiving technology and may utilize a sweeping beam or a phased array or the like for scanning or sensing or interrogating the area in front of the vehicle. Optionally, the forward facing radar sensing system may include or may be associated with a forward facing camera or imaging sensor <NUM> (which may be disposed at or in the windshield electronics module or accessory module or overhead console or at another accessory module or windshield electronics module or at the interior rearview mirror assembly <NUM> or the like), which has a forward field of view in the forward direction of travel of the vehicle. The sensing system may function to perform a "sweep" of the area in front of the vehicle and if an object or the like is detected (e.g., the radar sensing system detects a "blip"), the radar sensor device and system may hone in on or focus on or further interrogate the region where the object is detected and may perform a more focused or enhanced interrogation of the area at which the object was detected to determine if the object is an object of interest. Optionally, for example, the system may control enhanced interrogation of a detected object by the radar sensor device (such as a beam aiming or beam selection technique, such as by digital beam forming in a phased array antenna system or such as by digital beam steering). Such enhanced interrogation by the radar sensor device may be in response to the forward facing image sensor detecting an object in its forward field of view.

According to the invention, the forward facing camera guides or initiates or controls the more focused interrogation of the suspected object of interest (such as further or enhanced interrogation by the camera and imaging system) in response to the initial detection by the radar sensing system. For example, the radar sensing system may initially detect an object and the forward facing camera may be directed toward the detected object or otherwise controlled or processed to further interrogate the detected object (or area at which the object is detected) via the camera and image processing, or, alternately, the forward facing camera may initially detect an object and the system may select or aim a radar beam in a direction of a detected object. The enhanced interrogation of the object area by the forward facing camera may be accomplished via control of the camera's field of view or degree of zoom [for example, the camera may zoom into the area (via adjustment of a lens of the camera to enlarge an area of the field of view for enhanced processing) at which the object is detected] or via control of the image processing techniques. For example, the image processor may provide enhanced processing of the captured image data at the area or zone at which the object is detected, such as by enhanced or intensified algorithmic processing of a portion of the image plane of the image sensor that is spatially related to the location of the detected object in the forward field of view of the image sensor, such as by enhanced processing of pixel outputs of pixels within a zone or sub-array of a pixelated imaging array sensor, such as by utilizing aspects of the imaging systems described in <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>, and/or <CIT>; and/or Ser. No. <CIT>, which are all hereby incorporated herein by reference in their entireties.

Thus, the sensing system of the present disclosure provides for cooperation or collaboration between the radar sensor device and the forward facing camera or image sensor in a way that benefits or enhances the sensing capabilities of the forward facing sensing system. The sensing system may thus operate with reduced processing until an object is initially detected, and then may provide further processing to determine if the object is an object of interest to the forward facing sensing system.

Optionally, and desirably, the radar sensor device and forward facing camera may be commonly established on a semiconductor substrate, such as a substrate comprising a germanium substrate, a gallium arsenide substrate or a silicon germanium substrate or the like. The substrate may include or may incorporate at least some of the control circuitry for the radar sensor device and camera and/or may include or incorporate common circuitry for the radar sensor device and camera.

Because the radar sensor device and camera may be disposed on a common substrate and are disposed within a windshield electronics module, the forward facing sensing system is removably installed at the vehicle and may be removed therefrom, such as for service or replacement. Thus, the sensing system (including the radar sensor device and camera) comprises a self-contained unit or system that is disposed at the upper region of the windshield. Optionally, the radar sensor device and/or camera may be disposed within a windshield electronics module or the like, such as by utilizing aspects of the modules described in <CIT> for VEHICLE ACCESSORY MODULE, now <CIT>; and/or Ser. No. <CIT>, and/or <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>, and/or in <CIT> by Donnelly Corp. for ACCESSORY SYSTEM FOR VEHICLE, and published Jul. <NUM>, <NUM> as International Publication No. <CIT>, and/or <CIT>; Ser. No.<CIT>; and Ser.

Optionally, the mirror assembly and/or windshield electronics module may include or incorporate a display, such as a static display, such as a static video display screen (such as a display utilizing aspects of the displays described in <CIT> and/or <NUM>,<NUM>,<NUM>, or a display- on-demand or transflective type display or other display utilizing aspects of the displays described in <CIT>; <CIT> and/or <CIT>, and/or <CIT> for VEHICULAR LIGHTING SYSTEM, now <CIT>; <CIT>. for ELECTRO-OPTIC MIRROR CELL, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT> for ACCESSORY SYSTEM FOR VEHICLE; Ser. No.<CIT>; Ser. No. <NUM>/<NUM>,<NUM>, filed Nov. <NUM>, <NUM> and/or Ser. No. <CIT>, and/or <CIT> by Donnelly Corp. , published Nov. <NUM>, <NUM> as <CIT>; and/or <CIT>, published May <NUM>, <NUM> as <CIT>; and <CIT> by Weller et al. for INTERIOR REARVIEW MIRROR ASSEMBLY WITH DISPLAY; Ser. No. <NUM>/<NUM>,<NUM>, filed Jan. <NUM>, <NUM>; and Ser. No. <NUM>/<NUM>,<NUM>, filed Nov. <NUM>, <NUM>, and/or <CIT> by Donnelly Corp. for ACCESSORY SYSTEM FOR VEHICLE, and published Jul. <NUM>, <NUM> as <CIT>). Alternately, the display screen may comprise a display (such as a backlit LCD video display) that is movable to extend from the mirror casing when activated, such as a slide-out display of the types described in <CIT> by Hutzel et al. for ACCESSORY SYSTEM FOR VEHICLE; and/or Ser. No. <NUM>/<NUM>,<NUM>, filed Nov. <NUM>, <NUM>, and/or <CIT> by Donnelly Corp. , and published Nov. <NUM>, <NUM> as <CIT>; and/or <CIT>, and published May <NUM>, <NUM> as <CIT>; and <CIT> by Weller et al. for INTERIOR REARVIEW MIRROR ASSEMBLY WITH DISPLAY; Ser. No. <NUM>/<NUM>,<NUM>, filed Jan. <NUM>, <NUM>; and Ser. No. <NUM>/<NUM>,<NUM>, filed Nov. <NUM>, <NUM>. Optionally, and preferably, the display is episodically extended and/or actuated, such as to display driving instructions to the driver as the vehicle approaches a waypoint or turn along the selected route, and then retracted after the vehicle has passed the waypoint and continues along the selected route.

Optionally, the display on the video screen may be operable to display an alert to the driver of a potential hazardous condition detected ahead of or in the forward path of the vehicle. For example, an output of a forward-viewing active night vision system incorporating an imaging sensor or camera device and near-IR floodlighting (such as those described in <CIT> and <CIT> for IMAGE SENSING SYSTEM FOR A VEHICLE, now <CIT>), or an output of another suitable forward facing sensor or system such a passive far-IR thermal imaging night vision sensor/camera, may be processed by an image processor, such as, for example, an EyeQ(TM) image processing chip available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel. Such image processors include object detection software (such as the types described in <CIT>; and/or Ser. No. <CIT> for OBJECT DETECTION SYSTEM FOR VEHICLE), and they analyze image data to detect objects. The image processor or control may determine if a potentially hazardous condition (such as an object or vehicle or person or deer or the like) may exist in the vehicle path and may provide an alert signal (such as by actuation of a visual indicator or an audible indicator or by an enhancement/overlay on a video display screen that is showing a video image to the driver of what the night vision sensor/camera is seeing) to prompt/alert the driver of a potential hazard (such as a deer or a pedestrian or a fallen rock or the like) as needed or appropriate. The display thus may provide an episodal alert so that the driver's attention is drawn to the display alert only when there is a potential hazard detected. Such a system avoids the driver from having to look forward out the windshield while often looking to or watching a monitor running a video of the camera's output, which is not particularly consumer-friendly and simply loads the driver with yet another task.

Optionally, the mirror assembly and/or windshield electronics module may include or incorporate a navigation device that may include navigational circuitry and a GPS antenna to determine the geographical location of the vehicle and to provide routes to targeted or selected destinations, such as by utilizing aspects of known navigational devices and/or the devices of the types described in <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and/or<CIT>, and/or <CIT> for VEHICLE NAVIGATION SYSTEM FOR USE WITH A TELEMATICS SYSTEM, now <CIT>; <CIT>. for MICROPHONE SYSTEM FOR VEHICLE; Ser. No.<CIT> for ACCESSORY SYSTEM FOR VEHICLE; Ser. No. <NUM><NUM>/<NUM>,<NUM>, filed Sep. <NUM>, <NUM>; and/or Ser. No. <NUM>/<NUM>,<NUM>, filed Oct. <NUM>, <NUM>, now <CIT>, and/or <CIT>; <CIT>; and/or Ser. No. <CIT>for MULTIMEDIA MIRROR ASSEMBLY FOR VEHICLE. Optionally, the mirror or navigation device may include a microphone, whereby the mirror or navigation device may provide voice activated control of the navigation device.

Optionally, for example, the mounting structure and/or mirror casing and/or windshield electronics module may support compass sensors, such as compass sensors of the types described in may utilize aspects of the compass systems described in <CIT> by Blank et al. for INTERIOR REARVIEW MIRROR SYSTEM WITH COMPASS; Ser. No. <NUM>/<NUM>,<NUM>, filed Jan. <NUM>, <NUM> by Schierbeek et al. for VEHICLE COMPASS COMPENSATION, now <CIT>; <CIT>; <CIT>; and/or Ser. No. <NUM>/<NUM>,<NUM>, filed Sep. <NUM>, <NUM> by Kulas et al. for INTERIOR REARVIEW MIRROR ASSEMBLY, now <CIT>; and/or <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>, and/or <CIT> by Donnelly Corp. for MIRROR ASSEMBLY FOR VEHICLE, and published Dec. <NUM>, <NUM> as <CIT>, and/or European patent application, published Oct. <NUM>, <NUM> under <CIT>. The compass circuitry may include the compass sensor, such as a magneto-responsive sensor, such as a magneto-resistive sensor, such as the types disclosed in <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>, a magneto-capacitive sensor, a Hall-effect sensor, such as the types described in <CIT>; <CIT> and <CIT>, a magneto-inductive sensor, such as described in <CIT>, a magneto- impedance sensor, such as the types described in <CIT>, or a flux-gate sensor or the like, and/or may comprise a compass chip, such as described in <CIT>; and/or Ser. By positioning the compass sensors at a fixed location, further processing and calibration of the sensors to accommodate adjustment or movement of the sensors is not necessary.

Optionally, the mounting structure and/or mirror casing and/or windshield electronics module may support one or more imaging sensors or cameras, and may fixedly support them with the cameras set with a desired or appropriate forward and/or rearward field of view. For example, the camera may be operable in conjunction with a forward facing imaging system, such as a rain sensing system, such as described in <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>, and in <CIT> by Schofield et al. for VEHICLE ACCESSORY MODULE, now <CIT>; and/or Ser. No. <NUM>/<NUM>,<NUM>, filed Aug. <NUM>, <NUM> by DeWard et al. for ACCESSORY MODULE FOR VEHICLE. The mounting structure and/or mirror casing may be pressed or loaded against the interior surface of the windshield to position or locate the image sensor in close proximity to the windshield and/or to optically couple the image sensor at the windshield. The mounting structure and/or mirror casing may include an aperture or apertures at its forward facing or mounting surface and the windshield may include apertures through the opaque frit layer (typically disposed at a mirror mounting location of a windshield) or the windshield may not include such a frit layer, depending on the particular application.

Optionally, the image sensor may be operable in conjunction with a forward or rearward vision system, such as an automatic headlamp control system and/or a lane departure warning system or object detection system and/or other forward vision or imaging systems, such as imaging or vision systems of the types described in <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>, and/or <CIT>et al. for VEHICLE HEADLIGHT CONTROL USING IMAGING SENSOR; Ser. No.<CIT>, now <CIT>; <CIT>; <CIT>; and/or Ser. No. <CIT> for OBJECT DETECTION SYSTEM FOR VEHICLE, and/or <CIT>et al. for IMAGING AND DISPLAY SYSTEM FOR VEHICLE; Ser. No.<CIT>; Ser. No.<CIT> for VEHICLE IMAGING SYSTEM; Ser. No.<CIT>for CAMERA MODULE FOR VEHICLE VISION SYSTEM; and/or Ser. No.<CIT>for CAMERA MOUNTED AT REAR OF VEHICLE, and/or International PCT Application No. <CIT>, and published May <NUM>, <NUM> as International Publication No. <CIT>. The mirror casing thus may support one or more rearward facing imaging sensors or cameras, such as for rearward vision or imaging systems, such as for a rear vision system or back up aid of the types described in <CIT> and/or <CIT>, and/or a cabin monitoring system or baby view system of the types described in <CIT>, and/or the like.

Optionally, the fixed mounting structure and/or mirror casing and/or windshield electronics module may house or support a display device, such as a heads up display device (such as the types described in <CIT>; and <CIT>, now <CIT>) that is operable to project a display at the area in front of the driver to enhance viewing of the display information without adversely affecting the driver's forward field of view. For example, the mirror casing may support a heads up display (HUD), such as a MicroHUD(TM) head-up display system available from Micro Vision Inc. of Bothell, WA, and/or such as a HUD that utilizes aspects described in <CIT>; and <CIT>, now <CIT>. For example, Micro Vision's MicroHUD(TM) combines a MEMS-based micro display with an optical package of lenses and mirrors to achieve a compact high-performance HUD module that reflects a virtual image off the windscreen that appears to the driver to be close to the front of the car. This laser-scanning display can outperform many miniature flat panel LCD display screens because it can be clearly viewed in the brightest conditions and also dimmed to the very low brightness levels required for safe night-time driving. For example, such a display device may be located at or in the mirror casing/mounting structure/windshield electronics module and may be non-movably mounted at the mirror casing or mounting structure or windshield electronics module, and may be operable to project the display information at the windshield of the vehicle so as to be readily viewed by the driver of the vehicle in the driver's forward field of view.

The mounting structure and/or mirror casing and/or windshield electronics module may be fixedly attached to or supported at the vehicle windshield and may extend upward toward the headliner of the vehicle. Thus, the mirror assembly may have enhanced wire management and may substantially conceal the wiring of the electronic components/accessories between the circuitry within the mirror casing and the headliner at the upper portion of the vehicle windshield. Optionally, the mirror assembly may include wire management elements, such as the types described in <CIT>; and/or Ser. No. <CIT>; and/or <CIT>, to conceal the wires extending between an upper portion of the mirror casing and the vehicle headliner (or overhead console). Optionally, the mirror casing and/or mounting structure and/or windshield electronics module may abut the headliner and/or may be an extension of an overhead console of the vehicle (such as by utilizing aspects described in <CIT> for ACCESSORY SYSTEM FOR VEHICLE, and/or <CIT> for COVER MODULE, now <CIT>. The mirror assembly thus may allow for utilization of the area above the mirror reflective element for additional mirror content, such as additional electronic accessories or circuitry, and thus may provide for or accommodate additional mirror content/circuitry and/or vehicle content/circuitry.

Claim 1:
A forward-facing sensing system (<NUM>) for a vehicle (<NUM>), the vehicle having a windshield (12a), said forward-facing sensing system (<NUM>) comprising:
a radar sensor device (<NUM>), wherein said radar sensor device has a sensing direction forward of the vehicle;
a forward-facing image sensor (<NUM>) having a forward field of view through windshield (12a);
a control responsive to an output of said radar sensor device and responsive to an output of said forward-facing image sensor;
both said radar sensor device (<NUM>) and said forward-facing image sensor (<NUM>) being disposed within a windshield electronics module (<NUM>) that is removably installed within the vehicle cabin at an upper region of the vehicle windshield (12a) and that may be removed therefrom as a self-contained unit, such as for service or replacement;
characterized by said radar sensor device (<NUM>) and said forward-facing image sensor (<NUM>) collaborating in a way that enhances the sensing capabilities of the forward-facing sensing system to detect objects present in the path of forward travel of the vehicle by:
- the forward-facing sensing system (<NUM>) enhancing interrogation of an area in front of the vehicle at which an object is detected by said radar sensor device (<NUM>) to determine if the detected object is an object of interest causing a potentially hazardous condition; and
- wherein enhancing interrogation of the area at which the object is detected comprises enhanced processing by an image processor of image data captured by said forward-facing image sensor (<NUM>) at the area at which the object is detected.