Automatic rain response system

An automatic rain response system includes microphones of a vehicle, windshield wipers, and a controller. The microphones generate acoustic signals in response to rain. The controller is configured to receive a local weather condition, extract acoustic features from the acoustic signals, form feature vectors in response to the acoustic features and the local weather condition, classify the feature vectors to determine a current class among multiple classes, activate the windshield the wipers at a high speed in response to the current class being a heavy rain on a windshield class, activate the windshield wipers at a medium speed in response to the current class being a freezing rain on the windshield class, activate the windshield wipers at a low speed in response to the current class being a light rain on the windshield class, and deactivate the windshield wipers in response to the current class being a no rain class.

INTRODUCTION

The present disclosure relates to a system and a method for an automatic rain response system.

Current higher-trim vehicles are equipped with dedicated rain-sensing systems that activate windshield wipers while rain is falling. The rain-sensing systems transmit a beam of infrared light that is reflected back by the rain on the windshield. The reflections are used to determine when to activate and when to stop the windshield wipers. The rain-sensing systems involve an expensive set of components that add costs to the vehicles.

Accordingly, those skilled in the art continue with research and development efforts in the field of low-cost automatic rain response systems for automotive vehicle windshields.

SUMMARY

An automatic rain response system is provided herein. The automatic rain response system includes a plurality of microphones of a vehicle, a plurality of windshield wipers, and a controller. The plurality of microphones is configured to generate a plurality of acoustic signals in response to rain striking a windshield. The controller is in communication with the plurality of microphones, and is configured to receive a local weather condition, extract a plurality of acoustic features from the plurality of acoustic signals, form a plurality of feature vectors in response to the plurality of acoustic features and the local weather condition, classify the plurality of feature vectors to determine a current class among a plurality of classes, activate the plurality of windshield wipers at a high speed in response to the current class being a heavy rain on a windshield class, activate the plurality of windshield wipers at a medium speed in response to the current class being a freezing rain on the windshield class, activate the plurality of windshield wipers at a low speed in response to the current class being a light rain on the windshield class, and deactivate the plurality of windshield wipers in response to the current class being a no rain class.

In one or more embodiments, the automatic rain response system includes a windshield heater in communication with the controller. The controller is further configured to activate the windshield heater in response to the current class being the freezing rain on the windshield class.

In one or more embodiments, the automatic rain response system includes a transceiver in communication with the controller, and configured to transmit the current class to a neighboring vehicle using vehicle-to-vehicle communication.

In one or more embodiments, the automatic rain response system includes a transceiver in communication with the controller, and configured to transmit the current class external to the vehicle using vehicle-to-everything communication.

In one or more embodiments, the automatic rain response system includes a navigation system in communication with the controller and configured to determine a location of the vehicle, and a transceiver in communication with the controller, configured to transmit the location from the vehicle to a weather service provider external to the vehicle, and configured to receive the local weather condition at the vehicle from the weather service provider based on the location of the vehicle.

In one or more embodiments, the automatic rain response system includes a temperature sensor configured to measure an ambient temperature around the vehicle. The ambient temperature determines the local weather condition.

In one or more embodiments, the automatic rain response system includes a receiver configured to receive a neighboring current class from a neighboring vehicle in proximity to the vehicle. The current class of the vehicle is set to the neighboring current class.

In one or more embodiments of the automatic rain response system, the controller is further configured to determine if appropriate to close a window, and recommend or command a closing of the window where appropriate and in response to the classifying being one of the heavy rain on the windshield class, the light rain on the windshield class, or the freezing rain on the windshield class.

In one or more embodiments of the automatic rain response system, the plurality of classes includes a windshield wiper noise class, a turn signal noise class, and an occupant sneezing noise class.

A method for automatic rain response for a vehicle is provided herein. The method includes generating a plurality of acoustic signals with a plurality of microphones of the vehicle in response to rain striking a windshield, extracting a plurality of acoustic features from the plurality of acoustic signals, receiving a local weather condition, forming a plurality of feature vectors in response to the plurality of acoustic features and the local weather condition, classifying the plurality of feature vectors to determine a current class among a plurality of classes, activating a plurality of windshield wipers at a high speed in response to the current class being a heavy rain on a windshield class, activating the plurality of windshield wipers at a medium speed in response to the current class being a freezing rain on the windshield class, activating the plurality of windshield wipers at a low speed in response to the current class being a light rain on the windshield class, and deactivating the plurality of windshield wipers in response to the current class being a no rain class.

In one or more embodiments, the method includes activating a windshield heater in response to the current class being the freezing rain on the windshield class.

In one or more embodiments, the method includes transmitting the current class to a neighboring vehicle using vehicle-to-vehicle communication.

In one or more embodiments, the method includes transmitting the current class external to the vehicle using vehicle-to-everything communication.

In one or more embodiments, the method includes determining a location of the vehicle, transmitting the location from the vehicle to a weather service provider external to the vehicle, and receiving the local weather condition at the vehicle from the weather service provider based on the location of the vehicle.

In one or more embodiments of the method, the local weather condition is determined by measuring an ambient temperature around the vehicle with a temperature sensor of the vehicle.

In one or more embodiments, the method includes receiving a neighboring current class from a neighboring vehicle in proximity to the vehicle, and setting the current class of the vehicle to the neighboring current class.

In one or more embodiments, the method includes determining if appropriate to close a window, and recommending or commanding a closing of the window where appropriate and in response to the classifying being one of the heavy rain on the windshield class, the light rain on the windshield class, or the freezing rain on the windshield class.

In one or more embodiments of the method, the plurality of classes includes a windshield wiper noise class, a turn signal noise class, and an occupant sneezing noise class.

A vehicle is provided herein. The vehicle includes a windshield and an automatic rain response system. The automatic rain response system is in communication with the windshield. The automatic rain response system includes a plurality of microphones, a plurality of windshield wipers, and a controller. The plurality of microphones is configured to generate a plurality of acoustic signals in response to rain striking the windshield. The controller is in communication with the plurality of microphones, and is configured to receive a local weather condition, extract a plurality of acoustic features from the plurality of acoustic signals, form a plurality of feature vectors in response to the plurality of acoustic features and the local weather condition, classify the plurality of feature vectors to determine a current class among a plurality of classes, activate the plurality of windshield wipers at a high speed in response to the current class being a heavy rain on a windshield class, activate the plurality of windshield wipers at a medium speed in response to the current class being a freezing rain on the windshield class, activate the plurality of windshield wipers at a low speed in response to the current class being a light rain on the windshield class, and deactivate the plurality of windshield wipers in response to the current class being a no rain class.

In one or more embodiments of the vehicle, the automatic rain response system is characterized by a lack of a dedicated rain sensor.

DETAILED DESCRIPTION

Embodiments of the disclosure an automatic rain response system that utilizes microphone components to capture non-speech transient noise caused by rain drops falling on a windshield of a vehicle. The non-speech noise is classified using garbage modeling to isolate the rain drop noise from other transient noise types, such as windshield wiper noises, lip smacking noises, sneezing noises, turn signal noises, and the like. The classification also quantifies an intensity of the rain drop noises. For example, the rain may be classified as a heavy rain, a light rain, a freezing (medium) rain, and no rain. A rejection threshold (e.g., posterior conditional probability) may be calibrated to a high level to minimize false alerts. In some embodiments, the microphones may be on an exterior of the vehicle to accurately capture road noises. In various embodiments, interior microphones, such as rear view mirror microphones, headliner microphones and/or overhead console microphones may be used to detect the rain striking the windshield. Upon successful audio-based classification of the rain drops falling on the windshield, a vehicle network-based message may be triggered and windshield wipers activated to a default level. The automatic rain response system may serve as a replacement for dedicated rain-sensing systems and thus helps to save component costs. Thus, the vehicle is characterized by a lack of a dedicated rain sensor.

Referring toFIG.1, a schematic plan diagram of an example context of an environment60around a vehicle70is shown in accordance with one or more exemplary embodiments. The environment60may include an atmosphere at an ambient temperature62and rain64at various times. The vehicle70resides on the ground (e.g., a roadway) at a current location72. One or more neighboring vehicles74(one shown) may be in the vicinity of the vehicle70. One or more weather service providers76(one shown) may be within radio-communication range of the vehicle70.

The vehicle70is designed to carry a driver90and one or more passengers (not shown). The vehicle70includes an automatic rain response system100. The automatic rain response system100includes a windshield92, one or more windows94(one shown), and an automatic rain response system100. The automatic rain response system100generally includes multiple microphones110, a temperature sensor120, a controller130, multiple windshield wipers140(front wipers are shown), a windshield heater150(front heater is shown), a transceiver160, a navigation system170, an active noise cancellation system180, and a communication bus190.

The rain64striking the windshield92may generate a rain noise96. Characteristics of the rain noise96depend on a density of the rain64, a windspeed, and a speed of the vehicle70. The rain noise96is sensed by the microphones110. The microphones110may also sense other ambient noise98originating from both inside the vehicle70and outside the vehicle70.

Vehicle-to-vehicle (V2V) communication80may be established between the automatic rain response system100and the neighboring vehicle(s)74. The vehicle-to-vehicle communication80may transfer information for a current class80aof the rain64detected by the vehicle70. The current class80amay be used by the neighboring vehicle(s)74as a status of the rain64in the area. The vehicle-to-vehicle communication may transfer information for a neighboring current class80bof the rain64detected by the neighboring vehicle(s)74. The neighboring current class80bmay be used by the vehicle70as a status of the rain64at the location72.

Vehicle-to-everything (V2X) communication82may be established between the automatic rain response system100and the weather service provider(s)76. The vehicle-to-everything communication82may transfer the location72of the vehicle70to the weather service provider(s)76. In response, the weather service provider(s)76returns a local weather condition82bto the vehicle70based on the location72. The local weather condition82bmay include one or both of a status of the rain64at the location72and/or an ambient temperature62at the location72of the vehicle70. In some embodiments, the automatic rain response system100acts as a roving weather station and transmits the current class80aof the rain64to the weather service provider(s)76.

Multiple acoustic signals112are generated by the microphones110and transferred to the controller130via the communication bus190. The acoustic signals convey both the rain noise96and the ambient noise98.

A temperature signal122is generated by the temperature sensor120and transferred to the controller130on the communication bus190. The temperature signal carries data for the ambient temperature62of the air around the vehicle70.

The vehicle70implements a gas-powered vehicle, an electric vehicle, a hybrid vehicle, or a plug-in hybrid vehicle. In various embodiments, the vehicle70may include, but is not limited to, a passenger vehicle, a truck, an autonomous vehicle, a motorcycle, a boat, and/or an aircraft. Other types of vehicles70may be implemented to meet the design criteria of a particular application.

The windshield92implements a typical vehicle front windshield. The windshield92may be subjected to the rain64from time to time. The rain64striking the windshield92generates the rain noise96. Although the front windshield is discussed herein, the technique of the disclosure may be applied to a read windshield with rear wipers and/or a read windshield heater.

The windows94implement one or more of side windows and/or sun roofs. The windows94may be powered to transition between an open condition and a closed condition in response to control signals received via the communication bus190.

The automatic rain response system100implements a distributed rain sensing and windshield wiper control system. The automatic rain response system100is operational to generate the acoustic signals112in response to the rain64hitting the windshield92, extract acoustic features from the acoustic signals112, and determine a local weather condition from the temperature sensor120, the neighboring current class80breceived from a neighboring vehicle74and/or from the weather service provider76. The automatic rain response system100subsequently forms feature vectors in response to the acoustic features and the local weather condition, and classifies the feature vectors to determine the current class80aamong multiple possible classes. The automatic rain response system100responds to rain classes, where detected, by activating the windshield wipers140. The windshield wipers140may be set to a high speed, a medium speed, a low speed, or off (e.g., no speed) based on the classification results. Where the classification determines that the rain64is a freezing rain, the windshield wipers140may be set to the medium speed and the windshield heater150is activated.

In various embodiments, the automatic rain response system100may transmit the current class80ato one or more neighboring vehicles74and/or to one or more weather service providers76. If the windows94are open when the rain64is initially detected, the automatic rain response system100sends signals to close the open windows94, even if the vehicle70is powered down (e.g., engine off).

The microphones110implement exterior and/or interior microphones. In various embodiments, the microphones110may include road noise cancellation (RNC) microphones. The microphones110are operational to convert the rain noise96and the ambient noise98into the acoustic signals112. The acoustic signals112are transferred to the controller130via the communication bus190.

The temperature sensor120implements an air temperature sensor. The temperature sensor120is operational to detect the ambient temperature62of the air around the vehicle70and report the detected temperature in the temperature signal122to the controller130. The temperature sensor120enables the controller130to differentiate between freezing rain and non-freezing rain.

The controller130implements one or more electronic control units. In various embodiments, the controller130may be implemented in a body controller of the vehicle70. The controller130is operational to receive the acoustic signals112, the temperature signal122and the local weather condition82bfrom the weather service providers76and/or the neighboring current class80bfrom the neighboring vehicles74. The controller130uses the gathered information to extract the features from the acoustic signals112, form the feature vectors in response to the acoustic features and the local weather condition82band/or the neighboring current class80b, classify the feature vectors to determine the current class80a, and activate/deactivate windshield wipers140and the windshield heater150while the current class80ais one of the rain classifications.

In various embodiments, the controller130generally comprises at least one microcontroller. The at least one microcontroller may include one or more processors, each of which may be embodied as a separate processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a dedicated electronic control unit. The at least one microcontroller may be an electronic processor (implemented in hardware, software executing on hardware, or a combination of both). The at least one microcontroller may also include tangible, non-transitory memory (e.g., read-only memory in the form of optical, magnetic, and/or flash memory). For example, the at least one microcontroller may include application-suitable amounts of random-access memory, read-only memory, flash memory and other types of electrically-erasable programmable read-only memory, as well as accompanying hardware in the form of a high-speed clock or timer, analog-to-digital and digital-to-analog circuitry, and input/output circuitry and devices, as well as appropriate signal conditioning and buffer circuitry.

Computer-readable and executable instructions embodying the present method may be recorded (or stored) in the memory and executed as set forth herein. The executable instructions may be a series of instructions employed to run applications on the at least one microcontroller (either in the foreground or background). The at least one microcontroller may receive commands and information, in the form of one or more input signals from various controls or components in the vehicle70and communicate instructions to the other electronic components.

The windshield wipers140implement one or a pair of front wipers and/or rear wipers. The windshield wipers140are operational to clear the rain64from the windshield92when active. An on/off state and a speed of the windshield wipers140is controlled via commands on the communication bus190. The commands may originate from the controller130and from manual controls available to the driver90.

The windshield heater150implements a front windshield defroster and/or rear windshield defroster. The windshield heater150is operational to clear freezing rain64, sleet, snow and frost from the windshield92when active. An on/off state and a speed of the windshield heater150is controlled via commands on the communication bus190. The commands may originate from the controller130and from manual controls available to the driver90.

The transceiver160implements a radio-frequency transmitter and receiver. The transceiver160is operational to provide bidirectional communications between the controller130and the neighboring vehicles74(e.g., V2V communication) and between the controller130and the weather service providers76(e.g., V2X communication).

The navigation system170implements an inertial navigation system, a cellular triangulation system, and/or a Global Positioning System (GPS) receiver. The navigation system170is operational to determine the location72of the vehicle70in real time. The location72is transferred to the controller130via the communication bus190.

The active noise cancellation system180is operational to dampen (or lower) road noises experienced inside the vehicle70. The active noise cancellation system180is generally activated while the classification of the acoustic signals112determines that a moderate to heavy rain64is present.

The communication bus190implements a bidirectional digital bus. The communication bus190is operational to exchange data among at least the microphones110, the temperature sensor120, the controller130, the windshield wipers140, the windshield heater150, the transceiver160and the navigation system170. Other electronics within the vehicle70may communicate on the communication bus190to meet the design criteria of a particular application.

In various embodiments, the controller130implements a speech recognition engine that performs the feature vector formation and classification. The speech recognition engine may be located within an infotainment head unit and/or other electronic control unit(s). The speech recognition engine has a garbage modeling feature that identifies non-speech sounds (e.g., noise types). The garbage modeling may accurately classify transient noise types, such as rain drops, windshield wipers, lip smacks, sneezing, turn signals, and the like. Windshield wiper speed, window/sunroof level and/or heated windshield on/off are controlled according to the recognized rain level and in accordance with Federal Motor Vehicle Safety Standard (FMVSS) 118/S6 for remote closure control of movable glass (e.g., windows, sunroof, etc.).

Referring toFIG.2, a schematic diagram of an example implementation of the automatic rain response system100within the environment60is shown in accordance with one or more exemplary embodiments. Features implemented in the controller130include feature extraction200, feature vector formation202, a multiclass classifier204, and level decision206.

The rain64striking the windshield92is converted into the acoustic signals by the microphones110. The feature extraction200determines acoustic features from the acoustic signals112. The acoustic features are extracted with a high discrimination power to recognize multiple rain levels (e.g., no rain, light rain, heavy rain). By way of example, the acoustic features230may include, but are not limited to, Mel-frequency cepstral coefficients (MFCCs), filterbank energies, log filterbank energies, and spectral subband centroids. The acoustic features230are subsequently presented to the feature vector formation202.

The feature vector formation202receives the acoustic features230from the feature extraction200, the local weather condition82bfrom the weather service provider via the controller130, and an ambient temperature in the ambient temperature signal122. Based on the data received, the feature vector formation202is operational to generate a sequence of feature vectors232. The feature vector formation202includes features that lead to maximum variance in terms of Eigen values and Eigen vectors. Besides Mel cepstrum and subspace centroids, other features such as delta cepstrum, perceptual linear prediction (PLP), and linear prediction coefficients (LPC) provide good classification of transient non-speech sounds. The feature vectors232are transferred to the multiclass classifier204.

The multiclass classifier204is operational to classify the feature vectors232among multiple classes210a-210gand determine the current class80a. The classes210a-210gmay include, but are not limited to, a heavy rain on a windshield class210a, a light rain on the windshield class210b, a freezing rain on the windshield class210c, a no rain class210d, a windshield wiper noise class210e, a turn signal noise class210f, and an occupant sneezing noise class210g. The current class80ais presented to the level decision206, fed back to the multiclass classifier204for adaptation, and optionally provided to the transceiver160for transmission to other destinations remote from the vehicle70. In various embodiments, the rain recognition classification is continuously adapted using the data collected through retraining and over the air updates (OTA) following transfer learning and/or continual learning approaches.

The level decision206is operational to examine the current class80a. If the current class80ais one of the rain-related classes210a-210d, the level decision206determines which rain-related class210a-210dhas been detected. Where the current class80ais the heavy rain on the windshield class210a, the level decision206activates212athe windshield wipers140to a high speed. The level decision206may also enable the active noise cancellation system214, if implemented, to lower the sound of the heavy rain inside the passenger compartment of the vehicle70.

The active noise cancellation system214is switched on if heavy rain is detected in order to reduce transient noise effects. The transient noise data includes noise from windshield wipers, beeps, phone or dual tone multi-frequency (DTMF) tones, etc. The vehicle70may have pre-loaded noise acoustic models that are trained offline and persistently adapted using transient noise data. The active noise cancellation system214uses noise acoustic models and real-time rain classification result, to reduce the transient noise in vehicle70.

In some instances, the transceiver160may share the data in the current class80ausing vehicle-to-vehicle communications220. The vehicle-to-vehicle communications220alerts the neighboring vehicle(s)74of the possible rain within a geofenced area to equip the neighboring vehicle(s)74with a virtual rain sensor. The transceiver160may also share the data in the current class80ausing vehicle-to-everything communications222such that the vehicle70acts as a mobile weather station within an infrastructure.

Referring toFIG.3, a flow diagram of an example implementation of a method240for automatic rain response in the vehicle70is shown in accordance with one or more exemplary embodiments. The method (or process)240is implemented by the automatic rain response system100. The method240generally includes steps242to266, as illustrated. The sequence of steps is shown as a representative example. Other step orders may be implemented to meet the criteria of a particular application.

In the step242, the acoustic signals112are generated with the microphones110of the vehicle70. The acoustic features230are extracted from the acoustic signals in the step244by the feature extraction200in the controller130. In the step246, the local weather condition82bis determined. In various embodiments, the local weather condition82bmay be determined by measuring the ambient temperature62around the vehicle70with the temperature sensor120. The feature vectors232are formed in response to the acoustic features230and the local weather condition82bin the step248using the feature vector formation202.

In the step250, the multiclass classifier204may classify the feature vectors to determine a current class80aamong the possible classes210a-210g. The level decision206activates212athe windshield wipers140at the high speed in the step252in response to the current class80abeing a heavy rain on a windshield class210a. In the step254, the level decision206activates212cthe windshield wipers140at the medium speed in response to the current class80abeing the freezing rain on the windshield class210c. The level decision206may also activate the windshield heater150in the step256in response to the current class80abeing the freezing rain on the windshield class210c. In the step258, the level decision206activates212cthe windshield wipers140at the low speed in response to the current class80abeing the light rain class on the windshield class210c. In the step260, the level decision206deactivates212dthe windshield wipers140in response to the current class80abeing the no rain class210d. In the step262, the controller130may command the transceiver160to transmit the current class80athat represents the rain/no rain state (e.g., classes210a-210d) to the neighboring vehicle(s)74.

In the step264, the controller130may determine if closing the window(s)94is appropriate. If the appropriate criteria are meet, the controller130may recommend or command closing the window(s)94in response to the current class80abeing one of the heavy rain on the windshield class210a, the light rain on the windshield class210b, or the freezing rain on the windshield class210cin the step266. In various embodiments, the controller130may leave the window(s)94unmoved despite the presence of the rain64.

Referring toFIG.4, a flow diagram of an example implementation of a step246afor determining the local weather condition is shown in accordance with one or more exemplary embodiments. The step246ais implemented by the automatic rain response system100and a weather service provider76. The step246ais a variation of the step246. The step246agenerally includes steps280to288, as illustrated. The sequence of steps is shown as a representative example. Other step orders may be implemented to meet the criteria of a particular application.

In the step280, the navigation system170determines the location72of the vehicle70. The controller130commands the transceiver160to transmit the location72in the step282from the vehicle70to a weather service provider76that is external to the vehicle70. The weather service provider76responds to the location72in the step284by returning the local weather condition82bat the location72. In the step286, the transceiver160receives the local weather condition82bat the vehicle70from the weather service provider76based on the location72. The controller130substitutes or blends the features of the local weather condition82breceived from the weather service provider76with the acoustic features230determined by the feature extraction200in the step288.

Referring toFIG.5, a flow diagram of an example implementation of a step250afor determining the current class is shown in accordance with one or more exemplary embodiments. The step250ais implemented by the automatic rain response system100and a neighboring vehicle74. The step250ais a variation of the step250. The step250agenerally includes steps290to292, as illustrated. The sequence of steps is shown as a representative example. Other step orders may be implemented to meet the criteria of a particular application.

In the step290, the transceiver160may receive the neighboring current class80b(FIG.1) from the neighboring vehicle74in proximity to the vehicle70. In the step292, the controller130sets the current class80ato the neighboring current class80bas received from the neighboring vehicle74. Therefore, the vehicle70may be aware of the falling rain64while the vehicle70is in a tunnel, or other shelter that blocks the rain64, by using the neighboring vehicle74as a nearby weather station.