Patent Description:
Nevertheless, it is undesirable for the cabin to be completely isolated from the environment. There are, after all, sounds that are important to hear. Among these sounds, referred to herein as "desired audio signals," are those that signal danger. Examples include sirens from emergency vehicles, back-up sounds from nearby vehicles, proximity warnings from nearby vehicles and horns from nearby vehicles, including the sound of bicycle bells. Other examples are sounds that originate from living beings, such as animal noises, for example, the sound of barking dogs, or the crying or shouting children. Other examples include sounds with actual semantic content. These include specific safety-related utterances such as "Help!" or "Look out! ".

There also exist desirable audio signals that are not necessarily related to safety. For example, there exist drive-ins with kiosks that require spoken interaction to place orders for food.

From <CIT> it is known to amplify acoustic signals from a direction of interest. From <CIT> a vehicle situation awareness device is known. From <CIT> an audio system for a vehicle is known. From <CIT> it is known to amplify environmental sound collected by a sound collection unit and to output the amplified signal to an inside of a vehicle. From <CIT> a communication assembly for a vehicle for communicating audio sound in an environment outside of the passenger compartment is known. From <CIT> a sound classification system for a vehicle is known. From <NPL> using a Wiener filter in sound enhancement is known. From <CIT> sound damping in a vehicle is known. From <CIT> a system for audio events detection in a vehicle is known.

To mitigate the risk of the driver and/or passengers missing desirable audio signals, it is useful to provide circuitry to cause the technical effect of transmitting desirable audio signals into the driving cabin of a vehicle while continuing to passively or actively suppress unwanted noise.

The claimed invention is directed to an apparatus according to claim <NUM> and a method according to claim <NUM>.

Some practices of the method include steering the external-microphone set and others include receiving an instruction from an occupant to steer the external-microphone set to form a beam that points to a designated angle.

Still other practices include displaying an image obtained outside the vehicle on a display in the cabin.

Some practices include identifying a threat in the raw audio signal or in the raw audio signal. Examples of such signals include a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, and a spoken utterance. Among these are practices in which identifying the signal corresponding to a threat is carried out based on a machine-learning algorithm. A suitable machine-learning algorithm for such practices is one that has been trained on pre-selected signals. These pre-selected signals correspond to various kinds of threats.

Filters, and in particular, digital filters, come in many forms. In some practices, the filter that is used for application to the raw audio signal is a Wiener filter.

Further practices include displaying a visual alert while broadcasting the filtered audio signal through the cabin loudspeaker(s). Among these practices are those in which the visual alert includes a characterization of the filtered audio signal. Examples of such characterizations include characterizations that identify the nature of the signal, for example whether the signal being broadcast is a siren, a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, or a spoken utterance.

Among the additional practices are those that include displaying a visual characterization of the filtered audio signal while the filtered audio signal is being broadcast into the cabin. This visual characterization provides information indicative of the type of filtered audio signal that is being broadcast. Among these are practices in which the visual information is indicative of whether the filtered audio signal includes one of a siren, a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, and a spoken utterance.

Further practices include selecting the filter that is to be applied to the raw audio signal based on the vehicle's state.

Also among the practices of the method are those that further include broadcasting sound that originated inside the vehicle outside the vehicle using an outbound audio channel. This outbound audio channel includes both an external-loudspeaker set that includes one or more exterior loudspeakers that are disposed to broadcast sound outside the vehicle and an internal-microphone set that includes one or more internal microphones positioned to receive an audio signal from inside the cabin.

Among the foregoing practices are those that include preventing the outbound audio channel from broadcasting the sound during a selected period. These, in turn, include practices in which preventing the outbound audio channel from broadcasting the sound during a selected period includes causing the outbound audio channel to broadcast noise outside the vehicle during the selected period, practices in which preventing the outbound audio channel from broadcasting the sound during a selected period includes silencing the outbound channel during the selected period, and practices in which preventing the outbound audio channel from broadcasting the sound during a selected period includes causing the outbound channel to broadcast selected programming outside the vehicle during the selected period.

For those cases in which the external-loudspeaker set includes a first external-loudspeaker and a second external-loudspeaker, there exist practices of the method that further include suppressing output of sound from the first external-loudspeaker while concurrently permitting the sound to be broadcast from the second external-loudspeaker.

Still other practices of the method include causing the sound to be broadcast on only one side of the vehicle.

Alternative practices of the method include applying first and second weights to first and second internal microphones, respectively, from the internal-microphone set. In such practices, the first and second weights differ in one or more of their respective real and imaginary parts or in one or more of their respective magnitudes and exponents.

Still other practices include those in which the various signal processing steps are carried out in the frequency domain, for example using a bank of filters and using a Fourier transform, such as that implemented using an FFT. In such cases, signals are represented by complex values. Embodiments include those in which filtering is carried out by changing the real and imaginary parts of the signal and those in which filtering is carried out using amplitude and phase of the signal. Among these are practices that include causing a time delay in a signal by incorporating a phase shift and those in which a time delay is caused by multiplying a signal by a complex exponential having an argument that is related to the desired delay. Among these practices are those in which the argument of the complex exponential is obtained by taking the ratio of the imaginary part to the real part and causing, as a delay, a value that corresponds to that phase angle having that ratio as a tangent thereof.

Still other practices include providing, to the external-loudspeaker set, an inner product of a weight vector and an internal-microphone vector, the internal-microphone vector including, as elements thereof, values representative of signals generated by corresponding internal microphones from the internal-microphone set.

Embodiments also include those in which the external-microphone set includes a microphone array having a steerable beam and those that include a user interface to permit an occupant of the cabin to steer a beam formed by the external-microphone set.

Also among the embodiments are those that include an external-camera system including a camera and a user interface that displays an image received by the camera and those that include a user interface to permit an occupant of the cabin to point a camera of an external-camera set to different locations outside the vehicle.

In some of the foregoing embodiments, the signal processor is further configured to identify a threat in the filtered audio signal. Among these are embodiments in which the signal processor is further configured to identify, in the filtered audio signal, at least one of a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, and a spoken utterance and those in which the signal processor is further configured to identify a threat in the sound from outside the cabin and to do so based on a machine-learning algorithm that has been trained on pre-selected signals that correspond to threats.

Also among the embodiments that filter the raw audio signal are those in which the filter includes a Wiener filter.

Also among the embodiments are those that include an outbound audio channel that includes an external-loudspeaker set and an internal-microphone set. The external-loudspeaker set includes one or more exterior loudspeakers that are disposed to broadcast sound outside the vehicle. The internal-microphone set includes one or more internal microphones positioned to receive an audio signal from inside the cabin. In such embodiments, the signal processor is further configured to transition between broadcasting an audio signal detected by the internal-microphone set via the external-loudspeaker set and broadcasting an audio signal detected by the external-microphone set via the internal-loudspeaker set.

Further embodiments include a user interface that displays a visual alert when the filtered audio signal is being broadcast into the cabin. Among these are embodiments in which the user interface also displays a visual indication of a characteristic of the filtered audio signal when the filtered audio signal is being broadcast into the cabin. Among these embodiments are those in which the visual indication identifies the existence, in the filtered audio signal, of at least one of: a siren, a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, and a spoken utterance.

Further embodiments include those in which the signal processor is configured to apply the filter that has been selected based on a state of the vehicle.

Additional embodiments include an outbound audio channel that includes an external-loudspeaker set and an internal-microphone set. The external-loudspeaker set includes one or more exterior loudspeakers that are disposed to broadcast sound outside the vehicle. The internal-microphone set includes one or more internal microphones positioned to receive an audio signal from inside the cabin. The inbound and outbound channels collectively form a trans-cabin communication system.

Among the foregoing embodiments are those that include a user interface to provide an ability to control operation of the outbound audio channel. In such embodiments, the ability to control the outbound audio channels includes one or more of an ability to prevent the outbound audio channel from operating during selected periods of operation of the trans-cabin communication system, an ability to cause the outbound audio channel to carry noise, an ability to silence the outbound audio channel, and an ability to cause the outbound audio channel to transition between providing a signal from inside the cabin and playing selected programming.

Other features and advantages of the invention are apparent from the following description, and from the claims, in which:.

<FIG> is a cross-section of a vehicle <NUM> showing its cabin <NUM>. The cabin <NUM> accommodates one or more occupants. The illustrated vehicle is a motorized vehicle configured for travel along roads. Examples of such vehicles include automobiles, trucks, vans, and sport-utility vehicles.

The vehicle <NUM> includes an internal-loudspeaker set that comprises one or more internal loudspeakers <NUM>, an external-loudspeaker set that comprises one or more external loudspeakers <NUM>, an external-microphone set that comprises one or more external microphones <NUM>, an internal microphone set that comprises one or more internal microphones <NUM>, and an external-camera set that comprises at least one camera <NUM>. These cooperate to form a trans-cabin communication system <NUM> in which the external microphones <NUM> and internal loudspeakers <NUM> form an inbound audio channel <NUM>, the internal microphones <NUM> and external loudspeakers <NUM> form an outbound audio channel <NUM>, and the camera <NUM> forms an inbound video channel <NUM>.

In a preferred embodiment, the inbound and outbound channels <NUM>, <NUM> are simultaneously operable, as a result of which the trans-cabin communication system <NUM> is full duplex. Such a trans-cabin communication system <NUM> permits two-way communication between an occupant of the cabin <NUM> and an entity outside the vehicle <NUM> without the need to open a window or door.

The vehicle <NUM> also includes a user-interface <NUM>. In some implementations, the user-interface <NUM> includes a screen of a vehicle infotainment system. In other implementations, the user-interface <NUM> includes one or more lights that are activated in conjunction with a transmitted audio signal.

The user-interface <NUM> provides a location for implementing a human-machine interface to control the trans-cabin communication system <NUM>, for example by controlling the operation of the external and internal microphones <NUM>, <NUM>, the operation of the internal and external loudspeakers <NUM>, <NUM>, the availability of the inbound audio and outbound audio channels <NUM>, <NUM>, and operation of the camera <NUM>, including pointing the camera <NUM> in a desired direction so as to cause an image to be displayed on the screen of the infotainment system.

The user-interface <NUM> thus provides a way to manually activate, pause, or stop operation of the inbound audio channel <NUM>, the outbound audio channels <NUM>, or the inbound video channel <NUM> in any combination.

In addition, the user-interface <NUM> provides the ability to operate the trans-cabin communication system <NUM> in a private mode. Private mode disables the outbound audio channel <NUM>, thus enabling occupants to talk amongst themselves without a person outside the vehicle <NUM> being able to listen, provide of course that the windows are shut.

In some cases, conversation amongst the occupants is loud enough to be partially audible outside the vehicle <NUM> even with the outbound audio channel <NUM> having been disabled. To further ensure privacy, the private mode provides the ability to play a masking sound through the external loudspeakers <NUM>. Examples of suitable masking sounds include stationary noise or music.

The vehicle <NUM> also includes signal processor <NUM> with which the microphones <NUM>, <NUM>, loudspeakers <NUM>, <NUM>, and camera <NUM> are in data communication. The signal processor <NUM> distinguishes desirable audio signals <NUM> from undesirable audio signals <NUM> and ultimately transmits a filtered audio signal <NUM> into the driving cabin via one or more of the internal loudspeakers <NUM>. In addition, the signal processor <NUM> determines weights to apply to any one or more of the microphones <NUM>, <NUM> and the loudspeakers <NUM>, <NUM> while operating the trans-cabin communication system <NUM>.

The external microphones <NUM> are mounted on the vehicle <NUM> so as to permit them to detect audio signals <NUM>, <NUM> from sources outside the cabin <NUM>. A convenient place to mount the external microphones <NUM> is to integrate them into an existing antenna module.

In some implementations, signals provided by the external microphones <NUM> are used to estimate the direction-of-arrival of an audio signal <NUM>, <NUM>. The number of external microphones <NUM> governs the resolution with which this can be accomplished. Practical embodiments, include two, three, or four external microphones <NUM>.

The internal microphones <NUM> are mounted in the vehicle <NUM> such that they detect audio signals from the occupants inside the driving cabin <NUM>. Embodiments include those in which each seat in the cabin <NUM> has its own internal microphone <NUM>. The signal processor <NUM> sums the signals from the internal microphones <NUM> and provides them to the external speakers <NUM>.

When using the trans-cabin communication system <NUM>, it is useful for all occupants in the cabin to be perceived, by a listener outside the vehicle <NUM>, as having voices of roughly equal amplitude. Since the occupants are at different locations relative to the internal microphones <NUM>, there is no guarantee that this will be the case. As a result, in some embodiments, the signal processor <NUM> applies different complex-valued weights to the signals received from the internal microphones <NUM> and provides the resulting weighted sum to the external loudspeakers <NUM>.

The signal processor's ability to weight signals is particularly useful in connection with ensuring that spatial cues from outside the vehicle <NUM> are made available to occupants of the cabin <NUM>. A difficulty that arises when simply passing a signal into the cabin is that the occupant may have a mistaken impression of where the sound is coming from.

As an example, if the external microphones <NUM> detect a siren from the left of the vehicle <NUM> and the signal processor <NUM> simply passes that sound to all of the internal loudspeakers <NUM>, then each loudspeaker <NUM> will play the siren at the same amplitude. Thus, the occupants will have the impression that sirens are present in every direction.

The availability of plural internal loudspeakers <NUM> makes it possible to weight the outputs of the individual internal loudspeakers <NUM> makes it possible to provide spatial cues to suggest a location from which a sound from outside the vehicle <NUM> originated. For example, if, as a result of having exercised the ability of the external microphones <NUM> to determine a direction-of-arrival of an audio signal, it has been determined that a sound has arrived from a particular direction, it becomes possible to apply suitable complex weights to the outputs of the internal loudspeakers <NUM> in such a way as to cause the occupants of the cabin <NUM> to perceive that sound as coming from that direction. The details of this operation, referred to as "spatialization," are discussed in connection with <FIG> and <FIG>.

The availability of plural external loudspeakers <NUM> makes it possible to control the direction in which the vehicle <NUM> emits sound. For example, if a sound is to be directed towards a particular angle, only those speakers that face in the general direction of that angle are activated. This feature is particularly useful in those cases in which one wishes to communicate with a person or machine that is on one side of the vehicle <NUM>. In such cases, it is desirable to deactivate those external loudspeakers <NUM> that face away from that side of the vehicle <NUM>.

For example, when communicating with a person who is standing by one side of the vehicle, it is possible for an occupant, using the user-interface <NUM>, to both disable those external loudspeakers <NUM> that are not useful for such interaction and to steer the beam formed by external microphones <NUM> in whatever direction is most useful for such interaction.

The availability of plural external microphones <NUM> makes it possible to receive sounds from any direction. However, it also opens up the possibility of applying complex weights to the outputs of those external microphones <NUM>, thereby forming a steerable microphone array. Embodiments include those in which an occupant steers the beam using the user-interface <NUM> and those in which the beam steers itself automatically, for example by source localization.

<FIG> shows hardware for implementing a filtering system <NUM> within the vehicle <NUM>. The filtering system <NUM> includes a filter selector <NUM>, vehicle-state data <NUM>, a filter calculator <NUM>, one or more sound models <NUM>, and an input/output module <NUM> that is connected to an external microphone <NUM> and an internal loudspeaker <NUM>.

The filter selector <NUM> selects filter characteristics that will be used to distinguish desirable and undesirable audio signals. For the sake of clarity and convenience, the words "desirable" and "undesirable" are used in this document in connection with audio signals. In the context of a particular filter, a "desirable" signal is a signal that satisfies the conditions for re-transmission into the cabin of a vehicle. In the context of a particular filter, the words "desirable" or "undesirable" are not meant to connote any subjective valuation or characterization of an audio signal. Instead, an "undesirable" audio signal is anything removed by a particular filter, and a "desirable" one is anything that is passed through to the cabin.

During operation, a vehicle <NUM> transitions between various vehicle states. In some implementations, it is advantageous to use different filter characteristics in different vehicle states. For example, when the vehicle is in the state of moving in reverse, a filter that is relatively permissive to spoken utterances is appropriate to mitigate the risk of accidents. In contrast, when a vehicle <NUM> is in the state of being a parked, the low risk of accidents makes a less permissible filter desirable.

According to the claimed invention, the filter selector <NUM> accesses vehicle-state data <NUM> to identify the vehicle's state. This vehicle-state data <NUM> includes any measurable data about the car or its environment. Examples of vehicle-state data <NUM> include the vehicle's speed, its direction of travel, its tachometer reading, and the vehicle's engine-noise level, which is readily measured by one or more on-board microphones. Other examples of vehicle-state data <NUM> include which gear is engaged, whether any subsystem (e.g., the radio subsystem or the air-conditioning subsystem) is activated, and if so, what those subsystems' settings are, ambient temperature as measured by one or more thermometers, the vehicle's location as measured by GPS, and various user preferences, such as how permissive or aggressive the level of sound filtering should be set.

The ensemble of all possible vehicle states, including user preferences defines a vehicle-state space. In some embodiments, the vehicle-state space is partitioned into non-overlapping regions such that a set of unique filter characteristics is associated with each region. In this case, the filter calculator <NUM> determines the set of unique filter characteristics associated with the current vehicle state.

Once particular filter characteristics have been selected, the filter calculator <NUM> is operable to filter incoming audio signals using the selected filter. Further details of filter computation are provided below.

In some implementations, the filter that is used to distinguish stationary audio signals from transitory audio signals is a Wiener filter. As used herein, a stationary signal is one whose spectrum is approximately constant over time. A transitory signal is a signal that is not a stationary signal.

In this context, "approximately constant" means constant to within a pre-defined threshold supplied by a user or a manufacturer. In the context of operating a vehicle, road noise and engine noise are significant contributors of stationary audio signals and may therefore be filtered out by some filters.

In some implementations, the filter involves one or more classifications of input sounds. Examples include a filter classifies an input audio signal as being a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, or a spoken utterance. Such filters are created in a variety of ways, among which is that of training one or more machine-learning algorithms on various inputs that have been pre-classified as falling into these categories.

In a typical case, a human carries out the pre-classification and a machine-learning algorithm uses the initial classification to learn how to classify on its own. Examples of machine-learning algorithms include neural nets and deep-learning algorithms.

In some implementations, acoustic features corresponding to the signals described above are provided as input to the machine learning algorithms. Such features are embodied in various ways, such as by Mel-frequency Cepstral Coefficients (MFCC), a magnitude spectrum and/or logarithmic magnitude spectrum, spectral centroids, time samples, and noise estimates.

In those implementations that include classification of input signals, the filter calculator <NUM> refers to one or more pre-defined sound models <NUM>. Such sound models <NUM> embody the result of pre-computation used to characterize a given input audio signal as belonging to one or more classifications, such as a siren, a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, or a spoken utterance.

In some cases, an occupant of the vehicle <NUM> becomes interested in all audio signals <NUM>, <NUM> from outside the cabin <NUM>. In such cases, the occupant activates a toggle control that re-broadcasts whatever the external microphones <NUM> are receiving directly into the cabin <NUM>. This signal, which remains unfiltered, will be referred as the "raw audio signal.

To enhance the occupant's experience, the filtering system <NUM> carries out a spatialization procedure, which is discussed in connection with <FIG> and <FIG>, before re-broadcasting the raw audio signals into the cabin <NUM>. As a result of this procedure,.

In some implementations, activating the toggle control also engages a "conversation mode" that activates the trans-cabin communication system <NUM>. This permits an occupant of the vehicle <NUM> to converse with a person outside the vehicle <NUM> and to do so without having to roll down a window or open a door. This may be desirable, for example, in inclement weather or in an effort to enhance security.

Yet other techniques for identifying audio signals of interest are possible. Examples of such techniques are described in <CIT>, <CIT>, <CIT>, and <CIT>.

<FIG> shows hardware for carrying a filtering method as shown in <FIG>.

The illustrated hardware includes an A/D converter <NUM> that receives an analog signal from an external microphone <NUM>. The resulting digital signal is provided to a filter calculator <NUM> and to a filter <NUM> that receives calculated results from the filter calculator <NUM>. The filter <NUM> filters the signal and provides the filtered signal to a D/A converter <NUM> for communication to occupants of the cabin via an internal loudspeaker <NUM> or using the display at the user interface <NUM>.

The method <NUM> begins with receiving a raw audio signal (step <NUM>) and carrying out analog-to-digital conversion (step <NUM>).

Based on pre-selected filter characteristics, the filter calculator <NUM> computes the effect of the filter on the raw audio signal (step <NUM>) and provides it to the filter <NUM>. The filter <NUM> uses the result to filter the raw audio signal (step <NUM>), thereby producing a filtered audio signal.

The D/A converter <NUM> converts the filtered audio signal to an analog signal (step <NUM>) and provides it for broadcast into the cabin <NUM> via an internal loudspeaker <NUM> (step <NUM>), and optionally via the user interface <NUM>.

In some implementations, the foregoing procedure results in a visual alert. One example of a visual alert is a message on the user interface <NUM> vehicle's infotainment screen. Another example of a visual alert is activating one or more cabin lights.

In other implementations, the broadcasting step <NUM> includes applying a gain factor to the filtered signal at the internal loudspeaker <NUM> to increase the likelihood of drawing the occupant's attention, thus reducing the risk of missing the alert altogether. Among these implementations are those that include determining the gain factor based on ambient acoustic conditions within the cabin <NUM>.

Examples of conditions used to determine the gain factor include the properties of road noise and/or engine noise, including amplitude and/or frequency, whether measured directly or determined from a model based on the engine's state), the volume level of the vehicle's stereo, and/or an ambient cabin noise level from other sources, such as passenger conversation or audio playback on an external device, both of which are measurable by an internal microphone <NUM>.

In still other implementations, the broadcasting step <NUM> includes reducing or otherwise adjusting the volumes of one or more other audio sources to enhance the driver's perception of the broadcasted audio. Examples of such audio sources include an in-vehicle entertainment system and telephone audio played through the vehicle's speakers.

<FIG> shows hardware for carrying out a more refined filtering method shown in <FIG>.

The illustrated hardware includes an A/D converter <NUM> that receives an analog signal from an external microphone <NUM>. The resulting digital signal is provided to a filter calculator <NUM>, to a filter <NUM> that receives calculated results from the filter calculator <NUM>, and to a sound classifier <NUM> that classifies the sound and provides the resulting class to the filter calculator <NUM>. The filter <NUM> filters the signal and provides the filtered signal to a D/A converter <NUM> for communication to occupants of the cabin via an internal loudspeaker <NUM> or using the display at the user interface <NUM>.

The sound classifier <NUM> then classifies relevant aspects of the raw audio signal during a classifying step <NUM>. In some implementations, the raw audio signal is classified as containing a signal-of-interest. Examples of signals-of-interest include a siren, a back-up alarm, a vehicle horn, a proximity warning from a vehicle, a bicycle bell, a dog's bark, a crying child, a shouting voice, and a spoken utterance.

Based on a pre-selected filter, the filter calculator <NUM> carries out a computation step <NUM> in which it computes the effect of the filter on the raw audio signal. It then provides the result to the filter <NUM>, which then filters the raw audio signal during a filtering step <NUM>. In some embodiments, the filter calculator <NUM> uses received vehicle-state information to either select which filter characteristics to use and/or to determine certain parameters or coefficients in the selected filter.

In some implementations, the filtering step <NUM> includes an extraction step in which the filter <NUM> extracts the signal-of-interest from the raw audio signal.

In other implementations, the filter <NUM> replaces the raw audio signal, or a portion thereof, with a pre-recorded representative signal that corresponds to the signal-of-interest classified by the sound classifier <NUM> during the classifying step <NUM>. For example, if the raw audio signal included an ambulance siren, the filter <NUM> would replace the raw audio signal with a pre-recorded sample of a siren during the filtering step <NUM>.

The D/A converter <NUM> then converts the filtered audio signal into an analog signal (step <NUM>) and provides it for broadcast into the cabin <NUM> (step <NUM>). In some implementations, this includes providing a visual alert, such as displaying a message on the user interface <NUM>, activating one or more cabin lights, or displaying information indicative of the classification carried out by the sound classifier <NUM> during the classifying step <NUM>. For example, if the sound classifier <NUM> classified the raw audio signal as a siren, then the visual alert would reference a siren.

<FIG> shows hardware for carrying out the filtering method shown in <FIG>.

The illustrated hardware includes first and second A/D converters 502a, 502b, each of which connects to a corresponding external microphone <NUM>. The resulting first and second digital signals are provided to a direction finder <NUM>, which uses them to estimate a direction-of-arrival. The resulting estimate is provided to spatializing circuitry <NUM>, which determines weights to apply to first and second output signals that are provided to corresponding first and second D/A converters 514a, 514b.

The outputs of the first and second A/D converters 502a, 502b are also provided to a signal fuser <NUM>, which outputs a combined signal to both a filter calculator <NUM> and to a filter <NUM>.

The method <NUM> begins with receiving raw audio signals (step <NUM>) from corresponding external microphones <NUM> followed by A/D conversion at the first and second A/D converters 502a, 502b.

During an estimation step <NUM>, the direction finder uses the two spatial samples provided by the two external microphones <NUM> to estimate a direction-of-arrival. In some embodiments, the direction finder <NUM> relies on a time lag or a phase difference between the signals provided by the external microphones <NUM>, thereby computing a time-of-flight required for sound waves giving rise to corresponding features to travel from one microphone to another. This can then be used to direction-of-arrival via triangulation. Other methods for acoustic source localization or estimating direction-of-arrival based on signals from two or more different microphones are known in the art. Also among such methods are those that rely on outputs from various types of sensors, such as audio signals from the external microphones <NUM> and video signals from the camera <NUM> or from plural cameras.

Since the external microphones <NUM> are in different locations, one cannot simply add their signals together. Instead, during a fusing step <NUM>, the fuser <NUM> fuses the raw audio signals into a single signal in a manner that avoids the interference that would result from simple addition. In some implementations, the fuser <NUM> adds time-shifted copies of input signals together, where the time shift is given by the time lag between corresponding features of each audio signal. Other techniques are known in the art for fusing the raw audio signals into a single signal. Examples include automatic microphone mixing and/or microphone diversity combining.

In some embodiments, the downstream hardware components have dual inputs, in which case there is no need to fuse the signals. Instead, the various signal processing steps are simply carried out separately for each external-microphone signal.

During a filter-calculation step <NUM>, the filter calculator <NUM> uses pre-selected filter characteristics to compute the effect of the filter on the raw audio signal. The result is then provided to the filter <NUM>, which proceeds to filter the raw audio signal during a filtering step <NUM>.

In some implementations, the filter calculator <NUM> extracts a signal-of-interest from the raw audio signal(s), as described above.

In other implementations, the filter calculator <NUM> replaces the raw audio signal(s), or corresponding portions thereof, with a pre-recorded representative signal that corresponds to the nature of the signal-of-interest as provided by, for example, a classifier <NUM>. For example, if the raw audio signal were found to comprise an ambulance siren, the filter calculator <NUM> would replace the raw audio signal with a pre-recorded sample of a siren. This is particularly useful to avoid confusion as a result of different jurisdictions having different types of sirens.

The filter <NUM> provides the filtered audio signal to spatializing circuitry <NUM> during a spatializing step <NUM>. During the spatializing step <NUM>, the spatializing circuitry <NUM> determines certain acoustic parameters with which to weight signals provided to the first and second D/A converters 514a, 514b. Examples of such weights are those that cause differences in volume and those that cause difference in timing. The selection of weights, together with information concerning an occupant's position and orientation, causes the sound perceived by the occupant to match the direction-of-arrival estimated by the direction finder <NUM>.

The resulting filtered audio signals, suitably weighted, are converted into corresponding analog signals by the D/A converters 514a, 514b and broadcast into the cabin <NUM> during a broadcasting step <NUM>.

In some implementations, the broadcasting step <NUM> includes providing a visual alert, such as displaying a message on the user interface <NUM> or activating one or more cabin lights.

In other implementations, the visual alert incorporates the estimate of direction-of-arrival provided by the direction finder <NUM>. For example, if the raw audio signals were determined to have arrived from the left side of the vehicle, the visual alert would then reference the left side of the vehicle.

Claim 1:
An apparatus comprising
a vehicle (<NUM>) configured for travel along a road, the vehicle comprising a cabin (<NUM>) in which at least one occupant rides and an inbound audio channel (<NUM>), the inbound audio channel comprising an external-microphone set that includes one or more microphones (<NUM>) disposed to receive a raw audio signal from outside the cabin and an internal-loudspeaker set that comprises one or more loudspeakers (<NUM>) disposed to radiate sound into the cabin in response to the external-microphone set having received the raw audio signal, and
a signal processor (<NUM>) that is configured to apply a filter to the raw audio signal to generate a filtered audio signal that is then broadcast into the cabin using the internal-loudspeaker set,
wherein the signal processor is further configured to identify, in the raw audio signal, a siren, the apparatus being characterised in that
the filter is configured to distinguish a stationary signal from a transitory signal, wherein the stationary signal is one whose spectrum is approximately constant over time and the transitory signal is a signal that is not a stationary signal,
wherein the signal processor is further configured to transition between broadcasting the raw audio signal into the cabin and broadcasting the filtered signal into the cabin.