Patent Description:
As a technique for supporting communication based on speech between users in different areas in an automobile, a technique for collecting speech voices of users in a first area of the automobile with a microphone in the first area, and outputting, from a speaker in a second area, the speech voices of which a gain has been adjusted so that the users in the second area can clearly hear the speech voice is known (for example, <CIT>).

Furthermore, as a technique for supporting communication based on speech in a vehicle, as illustrated in <FIG>, in a system for supporting a conversation between a user in a first area and a user in the second area by outputting, from a speaker <NUM> in a second area, a voice of the user picked up by a microphone <NUM> in the first area and outputting, from a speaker <NUM> in the first area, a voice of the user picked up by a microphone <NUM> in the second area, there is also known an echo cancellation technique in which an echo canceller <NUM> cancels out an echo introduced from the speaker <NUM> in the second area and routed to the microphone <NUM> in the second area (for example, <CIT>).

The echo canceller <NUM> includes an adaptive filter <NUM> that receives the output of the microphone <NUM> in the first area as an input, and an adder <NUM> that adds the output of the adaptive filter <NUM> and the output of the microphone <NUM> in the second area and outputs the result to the speaker <NUM> in the first area, and cancels an echo by performing an adaptive operation using the output of the adder <NUM> as an error in the adaptive filter <NUM>.

In addition, there is also known a howling canceller technique for canceling out howling using an adaptive filter (for example, <CIT>).

<FIG> illustrates a configuration in which such a howling canceller <NUM> is applied to suppress occurrence of howling due to sound introduced from the speaker <NUM> in the first area and routed to the microphone <NUM> in the second area.

As illustrated in the drawing, the howling canceller <NUM> includes an adaptive filter <NUM> and an adder <NUM> that adds the output of the adaptive filter <NUM> and the output of microphone <NUM> in the second area and outputs the result to the speaker <NUM> in the first area. The output of the adder <NUM> is used as an input of the adaptive filter <NUM>, and the adaptive filter <NUM> performs an adaptive operation using the output of the adder <NUM> as an error, thereby suppressing occurrence of howling due to sound introduced from the speaker <NUM> in the first area and routed to the microphone <NUM> in the second area.

There is also known a road noise canceller technique for canceling out road noise of an automobile using an adaptive filter (for example, <CIT>).

<FIG> illustrates a configuration in which such a road noise canceller <NUM> is applied to cancel road noise audible to the user in the first area.

As illustrated in the drawing, the road noise canceller <NUM> includes an adaptive filter <NUM>, an adder <NUM> that adds the output of the adaptive filter <NUM> and the output of the microphone <NUM> in the second area and outputs the result to the speaker <NUM> in the first area, and a reference signal generation unit <NUM>.

The reference signal generation unit <NUM> generates and outputs a reference signal simulating node noise from an output of a sensor <NUM> that detects a signal correlated with road noise of an acceleration sensor or the like. Then, the reference signal output from the reference signal generation unit <NUM> is used as an input to the adaptive filter <NUM>, and the adaptive filter <NUM> performs an adaptation operation with the output of the microphone <NUM> in the second area as an error, thereby suppressing road noise audible to the user in one area.

In a case where communication based on speech between users in different areas inside the automobile is supported by collecting a speech voice of a user in a first area of the automobile with a microphone in the first area and outputting the collected speech voice from a speaker in a second area, if road noise becomes large due to highspeed traveling, the speech voice is buried in the road noise, and good support cannot be performed.

In addition, as illustrated in <FIG>, in a case where an echo canceller or a howling canceller is provided to support mutual conversation between the user in the first area and the user in the second area, if the road noise increases, the adaptive operation of the adaptive filter diverges, and appropriate support may not be performed.

<CIT> discloses a digital voice enhancement communication system (DVE) using microphones to detect speech signals in an acoustic enclosure, for example an automotive cabin and other enclosures. The microphones in conjunction with the loudspeakers located in the acoustic space create electroacoustic transfer functions. Particularly, a communication system includes a first acoustic zone, a second acoustic zone, a first microphone at the first zone, a first loudspeaker at the first zone, a second microphone at the second zone and having an output supplied to first loudspeaker such that a first person at first zone can hear the speech of a second person at second zone as transmitted by second microphone and first loudspeaker, and a second loudspeaker at second zone and having an input supplied from first microphone such that the second person at the second zone can hear the speech of the first person at the first zone as transmitted by the first microphone and second loudspeaker. Each of the zones is subject to noise. A noise responsive high pass filter between the first microphone and second loudspeaker has a filter cutoff effective at elevated noise levels and reducing bandwidth and making more gain available, to improve intelligibility of speech of the first person transmitted from the first microphone to second loudspeaker <NUM>.

Therefore, an object of the present invention is to satisfactorily support with a communication support system communication by speech between users in different areas even when a large road noise is being generated.

The invention relates to a communication support system according to the appended independent claim <NUM>.

According to the communication support system as described herein, during a period in which a large road noise is being generated, a low frequency band in which road noise is concentrated is excluded, and only a treble band that is a frequency band including a high sound band in which sound easily passes is relayed to a speaker for a user in another area from a voice of the user picked up by a microphone. Therefore, even when a large road noise is being generated, communication based on a speech between users can be supported relatively well.

According to the present invention, even when a large road noise is being generated, it is possible to satisfactorily support communication by speech between users in different areas.

Hereinafter, embodiments of the present invention will be described by taking an application to an in-vehicle system that supports communication by speech in a vehicle between a user in a front seat and a user in a rear seat of an automobile as an example.

<FIG> illustrates a configuration of an in-vehicle system according to an embodiment.

The in-vehicle system is a system mounted on an automobile, and includes according to an embodiment, as illustrated in the drawing, a signal processing processor <NUM> to which the following units are connected: a front seat microphone <NUM> which is a microphone for a user in a front seat area in a vehicle interior, a front seat speaker <NUM> which is a speaker for a user in the front seat area, a rear seat microphone <NUM> which is a microphone for a user in a rear seat area in the vehicle interior, and a rear seat speaker <NUM> which is a speaker for a user in the rear seat area. In addition, the signal processing processor <NUM> is connected to an external system <NUM> that detects and manages various states of the automobile.

Here, the front seat microphone <NUM> and the rear seat microphone <NUM> output audio data having a predetermined sampling frequency representing picked up sound. In addition, the front seat speaker <NUM> and the rear seat speaker <NUM> emit voice represented by input audio data having a predetermined sampling frequency.

The signal processing processor <NUM> outputs the voice of the user in the rear seat area picked up by the rear seat microphone <NUM> in the rear seat area to the front seat speaker <NUM> in the front seat area, and outputs the voice of the user in the front seat area picked up by the front seat microphone <NUM> in the front seat area to rear seat speaker <NUM> in the rear seat area, thereby supporting communication by conversation between the user in the rear seat area and the user in the front seat area.

Here, as illustrated in <FIG>, the rear seat area is, for example, an area of a seat behind a driver's seat of an automobile, and the rear seat speaker <NUM> and the rear seat microphone <NUM> are disposed in the rear seat area. In addition, the front seat area is an area of a driver's seat of an automobile, and front seat speaker <NUM> and front seat microphone <NUM> are disposed in the front seat area.

Next, <FIG> illustrates a functional configuration of a signal processing processor <NUM> according to an embodiment.

Here, each functional unit is implemented by software processing, and each functional unit shares the same resources as those of the signal processing processor <NUM>.

As illustrated in the drawing, the signal processing processor <NUM> includes, as functional units, a control unit <NUM>, a front seat voice processing unit <NUM> that processes a voice picked up by the front seat microphone <NUM> and output to the rear seat speaker <NUM>, and the rear seat voice processing unit <NUM> that processes a voice picked up by the rear seat microphone <NUM> and output to the front seat speaker <NUM>.

The front seat voice processing unit <NUM> and the rear seat voice processing unit <NUM> have the same configuration, and each include an IIR filter <NUM>, a downsampling unit <NUM>, an echo canceller <NUM>, a howling canceller <NUM>, and an upsampling unit <NUM>.

Operations of the front seat voice processing unit <NUM> and the rear seat voice processing unit <NUM> will be described below.

An operation of each unit of the front seat voice processing unit <NUM> will be described first.

The IIR filter <NUM> is a frequency filter that receives audio data output from the front seat microphone <NUM> as an input, and an output of the IIR filter <NUM> is output to the downsampling unit <NUM>. The downsampling unit <NUM> down-samples the audio data input from the IIR filter <NUM> and converts the sampling frequency into a lower sampling frequency.

The echo canceller <NUM> cancels a component correlated with audio data output from the downsampling unit <NUM> of the rear seat voice processing unit <NUM> from the audio data output from the downsampling unit <NUM>, thereby canceling out and outputting an echo introduced from the front seat speaker <NUM> and routed to the front seat microphone <NUM>.

Here, as the configuration of the echo canceller <NUM>, for example, the configuration of the echo canceller <NUM> illustrated in <FIG> can be used.

The howling canceller <NUM> cancels a component correlated with the audio data already output from the howling canceller <NUM> to remove a voice component introduced from the rear seat speaker <NUM> and routed to the front seat microphone <NUM> from the audio data output from the downsampling unit <NUM>, thereby suppressing occurrence of howling.

Here, as the configuration of howling canceller <NUM>, for example, the configuration of howling canceller <NUM> shown in <FIG> can be used.

The upsampling unit <NUM> then upsamples the audio data output from the howling canceller <NUM> by interpolation, generates audio data having a predetermined sampling frequency for output from the speakers, and outputs the audio data to the rear seat speaker <NUM>.

Here, the filter characteristics of the IIR filter <NUM>, the sampling frequency of the audio data downsampled by the downsampling unit <NUM>, and the characteristics of the upsampling performed by the upsampling unit <NUM> can be controlled and changed by the control unit <NUM>.

Next, the operations of the respective units of the rear seat voice processing unit <NUM> are described by replacing "front seat" and "rear seat" in the above description of the operations of the respective units of the front seat voice processing unit <NUM>.

Hereinafter, the operation of the control unit <NUM> will be described.

<FIG> illustrates a procedure of the assist characteristic switching processing performed by the control unit <NUM> according to an embodiment. Here, in the following description, it is assumed that sampling frequencies of audio data output from front seat microphone <NUM> and the rear seat microphone <NUM> and sampling frequencies of input audio data input from the front seat speaker <NUM> and the rear seat speaker <NUM> are both <NUM>.

As illustrated in the drawing, the control unit <NUM> checks whether the road noise is currently large in the assist characteristic switching processing (step <NUM>).

Here, in step <NUM>, in a case where a level of components of <NUM> or less having a large correlation with each other, included in outputs of the front seat microphone <NUM> and the rear seat microphone <NUM>, is larger than a predetermined threshold, it is detected or determined that the road noise is currently large.

Alternatively, in step <NUM>, information on the automobile speed and the rotation speed of the automobile is acquired from the external system <NUM>, and when the acquired information indicates that the vehicle is continuously traveling at a high speed of a predetermined speed or more for a predetermined time or more, it is detected or determined that the road noise is currently large.

Then, when it is determined that the road noise is not currently large (step <NUM>), the assist band is controlled to be the standard band (step <NUM>).

Here, the assist band indicates a frequency band of the output of the front seat microphone <NUM> that the signal processing processor <NUM> targets for relaying to the rear seat speaker <NUM> and indicates the frequency band of the output of the rear seat microphone <NUM> that the signal processing processor <NUM> targets for relaying to the front seat speaker <NUM>.

The standard band is, for example, a band of <NUM> or less.

In a case where the standard band is the band of <NUM> or less, in step <NUM>, in order to set the assist band to the band of <NUM> or less, the filter characteristic of the IIR filter <NUM> is set to the filter characteristic for cutting off the component of the frequency band exceeding <NUM>, and the sampling frequency after the downsampling of the downsampling unit <NUM> is set to <NUM>.

Here, by setting the filter characteristic of the IIR filter <NUM> to cut off a low-frequency component exceeding <NUM>/<NUM> of the sampling frequency after the downsampling, anti-aliasing is performed to prevent appearance of the folded noise in the audio data after the downsampling. Furthermore, the sampling frequency after the downsampling of the downsampling unit <NUM> is set to <NUM> in order to remove an unnecessary frequency band exceeding <NUM> and to reduce the processing amount after the sample rate of the audio data is reduced.

In step <NUM>, echo canceller <NUM> and howling canceller <NUM> are set to process the audio data having the sampling frequency of <NUM>, and upsampling unit <NUM> is set to upsample the audio data having the sampling frequency of <NUM> into the audio data having the sampling frequency of <NUM>. In addition, the upsampling unit <NUM> is set to perform upsampling so that the audio data to be output does not include a component exceeding <NUM>.

Then, it is repeatedly checked whether the road noise is currently large until it is determined that the road noise is large (step <NUM>), and when it is determined that the road noise is large, the processing proceeds to step <NUM>.

On the other hand, in a case where it is determined in step <NUM> that the road noise is currently large, the processing also proceeds to step <NUM>.

When the processing proceeds from step <NUM> or step <NUM> to step <NUM>, the operations of the howling canceller <NUM> in the front seat voice processing unit <NUM> and the rear seat voice processing unit <NUM> are invalidated.

Here, the howling canceller <NUM> whose operation is invalidated performs a through operation of outputting input audio data as it is.

Then, next, control is performed to set the assist band to the treble band (step <NUM>).

The high sound band does not include the low frequency band included in the standard band but includes a band having a higher frequency than the standard band. For example, the treble band is a band from <NUM> to <NUM>.

In a case where the treble band is a band from <NUM> to <NUM>, in step <NUM>, in order to set the assist band from <NUM> to <NUM>, the filter characteristic of the IIR filter <NUM> is set to a filter characteristic for cutting off a component in a frequency band other than the frequency band from <NUM> to <NUM>, and the sampling frequency after the downsampling of the downsampling unit <NUM> is set to <NUM>.

In step <NUM>, echo canceller <NUM> and howling canceller <NUM> are set to process the audio data having the sampling frequency of <NUM>, and upsampling unit <NUM> is set to upsample the audio data having the sampling frequency of <NUM> into the audio data having the sampling frequency of <NUM>. In addition, the upsampling unit <NUM> performs setting so as to perform upsampling so that the audio data to be output does not include a component of a frequency band other than the frequency band of <NUM> to <NUM>.

Then, it repeatedly checks whether the road noise is currently large until it is determined that the road noise is not large (step <NUM>), if it is determined that the road noise is not large, invalidation of operation of the howling canceller <NUM> of the front seat voice processing unit <NUM> and the rear seat voice processing unit <NUM> is canceled (step <NUM>), and the processing proceeds from step <NUM>. Here, the howling canceller <NUM> whose invalidation of the operation has been released restarts the operation for suppressing howling described above.

The assist characteristic switching processing performed by the control unit <NUM> has been described above.

According to such an assist characteristic switching processing, the operations of the front seat voice processing unit <NUM> and the rear seat voice processing unit <NUM> are performed with the assist band as the standard band (for example, a band of <NUM> or less) when the road noise is not large, and with the assist band as the treble band (for example, a band from <NUM> to <NUM>) not including the low frequency band included in the standard band but including the band having a higher frequency than the standard band when the road noise is large.

Here, when the road noise that hinders the collection and listening of the speech voice is not large, even if the assist band is set to a low frequency band, it is possible to sufficiently support the communication based on the speech between the user in the front seat and the user in the rear seat of the automobile.

Further, when the road noise is not large, the echo canceller <NUM>, the howling canceller <NUM>, and the like are operated for the audio data having a low sampling frequency, so that the processing road of the signal processing processor <NUM> can be suppressed to be small.

On the other hand, even when the road noise is large, the high-frequency voice can be picked up by the front seat microphone <NUM> and the rear seat microphone <NUM> relatively well and can be heard by the user relatively well.

In addition, since the road noise is concentrated in a low frequency band lower than <NUM>, this low frequency band is excluded from the processing target, and the processing is performed only for a high frequency band, whereby the occurrence of divergence of the adaptive operation of the adaptive filter of the echo canceller <NUM> or the howling canceller <NUM> is suppressed.

Therefore, according to the present embodiment in which the assist band is the treble band (for example, a band from <NUM> to <NUM>) not including the low frequency band included in the standard band but including the band having a higher frequency than the standard band when the road noise is large, it is possible to support communication by speech between the user in the front seat and the user in the rear seat of the automobile as well as possible even when the road noise is large.

In addition, in the present embodiment, when the road noise is large, the echo canceller <NUM> and the like are operated for audio data having a high sampling frequency. Therefore, the processing road of the signal processing processor <NUM> increases accordingly, but instead, the operation of the howling canceller <NUM> is invalidated, so that an increase in the processing road of the signal processing processor <NUM> can be suppressed.

Here, when the road noise is large, the S/N of the path in which the howling sound loops due to disturbance becomes small, and howling hardly occurs. In addition, since the speaker usually has high directivity in a high frequency band, howling hardly occurs in the high frequency band.

Accordingly, invalidating the operation of the howling canceller <NUM> causes no significant problem when a treble band is set as a frequency band to be relayed by front seat voice processing unit <NUM> and the rear seat voice processing unit <NUM> due to large road noise.

Here, a road noise canceller that cancels road noise may be further provided as a functional unit of the signal processing processor <NUM> in the above embodiment.

In other words, in this case, for example, as illustrated in <FIG>, a road noise canceller <NUM> that generates a cancellation sound so that the component correlated with the road noise contained in the output of the front seat microphone <NUM> is minimized using the output of the sensor <NUM> that detects a signal correlated with road noise, such as an acceleration sensor, and the output of the front seat microphone <NUM> and cancels road noise in the front seats by adding cancellation sound to the audio data output from the rear seat voice processing unit <NUM> and outputting the audio data to the front seat speaker <NUM> and a road noise canceller <NUM> that generates a cancellation sound so that the component correlated with the road noise contained in the output of the rear seat microphone <NUM> is minimized using the output of the sensor <NUM> and the output of the rear seat microphone <NUM> and cancels road noise in the rear seats by adding the cancellation sound to the audio data output from the front seat voice processing unit <NUM> and outputting the audio data to the rear seat speakers <NUM> are provided.

Here, as a configuration of the road noise canceller <NUM>/<NUM>, for example, a configuration of road noise canceller <NUM> illustrated in <FIG> can be used.

Furthermore, in the above embodiment, the application to the support of communication by speech between the front seat and the rear seat has been described as an example, but the above embodiment can be similarly applied to a case of supporting communication by speech between seats in a combination of arbitrary seats other than the front seat and the rear seat.

Claim 1:
A communication support system that is configured to support communication by speech between a user in a first area and a user in a second area in an automobile, the communication support system comprising:
a first area microphone that is a microphone disposed in the first area;
a second area speaker that is a speaker disposed in the second area;
a road noise detection unit that is configured to determine whether a large road noise is being generated;
a control unit (<NUM>); and
a voice processing unit that is configured to relay a voice picked up by the first area microphone to the second area speaker,
wherein the control unit (<NUM>) is configured to cause the voice processing unit to extract a component of a standard band that is a preset frequency band of the voice picked up by the first area microphone and to relay the extracted component to the second area speaker, during a period in which the road noise detection unit does not determine that a large road noise is being generated, and is configured to cause the voice processing unit to extract a component of a treble band, which is a preset frequency band that does not include at least a band on a low frequency side of the standard band but includes a band having a higher frequency than the standard band of the voice picked up by the first area microphone and to relay the extracted component to the second area speaker, during a period in which the road noise detection unit determines that a large road noise is being generated.