Proceedings, APSIPA Annual Summit and Conference 2020

7-10 December 2020, Auckland, New Zealand

Applying Virtual Microphones to Triangular
Microphone Array in in-Car Communication
Hanako Segawa, Riki Takahashi, Ryoga Jinzai, Shoji Makino, Takeshi Yamada
University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8577 Japan

Abstract—We have evaluated the application of the
wavelength–proportional arrangement of virtual microphones(WPVM) to a triangular microphone array in a real car environment. Beamforming is effective for speech enhancement
in in-car communication. However, it is difﬁcult to install a
sufﬁcient number of microphones for beamforming in a car.
In this paper, we propose a virtual microphone technique that
increases the number of microphones virtually to achieve highspeech enhancement performance with a small number of microphones in a car. We applied the virtual microphone techniqueto a triangular microphone array in a car and evaluated theadaptive beamforming performance. As a result, the WPVM wasshown to be effective for improving the performance of speechenhancement in a car.

I. I NTRODUCTION
Information and communication technology (ICT) is developing rapidly. Recently, several studies of the application of
ICT to in-car communication (ICC) have been carried out [1][2]. Owing to the seat arrangement and background noise,communication within a car is difﬁcult. In particular, backseat
passengers may feel uncomfortable listening to the speechfrom front seat passengers. The situation can be improved byenhancing particular parts of speech by a loudspeaker locatednear the back seats.
Beamforming is one of the methods used to enhance target
speech. In particular, beamforming speech enhancement iseffective because the positions of people, or the directionsof speech arrival, are easily predictable in a car. However,beamformers do not work properly when there are fewer microphones than sources, i.e., underdetermined conditions. The
performance of beamforming degrades in underdeterminedsituations. In ICC, the performance degrades because of theinability to cope with an increase in the number of speakers
and background noise.
A simple solution to this problem is to increase the numberof microphones. However, in a car, this means that the microphone array will be larger and more expensive. Considering
the increased costs and the balance with the design and otherequipment in a car, this solution is not practical.
In this situation, the virtual microphone technique is one of
the effective methods. A virtual microphone signal is generated
from the observed signals of two real microphones, and avirtual microphone is deployed on the line connecting the tworeal microphones. By using the virtual signal, we can increase
the number of channels of the beamformer. In experimentswith two microphones, it has been shown that the performance

978-988-14768-8-3/20/$31.00 ©2020 APSIPA

Fig. 1. Arrangement of real and virtual microphones in interpolation techniqueof speech enhancement with a virtual microphone is better
than that without the virtual microphone [3]. The advantageof the virtual microphone technique is that it is easy to installbecause it has a low cost and requires no space.
In this paper, we consider the virtual microphone techniquein ICC. Speciﬁcally, we generate virtual signals from thesignals obtained by a triangular microphone array in a car
and evaluate the adaptive beamforming performance. In an
experiment, we use the impulse response measured in a car.
We examine the effectiveness of the wavelength–proportionalarrangement of virtual microphones (WPVM) for speech enhancement in ICC.
II. I NCREASING NUMBER OF CHANNELS BY VIRTUALMICROPHONE TECHNIQUE

A. Interpolation of Virtual Microphone Signals
In this section, we introduce interpolation by the virtual
microphone technique [4] [5]. In the virtual microphone
technique, a virtual microphone signal v(ω, t, α) is generated
from observed signals of two real microphones x1 , x2 in thetime-frequency domain, where xi (ω, t) is the ith microphonesignal (i = 1, 2) at the angular frequency ω in the tth timeframe. α is the coefﬁcient of the interpolation of the virtual
microphone; a virtual microphone is located at the point
obtained by internally dividing the line joining the two real
microphones in the ratio α to (1 − α). The arrangement ofreal and virtual microphones is shown in Fig. 1.
In a situation where there are multiple sounds arriving fromdifferent directions, the relationship between the microphone
position and the waveform is complicated and is difﬁcult tointerpolate. Therefore, in this method, by assuming the Wdisjoint orthogonality (W-DO) [6] of the observed signals, we
simplify the modeling of the relationship. W-DO implies thestrong sparsity of the signal in the time-frequency domain,
assuming that the component from a sound source dominates

421

APSIPA-ASC 2020

Proceedings, APSIPA Annual Summit and Conference 2020

7-10 December 2020, Auckland, New Zealand

B. Extrapolation of Virtual Microphone Signals

Fig. 2. Arrangement of real and virtual microphones in extrapolation technique

one time–frequency slot of a discrete short–time Fouriertransform (STFT).
By assuming this, when multiple sounds arrive, they canbe regarded as a single sound in each time–frequency slot.
Thus, we can interpolate the virtual microphone signal. In thevirtual microphone, the phase and amplitude are interpolated
individually. The phase and amplitude of the observed signal
xi (ω, t) are respectively denoted asϕi=∠xi (ω, t) = tan−1

Ai=|xi (ω, t)|.

Im(xi (ω, t)),Re(xi (ω, t))

(1)
(2)

The phase ϕi of the virtual microphone signal is interpolated
linearly as follows.
ϕv= ϕ1 + α(ϕ2 − ϕ1 )= (1 − α)ϕ1 + αϕ2 .

(3)The values of the phase are ϕi ±2nπ for an arbitrary natural
number n. Therefore, the phase of the virtual microphone isinterpolated with the assumption that|ϕ1 − ϕ2 | ≤ π.

(4)

The appropriate interpolation of the amplitude of the virtual microphone depends on reverberation and the distance
between the source and the microphones in addition to thedirection of arrival. Thus, it is difﬁcult to faithfully model theactual amplitude attenuation. However, The signal amplitude
must also be interpolated in accordance with an appropriate
rule. Consequently, this method uses β-divergence for amplitude interpolation instead of some physical model [4] [5]. βdivergence is used distance measure for nonnegative values
such as amplitude. The amplitude of the virtual microphone
is interpolated asexp ((1 − α) log A1 + α log A2 ) (β = 1)
) 1
Av = ((5)
 (1 − α)Aβ−1 + αAβ−1 β−1(otherwise).
1
2
Using the parameter β, it is possible to nonlinearly interpolate
the amplitude of the virtual microphone using the amplitudesof the two real microphones.
From the above, the virtual microphone signal v(ω, t) isrepresented asv(ω, t, α) = Av exp (jϕv ).

(6)

Next in this section, we introduce extrapolation by thevirtual microphone technique [7]. The arrangement of real andvirtual microphones is shown in Fig. 2. For the extrapolationof a virtual microphone signal, we have to check the validityof the generation method for a virtual microphone describedin the previous section. For phase extrapolation, we can use
the same equation as in the previous phase interpolation, but
the amplitude interpolation is difﬁcult. Extrapolation based on
β-divergence sometimes outputs unrealistic amplitudes suchas a negative amplitude or diverges to positive inﬁnity. Such
impossible extrapolation must be avoided. Therefore, in thismethod, since interaural time difference (ITD) is dominant in
the frequency range below 1.5 kHz [8] [9], we use the amplitude of the real microphone closest to the virtual microphoneas the amplitude of the extrapolated virtual microphone.
{
A1 (α < 0)Av =(7)
A2 (α > 1).
From the above, the extrapolated virtual microphone signalv(ω, t, α) is represented in the same way as in the interpolation.
v(ω, t, α) = Av exp (jϕv ).
(8)
C. Wavelength–Proportional Arrangement of Virtual Microphones (WPVM)
In this section, we introduce the wavelength–proportionalarrangement of virtual microphones (WPVM) [3]. In thismethod, the coefﬁcient of the position of the virtual microphone α is denoted as
2πck
λk
=
.
(9)
d
ωd
where λ is the wavelength, k is the scaling of the interval
between the reference microphone and the virtual microphone
relative to wavelength, d is the distance between the real
microphones, and c is the velocity of sound. This equation implies that the virtual microphone is placed at a position k times
the wavelength corresponding to the processing frequency
from x1 ; therefore, the total length of the microphone arrayincluding the virtual microphone is larger at lower frequencies
and smaller at higher frequencies. By setting an appropriate
parameter k, spatial aliasing does not occur and a sufﬁcientphase difference between microphones can be obtained at all
frequencies. In theory, a sufﬁcient phase difference is not
obtained at k = 0.25 and spatial aliasing may occur at k
= 2. A sufﬁcient phase difference is obtained at k = 0.5
and good performance is expected. By this method, it was
shown that speech enhancement performance increased in a
two-microphone experiment in an underdetermined situation.
According to research on WPVM, an appropriate value of k
is to be 0.5 or 1 [3].
α(ω) =

D. Triangular Beamformers in a Car
Finally, we introduce triangular beamformers in a car used
for speech enhancement in the experiment. In this experiment,422

Proceedings, APSIPA Annual Summit and Conference 2020

7-10 December 2020, Auckland, New ZealandFig. 3. Sound source and microphone layout in experimentFig. 5. Actual triangular microphone array
Fig. 4. Arrangement of triangular microphone arrayto evaluate the performance of the virtual microphone technique, we apply an increasing number of channels by using thevirtual microphone technique to a maximum SNR beamformer[10] [11]. The maximum SNR beamformer requires prior information on the covariance matrices of the target-only period
and interference-only period. From the prior information of
the target and interference, the maximum SNR beamformer
constructs a spatial ﬁlter so that the power ratio of the target
to the interference becomes maximum. In principle, the virtual
microphone can be similarly applied to other microphone arraysignal processing techniques as well as the maximum SNR
beamformer.
As a comparison, we use a minimum variance distortionlessresponse (MVDR) beamformer without virtual microphones
for speech enhancement [10] [12]. When there is no correlation between the target and the interference sound, since thevariance of the observed signal is the sum of the variances of
the target and interference, the MVDR beamformer reduces
the effect of the interference by minimizing the variance ofthe observed signal.
In this paper, we use WPVM and perform extrapolation ofvirtual microphone signals with a triangular microphone array.
In addition, we apply this technique to speech enhancement
using a maximum SNR beamformer.
III. E XPERIMENTS
In the experiments, we examined whether WPVM and theextrapolation of virtual microphones are effective for triangularmicrophones in ICC, using an observed signal obtained fromfour speakers and the measured impulse responses in a car. Bychanging k of WPVM, α used in extrapolation, the speakers

Fig. 6. Arrangement of real and virtual microphones

to be enhanced, and the direction of the virtual microphone,
we examined whether the virtual microphone affects theperformance. We used signal-to-distortion ratio (SDR), sourceto-interference ratio (SIR), and sources-to-artifacts ratio (SAR)to evaluate the enhancement performance [13].
A. Experimental Conditions
In the experiments, we used data for a total of 10 speakers(6 male speakers and 4 female speakers), and 503 phoneme–balanced sentences contained in set B of the ATR digitalspeech database [14]. We selected four random individuals
from this database. We generated 20 patterns of observed
signals by convolving measured impulse responses into the
speech signals. The number of real microphones was 3 andthe number of virtual microphones was 1. There are 2 patternsof virtual microphone positions. The virtual microphones areused individually rathar than both at the sametime.

423

Proceedings, APSIPA Annual Summit and Conference 2020

7-10 December 2020, Auckland, New Zealand

TABLE ITABLE II

EXPERIMENTAL CONDITIONSRESULTS OF DRIVER ’ S SPEECH ENHANCEMENT

Sampling rate
Signal-to-noise ratio (SNR)FFT frame length
FFT shift
Reverbration time

8 kHz
0 dB
1024 samples
256 samples
58 ms

We measured the impulse responses by using a time–
stretched pulse (TSP) recorded in a car. In recording the TSP,
we set speakers at a driving seat, a passenger seat, and two rearseats in the car and microphones at a map lamp. The layout of
the sound sources and triangular microphone array is shown in
Fig. 3. The arrangement of the triangular microphone array isshown in Fig. 4. An image of the actual triangular microphonearray is shown in Fig. 5. Other experimental conditions arelisted in Table I. In this experiment, a virtual microphone
was applied to the triangular microphone array. The combinations of two real microphones used to generate the virtual
microphones were {(real mic1, real mic2): virtual mic1, (realmic1, real mic3): virtual mic2}, and virtual microphones are
generated towards the rear of the car. The arrangement of thevirtual microphone is shown in Fig. 6. Virtual mic1 is a virtual
microphone toward the passenger seat and the virtual mic2 is avirtual microphone toward the driver’s seat. The combinationsof two real microphones was set so that virtual microphones
are close to the angle of the enhanced speech.
In addition, we set k of WPVM to {0.25, 0.5, 1, 2} and set
α to 10. We decided the values of k on the basis of the values
tested in WPVM research and the value of α on the basis ofpreviously obtained good results [3].
In the experiment, we used the MVDR beamformer [10][12] and maximum SNR beamformer [10] [11] in only thetriangular microphone array, the maximum SNR beamformer
was used with a virtual microphone for the driver’s or passenger’s speech enhancement, and we evaluated SDR, SIR,
and SAR. The maximum SNR beamformer and MVDR beamformer were given prior information about the target-only
period and interference-only period. The speech given as priorinformation is another speech of the same speaker as the test.
SDR, SIR, and SAR were averaged over the 20 patterns
corresponding to different combinations of the four speakers.
B. Results and Discussion
We evaluated the speech enhancement performance using
the average of the results of 20 patterns. Table II shows thedriver’s speech enhancement and Table III shows the passenger’s speech enhancement. The maximum SNR beamformer
has better performance than the MVDR. Next, the performance
was better with a virtual microphone than without it. Bychanging from a situation of three microphones and four soundsources (underdetermined situation) to that of four microphones and four sound sources (determined situation) using
a virtual microphone, the performance was improved. Fromthese results, the virtual microphone technique is effective ina triangular microphone array in an underdetermined situation

Virtual mic

k 1, α2

SDR

SIR

SAR

no virtual mic

MVDR
max SNRk = 0.25
k = 0.5k=1
k=2
α = 10k = 0.25
k = 0.5k=1
k=2
α = 10

8.31
9.02
10.33
10.60
10.41
9.41
10.19
10.38
10.47
10.07
9.46
10.39

18.00
14.76
16.99
17.64
17.45
15.64
16.76
16.93
17.25
16.71
15.64
16.81

8.90
10.55
11.50
11.66
11.47
10.74
11.39
11.59
11.61
11.27
10.82
11.64

virtual mic1

virtual mic2

1
2

Wavelength coefﬁcient in WPVM.
Coefﬁcient of position of virtual microphone inextrapolation.
TABLE IIIRESULTS OF PASSENGER ’ S SPEECH ENHANCEMENT

Virtual mic

k 1, α2

SDR

SIR

SAR

no virtual mic

MVDR
max SNRk = 0.25
k = 0.5k=1
k=2
α = 10k = 0.25
k = 0.5k=1
k=2
α = 10

4.86
8.96
10.29
10.46
10.15
9.36
10.20
9.66
9.87
9.84
9.35
9.71

15.22
15.18
17.10
17.57
17.32
16.00
17.01
16.74
16.85
16.75
15.93
16.54

5,45
10.33
11.42
11.51
11.20
10.58
11.34
10.82
10.98
10.97
10.97
10.86

virtual mic1

virtual mic2

in a car.
Next, we found that the use of WPVM is the best when
an appropriate k (k = 0.5) is chosen where we apply a virtual
microphone. In contrast, when an inappropriate k, especially k= 2, was used, the results were worse than those for extrapolation with α = 10. By comparing WPVM with an appropriate k
and the maximum SNR beamformer with only the triangularmicrophone array, the SDR was improved by up to 1.6 dB
for the former. These results show that the performance isslightly better using WPVM with an appropriate k than using
the extrapolation of a virtual microphone with α = 10.
The directivity pattern did not change signiﬁcantrly when
the driver’s speech was enhanced, and SDR was improved by
up to about 0.6 dB when the passenger’s speech was enhanced.
Looking at the individual results, we found that the resultsfor passenger’s speech enhancement were often better with424

Proceedings, APSIPA Annual Summit and Conference 2020

7-10 December 2020, Auckland, New Zealandvirtual mic1, whereas the results for the driver’s speechenhancement were good in some cases and the results were
bad in others with virtual mic2. This result shows that the performance is not uniformly affected by the placement direction,
but it is more likely that the virtual microphones are the bestplaced in the direction of the target.
These results are similar to those of the two-microphone
experiments [3], except for the direction of placement, whichindicates that the virtual microphone is effective on a surface
with a triangular microphone array in ICC.
IV. C ONCLUSION
In this paper, we applied the wavelength–proportional arrangement of virtual microphones (WPVM) to speech enhancement with a triangular microphone array in in-car communication (ICC) in an underdetermined situation. By thevirtual microphone technique, it was found possible to mitigate
the degradation of performance due to underdetermined conditions in three-microphone experiments. In these experiments,
we generated observed signals by convolving measured impulse responses into speech in a car. We compared the resultsof speech enhancement by beamforming with and without avirtual microphone.
In the experiment, the virtual microphone technique wasshown to be effective for improving speech enhancement performance by using a triangular microphone array in ICC andan underdetermined situation. The WPVM with an appropriate
k showed especially the best enhancement performance.
V. ACKNOWLEDGMENT
This work was supported by the Japan Society for the
Promotion of Science (JSPS) KAKENHI through a Grant-inAid for Scientiﬁc Research under Grants 19H04131, and theStrategic Core Technology Advancement Program (Supporting
Industry Program).
R EFERENCES[1] Rabea Landgraf, Johannes Köhler-Kaeß, Christian Lüke,
Oliver Niebuhr, Gerhard Schmidt, “Can you hear me now?
Reducing the Lombard effect in a driving car using an incar communication system,” in Proc. Speech Prosody, pp.
479–483, 2016.
[2] Anne Theiss, Gerhard Schmidt, Jochen Withopf, Christian
Lueke, “Instrumental evaluation of in-car communication
systems,” in Proc. Speech Communication; 11. ITG Symposium, pp. 1–4. VDE, 2014.
[3] Ryoga Jinzai, Kouei Yamaoka, Mitsuo Matsumoto, Shoji
Makino, Takeshi Yamada, “Wavelength proportional arrangement of virtual microphones based on interpolation/extrapolation for underdetermined speech enhancement,” in Proc. European Signal Processing Conference(EUSIPCO), pp. 1–5, 2019.
[4] Hiroki Katahira, Nobutaka Ono, Shigeki Miyabe, Takeshi
Yamada, Shoji Makino, “Nonlinear speech enhancementby virtual increase of channels and maximum SNR
beamformer,” EURASIP Journal on Advances in Signal
Processing, vol. 2016, no. 1, pp. 1–8, 2016.

[5] Kouei Yamaoka, Shoji Makino, Nobutaka Ono, Takeshi
Yamada, “Performance evaluation of nonlinear speechenhancement based on virtual increase of channels inreverberant environments,” in Proc. European Signal Processing Conference (EUSIPCO), pp. 2324–2328, 2017.
[6] Ozgur Yilmaz, Scott Rickard, “Blind separation of speech
mixtures via time-frequency masking,” IEEE Transactions
on Signal Processing, vol. 52, no. 7, pp. 1830–1847, 2004.
[7] Ryoga Jinzai, Kouei Yamaoka, Mitsuo Matsumoto,Takeshi Yamada, Shoji Makino, “Microphone position
realignment by extrapolation of virtual microphone,” inProc. Asia-Paciﬁc Signal and Information Processing
Association Annual Summit and Conference (APSIPA
ASC), pp. 367–372, 2018.
[8] Brian Moore, “An introduction to the psychology ofhearing,” Academic Press, 1997.
[9] Jens Blauert, “Spatial hearing: the psychophysics of human sound localization,” MIT Press, 1997.
[10] H. L. Van Trees, “Optimum array processing,” John
Wiley & Sons, 2002.
[11] Shoko Araki, Hiroshi Sawada, Shoji Makino, “Blindspeech separation in a meeting situation with maximum SNR beamformers,” in Proc. IEEE International
Conference on Acoustics, Speech and Signal Processing(ICASSP), pp. 41-44, 2007.
[12] Otis Lamont Frost, “An algorithm for linearly constrained
adaptive array processing,” Proceedings of the IEEE, vol.
60, no. 8, pp. 926–935, 1972.
[13] Emmanuel Vincent, Hiroshi Sawada, Pau Boﬁll, Shoji
Makino, Justinian P. Rosca, “First stereo audio source
separation evaluation campaign: data, algorithms and results,” in Proc. International Conference on Independent
Component Analysis and Signal Separation (ICA), pp.
552–559, 2007.
[14] Akira Kurematsu, Kazuya Takeda, Yoshinori Sagisaka,
Shigeru Katagiri, Hisao Kuwabara, Kiyohiro Shikano,“ATR Japanese speech database as a tool of speech
recognition and synthesis,” Speech Communication, vol.
9, no. 4, pp. 357–363, 1990.

425