# Data-independent Beamforming for End-to-end Multichannel Multi-speaker ASR

Can Cui, Paul Magron, Mostafa Sadeghi, Emmanuel Vincent

<figure><img src="image_1.png" /></figure>

## To cite this version:

Can Cui, Paul Magron, Mostafa Sadeghi, Emmanuel Vincent. Data-independent Beamforming for End-to-end Multichannel Multi-speaker ASR. IEEE 27th International Workshop on Multimedia Signal Processing (MMSP 2025), IEEE, Sep 2025, Pékin, China. hal-05252233

HAL Id: hal-05252233

https://hal.science/hal-05252233v1

Submitted on 12 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
Data-independent Beamforming for
End-to-end Multichannel Multi-speaker ASR

Can Cui

Paul Magron, Mostafa Sadeghi, Emmanuel Vincent

Research and Development Group

Multispeech

iFLYTEK Co., Ltd.

Université de Lorraine, CNRS, Inria, LORIA, F-54000

Shanghai, China

Nancy, France

cancui11@iflytek.com

{paul.magron, mostafa.sadeghi, emmanuel.vincent}@inria.fr

***Abstract***—Automatic speech recognition (ASR) in multichan-
nel, multi-speaker scenarios remains challenging due to ambient
noise, reverberation and overlapping speakers. In this paper, we
propose a beamforming approach that processes specific angular
sectors based on their spherical polar coordinates before applying
an end-to-end multichannel, multi-speaker ASR system. This
method is data-independent and training-free. We demonstrate
that using a group of beamformed signals improves ASR perfor-
mance compared to using the same number of raw microphone
signals. Moreover, increasing the number of signals used for
beamforming further enhances recognition accuracy, leading to
a more efficient use of multichannel signals while reducing the
overall input load for the ASR system. We conduct experiments
on the AMI meeting corpus, where the proposed method reduces
word error rate by up to 11% and improves speaker counting
accuracy by up to 27% relative compared to a multichannel ASR
baseline system that does not exploit beamforming.

***Index Terms***—multichannel multi-speaker speech recognition, data-independent beamforming, spatial information

I. INTRODUCTION

Transcribing real-world distant-microphone multi-speaker
meetings recordings is an active research area [1]-[3]. How-
ever, it remains a challenging task due to noise, reverbera-
tion, and overlapping speech. To address these challenges,
numerous studies [4]-[8] have explored end-to-end multichan-
nel architectures. Multichannel audio processing introduces a
wealth of spatial information, making it a valuable approach
for speech separation and enhancement. By utilizing mul-
tiple microphones, spatial cues from various directions can
be harnessed to improve the overall robustness and perfor-
mance of these systems. Inspired by single-channel atten-
tion [9], the multi-frame cross-channel attention (MFCCA)
mechanism [6] either concentrates on understanding global
correlations between sequences across different channels or
efficiently utilizes detailed channel-wise information at each
time step. This mechanism can then be integrated within
an ASR encoder to effectively capture spatial information
from multiple microphones, where it has been shown to
outperform single-channel ASR systems [6], [8]. However, it
directly processes raw far-field microphone signals, making
it susceptible to ambient noise and reverberation, which can
degrade recognition performance.

This work was mostly conducted while Can Cui was pursuing her PhD
with Inria Centre at Université de Lorraine (Nancy, France).

Several studies have been conducted to enhance multichan-
nel input quality, particularly through beamforming [10]–[12]
and/or speech separation [2], [13], [14]. While these methods
improve speech quality, they typically rely on subsequent
single-channel ASR architectures, leading to the loss of spatial
information during recognition. To mitigate this, [15] proposes
feeding a multichannel ASR system with beamformed outputs
as separate input channels, while [16] explores selecting a
subset of audio channels that are optimal for the task at hand.
However, these approaches require data-dependent beamform-
ing and additional training, making the process more complex.

In this paper, we alleviate the afore-mentioned issues by
leveraging a beamforming approach that extracts signals from
specific angular sectors. This allows to enhance speech quality
by capturing speech information from various participant po-
sitions. This multichannel input is then fed into an end-to-end
ASR system based on serialized output training (SOT) [17]
within a multichannel architecture using the MFCCA mech-
anism to further exploit spatial information. Our proposed
method is data-independent and does not require training for
beamforming, which makes it adaptable to any multichannel
ASR system. We conduct experiments on the real-world AMI
meeting dataset [18]. Our results show that the proposed
approach reduces word error rate (WER) by up to 11 % relative
and speaker counting accuracy by up to 27 % relative, when
compared to an ASR that uses raw multichannel signals as
input. Our system also reduces WER by 10 % relative to an
MVDR-based single-channel ASR baseline.

The rest of this paper is organized as follows. Section II
presents the proposed data-independent beamforming tech-
nique. We then describe our experimental setup and results
in Section III. Finally, Section IV concludes the paper.

II. DATA-INDEPENDENT BEAMFORMING BASED ON
TARGET ANGULAR SECTORS

We propose to preprocess the multichannel signals with
data-independent beamformers, so that each resulting beam-
formed signal focuses on sound incoming from a specific
angular sector. This preprocessing makes it possible to exploit
the phase information corresponding to distinct speaker posi-
tions. The beamformers for each angular sector are calculated
from all microphones, so the number of microphones and the
<figure><img src="image_3.png" /><figcaption>Fig. 1: Bird's-eye view of the acoustic scene for original microphone channels (left), and beamformed channels corresponding to 4 angular sectors from either 4 microphones (center) or 8 microphones (right).</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2: Geometrical illustration of the azimuth $\theta$, the elevation $\phi$ and the unit vector $\mathbf{k}$ between the source and the microphone array center.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 3: Angular sector corresponding to $\theta \in [45^\circ, 135^\circ]$ and $\phi \in [10^\circ, 60^\circ]$.</figcaption></figure>

number of angular sectors are two independent concepts, as illustrated in Figure 1. Let $I$ and $S$ denote the number of microphones and angular sectors, respectively. Multichannel signals in the short-time Fourier transform (STFT) domain are denoted $\mathbf{x}(f) \in \mathbb{C}^I$ at frequency $f$, as we discard the time frame index for clarity.

### A. Data-independent beamforming

We consider the data-independent beamformer presented in [19, Chap.10.3], which is designed to extract signals from specific angles by leveraging prior knowledge about the speakers' positions. Assuming that the sources are in the far field of the array [20], the problem can be defined as finding the beamformer $\mathbf{w}(f) \in \mathbb{C}^I$, whose spatial response is closest to a predefined target $b^{\text{igt}}(\theta, \phi, f)$:

$$ \underset{\mathbf{w}}{\operatorname{argmin}} \frac{1}{4\pi} \int_{\Omega} |\mathbf{w}^H \mathbf{d}(\theta, \phi, f) - b^{\text{igt}}(\theta, \phi, f)|^2 \cos \theta d\theta d\phi, \quad (1) $$

where $\Omega$ denotes the set of all azimuth $\theta$ and elevation $\phi$ angles (these are illustrated in Figure 2), $\mathbf{H}$ denotes the Hermitian transpose, and:

$$ \mathbf{d}(\theta, \phi, f) = \begin{bmatrix} e^{-2j\pi\mathbf{k}^T(\theta, \phi, f)\mathbf{m}_1/\lambda} \\ \vdots \\ e^{-2j\pi\mathbf{k}^T(\theta, \phi, f)\mathbf{m}_I/\lambda} \end{bmatrix} \quad (2) $$

is the steering vector with $\mathbf{m}_i$, the Cartesian coordinates of microphone $i$, $\lambda$ is the center wavelength of the narrowband signal, and:

$$ \mathbf{k} = \begin{bmatrix} \cos \theta \cos \phi \\ \sin \theta \cos \phi \\ \sin \phi \end{bmatrix} \quad (3) $$

is the unit vector from the center of the microphone array to the target source.

Since Equation (1) has no general closed-form solution, several solutions have been proposed for specific array geometries, predefined target models $b^{\text{igt}}$, and/or exploiting numerical approximation schemes (see [19, Chap.10.3] for an overview). In what follows we propose a simple predefined target model for $b^{\text{igt}}$ which allows to derive a closed-form beamformer for each angular sector.

### B. Proposed solution

Since the speech source is omnidirectional, we aim to extract a response from every spatial direction. Therefore, based on Equation (1), the beamformer is obtained by integrating the signal over the responses from all azimuth and elevation angles. However, in this work we rather perform integration over angular sectors, which represent groups of target position points in a given portion of space. Each angular sector $\Psi_s$, $s \in [1, S]$ is defined as a set of azimuth and elevation ranges,
<figure><img src="image_6.png" /><figcaption>Fig. 4: Overview of the proposed system combining data-invariant beamforming and multichannel MFCCA-based ASR. The MFCCA module extracts and integrates spatial-temporal features across the *S* sectors, but can also directly process the *I* raw multichannel signals.</figcaption></figure>

as illustrated in Figure 3. We consider the following predefined target for each angular sector:

$$b_s^{\text{tgt}}(\theta, \phi, f) = \begin{cases} 1 & \text{if } (\theta, \phi) \in \Psi_s, \\ 0 & \text{otherwise,} \end{cases} \quad (4)$$

which allows to derive the following closed-form solution for Equation (1):

$$\hat{\mathbf{w}}_s(f) = \left( \int_{\Omega} \cos\theta \times \mathbf{d}(\theta, \phi, f) \mathbf{d}(\theta, \phi, f)^H d\theta d\phi \right)^{-1} \times \int_{\Psi_s} \cos\theta \times \mathbf{d}(\theta, \phi, f) d\theta d\phi. \quad (5)$$

Once estimated, we apply the set of beamformers $\hat{\mathbf{W}}(f) \in \mathbb{C}^{I \times S}$ to the input multichannel signal via $\hat{\mathbf{W}}_s^H(f)\mathbf{x}(f)$, and these are then reverted back to the time domain via inverse STFT.

Finally, the resulting beamformed signals are fed to a multichannel ASR system, as illustrated in Figure 4. We consider an MFCCA-based ASR model, which captures angle-specific information and encodes dependencies across different spatial directions and time steps (more details in Section III-B).

### C. Proposed angular sectors

In this paper, we consider four equally-spaced angular sectors ($S = 4$), which corresponds to a four-speaker scenario as encountered in our dataset (see Section III-A). Note that this number can be adjusted to accommodate different meeting configurations. For realistic consideration of speakers’ possible positions with respect to the microphone array, we choose an elevation range from 10 to 60 degrees for all sectors. The angular sectors cover a width of 90° azimuth each, ranging from 315 to 45°, from 45 to 135°, from 135 to 225°, and from 225 to 315°, respectively. While prior knowledge of the speaker’s position is not essential for applying our data-independent beamformer, the beamformed signals are expected to isolate the target speakers given such a spatial setup. Besides, even though each beamformed signal does not perfectly extract each speaker, this preprocessing is expected to improve MFCCA’s ability to model cross-channel dependencies.

We consider either 4 or 8 microphones arranged in a circular array, thus yielding three possible inputs to the multichannel ASR system, as illustrated in Figure 1: the original 4-channel signals from 4 microphones, the 4-channel

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 5: Response of the fixed beamformer associated with the second angular sector. The dashed red and white lines define regions with azimuth values from 45 to 135°, and from 225 to 315°, respectively.</figcaption></figure>

beamformed signals extracted from 4 microphones, and the 4-channel beamformed signals extracted from 8 microphones. Figure 5 illustrates the response of the beamformer in the second angular sector, i.e., $\hat{\mathbf{w}}_2^H(f)\mathbf{d}(\theta, \phi, f)$, over frequencies and azimuth angles, for various elevation angles (10°, 25°, 40°, and 55°), and computed from either 4 or 8 microphones. The 8-microphone setup maintains accurate performance (i.e., a high spatial response) up to 4,000 Hz, while the 4-microphone
setup starts to degrade above 2,000 Hz. The values within the target angular sector, an azimuth range from 45 to 135°, are higher than those in other sectors. For the 8-microphone setting, the quality of the response degrades above 4,000 Hz, especially at elevations of 10° and 55°, because they are closer to the border of the target elevation range, which is from 10° to 60°. At an elevation of 55° for both the 4- and 8-microphone settings, higher values can be found in other sectors, such as the fourth sector with an azimuth range from 225° to 315°. Overall, the 8-microphone responses are sharper and more accurate than those of the 4-microphone setting.

### III. EXPERIMENTS

This section details our experimental setup and results. For reproducibility, our code is available online.<fnref n="1" />

#### A. Datasets and metrics

1) **Datasets: Multi-speaker LibriSpeech** — To achieve good performance on real-life meeting corpus, the ASR model must be pretrained on a larger simulated distant-microphone multi-speaker dataset [21]. We created a 960 h training set and a 20 h development set from the LibriSpeech train-960 and dev-clean sets [22]. We adopted the room simulation settings as described in [8]. Each mixture contains 1 to 3 speakers. Each speaker's utterance begins after a random delay of more than 0.5 s from the start of the previous speaker's utterance, to better reflect realistic speaker turn-taking. This also guarantees the first-in, first-out principle behind SOT. The sentences of each speaker are concatenated together and separated from the next speaker using a special <sc> token.

**Real AMI** — After it has been pretrained on Multi-speaker LibriSpeech, the ASR model is fine-tuned and evaluated on real AMI multiple distant microphone (MDM) data. We utilize the segmentation method in [8] to partition the MDM data into 5 s chunks and adjust the chunk start/end times to non-overlapped word boundaries. The resulting AMI dataset contains respectively 165 h, 19 h, and 19 h for training, development, and test. We consider 4- and 8-channel taken from Array1,<fnref n="2" /> namely channels 1, 3, 5, and 7 in the 4-channel setting, and all 8 channels in the 8-channel setting. We use the Full-corpus-ASR partition of the dataset.<fnref n="3" />

For both datasets, audio signals are sampled at 16 kHz. Log-Mel spectrograms are extracted using a 25 ms window, a hop size of 10 ms, and 80 Mel bands. Note that even though in this study we consider static speakers and circular microphone arrays for simplicity, our proposed method can accommodate moving sources and any array geometry.

2) **Metrics:** We use the WER to evaluate the ASR task. The ASR model we employed transcribes speech from multiple speakers by inserting a speaker change token <sc> in a first-in, first-out order. This allows for calculating the number of speakers in both the ground truth and estimated transcriptions,

determined by counting the occurrences of <sc> tokens in the ASR output. As a result, we can also evaluate the speaker counting accuracy [23] via the confusion scores:

$$ \text{Confusion\_score}(i, k) = \frac{N_k^i}{N_k}, \qquad (6) $$

where $N_k^i$ is the number of test signals where the estimate indicates i speakers when the ground truth has k speakers, and $N_k$ represents the number of test signals where the ground truth has k speakers.

We employed the SCTK toolkit<fnref n="4" /> to conduct a Matched Pair Sentence Segment significance test. We indicate with bold fonts the best numerical results at a 0.05 significance level in the corresponding tables.

#### B. Models and training setup

We implement our proposed beamformer as detailed in Section II. To compute it in practice, the integrals in Equation (5) are replaced with sums using a step size of 1 degree over the considered range, which allows for a good trade-off between computational burden and precision.

We then consider an MFCCA-based multichannel ASR model. It features 12 layers in the encoder and 6 layers in the decoder. The number of filterbank output channels and the number of bins in the estimated masks are both set at 513. All multi-head attention mechanisms have 4 heads, the model dimension D is set to 256, and the size of the feedforward layer is 2,048. Following [6], the context frame length F of MFCCA is set to 2. Our text tokenizer is a SentencePiece model [24] with a vocabulary of 5,000 tokens.

To evaluate the effectiveness of the proposed beamforming method, as a comparison baseline we consider a neural MVDR beamformer [25], [26], as implemented in the torchaudio library [27]. This yields a single-channel signal which is then fed to a single-channel SOT-based ASR (see details above).

Note that even though more recent methods such as [15], [23] would be interesting comparison baselines, we could unfortunately not reproduce their results due to a lack of available code, data, and limited computational resources.

Our experiments were implemented using the SpeechBrain toolkit [28]. Pretraining on LibriSpeech and fine-tuning on Real AMI are conducted using the Adam optimizer with a learning rate of $5 \times 10^{-4}$ and $1 \times 10^{-4}$, and a global batch size (batch size × number of GPUs × gradient accumulation factor) of 160 and 80, respectively.

#### C. Evaluation results

1) **Improvements on speech quality:** First, we compare signals before and after beamforming by visualizing the spectrogram of a Real AMI meeting chunk in the first array / sector. As shown in Figure 6, both beamformed signals (i.e., from either 4 or 8 microphones) significantly reduce ambient noise and reverberation compared to the original far-field channel. This illustrates the ability of the proposed beamformer to

<footnote n="1">¹https://github.com/can-cui/data-invariant-beamforming-mc-asr</footnote>

<footnote n="2">²A few meetings have two arrays, each consisting of 8 microphones.</footnote>

<footnote n="3">³See partition detail at https://groups.inf.ed.ac.uk/ami/corpus/datasets.shtml.</footnote>

<footnote n="4">⁴https://github.com/usnistgov/SCTK.git</footnote>
<table><caption>TABLE I: ASR performance (WER in %) on the LibriSpeech and Real AMI test sets, for several methods, according to the ASR input type (unprocessed, using the proposed beamforming, or an MVDR beamformer).</caption><thead><tr><th rowspan="2">Input type</th><th rowspan="2"># Microphones</th><th rowspan="2"># Sectors</th><th colspan="4">LibriSpeech</th><th colspan="4">Real AMI</th></tr><tr><th>1-speaker</th><th>2-speaker</th><th>3-speaker</th><th>1-3-speaker</th><th>1-speaker</th><th>2-speaker</th><th>3-speaker</th><th>1-4-speaker</th></tr></thead><tbody><tr><td>Unprocessed</td><td>4</td><td>-</td><td>7.21</td><td>14.82</td><td>19.76</td><td>16.30</td><td>25.89</td><td>41.70</td><td>54.68</td><td>45.25</td></tr><tr><td rowspan="2">Proposed beamforming</td><td>4</td><td>4</td><td>7.46</td><td>12.27</td><td>22.99</td><td>17.64</td><td>24.52</td><td><b>39.61</b></td><td>52.19</td><td>43.14</td></tr><tr><td>8</td><td>4</td><td><b>6.73</b></td><td><b>10.53</b></td><td><b>19.06</b></td><td><b>14.26</b></td><td><b>22.96</b></td><td>40.01</td><td><b>49.59</b></td><td><b>41.64</b></td></tr><tr><td rowspan="2">MVDR</td><td>4</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>25.11</td><td>40.07</td><td>54.17</td><td>44.43</td></tr><tr><td>8</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>26.35</td><td>42.13</td><td>55.89</td><td>46.08</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>Fig. 6: Spectrogram of one Real AMI test chunk: original 1st array channel (top), and beamformed signal in the 1st sector from 4 microphones (middle) or 8 microphones (bottom).</figcaption></figure>

enhance the signal quality. We also observe interfering speech reduction in the beamformed signal. Thus, even though the exact positions of the speakers are unknown, the proposed beamforming method is able to enhance a specific speaker (i.e., the one in the first angular sector) and reduce the other ones. Similar results are obtained from other arrays / sectors.

Furthermore, by comparing the signals extracted from 4 and 8 microphones, particularly in the zone between the white lines, we observe that the latter provides better dereverberation in the speech region by reducing interfering speech and enhancing the target speech. Similar phenomena can be observed in other speech segments, where the signal from 8 microphones shows clearer speech formants. This demonstrates that using more microphones better exploits the available spatial information and thus yields cleaner speech signals.

2) Speech recognition performance: We then compare the results of ASR systems trained on original channels versus data-independent beamformed channels. Table I displays the results on the LibriSpeech and Real AMI test sets using the pretrained and fine-tuned models, respectively. On the LibriSpeech test set, the model trained on 4-channel beamformed signals from 4 microphones (“beamformed-4”) does not show improvement compared to the model trained on the unprocessed 4-channel signals, with WERs of 17.64% vs. 16.30% on the 1,2,3-speaker mixture test set, respectively. However, the model trained on 4-channel beamformed signals from 8 microphones (herein denoted “beamformed-8”) demonstrates a reduction of 13% relative in WER compared to beamformed-4, reducing the WER from 16.30% to 14.26% on the 1,2,3-speaker mixture test set. For the 2-speaker mixture test set, both beamformed-4 and -8 show better performance compared

<table><caption>TABLE II: Speaker counting accuracy (confusion score in %) on the Real AMI test set for our proposed method.</caption><thead><tr><th rowspan="2">Input type</th><th rowspan="2"># Speakers</th><th colspan="5">Estimated # speakers</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>&gt;4</th></tr></thead><tbody><tr><td rowspan="4">Unprocessed</td><td>1</td><td>96.84</td><td>3.01</td><td>0.14</td><td>0.00</td><td>0.00</td></tr><tr><td>2</td><td>23.21</td><td><b>74.65</b></td><td>2.09</td><td>0.03</td><td>0.00</td></tr><tr><td>3</td><td>6.39</td><td>63.90</td><td>29.08</td><td>0.61</td><td>0.00</td></tr><tr><td>4</td><td>2.85</td><td>39.82</td><td>52.60</td><td>4.71</td><td>0.00</td></tr><tr><td rowspan="4">Beamformed (8 mic.)</td><td>1</td><td><b>97.86</b></td><td>2.04</td><td>0.09</td><td>0.00</td><td>0.00</td></tr><tr><td>2</td><td>25.89</td><td>72.05</td><td>1.98</td><td>0.06</td><td>0.00</td></tr><tr><td>3</td><td>6.29</td><td>56.71</td><td><b>36.94</b></td><td>0.04</td><td>0.00</td></tr><tr><td>4</td><td>1.86</td><td>33.87</td><td>58.31</td><td><b>5.70</b></td><td>0.24</td></tr></tbody></table>

to the unprocessed model, with a 14% relative reduction in WER (from 14.82% to 12.27%) and a 29% relative reduction in WER (from 14.82% to 10.53%), respectively.

On the Real AMI test set, both beamformed models yield a more substantial improvement compared to using the unprocessed signals. In particular, the beamformed-8 model achieves a reduction in WER of 11%, 4%, 9%, and 8% relative compared to the unprocessed model for the 1, 2, 3, and 1-4-speaker groups, respectively. Overall, these improvements highlight the efficiency of fine-tuning on beamformed signals to better exploit spatial information. Moreover, the beamformed-8 model consistently outperforms the beamformed-4 model on both the LibriSpeech and Real AMI test sets. This demonstrates that using beamformed signals from a larger number of microphones results in better performance for the MFCCA-vased multichannel ASR model. This conclusion is consistent with the fact that a larger number of microphones feeds more information to a given sector, thereby providing better enhancement of signals from that sector (see Figure 5).

3) Speaker counting accuracy: Table II presents the speaker counting accuracy on the Real AMI test set.<fnref n="5" /> The results show that the model utilizing beamformed channels generally outperforms the one using original channels. Specifically, the proposed method improves speaker counting accuracy by 27% relative (from 29.08% to 39.94%) in 3-speaker scenarios and by 21% relative (from 4.71% to 5.70%) in 4-speaker scenarios. These findings highlight that beamforming improves the prediction of speaker change tokens by more effectively leveraging spatial information, particularly as the number of speakers increases.

<footnote n="5">⁵Note that these results are not directly comparable to similar studies utilizing speaker-attributed ASR [8], [12], as these incorporate an additional speaker embedding model to enhance speaker prediction accuracy.</footnote>
4) *Comparison to MVDR-based beamforming:* We report the performance of the MVDR-based ASR method (see Section III-B) in the last rows of Table I. We only consider test results on the AMI dataset, as it is more realistic. We observe that the model using raw multichannel input (i.e., no beamforming) does not yield a consistent improvement over the MVDR-ASR system. This suggests that the spatial information may not be fully exploited by the ASR model when it is not first processed by e.g., a beamformer. In contrast, when using our proposed data-invariant beamformed input, the performance improves significantly for both 4- and 8-channel configurations. Notably, the model using 8 microphones achieves a 10% relative reduction in WER, decreasing from 46.08% in the MVDR-based system to 41.64% on the mixed 1-4-speaker scenarios. This result highlights the benefit of a robust, data-independent beamforming method that generalizes well across channel and speaker conditions, without relying on neural network-based learning.

## IV. CONCLUSION

In this paper, we have proposed a data-independent, training-free beamforming approach as input to an end-to-end multichannel multi-speaker ASR system. This beamformer captures speech from specific elevation and azimuth sectors, aligned with typical participant positions in meetings. Our method achieves a reduced WER by up to 11% and improves speaker counting accuracy by up to 27% relative compared to using unprocessed multichannel signals, and also outperforms a competing method based on data-dependent MVDR beamforming. These results highlight the potential of data-invariant spatial filtering for efficient multi-speaker ASR. Future work will explore adaptive integration with downstream models, and deployment in more diverse acoustic conditions, including moving sources and ad-hoc microphone arrays.

## V. ACKNOWLEDGMENTS

Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations.

## REFERENCES

[1] F. Yu, S. Zhang, Y. Fu, L. Xie, S. Zheng, Z. Du, W. Huang, P. Guo, Z. Yan et al., "M2Met: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge," in *Proc. of IEEE ICASSP*, 2022.

[2] S. Cornell, M. S. Wiesner, S. Watanabe, D. Raj, X. Chang, P. Garcia, Y. Masuyam, Z.-Q. Wang, S. Squartini, and S. Khudanpur, "The CHIME-7 DASR Challenge: Distant meeting transcription with multiple devices in diverse scenarios," in *Proc. of CHIME*, 2023.

[3] S. Cornell, T. J. Park, H. Huang, C. Boeddeker, X. Chang, M. Maciejewski, M. S. Wiesner, P. Garcia, and S. Watanabe, "The CHIME-8 DASR Challenge for generalizable and array agnostic distant automatic speech recognition and diarization," in *Proc. of CHIME*, 2024.

[4] F.-J. Chang, M. Radfar, A. Mouchtaris, and M. Omologo, "Multi-channel transformer transducer for speech recognition," in *Proc. of Interspeech*, 2021.

[5] J. Li, Y. Zhu, D. Luo, Y. Liu, G. Cui, and Z. Li, "The PCG-AIID system for L3DAS22 challenge: MIMO and MISO convolutional recurrent network for multi channel speech enhancement and speech recognition," in *Proc. of IEEE ICASSP*, 2022.

[6] F. Yu, S. Zhang, P. Guo, Y. Liang, Z. Du, Y. Lin, and L. Xie, "MFCCA: Multi-frame cross-channel attention for multi-speaker ASR in multi-party meeting scenario," in *Proc. of IEEE SLT*, 2023.

[7] M. Shi, J. Zhang, Z. Du, F. Yu, Q. Chen, S. Zhang, and L.-R. Dai, "A comparative study on multichannel speaker-attributed automatic speech recognition in multi-party meetings," in *Proc. of APSIPA ASC*, 2023.

[8] C. Cui, I. Sheikh, M. Sadeghi, and E. Vincent, "End-to-end multichannel speaker-attributed ASR: Speaker guided decoder and input feature analysis," in *Proc. of IEEE ASRU*, 2023.

[9] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," *Advances in Neural Information Processing Systems*, vol. 30, 2017.

[10] Z. Zhang, T. Yoshioka, N. Kanda, Z. Chen, X. Wang, D. Wang, and S. E. Eskimez, "All-neural beamformer for continuous speech separation," in *Proc. of IEEE ICASSP*, 2022.

[11] Y. Masuyama, X. Chang, S. Cornell, S. Watanabe, and N. Ono, "End-to-end integration of speech recognition, dereverberation, beamforming, and self-supervised learning representation," in *Proc. of IEEE SLT*, 2023.

[12] C. Cui, I. A. Sheikh, M. Sadeghi, and E. Vincent, "Joint beamforming and speaker-attributed ASR for real distant-microphone meeting transcription," in *Proc. of EUSIPCO*, 2025.

[13] D. Raj, P. Denisov, Z. Chen, H. Erdogan, Z. Huang, M. He, S. Watanabe, J. Du, T. Yoshioka, Y. Luo et al., "Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis," in *Proc. of IEEE SLT*, 2021.

[14] J. Shi, X. Chang, S. Watanabe, and B. Xu, "Train from scratch: Single-stage joint training of speech separation and recognition," *Computer Speech & Language*, vol. 76, p. 101387, 2022.

[15] N. Kanda, J. Wu, X. Wang, Z. Chen, J. Li, and T. Yoshioka, "Vararray meets T-Sot: Advancing the state of the art of streaming distant conversational speech recognition," in *Proc. of IEEE ICASSP*, 2023.

[16] B. Mu, P. Guo, D. Guo, P. Zhou, W. Chen, and L. Xie, "Automatic channel selection and spatial feature integration for multi-channel speech recognition across various array topologies," in *Proc. of IEEE ICASSP*, 2024.

[17] N. Kanda, Y. Gaur, X. Wang, Z. Meng, and T. Yoshioka, "Serialized output training for end-to-end overlapped speech recognition," in *Proc. of Interspeech*, 2020.

[18] J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal et al., "The AMI meeting corpus: A pre-announcement," in *Proc. of MLMI*, 2005.

[19] E. Vincent, T. Virtanen, and S. Gannot, *Audio Source Separation and Speech Enhancement*. John Wiley & Sons, 2018.

[20] D. H. Johnson and D. E. Dudgeon, *Array Signal Processing: Concepts and Techniques*. Prentice Hall., 1993.

[21] M. Yang, N. Kanda, X. Wang, J. Wu, S. Sivasankaran, Z. Chen, J. Li, and T. Yoshioka, "Simulating realistic speech overlaps improves multitalker ASR," in *Proc. of IEEE ICASSP*, 2023.

[22] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, "Librispeech: An ASR corpus based on public domain audio books," in *Proc. of IEEE ICASSP*, 2015.

[23] N. Kanda, Y. Gaur, X. Wang, Z. Meng, Z. Chen, T. Zhou, and T. Yoshioka, "Joint speaker counting, speech recognition, and speaker identification for overlapped speech of any number of speakers," in *Proc. of Interspeech*, 2020.

[24] T. Kudo and J. Richardson, "Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing," in *Proc. of EMNLP*, 2018.

[25] Y.-J. Lu, S. Cornell, X. Chang, W. Zhang, C. Li, Z. Ni, Z.-Q. Wang, and S. Watanabe, "Towards low-distortion multi-channel speech enhancement: The ESPNet-SE submission to the L3DAS22 challenge," in *Proc. of IEEE ICASSP*, 2022.

[26] M. Kim, S. Cheong, and J. W. Shin, "DNN-based Parameter Estimation for MVDR Beamforming and Post-filtering," in *Proc. of Interspeech*, 2023.

[27] Y.-Y. Yang, M. Hira, Z. Ni, A. Chourdia, A. Astafurov, C. Chen, C.-F. Yeh, C. Puhrsch, D. Pollack, D. Genzel, D. Greenberg, E. Z. Yang, J. Lian, J. Mahadeokar, J. Hwang, J. Chen, P. Goldsborough, P. Roy, S. Narenthiran, S. Watanabe, S. Chintala, V. Quenneville-Bélair, and Y. Shi, "Torchaudio: Building blocks for audio and speech processing," *arXiv preprint arXiv:2110.15018*, 2021.

[28] M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Logosch, C. Subakan, N. Dawalatabad, A. Heba et al., "SpeechBrain: A general-purpose speech toolkit," 2021, arXiv:2106.04624.