## 3 Elaborated Model of the Proposed Methods
### Speaker-Attributed Training Using Speaker Profiles  
Because speaker embedding vectors are provided as the input to \(\operatorname{Enc}_{\text{mix}}\), they should be as relevant to the speakers as possible in order to obtain optimal splitting performance. However, because \(\operatorname{Enc}_{\text{speaker}}\) and \(\operatorname{Dec}_{\text{speaker}}\) were trained by single-talker recordings, the extracted speaker embedding vectors may not have reliably modeled the mixed speakers. Therefore, we adopted attention-weighted averaging of the speaker profiles [8; 32; 35] to better match the input to the speaker embedding vectors in the target profile. Through using a speaker encoder-decoder on the training data, speaker embedding vectors were extracted, and those embedding vectors were clustered into a predefinednumber of profile vectors [32], which are denoted as \(\mathbf{P}=\{\mathbf{p}_{1},\mathbf{p}_{2},\ldots,\mathbf{p}_{K}\}\), where \(K\) is the number of profile vectors. In this paper, \(\mathbf{P}\) was implemented by a matrix, where each column represents a profile embedding vector, and the number of profiles used was 40. For the input query vector \(\mathbf{q}_{t}\), its closeness to profile \(k\) was computed by a cosine similarity between two vectors as follows [32]:  
\[b_{t,k}=\frac{\mathbf{q}_{t}\cdot\mathbf{p}_{k}}{\|\mathbf{q}_{t}\|\,\|\,\| \mathbf{p}_{k}\|}\,,\quad t=1,2,\ldots T\,, \tag{9}\]  
where the operator \(\cdot\) is an inner product, \(t\) is the frame index over time (in our work, it was extracted at every 10 milliseconds), \(k\) is the profile vector index in the profile matrix \(\mathbf{P}\), and \(T\) is the number of speaker embedding vectors. In our implementation, the inner product \(\mathbf{q}_{t}\cdot\mathbf{p}_{k}\) was efficiently computed by performing \(\mathbf{Q}^{\mathrm{T}}\mathbf{P}\), where T is the matrix transpose operator that generates \(T\times K\) inner products in a single step with a GPU. The cosine similarity between embedding vector \(\mathbf{q}_{t}\) and profile vector \(\mathbf{p}_{k}\), \(b_{t,k}\) ranged from \(-1\) to \(1\), so the simple linear weighting may be greatly varied. Therefore, \(b_{t,k}\) was scaled by a softmax normalization so that the resultant attention weight \(\beta_{t,k}\) ranged from 0 and 1:  
\[\beta_{t,k}=\frac{\exp(b_{t,k})}{\sum_{j=1}^{K}\exp(b_{t,j})}\,. \tag{10}\]  
The attention-weighted speaker profile \(\tilde{\mathbf{q}}_{t}\) was calculated as a linear combination of the profile vectors weighted by \(\beta_{t,k}\) as  
\[\tilde{\mathbf{q}}_{t} = \sum_{k=1}^{K}\beta_{t,k}\mathbf{p}_{k}\,, \tag{11}\] \[\tilde{\mathbf{Q}}_{mix} = \{\tilde{\mathbf{q}}_{1},\tilde{\mathbf{q}}_{2},\ldots,\tilde{ \mathbf{q}}_{T}\}\,. \tag{12}\]  
The speaker embedding set \(\mathbf{Q}_{mix}\) in Equation (5) was replaced with \(\tilde{\mathbf{Q}}_{mix}\) in Equation (12). Once the profile matrix \(\mathbf{P}\) was prepared, only cosine similarity computation was required, resulting in minimal computational overhead during the inference step. Figure 4 illustrates the speaker-attributed ASR using speaker profiles. The speaker profiling block was added to replace the speaker embedding vector with the weighted average of the profile vectors. The speaker profiling was expected to convert the unknown speaker embedding into a linear combination of known speaker embedding vectors; therefore, \(\mathrm{Enc}_{\mathrm{mix}}\) may have better split the speaker information into \(\mathbf{H}^{(1)}\) and \(\mathbf{H}^{(2)}\). Figure 4: Overview of the SAT-ASR system when using speaker profiles. The speaker embedding vector \(\mathbf{q}_{\mathrm{mix}}\) was passed through an additional block \(\mathrm{Attention}_{\mathrm{speaker}}\) and then sent to \(\mathrm{Enc}_{\mathrm{mix}}\). \(\mathbf{P}\) is a profile matrix composed of the speaker embedding vectors obtained from the training dataset, and \(\beta\) is a set of computed attention weights.