# Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics +
## 5 Experimental Framework
### Quality Measurement
#### 5.2.2 Alignment Metrics  
These metrics aim to obtain the number of correctly generated cloned utterances by matching the input text sequence with the resulting waveform. As an example, an estimation of the number of correct sentences can be computed by means of an automatic force aligner such as MFA, trying to match the generated audio with the corresponding input text with the lowest beam possible. However, a successful forced alignment does not ensure that a particular sentence has been correctly generated.  
In this context, we propose a complementary alignment metric that takes advantage of the characteristics of the voice-cloning system Tacotron-2, its attention matrix specifically, for computing the number of input characters that have been correctly synthesised from each sentence. Given that the weights of the attention matrix for successfully generated audio should present a diagonal line, its presence can be easily checked by using a series of sliding rectangular windows:  
Let \(A=\left(a_{ij}\right)\) be an attention matrix of dimensions \(E\times D\) where \(E\) is the length of the input sequence, \(D\) is the length of the output spectrogram, and every \(a_{ij}\) represents the attention score of the matrix. In order to check if all these elements have scored a minimum threshold value, they were clustered using sliding rectangles of a fixed size with values of the width \(w\) and height \(h\). The algorithm starts with the first rectangle at position \(x=0\), \(y=0\) (lower-left corner). Every element \(a_{ij}\) of the attention matrix inside this rectangle is checked to be higher than a given threshold value \(\theta\), which can be modified as an input parameter in order to be more or less aggressive. The rectangle is given a margin of \(\frac{1}{3}w\) to the left in order to facilitate the search of aligned characters inside the region. If there exists a value \(a_{ij}>\theta\) inside this rectangle, then the \(i\)-th input character is considered to be correctly aligned. This process is then repeated, sliding the rectangle to the position of the last correctly aligned character until:  
* No correctly aligned character is found inside the region: there is no correctly synthesised character beyond this point,
* Its uppermost point exceeds \(E\): the characters at the end of the sentence are aligned,
* Or its leftmost point exceeds \(D\): the alignment algorithm reaches the end of the spectrogram.  
The algorithmic implementation is shown in Algorithm 1, and an example of this procedure can be found in Figure 4.  
In this work, all the aligned characters were calculated with the following values: \(w=150\) spectrogram windows, \(h=8\) characters, and \(\theta=0.7\), as they performed the best in our previous experiments.  
```
1:functionAligned characters(\(A,w,h,\theta\))
2:\(x,y\gets 0,0\)
3:aligned_total \(\gets 0\)
4:while\(y+h<E\)and\(x+\frac{2}{3}w<D\)do
5:\(\#\)Attention scores higher than \(\theta\) inside the rectangle:
6:\(N\leftarrow\left\{a_{ij}\ :\ a_{ij}>\theta\ \ \text{and}\ \ y<i\leq y+h\ \ \text{and}\ \ x-\frac{1}{3}w<j\leq x+\frac{2}{3}w\right\}\)
7:\(\#\)Get the number of characters with a score higher than \(\theta\) inside the rectangle:
8:aligned_rect \(\leftarrow\left|\left\{i:a_{ij}\in N\right\}\right|\)
9:ifaligned_rect \(=0\)then
10:break
11:endif
12:aligned_total \(\leftarrow\)aligned_total \(+\)aligned_rect
13:\(\#\)Slide the rectangle to the position of the last aligned element:
14:\(y\leftarrow\max\{i:a_{ij}\in N\}\)
15:\(x\leftarrow\max\{j:a_{ij}\in N\}\)
16:endwhile
17:return aligned_total
18:endfunction
```  
**Algorithm 1** Character alignment algorithm using the attention matrix of Tacotron-2, given an attention matrix \(A=(a_{ij})\) of dimensions \(E\times D\), a rectangle of width \(w\) and height \(h\), and threshold value \(\theta\). The algorithm is explained in detail in Section 5.2.2