## 4 Experiment
### Results
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Development**} & \multicolumn{2}{c}{**Evaluation**} \\ \cline{3-4}  & & **Male** & **Female** \\ \hline _max\_front\_const_ & 5.9814 & 5.4696 & 5.7919 \\ _max\_front\_rept_ & **5.0286** & 4.7755 & 5.2656 \\ _max\_end\_const_ & 6.0851 & 5.2865 & 5.9126 \\ _max\_end\_rpt_ & 5.1067 & 4.7189 & 5.2431 \\ \hline _mean\_front\_const_ & 6.2526 & 5.5981 & 5.7569 \\ _mean\_end\_rpt_ & 6.0607 & 5.5753 & 5.9489 \\ _mean\_end\_const_ & 5.8732 & 5.1386 & 5.8918 \\ _mean\_end\_rpt_ & 5.9291 & 5.3014 & 5.8448 \\ \hline _bmax\_front\_const_ & 6.148 & 5.235 & 5.1985 \\ _bmax\_front\_rept_ & 5.3502 & 4.5813 & 5.2674 \\ _bmax\_end\_const_ & 6.4357 & 5.2748 & 5.4604 \\ _bmax\_end\_rpt_ & 5.2754 & 5.002 & 5.4826 \\ \hline _bmean\_front\_const_ & 6.3489 & 5.4201 & 5.7655 \\ _bmean\_front\_rept_ & 6.515 & 5.4798 & 6.266 \\ _bmean\_end\_const_ & 6.1184 & 5.3527 & 5.6719 \\ _bmean\_end\_rpt_ & 6.0218 & 5.6071 & 6.3661 \\ \hline \multicolumn{2}{c}{PackedSequence} & 5.3949 & 4.6406 & 5.2572 \\ \hline \multicolumn{2}{c}{Proposed} & 5.1206 & **4.2291** & **5.039** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The EERs (%) of systems with self-attention on the development and evaluation trials of RSR 2015 part 2. The lowest EERs among all the methods are highlighted in bold, the lowest EERs among all the methods except the proposed method are underlined. If attention can perfectly ignore (i.e., give zero attention weight) the frames that contain unnecessary information, the EERs of _max/bmax_/PackedSequence (i.e., when there is no information loss) may be very close to those of the proposed method. If so, it is not necessary to use the proposed method. In practice, however, the proposed method showed relatively about 11.2%, 11.1%, and 6% lower EERs on average than _max_, _bmax_, and PackedSequence, respectively. Except on the development trials, the proposed method shows the lowest EERs, exhibiting relative reductions of from approximately 7.7% (i.e., compared with _bmax_front_rept_) to 24.6% (i.e., compared with _bmax_end_rept_) on the male evaluation trials, and from 3.1% (i.e., compared with _bmax_front_const_) to 20.8% (i.e., compared with _bmean_end_rept_) on the female evaluation trials. This means that attention cannot perfectly remove the unnecessary information. When using the conventional methods, which cause information distortion, the attention weights can also be distorted because they were trained using the frames that have distorted information. Therefore, the proposed method is required to remove the information distortion. On the development trials, the _max_front_rept_ and _max_end_rept_ showed the lowest and second lowest EERs, respectively. The proposed method showed the third lowest EER with relative about 1.8% and 0.3% higher than the _max_front_rept_ and _max_end_rept_, respectively. It seems that the network trained with a sequence that includes some unnecessary information may be more suitable for the embedding from the sequence (that includes some unnecessary information) rather than the embedding from the distortion-free sequence. Table 3 shows the average execution time for extracting one embedding with/without attention. In our experiments, execution time is proportion to both computation amount and memory usage because we did not use the multi-processing method. Naturally, _max_, which requires the most amount of computation, shows the longest execution times. Also, in most cases, the methods with attention show longer execution times, relatively about 7.4%. _bmax_ showed relatively shorter execution times of approximately 88.8% than _max_, and _bmean_ showed relatively shorter execution times of approximately 66.7% than _mean_. Notice that there is a trade-off between _max/mean_ and _bmax/bmean_. Generally, _max/mean_ require more computation than _bmax/bmean_. Instead, _bmax/bmean_ require an overhead for truncation/padding because the target length is changed for every mini-batch.