# Parameter-Free Ordered Partial Match Alignment with Hidden State Time Warping
## 3 Results
### Speech Alignment & Verification Results  
For the speech task, we compared the performance of seven different parameter-free alignment systems. The first system (\"DTW-111\") was subsequence DTW with (query, reference) transitions \(\{(1,1),(0,1),(1,0)\}\) and corresponding transition weights \(\{1,1,1\}\). For each element \((i,j)\) on the optimal alignment path, the pairwise cost \(C[i,j]\) was imputed as a score to the \(ith\) frame in the query (i.e., the potentially tampered recording), and each query frame's score was compared to a threshold to determine if the frame was a match or a non-match. This threshold can be varied to sweep out an ROC curve. The second (\"DTW-121\") and third (\"DTW-112\") systems were the same as the first system but with transition weights \(\{1,2,1\}\) and \(\{1,1,2\}\), respectively. The fourth system (\"NW-threshold\") was Needleman-Wunsch alignment [30], which allows for regular \((1,1)\) transitions as well as \((0,1)\) and \((1,0)\) transitions with a fixed insertion/deletion penalty \(\gamma\) that does not depend on the pairwise cost matrix \(C\). Once the optimal alignment path was estimated, scores were imputed to the query frames and thresholded, as before. The fifth system (\"NW-skip\") was identical to the fourth system but simply classified insertions and deletions as non-matches and \((1,1)\) transitions as matches. The sixth system (\"HSTW-threshold\") used our proposed algorithm to estimate an optimal subsequence alignment path, imputed scores to each query frame based on the pairwise cost matrix, and then applied a threshold. The seventh system (\"HSTW-plane\") estimated the alignment with our proposed algorithm and simply classified path elements in the visible plane as matches and path elements in the hidden plane as non-matches. For each system above, hyperparameters were tuned on the training set.  
Figure 2 shows the frame classification results on the speech verification task for \(L=1,2,3,4\) s tampering durations. For all threshold-based systems (i.e., all systems except NW-skip and HSTW-plane), the bars indicate the EER of classification performance. Because NW-skip and HSTW-plane are not threshold-based systems, their performance is defined by a single point in the ROC plane, as shown in Figure 3 for \(L=2\) s tampering durations. For these two systems, an approximation of EER was estimated by averaging the false positive and false negative rate.  
There are three phenomena to notice in Figures 2 and 3. First, HSTW-plane shows the best classification performance by a wide margin. For example, for \(L=2\) s tampering operations, HSTW-plane achieved a 0.01% false positive rate and a 0.25% false negative rate, while the best non-HSTW system achieved a 3.04% equal error rate. Second, HSTW-plane greatly outperformed HSTW-threshold, even though both systems have identical alignment paths. This indicates that there is relevant information in the visible/hidden dimension that is not captured by the 2D alignment between the two sequences. Third, the performance of non-HSTW systems varied drastically based on the tampering duration, but that of the HSTW-based systems was much more stable.  
Figure 3: ROC curves for all systems on the speech alignment & verification task with a \(L=2\) s tampering duration. NW-skip and HSTW-plane are not threshold-based systems, so their performance is characterized by a single point.  
Figure 2: Results on the speech alignment & verification task. Bars indicate the equal error rate for classifying audio frames as matching vs. non-matching. Systems with \({}^{*}\) are not threshold-based and therefore do not sweep out a full ROC curve. For these systems, the bar indicates the mean values of the false positive and false negative rates.