# An Agent-Ensemble for Thresholded Multi-Target Classification
## 5 Experiments
### Classification of Ground Vehicles Using Acoustic Signatures
We estimate the predominant fundamental using an algorithm similar to the harmonic relationship detection algorithm described in [33]. The algorithm detects the presence of harmonic sets with fundamental frequencies between six and twenty Hz on a scan-by-scan basis by looking for a repeated pattern of narrowband energy at the correct harmonics within the spectrum. We smooth the resulting estimate using a median filter, and take the magnitude of the spectrum at the second through twelfth harmonics, normalized to have a maximum of one, as the feature vector for the scan. This results in eighty features per event. These features are suitable for this application due to their invariance to changes in the vehicle's speed [33]. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Vehicle** & **Number of Events** & **Number of Scans** \\ \hline
1 & 62 & 4960 \\
2 & 37 & 2960 \\
3 & 9 & 720 \\
4 & 27 & 2160 \\
5 & 39 & 3120 \\
6 & 37 & 2960 \\
7 & 7 & 560 \\
8 & 35 & 2800 \\
9 & 21 & 1680 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Number of closest point of arrival (CPA) events for each type of vehicle in the ACIDS dataset. Figure 3: Normalized magnitude spectrogram, in dB, for the first vehicular event in the ACIDS dataset. Vehicles 1, 2, 8, and 9 form a group of heavy tracked vehicles and we define these four vehicle types as our targets and the others as clutter. We perform ten-fold cross-validation. In each fold, we set aside ten percent of the events as test and the rest as training. We use the scans in the training data to train an ensemble of Gaussian RBF SVM classifier-agents, one for each target vs. clutter. We choose the SVM parameters for each classifier agent by ten fold cross-validation on the training data and also get unbiased data for combiner training by ten-fold cross-validation. We compare the proposed combiner with weighted-likelihood ratio normalization, where we model the likelihood using Platt probabilistic normalization, Neyman-Pearson combination using a joint Gaussian model, Neyman-Pearson combination using an independent Gaussian model, the any-combiner using Z, F, and EER normalization and a meta-SVM combiner using a non-linear Gaussian RBF SVM trained to classify the output of the combiners. We train the meta-SVM combiner using the same parameters and procedure with which we train the classifier agents. We repeat this experiment twenty-five times, randomizing the cross-validation indices each time, to get our final set of results. The SVM agents classify scans. In order to classify the events, we combine the scan-by-scan scores along the eighty scans for each event. Define \(c_{i},i=1,...,80\) as the combiner output along the eighty scans for an event. Assuming that the scans are independent, we combine the scores for an event to get a likelihood ratio of target vs. clutter according to:  
\[\Lambda=\sum_{i=1}^{80}\log\left(\frac{p(c_{i}|y=1)}{p(c_{i}|y=-1)}\right). \tag{27}\]  
The two Neyman-Pearson combiners as well as the proposed combiner give score outputs that are already in the form of (27), so we sum the log of the combiner outputs on scans to get the event score. In order to convert the Z-norm, F-norm, and EER-norm combiner outputs to a probability, we make the assumption that the output of the combiner is conditionally Gaussian. For the Meta-SVM, we train a Platt-scaling function for the Meta-SVM output and convert it to a likelihood ratio as in Section 4.4. Table 7 gives the PAUC, averaged over the twenty-five experiments, in the 0%-10% PFA operating region when classifying scans and events. The result shows that the proposed combiner with Platt weighted-likelihood normalization results in the highest PAUC in this region for both scans and events, and that the result is statistically significant at 95 % confidence.