# An Agent-Ensemble for Thresholded Multi-Target Classification
## 4 Proposed Agent Combiner
### Relationship to the Any-Combiner Rule and a Comparison of Normalization Functions  
If we replace the sum in (3) with a maximum, then we arrive at the any-combiner agent combination function that was originally proposed in [5]:  
\[\hat{y}(\mathbf{x})=\text{sign }\left(\max_{j=1,\ldots,L}f(\mathbf{x}_{j})- \alpha\right). \tag{16}\]  
The any-combiner given in (16) is more flexible than the proposed combiner in that it can use a number of different normalization functions. Several other score-normalization functions have been proposed in the biometrics literature for user verification. In user verification, a user claims an identity, and thus only one of the \(L\) agents is used at test time. Therefore, the complete ensemble is not evaluated at test time; however score-normalization is still useful for biometric verification so that each agent can be compared to a common threshold or so that the outputs of multiple biometric systems can be combined for different users in a consistent manner [18]. As these score normalization functions are still useful for our problem, we describe them here and evaluate their effectiveness in the experiments section. [PERSON] and [PERSON] [18] and [PERSON] [24] review score normalization functions and compare their use for biometric verification. Define \(\hat{\mu}_{j}^{(-1)}\) and \(\hat{\mu}_{j}^{(j)}\) as estimates of the mean value of \(x_{j}\) given clutter and target-type \(j\), respectively, and define \(\hat{\sigma}_{j}^{(-1)}\) and \(\hat{\sigma}_{j}(j)\) as the estimated standard deviations. The normalization functions that [18,24] analyze are the Z-norm  
\[f_{Z}(x_{j})=\frac{x_{j}-\hat{\mu}_{j}^{(-1)}}{\hat{\sigma}_{j}^{(-1)}}, \tag{17}\]  
F-norm  
\[f_{\text{F}}(x_{j})=\frac{x_{j}-\hat{\mu}_{j}^{(-1)}}{\hat{\mu}_{j}^{(j)}-\hat {\mu}_{j}^{(-1)}}, \tag{18}\]  
MS-LLR norm (model-specific log-likelihood ratio)\[f_{MSLLR}(x_{j})=\log\frac{\hat{p}(x_{j}|Y=j)}{\hat{p}(x_{j}|Y=-1)}, \tag{19}\]  
and EER-norm (equal error rate)  
\[f_{EER}(x_{j})=x_{j}-\Delta_{j}, \tag{20}\]  
where \(\Delta_{j}\) is an estimate of the score at which the probability of false positive equals the probability of false negative for agent \(j\). The Z-norm requires only clutter data for training, and we note that the Z-norm is a Gaussian parametric form of equal-PFA normalization proposed in [5]:  
\[f_{\text{EQ}PFA}(x_{j})=1-\hat{\text{F}}(x_{j}|Y=-1), \tag{21}\]  
where the latter term in (21) is an estimate of the conditional cumulative distribution function of agent \(j\)'s output given clutter. Both of these normalization functions are designed so that agents commit the same number of false alarms at any operating point. Thus, they can be motivated by a _fairness_ perspective if one views the system-level PFA rate as a resource that is shared among the agents. However, these normalization functions can result in lower system-level probabilities of detection as they do not consider the difference in probability of detection among the different agents. The F-norm, EER-norm and MS-LLR norm all use the target and clutter data to train the normalization functions. The F-norm, however, only uses the mean of the score distributions, and can thus perform poorly if the higher-order statistics of the score-outputs differ significantly. The EER norm considers second order statistics, but is designed only for a threshold of \(\alpha=0\). If this threshold is far from the desired operating point, then the EER norm will perform poorly. The MS-LLR norm is very similar to the proposed weighted likelihood ratio normalization; however, by not including the a-priori probabilities of the various target-types, this normalization will result in poor performance when the target sub-type distributions or risks are asymmetric. An additional normalization function that has been used previously in the literature for SVM agents is Platt probabilistic normalization [4; 5; 25].