# Forensic Speaker Comparison Using Evidence Interval in Full Bayesian Significance Test

Authors: Adelino P. Silva; Maurílio N. Vieira; Adriano V. Barbosa
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

This paper describes the application of a full Bayesian significance test (FBST) to compute evidence intervals in forensic speaker comparison (FSC). In the FBST approach, the challenge is to apply the test to a large number of observations and to formulate an equation to solve the test quickly. The contribution of the present work is that it proposes an application of the FBST to FSC and develops a method to calculate the FBST for the distribution of expected values (mean) with unknown variance without using Monte Carlo Markov chains (MCMC). Comparisons with other interval inference methodologies indicate that the evidence interval size is 49% greater than that computed with the Gosset approach. The evidence interval presented 71% fewer classification errors than the punctual inference did for the signal-to-noise ratio (SNR) of 17 dB.


---

## Body

## 1. Introduction

The main task in forensic speaker comparison (FSC) is to analyze two or more voice records to infer whether they come from the same speaker. FSC differs from biometric voice recognition in the hypothesis test approach and in the nature of the voice samples. In the FSC scenario, aquestioned-voice is compared to a known-voice, whereas in biometric recognition, the comparison is made among multiple speakers [1, 2].The questioned-voice (or voice evidence) is an audio recording accepted as a vestige or evidence in a criminal investigation. The questioned-voice may be recorded in different situations, such as lawful phone interception (wiretapping), recordings of face-to-face conversation, or audio broadcasting.In FSC, the hypothesisH0 considers that both the questioned- and known-voices come from different speakers, whereas H1 assumes that the questioned- and known-voices come from the same speaker.However, the “individualization” that the hypotheses above propose has been considered a fallacy. This individualization assumes that the result of the confrontation between the questioned and standard voice is unique, withouta priori probability and without repeating the test for the entire population [3, 4]. According to Saks and Koehler [3], the most reasonable hypotheses would be(1)H:H0:the features of the questioned−voice are not compatiblewith the features of the known−voice,H1:the features of the questioned−voice are compatible withthe features of the known−voice.Punctual inference in FSC is based on a score of (dis)similarity [5–7]. Interval inference is a tradeoff between precision and confidence because it sacrifices some precision of the estimate by moving from a point to a range, but results in greater confidence that the statement is correct (inside interval) [8] (pp. 418).Reports on interval inference in automatic speaker recognition (ASR) began with Bisani and Ney [9], who used bootstrap [10] to compute confidence intervals. Subsequently, Campbell et al. [11] computed confidence intervals using multilayer perceptron (MLP) based on statistical entropy. Later, Koval and Lokhanova [12] used a sigmoid function to approximate the a posteriori probability PH0x⟶, where x⟶is the voice data and H0 is the null hypothesis, using Platt scaling [13] and estimated credibility intervals. The credibility interval can also be computed by empirical methods (Morrison et al. [14]).The present work proposes the application of the full Bayesian significance test (FBST) to computeevidence intervals of FSC. This proposal aims to obtain the same confidence of capturing the parameter of interest in FSC and to reduce type I errors, reinforcing the legal aphorism of Absolvere nocentem satius est, quam condemnare innocentem. One of the motivations of this work, among others, is to establish a confidence limit of the automatic speaker comparison techniques, primarily when used as a support to quantify an FSC [15].Applications of the FBST to FSC were not found during the bibliographic survey in the development of this research. Thus, the main contribution of this work is that it proposes an application of the FBST to FSC and develops a method to calculate the FBST for the distribution of the expected value (mean) with unknown variance without using Monte Carlo Markov chains (MCMC).The results indicate that the application of the FBST to FSC can improve the evaluation of results by the LR framework, reducing the occurrence of type I errors. The FBST also supports decisions on multispeaker comparisons.The paper is organized as follows.Section 1 presents the FBST and our proposed improvements and proposes adaptations for FSC (the GMM-UBM method was chosen because it presented more satisfactory results in previous experiments than the i-vector- and x-vector-based methods with deep neural networks (DNN). These experiments were performed with database in Portuguese, quoted in this article, and with voices provided by the Civil Police of Minas Gerais (Brazil) forensic sector. The result of this experiment is in the process of being published). Section 1 compares the evidence interval to other methods. Section 1 presents the conclusion and future research directions.


## 2. Evidence FSC Interval with the FBST

### 2.1. Interval Inference in FSC

In classical FSC, the comparison is performed between features of the questioned-voice,x⟶Q, and features of the known-voice, x⟶K. The features of the universal background model (UBM), x⟶UBM, represent the average speaker [5].The LR can be computed using a GMM-UBM. In this case, the LR equivalent scoreLRx⟶Q is computed as follows:(2)LRx⟶Q=px⟶QλKpx⟶QλUBM≤ζ0,H0is not rejected,>ζ0,H0is rejected.Furthermore,px⟶QλK and px⟶QλUBM are, respectively, the evaluation of the data x⟶Q of the GMM of the known-voice, λK, and of the UBM λUBM.The GMM-UBM is a methodology applied to voice comparison [7, 16, 17]. In the first studies [5, 18], the GMM-UBM methodology was applied using Mel-frequency cepstrum coefficients (MFCC).The first step in the GMM-UBM procedure is to compute the GMM of the known-voice,λK, and of the UBM λUBM, which can be computed using the expectation-maximization (EM) algorithm [5]. In the second step, the Score of the comparison LRx⟶Q is obtained as a ratio between two likelihoods: the questioned-voice x⟶Q versus the known-voice λK and the questioned-voice versus the UBM model λUBM.Thescore proposed by Reynolds et al. [5] is the sample mean of the log-likelihood ratio (LLR) over T speech frames:(3)LRx⟶Q=px⟶QλKpx⟶QλUBMlog⟶LLRx⟶Q=logpx⟶QλKpx⟶QλUBM.Because the featuresx⟶Q=xQ0,xQ1,…,xQT−1are not independent and not identically distributed (i.i.d.), the resulting values are not, technically, a likelihood ratio. Normalization by the number of frames, T, also removes the duration effects from the log-likelihood value. However, the LLRx⟶Q of equation (3) allows us to include an interval-based inference.Calculating the interval inference is possible empirically or analytically over the sample space. The widespread empirical approaches include bootstrap [10], jackknife [19], and the method proposed by Morrison et al. [14]. One possible analytical method uses the t-Student distribution of Gosset [20, 21]:(4)tα/2,T−1σ^T≤LLRx⟶Q−μ≤t1−α/2,T−1σ^T,where σ^ is the sample standard deviation, µ is the expected value of LLRx⟶Q,andtα/2,T−1 is a t-Student distribution with significance α and T−1 degrees of freedom.In Section1, we compare our evidence interval computed using the FBST to Morrison’s credibility/confidence intervals, the analytical method in equation (4).Morrison’s approach [14, 22] uses two samples of voice per speaker and measures the LLR from the vowel formants. In these works, the credibility intervals were computed from raw data rather than from a statistic such as the mean. We propose a small modification to Morrison’s approach such that the computation is based on the sample mean instead of the raw data.


### 2.2. Full Bayesian Significance Test

The FBST can be used to compute evidence against a precise hypothesisLLRx⟶Q=η, where η is a value in the parametric space of LLR of equation (2).The FBST [23, 24] is a coherent Bayesian significance test for sharp hypotheses. The test is based on an evidence concept value, whose original definition was motivated by practical, juridical, and epistemological requirements. Consider the parametric space Θ and a subset θ∈Θ⊆ℝnand a precise (null) hypothesis H0 that the parameter lies in the null set, defined by the inequality gθ and equality hθ constraints given by the vector functions g and h in the parameter space:(5)ΘH=θ∈Θgθ≤0∧hθ=0.For the experimental datax⟶, the a posteriori density of a precise hypothesis is proportional to the product of the likelihood and the a priori density [25]:(6)fnθx⟶∝fθℒθx⟶,where fθis an a priori density and ℒθx⟶ is the likelihood. The points of the parameter space with highest “surprise” in the null set H0 are(7)θ∗=arg maxθ∈ΘHfnθx⟶,while the highest relative surprise set (HRSS), T∗, is(8)T∗=θ∈Θfnθx⟶>fnθ∗x⟶.TheBayesian evidence value against H0is the a posteriori probability of the “tangent” set; that is,(9)ev¯=Prθ∈T∗x⟶=∫T∗fnθx⟶dx,where Prθ∈T∗x⟶ is the probability that the parameter θ is inside T∗. The e-value associated with the FBST is(10)e­value=1−Prθ∈T∗x⟶.Thee-value is a probability in the parameter space (μ and ρ), whereas the p value is a probability in the sample space [26]. In Section 1, we use the e-value and ev¯ (Bayesian evidence value against H0) to compute the evidence interval on FSC using hypothesis H:LLRx⟶Q=η.


#### 2.2.1. Improvement of the FBST over the Mean with an Unknown Variance

This section describes a method to compute the FBST for a distribution of the mean (expected value) with an unknown variance. To lower the computational cost, we focus on a mostly analytical development. This is important in order to limit the computation time of thee-value over the η-space.Consider a normally distributed samplex∈X with n i.i.d. observations, Xμ,1/ρ, where µ is the expected value and ρ=1/σ2 is the precision. The minimal sufficient statistic could be the sample mean x¯ and total sum of squares Q=∑i=1nxi−x¯2. The likelihood function for μ∈−∞,∞and ρ∈0,∞ [26] is(11)ℒμ,ρn,x¯,Q∝ρn/2e−ρQ/21+n/Qμ−x¯2.Taking thea priori noninformative distribution pμ,ρ=dμdρ/ρ [27], the a posteriori probability density function (PDF) is [26](12)Pnμ,ρn,x¯,Q=cρn/2−1e−ρQ/21+n/Qμ−x¯2,where(13)c=Qn−1/2n2n/2πΓn−1/2,and c is calculated such that the integral over equation (12) is 1. The gradient is given by the partial derivatives of Pnμ,ρ(henceforth, the we write the PDF Pnμ,ρ|n,x¯,Q as Pnμ,ρ) lead to the maximum Pμ∗,ρ∗:(14)μ∗=x¯,ρ∗=n−2Q.Figure1 shows an example of the FBST evaluation over H0:μ=0. The bell-shaped surface is Pnμ,ρ and the solid black line is the restriction of the null hypothesis μ=0. The maximum value of the black line delimits the “tangent” T∗set, represented as a dash-dot line. The dotted line is the restriction Pnμ,ρ=ρ∗.Figure 1
Probability density functionPnμ,ρn,x¯,Qshowing the restriction of the null hypothesis and the “tangent” set. The bell shape is Pnμ,ρ, while the solid black line is the restriction of the null hypothesis H0:μ=0. The maximum value of the black line delimits the “tangent” set T∗ (dash-dot line). The dotted line is the restriction Pnμ,ρ=ρ∗.The evidence against the null hypothesisH0:μ=η=0 is evaluated by equation (9). Main works on the FBST over the distribution of a mean with an unknown variance [26, 28, 29] use MCMC to solve the integral of fnin equation (9). However, specifically for equation (12), it shows that the “tangent” set T∗ has extreme points ρA, ρB, ρCe and ρD(as in Figure 2), where(15)ρA=n−2Q1+n/Qη−x¯2,ρB=n−2Q1+n/Q2x¯−η−x¯2=ρA.Figure 2
Boundary points of the “tangent” setT∗. The solid black line is the restriction of the null hypothesis H0:μ=0. The tangential set T∗ is represented by the dash-dot line. The dotted line is the restriction Pnμ,ρ=ρ∗.MakingPnx¯,ρ=Pnη,ρA for equation (12) results in(16)cρn/2−1e−ρQ/2=cρAn/2−1e−ρAQ/21+n/Qη−x¯2,and grouping variables and taking the natural logarithm in both sides yields(17)ρ−n−2Qlogρ+n−2QlogρA−ρA1+nQη−x¯2=0,with the roots being(18)ρD=exp−W−1e−β/αα+βα,ρC=exp−W0e−β/αα+βα,where Wn⋅ is the Lambert-W function [30]. By the symmetry of T∗ over the µ- axis, we can compute the evidence ev¯ by(19)ev¯=2∫ρCρD∫x¯μρcρn/2−1e−ρQ/21+n/Qμ−x¯2dμdρ,where μρ is the contour function (from any boundary) on the µ-axis of the “tangent” set T∗(Figure 2). The contour of T∗ can be defined as(20)Pnμ,ρ=cρn/2−1e−ρQ/21+n/Qμ−x¯2=Pn∗↓logμ2−2x¯μ+x¯2+Qn−2ρnn−22logρ−logPn∗c=0,where Pn∗=cρAn/2−1e−ρAQ/21+n/Qη−x¯2. The roots of equation (20) in µ define the left and right sides of the contour (see Figure 2):(21)μρ=x¯±2ρnn−22logρ−Pn∗c−Qn.Note thatμρ is a contour for values greater and less than x¯. By symmetry, we compute equation (19) as(22)where erf⋅ is the error function. We can simplify the argument of this function as(23)νρ,η=n−22logρρA−Q2ρ−ρA+ρAn2η−x¯2,where ρA is the inferior limit of ρ, and η is the hypothesis test H0:μ=η. Thus, we can rewrite equation (19) as the one-dimensional integral:(24)ev¯=1Γn−1/2Q2n−1/2×∫ρCρDρn−3/2e−ρQ/2erfνρ,ηdρ.The integral in equation (24) does not need MCMC techniques, thus demanding less computational effort than equation (9) does.


### 2.3. Proposed Method

This section proposes a method to compute the evidence interval with a Bayesian evidence levelα, which can be computed using equation (24). The result in the GMM-UBM scenario is the sample mean LLRx⟶Q of the time series LLRxtQ, as equation (3) shows, on the parametric space η.Consider the time seriesLLRxtQ with a parametric mean (expected value) of μ, precision ρ, and sample mean LLRx⟶Q. From this, it is possible to define the evidence interval of µ as the subspace ηL≤μ≤ηH, where ηL and ηH are values above and below LLRx⟶Q, respectively. The Bayesian evidence ev¯ against the precise hypotheses H:μ=ηL and H:μ=ηH is 1−α (see equation (24)).Outside this range of the LLR,ηL≤LLRx⟶Q≤ηH, the evidence (e-value computed by the FBST) that the parametric mean (μ) is higher than ηH or lower than ηLis less than α.We are aware that the definition above does not fit the traditional confidence (or credibility) interval as defined in [31]. However, it is an analytical method based on the parameter space and represents the limits of evidence that the sample can provide Bayesian evidence (“significance”) of 1−α.For example, consider that the comparison between a questioned-voice and a known-voice generates a time seriesLLRxQt, where the values of frames x⟶Q=xQ0,xQ1,…,xQT−1 in equation (2) are used. Figure 3 shows the statistical distribution of these LLR values on the normalized histogram (Norm. Hist.) in the left panel. In this panel, the solid light gray line is the empirical PDF (emp. PDF) and the small circle over this curve indicates the sample mean LLRx⟶Q. The dash-dotted rectangle on the left graph is the region on the right graph. The sample mean of the LLRxQt series is LLRx⟶Q≈−0.8 Np (nepers) (neper is the natural logarithm of ratios, named after John Napier).Figure 3
The panel on the left shows the normalized histogram ofLLRxQt occurrences and the empirical PDF (thick line). On the same panel, the dashed rectangle indicates the region shown in the panel on the right. In this panel, the bell-shaped dark gray solid line indicates the e-value curve and the horizontal solid black error bar indicates the evidence interval and the sample mean.
(a)
(b)The evaluation of the hypothesisH:LLRx⟶Q=η along the variable η in the LLR space with the FBST (equation (24) yields the e-value curve. The variation of η values results in the e-value curve (ev-curve, solid dark gray) indicated in the right graph of Figure 3. This curve is computed by sampling the η space and solving equation (24) for each sample. On this graph, the horizontal dash-dotted line (ev = 0.05) indicates the Bayesian evidence (significance) α=0.05 (evidence value against hypothesis ev¯=95% or e-value = 0.05). The horizontal solid black error bar ev>0.05 indicates the evidence interval and the sample mean.


## 2.1. Interval Inference in FSC

In classical FSC, the comparison is performed between features of the questioned-voice,x⟶Q, and features of the known-voice, x⟶K. The features of the universal background model (UBM), x⟶UBM, represent the average speaker [5].The LR can be computed using a GMM-UBM. In this case, the LR equivalent scoreLRx⟶Q is computed as follows:(2)LRx⟶Q=px⟶QλKpx⟶QλUBM≤ζ0,H0is not rejected,>ζ0,H0is rejected.Furthermore,px⟶QλK and px⟶QλUBM are, respectively, the evaluation of the data x⟶Q of the GMM of the known-voice, λK, and of the UBM λUBM.The GMM-UBM is a methodology applied to voice comparison [7, 16, 17]. In the first studies [5, 18], the GMM-UBM methodology was applied using Mel-frequency cepstrum coefficients (MFCC).The first step in the GMM-UBM procedure is to compute the GMM of the known-voice,λK, and of the UBM λUBM, which can be computed using the expectation-maximization (EM) algorithm [5]. In the second step, the Score of the comparison LRx⟶Q is obtained as a ratio between two likelihoods: the questioned-voice x⟶Q versus the known-voice λK and the questioned-voice versus the UBM model λUBM.Thescore proposed by Reynolds et al. [5] is the sample mean of the log-likelihood ratio (LLR) over T speech frames:(3)LRx⟶Q=px⟶QλKpx⟶QλUBMlog⟶LLRx⟶Q=logpx⟶QλKpx⟶QλUBM.Because the featuresx⟶Q=xQ0,xQ1,…,xQT−1are not independent and not identically distributed (i.i.d.), the resulting values are not, technically, a likelihood ratio. Normalization by the number of frames, T, also removes the duration effects from the log-likelihood value. However, the LLRx⟶Q of equation (3) allows us to include an interval-based inference.Calculating the interval inference is possible empirically or analytically over the sample space. The widespread empirical approaches include bootstrap [10], jackknife [19], and the method proposed by Morrison et al. [14]. One possible analytical method uses the t-Student distribution of Gosset [20, 21]:(4)tα/2,T−1σ^T≤LLRx⟶Q−μ≤t1−α/2,T−1σ^T,where σ^ is the sample standard deviation, µ is the expected value of LLRx⟶Q,andtα/2,T−1 is a t-Student distribution with significance α and T−1 degrees of freedom.In Section1, we compare our evidence interval computed using the FBST to Morrison’s credibility/confidence intervals, the analytical method in equation (4).Morrison’s approach [14, 22] uses two samples of voice per speaker and measures the LLR from the vowel formants. In these works, the credibility intervals were computed from raw data rather than from a statistic such as the mean. We propose a small modification to Morrison’s approach such that the computation is based on the sample mean instead of the raw data.


## 2.2. Full Bayesian Significance Test

The FBST can be used to compute evidence against a precise hypothesisLLRx⟶Q=η, where η is a value in the parametric space of LLR of equation (2).The FBST [23, 24] is a coherent Bayesian significance test for sharp hypotheses. The test is based on an evidence concept value, whose original definition was motivated by practical, juridical, and epistemological requirements. Consider the parametric space Θ and a subset θ∈Θ⊆ℝnand a precise (null) hypothesis H0 that the parameter lies in the null set, defined by the inequality gθ and equality hθ constraints given by the vector functions g and h in the parameter space:(5)ΘH=θ∈Θgθ≤0∧hθ=0.For the experimental datax⟶, the a posteriori density of a precise hypothesis is proportional to the product of the likelihood and the a priori density [25]:(6)fnθx⟶∝fθℒθx⟶,where fθis an a priori density and ℒθx⟶ is the likelihood. The points of the parameter space with highest “surprise” in the null set H0 are(7)θ∗=arg maxθ∈ΘHfnθx⟶,while the highest relative surprise set (HRSS), T∗, is(8)T∗=θ∈Θfnθx⟶>fnθ∗x⟶.TheBayesian evidence value against H0is the a posteriori probability of the “tangent” set; that is,(9)ev¯=Prθ∈T∗x⟶=∫T∗fnθx⟶dx,where Prθ∈T∗x⟶ is the probability that the parameter θ is inside T∗. The e-value associated with the FBST is(10)e­value=1−Prθ∈T∗x⟶.Thee-value is a probability in the parameter space (μ and ρ), whereas the p value is a probability in the sample space [26]. In Section 1, we use the e-value and ev¯ (Bayesian evidence value against H0) to compute the evidence interval on FSC using hypothesis H:LLRx⟶Q=η.


### 2.2.1. Improvement of the FBST over the Mean with an Unknown Variance

This section describes a method to compute the FBST for a distribution of the mean (expected value) with an unknown variance. To lower the computational cost, we focus on a mostly analytical development. This is important in order to limit the computation time of thee-value over the η-space.Consider a normally distributed samplex∈X with n i.i.d. observations, Xμ,1/ρ, where µ is the expected value and ρ=1/σ2 is the precision. The minimal sufficient statistic could be the sample mean x¯ and total sum of squares Q=∑i=1nxi−x¯2. The likelihood function for μ∈−∞,∞and ρ∈0,∞ [26] is(11)ℒμ,ρn,x¯,Q∝ρn/2e−ρQ/21+n/Qμ−x¯2.Taking thea priori noninformative distribution pμ,ρ=dμdρ/ρ [27], the a posteriori probability density function (PDF) is [26](12)Pnμ,ρn,x¯,Q=cρn/2−1e−ρQ/21+n/Qμ−x¯2,where(13)c=Qn−1/2n2n/2πΓn−1/2,and c is calculated such that the integral over equation (12) is 1. The gradient is given by the partial derivatives of Pnμ,ρ(henceforth, the we write the PDF Pnμ,ρ|n,x¯,Q as Pnμ,ρ) lead to the maximum Pμ∗,ρ∗:(14)μ∗=x¯,ρ∗=n−2Q.Figure1 shows an example of the FBST evaluation over H0:μ=0. The bell-shaped surface is Pnμ,ρ and the solid black line is the restriction of the null hypothesis μ=0. The maximum value of the black line delimits the “tangent” T∗set, represented as a dash-dot line. The dotted line is the restriction Pnμ,ρ=ρ∗.Figure 1
Probability density functionPnμ,ρn,x¯,Qshowing the restriction of the null hypothesis and the “tangent” set. The bell shape is Pnμ,ρ, while the solid black line is the restriction of the null hypothesis H0:μ=0. The maximum value of the black line delimits the “tangent” set T∗ (dash-dot line). The dotted line is the restriction Pnμ,ρ=ρ∗.The evidence against the null hypothesisH0:μ=η=0 is evaluated by equation (9). Main works on the FBST over the distribution of a mean with an unknown variance [26, 28, 29] use MCMC to solve the integral of fnin equation (9). However, specifically for equation (12), it shows that the “tangent” set T∗ has extreme points ρA, ρB, ρCe and ρD(as in Figure 2), where(15)ρA=n−2Q1+n/Qη−x¯2,ρB=n−2Q1+n/Q2x¯−η−x¯2=ρA.Figure 2
Boundary points of the “tangent” setT∗. The solid black line is the restriction of the null hypothesis H0:μ=0. The tangential set T∗ is represented by the dash-dot line. The dotted line is the restriction Pnμ,ρ=ρ∗.MakingPnx¯,ρ=Pnη,ρA for equation (12) results in(16)cρn/2−1e−ρQ/2=cρAn/2−1e−ρAQ/21+n/Qη−x¯2,and grouping variables and taking the natural logarithm in both sides yields(17)ρ−n−2Qlogρ+n−2QlogρA−ρA1+nQη−x¯2=0,with the roots being(18)ρD=exp−W−1e−β/αα+βα,ρC=exp−W0e−β/αα+βα,where Wn⋅ is the Lambert-W function [30]. By the symmetry of T∗ over the µ- axis, we can compute the evidence ev¯ by(19)ev¯=2∫ρCρD∫x¯μρcρn/2−1e−ρQ/21+n/Qμ−x¯2dμdρ,where μρ is the contour function (from any boundary) on the µ-axis of the “tangent” set T∗(Figure 2). The contour of T∗ can be defined as(20)Pnμ,ρ=cρn/2−1e−ρQ/21+n/Qμ−x¯2=Pn∗↓logμ2−2x¯μ+x¯2+Qn−2ρnn−22logρ−logPn∗c=0,where Pn∗=cρAn/2−1e−ρAQ/21+n/Qη−x¯2. The roots of equation (20) in µ define the left and right sides of the contour (see Figure 2):(21)μρ=x¯±2ρnn−22logρ−Pn∗c−Qn.Note thatμρ is a contour for values greater and less than x¯. By symmetry, we compute equation (19) as(22)where erf⋅ is the error function. We can simplify the argument of this function as(23)νρ,η=n−22logρρA−Q2ρ−ρA+ρAn2η−x¯2,where ρA is the inferior limit of ρ, and η is the hypothesis test H0:μ=η. Thus, we can rewrite equation (19) as the one-dimensional integral:(24)ev¯=1Γn−1/2Q2n−1/2×∫ρCρDρn−3/2e−ρQ/2erfνρ,ηdρ.The integral in equation (24) does not need MCMC techniques, thus demanding less computational effort than equation (9) does.


## 2.2.1. Improvement of the FBST over the Mean with an Unknown Variance

This section describes a method to compute the FBST for a distribution of the mean (expected value) with an unknown variance. To lower the computational cost, we focus on a mostly analytical development. This is important in order to limit the computation time of thee-value over the η-space.Consider a normally distributed samplex∈X with n i.i.d. observations, Xμ,1/ρ, where µ is the expected value and ρ=1/σ2 is the precision. The minimal sufficient statistic could be the sample mean x¯ and total sum of squares Q=∑i=1nxi−x¯2. The likelihood function for μ∈−∞,∞and ρ∈0,∞ [26] is(11)ℒμ,ρn,x¯,Q∝ρn/2e−ρQ/21+n/Qμ−x¯2.Taking thea priori noninformative distribution pμ,ρ=dμdρ/ρ [27], the a posteriori probability density function (PDF) is [26](12)Pnμ,ρn,x¯,Q=cρn/2−1e−ρQ/21+n/Qμ−x¯2,where(13)c=Qn−1/2n2n/2πΓn−1/2,and c is calculated such that the integral over equation (12) is 1. The gradient is given by the partial derivatives of Pnμ,ρ(henceforth, the we write the PDF Pnμ,ρ|n,x¯,Q as Pnμ,ρ) lead to the maximum Pμ∗,ρ∗:(14)μ∗=x¯,ρ∗=n−2Q.Figure1 shows an example of the FBST evaluation over H0:μ=0. The bell-shaped surface is Pnμ,ρ and the solid black line is the restriction of the null hypothesis μ=0. The maximum value of the black line delimits the “tangent” T∗set, represented as a dash-dot line. The dotted line is the restriction Pnμ,ρ=ρ∗.Figure 1
Probability density functionPnμ,ρn,x¯,Qshowing the restriction of the null hypothesis and the “tangent” set. The bell shape is Pnμ,ρ, while the solid black line is the restriction of the null hypothesis H0:μ=0. The maximum value of the black line delimits the “tangent” set T∗ (dash-dot line). The dotted line is the restriction Pnμ,ρ=ρ∗.The evidence against the null hypothesisH0:μ=η=0 is evaluated by equation (9). Main works on the FBST over the distribution of a mean with an unknown variance [26, 28, 29] use MCMC to solve the integral of fnin equation (9). However, specifically for equation (12), it shows that the “tangent” set T∗ has extreme points ρA, ρB, ρCe and ρD(as in Figure 2), where(15)ρA=n−2Q1+n/Qη−x¯2,ρB=n−2Q1+n/Q2x¯−η−x¯2=ρA.Figure 2
Boundary points of the “tangent” setT∗. The solid black line is the restriction of the null hypothesis H0:μ=0. The tangential set T∗ is represented by the dash-dot line. The dotted line is the restriction Pnμ,ρ=ρ∗.MakingPnx¯,ρ=Pnη,ρA for equation (12) results in(16)cρn/2−1e−ρQ/2=cρAn/2−1e−ρAQ/21+n/Qη−x¯2,and grouping variables and taking the natural logarithm in both sides yields(17)ρ−n−2Qlogρ+n−2QlogρA−ρA1+nQη−x¯2=0,with the roots being(18)ρD=exp−W−1e−β/αα+βα,ρC=exp−W0e−β/αα+βα,where Wn⋅ is the Lambert-W function [30]. By the symmetry of T∗ over the µ- axis, we can compute the evidence ev¯ by(19)ev¯=2∫ρCρD∫x¯μρcρn/2−1e−ρQ/21+n/Qμ−x¯2dμdρ,where μρ is the contour function (from any boundary) on the µ-axis of the “tangent” set T∗(Figure 2). The contour of T∗ can be defined as(20)Pnμ,ρ=cρn/2−1e−ρQ/21+n/Qμ−x¯2=Pn∗↓logμ2−2x¯μ+x¯2+Qn−2ρnn−22logρ−logPn∗c=0,where Pn∗=cρAn/2−1e−ρAQ/21+n/Qη−x¯2. The roots of equation (20) in µ define the left and right sides of the contour (see Figure 2):(21)μρ=x¯±2ρnn−22logρ−Pn∗c−Qn.Note thatμρ is a contour for values greater and less than x¯. By symmetry, we compute equation (19) as(22)where erf⋅ is the error function. We can simplify the argument of this function as(23)νρ,η=n−22logρρA−Q2ρ−ρA+ρAn2η−x¯2,where ρA is the inferior limit of ρ, and η is the hypothesis test H0:μ=η. Thus, we can rewrite equation (19) as the one-dimensional integral:(24)ev¯=1Γn−1/2Q2n−1/2×∫ρCρDρn−3/2e−ρQ/2erfνρ,ηdρ.The integral in equation (24) does not need MCMC techniques, thus demanding less computational effort than equation (9) does.


## 2.3. Proposed Method

This section proposes a method to compute the evidence interval with a Bayesian evidence levelα, which can be computed using equation (24). The result in the GMM-UBM scenario is the sample mean LLRx⟶Q of the time series LLRxtQ, as equation (3) shows, on the parametric space η.Consider the time seriesLLRxtQ with a parametric mean (expected value) of μ, precision ρ, and sample mean LLRx⟶Q. From this, it is possible to define the evidence interval of µ as the subspace ηL≤μ≤ηH, where ηL and ηH are values above and below LLRx⟶Q, respectively. The Bayesian evidence ev¯ against the precise hypotheses H:μ=ηL and H:μ=ηH is 1−α (see equation (24)).Outside this range of the LLR,ηL≤LLRx⟶Q≤ηH, the evidence (e-value computed by the FBST) that the parametric mean (μ) is higher than ηH or lower than ηLis less than α.We are aware that the definition above does not fit the traditional confidence (or credibility) interval as defined in [31]. However, it is an analytical method based on the parameter space and represents the limits of evidence that the sample can provide Bayesian evidence (“significance”) of 1−α.For example, consider that the comparison between a questioned-voice and a known-voice generates a time seriesLLRxQt, where the values of frames x⟶Q=xQ0,xQ1,…,xQT−1 in equation (2) are used. Figure 3 shows the statistical distribution of these LLR values on the normalized histogram (Norm. Hist.) in the left panel. In this panel, the solid light gray line is the empirical PDF (emp. PDF) and the small circle over this curve indicates the sample mean LLRx⟶Q. The dash-dotted rectangle on the left graph is the region on the right graph. The sample mean of the LLRxQt series is LLRx⟶Q≈−0.8 Np (nepers) (neper is the natural logarithm of ratios, named after John Napier).Figure 3
The panel on the left shows the normalized histogram ofLLRxQt occurrences and the empirical PDF (thick line). On the same panel, the dashed rectangle indicates the region shown in the panel on the right. In this panel, the bell-shaped dark gray solid line indicates the e-value curve and the horizontal solid black error bar indicates the evidence interval and the sample mean.
(a)
(b)The evaluation of the hypothesisH:LLRx⟶Q=η along the variable η in the LLR space with the FBST (equation (24) yields the e-value curve. The variation of η values results in the e-value curve (ev-curve, solid dark gray) indicated in the right graph of Figure 3. This curve is computed by sampling the η space and solving equation (24) for each sample. On this graph, the horizontal dash-dotted line (ev = 0.05) indicates the Bayesian evidence (significance) α=0.05 (evidence value against hypothesis ev¯=95% or e-value = 0.05). The horizontal solid black error bar ev>0.05 indicates the evidence interval and the sample mean.


## 3. Comparison with Other Methods

This section presents an experiment and a case study involving the range of evidence. We conducted training and testing stage with a voice data set CEFALA-1 [32], containing 104 speakers (55 men and 49 women) recorded with five microphones (generating 520 records). The validation step used 50 recordings that do not belong to the corpus CEFALA-1. This validation emulates an open-set database in speaker comparison.We designed an experiment to compare the proposed interval inference method with other methods used in FSC. The experiment used 104 voices narrowband filtered (4th order butterworth) in the 300–3500 Hz range and resampled to 8 kHz, compatible with the Brazilian mobile phone system.In order to compare the various interval inference methods, we need to use the speech database to define the known-voice and questioned-voice sets. We do this as follows. For each subject 50% of voice content was used as known-voice and 50% as questioned-voice, both in the CEFALA-1 corpus and in the validation recordings.In order to emulate forensic conditions, both the known-voice and questioned-voice data are subject to 3 types of degradation. First, the data are contaminated with pink noise at the following SNR levels: 25 dB, 23 dB, 20 dB, 17 dB, 15 dB, and 12 dB. Next, the data are encoded and then decoded by a GSM 06.60 codec [33]. Finally, the data are run through a narrowband filter (300–3500 Hz).The features were extracted with MFCCcn using 13 critical bands (filters), a frame length of 25ms, and frame step of 10ms. The features include delta Δcn and delta-delta Δ2cn. We used Sonh’s [34] method for voice activity detection (VAD) to identify the voiced frames.The methods used to compute interval inference (significanceα=0.05) wereGosset: confidence interval computed by equation (4)Morrison: empirical credibility interval computed by combining thek-nearest neighborhood (KNN) with the linear regression, as described by Morrison [14]FBST: the proposed method that computes the evidence as a subspace of the parametric space, where thee-value is αFigure4 presents examples of the interval inference. In the figure, we show the LLR values along the horizontal axis. The inference intervals are shown as horizontal lines, with the circles indicating the mean values and dot-dashed vertical line indicating the decision threshold.Figure 4
Scenarios of the inference intervals. The dot-dashed vertical line indicates the decision threshold. The light gray line indicates a same-speaker comparison, and dark gray indicates a different-speaker. The scenarios are (a) correct comparison ou “Right comp,” (b) an intermediate or “middle result” region where the comparison threshold is within the inference interval, and (c) comparison error (Type I or Type II) ou “Miss comp.”The horizontal light gray line indicates a same-speaker comparison, and dark gray indicates a different-speaker comparison. The scenarios are (a) correct comparison, (b) an intermediate region where the comparison threshold is within the inference interval, and (c) comparison error (Type I or Type II).We used the method proposed by Morrison et al. [14] to compute the credibility interval over the data themselves, not over the mean (expected value) of the data. Morrison’s method was adapted to compute the mean of 50 subsamples with replacement (similar to bootstrap [10]).We evaluated the performance of each interval inference method based on results presented in Figure4. We expected that a comparison between the GMM model of a given speaker and a set of features coming from that speaker (same-speaker comparison hereafter) results in a higher LLR value than a comparison between that same GMM model and a set of features coming from a different speaker (different-speaker comparison hereafter). The training and testing stage, using only samples from the CEFALA-1 corpus with contaminations between 12 and 25 dB, presented an equal error rate (EER) of 8.1% with threshold at LLR = 0.25 Np. The results presented below cover the test and validation steps.Figure5 shows the number of correct classifications in scenario (a). The occurrences of correct classifications for the evidence interval (vertical light gray bar) is smaller than that of other methods (interval and punctual). The comparisons of the best interval methods yield values of 84.0% against 84.4% for SNR 12 dB, 84.4% against 88.6% for 15 dB, and less than 1% for the other SNR values. These values represent a loss of the accuracy of less than 0.5% compared to interval inference. Compared to the punctual inference, the loss in the accuracy is less than 1.6% for the other SNR values.Figure 5
Percentage of correct classifications, scenario (a), for each method at different SNR values.The intermediate results, in which the intervals overlap, are exemplified in Figure4 by comparisons (b). These scenarios are deemed inconclusive and represent an In dubio pro reo condition, meaning that a defendant should not be convicted when doubts remain about his or her guilt (association between questioned- and known-voices).In the punctual inference, scenario (b) does not occur, and there is no transition region. Thus, in the interval inference, scenarios (a) and (c) are decisive, and the intermediate scenario, (b), indicates that the results have some equivalence; that is, there is a chance that the comparison between different speakers will be larger (or smaller) than the comparison between the same speakers.Figure6 shows the comparison results for various interval inference methods (Gosset, Morrison’s method, and FBST). The results are grouped by the SNR level. The panel indicates the percentage of inconclusive interval inferences (b), wrong interval inferences (c), and punctual error inferences (dashed vertical line).Figure 6
Results from the interval inference method. Each chart—grouped by SNR—shows the percentage of scenarios described in Figure4. The horizontal bars indicate the percentages by the method, while the dark gray-dashed vertical line indicates the error percentage of the inference without an interval. (a) SNR 12 dB. (b) SNR 15 dB. (c) SNR 17 dB. (d) SNR 20 dB. (e) SNR 23 dB. (f) SNR 25 dB.
(a)
(b)
(c)
(d)
(e)
(f)Compared to the punctual inference (dashed vertical line), the evidence interval computed by the FBST (horizontal light gray bar) reduces the number of wrong inferences in 1.6%, 1.1%, 0.9%, 0.7%, 0.6%, and 0.4%, respectively, for SNRs from 12 dB to 25 dB (see Figure6). Compared to the other methods of the interval inference, the evidence interval (horizontal light gray bar) presents an incorrect number of inferences (c) less than or equal to the other methods (horizontal bars).These results can be explained by checking the size of the intervals for each method in Figure7. In this figure, points represent the raw data (jittered horizontally), the horizontal line shows the sample mean, and the lateral lines represent a smoothed density. Table 1 summarizes the values contained in Figures 5, 6, and 7.Figure 7
RDI (raw description and inference) of the interval length for different computational methods. Each column points represent the raw data (jittered horizontally), the horizontal line shows the sample mean, and the lateral lines represent a smoothed density.Table 1
Occurrence of the scenarios described in Figure4 for interval inference methods.
Percent of occurrences±0.1(%)
Scenario
SNR (dB)
Gosset
Morrison
FBST
Punctual
(a) Correct classification
12
84.4
84.3
84.0
85.6
15
88.6
88.6
88.4
89.6
17
91.0
91.0
90.7
91.7
20
93.7
93.7
93.5
94.2
23
95.4
95.5
95.3
95.9
25
96.4
96.4
96.3
96.8
(b) Middle interval
12
2.4
2.6
3.2
15
1.8
1.9
2.3
17
1.5
1.5
1.9
20
1.0
1.0
1.4
23
0.9
0.9
1.2
25
0.7
0.7
0.9
(c) Miss classification
12
13.2
13.1
12.8
14.4
15
9.6
9.5
9.3
10.4
17
7.5
7.5
7.4
8.3
20
5.3
5.3
5.1
5.8
23
3.7
3.6
3.5
4.1
25
2.9
2.9
2.8
3.2
Average interval length (Np)
0.074
0.080
0.092On an average, the length of the evidence interval (computed by the FBST) is 24% larger than the interval calculated by the Gosset method and 15% larger than the interval calculated by Morrison’s method (see Table1). They also present a higher dispersion than the other methods do.Another attempt to measure the influence of interval inference is to exclude from the confusion matrix the comparisons that result in scenario (b) of Figure4. In this way, a fifth category, “In dubio pro reo,” may be included. The Table 2 presents a comparison of how the inclusion of the interval inference, when including “In dubio pro reo,” changes the percentage of true positives, true negatives, false positives, and false negatives. The table shows the EER calibration of 8.1%. However, for open-set validation, the GMM-UBM methodology presents false positive rates of 9.4%, which reduces to 8.4% using the range of evidence calculated from the FBST.Table 2
Percentage of true positives, true negatives, false positives, and false negatives obtained in the test and validation steps.
True positive (%)
True negative (%)
False positive (%)
False negative (%)
In dubio pro reo (%)
Train
Punctual
91.9
91.9
8.1
8.1
Gosset
91.1
90.8
7.0
7.4
1.8
Morrison
91.2
90.9
7.1
7.5
1.7
FBST
90.8
90.0
6.79
7.12
2.4
Validation
Punctual
85.3
90.6
9.4
14.7
Gosset
85.0
90.0
8.8
14.0
1.1
Morrison
85.0
90.0
8.8
13.9
1.2
FBST
85.0
89.9
8.7
13.8
1.3


## 4. Conclusion and Future Work

This paper presented an improvement to the FBST calculation for the distribution of a mean with an unknown variance. These improvements obviate the need for MCMC techniques to calculate the FBST integral. Compared with other methods, the evidence interval was more conservative, reducing incrementally Type I and Type II errors in low-SNR scenarios.Although the results do not present a significant improvement in the reduction of the false positive rate, for open sets, the present work helps to understand the limits of the GMM-UBM methodology applied to FSC. The contribution of the range of evidence may seem insignificant. However, in the case of sex crimes, especially against children, understanding the limits of each tool in the FSC helps the forensic expert to make more informed decisions.Possible developments of the present work include improving the FBST for the Behrens–Fisher problem, combining the evidence interval with background database calibration and tests with different features such as Power Normalized Cepstral Coefficients (PNCC), Perceptual Linear Predictive (PLP), and noise. The application of the interval inference in speaker verification techniques, such asi-vector and x-vector, are under development and should be discussed in future work.



---

*Source: 2914942-2020-03-24.xml*