SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Based on Theorem 4 we propose a modified version of our statistical test named  $\ell$ -C2ST-NF. The new null hypothesis associated with the consistency of the posterior approximation  $q_{\phi}$  at  $x_{\mathrm{o}}$  is
$$
\mathcal {H} _ {0} ^ {\mathrm {N F}} \left(x _ {\mathrm {o}}\right): p \left(T _ {\phi} ^ {- 1} (\theta ; x _ {\mathrm {o}}) \mid x _ {\mathrm {o}}\right) = \mathcal {N} \left(0, I _ {m}\right), \tag {17}
$$
which leads to a new binary classification framework
$$
\left(Z, X\right) \mid (C = 0) \sim \mathcal {N} \left(0, I _ {m}\right) p (x) \quad \text {v s .} \quad \left(Z, X\right) \mid (C = 1) \sim p \left(T _ {\phi} ^ {- 1} (\theta ; x), x\right). \tag {18}
$$
Algorithm 3 describes how to sample data from each class and train a classifier to discriminate them. The classifier is then evaluated on  $N_{v}$  samples  $Z_{n} \sim \mathcal{N}(0, I_{m})$  which are independent of  $x_{\mathrm{o}}$ .
A remarkable feature of  $\ell$ -C2ST-NF is that calculating the test statistics under the null hypothesis is considerably faster than for  $\ell$ -C2ST. In fact, for each null trial  $h = 1, \ldots, N_{\mathcal{H}}$  we use the dataset  $\mathcal{D}_{\mathrm{cal}}$  only for recovering the samples  $X_{n}$  and then independently sample new data  $Z_{n} \sim \mathcal{N}(0, I_{m})$ . As such, it is possible to pre-train the classifiers without relying on a permutation procedure (cf. Algorithm 4), and to re-use them to quickly compute the validation diagnostics for any choice of  $x_{\mathrm{o}}$  or new posterior estimator  $q_{\phi}$  of the given inference task. It is worth mentioning that this is not possible with the usual  $\ell$ -C2ST, as it depends on  $q$  and it would require new simulations for each trial.
Algorithm 3:  $\ell$ -C2ST-NF - training the classifier on the joint distribution
Input: NF posterior estimator  $q_{\phi}$ ; calibration data  $\mathcal{D}_{\mathrm{cal}} = \{\Theta_n,X_n\}_{n = 1}^{N_{\mathrm{cal}}}$ ; classifier  $f$   
Output: estimate  $d$  of the class probabilities  
/* Construct classification training set  
for  $n$  in  $1,\ldots ,N_{\mathrm{cal}}$  do  
\[
Z_{n}\sim \mathcal{N}(0,I_{m});Z_{n}^{q} = T_{\phi}^{-1}(\Theta_{n};X_{n}) / *\text{inverse NF - transformation} * / W_{2n} = (Z_{n},X_{n});C_{2n} = 0
W_{2n + 1} = (Z_{n}^{q},X_{n});C_{2n + 1} = 1
\]  
 $\mathcal{D}\gets \{W_n,C_n\}_{n = 1}^{2N_{\mathrm{cal}}}$   
/* Get estimate  $d$  of the class probabilities  
Train the classifier  $f$  on  $\mathcal{D}$ $d\gets f_{\mathrm{probability}}$   
return  $d$
Algorithm 4:  $\ell$ -C2ST-NF - precompute the null distribution for any estimator
Input: calibration data  $\mathcal{D}_{\mathrm{cal}} = \{\Theta_n,X_n\}_{n = 1}^{N_{\mathrm{cal}}}$  classifier  $f$  ; number of null samples  $N_{\mathcal{H}}$  Output: estimates  $\{d_1,\dots ,d_{N_{\mathcal{H}}}\}$  of the class probabilities under the null for  $h$  in 1,...  $N_{\mathcal{H}}$  do  $\begin{array}{rl}{/\ast}&{\mathrm{Construct~classification~training~set}}\\{\mathrm{Sample~}Z_n\sim\mathcal{N}(0,I_m)\mathrm{~for~}n=1,\ldots,2N_{\mathrm{cal}}}\\{\mathcal{D}\leftarrow\{(Z_{2n},X_n),0\}_{n = 1}^{N_{\mathrm{cal}}}\cup\{(Z_{2n + 1},X_n),1\}_{n = 1}^{N_{\mathrm{cal}}}}\\{/^*}&{\mathrm{Get~estimate~}d\mathrm{~of~the~class~probabilities}}\\{\mathrm{Train~the~classifier~}f\mathrm{~on~}\mathcal{D}}\\{dh\leftarrow f_{\mathrm{probability}}}\end{array}$  return  $\{d_1,\dots ,d_{N_\mathcal{H}}\}$
All experiments were implemented with Python and the sbi package [44] combined with PyTorch [38] and nflows [12] for neural posterior estimation. Classifiers on the C2ST framework use the MLPClassifier from scikit-learn [39] with the same parameters as those used in sbibm [31].
4.1 Two benchmark examples for SBI
We illustrate  $\ell$ -C2ST on two examples: Two Moons and SLCP. These models have been widely used in previous works from SBI literature [18, 36] and are part of the SBI benchmark [31]. They both represent difficult inference tasks with locally structured multimodal true posteriors in, respectively, low ( $\theta \in \mathbb{R}^2$ ,  $x \in \mathbb{R}^2$ ) and high ( $\theta \in \mathbb{R}^5$ ,  $x \in \mathbb{R}^8$ ) dimensions. See [31] for more details. To demonstrate the benefits of  $\ell$ -C2ST-NF, all experiments use neural spline flows [11] trained under the amortized paradigm for neural posterior estimation (NPE) [37]. We use implementations from the sbibm package [31] to ensure uniform and consistent experimental setups. Samples from the true posterior distributions for both examples are obtained via MCMC and used to compare  $\ell$ -C2ST(NF) to the oracle-C2ST framework. We include results for local-HPD implemented using the code repository of the authors of [48] with default hyper-parameters and applied to HPD.
First, we evaluate the local consistency of the posterior estimators of each task over ten different observations  $x_{\mathrm{o}}$  with varying  $N_{\mathrm{train}}$  and fixed  $N_{\mathrm{cal}} = 10^{4}$ . The first column of Figure 2 displays the MSE statistics for the oracle and  $\ell$ -C2ST frameworks. As expected, we observe a decrease in the test statistics as  $N_{\mathrm{train}}$  increases: more training data usually means better approximations. For Two Moons the statistic of  $\ell$ -C2ST decreases at the same rate as oracle-C2ST, with notably accurate results for  $\ell$ -C2ST-NF. However, in SLCP both  $\ell$ -C2ST statistics decrease much faster than the oracle. This is possibly due to the higher dimension of the observation space in the latter case, which impacts the training-procedure of  $\ell$ -C2ST on the joint distribution.
We proceed with an empirical analysis based on 50 random test runs for each validation method and computing their rejection-rates to the nominal significance level of  $\alpha = 0.05$ . Column 4 in Figure 2 confirms that the false positive rates (or type I errors) for all tests are controlled at desired level. Column 2 of Figure 2 portrays the true positive rates (TPR), i.e. rejecting  $\mathcal{H}_0(x_{\mathrm{o}})$  when  $p \neq q$ , of each test as a function of  $N_{\mathrm{train}}$ . Both  $\ell$ -C2ST strategies decrease with  $N_{\mathrm{train}}$  as in Column 1, with higher TPR for  $\ell$ -C2ST-NF in both tasks. The oracle has maximum TPR and rejects the local consistency of all posterior estimates across all observations at least  $90\%$  of the time. Note that SLCP is designed to be a difficult model to estimate<sup>6</sup>, meaning that higher values of TPR are expected ( $\hat{t} \neq 0$  in Column 1). In Two Moons, the decreasing rejection rate can be seen as normal as it reflects the convergence of the posterior approximator ( $\hat{t} \to 0$  in Column 1): as  $N_{\mathrm{train}}$  increases, the task of differentiating the estimator from the true posterior becomes increasingly difficult.
We fix  $N_{\mathrm{train}} = 10^3$  (which yields inconsistent  $q_{\phi}$  in both examples) and investigate in Column 3 of Figure 2 how many calibration samples are needed to get a maximal TPR in each validation method.
$^{4}$ Code is available at https://github.com/JuliaLinhart/1c2st.
The average run-times for each validation method are provided in Appendix C.
$^6$ SLCP = simple likelihood, complex posterior
Figure 2: Results on two examples from the SBI benchmark: SLCP and Two Moons. We compare  $\ell$ -C2ST and  $\ell$ -C2ST-NF (dashed) to the oracle C2ST and local-HPD. Columns 1 and 2 display the test statistic and empirical power as a function of  $N_{\mathrm{train}}$ , while Columns 3 and 4 show the empirical power and type I error for varying  $N_{\mathrm{cal}}$ . The  $\ell$ -C2ST(-NF) statistics are comparable to the oracle, as their decreasing behaviour reflects the convergence of NPE to the true posterior for large training datasets. We also note that  $\ell$ -C2ST-NF is uniformly better than  $\ell$ -C2ST (i.e. higher power for all  $N_{\mathrm{train}}$  and increases faster with  $N_{\mathrm{cal}}$ ), and both reach maximum statistical power with smaller  $N_{\mathrm{cal}}$  than local-HPD. All Type I errors are controlled at  $\alpha = 0.05$ . Experiments were performed over 10 different observations  $x_{\mathrm{o}}$  (mean and std) and Columns 2-4 used additional 50 random test runs.
SLCP is expected to be an easy classification task, since the posterior estimator is very far from the true posterior (large values of  $\hat{t}$  in Column 1). We observe similar performance for  $\ell$ -C2ST-NF and  $\ell$ -C2ST, with slightly faster convergence for the latter. Both methods perform better than local-HPD, that never reaches maximum TPR. Two Moons represents a harder discrimination task, as  $q_{\phi}$  is already pretty close to the reference posterior (see Column 1). Here,  $\ell$ -C2ST-NF attains maximum power at  $N_{\mathrm{cal}} = 2000$  and outperforms all other methods. Surprisingly, the regression-based oracle-C2ST performs comparably to local-HPD, converging to TPR = 1 at  $N_{\mathrm{cal}} = 5000$ .
4.2 Jansen-Rit Neural Mass Model (JRNMM)
We increase the complexity of our examples and consider the well known Jansen & Rit neural mass model (JRNMM) [25]. This is a neuroscience model which takes parameters  $\pmb{\theta} = (C, \mu, \sigma, g) \in \mathbb{R}^4$  as input and generates time series  $x \in \mathbb{R}^{1024}$  with properties similar to brain signals obtained in neurophysiology. Each parameter has a physiologically meaningful interpretation, but they are not relevant for the purposes of this section; the interested reader is referred to [3] for more details.
The approximation  $q_{\phi}$  of the model's posterior distribution is a conditioned masked autoregressive flow (MAF) [35] with 10 layers. We follow the same experimental setting from [3], with a uniform prior over physiologically-relevant parameter values and a simulated dataset from the joint distribution including  $N_{\mathrm{train}} = 50000$  training samples for the posterior estimator and  $N_{\mathrm{cal}} = 10000$  samples to compute the validation diagnostics. An evaluation set of size  $N_{v0} = 10000$  is used for  $\ell$ -C2ST-NF.
We first investigate the global consistency of our approximation, which informs whether  $q_{\phi}$  is consistent (or not) on average with the model's true posterior distribution. We use standard tools for this task such as simulation-based calibration (SBC) [42] and coverage tests based on highest predictive density (HPD) [48]. The results are shown in the left panel of Figure 3. We observe that the empirical cdf of the global HPD rank-statistic deviates from the identity function (black dashed line), indicating that the approximator presents signs of global inconsistency. We also note that the marginal SBC-ranks are unable to detect any inconsistencies in  $q_{\phi}$ .
We use  $\ell$ -C2ST-NF to study the local consistency of  $q_{\phi}$  on a set of nine observations  $x_0^{(i)}$  defined as<sup>7</sup>
$$
x _ {\mathrm {o}} ^ {(i)} = \operatorname {J R N M M} \left(\theta_ {\mathrm {o}} ^ {(i)}\right) \quad \text {w i t h} \quad \theta_ {\mathrm {o}} ^ {(i)} = (1 3 5, 2 2 0, 2 0 0 0, g _ {\mathrm {o}} ^ {(i)}) \quad \text {a n d} \quad g _ {\mathrm {o}} ^ {(i)} \in [ - 2 0, + 2 0 ]. \tag {19}
$$
The right panel of Figure 3 shows that the test statistics of  $\ell$ -C2ST-NF vary in a U-shape, attaining higher values as  $g_{\mathrm{o}}$  deviates from zero and at the borders of the prior. The plot is overlaid with the  $95\%$  confidence region, illustrating how much the test statistics deviate from the local null hypothesis.
Note that the uniform prior for  $g$  is defined in  $[-20, +20]$  when training  $q_{\phi}$  with NPE.
Figure 3: Results for global and local tests on JRNMM. Left: PP-plots for the marginal SBC and global HPD rank statistics. Right: Test statistics for  $\ell$ -C2ST-NF on observations with varying  $g_{\mathrm{o}}$ . SBC fails to detect any inconsistency of  $q_{\phi}$ , while HPD only provides a global assessment, unlike  $\ell$ -C2ST which locally explains the inconsistencies in  $q_{\phi}$ .
Figure 4: Graphical diagnostics of  $\ell$ -C2ST-NF for JRNMM. Top right panel displays the empirical CDF of the classifier (blue) overlaid with the theoretical CDF of the null hypothesis (step function at 0.5) and  $95\%$  confidence region of estimated classifiers under the null displayed in gray. The pairplot displays histograms of samples from  $q_{\phi}$  within the prior region and dashed lines indicate values of  $\theta_{\mathrm{o}}$  used to generate the conditioning observation  $x_{\mathrm{o}}$ . The predicted probabilities are mapped on the colors of the bins in the histogram. Blue-green (resp. orange-red) regions indicate low (resp. high) predicted probabilities of the classifier. Yellow regions correspond to chance level, thus  $q_{\phi} \approx p$ .
We demonstrate the interpretability of the results for  $\ell$ -C2ST-NF with a focus on the behavior of  $q_{\phi}$  when conditioned on an observation for which  $g_{\mathrm{o}} = 10$ . The local PP-plot in Figure 4 summarises the test result: the predicted class probabilities deviate from 0.5, outside of the  $95\%$  CR, thus rejecting the null hypothesis of local consistency at  $g_{\mathrm{o}} = 10$ . The rest of Figure 4 displays 1D and 2D histograms of samples  $\Theta^q \sim q_{\phi}(\theta \mid x_{\mathrm{o}})$  within the prior region, obtained by applying the learned  $T_{\phi}$  to samples  $Z \sim \mathcal{N}(0, I_4)$ . The color of each histogram bin is mapped to the intensity of the corresponding predicted probability in  $\ell$ -C2ST-NF and informs the regions where the classifier is more (resp. less) confident about its choice of predicting class 0. $^8$  This relates to regions in the parameter space where the posterior approximation has too much (resp. not enough mass) w.r.t. to the true posterior:  $q_{\phi} > p$  (resp.  $q_{\phi} < p$ ). We observe that the ground-truth parameters are often outside of the red regions, indicating positive bias for  $\mu$  and  $\sigma$  and negative bias for  $g$  in the 1D marginal. It also shows that the posterior is over-dispersed in all 2D marginals. See Appendix D for results on all observations  $x_{\mathrm{o}}^{(i)}$ .
We have presented  $\ell$ -C2ST, an extension to the C2ST framework tailored for SBI settings which requires only samples from the joint distribution and is amortized along conditioning observations. Strikingly, empirical results show that, while  $\ell$ -C2ST does not have access to samples from the true posterior distribution, it is actually competitive with the oracle-C2ST approach that does. This comes at the price of training a binary classifier on a potentially large dataset to ensure the correct calibration
${}^{8}$  Specifically,we compute the average predicted probability of class 0 for data points  $Z \sim  \mathcal{N}\left( {0,{I}_{4}}\right)$  corresponding to samples  ${T}_{\phi }\left( {Z;{x}_{\mathrm{o}}}\right)  \sim  {q}_{\phi }\left( {\theta  \mid  {x}_{\mathrm{o}}}\right)$  within each histogram bin.
of the predicted probabilities. Should this be not the case, some additional calibration step for the classifier can be considered [10].
Notably,  $\ell$ -C2ST allows for a local analysis of the consistency of posterior approximations and is more sensible, precise, and computationally efficient than its concurrent method, local-HPD. Appendix F.4 provides a detailed discussion of these statements, based on results obtained for additional benchmark examples. When exploiting properties of normalizing flows,  $\ell$ -C2ST can be further improved as demonstrated by encouraging results on difficult posterior estimation tasks (see Table 2 in Appendix F). We further analyze the benefits of this -NF version in Appendix F.3.  $\ell$ -C2ST provides necessary, and sufficient, conditions for posterior consistency, features that are not shared by other standard methods in the literature (e.g. SBC). When applied to a widely used model from computational neuroscience, the local diagnostic proposed by  $\ell$ -C2ST offered interesting and useful insights on the failure modes of the SBI approach (e.g. poor estimates on the border of the prior intervals), hence demonstrating its potential practical relevance for works leveraging simulators for scientific discoveries.
6 Limitations and Perspectives
Training a classifier with finite data. The proposed validation method leverages classifiers to learn global and local data structures and shows great potential in diagnosing conditional density estimators. However, it's validity is only theoretically guaranteed by the optimality of the classifier when  $N_v \to \infty$ . In practice, this can never perfectly be ensured. Figure 6 in Appendix F.2 shows that depending on the dataset,  $\ell$ -C2ST can be more or less accurate w.r.t. the true C2ST. Therefore, one should always be concerned about false conclusions due to a far-from-optimal classifier and make sure that the classifier is "good enough" before using it as a diagnostic tool, e.g. via cross-validation. Note, however, that the MSE test statistic for  $\ell$ -C2ST is defined by the predicted class probabilities and not the accuracy of the classifier, thus one should also check how well the classifier is calibrated.
Why Binary Classification? An alternative to binary classification would have been to train a second posterior estimate in order to assess the consistency of the first one. Indeed, one could ask whether training a classifier is inherently easier than obtaining a good variational posterior, the response to which is non-trivial. Nevertheless, we believe that adding diversity into the validation pipeline with two different ML approaches might be preferable. Furthermore, building our method on the C2ST framework was mainly motivated by the popularity and robustness of binary classification: it is easy to understand and has a much richer and stable literature than deep generative models. As such, we believe that choosing a validation based on a binary classifier has the potential of attracting the interest of scientists across various fields, rather than solely appealing to the probabilistic machine learning community.
Possible extensions and improvements. Future work could focus on leveraging additional information of  $q$  while training the classifier as in [47]. For example, by using more samples from the posterior estimator  $q$  or its explicit likelihood function (which is accessible when  $q$  is a normalizing flow). On a different note, split-sample conformal inference could be used to speed up the  $p$ -value calculations (avoiding the time-consuming permutation step in Algorithm 1).
In summary, our article shows that  $\ell$ -C2ST is theoretically valid and works on several datasets, sometimes even outperforming local-HPD, which to our knowledge is the only other existing local diagnostic. Despite facing some difficulties for certain examples (just like for other methods as well), an important feature of  $\ell$ -C2ST is that one can directly leverage from improvements in binary classification to adapt and enhance it for any given dataset and task. This makes  $\ell$ -C2ST a competitive alternative to other validation approaches, with great potential of becoming the go-to validation diagnostic for SBI practitioners.
Julia Linhart is recipient of the Pierre-Aguilar Scholarship and thankful for the funding of the Capital Fund Management (CFM). Alexandre Gramfort was supported by the ANR BrAIN (ANR-20-CHIA0016) grant while in his role at Université Paris-Saclay, Inria.
[44] Alvaro Tejero-Cantero, Jan Boelts, Michael Deistler, Jan-Matthi Lueckmann, Conor Durkan, Pedro J. Goncalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulation-based inference. Journal of Open Source Software, 5(52):2505, 2020. doi: 10.21105/joss.02505.
[45] Vasist, Malavika, Rozet, François, Absil, Olivier, Mollière, Paul, Nasedkin, Evert, and Louppe, Gilles. Neural posterior estimation for exoplanetary atmospheric retrieval. A&A, 672:A147, 2023. doi: 10.1051/0004-6361/202245263. URL https://doi.org/10.1051/0004-6361/202245263.
[46] Daniel Ward, Patrick Cannon, Mark Beaumont, Matteo Fasiolo, and Sebastian M Schmon. Robust neural posterior estimation and statistical model criticism. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=MHE27tjD8m3.
[47] Yuling Yao and Justin Domke. Discriminative calibration. 2023.
[48] David Zhao, Niccolò Dalmasso, Rafael Izbicki, and Ann B. Lee. Diagnostics for conditional density models and bayesian inference algorithms. In Cassio de Campos and Marloes H. Maathuis, editors, Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pages 1830-1840. PMLR, 27-30 Jul 2021. URL https://proceedings.mlr.press/v161/zhao21b.html.
A Proofs 16
A.1 Proof of Theorem 1 16
A.2 Proof of Theorem 2 17
A.3 Proof of Theorem 3 17
B Algorithms 19
B.1 Generating PP-plots with  $\ell$ -C2ST 19
C Run-times for each validation procedure 20
D Graphical diagnostics (  $\ell$  -C2ST-NF) for the JRNMM posterior estimator 21
E On the cross-entropy loss for  $\ell$  -C2ST 23
F Additional Experiments on several benchmark examples 24
F.1 Scalability to high dimensions 24
F.2 Accuracy of  $\ell$ -C2ST(-NF) w.r.t. the true C2ST 25
F.3 Benefit of the -NF version 27
F.4 Advantages of  $\ell$ -C2ST w.r.t. local-HPD 27
In what follows, we assume that it is sufficient for the null hypothesis  $\mathcal{H}_0$  to hold on any set  $\mathcal{C} \subseteq \mathbb{R}^m$  of strictly positive measure, rather than requiring it to hold for all points  $\theta \in \mathbb{R}^m$ . In practice, it generally has no practical implications if the posterior estimator is inconsistent with the true posterior  $(q \neq p)$  on a set of measure zero, since those sets don't have any real statistical significance.
For the proofs of Theorems 1 and 2, we will consider a classifier  $f_{x_0}$  defined for a fixed observation  $x_0 \in \mathbb{R}^d$  on  $S_{x_0} = \{\theta \in R^m, q(\theta \mid x_0) + p(\theta \mid x_0) > 0\}$ .
Theorem 1 (Local consistency and classification accuracy). If  $f_{x_{\mathrm{o}}}$  is Bayes optimal and  $N_v \to \infty$ , then  $\hat{t}_{\mathrm{Acc}}(f_{x_{\mathrm{o}}}) = 1/2$  is a necessary and sufficient condition for the local consistency of  $q$  at  $x_{\mathrm{o}}$ .
Proof. As  $S_{x_0}$  contains all data points  $\Theta_n^q \sim q(\theta \mid x_0)$  ( $C_n = 0$ ) and  $\Theta_n^p \sim p(\theta \mid x_0)$  ( $C_n = 1$ ), the empirical accuracy  $\hat{t}_{\mathrm{Acc}}(f_{x_0})$  over the validation set  $\mathcal{D}_v = \{(\Theta_n, C_n)\}_{n=1}^{2N_v}$  is well defined (7) and
$$
\hat {t} _ {\mathrm {A c c}} (f _ {x _ {\mathrm {o}}}) \underset {N _ {v} \rightarrow \infty} {\longrightarrow} \operatorname {A c c} (f _ {x _ {\mathrm {o}}}) = \operatorname {P} (f _ {x _ {\mathrm {o}}} (\Theta) = C) = \frac {1}{2} \left(\operatorname {P} (f _ {x _ {\mathrm {o}}} (\Theta^ {q}) = 0) + \operatorname {P} (f _ {x _ {\mathrm {o}}} (\Theta^ {p}) = 1)\right).
$$
Let's show that if  $f_{x_{\mathrm{o}}}$  is Bayes optimal, then  $\mathcal{H}_0(x_{\mathrm{o}})$  holds  $\Longleftrightarrow \operatorname{Acc}(f_{x_{\mathrm{o}}}) = \frac{1}{2}$ .
$(\Rightarrow)$ : Suppose  $\mathcal{H}_0(x_{\mathrm{o}})$  holds. The optimality of  $f_{x_{\mathrm{o}}}$  implies that  $f_{x_{\mathrm{o}}}(\Theta^{q})$  and  $f_{x_{\mathrm{o}}}(\Theta^{p})$  are Bernoulli random variables  $\mathcal{B}(\frac{1}{2})$  (see interpretation of equation (5)), and so  $\operatorname{Acc}(f_{x_{\mathrm{o}}}) = \frac{1}{2}\left(\frac{1}{2} + \frac{1}{2}\right) = \frac{1}{2}$ .
$(\Leftarrow)$ : Let's proceed by showing the contraposition: if  $\mathcal{H}_0(x_{\mathrm{o}})$  does not hold, then  $\operatorname{Acc}(f_{x_{\mathrm{o}}}) \neq \frac{1}{2}$ .