SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zeyu Zhang*
Gaoling School of Artificial Intelligence
zeyuzhang@ruc.edu.cn
xu.chen@ruc.edu.cn
Chaozhuo Li *
Microsoft Research Asia
cli@microsoft.com
Xing Xie
xingx@microsoft.com
This paper studies the problem of active causal discovery when the experiments can be done based on multi-fidelity oracles, where higher fidelity experiments are more precise and expensive, while the lower ones are cheaper but less accurate. In this paper, we formally define the task of multi-fidelity active causal discovery, and design a probabilistic model for solving this problem. In specific, we first introduce a mutual-information based acquisition function to determine which variable should be intervened at which fidelity, and then a cascading model is proposed to capture the correlations between different fidelity oracles. Beyond the above basic framework, we also extend it to the batch intervention scenario. We find that the theoretical foundations behind the widely used and efficient greedy method do not hold in our problem. To solve this problem, we introduce a new concept called  $\epsilon$ -submodular, and design a constraint based fidelity model to theoretically validate the greedy method. We conduct extensive experiments to demonstrate the effectiveness of our model.
Causal discovery aims to learn the causal structure of a set of variables, which is fundamental for many real-world applications, including health caring [1], education [2] drug discovery [3] and protein synthesis [4]. In general, causal structure learning is an NP-hard problem [5], and purely based on the observational datasets, one cannot identify the unique causal structure, where the best result is discovering its Markov equivalence class [6].
To more accurately identify the unique causal structure, a promising direction is active causal discovery (ACD), where the model is allowed to actively intervene the causal structure to query key information for orienting the causal relations between different variables. For example, to study the causal relations between the drugs and diseases, one can conduct clinical tests via selectively administering the medicines to the patients. The key of active causal discovery is how to design effective experiments when the total cost (e.g., the number of experiments) is limited. To achieve this goal, recent years have witnessed many promising models. For example, Agrawal et al. [7] proposes to intervene on the variables which can orient as many as possible undirected edges. Tugas et al. [8]
\*Co-first authors.
designs a mutual information based method to determine the variables and values to be intervened, and study both single and batch intervention scenarios.
While the above models have achieved remarkable successes, they only allow to query a single oracle (e.g., the real causal structure) for the experiments<sup>3</sup>. However, in many real-world applications, the experiments can be done via different methods. For example, to investigate the drug-disease causal relations, in addition to the clinical tests, one can also build simulators to obtain the medicine effects on the patients [9]. Usually, each experimental method corresponds to a unique oracle, and different oracles have various fidelites. Higher-fidelity experiments are more accurate but expensive, for example, administering drugs to the real patients. Lower-fidelity experiments are cheaper but inaccurate, for example, using patient simulators. These different fidelity oracles may offer better cost-benefit trade-offs, which, however, cannot be handled by existing active causal discovery models.
To bridge the above gap, in this paper, we formally define the task of active causal discovery with multi-fidelity oracles, where the model has to actively select which variables and values to intervene at which fidelities. This task is non-trivial due to the following reasons: to begin with, because of the introduction of multi-fidelity oracles, the model has to strategically choose the lower-cost and informative enough experiments to uncover the real causal structure, which needs our special designs. Then, given the experiment results with different fidelities, how to infer the real causal structure is also not easy, since the experiment results can be not produced from the oracle corresponding to the real causal structure. In addition, in practice, an efficient experiment should allow simultaneously intervening multiple variables [10]. However, how to extend our model to the batch intervention scenario is still not clear.
To overcome the above challenges, we design a Bayesian active causal discovery model, which is composed of two components. The first one is a mutual information (MI) based acquisition function. It aims to select the interventional variables, values and fidelities which are more informative for the real causal structure. The second one is a cascading fidelity model. In specific, we first regard the highest fidelity oracle as the real causal structure, and then a cascading model is built to correlate different fidelity oracles, so that the experiment results at one fidelity can be leveraged to infer the oracle at another fidelity. To achieve more efficient experiments, we also extend our model to the batch intervention scenario. Previously, the greedy method is demonstrated to be an efficient and effective strategy for batch intervention [8]. However, we found that, by allowing multi-fidelity oracles, the theoretical foundations behind the greedy method do not hold. For alleviating this problem, we introduce a new concept called  $\epsilon$ -submodular, and design a constraint-based fidelity model to theoretically validate the greedy method.
The main contributions of this paper are summarized as follows: (1) we formally define the task of active causal discovery with multi-fidelity oracles, which, to our knowledge, is the first time in the field of causal discovery. (2) To solve the above task, we propose a Bayesian framework, which is composed of a mutual information based acquisition function and a cascading fidelity model. (3) To extend our framework to the batch intervention scenario, we introduce a constraint-based fidelity model, which provides theoretical guarantees for the efficient greedy method. (4) We conduct extensive experiments to demonstrate the effectiveness of our model.
2.1 Structure Causal Model
Structure causal model (SCM) is an effective language for describing and learning the causal relations between different random variables [11]. Usually, SCM is composed of a causal graph and a set of structure equation models (SEM).
For the causal graph, we denote it by  $G = \langle V, \mathbf{E} \rangle$ , where  $V$  is the node set, and  $\mathbf{E}$  is the adjacency matrix. Each node in  $V$  corresponds to a variable. Suppose there are  $d$  variables in our studied problem, then we use  $X_{V} = [X_{1}, X_{2}, \ldots, X_{d}]$  to denote the variable set. The adjacency matrix  $\mathbf{E} \in \{0, 1\}^{d \times d}$  describes the causal relations between different variables.  $\mathbf{E}_{ij} = 1$  means that  $X_{i}$  is a
<sup>3</sup>In the following, we may interchangeably use "oracle", "causal structure" and "structure causal model" to represent the underlying model for generating the results of the experiments.
parent of  $X_{j}$ , and there exists an edge from  $X_{i}$  to  $X_{j}$ , while  $\mathbf{E}_{ij} = 0$  indicates that there is no edge between  $X_{i}$  and  $X_{j}$ .
For the structure equation models, we denote them by  $\pmb{g} = \{g_1, g_2, \dots, g_d\}$ , where each  $g_i$  quantitatively describes the relation between  $X_i$  and its parents. Formally, we implement  $F$  with the commonly used additive noise models (ANM) [12], that is:
$$
X _ {i} = g _ {i} \left(p a (i); \gamma_ {i}\right) + \epsilon_ {i}, \quad \epsilon_ {i} \sim \mathcal {N} \left(0, \sigma_ {i} ^ {2}\right), \tag {1}
$$
where  $\gamma_{i}$  is the parameter set of  $g_{i}$ , and the noise term  $\epsilon_{i}$  follows the Gaussian distribution with  $\sigma_{i}^{2}$  as the variance. We denote the complete parameter set of  $\pmb{g}$  as  $\pmb{\theta} = \{\pmb{\gamma},\pmb{\sigma}\}$ , where  $\pmb{\gamma} = \{\gamma_1,\gamma_2,\dots ,\gamma_d\}$  and  $\pmb{\sigma} = \{\sigma_{1},\sigma_{2},\dots ,\sigma_{d}\}$ . Noted that, given the above equation, we can easily derive the distribution of  $X_{V}$ , that is,  $p(X_V) = \prod_{i = 1}^d\mathcal{N}(g_i(pa(i);\gamma_i),\sigma_i^2)$ .
Based on the above formulation, given an observational dataset  $D = \{x_{k}\}_{k = 1}^{N}\sim p(X_{V})$ , causal discovery aims to learn the adjacency matrix  $\mathbf{E}$ , or more generally, simultaneously identify  $\mathbf{E}$  and the SEM parameter  $\theta$ . In this paper, we focus on the general case, and denote  $\phi = (\pmb {\theta},\mathbf{E})$
2.2 Active Causal Discovery
Previous work has demonstrated that, purely based on the observational dataset, the real causal graph can only be identifiable to its Markov equivalence class. Active causal discovery holds the promise of identifying more accurate causal graph via designing interventional experiments.
Formally, an interventional experiment is represented by  $e = \{(j,v)\}$ , which means cutting all the causal relations pointing to  $X_{j}$ , and fixing the value of  $X_{j}$  as  $v$ . In causal learning, the experiment  $e$  can also be represented by  $\mathrm{do}(X_j = v)$ . Obviously, the distribution of  $X_{V}$  is changed after the experiment  $e$ , and we denote the experiment-induced distribution by  $p(X_V|\mathrm{do}(X_j = v))$ . In practice, we cannot access the implementation of  $p$ , but can only observe the experiment result sampled from  $p(X_V|\mathrm{do}(X_j = v))$ . The key of active causal discovery is to design a series of experiments within limited budgets, such that the results can be better leveraged to identify  $\phi$ .
2.3 Multi-Fidelity Active Causal Discovery
Existing ACD models mostly obtain the experiment results via interacting with the real SCM. However, in practice, the experiments can be done in different ways (e.g., real clinical tests or patient simulators). Each type of experiment corresponds to an underlying oracle, which produces the results of the experiments. Different oracles may provide better cost-benefit trade-offs for the experiment designs, which are failed to be considered by the previous work.
Formally, suppose we have  $M$  oracles, and the parameters of the  $i$ th oracle are denoted by  $\phi_i$ . Let the experiment cost of the  $i$ th oracle be  $\lambda_i$ , and without loss of generality, we assume  $\lambda_1 \leq \lambda_2 \leq \dots \leq \lambda_M$ . Intuitively, if an oracle is more accurate (i.e., has higher fidelity), then it should be more expensive. We regard the real SCM as the most accurate oracle, thus we set  $\phi_M = \phi$ . We denote all the oracle parameters and costs as  $\Phi = \{\phi_1, \phi_2, \dots, \phi_M\}$  and  $\Lambda = \{\lambda_1, \lambda_2, \dots, \lambda_M\}$ , respectively.
Due to the introduction of multi-fidelity oracles, the experiment in traditional active causal discovery is extended to be a triplet  $e = \{(j,v),m\}$ , where in addition to the intervention pair  $(j,v)$ , the fidelity  $m$  should also be considered in the experiment design. We define the dataset for model training as  $D = \{e_t,\pmb{x}_t\}_{t = 1}^T$ , where  $e_t = \{(j,v),m\}$  indicates the distribution for generating  $\pmb{x}_t$ . In general,  $\{(j,v),m\}$  means that  $\pmb{x}_t$  is generated from  $p_m(X_V|\mathrm{do}(X_j = v))$ , which is induced by  $\phi_{m}$  and the intervention  $(j,v)$ . If  $(j,v) = \emptyset$ , then  $\pmb{x}_t$  is an observational data, which is sampled from  $\phi_{m}$  without any intervention. Finally, we define the task of multi-fidelity active causal discovery as follows:
Definition 1 (Multi-Fidelity Active Causal Discovery (MFACD)). Given  $M$  oracles with different costs  $\Lambda$ , we need to design a model  $f$ , which can strategically determine the intervention pair  $(j, v)$  and fidelity  $m$  to achieve better cost-benefit trade-off in terms of identifying the real SCM  $\phi_{M}$ .
<sup>4</sup>Noted that if some oracle costs more than another one with higher fidelity, then this oracle is useless, since one may always choose to query the higher fidelity oracle.
3 The Licence Model
For solving the task of MFACD, we design a Bayesian framework called Licence (Multi-fidelity active learning for causal discovery), which is composed of two components. The first one is an acquisition function based on mutual information, which is responsible for designing the experiments. The second one is a cascaded fidelity model, which is designed to capture the correlation between different  $\phi_i$ 's. The experiment results obtained from the first component are leveraged to update the fidelity model, and  $\phi_M$  in the fidelity model is the finally predicted result. At last, we introduce how to extend our model to the batch intervention scenario.
3.1 MI-based Acquisition Function
Intuitively, a better experiment should leverage little cost to reveal as much as possible information about the real SCM  $\phi_M$ . Thus, we design the following acquisition function:
$$
f (j, v, m) = \frac {I (\pmb {x} ; \phi_ {M} | \pmb {e} , D)}{\lambda_ {m}},
$$
where  $e = \{(j,v),m\}$  is the experiment to be designed.  $D$  is the dataset already collected, which will be enlarged after each experiment.  $I(\pmb {x};\phi_M|e,D)$  is the mutual information, which indicates that if we conduct experiment  $e$ , then how much information the experiment result may share with the target parameter  $\phi_{M}$ . Obviously, we should select  $e$  which can lead to larger  $I(\pmb {x};\phi_M|e,D)$ .  $\lambda_{m}$  is the cost of  $e$ . By dividing  $I(\pmb {x};\phi_M|e,D)$  with  $\lambda_{m}$ , we trade-off the experiment informativeness and cost. To determine  $(j,v,m)$ , we derive an estimator for  $f(j,v,m)$  following the idea of Bayesian Active Learning by Disagreement (BALD) [13], that is:
$$
\begin{array}{l} f (j, v, m) = \frac {H (\boldsymbol {x} | \boldsymbol {e} , D) - H (\boldsymbol {x} | \phi_ {M} , \boldsymbol {e} , D)}{\lambda_ {m}} \\ = \frac {- \mathbb {E} _ {p (\boldsymbol {x} | \boldsymbol {e} , D)} \left[ \log \mathbb {E} _ {p \left(\phi_ {m} | \boldsymbol {e} , D\right)} \left[ p (\boldsymbol {x} | \boldsymbol {e} , \phi_ {m}) \right] \right] + \mathbb {E} _ {p \left(\phi_ {M} | D\right)} \left[ \mathbb {E} _ {p (\boldsymbol {x} | \boldsymbol {e}, \phi_ {M})} \left[ \log p (\boldsymbol {x} | \boldsymbol {e}, \phi_ {M}) \right] \right]}{\lambda_ {m}}, \tag {2} \\ \end{array}
$$
where  $p(\pmb{x}|\pmb{e}, D)$  and  $p(\phi_m|e, D)$  correspond to the distributions of  $\pmb{x}$  and  $\phi_m$  after observing  $D$  under the intervention  $(j, v)$ .  $p(\phi_M|D)$  is the posterior of  $\phi_M$  after observing  $D$ .  $p(\pmb{x}|\pmb{e}, \phi_m)$  is the probability of  $\pmb{x}$  based on  $\phi_m$  intervened by  $(j, v)$ . We approximate the expectation operator based Monte Carlo sampling. The detailed derivation process can be seen in the appendix.
Obviously, determining the best  $(j,v,m)$  equals to solving the following problem:
$$
\left\{j ^ {*}, v ^ {*}, m ^ {*} \right\} = \arg \max  _ {\{j, v, m \}} f (j, v, m). \tag {3}
$$
In our task, the intervention value  $v$  is continuous. Following the previous work, we firstly learn the optimal  $v$  for each  $(j,m)$  pair based on Bayesian optimization (BO). Then, we compare all the results, and select the solution which can lead to the largest  $f(j,v,m)$ . We present the detailed Bayesian optimization process in the appendix. It should be noted that one can also leverage more advanced BO methods for jointly learning  $(j,v,m)$  [14]. However, we do not find significant performance improvements by these models.
3.2 Cascaded Fidelity Model
Intuitively, the experiment results from different oracles may share common information. The samples at one fidelity may help to infer the oracles at other fidelities. To capture the correlations between different oracles, we build a cascaded probabilistic model (see Figure 1), where the oracles with different fidelities are successively connected, and the observed samples are only determined by their corresponding oracles.
Formally, to achieve more robust optimization, we first regard the discrete adjacency matrix  $\mathbf{E}$  as the samples from Bernoulli distribution. In specific, we let  $\mathbf{E}_{ij} \sim \mathrm{Bernoulli}(\sigma(\mathbf{S}_{i}^{T}\mathbf{T}_{\cdot j}))$ , where  $\mathbf{S}, \mathbf{T} \in \mathbb{R}^{K \times d}$  ( $K \ll d$ ) are two continuous matrices.  $\mathbf{S}_{\cdot i}$  and  $\mathbf{T}_{\cdot j}$  are the  $i$ th and  $j$ th columns of  $\mathbf{S}$  and  $\mathbf{T}$ , respectively. By replacing  $\mathbf{E}$  with  $\mathbf{S}$  and  $\mathbf{T}$ , we revise the parameter set  $\phi$  as  $(\boldsymbol{\theta}, \mathbf{S}, \mathbf{T})$ .
For each fidelity  $m$ , we assign a prior distribution of  $\phi_{m}$  given  $\phi_{m-1}$  as follows:
$$
\begin{array}{l} p (\boldsymbol {\phi} _ {1}) \propto e ^ {- \beta \cdot f (\mathbf {S} _ {1}, \mathbf {T} _ {1})} \cdot \mathcal {N} (\mathbf {0}, \boldsymbol {I}), \\ p \left(\phi_ {m} \mid \phi_ {m - 1}\right) \propto e ^ {- \beta \cdot f \left(\mathbf {S} _ {m}, \mathbf {T} _ {m}\right)} \cdot \mathcal {N} (\boldsymbol {a} \phi_ {m - 1} + \boldsymbol {b}, \sigma^ {2} \boldsymbol {I}), \quad m \geq 2, \tag {4} \\ \end{array}
$$
Figure 1: The cascaded probabilistic model is shown above. Different fidelity oracles are successively connected, and the observed samples are only determined by their corresponding oracles.
where we add subscript  $m$  to indicate different fidelities.  $f(\mathbf{S}_m,\mathbf{T}_m) = \mathbb{E}_{p(\mathbf{E}|\mathbf{S}_m,\mathbf{T}_m)}[\lambda_1\cdot$ $\{\mathrm{trace}\left(e^{\mathbf{E}}\right) - d\} +\lambda_2\cdot ||\mathbf{E}||_1]$  is a regularizer to encourage  $\mathbf{E}$  to be a sparse and directed acyclic graph [15].  $\lambda_{1},\lambda_{2},a,b$  and  $\sigma^2$  are hyper-parameters.
Since the real SCM is  $\phi_M$ , we need to infer the posterior  $p(\phi_M|D)$ , where  $D$  is the initially observational dataset or experiment results. Directly computing  $p(\phi_M|D)$  is not easy, since the dataset  $D$  may contain samples from different fidelity oracles. To solve this problem, we first obtain the joint distribution  $p(\Phi |D)$ , where  $\Phi = \{\phi_1,\phi_2,\dots,\phi_M\}$  is the collection of all the oracle parameters. Then we derive  $p(\phi_M|D)$  by marginalizing out  $\{\phi_1,\phi_2,\dots,\phi_{M - 1}\}$ .
To efficiently compute and sample from  $p(\Phi | D)$ , we introduce a variational approximator  $q(\Phi)$  which is specified as follows:
$$
q (\phi_ {1}) \sim \mathcal {N} (\mathbf {0}, \boldsymbol {I}),
$$
$$
q _ {\psi_ {m}} \left(\phi_ {m} \mid \phi_ {m - 1}\right) \sim \mathcal {N} \left(\boldsymbol {c} _ {m} \boldsymbol {\phi} _ {m - 1} + \boldsymbol {d} _ {m}, \boldsymbol {\sigma} _ {m} ^ {2} \boldsymbol {I}\right), \quad m \geq 2, \tag {5}
$$
$$
q (\Phi) = q (\phi_ {1}) \prod_ {m = 2} ^ {M} q (\phi_ {m} | \phi_ {m - 1}),
$$
where  $\psi_{m} = \{\pmb{c}_{m},\pmb{d}_{m},\pmb{\sigma}_{m}\}$  is the set of learnable parameters, and we denote  $\Psi = \{\psi_m\}_{m = 2}^M$  According to the theory of variational inference [16], we maximize the following evidence lower bound (ELBO) [16] to learn  $\Psi$  ..
$$
\operatorname {E L B O} = \mathbb {E} _ {\Phi \sim q (\Phi)} [ \log p (D | \Phi) - \operatorname {K L} (q (\Phi) | | p (\Phi)) ], \tag {6}
$$
$$
= \mathbb {E} _ {\boldsymbol {\Phi} \sim q (\boldsymbol {\Phi})} \left[ \log p (D | \boldsymbol {\Phi}) - \log q (\boldsymbol {\Phi}) + \log p (\boldsymbol {\Phi}) \right],
$$
where the likelihood  $p(D|\Phi)$  can be easily obtained based on equation (1). More detailed derivation on the ELBO can be seen in the appendix. Once we have learned  $\Psi$ , the posterior  $p(\Phi |D)$  is approximated by  $q(\Phi)$ , and further, we have the following theory:
Theorem 1. If  $p(\Phi | D) \approx q(\Phi)$ , then for any variable sets  $A, B \subseteq \Phi$ ,  $p(A|B, D) \approx q(A|B)$ . As a special case  $p(\phi_M | D) \approx q(\phi_M)$ .
The proof of this theory is immediate, since
$$
\begin{array}{l} p (A | B, D) = \frac {p (A , B | D)}{p (B | D)} = \frac {\int_ {\Phi_ {- A - B}} p (\Phi | D) \mathrm {d} \Phi_ {- A - B}}{\int_ {\Phi_ {- B}} p (\Phi | D) \mathrm {d} \Phi_ {- B}} \approx \frac {\int_ {\Phi_ {- A - B}} q (\Phi) \mathrm {d} \Phi_ {- A - B}}{\int_ {\Phi_ {- B}} q (\Phi) \mathrm {d} \Phi_ {- B}} \\ = \frac {q (A , B)}{q (B)} = q (A | B) \\ \end{array}
$$
Let  $A = \phi_M$  and  $B = \emptyset$ , we have  $p(\phi_M|D) \approx q(\phi_M)$ . Based on this theory, we leverage  $q$  to replace  $p$  in (2) for easy sampling.
Remark. According to the specification of  $q_{\psi_m}(\phi_m|\phi_{m-1})$ , we can easily demonstrate that  $q(\Phi)$  follows Gaussian distribution. Such property enables us to use the reparameterization trick [17] to relate  $\Phi$  with  $\Psi$ . Different from traditional variational models, in our objective, the adjacency matrix
Low-fidelity
High-fidelity
$\pmb{E}$  in  $f(\mathbf{S}_m,\mathbf{T}_m)$  is sampled from a discrete distribution, which cuts down the back-propagation signal. To solve this problem, we leverage gumbel-softmax to further associate  $\pmb{E}$  with  $(\mathbf{S}_m,\mathbf{T}_m)\in \Phi$ . Since  $\Phi$  can be further represented by  $\Psi$ , all the variables in (6) can be reparameterized by  $\Psi$ , which enables us to optimize it in an end-to-end manner. We present the detailed derivation process and the complete learning algorithm in the appendix.
3.3 Extension to Batch Intervention
In practice, simultaneously intervening multiple variables can be more efficient due to the lower frequency on interacting with the oracles. However, under the setting of batch intervention, the candidate intervention space exponentially increases with respect to the number of targets. Suppose we need to select  $c$  out of  $d$  variables for intervention, then the size of the candidate space is  $d^c$ . Previous work found that the greedy strategy is a both efficient and effective method for the batch intervention scenario [8]. From the efficiency perspective, the greedy method only need to search in a  $cd$ -sized candidate space. From the effectiveness perspective, people demonstrate that the mutual information obtained by the greedy strategy is not worse than that of the optimal solution multiplied by  $(1 - \frac{1}{e})$  [18]. In the following, we introduce how to extend our model to the batch intervention scenario, and leverage the greedy strategy to solve our task.
Objective for batch intervention in MFACD. We use the following objective for batch intervention scenario:
$$
\operatorname *{arg  max}_{\substack{\boldsymbol {e}_{i}\} _ {i = 1}^{n}\\ }}I(\{\boldsymbol{x}_{i}\}_{i = 1}^{n};\phi_{M}|\{\boldsymbol{e}_{i}\}_{i = 1}^{n},D),
$$
$$
s. t. \sum_ {i = 1} ^ {n} \lambda_ {i} \leq C, \tag {7}
$$
where  $e_i = \{(j_i, v_i), m_i\}$  is an experiment,  $x_i$  is the observed sample from the experiment  $e_i$ , that is,  $x_i \sim p_{m_i}(X_V | do(X_{j_i} = v_i))$ .  $\lambda_i$  is the cost of experiment  $e_i$ . This objective aims to design a series of experiments with budget C, which can reveal the information about  $\phi_M$  as much as possible. The number of intervention targets  $n$  is not a fixed value, which is constrained by the total budget.
The greedy method for MFACD. The greedy method designs each experiment independently, and at each step, it selects the experiment which can maximize the average information gain. Following [19], the  $k$ th experiment is determined based on the following objective:
$$
\underset {\boldsymbol {e} _ {k}} {\arg \max } \frac {I \left(\left\{\boldsymbol {x} _ {i} \right\} _ {i = 1} ^ {k - 1} \cup \boldsymbol {x} _ {k} ; \phi_ {M} \mid \left\{\boldsymbol {e} _ {i} \right\} _ {i = 1} ^ {k - 1} \cup \boldsymbol {e} _ {k} , D\right) - I \left(\left\{\boldsymbol {x} _ {i} \right\} _ {i = 1} ^ {k - 1} ; \phi_ {M} \mid \left\{\boldsymbol {e} _ {i} \right\} _ {i = 1} ^ {k - 1} , D\right))}{\lambda_ {m}} \tag {8}
$$