SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} I \left(\boldsymbol {x} _ {s}; \phi_ {M} | \boldsymbol {x}, e \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) + I \left(\boldsymbol {x} _ {t}; \phi_ {M} | \boldsymbol {x}, e \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) \\ = H \left(\boldsymbol {x} _ {s} \mid \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) + H \left(\boldsymbol {x} _ {t} \mid \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {t}, \boldsymbol {e} _ {t} \right\}, D\right) \\ - H \left(\boldsymbol {x} _ {s} \mid \phi_ {M}, \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) - H \left(\boldsymbol {x} _ {t} \mid \phi_ {M}, \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) \\ \geq H \left(\boldsymbol {x} _ {s}, \boldsymbol {x} _ {t} \mid \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) - H \left(\boldsymbol {x} _ {s}, \boldsymbol {x} _ {t} \mid \phi_ {M}, \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) - \epsilon \\ = I \left(\boldsymbol {x} _ {s}, \boldsymbol {x} _ {t}; \phi_ {M} | \boldsymbol {x}, \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D\right) - \epsilon . \\ \end{array}
$$
According to the basic mutual information property  $I(A,B;C) - I(B;C) = I(A;C|B)$ , we have:
$$
\begin{array}{l} I (\boldsymbol {x} \cup \boldsymbol {x} _ {s}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \}, D) - I (\boldsymbol {x}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \}, D) \\ + I (\boldsymbol {x} \cup \boldsymbol {x} _ {t}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \}, D) - I (\boldsymbol {x}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \}, D) \\ \geq I (\boldsymbol {x} \cup \left\{\boldsymbol {x} _ {t}, \boldsymbol {x} _ {s} \right\}; \phi_ {M} | \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D) - I (\boldsymbol {x}; \phi_ {M} | \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D) - \epsilon . \\ \end{array}
$$
$$
\begin{array}{l} I (\boldsymbol {x} \cup \boldsymbol {x} _ {s}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \}, D) + I (\boldsymbol {x} \cup \boldsymbol {x} _ {t}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \}, D) \\ \geq I (\boldsymbol {x} \cup \left\{\boldsymbol {x} _ {t}, \boldsymbol {x} _ {s} \right\}; \phi_ {M} | \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D) + I (\boldsymbol {x}; \phi_ {M} | \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D) - \epsilon . \\ \end{array}
$$
Since different experiments are independent, we have:
$$
\begin{array}{l} I (\boldsymbol {x} \cup \boldsymbol {x} _ {s}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {s} \}, D) + I (\boldsymbol {x} \cup \boldsymbol {x} _ {t}; \phi_ {M} | \boldsymbol {e} \cup \{\boldsymbol {e} _ {t} \}, D) \\ \geq I (\boldsymbol {x} \cup \left\{\boldsymbol {x} _ {t}, \boldsymbol {x} _ {s} \right\}; \phi_ {M} | \boldsymbol {e} \cup \left\{\boldsymbol {e} _ {s}, \boldsymbol {e} _ {t} \right\}, D) + I (\boldsymbol {x}; \phi_ {M} | \boldsymbol {e}, D) - \epsilon . \\ \end{array}
$$
Thus,  $I(\cdot ;\phi_{M}|\cdot ,D)$  is  $\epsilon$  -submodular.
F Proof of Theory 4
For clear presentation, we denote  $g(\{e_i\}_{i=1}^n) = I(\{x_i\}_{i=1}^n; \phi_M|\{e_i\}_{i=1}^n, D)$ , then we need to solve the following problem:
$$
\underset {\left\{\boldsymbol {e} _ {i} \right\} _ {i = 1} ^ {n}} {\arg \max } g \left(\left\{\boldsymbol {e} _ {i} \right\} _ {i = 1} ^ {n}\right), \tag {14}
$$
Suppose  $S^{*} = \{e_{i}^{*}\}_{i=1}^{n}$  is the optimal solution for objective (14), and the results of the greedy method is  $S = \{e_{i}\}_{i=1}^{n}$ , where the experiments are sequentially determined from  $e_{1}$  to  $e_{n}$ . We denote  $S_{1:j} = \{e_{i}\}_{i=1}^{j}$ , and  $\Delta(e|S_{1:j}) = g(S_{1:j} \cup e) - g(S_{1:j})$ , according to the greedy method, we have:
$$
\pmb {e} _ {j + 1} = \arg \max _ {\pmb {e}} \frac {\Delta (\pmb {e} | S _ {1 : j})}{\lambda_ {e}},
$$
where  $\lambda_{e}$  is the cost of experiment  $e$ .
Based on all the above notations, we have:
$$
\begin{array}{l} g (S ^ {*}) \leq g (S ^ {*} \cup S _ {1: j}) \\ = g \left(S _ {1: j}\right) + g \left(S _ {1: j} \cup e _ {1} ^ {*}\right) - g \left(S _ {1: j}\right) \\ + g \left(S _ {1: j} \cup \boldsymbol {e} _ {1} ^ {*} \cup \boldsymbol {e} _ {2} ^ {*}\right) - g \left(S _ {1: j} \cup \boldsymbol {e} _ {1} ^ {*}\right) \\ + \dots \\ + g \left(S _ {1: j} \cup \left\{\boldsymbol {e} _ {1} ^ {*},..., \boldsymbol {e} _ {n} ^ {*} \right\}\right) - g \left(X _ {1: i} \cup \left\{\boldsymbol {e} _ {1} ^ {*},..., \boldsymbol {e} _ {n - 1} ^ {*} \right\}\right) \\ = g \left(S _ {1: j}\right) + \sum_ {k = 1} ^ {n} \left[ g \left(S _ {1: j} \cup \left\{\boldsymbol {e} _ {1} ^ {*}, \dots , \boldsymbol {e} _ {k} ^ {*} \right\}\right) - g \left(X _ {1: i} \cup \left\{\boldsymbol {e} _ {1} ^ {*}, \dots , \boldsymbol {e} _ {k - 1} ^ {*} \right\}\right) \right] \\ \leq g \left(S _ {1: j}\right) + \sum_ {k = 1} ^ {n} \left[ g \left(S _ {1: j} \cup \left\{\boldsymbol {e} _ {k} ^ {*} \right\}\right) - g \left(S _ {1: j}\right) + \epsilon \right] \\ = g \left(S _ {1: j}\right) + \sum_ {k = 1} ^ {n} \left[ \Delta \left(\left\{e _ {k} ^ {*} \right\} \mid S _ {1: j}\right) + \epsilon \right], \\ \end{array}
$$
where the first inequality holds because of the non-decreasing property, and the second inequality holds because of the  $\epsilon$ -submodular property.
Since  $e_{j+1} = \arg \max_{e} \frac{\Delta(e|S_{1:j})}{\lambda_e}$ , we have  $\frac{\Delta(e|S_{1:j})}{\lambda_e} \leq \frac{\Delta(e_{j+1}|S_{1:j})}{\lambda_{e_{j+1}}}$  for any  $e$ , thus  $\Delta(e|S_{1:j}) \leq \frac{\lambda_e}{\lambda_{e_{j+1}}} \Delta(e_{j+1}|S_{1:j}) \leq B_\lambda \Delta(e_{j+1}|S_{1:j})$ . By bringing this result into the above equation, we have:
$$
\begin{array}{l} g \left(S ^ {*}\right) \leq g \left(S _ {1: j}\right) + \sum_ {k = 1} ^ {n} \left[ \Delta \left(\left\{e _ {k} ^ {*} \right\} \mid S _ {1: j}\right) + \epsilon \right] \\ \leq g \left(S _ {1: j}\right) + \sum_ {k = 1} ^ {n} \left[ B _ {\lambda} \Delta \left(e _ {j + 1} \mid S _ {1: j}\right) + \epsilon \right] \\ = g \left(S _ {1: j}\right) + n B _ {\lambda} \Delta \left(e _ {j + 1} \mid S _ {1: j}\right) + n \epsilon \\ \end{array}
$$
Let  $T_{j} = g(S^{*}) - g(S_{1:j})$ , we have:
$$
T _ {j} - T _ {j + 1} = g \left(S _ {1: j + 1}\right) - g \left(S _ {1: j}\right) = \Delta \left(\boldsymbol {e} _ {j + 1} \mid S _ {1: j}\right) \geq \frac {T _ {j} - n \epsilon}{n B _ {\lambda}}
$$
$$
\begin{array}{l} T _ {n} \leq \left(1 - \frac {1}{n B _ {\lambda}}\right) T _ {n - 1} + \frac {\epsilon}{B _ {\lambda}} \leq \left[ \left(1 - \frac {1}{n B _ {\lambda}}\right) \right] ^ {2} T _ {n - 2} + \left(1 - \frac {1}{n B _ {\lambda}}\right) \frac {\epsilon}{B _ {\lambda}} + \frac {\epsilon}{B _ {\lambda}} \\ \leq \ldots \leq [ (1 - \frac {1}{n B _ {\lambda}}) ] ^ {n} T _ {0} + [ (1 - \frac {1}{n B _ {\lambda}}) ] ^ {n - 1} \frac {\epsilon}{B _ {\lambda}} + \ldots + \frac {\epsilon}{B _ {\lambda}} \\ \end{array}
$$
Let  $B = [(1 - \frac{1}{nB_{\lambda}})]^{n - 1}\frac{\epsilon}{B_{\lambda}} +\ldots +\frac{\epsilon}{B_{\lambda}} = \frac{\epsilon}{B_{\lambda}}\sum_{i = 1}^{n}[(1 - \frac{1}{nB_{\lambda}})]^{i - 1}$ , and considering that  $[(1 - \frac{1}{nB_{\lambda}})]^n = e^{-\frac{1}{B_{\lambda}}}$ , we have:
$$
g (S ^ {*}) - g (S _ {1: n}) \leq e ^ {- \frac {1}{B _ {\lambda}}} g (S ^ {*}) + B
$$
Thus, we have  $g(S_{1:n}) \geq (1 - e^{-\frac{1}{B_\lambda}})g(S^*) - B$ .
Algorithm 1: Algorithm of Licence for Single Intervention Scenario
Input: Variable set  $X_{V}$  , number of oracles  $M$  , cost of oracles  $\pmb{\Lambda}$  , observational data  $D^{O}$  , total budget  $C$  , and learning rate  $\eta$  Output: Causal graph  $\phi_M$  1 Initialize the model parameter  $\Psi$  2 Optimize  $\Psi$  with the training process of ELBO under  $D^O$  3 Initialize  $D^{I} = \emptyset$  4 while Budget  $C$  does not run out do 5 Initialize  $j^{*},m^{*},v^{*}$  and let  $\zeta^{*} = -\infty$  6 for  $(j,m)$  in  $\{1,2,\dots,d\} \times \{1,2,\dots,M\}$  do 7 Calculate  $v^{*}(j,m)$  with BO. 8 if  $f(j,v^{*}(j,m),m) > \zeta^{*}$  then 9 Update  $j^{*}\gets j,m^{*}\gets m$  and  $v^{*}\gets v^{*}(j,m)$  10 Update  $\zeta^{*}\gets f(j,v^{*}(j,m),m)$  11 end 12 end 13 Subtract the budget with  $C\leftarrow C - \lambda_{m^*}$  14 Acquire  $(j^{*},v^{*},m^{*})$  towards the true causal graph to obtain  $\pmb{x}^{*}\sim p_{m}(X_{V}|do(X_{j} = v))$  15 Update  $D^{I}\gets D^{I}\cup \{\pmb{x}^{*}\}$  16 Optimize  $\Psi$  with training process of ELBO under  $D^O\cup D^I$  17 end 18 Sample  $\phi_M$  from  $p(\phi_M|D)$  19 return Causal graph  $\phi_M$
The algorithm for Licence method for single intervention scenario is shown in Algorithm 1. Moreover, the algorithm for Licence method for batch intervention scenario is shown in Algorithm 2.
H More Experiments
H.1 Experimental Settings
H.1.1 Datasets
The details of our experimental datasets are presented as follows:
- Erdős-Rényi (ER) [41] graph is a random graph introduced by Paul Erdős and Alfréd Rényi. For ER graph, a graph with  $n$  vertices is generated by connecting each pair of vertices with a probability  $p$ .
- Scale-Free (SF) [42] graph is a type of random graph that has a degree distribution following power law. A small number of vertices in SF graph own a large number of edges, while the vast majority of vertices have relatively few edges.
- DREAM [43] is the abbreviation for Dialogue for Reverse Engineering Assessments and Methods, which can estimate the reverse quality that causal discovery methods perform. Specifically, we use a biological graph generator GeneNetWeaver for our experiments, which is a real-word public dataset.
H.1.2 Baselines
The details of experimental baselines are demonstrated as follows. We utilize DiBS [53] as our basic graph representation component. For acquisition methods, we use AIT and CBED and obtain the query tuples of node and value.
- AIT [44] is an active learning method that utilize f-score to select intervention queries.
Algorithm 2: Algorithm of Licence for Batch Intervention Scenario
Input: Variable set  $X_{V}$  , number of oracles  $M$  , cost of oracles  $\pmb{\Lambda}$  , observational data  $D^{O}$  , total batch experiment step  $T$  , total budget  $C$  , and learning rate  $\eta$  Output: Causal graph  $\phi_M$  1 Initialize the model parameter  $\Psi$  .   
2 Optimize  $\Psi$  with training process of constraint based ELBO under  $D^O$    
3 Initialize  $B^{I} = \emptyset$    
4 for t in 1,2,...,T do   
5 while Budget  $C$  does not run out do.   
6 Initialize  $j^{*},m^{*},v^{*}$  and let  $\zeta^{*} = -\infty$  .   
7 for (j,m) in  $\{1,2,\dots,d\} \times \{1,2,\dots,M\}$  do Calculate  $v^{*}(j,m)$  with BO. if  $f(j,v^{*}(j,m),m) > \zeta^{*}$  then Update  $j^{*}\gets j,m^{*}\gets m$  and  $v^{*}\gets v^{*}(j,m)$  Update  $\zeta^{*}\gets f(j,v^{*}(j,m),m)$  end   
end   
14 Subtract the budget with  $C\leftarrow C - \lambda_{m^*}$    
15 Update  $B^{I}\gets B^{I}\cup \{(j^{*},v^{*},m^{*})\}$    
16 end   
17 Acquire  $B^{I}$  towards the true causal graph to obtain  $\{\pmb {x}^*\sim p_m(X_V|do(X_j = v))\}_{(j,v,m)\in B^I}$    
18 Update  $D^{I}\gets D^{I}\cup \{\pmb{x}^{*}\}_{(j,v,m)\in B^{I}}$    
19 Optimize  $\Psi$  with training process of constraint based ELBO under  $D^O\cup D^I$    
20 end   
21 Sample  $\phi_M$  from  $p(\phi_M|D)$    
22 return Causal graph  $\phi_M$
- CBED [8] is based on the calculation of mutual information (MI), which intend to select intervention queries with maximal MI scores after obtaining new samples under current queries.
For the batch intervention scenario, we extend above methods with greedy strategy, which can promise an lower bound for approximation with submodular property. For choosing the fidelities to query, we use two circumstances, i.e., REAL and RANDOM.
- REAL fidelity means the model always choose the highest fidelity to conduct experiments. This strategy is aligned with classic causal discovery under active learning paradigm without multi-fidelity settings, which can just choose the most accurate samples to conduct discovery process.
- RANDOM fidelity means the model choose different fidelities randomly with uniform probability.
H.1.3 Metrics
The details of experimental metrics are demonstrated as follows. We utilize SHD and AUPRC to reflect the topological structure discovering performance, and design RMSE to reflex the predicting performance of functional relations.
- SHD [45] is the abbreviation for Structural Hamming Distance, and it estimate the topological structure by counting the number of different edges on adjacency matrix. We calculate the expectation of SHD under multiple graph samplings.
- AUPRC [46] is the area under precision-recall curve, where we consider entities on the adjacency matrix as binary classification problem. The AUPRC is also under the expectation for multiple graph sampling.
- RMSE is designed for estimating the performance of grasping functional relations. We obtain several samples from the true causal graph, and let our model and the true causal function to conduct forward process respectively, then calculate the RMSE between the two results. We calculate RMSE by sampling graphs for multiple times.
Table 1: The left table demonstrate the details of the configuration of device and platform. The right table shows the details of time cost on computation.
Name	Details
CPU	Intel Xeon Platinum 8350C 2.60GHz
GPU	RTX A5000 (24GB)
Memory	42GB RAM
Python	Version 3.8
Java	Version 1.8.0 (Necessary for DREAM)
Model	Time (secs)
AIT-REAL	7.686
AIT-RANDOM	7.451
CBED-REAL	7.998
CBED-RANDOM	7.989
Licence	8.320
Table 2: The details of experimental settings.
Name	Explanation	Value
budget	The total budget for interventional experiments, (i.e., C).	10/20/30/40/50
oracle number	The number of oracles, (i.e., M)	3
oracle cost	The cost for each oracle, (i.e., Λ)	2, 8, 32
oracle noise	The extra additive noise for each oracle.	0.04, 0.02, 0.00
observation number	The number of observational samples.	1000
expect edge number	The number of expect edges.	2
additive noise	The value of additive noise during data generations.	0.01
H.2 Simulation of Oracles with Different Fidelities
For a given intervention  $(j,v)$ , suppose we have  $M$  oracles  $\{\phi_1,\phi_2,\dots,\phi_M\}$ , then the experiment results  $\{x_{j,v,1},x_{j,v,2},\ldots,x_{j,v,M}\}$  are specified as follows:
$$
x _ {j, v, m} = x _ {j, v, M} + \delta_ {m},
$$
$$
\delta_ {m} \sim N (0, \sigma_ {m}),
$$
where  $x_{j,v,M}$  is the ground truth, which can be directly obtained from the datasets. Since  $x_{j,v,m}$  is correlated with  $x_{j,v,M}$  by the first line, their underlying oracles  $\phi_m$  and  $\phi_M$  are correlated in our simulation. In our experiment, we set  $\delta_1 > \delta_2 > \ldots > \delta_M = 0$ . Suppose the cost of  $\phi_m$  as  $\lambda_m$ , then we set  $\lambda_1 < \lambda_2 < \ldots < \lambda_M$ .
H.3 Details of Configurations and Computation
The details of the configurations of device and platform are demonstrate in Table 1(left). We will show the details of the time cost on computation. We measure the time cost on the generation of each intervention per fidelity for all models, and the results are shown in Figure 1(right). We find that our method costs a little more than the baselines, which is probably due to the more complex sampling process in our model.
We also show the details of experimental settings for our overall experiments in Table 2. We carefully tune the hyper-parameters for baselines and our model, and the final values can be obtained in the configuration file in our codes.
H.4 Experiments on DREAM Dataset
We conduct experiments on a real-world biological dataset, called DREAM. Note that, DREAM does not support the calculation of RMSE, because of the lack of interface in this real-world dataset. We use two sub-datasets Ecoli and Yeast as our true causal graphs. The results are shown in Figure 4. We find that our model outperforms that other baselines on both Ecoli and Yeast, and both single and batch intervention scenario.
H.5 Experiments on More Nodes
In this section, we conduct further experiments on datasets with more nodes. We extend the number of nodes from 10 to 20, and experiment on the ER graph. The results are shown in Figure 3. We find that our model is still effective on the scenario of more nodes, and is better than baselines.
Figure 4: The performance among models on DREAM datasets with different datasets and budgets. Lower SHD, RMSE indicate better performances. We conduct each experiment for ten times, and report the average performances and error bars.
Table 3: SHD results of 20 nodes graphs on different budgets. Lower SHD indicates better performances. We conduct each experiment for ten times, and report average performances and error bars.
Model	Budget(10)	Budget(20)	Budget(30)	Budget(40)	Budget(50)
AIT-REAL	63.36±4.89	64.36±5.18	64.53±6.83	63.28±4.86	64.35±5.19
AIT-RANDOM	63.62±4.61	62.16±5.75	64.60±5.23	66.87±6.47	63.53±5.27
DiBS-REAL	63.58±6.35	61.50±7.69	63.50±6.86	63.56±6.34	61.45±7.69
DiBS-RANDOM	63.68±6.77	65.07±6.41	63.91±7.14	63.99±4.46	63.86±3.00
Licence	49.67±11.64	49.61±8.08	55.68±8.63	51.34±11.24	51.36±9.11
H.6 Supplementary Experiments on MAE Metric
We further compare our model with the baselines based on the Mean Absolute Error (MAE) metric. The experiments are conducted based on ER with different total budgets. The results are presented in Table 4.
Table 4: Results of the metric MAE  $(\%)$
Model	Budget(10)	Budget(20)	Budget(30)	Budget(40)	Budget(50)
AIT-REAL	3.46±0.01	2.43±0.01	2.63±0.01	3.46±0.01	2.42±0.01
AIT-RANDOM	3.71±0.02	2.42±0.01	2.82±0.01	2.68±0.00	2.54±0.01
CBED-REAL	3.73±0.01	2.56±0.00	2.54±0.00	3.69±0.01	2.52±0.00
CBED-RANDOM	4.00±0.02	2.53±0.00	2.88±0.00	3.14±0.01	2.70±0.01
Licence	2.06±0.00	2.20±0.01	1.70±0.00	1.77±0.00	2.07±0.00
The results indicate that our model surpasses the baselines in terms of MAE. This further provides evidence that the superior performance of our model is a general conclusion.
H.7 Supplementary Experiments on Different Oracle Settings
To demonstrate that our model is generally effective for different cost- and noisy-levels. We conduct experiments based on different sets of oracles. In specific, the experiments are conducted based on the following settings in Table 5. The results are presented in Figure 5.
From the results, we can see that our model can always perform better than the baselines on different sets of oracles.
Table 5: Settings for different cost and noise levels.
Setting 1	Setting 2
Oracle (m)	cost (λm)	noise (σm)	Oracle (m)	cost (λm)	noise (σm)
1	2	0.04	1	2	0.04
2	8	0.02	2	8	0.02
3	32	0	3	16	0
Setting 3	Setting 4
Oracle (m)	cost (λm)	noise (σm)	Oracle (m)	cost (λm)	noise (σm)
1	2	0.04	1	2	0.04
2	4	0.02	2	8	0.02
3	32	0	3	32	0
Setting 5	Setting 6
Oracle (m)	cost (λm)	noise (σm)	Oracle (m)	cost (λm)	noise (σm)
1	2	0.08	1	2	0.04
2	8	0.02	2	8	0.03
3	32	0	3	32	0
H.8 Supplementary Experiments on Regularization Coefficient  $\lambda$
In our model, the  $\epsilon$ -independent constraint in Equation 10 is an important contribution. In the optimization process, we convert it to the objective. We study the influence of the coefficient  $\lambda$  by tuning it in the range of  $[10^{-5}, 10^{-6}, 10^{-7}, 10^{-8}, 10^{-9}]$ . The results are presented in Figure 6.
From the results, we can see the performances of our model varies as we set different  $\lambda$ 's. In most cases, the best performance is achieved when  $\lambda$  is moderated (not too large or too small).
H.9 Supplementary Experiments on Ablation Studies
To study whether the correlation modeling between different oracles are necessary, we first build a variant of our model by regarding different oracles as independent components, that is, removing the links between different  $\phi$ 's in Figure 1, and then compare our model with such variant. The results are presented in Table 6.
Table 6: Results of the Licence and Licence without the cascaded relation.
Metrics	Model	Budget(10)	Budget(20)	Budget(30)	Budget(40)	Budget(50)
SHD ↓	Licence (w/o rel)	14.61±2.30	15.25±2.74	15.47±3.74	15.17±4.30	18.52±4.65
Licence	14.67±2.98	14.73±2.04	14.29±3.20	14.83±3.24	15.02±3.01
AUPRC (%) ↑	Licence (w/o rel)	28.96±1.28	35.05±1.35	31.74±1.50	37.85±2.03	33.70±4.36
Licence	35.75±2.13	25.12±2.01	41.79±1.71	40.89±2.92	41.76±3.19
RMSE (%) ↓	Licence (w/o rel)	3.34±0.04	3.10±0.02	2.46±0.00	2.61±0.00	2.83±0.00
Licence	2.82±0.01	2.75±0.00	2.40±0.00	2.68±0.01	2.69±0.01
We can see, in most cases, our model can achieve better performance than its variant without modeling the correlations between different oracles.
I Potentially Negative Social Impact
Causal discovery focuses on understanding causal relationships between variables. While causal discovery has the potential to bring about positive social impacts, it is important to consider both the positive and negative implications of its applications. In this response, I will focus on the negative impact of causal discovery.
- Reductionism and Oversimplification. Causal discovery techniques often aim to identify simple cause-and-effect relationships. However, complex social phenomena often involve a multitude of interconnected factors, making it difficult to capture the full complexity of the system. Relying solely on causal discovery may lead to oversimplification and reductionism, neglecting the nuanced interactions between variables.
Figure 5: Results of different settings of oracles in terms of cost and noise.
Figure 6: Results of extensive experiments on regularization  $\lambda$ .
- Ethical Concerns. Causal discovery can involve analyzing sensitive data, such as personal information or medical records. If not handled carefully, the use of this data can raise significant ethical concerns related to privacy, consent, and potential discrimination. Improper handling of data could lead to violations of privacy and unfair treatment of individuals or groups.
- Overreliance on Correlation. Causal discovery often relies on identifying statistical correlations between variables. However, correlation does not imply causation, and there is a risk of mistakenly inferring causal relationships based solely on correlation. Overreliance on such methods can lead to erroneous conclusions, leading to misguided decision-making and ineffective interventions.
- Social Bias and Inequality. Causal discovery relies on the data used for analysis, which can reflect existing biases and inequalities present in society. If the data used is biased, the causal relationships discovered may perpetuate or exacerbate existing social inequalities. Causal discovery methods need to be sensitive to potential biases and strive for fairness and inclusivity in both data collection and analysis.
In conclusion, while causal discovery holds promise in understanding complex systems, it is crucial to consider its potential negative impacts. Oversimplification, ethical concerns, overreliance on correlation, and social bias are all factors that need to be addressed to ensure responsible and beneficial applications of causal discovery. It is essential to approach this field with caution and incorporate broader societal considerations to mitigate the negative impacts and harness its potential for positive social change.
J Limitations
In this section, we analyze the limitations of our work, including sub-optimum of greedy method, estimation of mutual information, and scale of causal graph.