SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 3: Mean statistics over 20 repetitions: 100 variables and sample size 1,000.
(a) Non-Gaussian
Figure 4: Mean statistics over 20 repetitions: scale-free, 100 variables, average degree 20 (when not varied), and sample size 1,000.
(b) High Dimension
important to note that nearly all of these simulations were infeasible for GRaSP before implementing it with GSTs. Table 2, the corresponding table, only reports results for B0SS since the two algorithms have nearly identical performance on all statistics except for running time. Full results are tabulated in the Supplement.
6 Validation on fMRI
The high spatial coverage of fMRI's has allowed researchers to study brain function at different scales, from voxels to cortical parcellations to functional systems (such as default mode or visual systems). Given the potential differences in spatial dimension, the expected connectivity density of functional brain networks remains unclear. For example, even after keeping only the 75th percentile stronger connections, structural connectivity networks from 90 cortical regions [23] had 1012 connections on average. Since structural connectivity supports functional connectivity, we could expect empirical
Table 2: Mean statistics over 10 repetitions: scale-free, average degree 20, and sample size 1,000.
Variables	Algorithm	BIC λ	Adj Pre	Adj Rec	Ori Pre	Ori Rec
500	BOSS	2	0.98	0.80	0.97	0.80
4	1.00	0.72	0.99	0.72
8	1.00	0.57	0.99	0.57
1000	BOSS	2	0.97	0.80	0.97	0.80
4	1.00	0.73	1.00	0.72
8	1.00	0.59	1.00	0.58
Table 3: fMRI simulated data with pseudo-empirical errors
Algorithm	Adj Pre	Adj Rec	Ori Pre	Ori Rec	ΔBIC	Edges	Seconds
BOSS	0.99	0.94	0.96	0.90	211.79	951.40	15.46
fGES	0.97	0.60	0.70	0.43	7784.48	617.35	5.16
DAGMA	1.00	0.69	0.98	0.67	3080.85	687.15	54.58
LINGAM	0.54	0.94	0.35	0.62	3868.93	1752.75	582.03
functional connectivity networks to be in that order of magnitude. Previous studies applying causal discovery methods to fMRI simulated networks with a high number of variables and connections have shown that while the adjacency recovery precision of these methods can be very high, they usually show a low adjacency recall [16, 20]. Despite possibly having low recall, the limited applications of causal discovery methods to real world data [3, 18] often recover models with average degree greater than 20. Considering this limitation and the likelihood that real fMRI brain networks have high average connectivity (degree), causal discovery methods capable of reducing the number of false-negative connections will substantially improve future analysis of fMRI data.
To demonstrate its practical utility in this important real-world domain, we apply BOSS to two types of resting-state fMRI data: simulated data with pseudo-empirical noise distributions derived from randomized empirical fMRI cortical signals and clinical data from 3T fMRI scans processed into cortical parcels.
6.1 Simulated fMRI
We simulated fMRI data following the approach in [19]. Networks were based on a directed random graphical model that prefers common causes and causal chains over colliders. 40 networks were simulated with 200 variables and an average degree of 10. Edge weights were sampled from a uniform distribution in the interval  $[0.1, 0.4]$ , randomly setting  $10\%$  of the coefficients to their negative value. Pseudo-empirical noise terms were produced by randomizing fMRI resting-state data across data points, regions, and participants from the Human Connectome Project (HCP). Using pseudo-empirical terms better captures the marginal distributional properties of the empirical fMRI. One thousand data points were generated from this procedure. These data are available for download at: https://github.com/cmu-phil/boss.
Accuracy and timing results are shown in Table 3 for BOSS, fGES, DAGMA, and LiNGAM. These results show that BOSS has by far the best BIC score of the group, compared to the BIC score of the true model, and that the running time of BOSS is very reasonable for a problem of this size. Precisions and recalls for BOSS are quite high, for both adjacencies and orientations. The poor performance of LiNGAM is due to insufficient non-Gaussian signal. More details are included in the Supplement.
6.2 Clinical fMRI
We applied BOSS to 171 3-Tesla resting state fMRI scans from patients beginning treatment for alcohol use disorder [3]. Before applying BOSS, the data were cleaned and parcellated into 379 variables representing biologically interpretable and spatially contiguous regions [6]. More details on the data collection, cleaning, and processing can be found in [3]. Figure 5 reports the in/out-degree distributions of the resulting models. The solid line depicts the median degree across all graphs and
Figure 5: In/out-degree distributions on clinical resting state fMRI data.
the color region shades in the area between the 2.5 and 97.5 percentiles. These plots indicate that the models are scale-free, with a small number of hub vertices with much higher degree than other vertices. Scale-free connectivity is consistent with biological expectations and prior connectivity analysis on fMRI data [18]. More details are included in the Supplement.
We have proposed a successor to the GRaSP algorithm [12] that retains its high accuracy but is faster and more scalable. BOSS implemented with GSTs comfortably scales to least 1000 variables with an average degree of at least 20. It can comfortably and informatively analyze data from 400 or even 1000 densely connected fMRI cortical parcellations. We show in simulations that our method is highly accurate for Erdős-Rényi graphs as well as scale-free graphs. Despite being developed for the linear Gaussian case, BOSS also performs well in the linear non-Gaussian case. BOSS is fast and accurate on simulated fMRI data and can rapidly produce informative and plausible models from clinical fMRI data. BOSS is available for use within the TETRAD project which includes Python and R wrappers [17].
The success of BOSS presents an opportunity to pursue further theoretical work showing how BOSS differs from GRaSP and what general lessons we may learn for constructing successor algorithms to BOSS. There is substantial room for additional improvements, as BOSS does have several limitations. For example, BOSS cannot handle most forms of unmeasured confounding, so it will be valuable to explore ways of adding this functionality while maintaining its accuracy and scalability. It will also be informative to apply BOSS to other types of data such as functional genomic data, financial data, and electronic health records.
We thank our anonymous reviewers for their detailed and insightful comments. BA was supported by the US National Institutes of Health under the Comorbidity: Substance Use Disorders and Other Psychiatric Conditions Training Program T32DA037183. JR was supported by the US Department of Defense under Contract Number FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software Engineering Institute. RSR was supported by the US National Institutes of Health under awards R01AG055556 and R01MH109520. JC was supported by the US National Institutes of Health under awards K01AA026349 and UL1TR002494, the UMN Medical Discovery Team on Addiction, and the Westlake Wells Foundation. EK was supported by the US National Institutes of Health under award UL1TR002494. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of these funding agencies. All authors have no conflicts of interest to report.
[19] Sánchez-Romero, R. and Cole, M. W. (2021). Combining multiple functional connectivity methods to improve causal inferences. Journal of cognitive neuroscience, 33:180-194.
[20] Sánchez-Romero, R., Ramsey, J. D., Zhang, K., Glymour, M. R., Huang, B., and Glymour, C. (2019). Estimating feedforward and feedback effective connections from fmri time series: Assessments of statistical methods. Network Neuroscience, 3:274-306.
[21] Schwarz, G. (1978). Estimating the dimension of a model. The annals of statistics, pages 461-464.
[22] Shimizu, S., Inazumi, T., Sogawa, Y., Hyvarinen, A., Kawahara, Y., Washio, T., Hoyer, P. O., Bollen, K., and Hoyer, P. (2011). DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. Journal of Machine Learning Research-JMLR, 12:1225-1248.
[23] Škoch, A., Rehák Bučková, B., Mares, J., Tintěra, J., Sanda, P., Jajcay, L., Horáček, J., Španiel, F., and Hlinka, J. (2022). Human brain structural connectivity matrices—ready for modelling. Scientific Data, 9:486.
[24] Solus, L., Wang, Y., and Uhler, C. (2021). Consistency guarantees for greedy permutation-based causal inference algorithms. Biometrika, 108:795-814.
[25] Spirtes, P., Glymour, C., and Scheines, R. (2000). Causation, prediction, and search. MIT press.
[26] Teyssier, M. and Koller, D. (2005). Ordering-based search: a simple and effective algorithm for learning Bayesian networks. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, pages 584-590.
[27] Uhler, C., Raskutti, G., Buhlmann, P., and Yu, B. (2013). Geometry of the faithfulness assumption in causal inference. The Annals of Statistics, 41:436-463.
[28] Zhang, J. (2013). A comparison of three Occam's razors for Markovian causal models. The British Journal for the Philosophy of Science, 64:423-448.
[29] Zhang, J. and Spirtes, P. (2008). Detection of unfaithfulness and robust causal inference. *Minds and Machines*, 18:239–271.