Paper:
Learning Bayesian Networks with Thousands of Variables
We present a method for learning Bayesian networks from data sets containing thousands of variables without the need for structure constraints. Our approach is made of two parts. The first is a novel algorithm that effectively explores the space of possible parent sets of a node. It guides the exploration towards the most promising parent sets on the basis of an approximated score function that is computed in constant time. The second part is an improvement of an existing ordering-based algorithm for structure optimization. The new algorithm provably achieves a higher score compared to its original formulation. Our novel approach consistently outperforms the state of the art on very large data sets.

Review:
The paper describes tricks to scale Bayesian network structure learning to thousands of variables. This is achieved by developing new heuristics for candidate parent set identification and the subsequent order based structure optimization.In general, the paper is clearly written and easy to read. There are issues in editing and style, but the problems do not affect readability (much). The suggested heuristics feel bit ad-hoc, thus the value of the work is eventually judged by empirical evaluation. In this regard, I feel that the work could be better. Notably, the comparisons to simple local greedy searches (that scale well to 10000 variables too) should be included or referred to.In the following, I will give more detailed comments:- Intro, second paragraph: "Usually" - I do not know where this "usually" comes from, but for me it is annoying. It also occurs in many other places in a text, and more often than not it provokes a counter-reaction. Yes, such a two phase strategy has been used before, but to say it is "usually" done this way is a statement that has to be proven right.- Page 2, second paragraph. It is inconsiderate to use citation as a part-of-speech as in [17]. Even for an expert reader, that requires stopping reading and looking for the reference. - Page 2, the BIC formula looks a bit ugly. It would have been better to do $BIC(G)=\sum_{i=1}^n BIC(X_i,\Pi_i)$, and  $$ BIC(X_i,\Pi_i) =...- Page 3, 3.1. The explanation makes the reader suspicious. It is exactly the non-monotonic nature of the effect of combining the parent sets that makes the structure learning hard. How would BIC*  manage with Z=X xor Y (or more generally a parity function). But, well, the experiments will tell.- Page 5, Which data was used to produce Fig 1.- Page 5, while the space of orderings is indeed less that space of DAGS, 1000! is still quite a big number, so calling it "convenient" is a bit of a stretch.- Page 6, 5.1. Sample sizes would have been nice to have too. Maybe n/N in Table 1.- Page 7, last paragraph. Gobnilp is said to be better because it is exact solver. Now, it is exact solver also when n > 500, so how can it all of the sudden be worse? I mean, the explanations as they stand now, are not very convincing. - Page 7. How did you set the structure and parameters for random networks?All in all, it is a rather simple paper and I find it interesting to scale the BN structure learning to 10 000 variables. I really would have liked to see a better empirical comparison with heuristic methods like greedy local search also with big enough sample sizes that allow a big number of parents for the optimal network.