Paper:
A Likelihood-Free Inference Framework for Population Genetic Data using Exchangeable Neural Networks
An explosion of high-throughput DNA sequencing in the past decade has led to a surge of interest in population-scale inference with whole-genome data. Recent work in population genetics has centered on designing inference methods for relatively simple model classes, and few scalable general-purpose inference techniques exist for more realistic, complex models. To achieve this, two inferential challenges need to be addressed: (1) population data are exchangeable, calling for methods that efficiently exploit the symmetries of the data, and (2) computing likelihoods is intractable as it requires integrating over a set of correlated, extremely high-dimensional latent variables. These challenges are traditionally tackled by likelihood-free methods that use scientific simulators to generate datasets and reduce them to hand-designed, permutation-invariant summary statistics, often leading to inaccurate inference. In this work, we develop an exchangeable neural network that performs summary statistic-free, likelihood-free inference. Our framework can be applied in a black-box fashion across a variety of simulation-based tasks, both within and outside biology. We demonstrate the power of our approach on the recombination hotspot testing problem, outperforming the state-of-the-art.

Review:
In population genetics, a common problem is the inference of evolutionary events and parameters given some observed DNA sequencing data. This is typically done by leveraging genetic simulators based on the coalescence model. For example, in approximate Bayesian Computation (ABC) one can infer the posterior distribution of the evolutionary parameter of interest by (i) drawing its value from a prior, (ii) generate population genetic data from a simulator using the drawn parameter value and (iii) weight the importance of the sample based on the similarity between the simulated and the real data. Similarity is typically defined in terms of a pre-chosen set of summary statistics.  The authors propose an inference strategy that does not rely on any choice of summary statistics. Briefly, the authors propose to use a neural network to invert the generative process underlying the coalescent simulator. Specifically, one can use steps (i) and (ii) as in ABC to generate multiple pairs (parameter value, population genetic data), and then train a neural network that can predict the parameter value given the genetic data. After training, a posterior can be computed by forward passing the real data through the network. As training data is only based on simulators, the authors propose a simulation-on-the-fly method where each training point is generated afresh. Finally, the authors consider an exchangeable neural network architecture to account for the exchangeability of individual genetic sequences in the population (in the case of unrelated individuals).  In an application to the recombination hotspot testing problem, the authors demonstrate the advantages of employing a “simulation-on-the-fly” strategy and an exchangeable architecture. Finally, they show that their approach competes with a state-of-the-art method for recombination hotspot testing.   The method introduced in this paper is novel and interesting and can in principle be extended to tackle different problems in population genetics. However, I have some points I would like to bring up:  1) The authors say that the neural network is used to learn a posterior of the evolutionary parameters. While this is not wrong, I think it may be confusing. If my understanding is correct, I think the neural network is mostly used to invert the generative process underlying the coalescent simulator. It is trained using simulated data (where the groundtruth value of the parameter to infer is known) while after training, it can be used to “invert the real data into a posterior on the parameter of interest”.  2) The authors should make more explicit which is the loss of the exchangeable neural network. My understanding is that in most setting the evolutionary parameters can take a finite number of discrete values and that in this case, a cross-entropy loss would be used. Is this correct? Also, what do the authors recommend if the parameter to infer is continuous?  3) In the recombination hotspot testing problem, the neural network is trained to classify 20-SNP windows in positive/negative cases of recombination hotspots. While the window size may be fine for this inference problem, this choice could be suboptimal when inferring other parameters. Of course, one could increase the window size, however, the number of weights in the dense layers would also increase (if the same neural network architecture is the same). Do the authors have any comment on this?  Minor: - Related to point 1, in Section 3.1, I think that the authors should remind to the reader that there are observed genetic data and that the goal is to infer the posterior of some evolutionary parameter given the observed data. - Which test statistic is used to obtain the ROC curves in in figure 4 for the two methods?  I raised my score from 7 to 8 after the reading the response from the authors.