As expected, the number of live points K is an impor-tant setting but it is even more crucial in Nessai since it limits the amount of training data available. We find that a minimum of 1000 live points is required and for more complex problems, such as gravitational-wave inference, at least 2000 live points should be used.

There are a large number of settings which relate to the complexity of the normalising flow. Whilst tuning the sampler we found that the number of coupling trans-formations greatly affected convergence. If too many transforms were used the algorithm was prone to over-constraining the posterior distribution. We attribute this to the complexity of the iso-likelihood contour learnt by the flow, if the flow has too many trainable parameters it can over-fit the distribution and exclude regions of the parameter space which should be sampled. At the other extreme, if the model is too simple then resulting contour can “smooth” fine details and more samples are drawn outside of the initial likelihood constraint. These will not be accepted and the sampling process is therefore less ef-ficient. We use a similar logic for the number of neurons and layers in the neural network that parameterises the flow but we find that these parameters predominantly affect training time with a lesser effect on overall con-vergence. Another parameter that is important to con-sider is the batch size, during sampling the normalising flow can be training upwards of 100 times. Hence, a larger batch size is recommended since it can greatly re-duce training time, we also recommend increasing the batch size when using reparameterisations that increase the amount of training data, such as the boundary inver-sion described in section III B and appendix B.

We note that the size of pool of new samples effects the efficiency of the algorithm and the total run-time. If the pool-size is small then the normalising flow is fre-quently retrained, in extreme case where the proposal is inefficient due to, for example, the complexity of the parameter space, then the normalising can be retrained multiple times during a single iteration. Conversely, if the pool-size is large then if the flow is force-ably re-trained a number of points are discarded or, if the flow is only retrained once the pool is empty, then the rejection sampling becomes in-efficient since a large fraction of the potential new points will lie outside the likelihood bound. We instead opt to inversely scale the pool-size given the mean acceptance of the sampler since the last iteration the flow was trained. We recommend setting the base pool-size to the number of live points, only retraining the model when the pool is empty and setting the maxi-mum pool-size to be ten times the base pool-size. We use these settings for the results in section V and find that this results in a median of 263 training instances required to reach convergence.

As mentioned previously, approximately 40% of the run-time is spent on populating the pool of new sam-ples. This is directly attributable to the efficiency of the rejection sampling required to ensure samples are dis-tributed according to the prior. In section III A we pro- pose two methods for drawing samples within the con-tour in the latent space, these produce uniformly and normally distributed samples respectively. In practice we find the two methods comparable in most cases with the exception of when the latent radius lies in the tail of the chi-distribution that corresponds to the latent prior \( p _ { \mathcal { Z } } \) . In this case using the uniform distribution results in lower population and proposal acceptances which leads to longer run-times.

# VI. CONCLUSIONS

We have proposed a novel method for sampling within a given iso-likelihood contour according to the prior that can be incorporated into the standard nested sampling algorithm. Our method employs normalising flows to learn the density of current set of live points which, once trained, allows us to produce points within the contour by sampling from a simple distribution and using rejec-tion sampling. The use of normalising flows allows us to avoid using multiple bounding distributions and since new samples are independent of the previous samples we eliminate the need to use a random walk. We imple-ment this proposal method in our sampler, Nessai, and conduct a series of tests to verify that it recovers the cor-rect Bayesian posteriors and then compare our results to those obtained with another sampler to determine if our design does in fact result in a more efficient sampler.

We apply our sampler to 128 four second duration simulated signals from the coalescence of binary black hole systems sampled at 2048 Hz and we run two sepa-rate analyses, one with distance marginalisation and an-other without. The resulting P-P plots (fig. 3) show that our sampler more reliably recovers the posterior distribu-tions with distance marginalisation than without, how-ever both pass the P-P test. This indicates that our proposal method does not introduce any inherent biases.

We use dynesty for the comparison, which has been shown to produce results consistent with those used in previous LVK analyses [36]. We find that our sampler returns evidences consistent with dynesty, which serves as further verification of our results. Since we aim to pro-duce a more efficient sampler we also compare the like-lihood evaluations required to reach convergence. When not using distance marginalisation we find that Nessai requires 5.04 \( \times 1 0 ^ { 6 } \) likelihood evaluations, 2.07 times fewer than dynesty. When distance marginalisation is en-abled Nessai requires \( 7 . 2 2 \times 1 0 ^ { 6 } \), which, whilst still 1.34 fewer than dynesty, is more than with the marginali-sation disabled. As such, we recommend using Nessai without distance marginalisation for gravitational-wave inference.

However, this reduction in likelihood evaluations does not relate directly to the total computation time because of the additional costs associated with sampling, which for Nessai are associated with training the normalis-ing flow and populating the pool of new samples. We

find that the fraction of the time spent of each stage changes when using distance marginalisation. Without the marginalisation, on average, 8% of the total computa-tion time is spent on training and a further 40% on popu-lation. When using distance marginalisation this changes to 5% spent on training and 42% on population. We at-tribute the difference in population time to the efficiency of the rejection sampling, which is improved when includ-ing the reparameterisation for distance discussed in sec-tion III C. We find that without distance marginalisation the median run-time for Nessai is 2.32 times faster than dynesty. However when distance marginalisation is en-abled we observe that, on average, Nessai is only 1.40 times faster than dynesty. This further reinforces our recommendation to use Nessai with distance marginali-sation disabled.

We also show how our sampler can make use of paral-lelised likelihood functions by evaluating the likelihood of new live points during the population stage. We repeat the previous analysis for a single injection without dis-tance marginalisation and parallelise the likelihood com-putation with increasing number of threads up to 16. We observe that the reduction time evaluating the likelihood does not quite match the theoretical values, indicating that there is a small overhead associated with it. This also highlights how the limiting factor is the time spent training the normalising flow and populating the pool of new live point.

To aid in diagnosing potential biases during sampling, we include a series of diagnostics in our sampler which allow us to easily identify under and over-constraining. These diagnostics also help to tune the sampling settings and highlight how periodically re-training the normal-ising flow during sampling prevents the proposal from becoming inefficient during sampling.

We find that our algorithm is susceptible to under-sampling regions of the parameter space which are close to the prior bounds. We consequently introduce the previously described reparameterisations to mitigate this and a series of diagnostics to aid in diagnosing biases and correctly tuning the settings. We aim address this in fur-ther work with changes to the design of the normalising flows we have used.

It is natural to compare this work to [27–30] which use variational autoeconders and normalising flows to pro-duce posterior distributions. Our approach differs from these in that it requires no prior computation since train-ing occurs during sampling and we do not introduce any assumptions about the data other than those necessary to apply a nested sampling algorithm. Nessai is therefore a drop-in replacement for existing sampling algorithms that does not require changes to existing pipelines. In future work we aim to evaluate our sampler us-ing more expensive waveform models including those for longer duration signals, such as those from binary neu-tron star of neutron star-black hole system, and models which include higher-order modes. We will also investi-gate the suitability of other types of normalising flow transforms, such as the spline based transforms from[68] and flows which allow for specifying a manifold [69].These changes could improve the efficiency of the pop-ulation stage which is currently the slowest part of the algorithm. Another possible approach for reducing the cost of population is using alternative reparameterisa-tions for parameters such as the spins magnitudes, which we observe to be two of the most challenging parameters to sample.

In summary, we have proposed a novel variation of the standard nested sampling algorithm that incorpo-rates normalising flows specifically designed for inference with computationally expensive likelihood functions. We have applied our sampler to the problem of gravitational wave inference and shown that it consistently recovers the Bayesian posteriors distributions and evidences with 2.07 times fewer total likelihood evaluations than dynesty, another commonly used sampler, which translates to a 2.32 times reduction in computation time. Our sampler therefore serves as a more efficient drop-in replacement for existing samplers.

# ACKNOWLEDGMENTS

The authors gratefully acknowledge the Science and Technology Facilities Council of the United Kingdom. MJW is supported by the Science and Technology Fa-cilities Council [2285031]. JV and CM are supported by the Science and Technology Research Council [ST/L000946/1]. CM is also supported by the European Cooperation in Science and Technology (COST) action[CA17137]. The authors are grateful for computational resources provided by Cardiff University, and funded by an STFC grant supporting UK Involvement in the Oper-ation of Advanced LIGO.

Software: Nessai was initially developed using cp-nest [21] with permission from the authors and still shares a similar interface and other core codes. Nessai is implemented in Python and uses NumPy [70], SciPy[71], pandas [72, 73], nflows [47], PyTorch [46], mat-plotlib [74] and seaborn [75]. Gravitational wave in-jections were generated using Bilby and bilby pipe [25].Figures were prepared using matplotlib [74], seaborn[75], Bilby [25] and corner [76].