Original: Variational Likelihood-Free Gradient Descent
Revised: Variational Likelihood-Free Gradient Descent

Original: Anonymous Authors

Original: Anonymous Institution

Original: Abstract
Revised: Abstract

Original: In many scientific applications, we do not have explicit access to the likelihood function.
Revised: In many scientific applications, we do not have explicit access to the likelihood function.

Original: However simulations of the process of interest, using different parameter settings, may give us access to the likelihood function implicitly.
Revised: However simulations of the process of interest, using different parameter settings, may give us access to the likelihood function implicitly.

Original: The methodology for approximating likelihoods and posterior distributions based on simulated observations can be described as simulation-based inference.
Revised: The methodology for approximating likelihoods and posterior distributions based on simulated observations can be described as simulation-based inference.

Original: In this paper, we propose a simulation-based inference algorithm in which we iteratively update particles to more closely resemble the posterior.
Revised: In this paper, we propose a simulation-based inference algorithm in which we iteratively update particles to more closely resemble the posterior.

Original: Our approach utilises simulations to estimate a density ratio function at each iteration and then uses it to approximate the KL divergence between the particle density and the posterior density.
Revised: Our approach utilises simulations to estimate a density ratio function at each iteration and then uses it to approximate the KL divergence between the particle density and the posterior density.

Original: By alternating between gradient descent and density ratio estimation, the approximated KL divergence is minimized.
Revised: By alternating between gradient descent and density ratio estimation, the approximated KL divergence is minimized.

Original: We benchmark the performance of our algorithm on a
Revised: We benchmark the performance of our algorithm on a

Original: Gaussian mixture model and the M/G/1 queue process model and report promising results
Revised: Gaussian mixture model and the M/G/1 queue process model and report promising results.

Original: 1. Introduction
Revised: 1. Introduction

Original: Scientific enquiry requires the construction of testable and falsifiable models that describe phenomena of interest.
Revised: Scientific enquiry requires the construction of testable and falsifiable models that describe phenomena of interest.

Original: Historically, this has led to the development of likelihood- and
Revised: Historically, this has led to the development of likelihood- and

Original: Bayesian-based approaches to inference, focusing on closed-form representations of the likelihood p ( x | θ ) and tractable posterior distributions p ( θ | x ).
Revised: Bayesian-based approaches to inference, focusing on closed-form representations of the likelihood p ( x | θ ) and tractable posterior distributions p ( θ | x ).

Original: More recently there has been a trend to focus on the stochastic mechanism whereby the data are generated (Diggle and
Revised: More recently there has been a trend to focus on the stochastic mechanism whereby the data are generated (Diggle and

Original: Gratton, 1984).
Revised: Gratton, 1984).

Original: In this case the likelihood is defined implicitly and a variety of approaches have been proposed to utilise it effectively for statistical inference.
Revised: In this case the likelihood is defined implicitly and a variety of approaches have been proposed to utilise it effectively for statistical inference.

Original: The relationship between our desired posterior p ( θ | x ) and the likelihood is given in Bayes’ formula p ( θ | x ) =
Revised: The relationship between our desired posterior p ( θ | x ) and the likelihood is given in Bayes’ formula p ( θ | x ) =

Original: p ( x | θ ) p ( θ ) p ( x ) .
Revised: p ( x | θ ) p ( θ ) p ( x ) .

Original: Henceforth we shall call p ( θ ) the prior, and p ( x ) the marginal.
Revised: Henceforth we shall call p ( θ ) the prior, and p ( x ) the marginal.

Original: Of course exact calculations of the posterior are unfeasible in the absence of the likelihood function.
Revised: Of course exact calculations of the posterior are unfeasible in the absence of the likelihood function.

Original: Despite this, much work has been done in literature to give approximate solutions to these problems, e.g. Approximate Bayesian Computation (ABC) (Beaumont et al., 2002;
Revised: Despite this, much work has been done in literature to give approximate solutions to these problems, e.g. Approximate Bayesian Computation (ABC) (Beaumont et al., 2002;

Original: Sisson et al., 2018).
Revised: Sisson et al., 2018).

Original: For a more general overview see (Cranmer et al., 2020) which covers a variety of simulation-based inference (SBI) methods.
Revised: For a more general overview see (Cranmer et al., 2020) which covers a variety of simulation-based inference (SBI) methods.

Original: In the SBI literature the implicit likelihood is assumed to be computationally expensive, and thus the emphasis of these methods is to provide accurate inference whilst using a minimal number of simulations.
Revised: In the SBI literature the implicit likelihood is assumed to be computationally expensive, and thus the emphasis of these methods is to provide accurate inference whilst using a minimal number of simulations.

Original: In recent years, it has been noticed that if we can approximate the ratio p ( θ | x ) p ( θ ) =
Revised: In recent years, it has been noticed that if we can approximate the ratio p ( θ | x ) p ( θ ) =

Original: p ( x | θ ) p ( x ) by a function r ( θ , x ), then given an observation x obs , we can easily approximate the posterior p ( θ | x obs ) by using r ( θ , x obs ) p ( θ ).
Revised: p ( x | θ ) p ( x ) by a function r ( θ , x ), then given an observation x obs , we can easily approximate the posterior p ( θ | x obs ) by using r ( θ , x obs ) p ( θ ).

Original: In Thomas et al.
Revised: In Thomas et al.

Original: (2021), they leverage logistic regression to estimate p ( x | θ 0 ) p ( x ) (where θ 0 is fixed) using samples from a simulator and samples from the marginal p ( x ).
Revised: (2021) they leverage logistic regression to estimate p ( x | θ 0 ) p ( x ) (where θ 0 is fixed) using samples from a simulator and samples from the marginal p ( x ).

Original: Similarly, Hermans et al.
Revised: Similarly, Hermans et al.

Original: first estimates [Equation] using samples from the joint probability p ( x , θ ) and the marginals p ( x ) and p ( θ ), then use MCMC to sample from the approximated posterior using r ( θ , x obs ) p ( θ ).
Revised: first estimates p ( x , θ ) p ( x ) p ( θ ) using samples from the joint probability p ( x , θ ) and the marginals p ( x ) and p ( θ ), then use MCMC to sample from the approximated posterior using r ( θ , x obs ) p ( θ ).

Original: Instead of estimating the density ratio in one-shot, Hermans et al.
Revised: Instead of estimating the density ratio in one-shot, Hermans et al.

Original: also proposes a sequential approach which focuses on estimating the ratio accurately for θ parameters that likely generated x obs .
Revised: also proposes a sequential approach which focuses on estimating the ratio accurately for θ parameters that likely generated x obs .

Original: Durkan et al.
Revised: Durkan et al.

Original: points out the links between this sequential approach and conditional density estimators such as Papamakarios and Murray (2016); Lueckmann et al.
Revised: points out the links between this sequential approach and conditional density estimators such as Papamakarios and Murray (2016); Lueckmann et al.

Original: Motivated by variational inference algorithms (Blei et al., 2017), we propose to minimize the
Revised: Motivated by variational inference algorithms (Blei et al., 2017), we propose to minimize the

Original: KL divergence between the approximate posterior and the true posterior.
Revised: KL divergence between the approximate posterior and the true posterior.

Original: Specifically, we were inspired by a variational approach called Stein Variational Gradient Descent (SVGD) (Liu and Wang, 2016) and optimise a particle distribution to minimize the aforementioned
Revised: Specifically, we were inspired by a variational approach called Stein Variational Gradient Descent (SVGD) (Liu and Wang, 2016) and optimise a particle distribution to minimize the aforementioned

Original: KL divergence.
Revised: KL divergence.

Original: Note that SVGD cannot be readily applied to simulation-based inference due to lack of a tractable likelihood.
Revised: Note that SVGD cannot be readily applied to simulation-based inference due to lack of a tractable likelihood.

Original: In our method, we use density ratio estimation to approximate the log ratio term in the KL divergence and perform gradient updates on particles to reduce the approximated KL divergence.
Revised: In our method, we use density ratio estimation to approximate the log ratio term in the KL divergence and perform gradient updates on particles to reduce the approximated KL divergence.

Original: We obtain promising results on toy and real-world problems using the proposed method.
Revised: We obtain promising results on toy and real-world problems using the proposed method.

Original: In the rest of the paper, we formally introduce our method in Section 2, show toy and realworld experiment results in Section 3, andfinally discuss and conclude insection Section 4.
Revised: In the rest of the paper: we formally introduce our method in Section 2, show toy and real-world experiment results in Section 3, and discuss and conclude in Section 4.

Original: 2. The Proposed Method
Revised: 2. The Proposed Method

Original: Suppose we have a set of particles Θ q = { θ i } n q i =1 with an empirical probability density function q .
Revised: Suppose we have a set of particles Θ q = { θ i } n q i =1 with an empirical probability density function q .

Original: We hope to iteratively update Θ q so that q converges to a good approximation of the target posterior.
Revised: We hope to iteratively update Θ q so that q converges to a good approximation of the target posterior.

Original: To achieve this, we minimize the KL divergence between q and the target posterior p ( θ | x obs ).
Revised: To achieve this, we minimize the KL divergence between q and the target posterior p ( θ | x obs ).

Original: Specifically, our algorithm repeats the following two steps:
Revised: Specifically, our algorithm repeats the following two steps:

Original: • Approximate KL[ q ( θ ) || p
Revised: • Approximate KL[ q ( θ ) || p

Original: ( θ | x obs )] using particles Θ q and simulations.
Revised: ( θ | x obs )] using particles Θ q and simulations.

Original: • Perform gradient descent update on particles in Θ q to reduce the approximated KL divergence.
Revised: • Perform gradient descent update on particles in Θ q to reduce the approximated KL divergence.

Original: Our method, Variational Likelihood Free Gradient Descent (VLFGD), can be seen as an extension to the classic SVGD algorithm (Liu and Wang, 2016; Liu, 2017).
Revised: The similarities of our method, Variational Likelihood-Free Gradient Descent (VLFGD) to the classic SVGD algorithm (Liu and Wang, 2016; Liu, 2017) are seen in this second point: both perform gradient descent on a set of particles to minimize the KL divergence between them and the target posterior.

Revised: However, beyond this the differences in the two methods are considerable - they have separate motivations, derivations and applications - and thus they should be considered as distinct methods.

Original: 2.1. Approximating the KL Divergence
Revised: 2.1. Approximating the KL Divergence

Original: Like other variational inference methods, we hope to find a variational distribution q that minimizes the KL divergence between q and the target posterior p ( θ | x obs ):
Revised: Like other variational inference methods, we hope to find a variational distribution q that minimizes the KL divergence between q and the target posterior p ( θ | x obs ):

Original: [Equation]
Revised: [Equation]

Original: Although in Bayesian inference, p ( x obs ) is usually seen as a constant, here we treat it as the density p ( x ) evaluated at x obs .
Revised: Although in Bayesian inference, p ( x obs ) is usually seen as a constant, here we treat it as the density p ( x ) evaluated at x obs .

Original: This treatment is crucial for developing our methodology.
Revised: This treatment is crucial for developing our methodology.

Original: Without access to density functions q ( θ ) , p ( x ) and p ( x , θ ), it is impossible to evaluate log (cid:104) q
Revised: Without access to density functions q ( θ ) , p ( x ) and p ( x , θ ), it is impossible to evaluate log (cid:104) q

Original: ( θ ) p ( x ) p ( x , θ ) (cid:105) in the KL divergence.
Revised: ( θ ) p ( x ) p ( x , θ ) (cid:105) in the KL divergence.

Original: However, it is possible to estimate the log density ratio function using samples from the constituent densities (Sugiyama et al., 2012).
Revised: However, it is possible to estimate the log density ratio function using samples from the constituent densities (Sugiyama et al., 2012).

Original: One of the simplest log density ratio estimators is the logistic regression classifier (see 
Revised: One of the simplest log density ratio estimators is the logistic regression classifier (see Chapter 4 in

Original: Chapter 4 in Sugiyama et al.
Revised: Sugiyama et al.

Original: Suppose density functions q ( z ) and p ( z ) are densities from which positive and negative samples are drawn in a binary classification task.
Revised: Suppose density functions q ( z ) and p ( z ) are densities from which positive and negative samples are drawn in a binary classification task.

Original: Logistic regression (assuming perfectly balanced class priors) solves the following optimization:
Revised: Logistic regression (assuming perfectly balanced class priors) solves the following optimization:

Original: [Equation]
Revised: [Equation]

Original: Some algebra shows that the minimum is attained when g = log qp .
Revised: Some algebra shows that the minimum is attained when g = log qp .

Original: When solving (1) in practice, we approximate the objective using empirical samples from q and p and restrict g to belong to some parametric family.
Revised: When solving (2) in practice, we approximate the objective using empirical samples from q and p and restrict g to belong to some parametric family.

Original: Therefore, to estimate the ratio q ( θ ) p ( x ) p ( x , θ ) , we minimize min g
Revised: Therefore, to estimate the ratio q ( θ ) p ( x ) p ( x , θ ) , we minimize min g

Original: [Equation]
Revised: [Equation]

Original: Suppose Z p :=
Revised: Suppose Z p :=

Original: (cid:110)(cid:16) θ (0) i ,x (0)
Revised: (cid:110)(cid:16) [Equation] (0) i ,

Original: i (cid:17)(cid:111) n p i =1 are unbiased samples from the joint probability p ( x , θ ), we can approximate the objective function in (2) using sample averages: minw
Revised: [Equation] [Equation] (cid:17)(cid:111) n p i =1 are unbiased samples from the joint probability p ( x , θ ), we can approximate the objective function in (3) using sample averages: min

Original: [Equation]
Revised: [Equation]

Original: where we have restricted g to belong to a function family parameterized by w .
Revised: where we have restricted g to belong to a function family parameterized by w .

Original: In this work, we parameterize g as a neural network and w is the weights of this neural network.
Revised: In this work, we parameterize g as a neural network where w are the weights of the neural network.

Original: After obtaining the minimizer (cid:98) w , we use g (cid:98) w ( · , x obs ) as an estimator of log (cid:104) q ( · ) p ( x obs ) p ( x obs , · ) (cid:105) .
Revised: After obtaining the minimizer (cid:98) w , we use g (cid:98) w ( · , x obs ) as an estimator of log (cid:104) q ( · ) p ( x obs ) p ( x obs , · ) (cid:105) .

Original: 2.2. Gradient Descent on KL Divergence
Revised: 2.2. Gradient Descent on KL Divergence

Original: In the proposed method, we want to slightly perturb our particles so that the updated particles will more closely resemble the true target posterior.
Revised: In the proposed method, we want to slightly perturb our particles so that the updated particles will more closely resemble the true target posterior.

Original: Consider a perturbed particle set Θ ′ q
Revised: Consider a perturbed particle set Θ ′ q

Original: = { θ ′ i } n q i =1 where θ ′ i
Revised: = { θ ′ i } n q i =1 where θ ′ i

Original: =
Revised: =

Original: θ i
Revised: θ i

Original: + ϵ i .
Revised: + ϵ i .

Original: Suppose particles in Θ ′ q have an empirical probability density function q ϵ , then the “optimal update” is obtained by the following optimization:
Revised: Suppose particles in Θ ′ q have an empirical probability density function q ϵ , then the “optimal update” is obtained by the following optimization:

Original: [Equation]
Revised: [Equation]

Original: where ϵ 0 is a small constant.
Revised: where ϵ 0 is a small constant.

Original: Since ϵ i are only small perturbations ( ∥ ϵ i ∥ ≤ ϵ 0 ), we can assume log q ϵ ( θ ) p ( x obs ) p( θ
Revised: Since ϵ i are only small perturbations ( ∥ ϵ i ∥ ≤ ϵ 0 ), we can assume log q ϵ ( θ ) p ( x obs [Equation] p

Original: [Equation]
Revised: [Equation]

Original: x obs )
Revised: x obs )

Original: [Equation]
Revised: [Equation]

Original: for all θ , so that g (cid:98) w ( · , x obs ) is still a reasonable estimator of log q ϵ ( · ) p ( x obs ) p ( · , x obs ) .
Revised: for all θ , so that g (cid:98) w ( · , x obs ) is still a reasonable estimator of log q ϵ ( · ) p ( x obs ) p ( · , x obs ) .

Original: The minimization in (4) can be approximated by
Revised: The minimization in (5) can be approximated by

Original: [Equation]
Revised: [Equation]

Original: Instead of actually solving (5) [Equation] we use an approximate solution
Revised: Instead of actually solving (6), we use an approximate solution

Original: [Equation]
Revised: [Equation]

Original: where η is a small constant (e.g. 0.0001) chosen post hoc.
Revised: where η is a small constant (e.g. 0.0001) chosen post hoc.

Original: Simply speaking, in order to move our particles closer to the true posterior, we push them along the negative gradient of the approximate KL divergence by a small amount.
Revised: Simply speaking, in order to move our particles closer to the true posterior, we push them along the negative gradient of the approximate KL divergence by a small amount.

Original: 2.3. Alternating Between KL Estimation and Gradient Update
Revised: 2.3. Alternating Between KL Estimation and Gradient Update

Original: Our estimation in Section 2.1 is tied to a specific q ( θ ).
Revised: Our estimation in Section 2.1 is tied to a specific q ( θ ).

Original: In other words, once we update particles in Θ q , our log ratio estimator is no longer accurate.
Revised: In other words, once we update particles in Θ q , our log ratio estimator is no longer accurate.

Original: Therefore, we need to reestimate the log density ratio every time after our particles are perturbed.
Revised: Therefore, we need to reestimate the log density ratio every time after our particles are perturbed.

Original: This naturally leads to a procedure alternating between the KL divergence estimation in Section 2.1 and the gradient update in Section 2.2.
Revised: This naturally leads to a procedure alternating between the KL divergence estimation in Section 2.1 and the gradient update in Section 2.2.

Original: This idea is summarised in Algorithm 1.
Revised: This idea is summarised in Algorithm 1.

Original: Algorithm 1: VLFGD
Revised: Algorithm 1: VLFGD

Original: Inputs: Observation x obs ; initial particles Θ (1) q ; sampler of prior p ( θ ); observation simulator sim( θ ); step size η ; maximum iteration T .
Revised: Inputs: Observation x obs ; initial particles Θ (1) q ; sampler of prior p ( θ ); observation simulator sim( θ ); step size η ; maximum iteration T .

Original: Outputs: Particles at iteration T : Θ ( T ) q .
Revised: Outputs: Particles at iteration T : Θ ( T ) q .

Original: Draw unbiased samples Z p :=
Revised: Draw unbiased samples Z p :=

Original: (cid:110)(cid:16) θ (0) i , x (0)
Revised: (cid:110)(cid:16) θ (0) i , x (0)

Original: i (cid:17)(cid:111) n p i =1 where θ (0) i ∼ p
Revised: i (cid:17)(cid:111) n p i =1 where θ (0) i ∼ p

Original: ( θ ) , x 
Revised: ( θ ) , x (0)

Original: i = sim (cid:16) θ (0) i (cid:17) .
Revised: i = sim (cid:16) θ (0) i (cid:17) .

Original: for t from 1 to T do
Revised: for t from 1 to T do

Original: Obtain (cid:98) w by solving (3) θ ( t +1) i :
Revised: Obtain (cid:98) w by solving (4) θ ( t +1) i :

Original: = θ ( t ) i + (cid:98) ϵ i , where (cid:98) ϵ
Revised: = θ ( t ) i + (cid:98) ϵ i , where (cid:98) ϵ

Original: i is defined in (6).
Revised: i is defined in (7).

Original: end
Revised: end

Original: 3. Experiments 3.1. Mixture of two Gaussians
Revised: 3. Experiments 3.1. Mixture of two Gaussians

Original: We apply our method to a simple model with a tractable posterior.
Revised: We apply our method to a simple model with a tractable posterior.

Original: The likelihood and prior are defined as
Revised: The likelihood and prior are defined as

Original: [Equation] [Equation]
Revised: [Equation] respectively.

Original: + [Equation] N ( θ, σ 22 ) , respectively. As in (Papamakarios et al., 2019), we set 
Revised: As in (Papamakarios et al., 2019), we set θ low = − 10 , θ high = 10 , α = 0 .

Original: [Equation] [Equation]
Revised: 5 , σ 21 = 1 , σ 22 = 0 .

Original: = 0 . and x obs = 
Revised: 01, and x obs = 0 .

Original: The tractable posterior p ( θ | x obs ) ∝ 0 . 5 N 
Revised: The tractable posterior p ( θ | x obs ) ∝ 0 . 5 N (0 , σ

Original: [Equation]
Revised: [Equation]

Original: 0 , σ 21 ) + 0 .
Revised: + 0 .

Original: 5 N (0 , σ 22 ).
Revised: 5 N (0 , σ 22 ).

Original: Because the parameter we wish to infer is one dimensional, it allows us to visualize our update steps.
Revised: Because the parameter we wish to infer is one dimensional, it allows us to visualize our update steps.

Original: We run Algorithm 1 with 1000 particles ( n q = 1000).
Revised: We run Algorithm 1 with 1000 particles ( n q = 1000).

Original: The histograms of particles Θ ( t ) q at iterations t = 1 , 15 , 200 are plotted at the top row of Figure 1.
Revised: The histograms of particles Θ ( t ) q at iterations t = 1 , 15 , 200 are plotted at the top row of Figure 1.

Original: It can be seen that Θ ( t ) q converges  to the target posterior p ( θ | x obs ) as t increases.
Revised: It can be seen that Θ ( t ) q converges [Figure] to the target posterior p ( θ | x obs ) as t increases.

Original: For comparison, we plot the histograms of SVGD particles at the bottom row.
Revised: For comparison, we plot the histograms of SVGD particles at the bottom row.

Original: We can see that our VLFGD updates mimic the updates observed in SVGD without accessing the true posterior.
Revised: We can see that our VLFGD updates mimic the updates observed in SVGD without accessing the true posterior.

Original: Moreover, we plot the estimated log ratio g (cid:98) w ( · , x obs ) for VLFGD updates.
Revised: Moreover, we plot the estimated log ratio g (cid:98) w ( · , x obs ) for VLFGD updates.

Original: It can be seen that at iteration 1 and 15, the negative gradient of g (cid:98) w ( · , x obs ) points to 0 where the true posterior is high.
Revised: It can be seen that at iteration 1 and 15, the negative gradient of g (cid:98) w ( · , x obs ) points to 0 where the true posterior is high.

Original: This confirms that g (cid:98) w ( · , x obs ) can indeed guide the Θ q toward a good posterior approximate.[Figure]
Revised: This confirms that g (cid:98) w ( · , x obs ) can indeed guide the Θ q toward a good posterior approximate.

Original: 3.2. M/G/1 Queue
Revised: 3.2. M/G/1 Queue

Original: The M/G/1 process aims to model a simple queue with customers arriving at random times v i and being served for random amounts of time s i by a single server.
Revised: The M/G/1 process aims to model a simple queue with customers arriving at random times v i and being served for random amounts of time s i by a single server.

Original: The output of the process is the inter-departure times d i − d
Revised: The output of the process is the inter-departure times d i − d

Original: i − 1 of the customers.
Revised: i − 1 of the customers.

Original: The distribution of these objects is dictated by 3 parameters θ =
Revised: The distribution of these objects is dictated by 3 parameters θ =

Original: ( θ 1 , θ 2 , θ 3 ).
Revised: ( θ 1 , θ 2 , θ 3 ).

Original: Precise details of the model are found in Appendix B, and we employ the same set up as (Papamakarios et al., 2019).
Revised: Precise details of the model are found in Appendix C, and we employ the same set up as (Papamakarios et al., 2019).

Original: It is easy to simulate observations using this model.
Revised: It is easy to simulate observations using this model.

Original: However, its likelihood is intractable and consequently its posterior is intractable.
Revised: However, its likelihood is intractable and consequently its posterior is intractable.

Original: Inference is considerably more challenging than that in Section 3.1 not only because the dimension of both x and θ have increased, but also due to the highly non-linear and noisy nature of the simulator.
Revised: Inference is considerably more challenging than that in Section 3.1 not only because the dimension of both x and θ have increased, but also due to the highly non-linear and noisy nature of the simulator.

Original: In Figure 3.2, we compare our method, VLFGD, to ABC-Rejection (Pritchard et al., 1999), 
Revised: In Figure 3.2, we compare our method, VLFGD, to ABC-Rejection (Pritchard et al., 1999), Neural Ratio

Original: Neural Ratio Estimation (NRE) and its sequential version Sequential Ratio Estimation (SRE) described in (Hermans et al., 2020) with the latter two implemented in Python with the toolbox sbi (Tejero-Cantero et al., 2020).
Revised: Estimation (NRE) and its sequential version Sequential Ratio Estimation (SRE) described in (Hermans et al., 2020) with the latter two implemented in Python with the toolbox sbi (Tejero-Cantero et al., 2020).

Original: The marginal and joint posteriors resulting from VLFGD, Rejection ABC, and SRE have reasonable amount of mass near the ground-truth parameters θ ⋆ .
Revised: The marginal and joint posteriors resulting from VLFGD, Rejection ABC, and SRE have reasonable amount of mass near the ground-truth parameters θ ⋆ .

Original: This is in contrast to
Revised: This is in contrast to

Original: NRE which has almost no mass assigned to the ground-truth parameter θ ⋆ 2 .
Revised: NRE which has almost no mass assigned to the ground-truth parameter θ ⋆ 2 .

Original: Both NRE,
Revised: Both NRE,

Original: SRE and our method give strongly peaked marginal posteriors.
Revised: SRE and our method give strongly peaked marginal posteriors.

Original: The posteriors achieved from our method and SRE are very similar, with peaks in almost identical locations.
Revised: The posteriors achieved from our method and SRE are very similar, with peaks in almost identical locations.

Original: All methods result in the main mode of p ( θ 2 | x obs ) in a different location to the ground truth θ ⋆ 2 .
Revised: All methods result in the main mode of p ( θ 2 | x obs ) in a different location to the ground truth θ ⋆ 2 .

Original: 4. Discussions and Conclusion
Revised: 4. Discussions and Conclusion

Original: In this paper, we proposed an iterative algorithm to produce a particle distribution that resembles the target posterior.
Revised: In this paper, we proposed an iterative algorithm to produce a particle distribution that resembles the target posterior.

Original: To achieve this, we move the particles according to the direction that minimizes a KL divergence between the particle distribution and the true posterior.
Revised: To achieve this, we move the particles according to the direction that minimizes a KL divergence between the particle distribution and the true posterior.

Original: The KL divergence is approximated using logistic regression.
Revised: The KL divergence is approximated using logistic regression.

Original: Experiments on a
Revised: Experiments on a

Original: Gaussian mixture model and the M/G/1 queuing model show promising results.
Revised: Gaussian mixture model and the M/G/1 queuing model show promising results.

Original: Methodologically, our method is similar to the sequential approach used in Hermans et al.
Revised: Methodologically, our method is similar to the sequential approach used in Hermans et al.

Original: (2020) in the sense that both approaches estimate a ratio function repeatedly while the particles (or the sampler) are being updated.
Revised: (2020) in the sense that both approaches estimate a ratio function repeatedly while the particles (or the sampler) are being updated.

Original: We estimate q ( θ ) p ( x ) p ( x , θ ) but Hermans et al.
Revised: We estimate q ( θ ) p ( x ) p ( x , θ ) but Hermans et al.

Original: estimates p ( x | θ )
Revised: estimates p ( x | θ )

Original: ˜ p
Revised: ˜ p

Original: ( θ ) ˜
Revised: ( θ ) ˜

Original: p ( x )˜ p
Revised: p ( x )˜ p

Original: ( θ ) where ˜ p ( θ ) is a proposal prior and ˜ p
Revised: ( θ ) where ˜ p ( θ ) is a proposal prior and ˜ p

Original: ( x ) is the marginal of p ( x | θ )
Revised: ( x ) is the marginal of p ( x | θ )

Original: ˜ p
Revised: ˜ p

Original: ( θ ).
Revised: ( θ ).

Original: At convergence, q ( x ) = p ( θ | x ) and thus our ratio converges to 1, while the ratio used by
Revised: At convergence, q ( θ ) = p ( θ | x ) and thus our ratio converges to 1, while the ratio used by

Original: Hermans et al.
Revised: Hermans et al.

Original: converges to the likelihood to evidence ratio.
Revised: converges to the likelihood to evidence ratio.

Original: Instead of modelling the posterior directly, Tran et al.
Revised: Instead of modelling the posterior directly, Tran et al.

Original: considers modelling the likelihood q w ( x | θ ) and minimizing the KL divergence between q ( θ | x obs ) and p ( θ | x obs ).
Revised: considers modelling the likelihood q w ( x | θ ) and minimizing the KL divergence between q ( θ | x obs ) and p ( θ | x obs ).

Original: The KL divergence, up to a constant, can be written as (See (4) in Tran et al.
Revised: The KL divergence, up to a constant, can be written as (see (4) in Tran et al.

Original: (2017) for details) 
Revised: (2017) for details) (cid:82) q w ( θ | x obs ) log q w ( x obs | θ ) p ( x obs | θ )

Original: [Equation]
Revised: d θ +

Original:  where F w ( θ ) is a term that can be easily approximated using Monte-Carlo via samples of q ( θ ).
Revised: F w ( θ ) , where F w ( θ ) is a term that can be easily approximated using Monte-Carlo via samples of q ( θ ).

Original: If q w ( x | θ ) is an implicit generative model (i.e., a neural sampler that produces x given an input θ ), one can use logistic regression to estimate the likelihood ratio q w ( x | θ ) p ( x | θ ) given a θ and then use it to approximate the KL.
Revised: If q w ( x | θ ) is an implicit generative model (i.e., a neural sampler that produces x given an input θ ), one can use logistic regression to estimate the likelihood ratio q w ( x | θ ) p ( x | θ ) given a θ and then use it to approximate the KL.

Original: The optimal w can be found by minimizing this approximated KL.
Revised: The optimal w can be found by minimizing this approximated KL.

Original: In comparison, we directly model the posterior using a particle distribution and estimate the ratio between the approximated posterior and the true posterior.
Revised: In comparison, we directly model the posterior using a particle distribution and estimate the ratio between the approximated posterior and the true posterior.

Original: Put simply, Tran et al.
Revised: Put simply, Tran et al.

Original: models the likelihood function and estimates the likelihood ratio, while we model the posterior and estimate the posterior ratio.
Revised: models the likelihood function and estimates the likelihood ratio , while we model the posterior and estimate the posterior ratio .

Original: Although our method is almost completely free from tuning, in the future we can consider a more automated algorithm where hyper-parameters (such as η ) in our algorithm can be tuned according to some criterion, since low-maintenance methods are always preferred by scientific users.
Revised: Although our method is almost completely free from tuning, in the future we can consider a more automated algorithm where hyper-parameters (such as η ) in our algorithm can be tuned according to some criterion, since low-maintenance methods are always preferred by scientific users.

Original: Further efficiencies can arise from reusing earlier samples in the estimation of the density ratio.

Original: We could also consider a sequential version of our method by gradually zeroing in on estimating a log-ratio which is more relevant to our observation.
Revised: Further, we could also consider a sequential version of our method by gradually zeroing in on estimating a log-ratio which is more relevant to our observation.

Revised: Appendix A. Particle Collapse

Revised: Observe that the minimum of Equation (1) for an empirical distribution q is achieved when all particles take values equal to the mode(s) of the target distribution.

Revised: However, we desire approximate samples of the posterior rather than approximations of the mode(s).

Revised: Fortunately, this potential issue is circumvented as a density ratio estimator is used to estimate the KL divergence instead of approximating the KL using the empirical density functions (delta functions) represented by particles.

Original: Appendix A. NRE Details
Revised: Appendix B. NN Details

Original: Specifications of our NN:
Revised: Specifications of our NN:

Original: • Mixed Gaussian, Section 3.1: MLP with 50 hidden units, 3 layers, batch-size 64, learning rate 1e-4, max epochs 1000, optimizer Adam • M/G/1, Section 3.2: MLP with 50 hidden units, 3 layers, batch-size 256, learning rate 1e-4, max epochs 1000, optimizer Adam
Revised: • Mixed Gaussian, Section 3.1: MLP with 50 hidden units, 3 layers, batch-size 64, learning rate 1e-4, max epochs 1000, optimizer Adam • M/G/1, Section 3.2: MLP with 50 hidden units, 3 layers, batch-size 256, learning rate 1e-4, max epochs 1000, optimizer Adam

Original: Appendix B. M/G/1 Queue Experiment Set-up
Revised: Appendix C. M/G/1 Queue Experiment Set-up

Original: We follow a very similar implementation to that of (Papamakarios et al., 2019).
Revised: We follow a very similar implementation to that of (Papamakarios et al., 2019).

Original: The model is defined by three equations
Revised: The model is defined by three equations

Original: [Equation]
Revised: [Equation]

Original: These can be interpreted as follows: the server takes s i ∼ U ( θ 1 , θ 2 ) to serve customer i .
Revised: These can be interpreted as follows: the server takes s i ∼ U ( θ 1 , θ 2 ) to serve customer i .

Original: The time between customer i ’s arrival v i and customer ( i − 1)’s arrival v i − 1 , is such that v i − v i − 1 ∼
Revised: The time between customer i ’s arrival v i and customer ( i − 1)’s arrival v i − 1 , is such that v i − v i − 1 ∼

Original: Exp( θ 3 ).
Revised: Exp( θ 3 ).

Original: Parameters ( θ 1 , θ 2 , θ 3 ) are the parameters we wish to do inference on.
Revised: Parameters ( θ 1 , θ 2 , θ 3 ) are the parameters we wish to do inference on.

Original: The priors we place on these parameters are given by
Revised: The priors we place on these parameters are given by

Original: [Equation]
Revised: [Equation]

Original: The output from the process is the inter-departure times d
Revised: The output from the process is the inter-departure times d

Original: i − d i − 1 where d i denotes the departure time of customer i .
Revised: i − d i − 1 where d i denotes the departure time of customer i .

Original: We have d i − d
Revised: We have d i − d

Original: i − 

Revised: i −

Original: =
Revised: =

Original: s i + max(0 , v i − d i − 1 ).
Revised: s i + max(0 , v i − d i − 1 ).

Original: For our experiments we take the total number of customers I = 50.
Revised: For our experiments we take the total number of customers I = 50.

Original: To reduce the dimensionality and make inference feasible we take a summary statistic of the output: we record the (0, 25, 50, 75, 100)th quantiles of the inter-departure times and output that.
Revised: To reduce the dimensionality and make inference feasible we take a summary statistic of the output: we record the (0, 25, 50, 75, 100)th quantiles of the inter-departure times and output that.

Original: In
Revised: In

Original: Papamakarios and Murray (2016)
Revised: Papamakarios and Murray (2016)

Original: they introduce a pilot run of 100K simulations to allow for the normalisation of x (to have zero mean and unit covariance) - we do not employ this as we feel this would be unfeasible in many other scientific applications.
Revised: they introduce a pilot run of 100K simulations to allow for the normalisation of x (to have zero mean and unit covariance) - we do not employ this as we feel this would be unfeasible in many other scientific applications.

Original: We take the ground truth parameters θ ⋆ =
Revised: We take the ground truth parameters θ ⋆ =

Original: and simulate x obs from these parameters.
Revised: and simulate x obs from these parameters.