Learning implicit hidden Markov models using neural likelihood-free inference
Authors: Sanmitra Ghosh
Paul Birrell
Daniela De Angelis
Venue: Submitted to ICLR 2023 (2023)
Keywords: likelihood-free
Bayesian inference
simulation based inference
ABC-SMC
HMM
simulator
implicit models

Abstract: Likelihood-free inference methods for implicit models based on neural conditional density estimation were shown to drastically reduce the simulation burden in comparison to classical methods such as ABC. However, when applied in the context of any latent variable model, such as a Hidden Markov model (HMM), these methods are designed to only estimate the parameters rather than the joint posterior distribution of both the parameters and the hidden states. Naive application of these methods to a HMM, ignoring the inference of this joint posterior distribution, will result in overestimation of uncertainty of the posterior predictive. We propose a postprocessing step that can rectify this problem. Our approach relies on learning directly the intractable posterior distribution of the hidden states, using an autoregressive-flow, by exploiting the Markov property. Upon evaluating our approach on some intractable HMMs, we found that the quality of the estimates retrieved using our postprocessing is comparable to what can be achieved using a computationally expensive particle-filtering which additionally requires a tractable data distribution.

Under review as a conference paper at ICLR 2023
LEARNING IMPLICIT HIDDEN MARKOV MODELS USING NEURAL LIKELIHOOD-FREE INFERENCE
Anonymous authors Paper under double-blind review
ABSTRACT
likelihood-free inference methods based neural conditional density estimation were shown to drastically reduce the simulation burden in comparison to classical methods such ABC. However, when applied in the context of any latent variable model, such as a Hidden Markov model (HMM), these methods are designed to only estimate the parameters rather than the joint posterior distribution of both the parameters and the hidden states. Naive application of these methods to a HMM, ig- noring the inference of this joint posterior distribution, will result in overestimation of uncertainty of the posterior predictive. We propose a postprocessing step that can rectify this problem for HMMs with a continuous state space. Our approach relies on learning directly the intractable posterior distribution of the hidden states, using an autoregressive- our approach some implicit HMMs, we found that the quality of the estimates retrieved using our postprocessing is comparable to what can be achieved using a computationally expensive particle-filtering which additionally requires tractable data distribution.
by exploiting the Markov property. Upon evaluating
INTRODUCTION
We consider the task of Bayesian inference of Hidden Markov mode mode[[whose whose likelihood is ana- lytically intractable and the model is only available as a simulator. Due to the unavailability of the likelihood standard Bayesian inference methods cannot be applied such model. Inference of such a model is generally carried out using approximate Bayesian computation (ABC) (Sisson et al. 2018), which only require forward simulations from the model, see for example Martin al. (2019);
Picchini (2014): Toni et al. (2009).
Recently, new class of likelihood-free inference methods, see Cranmer et al. (2000) for a review, were developed that use a neural network based emulator of the posterior density, the likelihood density and the likelihood ratio. Such methods were empirically shown to be much more sample efficient (Lueckmann et al. [02I]) in comparison to ABC. Additionally, these methods do not require the user to specify difficult-o-chhoose algorithmic parameters and they perform equally well across different models without (much) problem specific choice of the neural network's architecture. Naturally, these methods appear as more preferable algorithmic choices for carrying out inference in implicit HMM, in comparison ABC. We like to point out that these neural likelihood-fee approaches (NIFI) are usually applied to estimate the posterior of the parameters only. This is since naive implementation of neural network based emulator may perform unreliably in estimating the joint posterior of the parameters and the high-dimensional hidden states, potentially for a lack of inductive biases. Estimation of the hidden states may may not be of interest within a particular application domain. However, without estimating the joint posterior of the parameters and the hidden states the goodness-of cannot be correctly assessed. This indeed a severe limitation. Note that although ABC theoretically targets the joint distribution it fails to estimate (we will demonstrate this later in the experiments) the hidden states adequately within a reasonable simulation budget.
In this paper we present novel technique to estimate the hidden states by learning an approximation of the incremental posterior distribution of the states using a neural density estimator. After learning the incremental posterior density, the density estimator can be used to draw the full path of the hidden states recursively. Following are our salient contributions:
In some literature the term state-space model used interchangeably refer to Hidden Markoy model.

---

Under review as a conference paper at ICLR 2023
We highlight the problem of neural likelihood-free methods in estimating the joint density of the unknown variables within a HMM. We then propose postprocessing technique to mitigate this limitation.
Our approach can be used as a postprocessing technique for any likelihood-free method, in order to avoid overestimation of uncertainty.
We develop a method to obtain an amortised approximation of the posterior of a latent Markov process, with intractable transition and observation densities, without using ABC.
2
BACKGROUND
We begin by first introducing the implicit HMM and then we will discuss the challenges of carrying out Bayesian inference. We can describe a HMM, for a continuous latent Markov process X, IR with a K -ddimensional state-space, as follows:
X (x-10)
(I)
Y/ g Xx 0)
where 9 parameterise the transition f(X, 9) and the observation ( X, 0) densities re- spectively. We consider 9 to include the initial state Xo. Given a set of noisy observations of out of the K states y R- M experimental time points, of the latent process, our goal is infer the joint posterior distribution p(0, x æly), where x = (X,,... x is the unobserved sample path of the process the hidden states. The expression for the unnormalised posterior is given by
(
/M-1
)))
(II
( II [II
M-11
(0. & p(0)
II t=
g XT (x.))
(2)
[=1
where p(0) is the prior distributions over the parameters and the initial values. The trickier part is the inference of the hidden states. We are interested in the case where one can draw samples from f())
and g(-), but cannot evaluate either or both of these densities. Note that indeed when these densities are known, and are Gaussian, we can apply classical fllteing//moothing techniques (Särkkä 2013). For non non-Gaussian densities, or when only g()) is known, particle-filte filtering can be used. However, when g(-) is not known then likelihood-free methods are generally used.
3 NEURAL
INFERENCE
If instead of the joint p(0, xly) we only wish to estimate the marginal p(0ly), then a number of strategies based on conditional density estimation can be employed. For example, can simulate pairs of 0 y from their joint distribution and then subsequently create a training dataset, of N samples (0n, yd) ) n=1 which can be utilised to train conditional density estimator that can approximate the posterior (Papamakarios & Murray 2016) p(0ly) R qu (0lg) or the likelihood Papamakarios ct al. (2019) Oi) q (y)0). In the former case once we have trained an approximation to the posterior, using density estimator, we can directly draw samples O ~ Te. ( yo) by conditioning on a particular dataset yo In the latter case we can use the trained density estimator to approximate the posterior p(0) b x L (B. p)p(()))))))))))))))))))))))))) and then draw samples from it using Markov chain Monte Carlo (MCMC). In principle we can choose any density estimator for (-I-)) However, the literature surrounding NLFI, a neural network based density estimator is generally used. A neural network is used in this context either as a nonlinear transformation of the conditioning variables, within a mixture-of- Gaa-sssaaaaaaannnnnaa density
normalizing fflo (Rezende & Mohamed| 2015 Paaamakarios et al. 2021) that builds a transport map (Parno) (015) between a simple distribution (such as standard Gaussian) and complex one such the likelihood/posterion density. Following the seminal work of Tabak & Turner (2013) a large amount research is undertaken build such transport maps using samples from the respective measures. An alternative formulation of NLFI utilises the duality (Cranmer et al. (0155) between the optimal decision function
proposed in Bishop (1994),
of probabilistic classifier and the likelihood ratio, r(0", 0b)
p(vle")
evaluated using two samples pl
0" and 6", to approximate the latter through training a binary classifier using samples from p(y, 0).
This likelihood ratio can then be used as proxy within a MCMC scheme as follows:
min [1 p(y. 0 koo (000000)))))) pp(0))
Ko (0 0 (p(()))))
~ min 1, 1,r
(3)
ply. 0)ko 0 (0)())())
ko 0" ())(
2

---

Under review as a conference paper at ICLR 2023
where (-(-) is proposal density. Note that these NLFI methods carry out amortised inference that is there is no need to re-learn the density/density-rato estimator for every new instances of the observations.
To increase sample efficiency of these methods one can use them in a sequential manner (Durkan et al. 2018). After an initial round of NLFI, we are left with samples of 0 from its posterior distribution. We can subsequently use these samples to generate further simulated data concentrated around the given observations yo- This constitute new training set on which second round of NLFI can be applied to further refine the approximations, This process can be repeated for a number of rounds. Note that when sequential process is used in conjunction with density estimator for the posterior then the parameter samples from the second round are no longer drawn from the prior. Thus, different adjustments had been proposed, leading to different algorithms (Greennerg et al. 2019 Lueckmann et al. 2017), to overcome this issue.
We like to further point out that NLFI is applied using some summary statistic s(y) of the data y. a practise carried over from the usage of ABC methods. There have been recent work on using a neural network to generate the summary, trained simultaneously with the network used for density//ensitty- ratio estimation.
Can we ignore the joint distribution? The various NLFI techniques discussed so far are designed to solve the marginal problem of estimating p(Oly)) The necessity of the estimation of the hidden states x is problem specific. In some applications estimation of the hidden states is of paramount importance. For example, in epidemiology the hidden states reflect how infectious disease has progressed and thus inference of both the parameters and the hidden states is necessary. However, in certain applications one may wish ignore the hidden states. Nonetheless, it is helpful if the methodology can accommodate estimation of the joint distribution p(0, x v))y), and the choice of estimating just 0 or x or both left to the practitioner's discretion. However, in the process of modelling physical phenomenon it is often necessary to asses the goodness-of-ñit Within the Bayesian context this is carried out by inspection of the posterior predictive distribution ((y"ly) of generating replicated data (Gelman et al. 1996) y". This distribution is given by
p(y (v)
p(y" [z, 0)p(x, 0 y dr)dd
(4)
But when instead of the joint distribution we only have access to the marginal distribution (that is access only samples of 0, the output of any NLFI method) then the above can only be obtained as follows
p(y"ly) R p(y"ly) = ff p(y"' x; 0)p(() 0 y dodd,
(5)
where the joint posterior of x, 0 is approximated as p(z, 0ly) = p(z|0))p((l)) which is akin to drawing a: from the prior p(a: ( = [(
of the latent Markov process. As result the credible intervals of p yy) would be erroneously inflated, since in this case the latent sample path not correctly constrained by the data, leading to an incorrect assessment of the goodness-of-ff This is a severe problem that needs to be addressed even in the case where we wish to estimate just the marginal p(Oly))
limitations of NFLI for inferring the joint: let us now consider the task of estimating the joint posterior p(0, mly) using NLFI method. If we want to approximate the posterior then we have to extend any chosen density estimator to target a high-dimensional vector (0, vec(=)), where vec : IR xxxM M + KM which would invariably require a larger training set, and thus more simulations, in comparison to estimating only 0 (see Appendix E for an example). Alternatively, if we choose to approximate the likelihood density, then note that the acceptance ratio of the MCMC step will be of the following form:
min {1
1, ge (yoll 0" p(("' 0 Nan (aa (0 To (y ir 0 p(n, n. k)k (xx" (I) ( 0+ (0)
(6)
where kew (-), (-) are the proposal densities. Due to the intractability of p(x, 0) our only option as a proposal kio (-) is its prior that is the transition density in equation I This will jeopardise the mixing of the MCMC sampler which in turn would require excessive simulation from the model. We would face the same limitation if we had chosen to emulate the likelihood ratio.
3

---

Under review as a conference paper at ICLR 2023
We like to point out that ABC indeed jointly samples the hidden states and the parameters. However, the states are updated using the prior of the Markov process as the proposal (see Appendix for details) resulting in poor estimation of æ. In what follows we develop novel technique to estimate a given samples of 0.
4 METHODS
4.1 INFERRING BOTH PARAMETERS AND IIDDEN STATES
We can decompose the joint density using the product rule as follows:
p(z, 0 p( a (Oly).
With the above decomposition into two sub-tasks of inferring separately the distributions p (0)y) and p(a: a y) Samples of æ can then be drawn given samples of 9. Note that the task of inferring p(0)g) y can be carried out, sample efficiently, using any off-the-shelf NLFI method. Thus, the challenge is to develop a NLFI strategy for inferring p(æ O. y). The advantage applying NLFI to infer p( x 0 y) rather than p(æ, 0ly y is that for the former case we can exploit the Markov property to escape the curse-of-dimensionaal
break down the task of inferring the joint distribution of a.
This is a non- sta-dsandard approach of drawing from the joint distribution, simply because there did not exist any method, before the advent of NLFI, which can target p 0 yy) without drawing æ. Any classical marggnall sampling method, targeting 0 (Beaumont, 2003 Andrieu Roberts 2009), that uses the marginal likelihood p (0). or its unbiased estimate, in the MCMC acceptance step invariably also draws æ.
Note that if the observation density is known analytically, then standard Bootstrap (Gordon et al. 1995) SMC algorithm can be used, since it does not require a tractable transition density, to infer p(a[ O0 y) Rs (2) (z)2g y, 0), given an estimate of p(0 (y) obtained using NLFI. The posterior predictive distribution can then be evaluated as follows:
ply"ly) p(y" (ily"ee ax, (ax(a 0 yydddd ~ / p(y""lz: (ar" P (a
(8)
Although in certain practical applications the observation density remains tractable, the SMC al- gorithm however would require large number of simulations from the model, thus defeating the purpose of NLFI. This is the reason we consider learning an approximation of p(æ y) using a neural density estimator, which turn next.
4.2 LEARNING AN INCREMENTAL POSTERIOR
We can decompose the posterior of æ, again using the product rule, as follows:
p 0, yy =
X1, M X,)0,y)
(9)
It can be easily verified (see Appendix A) that the posterior distribution of the last sample point X 1. due to the Markov property, is independent of the past history:
P(M X M X1, y) = p((MaM 2-
(10)
Again due to the Markov property (see again Appendix A), each of the remaining sample points (X,) M-2 is only conditioned on (X++1, X,-1, Vr. 9). Thus, if we drop the conditioning of the future sample X,41, then equation Cean be approximated as follows:
M-1
p 0. v)
II (xX.Xx-1.91.90
(11)
Although this is a lossy approximation, it will work reasonable well as long as the information in X(+1 is largely contained in the pair (X,-1. y:). We can now emulate the incremental posterior yr, 0) z 4 (X, Xx.1. Ve. 0). between any two consecutive time points t, - 1, using a neural density estimator (-). We then recursively sample the full path using the corresponding
4

---

Under review as a conference paper at ICLR 2023
How examples are created for training the IDE: X Nc.
Once trained samples of the latent path X ~ pxX( 8) drawn recursively using the IDE
0² ~ p())
targets
Inputs
0 ~ p(0 m)
x 93
Simulate from the HMM
X
X X vi
1)
@a ~ p
X
X3
X
V
Simulate from HMM
M
X
IDE
X Mi Training examples
96
X
Figure I: (Leftt Process of creation of dataset for training the IDE: Tol X, 1, yr. 0). We demonstrate this using M time points. (Right) Graphical model of the IDE showing how the sample path of the hidden states is drawn. Note that p(0 y) can be obtained using any off-the-shelf likelihood-free free method. We build the IDE using a masked autoregressive flow.
incremental density estimator (IDH) (see Figure 1.. We can approximate the posterior predictive in equation 4 now as follows: f / 0)
M-1
p(y'l)) ~
II
q (X. Mr. 0)p(O)p dxdd. t=1
(12)
In summary our strategy for drawing samples x, 0 from an approximation of their joint posterior is follows. We first infer the parameter marginal p(0ly) using any chosen NLFI method and draw samples of O. In parallel we train an IDE using a subset of simulations used in inferring p(0)y). Then for each sample 0, and an observed dataset you we can use the trained IDE recursively obtain the sample path x conditioned on the sample @ and the dataset yo-
Limitations: There are two fundamental assumptions behind our approach. Firstly, we assume that samples of 0 are drawn from good approximation to the true unknown posterior. However, this may not be true when inference of done on very limited simulation budget. Secondly, we assume that there is no model miss-specification and the observations come from the joint distribution p(ær 0, y) used for learning the IDE. This can be a strong assumption when modelling a new phenomenon. Finally, we like to also point out that our approximation equation II may perform poorly at time t there a regime change at time + 1 due to loosing information from X,+1.
4.3 TRAINING THE INCREMENTAL. DENSITY ESTIMATOR
We chose masked autoregressive flow (MAF) as the incremental density estimator. MAF is built upon the idea of chaining together a series of autoregressive functions, and can be interpreted as normalizing- flow (Papamakarios et al. 2017). That is can represent go (x,1, yr, O)
transformation of a standard Gaussian N (0, lJ (or another simple distribution) through a series of J autoregressive functions h1 ...,, h/ parameterised by Φ, each of which is dependent the triplet (X,-1.9.) 0). 0):
(O, 2, = (2)-1, 12
X, zJ, where
(13)
X Yc. 0)
Each h; is a bijection with a lower-triangular Jacobian matrix, implemented by Masked Autoencoder for Distribution Estimation (MADE) (Germain al., 22015). and is conditioned on -x,-1, 1, yt,, 0). Using the formula for change of variable the density is given by
(X Hr. N II det
(14)
iz,-1
We can learn the parameters & by maximising the likelihood. To do this we create a training dataset consisting of N examples. We first sample N values of the parameter (0") [ ] from its prior and
5

---

Under review as a conference paper at ICLR 2023
for each 8" we simulate (see Appendix C) the sample path of the states and the observations using equation I Each training example is then created by collecting the random variables: (X,", y 0"),, and X as the input-target pair, where (i.j) = (0. 1), (2.3).... (M - 2, M - 1). Clearly, even with small number of model simulations (a few thousands) we can create large training dataset to learn an expressive neural density estimator. In Figure [ we outline the process of creating this training dataset. Given these training examples Φb can be learnt, using gradient ascent, through maximising the total likelihood: Σ asl X X 1: y². O*), which is equivalent to minimising the Bellback-Leibler divergence KT X
X 1, Vt. 1) X, Vr. O)) (Papamakarios et al.
2019).
RELATED WORK IN LIKELIHOOD-FREE INFERENCE OF HMMS
The most common approaches to tackle the inference of an implicit HMM consist largely of ABC methods (Dean et al. 2014, Martin et al. 2019 Picchini 2014). Note that when the observational density is known analytically then the particle- MCMC (Andrieu et al. 2010) method can be used to carry out exact inference. However, the computational cost of this method is prohibitive, as in each step of MCMC particle filter with large number of particles is to calculate unbiased estimate of the marginal likelihood. Interestingly, a new avenue of research can be of combining our proposed IDE as importance density within Particle-MCMC scheme. An alternative approach which combines SMC with ABC was proposed in (Drovandi et al., 2016). However, this approach requires the problematic choices of ABC algorithmie parameters.
The incremental posterior density is also the optimal importance density for a SMC sampler (Doucet et al. 2009; Creal, 2012). This quantity is only available in closed form for simple linear Gaussian models only. For complex models with a nonlinear transition this is mostly approximated sub- optimally using either linearisation techniques (Creal 2012) or model approximations (Golightly & Wilkinson 2011). The proposed IDE, being a normalising flow, is a closed form approximation of this quantity. Thus, the IDE can be applied for novel algorithm design within the context of SMC.
6 EVALUATIONS
We evaluate our proposed approach to recover the hidden states, through learning the incremental posterior density, using three biological HMMs: (i) the stochastic Lotka- Volterra (LV) model (Wilkin son 2018), (ii) the prokaryotic auto-regulator (PKY) model (Golightly & Wilkinson 2011) and (iii) a SIR epidemic model (Anderson et al. 1992). For the L.V and PKY models the hidden states evolve as pure Markov jump process (MJP) whereas for the SIR model we consider a stochastic differential equation (SDE). For all of the models we used simulated data, so that we are cognisant of the ground truth. We also used a tractable observation density g(-) to facilitate SMC.
Our goal is to evaluate how well the proposed approach can estimate the posterior predictive distri- bution p(y" 3)) in comparison to other approaches. We chose the following competing approaches. First is the ABC-SMC algorithm (Toni et al., 2009). used only for the PKY and LV model, which produces samples from the joint distribution p xc ()) that be used to evaluate the posterior predictive in equation 4 Except ABC-SMC all other approaches, including the proposed one, rely on the availability of parameters samples from the marginal p(0 (y) which can be obtained using any off-the-shelf NLFI method such as the ones discussed in section Once samples of O become available then samples of can be drawn from its posterior using the IDE, the SMC (since 9(-) is available), or simply from the prior transition p( 0) (which we denote as PrDyn). Samples from p(y" (1) can then be drawn using equation 12 for IDE, equation 8 for SMC and equation 5 for PrDyn.
To estimate p(0ly), required for IDE, SMC and PrDyn approaches, we used two sequential NLFI methods. One based on learning the likelihood density (SNLE) and the other one based on learning the likelihood-ratio (SRE). It was recently shown in (Durkan et al. (020) that SRE (Hermans et al. 2020) is equivalent to a certain form of sequential learning of the posterior density (SNPE-C) (Greenberg et al. 2019) and both can be unified under common framework on contrastive learning (Gutmann & Hyvärinen 2010). Thus, by using SNLE and SRE we can cover the general ambit of sequential NLFI approaches. For SNLE again used a MAF the likelihood density estimator gu (s(z) ()) and for SRE we used a mlp classifier. For both the MAFs, q (X, (X, yr, 0) for the IDE and g, (0) s ssy) for SNLE, we used the same architecture. That is J = 5 transformations each of which has two

---

Under review as a conference paper at ICLR 2023
hidden layers of 50 units each and ReLU nonlinearities. For SRE we used residual network based classifier with two residual layers of 50 units each and ReLU nonlinearities. For training all the neural networks we used ADAM (Kingma & Ba 2015) with minibatch size of 256, and learning rate of 0.0005. Following, Papamakarios et al. (019)) we used 10% of the training data as a validation set, and stopped training if validation log likelihood did not improve after 20 epochs. Following Papamakarios et al.] (2019), we used the Slice Sampling algorithm (Neal 2003) to draw samples from the posterior while using SNLE and SRE.
Since, p(y" (g) is obtained by jointly marginalising out 0 and æ, and since the IDE is only useful in estimating p(z y). thus we want to rule out major differences in the estimates of among different methods. For this reason we used fixed budget of simulations respectively for ABC-SMC (which jointly estimates 0, ..) and SNLE/SRE (used for estimating 0). These simulation budgets were determined based on previous studies such as Lueckmann et al. (202T), on models such as the ones we used, that compared the sample efficiency between ABC-SMC and NLFI methods for estimating p(Oly). Thus, for all three models, while using SNLE/SNRE, we used 30 rounds and the posterior samples from the final round were collected. For each model we generated 5000 training examples in the first round and in the subsequent rounds we generated 1000 examples. Furthermore, we used the 5000 simulations generated in the first round to learn the parameters Φ of the IDE. We limited the ABC-SMC to no more than 10" simulations from the model. ABC-SMC is far less sample efficient in comparison to SNLE/SRE (Lueckmann et al. (02T). Hence, we used considerably more simulations, in case of ABC-SMC, ensure that the parameter estimates are as close as possible to SNLE/SRE so that the differences in estimates of æ, and subsequently y", cannot be attributed to differences in parameter estimates. Further details of ABC-SMC settings are given in Appendix B. Note that for inferring the parameters using ABC, SNLE and SRE we used summary statistics (hand-craftedd Training of the IDE requires the full data. In Appendix E we have also carried out evaluations on the PKY model without using summary statistics, using the SRE and ABC-SMC.
Note that since the SMC algorithm is (asymptoticalll) exact for the task of inferring p(aly, 0), thus we considered the estimates of p(y"ly) obtained using SMC to be the baseline. Hence, we compared different approaches by measuring the maximum mean discrepancy (MMD) (Gretton et al.) (2012) between an estimate obtained by SMC and that obtained using other competing approaches. In particular we considered the sum of the pathwise MMDs for comparison:
M
MMD Ppc
Σ 1=0
MMD2 plui lv), [n)]
(15)
where g is obtained using SMC and y² using approach "a" (IDE, PrDyn, ABC-SMC), We used a rbf kernel and 500 samples, from the posterior predictive distribution, for calculating the M MDD using the formula in Gretton et al.] (2012). For running the SMC algorithm we used a Bootstrap particle-file filter with 100 particles for all models, shown to be enough for these models in Golightly & Wilkinson (2011). We repeated this evaluation for 10 different simulated datasets.
In addition to the biological HMMs, we have also used nonlinear Gaussian state-space model, which has a tractable incremental posterior XXx [X, yt), with fixed parameters, to evaluate the quality of approximation provided by the IDE within a classical filterinn context. See Appendix F for further details of this evaluation.
Additionally, we have also run an experiment with the LV model to show the perils of trying to estimate x. jointly using a neural density estimator. See Appendix G for details.
The NLFI approaches were implemented using the sbil] package (TejeroCCanter et al. 2020). We implemented the stochastic simulation algorithm (Gillespie) 1977) in C++ to simulate the LV and PKY models, and an Euler-Manuyama solver (Kloeden & Jentzen 2007), implemented in Python, was used to simulate the SIR. All experiments were run on a high performance computing cluster. Our code is available at racted.
6.1 MODELS
The stochastic Lotka-Volterra model: This model (Wilkinson, 2018) has been widely used for benchmarking (see Fearnhead et al. (2014): Giagos (010)) a variety of inference algorithms
hhtps//wwwwmeekkelabnn/hhii

---

Under review as a conference paper at ICLR 2023
including ones that use NLFI. This model describes a population comprising of two competing species: predators which die with rate 02 and reproduce with rate ej by consuming prey, which in turn reproduce with rate The dynamics of the states X; (X.rre .xpre)) is governed by a MJP and the objective here is recover the posterior of the parameters 0 (ci, C22 c3) and the hidden states given noisy observations of the state X₁.
Prokaryotic autoregulatory gene network: Golightly Wilkinson 201 considered the infer- ence of an autoregulatory gene network that consists of four species X; (RN As, Pt. P2,, DN As) evolving according to MJP. The parameter vector = (C1, C2 C3, C4, C52 Cb C7, 0g) consists of eight reaction constants. The latent MJP is observed as sum of the proteins P, P2 each time-point. This model has traits of un-identifiability and had been used for benchmarking various inference methods.
The SIR model (Anderson et al. 1992) of infectious disease models the number of susceptible (S), infected (!), and recovered (R) people in a population subjected to an epidemic. The stochastic version of the SIR model, for a population of Npop people, can be described as a SDE consisting the state vector X, (S,. It). Note that for the SIR model we have Noop = St + It + R. The SDE is governed by two parameters that control the infection 3 and recovery 7 rates. In addition to 3, 7 we also estimated the fractional initial susceptibility, = So/N, assuming the initial recovered fraction To = and thus to = Sp. Hence, the parameter vector is
The SIR Compartmental Model:
0 (3, 7; so).
See Appendix C for further details such as priors, data-generation and summary statistics for the models introduced above.
6.2 RESULTS
In Table I we furnished the MMD estimates for all the models, which show that our proposed method produces an estimate of the posterior predictive that is closer to what can be achieved using SMC. we have compared the accuracy of parameters estimates obtained using ABC-SMC and the NLFI methods for the LV and PKY models. Accuracy of the estimates were evaluated as the log probability of the true parameter vector under mixture of multivariate Gaussian densities fitted to 500 samples drawn from an estimate of p(@ by) obtained using each method. We did not notice any drastic difference in accuracy and thus the difference in the MMD estimates were largely influenced by the estimates of the hidden states. The marginal densitics of the parameter posteriors, for one dataset, are shown in Appendix D.
In Figure 2
Posterior sample paths obtained by the IDE are almost indistinguishable from a SMC estimate (shown for one dataset in Figure 3). Notice the overestimation of uncertainty in case of PrDyn and ABC-SMC, for all models.
Table I: The sum of pathwise MMD (smaller the better, given by equation 15 between estimates of the posterior predictive distribution obtained using SMC and other competing approaches: IDE (proposed), PrDyn and ABC-SMC (used for LV and PKY model only). These MMDs summarised by the mean = standard deviation across 10 different simulated datasests.
COMPARISON OF THE ESTIMATE OF p(y"ly) WITH SMC (BASELINE)
(WHEN p(O'y) FOR SMC, IDE AND PRDYN OBTAINED USING SNLE)
MODEL
IDE
PRDYN
ABC-SMC
LV
1.8965 = 0.2432
17.2093 - 1.1659
15.4923 + 0.8170
PKY
4.4448 + 1.3243
20.9553 + 2.7296
53.2612 + 4.9731
SIR
1.1372 + 0.5208
5.0773 + 0.2570
(WHEN p(Oly) FOR SMC. IDE AND PRDYN OBTAINED USING SRE)
MODEL.
IDE
PRDyN
ABC-SMC
LV
1.8998 + 0.2491
17.1193 1 1.1592
15.4705 0,8785795
PKY
5.5491 + 1.4890
21.1983 1 2.0962
53.6201 + 0.0278
SIR
1.1922 1 0.5415
4.7694 1 0.2298
8

---

Under review as a conference paper at ICLR 2023
Log probabiiltyy of frtue paremeters
0.0
-2.5
-200
-5.0
-400
-7.5
-10.0
-600
-12.5
-15.0
-17.5
-1000
ABC-SMC SNLE
SRE
200.0
ABC-SMC SNLE
SRE
(a)
(b)
Figure Comparison of the accuracy of parameter estimates for the Lotka- Volterra (a) and Prokaryotic autoregulator (b) models, using the log probability of the generative parameter
vector, summarised across the 10 datasets.
ABC IDE
SMC
PrDyn
500
300
400
60
250
Predator
300
2P2
50
200
40
200
30
100
100
20
%
20
40
100
%
5
times
times
times
(a)
(b)
(c)
Figure 3: Posterior distributions of one component of the latent sample path æ summarised by the mean (solid lines) and 95% credible intervals (broken lines), for Lotka-Volterse (a), Prokaryotic autoregulator (b) and SIR (c) model. The baseline is SMC.
6.3 CHOICE BETWEEN SMC AND IDE IN TERMS SIMULATION BUDGET
From the experiments it became clear that the proposed IDE is the only approach comparable to SMC in terms of the quality of estimate of the posterior predictive. However, note that the total number of simulations required for SMC is the size of the particle population times the number of parameter samples, which for the experiments was 100 x 500. These are simulations required in addition to the needed learn 0. In comparison the IDE re-used the simulation needed to learn 0. Thus, IDE can be preferred over SMC, even when g(.) (-) is available.
7 CONCLUSION
Neural Iiclihhodd-ree methods have been previously benchmarked using implicit HMMs and are proposed as computationally cheaper alternative classical methods such as ABC-SMC in sur- rounding literature. We argued (and have shown empirically) that both classical as well as neural likelihood-free methods, by ignoring the state estimation, can lead to grossly incorrect assessment of the goodness-of-fit. We thus proposed novel postprocessing technique to approximately estimate the hidden states once samples from the posterior (of the parameters) have been obtained, by any likelihood-Fii ffreeoodd--eeeeeeo method. Our technique, based on learning the posterior Markov process, using autoregressive flow, produced estimate the hidden states closer to what can obtained using a SMC based postprocessing that can be only used for tractable observational densities.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Roy M Anderson, B Anderson, and Robert M May. Infectious diseases of humans: dynamics and control. Oxford university press, 1992.
Christophe Andrieu and Gareth O Roberts. The marginal approach for efficient monte carlo computations. The Annals of Statistics, 37(2):697-725, 2009.
Christophe Andrieu, Arnaud Doucet, and Roman Holenstein. Particle markov chain monte carlo methods. Journal of the Royal Statisticul Society: Series B (Statistical Methodology). 72(3): 269- 3499332,220 2010.
Mark A Beaumont. Estimation of population growth decline in genetically monitored populations. 2003.
Genetics,
Christopher M Bishop. Mixture density networks. 1994.
Kyle Cranmer, Juan Pavez, and Gilles Louppe. Approximating likelihood ratios with calibrated discriminative classifiers. arXiv preprint arXiv: 1506. 02169, 2015.
Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation based inference. Proceedings of the National Academy of Sciences, 1 100543) 2020.
Drew Creal. A survey of sequential monte carlo methods for economics and finance. Econometric reviews, 31(3):245-25 296, 2012.
Thomas A Dean, Sumeetpal Singh, Ajay Jasra, and Gareth W Peters. Parameter estimation for hidden markov models with intractable likelihoods. Scandinavian Journal of Statistics, 41(4): 970-987, 2014.
Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. An adaptive sequential monte carlo method for
approximate bayesian computation. Statistics and computing. 2209):1009 2012.
Arnaud Doucet, Adam M Johansen, et al. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of nonlineur filtering, 72(656-704)33, 2009.
Christopher C Drovandi, Anthony N Pettitt, and Roy A McCutchan. Exact and approximate bayesian inference for low integer-valued time series models with intractable likelihoods. Bayesian Analysis, 11(2):325 312,,,25-352,,, 2016.
Conor Durkan, George Papamakarios, and lain Murray. Sequential neural methods for likelihood-free inference. arXiv preprint arXiv:1811.08723, 2018.
Conor Durkan, lain Murray, and George Papamakarios. On contrastive learning for likelihood inference. In International Conference on Machine Learning, pp- 2771-2781. PMLR, 2020.
Paul Fearnhead, Vasilieos Giagos, and Chris Sherlock. Inference for reaction networks using the linear noise approximation. Biometrics, 70(2):457-466, 2014.
Sarah Filippi, Chris P Barnes, Julien Comebise, and Michael PH Stumpf. On optimality of kernels for approximate bayesian computation using sequential monte carlo. Statistical applications in genetics and molecular biology, 12(1):87-107, 2013.
Christiane Fuchs. Inference for diffusion processes: with applications in life sciences. Springer Science & Business Media, 2013.
Andrew Gelman, Xiao-Li Meng, and Hal Stem. Posterior predictive assessment of model fitness via realized discrepancies. Statistica sinica, pp. 733-760, 1996.
Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In International Conference Machine Learning, pp. 881-889. PMLR, 2015.
Vasileios Giagos. Inference for auto-regulatory genetic networks using diffussio process approxima- fions. PhD thesis, Lancaster University, 2010.
10

---

Under review as a conference paper at ICLR 2023
Walter R Gilks and Carlo Berzuini. Following moving target monte carlo inference for dynamic bayesian models. Journal of the Royal Statistical Sociery: Series B (Statistical Methodology), 63 (1):127-146, 2001.
Daniel Gillespie. Exact stochastic simulation coupled chemical reactions. The journal of physical chemistry, (1(25):330-2361, 1977.
Andrew Golightly and Colin S Gillespie. Simulation of stochastic kinetic models. In In Silico Systems Biology, pp. 187. Springer, 2013.
Andrew Golightly and Darren J Wilkinson. Bayesian parameter inference for stochastic biochemical network models using particle markov chain monte carlo. Interface 1(6):807-820, 2011.
Neil Gordon, David Salmond, and Craig Ewing. Bayesian state estimation for tracking and guidance using the bootstrap filter. Journal of Guidance, Control, and Dynamics, 18(6): 1434-1443,, 1995.
David Greenberg, Marcel Nonnenmacher, and Jakob Macke. Automatic posterior transformation for likelihood-free inference. International Conference on Machine Learning, pp. 2404-2414. PMLR, 2019.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(M37):733-773, 2012.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 297-304. JMLR Workshop and Conference Proceedings, 2010.
Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood- frre meme with amortized approxi- mate ratio estimators. In International Conference on Machine Learning, pp. 4239 -248, PMLR, 2020.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Peter E Kloeden and Arnulf Jentzen. Pathwise convergent higher order numerical schemes for random ordinary differential equations. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, (222008):22292294 2007.
Peter Eris Kloeden, Eckhard Platen, and Henri Schurz. Numerical solution of SDE through computer experiments. Springer Science & Business Media, 2012.
Jan-Mathis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural dynamics. Advances in neural information processing systems, 30, 2017.
Jan-Matthis I Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Bench- marking simulation- inference. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Pro- ceedings of Machine Learning Research, pp. 343-351. PMLR, 13-15 Apr 2021.
Jean-Michel Marin, Pierre Pudlo, Christian P Robert, and Robin J Ryder. Approximate bayesian computational methods. Statistics and Computing, 22(6): 167-1180, 2012.
Paul Marjoram, John Molitor, Vincent Plagnol, and Simon Tavaré. Markov chain monte carlo without likelihoods. Proceedings of the National Academy of Sciences, 100426;:1324411328, 2003.
Gael M Martin, Brendan PM McCabe, David T Frazier, Worapree Maneesoonthorn, and Christian P Robert. Auxiliary likelihood-bart approximate bayesian computation in state space models. Journal of Computational and Graphical Statistics, 28(3):508-522, 2019.
Radford M Neal. Slice sampling. The annals of statistics, 3133)705-777, 2003.
11

---

Under review as a conference paper at ICLR 2023
George Papamakarios and lain Murray. Fast e-free inference of simulation models with bayesian conditional density estimation. Advances in neural information processing systems, 29, 2016.
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. Advances in neural information processing systems, 30, 2017.
George Papamakarios, David Sterratt, and lain Murray. Sequential neural likelihood: Fast likelihood- free inference with autoregressive flows. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 837-848. PMLR, 2019.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1 2021.
Matthew David Parno. Transport maps for accelerated Bayesian computation. PhD thesis, Mas- sachusetts Institute of Technology, 2015.
Umberto Picchini. Inference for sde models via approximate bayesian computation. Journal of Computational and Graphical Statistics, 23(4): 1080-1100, 2014.
Michael K Pitt and Neil Shephard. Filtering via simulation: Auxiliary particle filters. Journal of the American statistical association, 94(446):590-599 1999.
Jonathan K Pritchard, Mark Seielstad, Anna Perez-Lezaun, and Marcus Feldman. Population growth of human chromosomes: study of y chromosome microsatellites. Molecular biology and evolution, 1611))1191-1199,, 1999.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on machine learning. pp. 1530-1538. PMLR, 2015.
Simo Särkkä. Bayesian filtering and smoothing. Number 3. Cambridge university press, 2013.
Scott A Sisson, Yanan Fan, and Mark Beaumont. Handbook of approximate Bayesian computation. CRC Press, 2018.
Esteban G Tabak and Cristina V Turner. A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics, 66(2):145-164, 2013.
Alvaro Tejero-Cantero, Jan Boelts, Michael Deistler, Jan-Matthis Lucckmann, Conor Durkan, Pedro J. Gonçalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulation-basee inference. Journal of Open Source Software, 5(52):2505, 2020. doi: 10211005/oss.005555 URL
hittps://dot org/10 wtt.o///......../..er.. joss 02505
T. Toni, D. Welch, N. Strelkowa, A. Ipsen, and M. P.H Stumpf. Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of the Royal Society Interface, 6(31):187-20) February 2009.
Darren Wilkinson. Stochustic modelling for systems biology. CRC press, 2018
A DERIVATIONS OF ABC AND INCREMENTAL POSTERIORS OF HMM
A. JOINT DISTRIBUTION FOR HMM USING ABC
NLFI methods are designed efficiently sample from the marginal distribution p(0ly). In ABC although the desired outcome often is the marginal distribution, however it is easy show that for latent variable model, such as an implicit HMM, ABC does indeed target an approximation of the joint distribution p(0, x x
In ABC we rely upon simulation of pseudo-data ÿ, when the likelihood p(y(0) is intractable. The operating principle of any standard ABC algorithm, based rejection sampling (Pritchard al.
12


Reviews:
[{"review_id": "uNG_ZdAIbo", "reviewer": "Reviewer_AdSN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors identify issues with neural likelihood-free methods for joint density estimation in hidden variable models and propose a \"post-processing\" technique using samples from the posterior of the parameters that can be generically applied to mitigate these issues. The proposed approach is of interest primarily in applications in which estimation of the hidden variables important.  The approach is validated using simulations from three different biological HMMs against an SMC baseline.", "summary_of_the_review": "An interesting approach the fills gaps in the literature, but the novelty might be limited.", "strength_and_weaknesses": "Strengths:  The work identifies and attempts to address weaknesses in current approaches to likelihood free inference.\n\nWeaknesses:  Estimating the hidden variables isn't always so useful (so applicability of this approach is somewhat limited).  Some novelty issues related to the approach, e.g., the key concepts have appeared elsewhere (just not in this specific context).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "s_-Qhcw3iZ", "reviewer": "Reviewer_DLUp", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is about learning the full (over parameters and hidden states) posterior of a Hidden Markov Model. The motivation for inferring the hidden states as well is that it is either an end in itself or a means to better estimate the posterior-predictive to assess the model's goodness of fit. The key insight is to break down the full posterior into two parts: a distribution over parameters and a distribution over hidden states. The first is learnt using Simulation-Based Inference powered by Deep Learning. The second term is learnt by maximizing the likelihood of a normalizing flow architecture that follows the simple Markov factorization of the HMM's hidden states. Experiments are conducted to show that this combination of methods can efficiently estimate the full posterior and, as a consequence, quantifiably improves the estimation of the posterior-predictive. ", "summary_of_the_review": "The authors presented a convincing-enough argument for why current methods cannot efficiently learn the posterior over hidden states (high-dimensionality of x, or the HMM prior is a bad choice of MCMC proposal). Their method is clearly explained and can be applied after estimating the posterior over parameters, making it a convenient option. It seems broadly applicable: first learn the posterior over parameters, then use samples of these parameters to simulate hidden variables which are learnt by an architecture that mimicks the conditional dependency structure that is assumed for these hidden variables. The experiments seem to support the efficiency of this method in practice.", "strength_and_weaknesses": "The authors motivate the need to estimate the full posterior and discuss the limitations of existing methods. Their experiments seem to support their claim that estimating the hidden states improves the estimation of the posterior-predictive. The relation to prior work is thoroughly discussed. \n\nI am curious as to why MMD (versus another IPM or sample-based divergence) is used to evaluate the posterior-predictive. \n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dZsc1xXSCu5", "reviewer": "Reviewer_pyEj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper discusses the use of neural density estimation problems for latent variables in hidden Markov models. To do so, the authors suggest to amend the neural posterior with a conditional density estimator that factorizes according to the Markov property of the hidden states. This enables more efficient learning. The method is demonstrated on three tasks where it is compared with classical ABC techniques and SNLE and SRE approaches using summary statistics. ", "summary_of_the_review": "The approach presented here might work well in practice, but the manuscript at the current stage seems rushed, containing many imprecise passages. The experiment section is weak, since important aspects for sources of differences haven't been properly investigated. For example, using the component-wise MMD as the only measure of the posterior predictive, or the use of summary statistics for competing models. I don't think the paper should be accepted before those points are adequately addressed. ", "strength_and_weaknesses": "__Strengths:__\n- The approach provides a sensible complement to neural posterior estimation techniques, providing an additional estimate of the latent states\n- The method is easy to understand and a straight-froward extension to neural SBI tools. \n- It can be implemented using existing conditional density estimators.\n- The method seems to perform well in experiments (although with caveats, see below).\n\n__Weaknesses:__\n- The authors use hand-crafted summary statistics for SNRE. That's an inappropriate comparison. A RNN or other temporal neural network should be used. \n- Provided alternatives use summary statistics. If those are not sufficient (and they likely are not) a loss in performance is expected. However, methods that can deal with time-series data exist. Such comparisons would be important because it's impossible to disentangle the loss of information due to summary statistics and the general performance of the density estimation approach. E.g. why not compare to an ABC method that can deal with time-series data?\n- Even for SNLE either the full data could be used (probably doesn't work too well) or approximately sufficient summary statistics could be learned [1].\n- I am not convinced the MMD is the best measure for comparing time-series output.\n- A completely tractable example would be helpful to see the performance vs. SMC. \n\n[1] Chen, Y., Zhang, D., Gutmann, M. U., Courville, A. C., & Zhu, Z. (2021, January). Neural Approximate Sufficient Statistics for Implicit Models. In ICLR.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_l8JQ5qG_xR", "reviewer": "Reviewer_eUPv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of approximating the joint posterior -- p( parameters, hidden state variables | data) -- for a hidden Markov model with continuous-valued states (not discrete ones). The assumed context is that the generative process can be sampled from, but is not available analytically (e.g. PDFs of the prior over state transitions or the likelihood that generates observations given states may not be easily evaluated). Thus, likelihood-free methods are pursued.\n\nThe claimed contributions are:\n\n1) Exposing the problem of using neural likelihood-free methods to estimate the posterior over parameters *and* hidden states simultaneously (discussed around Eq 5-6). Essentially, the problem is that including hidden states means too many parameters and difficult estimation.\n\n2) Proposing a \"post-processing\" technique to remedy the issue, with 3 steps:\n\na) Estimate p(parameters | data) via standard NLFI\nb) Approximating the posterior over states given parameters via an \"incremental\" factorization\n\n$$\np( x_1, ... x_T | params, data) \\approx \\prod_t q( x_t | x_t-1, params, data)\n$$\n\nc) Train a neural density estimator (masked autoreg. flow, Sec. 4.3) to use for this incremental posterior $q$\n\nThe approach overall (essentially steps b,c above) is called the \"Incremental Density Estimator\" (IDE). \n\nEvaluations on 3 different models suggest that the proposed IDE delivers:\n\n* posterior predictive samples for future observations that are closer to a gold-standard SMC than alternatives like ABC (see Tab 1)\n* better estimates of latent states x (see Fig 3)", "summary_of_the_review": "Overall I think the approach has promise, but I worry about scalability to bigger models (all models here cover 2-4 dimensional x) and lossy approximations (dropping all future observations is a restrictive assumption). I hope revisions can address my concerns.\n\n**Update after discussion**:\n\nRevisions addressed my concerns about assessing scalability and improving clarity. I've raised my score from 5 to 7. Technically, I can either give a 6 or an 8. I will enter 6 since I am still a bit worried about the lossy approximation, but to clarify I'm a bit more of a 7 than the entered score 6.", "strength_and_weaknesses": "\n# Strengths\n\n* Likelihood-free inference for continuous-valued HMMs is an important/interesting topic\n* Proposed IDE approach is rather elegant and clean\n* Plots in Fig 3 (esp 3b) show clear gains from this approach at predicting latent state trajectories x_1:T\n\n\n# Weaknesses\n\nI highlight the main issues impacting my scoring here... see below comments under relevant heading (Quality/Clarity) for detailed elaboration \n\n* W1: The incremental posterior idea is still quite a lossy approximation\n* <strike> W2: Scalability is difficult to assess, more experiments would help</strike>\n* <strike> W3: Need to clearly state that IDE only applies to continuous-state HMMs</strike>\n* <strike> W4: Initial state handling is unclear</strike>\n\n\n**Update after discussion**:\n* W2 has been addressed in Supp F\n* W3 and W4 have been satisfactorily addressed.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject