Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Disclaimer: I am not expert in SDE theory so my review will focus on the high level and clarity.

The authors are interested in the filtering problem in SDEs, i.e. computing the conditional distributions x_t|y<t, with x the latent process and y the observations.
The focus is on particule filtering, where the filtering distribution is approximated via discrete particules.
In particule filtering, a guiding function q or proposal is used to propagate the values of the particules in time.
A convenient guide is the 'Fully Adapted Auxiliary Particle Filter' which samples from the conditional p(x_t+1|y_t+1, x_t).

Doob's theorem happens to give an expression for the SDE corresponding to this conditional p(x_t+1|y_t+1, x_t), which shares the same
diffusion as the a priori SDE and whose drift function is equal to the prior drift function plus an offest, or control function. 
Sampling from this SDE for starting point x_t would give the desired sample of p(x_t+1|y_t+1, x_t) for the purpose of particule filtering

The authors propose an algorithm to find learn this control function c, without having to have to first solve a PDE first.
This function and the prior density is parameterized by neural network.
An objective is proposed to learn the control function.
An algorithm is proposed to optimize this objective.

The algorithm is evaluated to check the validity of the learned control in toy settings and in Particle filtering scenarios.

strengths_and_weaknesses: I find the proposed manuscript is well motivated.
The background on particule filtering is clear.

My SDE theory is less developed but I don't understand section 2.5,
not even at a high level. As a result I m not sure I trust the content of section 3.1 (the loss and algorithm)
I m possibly not the best reviewer (hence the low confidence) 
but I also think clarity could be greatly improved (see below).



questions: Some comments / questions:

two related comments first
* eta_x should depend on the conditioning end observation. 
If you drop this dependency, you are not sampling from the conditioned SDE. 
In your experiment (L252), you drop this dependency. Is this a typo, a design choice, an approximation?

* L159: "In practice, we will let the three sources of randomness be independent of each other."
What is lost with that approximation (is it an approximation)? If you choose T large, then the process reverts to the prior (independent of Y), stationary distribution of the uncontrolled SDE.
Is it what you do?

* The result section, shows that the learned control matches the exact control on a linear SDE scenario.
Many details are missing, an appendix with those would be great. 
as a reader, I don't want to have to fill all the gaps. Even less as a reviewer.
For example, for OU, you could give the exact expression for h, and v.

* The proposed objective seems to contain the control on both side of the square loss, the V_T and the log g(..).
I do not really understand or see why the optimal control is a minimizer of objective.
It also feels like hitting a moving target.

* It would be good to have a discussion of this optimization problem.
For example, how do the dynamics of learning look like?

* Could you be more precise on which variables you are taking expectations over and why?
This is important for the loss.

* Also could you give a justification for the 'stop gradient' comment?
You have a loss that has multiple dependence on c and you decide to only propagate through one part.

* CDT algorithm, it would help to have a more detailed pseudo code for the algorithm.

* Doob's transform: I suggest you either give more details or remove the sketch from the main paper. As is it is not useful.

* Where is the conclusion? It is good to end a paper with the conclusion summarizing the method and results.


limitations: None
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper addresses the task of online filtering for discretely observed nonlinear diffusion approaches. The paper proposes a method based on the fully adapted auxiliary particle filter (FA-APF). The paper discusses how a control function can be used to construct a controlled diffusion that takes into account a future observation. The desired control function is related to Doob’s h-transform and can be obtained by solving the backward Kolmogorov equations. This is computationally intractable, so the proposed strategy is to work with the value function (the negative log of h). The optimal control can then be expressed in terms of the value function and approximated with neural networks. These networks can be trained by simulating the controlled diffusion trajectories. After training, the online filtering can be performed using the approximately optimal control. 

The paper presents numerical results for three models: an Ornstein-Uhlenbeck model, a logistic diffusion model, and a cell model. The results are compared to a bootstrap particle filter. For the Ornstein-Uhlenbeck model, which is analytically tractable, there is also comparison with the exact APF. There superiority of the proposed method is clear in all three examples. 

strengths_and_weaknesses: Strengths
1.	The paper addresses the important problem of online inference for diffusion processes (although it doesn’t make a compelling case as to why this problem is important) and provides an elegant algorithm for performing the filtering. To the best of my knowledge, this represents a novel approach to solving this problem and I would rate it as a highly original approach. I think the work could have a significant impact on research addressing online inference for diffusion processes.  

2.	The paper is very well written. The material is relatively dense, but the authors do an excellent job of providing a clear description of the algorithm. It is developed step-by-step in an intuitive manner. 


Weaknesses
1.	Related work and performance comparison: there is very little discussion of existing methods for filtering partially observed diffusions. Multiple methods are acknowledged (and dismissed) with the single sentence “Specialized methodologies have been developed to circumvent or mitigate these issues.” Other methods like [R1] are ignored. 
Even if some of the methods are more limited in their applicability, there should have been an effort to compare with them experimentally for examples where they are applicable. Some discussion of the limitations of existing work should be included in the paper. 

2.	Motivation: The paper does not provide a compelling motivation for the analyzed problem. There are very general claims about the ubiquity of diffusion processes in the first few sentences but no citations to any literature to support these claims. There are no citations of work providing more practical examples of where the proposed filtering problem would prove beneficial. The studied examples, while providing a good illustration of the benefits of the technique, do not provide compelling evidence of the practical utility. The cell model and the logistic diffusion model for population size dynamics do not seem suited to a meaningful, practical filtering task.

3.	Experimental analysis: There is no performance comparison to any methods except for the basic bootstrap particle filter. There have been multiple approaches to address the filtering task, so there should either be a clear explanation as to why existing approaches are inapplicable or other methods should be included. I could not find the details of exactly how many particles are used in the simulations (apart from the specification that it increases linearly with K). This seems to be an important design choice and there should be an explanation as to how the value was selected. Preferably there should be an analysis of how performance changes with varying K. There is no reporting of computational requirements. It is not clear if the proposed filter and the BPF have the same (or very similar) computations. Presumably there are additional neural network evaluations for the proposed scheme, but these may be negligible compared to other computational overhead. If they are non-negligible, then a fairer comparison would use more particles for the BPF. 
 
[R1] Jasra, A., Law, K. J., & Yu, F. (2020). Unbiased filtering of a class of partially observed diffusions. Advances in Applied Probability, 1-27.

questions: 1.	How many particles were used in the experiments (or what was the scaling factor with K)? Why was this value chosen? What is the sensitivity to this choice? Is it possible to achieve improved performance with more particles or is the computational burden already considerable, i.e. if I were trying to implement a real-time system with a reasonably powerful computer, with a reasonable time between observations, how many particles could I use?

2.	What is the computational overhead? How does the computation of the proposed method compare to the BPF? What are the key contributors to the computational burden? 

3.	What are the state-of-the-art competing techniques? Why are these not considered in a quantitative performance comparison? Are methods like [R1] inapplicable for the studied settings? 

limitations: Yes.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a novel method for online filtering where the latent process has a higher frequency than the observations. The authors define an algorithm, Computational Doob’s h-Transform, for iteratively learning a control function over the latent space and an observation to drive a stochastic process to a specific point at time T. The learned control function is then later applied in online filtering, where no further learning is required but the controlled dynamics can be applied to compute the particle filtering weights and to propagate the particles. CDT is applied to low-dimensional settings in the experiments section, where it is compared to bootstrap particle filtering for nonlinear dynamics, and to the exact analytical solution when available.
strengths_and_weaknesses: Strengths
1. An original method for learning a control function for particle filtering using a Doob’s h-transform
2. Separation of learning the control and the filtering itself is an attractive idea for improving efficiency 
3. The work is reproducible (the code was provided and I was able to run it) 

Weaknesses
1. Limitations and related work were not discussed enough, the paper would have benefited from a well-structured Conclusions/Discussion 
     section at the end instead of scattered remarks. It was not clear to me how the method presented compares to earlier work. 
2. While I agree that covering the theory in detail is well justified, having three pages of background section would have been more 
     comfortable to read if there was more text motivating why the topics covered are important or intuitive/heuristic explanations. 
3.   The convergence properties of the iterative scheme were not studied empirically or theoretically, I was not able to develop an intuition 
     into how the control behaves at iteration 1 versus iteration N.
4.   Low dimensional experiments, only comparisons to a simple baseline method (bootstrap particle filter) and not to more recent work
5.  The figures were focused on presenting error metrics, the behaviour of the learned control or the filtering distribution were hardly 
      visualized

questions: Questions
1. On the iterative method: how many iterations of the iterative method were required until satisfactory results/convergence? Can you 
      point to any theoretical results or give heuristics on the behaviour of the learned control through the iterative method? 
2. On dimensionality: How does the method scale when the dimensionality of the problem is changed? I’m concerned of the learning of 
       the control function, and how representative is the sampling scheme for observations during training when the problem is high- 
      dimensional
3.  On experiments; Do they present a fair comparison? Is there some more recent work than bootstrap filters you could compare to in the 
        experiment settings where no analytical solution is available?
4.  On training of neural networks; Did training the initial value and control networks simultaneously present any issues? 	
5.   On the loss function: the loss function considers only the final state of the value function. Was no regularization needed on the 
         dynamics? Why, and are there some cases where it would be beneficial?
6. On the observations: There is a comment in the experiments section (Line 242) that the results are not sensitive to choice of T as long 
        as the discretization has a small enough step size. What is small enough? A heuristic explanation/plot showing how long it takes the 
       control to steer to the observation would have been informative.
7.   On clarity of text: (minor) some parts of the text had, to my knowledge, a number of typos and issues with grammar and notation. In 
      addition, there was some vague phrasing. Please consider going through the following lines

Lines to check:
Line 13-14: applications listed, but no references provided
Line 21: “is collected” while referring to multiple observations
Line 29-30: “the goal of the article is…” This sentence felt too vague on my initial read
Line 33: “these quantities” which quantities are you referring to here? Only the control function has been referred to so far 
Line 37: Efficient in what sense? 
Line 44: “an homogenous” should be “a”?
Line 68: comma before while
Equation 4: why p_T? It is confusing to the reader in the case k> 1. The notation makes sense after reading 88-90, but this equation is before that
Line 136: “Ito lemma” should be “Ito’s lemma”
Line 143: Period missing from the last sentence of the paragraph.
Equation 15: t is used as time on the left-hand size, but it is the integrated variable on the right-hand side
Line 217: why is the control evaluated at t-t_k?
Line 238: odd to say that all experiments had 10 observations, when nearly all experiments vary K?
Line 248: why was it required to grow the number of samples when the number of observations grows? Does this mean that when K=1000, there were 100k samples (since there were 1k samples for K=10)
Line 255: “Analytically tractability” 
Figure 1: the figure text was quite vague, I didn’t get what I should be concluding based on the plots? I can see that they somewhat match, but was there some additional insight I’m missing here?
Figure 2: The y-axis labels slightly overlap with the subplot borders

limitations: 1. Societal impact of the work was not discussed. As the paper is presenting theoretical work, it does not require a deeper investigation into such matters. 
2.  Scalability to higher dimensional problems and convergence of the iterative method were not sufficiently discussed or empirically tested, even though scalability to high-dimensional problems was explicitly stated as a motivating factor for not using standard numerical methods for solving PDEs (Line 126-128)
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper develops an approximation to the locally optimal particle filter in discretely observed diffusion processes under the assumption of a constant measurement rate. 
strengths_and_weaknesses: S1 - The method compares favourably to baseline solutions. 

W1 - Experiments are not entirely convincing. More specifically, only low-dimensional examples re considered (d <= 2), which undermines the rationale for their method in that we are not in a high-dimensional regime where numerical  solutions of PDEs become infeasible (paragraph 1 of sec 2.5). 

W2 - The method relies on fixed SDE dynamics so is not applicable to parameter inference. 

W3 - The method relies on constant sample rate, which is many times infeasible in practice due to e.g. missing measurements. 
questions: 
Q1 - The difference between CDT static scheme and bootstrap proposals needs to be elucidated. Choosing c = 0 yield the original SDE dynamics, i.e. bootstrap proposals, no? 
limitations: Its fine.
review_title: null

Meta-review: metareview: This is an interesting paper, but it also requires some additional work. The paper received mixed responses from the reviewers, but the main concerns were sorted out during the rebuttal and discussion. In the end, the reviewers ended up (weakly) recommending acceptance after the authors promised an extensive number of changes to the paper in the camera-ready stage. However, the promised changes appear very extensive and beyond what would be expected for a typical NeurIPS paper (list below). The authors have provided links to plots in the discussion, but have not revised the paper in OpenReview. Sorting out the issues counts as a 'major revision' and the conference review process does not recognise such a concept.

List of changes (based on rebuttals):
* Empirical comparison (& discussion) to GIRF instead of only bootstrap particle filter
* Adding a discussion section to the end of the paper
* Writing a more thorough Related work section
* A study on the impact of growing dimensionality (OU process)
* Plots and discussion on the learned dynamics (control)
* Plots and discussion on training loss
* A more intuitive explanation of the Doob's h-transform in the Background section
* Expanded Introduction to discuss applications (with references)
* Computational cost discussion (promised to add to the appendix)
* Figures illustrating Doob’s h-transform trajectories
* Discussion on how fine discretization is needed
* Discussion on parameter inference and the smoothing problem
* Discussion on irregular observation intervals
confidence: Less certain
Decision: Reject