Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper investigates particle-based state estimation under the presence of unknown observation and transition models. This is challenging for a number of reasons, in particular due to the non-differentiable resampling step in particle-based methods. Prior work has proposed leveraging Fisher’s identity to derive a maximum likelihood objective for the model parameters that bypasses the resampling step; however such existing approaches are computationally expensive for large models. This paper builds on this prior work and introduces a particle approximation that trades off bias for computations efficiency. The authors compare this approach with existing baselines and demonstrate superior performance on a real-world and synthetic AV dataset. 
strengths_and_weaknesses: Strengths:
* This paper studies a well-motivated problem — in real world scenarios the observation / transition models may be unknown, necessitating methods that can estimate the parameters of these models.
* The introduction and methods section are well-written. The paper does a good job detailing the shortcomings of prior work, providing background information for the method, and laying out the proposed approach in a step-by-step basis.
* The experimental results are convincing, demonstrating superior performance compared to a number of baselines across a number of quantitative metrics. 

Weaknesses:
* The paper highlights that this approach is useful in multi-agent settings, yet there is nothing about the method specifically tailored for multi-agent problems. While the experiments focus on a multi-agent setting, this method seems designed for any general state estimation problem. This makes the message of the paper somewhat confusing.
* Even after reviewing the appendix, the paper provides sparse details about the datasets. Ideally the authors can help to answer the following questions: who collected the real world dataset? What is the nature of the agents in the dataset? In which specific ways are the real and synthetic datasets different?
* The paper has a limited number of figures and visualizations. It would help to provide visualizations of the datasets and qualitative analysis of the results. Some of these are available in the appendix, perhaps they can be brought up to the main section of the paper.
* The limitations that the authors describe in section 6, namely scaling this method to more complex empirical domains with occlusions and high-dimensional observations. While these are not strictly necessary components, they can help to strengthen the empirical findings.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: While this paper builds upon a body of prior work, it introduces important technical contributions that enable efficient particle-based estimation for large scale settings. The authors can improve the paper by clarifying their contribution specifically for multi-agent settings, providing more details about the datasets, providing more figures for greater interpretability, and demonstrating results on more complex domains.

**Update**: I appreciate the improvements in clarity regarding the datasets, and the new experiments on (slightly) higher dimensional settings. These are relatively minor changes, so I am keeping my initial recommendation of weak accept.
issues: (prefer refer to the list of weaknesses)
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a method for learning observation and transition models for birds-eye view multi-car autonomous driving scenarios.  Using a fixed-lag approximation of the score function along with a deterministic motion model, inference of high-dimensional models can be achieved.
strengths_and_weaknesses: Strengths:

The paper is well-written, and the method is generally well-justified.  The method leverages specific assumptions from the bird's eye view autonomous driving scenario to learn observation and transition models. 

Weaknesses/Questions:

More evaluations on the fixed-lag size would be beneficial. At what point is path degeneracy an issue? At what point is a fixed-lag uninformative? Since this most likely depends on the number of particles as well, it would be nice to see a comparison with the fixed-lag window size and number of particles being modified simultaneously.  

Why does the real data show less performance difference when modifying fixed-lag? Why did real data cause large gradients across all methods? If performance is independent of fixed-lag size, what is the method gaining? Plot shows minimum of L=5 for real-data, but what happens for L=1,…,5? If this small of window size gives similar results for real-data, then learning the observation and transition models may not require longer horizon inference.

If DPF-SGR performs better in terms of accuracy for 25 steps, what is preventing methods from also limiting the number of steps that is used for training?

Since other methods may or may not differentiate through the marginal log-likelihood, does it make sense to even compare the baselines using this?

Relative bearing is usually not available as a measurement, and estimates would be somewhat noisy.  Is this a realistic setup?

Other comments:

Figure 1 is somewhat wasteful in terms of white space, and also does not illustrate much.  Consider improving the figure as this is the only one in the paper.  One of the figures in the supplementary material may better illustrate the method and application than the current Figure 1.

How sensitive is the method to the agent's state noise? In reality, this will be imperfect.  

Is assuming the motion model to be deterministic adequate? In real-world scenarios, this may not be the case.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, the method provides is well-written and provides interesting methodology.  While the evaluations demonstrate the effectiveness of the method, some additional experiments demonstrating the benefits of the proposed fixed-lag approximations is needed.  One of the baselines performs better on the 25 step simulate data, while the fixed-lag size does not show much difference in performance on the real data.


Update:
The authors have addressed the main concern about the effect of the fixed-lag length, as well as other questions.  Therefore, I have changed my recommendation to weak accept.
issues: See above.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper proposes a new approach for particle filtering approach for estimating the score function of state-space models (SSM). The authors do so using the Fisher Identity to circumvent the non-differentiable sampling step in particle filtering for estimating the score function. Moreover, they circumvent the potential issue of path degeneracy where the particles converge to a single one by using a fixed lag $L$ up to which the estimates are calculated, based on the assumption that observations after a time $t+L$ are not very useful for estimates at the current time step. They also derive the use of a motion model for policy-based approximations by showing that the gradient of the policy corresponds with that of the SSM, allowing it to be plugged into the gradient of the score function. Their results for approximating the states of external objects from Bird's eye views of a vehicle show good log-likelihoods and state estimates on both real and simulated datasets.
strengths_and_weaknesses: ### Strengths
- The proposal for using the Fisher Identity for approximating SSMs enables low variance state estimates and also circumvents the non-differentiable nature of the sampling process.
- The computational capacity is drastically reduced by considering a fixed time window up to which the estimates are calculated rather than going through the whole trajectory.
- The paper is very well written. It is very easy to follow and the relevant works, preliminaries and approaches are explained in simple, understandable terms and the paper has a good flow from start to end.

### Weaknesses
- The paper would benefit from providing some more extensive information about the real dataset that is used, such as how the data was collected, sensors used, example data etc. It would have been better to showcase the results on some existing datasets, like KITTI for example, or even on a subset of it. This would allow the results to be more comprehensible. 
- The paper is currently missing an ablation study to show the extent to which the performance is affected by the different components. These could be different Lag lengths in Sec. 4.2 

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The proposed approach is well motivated and follows a simple, intuitive and useful extension to improve the score functions of SSMs along with the added advantage of reduced computational capacity and incorporating the policy gradient in the score function estimate. The experimental evaluation shows the benefits of the proposed method on both synthetic and real data (although the real dataset is much smaller compared to traditional driving datasets like KITTI for example). Moreover, the paper is very well written and the ideas are presented well, making it a nice read.
issues: - There are some issues that I found in Lines 140-141 and Eq. 4 as $\alpha$ and $\delta$ are not defined. This needs to be cleared to properly understand the approach and how it could potentially affect the score function.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The model proposes training non-linear, non-Gaussian SSMs using gradients approximated with Fisher's identity. In order to efficiently compute the required smoothing distributions, it proposes using fixed-lag smoothing. The resulting method is evaluated on a state estimation task in an autonomous vehicle setting where 2D poses of surrounding vehicles need to be estimated from observations.  
strengths_and_weaknesses: The paper is well written and easy to follow, and the main idea, intuitions, and mathematical details are clear. 
Using fixed-lag smoothing to approximate the required smoothing estimates efficiently is a simple idea.
Yet, as this seems to be the paper's main algorithmic contribution the effects and limitations of this approximation would need further investigation. While I agree, that the assumption seems reasonable for many systems, ablations for different smoothing lengths $L$ would be good to see. 
In general, I do not believe the conducted experiment is sufficient to allow assessment of the method's full potential, as it relies on unrealistic assumptions (lines 218-220), largely pre-engineered transition and observation models, and only considers a narrow scope of applications. 

  
robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: While I like the writing and simplicity of the idea, I think the experiments are not sufficient to allow assessment of the method's potential impact beyond the narrow scope it was evaluated in. 
issues: Besides the above-mentioned points, I have minor issues:
- At first, I was a bit confused by Section 4.3, because I thought it to be a fundamental aspect of the method, not just a simplification of the dynamics model done for the specific conducted experiments (although I agree that the general method approach described in this section is applicable to a wider class of experiments). Maybe it could be considered to move this part to the experiment section.  
- If I understand correctly, the MLL is computed based on the filtered estimates, while ADE and AYE use smoothed estimates. Why are different beliefs used here and are the smoothed ones "fixed-lag" using the full trajectory?
review_title: null

Meta-review: metareview: The authors propose a multi-object state space estimation approach based on particle filters where the gradients are computed through the Fischer's identity.

Strength:

   - Clear story and well motivated (to avoid biased or high variance gradient approximation)
   - Well-structured paper
   - Comparison to two baseline methods
   - Evaluation on a real-world dataset and two synthetic datasets
   - Detailed discussion of all model assumptions and limitations

Weakness:

   - The real-world av tracking task is not described in sufficient detail
   - Therefore it is difficult to assess how relevant the approach is and how well it could work on more complex real-world tasks
   - In general, a discussion on the applicability of the model to other problems, not only multi-object tracking, would be important

Update: The modified Figure 1 and the additional explanations at multiple places improve the paper.
confidence: 3: The area chair is somewhat confident
Decision: Accept (Poster)