Paper:
Backprop KF: Learning Discriminative Deterministic State Estimators
Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the entire distribution over sensor readings. Discriminative models do not suffer from this limitation, but are typically more complex to train as latent variable models for state estimation. We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks. Our model can be viewed as a type of recurrent neural network, and the connection to probabilistic filtering allows us to design a network architecture that is particularly well suited for state estimation. We evaluate our approach on synthetic tracking task with raw image inputs and on the visual odometry task in the KITTI dataset. The results show significant improvement over both standard generative approaches and regular recurrent neural networks.

Review:
The paper describes that the standard Kalman Filter (KF) equations can be expressed in a computation graph, thereby integrating probabilistic state estimation with a per-frame convnet. This enables optimizing the convnet and the resulting low-dimensional features with respect to the filtered state directly on a batch of images with known target location using standard back-propagation. The hypothesis is that a KF provides useful domain knowledge for state estimation (i.e. through the linear motion equations and the parametric state probability distribution) over a generic LSTM.   The trained network yields thereby a better representation for state estimation of the target in novel tracking sequences, as compared to training the convnet on individual frames independently without considering the KF. Also, the convnet uses a novel normalization layer which resembles batch normalization. The method is evaluated on an artificial tracking problem consisting of a red disc moving through and occluded by various other colored discs, and compared to independent training of the convnet, and a version where the KF is replaced by LSTM. The results show that the proposed method outperforms these baselines on the artificial dataset. 
I like the idea to investigate integration of the well-known Kalman Filtering in back-propogation, to obtain best of both-worlds: data-driven feature learning and informed motion models. The paper is generally well written and easy to follow. Since one could expect learning motion models would also benefit from non-linear data-driven models (e.g. LSTM) with large amounts of data, instead of using highly informed models with few free parameters, the question when/where this trade-off occurs would be very interesting. The idea that this could improve state estimation in tracking specifically is also appealing,  especially if empirical evidence on real-world tracking tasks can be given.  However, this paper does not really explore the first question. It is unclear how much data is used, and if the exhibited movements are realistic or already very similar towards the linear equations used in the Kalman filter. Also, the presented tracking use-case seems very ad-hoc and not very relevant to any real-world tracking use-case: it presuposes that we posses extensive training data of the instance that we wish to track (a red circle) with a distinct and static appearance, sufficient to train a convnet for the specific appearance. Standard model-based tracking (e.g. multi-person tracking) or model-free tracking (e.g. track unknown object from initial bounding box) do not fit this task, as the instance appearances are not known a-priori.  So while the artificial scenario is good for illustration and explaining the method, it does not really "reflect typical challenges in visual state estimation" (line 202). Due to the limited experimental evaluation and unclear use-case, I rated both Technical Quality and the Potential Impact as substandard for NIPS. More details comments follow below.  ## Detailed comments: - line 104: In this example, what would the hidden system state x_t be? If the state represents the 'true' position and velocity, do you consider y_t a noisy version of x_t (if yes, then why consider y_t the output instead of x_t, if no why call it an observation in a Kalman Filter)?   - line 181. Should L not be defined as the negative log likelihood? I.e. higher likelihood = lower loss.  - line 136: Aren't the two terms in the brackets ("dphi/ds . dL/dphi" and "dL / ds") the same? I would expect the recursion to take the form of "dL / ds_{t-1} = dL / dphi_{t-1} . dphi_{t-1} / ds_{t-1} + ds/ds_{t-1} . dL/ds"  - line 183. introduces a new contribution, namely a novel response normalization layer. Such a novelty warrants extra experiments (e.g. comparing to batch normalization [9]), or at least some insight into why it is "substantially more effective" (line 188). Can you expand on/cite why standard approach "is know(n) to require additional care"?   - line 195.5: Are the KF parameters also trained using back propagation, e.g. Q, A, B, C ? If not, how are they determined?  - The experimental section provides no details on how much training, testing data is used, what kind of back-propogation technique, or any other details to reproduce results. Are the results averaged over multiple folds / new batches of artificial data? For instance, are the up-down patterns in Fig. 6b systematic or due to random effects in a single artificial sequence?  - In the presented artificial dataset, the target seems to have a clearly distinct and static appearance (its color) compared to almost all distractors. So, while technically the model is processing "raw pixel data", this seems to overstate the complexity of dealing with real-world raw pixel data a lot: Creating a good per-frame target response map appears trivial (a simple color threshold), which undermines the whole point of using a convnet to learn a robust representation.  - The main question therefore remains: does this approach help to improve state estimation in real tracking scenarios where the target is not known a-prior, where the appearance is dynamic, has lighting and shape changes, and fore and background as more prone to false positives? E.g. comparing tracking on VOT 2014/2015 sequences, or PETS pedestrian sequences would be more convincing. Also, in normal tracking scenarios you do not have labeled prior examples of the specific target *instance*. At best, you have labeled examples of the target class (e.g. pedestrians). A tracker still needs to online learn a way to discriminate other instances of the same class. For instance, could the method be used to improve and initial tracking results from a baseline tracking method providing inaccurate object labels?   ## Minor comments - Equations are not numbered, annoying for anyone referring to parts of the paper.  - line 138.5: At this point I had several questions (though these were somewhat clarified later in Section 5.2): How are these gradient terms obtained in practice? Why not provide the closed form solutions, or appropriate citations? Or do the authors rely on automatic derivation by a software package? Is there any risk of a gradient descent step leading to non positive-definite covariance?  - line 169: Extended Kalman Filter should have a citation, it was not introduced anywhere before.  - line 261.: "although the accuracy of the BKF can be matched by a recurrent LSTM network". Where is this shown in the experiments? How could one conclude this from Table 1, or Figure 6?