DNBP: Differentiable Nonparametric Belief Propagation
Authors: Anthony Opipari
Jana Pavlasek
Chao Chen
Shoutian Wang
Karthik Desingh
Odest Jenkins
Venue: ICLR 2022 Submitted (2022)
Keywords: Belief Propagation
Bayesian Inference
Nonparametric Inference

Abstract: We present a differentiable approach to learn the probabilistic factors used for inference by a nonparametric belief propagation algorithm. Existing nonparametric belief propagation methods rely on domain-specific features encoded in the probabilistic factors of a graphical model. In this work, we replace each crafted factor with a differentiable neural network enabling the factors to be learned using an efficient optimization routine from labeled data. By combining differentiable neural networks with an efficient belief propagation algorithm, our method learns to maintain a set of marginal posterior samples using end-to-end training. We evaluate our differentiable nonparametric belief propagation (DNBP) method on a set of articulated pose tracking tasks and compare performance with learned baselines. Results from these experiments demonstrate the effectiveness of using learned factors for tracking and suggest the practical advantage over hand-crafted approaches. The project webpage is available at: https://sites.google.com/view/diff-nbp

Under review as a conference paper at ICLR 2022
DNBP: DIFFERENTIABLE NONPARAMETRIC BELIEF PROPAGATION
Anonymous authors Paper under double-blind review
ABSTRACT
We present a differentiable approach to learn the probabilistic factors used for in- ference by a nonparametric belief propagation algorithm. Existing nonparametric belief propagation methods rely on fomaann-speciff features encoded in the prob- abilistic factors of graphical model. In this work, we replace each crafted factor with differentiable neural network enabling the factors to be learned using an efficient optimization routine from labeled data. By combining differentiable neu- ral networks with an efficient belief propagation algorithm, our method learns to maintain a set of marginal posterior samples using end-to-end training. We evalu- ate our differentiable nonparametric belief propagation (DNBP) method on set of articulated pose tracking tasks and compare performance with learned baselines. Results from these experiments demonstrate the effectiveness of using learned fac- tors for tracking and suggest the practical advantage over hand-crafted approaches. The project webpage is available at: https://sites.gogglt wwwglle.com/vvww/eee.
Observation Sequence
Extraction
Nonparametric Graphical
Maximum Likellhood
Estimates Reported Uncertainty
Model Inference
Estimation
Usoo Neural Network
Magiani
crt Goddem Desceet
Gradient
Figure I: Architecture diagram of differentiable nonparametric belief propagation. DNBP combines domain knowledge in the form of graphical models with differentiable neural networks for tractable inference in con- tinuous spaces. Input features from deep neural network and the probabilistic relationships encoded in graphical model are learned jointly an end-to-end fashion using backpropagation. Following offline training. DNBP can be applicd unseen data without hand-tuning.
INTRODUCTION
A significant challenge for robotic applications is the ability to estimate the pose of articulated objects in high noise environments. Nonparametric belief propagation (NBP) algorithms (Sudderth et al., 2003; Isard, 2003) have proven effective for inference in visual perception tasks such as human pose tracking (Sigal et al., 2004) and articulated object tracking in robotic perception (Desingh al., 2019; Pavlasek et al., 2020). Moreover, these algorithms are able to account for uncertainty in their estimates when environmental noise high and show promising computational properties in practice (Desingh et al., 2019; Ortiz et al., 2021). Their adaptability to applications, however, is limited by the need to define hand-crafted functions that describe the distinct statistical relationships in particular dataset. Current methods that utilize NBP rely extensive domain knowledge to parameterize these relationships. Reducing the domain knowledge required by NBP methods would enable their in broader range of applications.
As a form of probabilistic graphical model inference, NBP algorithms leverage domain knowledge encoded in graph- representations, such as the Markov random field (MRF). Their capacity to

---

Under review as a conference paper at ICLR 2022
perform inference using arbitrary graphs sets them apart from other algorithms such as the recursive Bayes filter (Thrun et al., 2005) (e.g. particle filter (Godsill, 2019)) and has been shown to be impor- tant in computational perception because it allows for modeling of non-causal relationships (Sud- derth et al., 2003). Data-driven approaches are an alternative for computational perception (Xiang al., 2018; Tremblay al., 2018). These methods generally avoid the need for extensive domain knowledge by learning from large amounts of labelled data. Data-driven approaches, however, are prone to noisy estimates and have limited capacity to represent uncertainty inherent in their esti- mates. In robotic applications, both of these limitations negatively impact the ability for robot to operate effectively in unstructured environments.
In this paper, we present differentiable nonparametric belief propagation (DNBP) method, hy- brid approach which leverages neural networks to parameterize the NBP algorithm. Through dif- ferentiable inference, DNBP leverages the explainability and robustness of probabilistic inference techniques and capitalizes on the efficiency and generalizability of data-driven approaches. Inspired by the differentiable partiele filter (DPF) from Jonschkowski et al. (2018) and the pull message pass- ing for nonparametric belief propagation (PMPNBP) algorithm (Desingh et al., 2019), we develop a differentiable nonparametrie belief propagation algorithm. DNBP performs end-to-end learning of each probabilistic factor required for graphical model inference.
The effectiveness of DNBP is demonstrated on two simulated articulated tracking tasks and real-world hand pose tracking tasks in challenging noisy environments. An analysis of the learned probabilistic factors and resulting tracking performance is used to validate the approach. Results show that our approach can leverage the graph structure to report uncertainty about its estimates while significantly reducing the need for prior domain knowledge required by previous NBP meth- ods. DNBP performs competitively in comparison to traditional learning- tracking tasks. Collectively, these results indicate that DNBP has the potential to be successfully applied to robotic perception tasks, where a notion of uncertainty in the inference is inevitable.
approaches on the
RELATED WORK
Belief Propagation: In the context of graphical models, inference refers to the process in which information about observed variables is used to derive the posterior distribution(s) of unobserved variables. Belief propagation (BP) is message passing algorithm for inference graphical models. BP computes exact marginal distributions on trees (Pearl, 1988), and has demonstrated empirical success on loopy graphs (Murphy et al., 1999; Sun et al., 2003; Lee et al., 2008; Lan et al., 2006). In order to apply inference techniques such as BP and LBP, the parameters of graphical model (e.g. the probabilistic factors) must be fully specified. Maximum likelihood estimation (MLE) has been shown to be an effective approach for learning the parameters of graphical model from data (Murphy, 2012; Koller & Friedman, 2009; Ping & Ihler, 2017). In contrast, this current study focuses on parameter learning for use with inference of continuous random variables.
Nonparametrie Belief Propagation: For continuous spaces, such as six degrees- ofwfreeooomm object pose, exact integrals called for in BP and LBP become intractable and approximate methods for inference have been considered. Nonparametric belief propagation (NBP) methods (Isard, 2003; Sudderth et al., 2003), have been proposed which represent the inferred marginal distributions using mixtures of Gaussians and define efficient message passing approximations for inference. Isard (2003) demonstrated the effectiveness of PAMPAS using set of synthetic visual datasets each modeled with hand-crafted factors. Sudderth al. (2003) applied their NBP method successfully to visual parts-based face localization task as well as a human hand tracking task (Sudderth et al., 2004). In both applications. NBP relied on factor models which were chosen based on task-level domain knowledge (e.g. valid configurations of human hands). Sigal et al. (2004) extended these NBP methods to human pose estimation and tracking using factors which were each trained separate from the inference algorithm using independent training objectives.
Ihler & McAllester (2009) described a conceptual theory of particle belief propagation, where mes- sages being sent to inform the marginal of particular variable could be generated using shared proposal distribution. Following the work of lhler and McAllester, Desingh et al. (2019) presented an efficient "pull" message passing algorithm (PMPNBP) which uses weighted particle set to ap- proximate messages between random variables. PMPNBP was shown to be effective on robot pose estimation tasks using hand-crafted factors. Using similar approximation of belief propagation,
2

---

Under review as a conference paper at ICLR 2022
Pavlasek et al. (2020) took step toward neural network-based potential functions by introducing a pre-trained image segmentation network to the unary factors. An important limitation of these works is they assume the probabilistic factors expressed in the graph provided imput rely on domain knowledge to separately model and train each function. The potential for neural networks to learn the parameters used by alternative inference techniques has been demonstrated (Do & Artières, 2010; Tompson et al., 2014). The recent work of Xiong & Ruozzi (2020) demonstrated promising inference performance on discrete classification and synthetic datasets using learned probabilistic factors with a Bethe free energy approximation. In contrast, this paper focuses on inference high dimensional continuous state spaces using learned probabilistic factors with the particle-based in- ference process NBP.
Differentiable Bayes Filtering: In the context of robot state estimation, many approaches have re- cently been proposed that incorporate neural networks with recursive inference algorithms in an end- to-end fashion. Haarnoja et al. (2016) introduced differentiable Kalman filter, and Jonschkowski & Brock (2016) proposed a differentiable, histogram- based Bayes filter algorithm. Jonschkowski et al. (2018) and Karkus et al. (2018) both proposed differentiable particle filter algorithms for modeling continuous state spaces. Kloss et al. (2021) evaluate recent differentiable filtering techniques. Yi al. (2021) propose an end-to-end learning method for inference over factor-graph models. In con- trast to these methods, which model a single object body using variants of the Bayes filter, this work sets out study the potential for NBP be used as an algorithmic prior for modeling multi-par articulated objects. Recently, this line of research on differentiable state estimation algorithms has extended into the planning domain (Karkus et al., 2019; Wang et al., 2020; Anderson et al., 2019). Exploration of embedding DNBP within a differentiable planning system is left as future work.
3 BELIEF PROPAGATION
Consider Markov Random Field (MRF) defined by the undirected graph 9 = (V,E), where de- notes set of nodes and & denotes set of edges, An example MRF model is shown in Fig. 2b. Each node in represents an observed (grey) or unobserved (white) random variable, while each edge in & represents a pairwise relationship between two random variables in V. The joint probability distribution for 9 is:
p(X
1
II Ved(X,, Xa) II oa Ya) deV
(1)
(a.d)EE
where X'' = (Xa / v) is the set of unobserved variables and y = (Y2 d E V} is the set of cor- responding observed variables. The scalar Z is normalizing constant. For each node, the function <a(-) is the unary potential, describing the compatibility of Xa with corresponding observed vari- able Ya. For each edge, the function (-) the pairwise potential, describing the compatibility of neighboring variables X2 and Xa. This work considers MRF models limited to pairwise clique potentials.
Given the factorization of the joint distribution defined in Eq. (1), BP provides an algorithm for in- ference of the marginal posterior distributions, know as the beliefs, bela(Xa). BP defines a message passing scheme for calculation of the beliefs as follows:
bela (Xa, Ya)
II (Xa)
sEp(d)
where p(s) denotes the set of neighboring nodes of s. message from node 8 to d defined as:
m m (xa)
/ Φz (X., Vsa Xa) X
II mu- (X.) d.X, HEP(s)yd
x.
Performing inference of random variables in continuous space causes the integral in Eq. (3) to be- come intractable. This motivates the use of efficient algorithms that approximate the message pass- ing scheme of Eq. (2) and Eq. (3).
3.1 NONPARAMETRIC BELIEF PROPAGATION
Nonparametric belief propagation (NBP) (Sudderth et al., 2003) uses Gaussian mixtures to repre- sent the belief' and messages for continuous random variables. Later works, including Ihler &
3

---

Under review as a conference paper at ICLR 2022
McAllester (2009) and Desingh et al. (2019), further improve upon the tractibility of approximate nonparametric inference by representing beliefs and messages with sets of weighted particles. These particle-based NBP methods infer an approximation of the beliefs using iterative message pass- ing algorithm, in which beliefs and messages are updated at each iteration t. In particular, Desingh al. (2019) avoid the expensive message generation of NBP by approximating Eq. (3) with "pull" strategy. A message, m outgoing from to d, is generated by first sampling M independent samples from bel (Xa) then reweighting and resampling from this set.
4 DIFFERENTIABLE NONPARAMETRIC BELIEF PROPAGATION
We propose a differentiable nonparametric belief propagation (DNBP) method. DNBP maintains a representation of the uncertainty in the estimate by efficiently approximating the marginal posterior distributions encoded in an MRF. Our method avoids the need to define hand-crafted functions for each domain by modeling the potentials needed for the computation of the distributions with neural networks that are trained end-to-end. This hybrid cenerative-discriminati approach leverages the strengths of both NBP and neural networks.
DNBP uses an iterative, differentiable message passing scheme to infer the beliefs over hidden variables in an MRF. DNBP approximates the belief and messages in Eq. (2) and Eq. (3) at iteration t by sets of N and M weighted particles respectively:
(Xa) {18
(
(1)
}
ud i=
(4)
m { {
(5)
DNBP relies on a "pull" message passing strategy similar to the one presented by Desingh et al. (2019). In this strategy, each iteration of the algorithm is defined in terms of a message update step and a belief update step. The message update generates a new set of message particles as a reweighted set of samples from the previous iteration's 's belief. Crucially, the weights associated with these updated message samples result from learned probabilistic factors as opposed to hand- crafted ones. Following a message update, the belief update combines information that is incoming to each node from the newly generated messages. Pseudocode of DNBP's message and belief update schemes is included in Appendix A. The following sections describe the networks used compute the message and belief updates.
Unary Potential Functions: According to the factorization of the MRF joint distribution in Eq. (1), each unobserved variable X,, for d E V. is related to corresponding observed variable Y. by the unary potential function (X(,, Ya). DNBP models each unary function with a feedforward neural network. The unary potential for a particle, Ta. given an observed image, Va. is:
(x Fd, Y. = yu) La (xa @ fa(ya))
(6)
where Sa is a convolutional neural network, la is a fully connected neural network, and the sym- bol @ denotes concatenation of feature vectors. Details of network architectures are given in Ap- pendix A.2, Table 1.
Pairwise Potential Functions: For any pair of hidden variables, X, and Xa. which are connected by an edge in E, a pairwise potential function, Ved(Xa, (a, represents the probabilistic relationship between the two variables. DNBP models each pairwise potential using a pair of feedforward, fully connected neural networks, (ed Xa) (t) (.) The pairwise density network, gie (+), evaluates the unnormalized potential for pair of particles. The pairwise sampling network,
sel (-), is used to form samples of node s conditioned on node d and vice versa. Details of network architectures are given the Appendix A.2, Table 1. The weight computation detailed in the pseudocode in Appendix A.I.
DNBP uses a learned particle diffusion model for each hidden variable, mod- eled as distinct feedforward neural networks, (-) for d E V. This diffusion model replaces the Gaussian diffusion models typically used by particle- bratile-bbsed inference methods. At the outset of mes- are resampled then passed
Particle Diffusion:
sage generation at iteration DNBP's belief particles from iteration through the diffusion model at the beginning of iteration i to form the messages used to update the distributions iteration i.
4

---

Under review as a conference paper at ICLR 2022
Middle Joint
End Exfector
(a)
(b)
(c)
(d)
Figure 2: a) Geometry and example configuration of the double pendulum. b) Graphical model used DNBP for double pendulum task. c) Geometry and an example configuration of the spider structure. d) Graphical model used by DNBP for the spider task.
Particle Resampling: The final operation of the belief update algorithm in NBP is weighted re- sampling of belief particles. This resampling operation is con-differentiable (Karkus et al., 2018; Jonschkowski et al., 201 It follows that the iterative belief update algorithm is non-differentiable due to the resampling step. DNBP addresses the non-diffcrentiabiiiy by relocating the resampling and diffusion operations to the beginning of the message update algo- rithm. With this modification, the belief update returns a weighted set of particles approximating the marginal beliefs. The resulting belief density estimate is differentiable up to the beginning of the message update, when particles from the previous iteration were resampled. The resulting algorithm differentiable through belief update and message passing updates.
the belief update algorithm
4.1 SUPERVISED TRAINING
DNBP's training approach is inspired by the work of Jonschkowski et al. (2018) with modifications to enable learning the potential functions distinct to DNBP. During training, DNBP uses a set of observation sequences, and corresponding set of ground truth sequences. Using the observation sequences, DNBP estimates belief of each unobserved variable at each sequence step. Then, by maximizing estimated belief at the ground truth label of each unobserved variable, DNBP learns its network parameters by maximum likelihood estimation. Further details regarding the implementa- tion of the training procedure are discussed in Appendix A.2.
Given a set of weighted particles representing the belief of Xa produced by the inference procedure at iteration t, the density of the belief can be expressed as mixture of Gaussians, with a component centered at each particle. The density of a sample æa can be computed as follows:
Objective Function:
N
(er)) =
Σ
(1)
N(aas Σ)
w
1=1
DNBP defines loss function one each hidden node d E g as:
L = - log(be(
where x²' denotes the ground truth label for node at sequence step t. The loss for each hidden node computed and optimized separately. At each sequence step during training, DNBP iterates through the nodes of the graph, updating each node's incoming messages and belief followed by single optimization step Eq. (8) using stochastic gradient descent.
RESULTS
The capability of DNBP demonstrated on three challenging articulated tracking tasks. The first two tasks involve visually tracking the articulated joints of simulated articulated structures, illus- trated in Fig. 2. To increase the difficulty of these tasks, simulated clutter! the form of static and
In this work, clutter ratio is defined the ratio of pixels occluded by simulated clutter to the total number of image pixels and is averaged over full sequence of images.
5

---

Under review as a conference paper at ICLR 2022
dynamic geometric shapes are rendered into the image sequences. In the second task, we evaluate DNBP on its ability to track the articulated pose human hands. In both experiments, DNBP is directly compared to learned baseline approaches that are not NBP.
5.1 DATASETS
Simulated Double Pendulum: To characterize DNBP's tracking performance under chaotic mo- tion, the double pendulum task was chosen as an initial evaluation. The double pendulum structure consists of two revolute joints connected to two rigid-body links in series (see Fig. 2a for illus- tration), which are acted on by gravity. The pose of the double pendulum is modeled by the 2- dimensional position of its two revolute joints, rendered as yellow circles, and one end effector. The training set on this task consists of 1024 total sequences with 20 frames per sequence while the vali- dation set consists of 150 total sequences with 20 frames per sequence. Both training and validation sequences are split evenly among three bins of clutter ratio: none, 0 to 0.04 and 0.04 to 0.1. Of the training and validation sequences with any amount of clutter, half contain static clutter and the other half contain dynamic clutter. The held-out test set is evenly split among clutter ratio deciles from O to 0.95, thus contains a shift in distribution from the training set, which was limited to clutter ratios below 0.1. Each decile contains 50 sequences with 100 frames per sequence. For test sequences with any amount of clutter, half contain static clutter and the other half contain dynamic clutter.
Simulated Articulated Spider:
The spider task was chosen to further characterize DNBP's per- formance using structure with added articulations and larger graphical model. As depicted in Fig, 2c, the spider is comprised of three revolute-prismatic joints, three purely revolute joints, and six rigid-body links. An example of the spider shown in Fig. 2c, in which the joints are rendered as yellow circles and the rigid-body links are rendered coloured rectangles. Unlike the double pendulum, which contained a stationary base joint, the spider is not tethered to any position and can move freely throughout the image under simulated joint control. The training, validation and test set for this task follow the same respective distributions of clutter as were used in the double pendulum datasets. The training set consists of 2, 048 total sequences and the validation set consists of 300 sequences. The training and validation sequences are split evenly among five bins of clutter ratio: none, 0 to 0.04 and 0.04 0.1, 0.1 0.2 and 0.2 to 0.3. There are 20 frames per sequence in each of the spider datasets. Both simulated tasks use images of size 128 x 128 pixels. Ground truth keypoint locations are represented continuous valued coordinates scaled to range of [-1,
The FPHAB dataset GGarcia-Hernando et al., 2018) con- sists of RGB-D image sequences taken from the first-person perspective. Thus, the dataset captures the pose and motion of human hands as they perform typical actions. This is a challenging dataset with extreme occlusions where complete observations of all the finger joints are rare. total, there are 1175 distinct sequences and 105459 individual image frames. Each image is labeled with the 3D position of 21 hand joints (illustration of joint relations shown in center column of Fig. 1). The ress-orrforing hand pose estimation baseline proposed by Herniaoo et al. (2018) is used for comparison in the current study. Just like Garcia- Heriindd et al. (2018), DNBP uses only depth observations. To ensure fair comparisons with the FPHAB baseline, this study follows the 1:1 cross-subject training protocol as described in FPHAB.
First-Person Hand Action Benchmark:
5.2 IMPLEMENTATION DETAILS
On all three tasks, Adam (Kingma & Ba, 2015) is used for network optimization with a batch size of 6 and models are trained until convergence of the validation loss. The graphs used by DNBP are shown in Figs. 1, 2b and 2d. While DNBP uses tree-structured graphs in these experiments, the inference strategy is compatible with graphs containing cycles. DNBP is trained using 100 particles per message and tested using 200 partieles per message. During training, one message update performed at each sequence step, while two message updates are used at test time. The pairwise density, pairwise sampling and diffusion sampling processes of DNBP are defined over the relative translations between neighboring nodes. The maximum weighted particle from each marginal belief set of DNBP is used during evaluation for comparison with the ground truth.
On both simulated tasks, DNBP is compared to an LSTM recurrent neural network (Hochreiter & Schmidhuber, 1997). Both models use image inputs that are normalized channel-wise based on training set statistics. The total number of trainable parameters between LSTM and DNBP were

---

Under review as a conference paper at ICLR 2022
50
joinc DNEP: joint DNEP: Endeffector
Avenge Eucclteeam Error
joint LSTM) Endeffector
Average kaatldean (Pixels)
(Pixeli)
10-
0.0
C1
02
3.3
0.1
0.5
04
0.7
9.8
0.9
Clutter Ratio
Clutter Ratio
Figure 3: Average enror of DNBP and LSTM predictions as function of clutter ratio and keypoint type for double pendulum tracking.
Figure 4: Average enror of DNBP and LSTM predictions as function clutter ratio for ar- ticulated 'spider' tracking.
chosen be similar. For hand tracking, the preprocessing protocol of Xiong et al. (2019), is fol- lowed. Notably, preprocessing on the hand tracking task assumes ground truth bounding boxes to ensure fair comparison with the baseline method published by Garcia-Hernande Hernando al. (2018). Sim- ilarly. the feature extractor used by DNBP in the following experiments was designed to emulate the feature extractor of compared baseline. Details of network parameters and inspection of learned relationships are included in the Appendices A.2 and A.6.
5.3 PERFORMANCE METRICS
As quantitative measure of tracking error, average Euclidean error is used. On the simulated tasks, Euclidean eтrоr is averaged over all images in the test set. On the hand tracking task, Euclidean error averaged over all joints per frame then used to calculate percent of frames satisfying variable error thresholds as used by Garcia-Hernande et al. (2018).
Discrete entropy (Shannon, 1948) used as a quantitative measure of uncertainty estimated by DNBP. Discrete entropy is calculated by binning samples from each marginal belief set. For qual- itative analysis of the uncertainty estimated by DNBP, samples from an approximation the joint posterior distribution (i.e. for collection of all unobserved variables) are formed using sequential Monte Carlo sampling approach (Naesseth et al., 2014). Visualization of these samples are formed by plotting rendered link between each pair of keypoint samples.
5.4 DOUBLE PENDULUM TRACKING RESULTS
As shown in Fig. 3. the keypoint tracking error DNBP is directly compared to that of the LSTM baseline the held-out test set for each keypoint type (base, middle and end effector) across the full range of clutter ratios. Results from this comparison show that DNBP's average keypoint tracking error is comparable to the LSTM's corresponding error for both the mid joint and end effector keypoints, independent of clutter ratio. For the base joint keypoint, which is stationary at the center position of every image, the LSTM was able memorize the correct position. DNBP, which diffuses particles based on the message passing scheme, does not memorize the base joint position and registers a consistently larger error which increased with clutter ratio.
DNBP provides measures of uncertainty associated with its predictions, which are generated ac- cording to the algorithmic prior of belief propagation. Next tested was the hypothesis that the DNBP model would generate increased uncertainty under conditions in which an occluding object is placed into the input images such that it covers portions of the double pendulum. This test was performed by rendering occluding block onto test sequence as shown in Fig. 5a-c. Under optimal condi- tions, in which the pendulum is minimally occluded (< 25% by surface area), the model's output indicates low level of uncertainty (see Fig. 5d.f.g.) ) for each keypoint and each frame. In contrast, under conditions in which the pendulum is occluded by the superimposed object, the model's output indicates relatively high levels of uncertainty precisely at frames in which the superimposed object
7

---

Under review as a conference paper at ICLR 2022
Faae
Foae
Frune
Prane
Tonre
Ground
E
Prodictions
LSTM
DNBP Predictions
DNBP Pretiictoon
Marginal Entropy Cha1
Framz
Figure 5: Tracking of double pendulum by DNBP under partial occlusion (orange block). Uncertainty associated with predictions is shown as samples from the joint distribution in pink and blue (d,e.f). (g) Marginal entropy for each keypoint across test sequence; base keypoint (red), middle keypoint (green), end- effector keypoint (blue). Sequence steps high- lighted by gray correspond to images in which > 25% the pendulum is occluded.
Figure 6: Comparison of articulated 'spider' tracking by LSTM (d,e.f) and DNBP (g.h.i) der cluttered conditions. Predicted and ground truth keypoints shown as yellow circles. Clutter shown as faded shapes for illustration to high- light predictions.
occludes a portion (> 25%) of the double pendulum (see Fig. Se.g.). These results demonstrate that the estimate of uncertainty produced by DNBP can identify predictions which unreliable.
5.5 ARTICULATED SPIDER TRACKING RESULTS
After having established the performance characteristics of DNBP on the relatively straightforward double pendulum task, next set determine DNBP's capability for tracking structures. To this end, the 3-arm spider structure was used as a more challenging articulated pose tracking task. Each model's performance was quantitatively assessed on the held-out test of the articulated spider tracking task using the same approach as described for the double pendulum ex- periment by varying clutter ratio (Fig. 4). Similar the results of the double pendulum experiment, average eтror on the spider task increases as a function of clutter ratio for both the LSTM and for DNBP. For clutter ratios between and 0.25, average error for both models remains near 6 pixels then increases consistently with clutter ratio, reaching above 30 pixels of average error for clutter ratios above 0.85. As in the case of the double pendulum experiment, these results demonstrate comparable performance between LSTM and DNBP
complex
articulated pose tracking task.
Next, a qualitative example of tracking performance under conditions of clutter shown in Fig. 6. In Fig. 6(a-c), the ground truth pose is shown amidst distracting shapes across selected frames of test sequence with clutter ratio of 0.25. Pose predictions generated by LSTM are shown in Fig. 6(d- and by DNBP in (g-i). Qualitative assessment of the images indicates both the LSTM and DNBP place their predictions in the correct region the image. Additionally, each model is shown to correctly predict the relative positions of the three arms. Over the sequence, both models track the motion of each keypoint, however appear to struggle with certain keypoint predictions.
5.6 HUMAN HAND TRACKING RESULTS
To evaluate DNBP's capability for application to real-world tasks, the algorithm's 's state estimation and tracking performance was evaluated on the FPHAB dataset. This a challenging dataset with extreme occlusions where complete observations of all the finger joints Firstly, Euclidean
8

---

Under review as a conference paper at ICLR 2022
Ground Truth
Thumb Uncertainty
Index Uncertainty
Middle Uncertainty
Ring Uncertainty
Pinky Uncertainty
Estimate
Figure 7: Output from DNBP on randomly sampled frames. See Appendix A.7 for more examples.
80-
Persentage of Franees
with crrer E
60-
30
Herrando et (Estiaatio)
DARP (Estimation)
DABP TTrackingo
30
60
Error Thresbokd
E
(cmm)
Figure 8: Quantitative comparison between DNBP and neural network baseline on hand pose tracking task of the FPHAB dataset. For cach model the percent of frames with predicted pose less than set threshold is calculated the threshold is varied from Omm 80mm.
error between the estimated and ground truth pose measured for every frame in the test set. For this first evaluation, DNBP is applied frame-by-frane estimator without maintaining its belief over time. The quantitative results from this experiment, are included in Fig. 8 with direct comparison to pure neural network baseline. The results from this experiment indicate that for error thresholds below 50mm, DNBP will consistently have an accuracy of 95% and above.
Following the comparison against a of the baseline, formance would improve when applied tracking method which maintains belief over time. To perform this test, DNBP was applied sequentially to each test sequence and evaluated under the same eгrоr metric. The result from this test, shown in Fig. 8, demonstrates that DNBP does im- prove in terms of frame error when allowed to track its uncertainty over time. Qualitative examples (on frames from randomly chosen sequences) showing DNBP's tracking performance are shown in Fig. 7 and Appendix A.7. The tracking videos showing the DNBP's estimates and belief are in- cluded in the supplementary material and project webpage: gopp:// www/view/difffffffcceeeeooooo nbp.
hypothesized that DNBP's per-
CONCLUSION
In this work, we proposed novel formulation of belief propagation which is differentiable and uses nonparametric representation of belief. It was hypothesized that combining maximum like- lihood estimation with the nonparametric inference approach would enable end-to-end learning of the probabilistic factors needed for inference. Results on both qualitative and quantitative exper- iments demonstrate successful application of this approach and highlight the capability DNBP estimate useful measures of uncertainty, which are crucial for applications where incorrect esti- mates lead catastrophic decisions, such robotics. The current approach limited by its use of non-differentiable resampling and its demand for graph model input. Exploration of methods to overcome these limitations, such by incorporating a soft-resampling strategy Karkus al. (2018), are left future work.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Peter Anderson, Ayush Shrivastava, Devi Parikh, Dhruv Batra, and Stefan Lec. Chasing ghosts: Instruction following as Bayesian state tracking. In Advances in Neural Information Processing Systems, pp. 369-379, 2019.
Greg Brockman, Vicki Cheung. Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016.
Alex Clark. Pillow (pil fork) documentation, 2015.
Karthik Desingh, Shiyang Lu, Anthony Opipari, and Odest Chadwicke Jenkins. Efficient nonpara- metric belief propagation for pose estimation and manipulation articulated objects. Science Robotics, 4(30), 2019. doi: 10.1 /2////cirobotiissaav45522
Trinh Minh Tri Do and Thierry Artières. Neural conditional random fields. In Yee Whye Teh and D. Mike Titterington (eds.), Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010, volume 9 of JMLR Proceedings, pp. 177-184, JMLR.org, 2010.
Guillermo Garcia-Hernande Hernando, Shanxin Yuan, Seungryul Back, and Tae-Kyun Kim. First-person hand action benchmark with rgb-d videos and hand pose annotations. In Proceedings of Computer Vision and Puttern Recognition ( (CVPR), 2018.
S. Godsill. Particle filtering: the first 25 years and beyond. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7760-7764. IEEE, 2019. doi: 10.1109/ICASSP. 8019.8683411.
Tuomas Haarnoja, Anurag Ajay, Sergey Levine, and Pieter Abbeel. Backprop KF: learning discrim- inative deterministic state estimators. In Advances in Neural Information Processing Systems, pp. 4376-4384, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 1735 1780, 1997. doi: 00.1162/neo.11977 1997.9.8.1735.
Alexander Thler and David McAllester. Particle belief propagation. Artificial Intelligence and Statistics, pp. 256-263, 2009.
Michael Isard. PAMPAS: vellevalued graphical models for computer vision. In Conference Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2003.
Rico Jonschkowski and Oliver Brock. End-to-end learnable histogram filters. In Workshop on Deep Learning for Action and Interaction at NeurIPS, December 2016.
Rico Jonschkowski, Divyam Rastogi, and Oliver Brock. Differentiable particle filters: End-to-end learning with algorithmic priors. In Robotics: Science and Systems (RSS), 2018. doi: 10.15607/ RSS.2018.XIV.001..
Péter Karkus, David Hsu, and Wee Sun Lee. Particle filter networks with application to visual localization. In Conference on Robot Learning (CoRL), volume 87, pp. 169-178. PMLR, 2018.
Péter Karkus, Xiao Ma, David Hsu, Leslie Pack Kaelbling, Wee Sun Lee, and Tomás Lozano- Pérez. Differentiable algorithm networks for composable robot learning. In Robotics: Science and Systems, 2019. doi: 20.15677/RSS.2019.XY.0999
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representattons, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Alina Kloss, Georg Martius, and Jeannette Bohg. How to train your differentiable filter. Autonomous Robots, pp. 1-18, 2021.
Daphne Koller and Nir Friedman. Probabilistic Graphical Models Principles and Techniques. MIT Press, 2009. ISBN 978-0-262-01319--2
10

---

Under review as a conference paper at ICLR 2022
Xiangyang Lan, Stefan Roth, Daniel P. Huttenlocher, and Michael J. Black. Efficient belief prop- agation with learned higher-order markov random fields. In European Conference on Computer Vision (ECCV), volume 3952 of Lecture Notes in Computer Science, pp. Springer, 2006. doi: 10. 1744047) 21.
Kuang-chih Lee, Dragomir Anguelov, Baris Sumengen, and Salih Burak Göktürk. Markov random field models for hair and face segmentation. In International Conference on Automatic Face and Gesture Recognition (FG 2008), pp. 1-6. IEEE Computer Society, 2008, doi: 10.1109/AFGR.
481334.
Kevin P. Murphy, Machine learning a probabilistic perspective. Adaptive computation and ma- chine learning series. MIT Press, 2012. ISBN 0262018020.
Kevin P. Murphy, Yair Weiss, and Michael I. Jordan. Loopy belief propagation for approximate inference: An empirical study. Conference on Uncertainty in Artificial Intelligence (UAI), pp- 467-475. Morgan Kaufmann, 1999.
Christian A. Naesseth, Fredrik Lindsten, and Thomas B. Schön. Sequential Monte Carlo for graph- ical models. In Advances in Neural Information Processing Systems, pp. 1862-1870, 2014.
Joseph Ortiz, Talfan Evans, and Andrew J. Davison. A visual introduction to gaussian belief propa- gation. arXiv preprint arXiv:2107.02. 02.00.088.. 2021.
Jana Pavlasek, Stanley Lewis, Karthik Desingh, and Odest Chadwicke Jenkins. Parts-based ticulated object localization in clutter using belief propagation. In International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.
Judea Pearl. Chapter 4 belief updating by network propagation. Judea Pearl (ed.), Probabilistic Reasoming in Intelligent Systems, pp. 143 - 237. Morgan Kaufmann, San Francisco (CA), 1988. ISBN 978-0-08 0489---
doi: https://doii.o http:://doi.org/Iiii wwtp//....////////0//
Wei Ping and Alexander T. Ihler. Belief propagation in conditional rbms for structured prediction. In Aarti Singh and Xiaojin (Jerry) Zhu (eds.). Proceedings of the 20th International Conference on Artificial Intelligence and Statisties, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, volume 54 of Proceedings of Machine Learning Research, pp. 1141-1149, PMLR, 2017.
Claude E. Shannon. A mathematical theory of communication. Bell Syst. Tech. J., 27(3):379-423, 1948. doi: 10.1002/j.1538 7305. 1948 1b01338.x.
Lconid Sigal, Sidharth Bhatia, Stefan Roth, Michael J. Black, and Michael Isard. Tracking loose- limbed people. In Computer Vision and Pattern Recognition (CVPR), pp. 421-428. IEEE Com- puter Society, 2004. doi: 10. 109/CVPR.2004.252
Erik B. Sudderth, Alexander T. Ihler, William T. Freeman, and Alan S. Willsky. Nonparametric belief propagation. In Computer Vision and Pattern Recognition (CVPR), pp. 605-612. IEEE Computer Society, 2003. doi: 10.1 109/CVPR.2003. 121 1409.
Erik B Sudderth, Michael I Mandel, William T Freeman, and Alan S Willsky. Visual hand tracking using nonparametric belief propagation. In IEEE Conference on Computer Vision and Pattern Recognition Workshop ( (CVPRW'OK) 004), pp. 189-189, 2004.
Jian Sun, Nanning Zheng, and Heung- Yeung Shum. Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell., 25(7):787-80) 800, 2003. doi: 10.1109/PA.....0003 100.0.0000..00 1206509.
Sebastian Thrun, Wolfram Burgard, and Dieter Fox. Probabilistic Robotics. MIT Press, 2005. ISBN 978-0-262-20162-9.
Jonathan Tompson, Arjun Jain, Yann LeCun, and Christoph Bregler. Joint training of convolutional network and graphical model for human pose estimation. In Advances in Neural Information Processing Systems, pp. 1799-1807, 2014.
Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Yu Xiang, Dieter Fox, and Stan Birch- field. Deep object pose estimation for semantic robotic grasping of household objects. In Confer- on Robot Learning (CoRI.), 2018.
11

---

Under review as a conference paper at ICLR 2022
Yunbo Wang, Bo Liu, Jiajun Wu, Yuke Zhu, Simon S. Du, Fei-Fei Li, and Joshua B. Tenenbaum. Dualsme: Tunneling differentiable filtering and planning under continuous pomdps. Interna- tional Joint Conference on Artificial Intelligence (IJCAI), pp. 4190-4198. ijcai.org. 2020. doi: 1t19963//caii22220/55.
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. PoseCNN: A convolutional neural network for 6D object pose estimation in cluttered scenes. In Robotics: Science and Systems (RSS). 2018.
Fu Xiong, Boshen Zhang. Yang Xiao, Zhiguo Cao, Taidong Yu, Joey Zhou Tianyi, and Junsong Yuan. A2j: Anchor-to-joint regression network for 3d articulated pose estimation from a single depth image. In International Conference Computer Vision (ICCV), 2019.
Hao Xiong and Nicholas Ruozzi. General purpose MRF learning with neural network potentials. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 2769-2776. ijeai.org, 2020. doi: 10.24963/ijaai 20220//3/ 2020/384.
Qianru Ye, Shanxin Yuan, and Tae- Tae-Kyun Kim. Spatial attention deep net with partial pso for hierar- chical hybrid hand pose estimation. In ECCV, 2016.
Brent Yi, Michelle Lee, Alina Kloss, Roberto Martín-Martín and Jeannette Bohg. Differentiable factor graph optimization for learning smoothers. In International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021.
12

---

Under review as a conference paper at ICLR 2022
A APPENDIX
A.1 ALGORITHM PSEUDOCODE
In this section, pseudo code of the DNBP message passing algorithm is given for reference. As discussed in Section this algorithm differentiable variant of the PMPNBP algorithm Desingh et al. (2019).
Algorithm 1: Message update
input Belief set bel" (Xa) [ww] Md
Incoming messages m" (X₂) {(wae, Mue J==l for each node p(s) \ \d
(i)
output: Outgoing messages, (ors)-d(a)) - ((u w J
Draw (1->"=1) M independent samples from bel (Xa)
(pldd (Xx) (1= 1).M
(0)
Apply particle diffusion to each sampled particle
Pad = 1d Ta(c);
(0)
Draw remaining 7 n- M samples independently from uniform proposal distribution;
foreach (8) do
for 8 [1:11:10 U do
Sample X, ( (
sad X = (9))
(1) Wanary + (X, Xz (6)
Wanary
Y.);
end
mi
umarn.
Wenary
U foreach E p(s) d do
10
Σ will wu
(3))
where w( Ved(X, puod Xy. p1
11
12
end
II W!O:
13
W.
neigh
wEp(e)\d
(i) sd Wanory
neigh?
15
end
16
LM Associate (w Jiml with tu M
(2()
to form outgoing m mXa)
Algorithm 2: Belief update
input Incoming messages, m (Xa) (1 (9)
sd Mdd M.M. for each node E p(d)
output: Belief set bel") (X,) = ((w)) .Pd ,1 )
foreach s E p(d) do
Update message weights
(.)
( x @a(Xa = add Ya) for i [1: M);
Normalize message weights
(i) w sd
i E [1: M];
Σ S
end
Form belief set (Xa) = U m (Xa)(
sEp(d)
Normalize belief weights
(6)
(1)
for E [1 T]:
Σx= Co
13

---

Under review as a conference paper at ICLR 2022
Table 1: Network parameters of learned DNBP potential functions used on both simulated articu- lated tracking tasks. Note s, d E V. and (s, d) E & Unary potentials: (.)) Pairwise potentials: (yed: W.d Particle diffusion; T
NETWORK
UNIT LAYERS
5 [conv(3x3, 10, stride=2, ReLU), maxpool(2x2, 2)]
2 x fc(64, ReI.U), fc(1, Sigmoid scaled to [0.005, 1)) 4 x fe(32, ReL.U,, fe(1, Sigmoid scaled [0.005, 1J)
8/0 V
ad
2 x fo(64, ReLU), fc(2)
Ts
2 x fc(64, ReLU), fc(2)
Results in this work were generated with U set to 10, while past related work (Desingh et al., 2019) used U 1. was observed that this modification improved training stability during preliminary development. Note that is a hyperparameter that controls the resampling strategy and is set to 0.9 in our experiments. y is used only during training: during evaluation, all M samples are drawn from bel. -1 (Xa).
A.2 NETWORK ARCHITECTURE & TRAINING
For both simulated articulated tracking tasks, the network architecture for each sub-network de- scribed in Section 4 is summarized in Table 1. For the hand tracking task, each network follows the same structure as those in Table 1, with two exceptions: (I) the feature extractor, fa(.), used for hand tracking is based on the architecture used by the FPHAB baseline that was introduced by Ye et al. (2016). (2) each node likelihood network, (-), has one additional feature reduction layer of [fc(64, BatchNorm,,ee.U) preceeding the layers of the corresponding network in Table 1.
The following sections describe specific implementation details used in the supervised training of DNBP. To ensure independence from spatial location, the pairwise density, pairwise sampling and diffusion sampling processes of DNBP are defined over the space of transformations between vari- ables. Specifically, each of these networks takes input produces output translation between samples of their corresponding random variables.
The belief weight, w f.(() of particle is proportional to the product of com- f.(s) ponent weights, Wumerya Wumary, x neigh," where s is the neighbor of node d from which particle i originated (see Algorithm 2). Since each of these component weights is produced by a separate potential network (cither da. Wi or e respectively), direct optimization of the belief density will lead to interdependence of the potential network gradients during training. In the context of DNBP, interdependence between different potential functions is inconsistent with the factorization given in g. Tompson et (2014) describe a similar phenomenon they refer to gradient coupling which was addressed by expressing a product of features in log-space which "decouples" the gradients.
Gradient Decoupling:
f.(() t.(1)
To avoid interdependence between potential functions during training, we consider the partial-belief densities which are defined for each node d E V mixtures of Gaussian density functions:
bel d.nnary Vat (Xa)
Σ
1.(0 unarya
NN(a; (0) Σ)
(9)
N
bel danarypts) (Xa)
w
t.(i) anary,
N((x;; (1) E)
(10)
bel b (Xa)
d.neigh
e.( neigh,
NNXx;
M
(6)
(11)
Using these definitions, direct interaction between the potential networks' gradients is avoided by maximizing the product of partial-belics beliefs at the ground truth of each node in log space. The product of partial-beliefs is defined:
(Xa)
bel bel' d. (Xa) x bel unarystas (Xa) x bel dmeighans (Xa)
(12)
14

---

Under review as a conference paper at ICLR 2022
Unary Potentials: During training of DNBP, only those gradients derived from the belief update of each node are used to update the corresponding node's 's unary potential network parameters. Any gradients derived from the outgoing messages of particular node are manually stopped from prop- agating to that node's unary network. This is done to avoid confounding the objective functions of neighboring nodes, which each rely on the others' unary network during message passing. This approach can be implemented with standard deep learning frameworks by dynamically stopping the parameter update of each unary network depending on where in the algorithm its forward pass was registered.
To speed up and stabilize the training of pairwise density potential networks, the following substitution made during training. While calculating wh) for outgoing message i from node s to d, the summation over incoming messages from p(s) to s is replaced by a single evaluation of:
Pairwise Density Networks:
(13) where x- is the ground truth label of sender node 8. This change improves inference time and reduces memory demands by removing a summation over M particles while also providing more stable training feedback to the network. This substitution is removed at test time after training is complete.
w (X, x", Xa Mdd
Pairwise Sampling Networks: The pairwise sampling networks, V. take random sample of Gaussian noise as input and generate conditional samples using the following rule:
e~ 10,
(14)
Fajd d = za + (C((
(15)
where Fald is the sample of variable X, conditioned on neighboring sample za and where is noise vector sampled from zero-mean, unit variance multivariate Gaussian distribution with dimm()) = 64. Similarly, for sampling in the opposite conditioning direction (node d conditioned on s), memory efficiency is gained by reusing the vi network but negating the sampled translation.
A.3 DOUBLE PENDULUM CLUTTER
As summarized in Section 5.1, the double pendulum dataset was generated using a modified ver- sion of the OpenAI Brockman et al. (2016) Acrobot environment. Synthetic geometric shapes are rendered into each image of the dataset to simulate noisy, cluttered environments. All simulated clutter on the double pendulum task is generated according to the following parameters: 50% of clutter is rendered visually beneath the pendulum while the remaining 50% is rendered on top of the pendulum. For dynamic clutter, each geometry simulates motion using a random, constant posi- tion velocity (+t, ý) and orientation velocity (é). Position velocities are sampled from N(00, 0.025). Orientation velocities are sampled from (O, 0.05). Clutter is simulated as either rectangles with 80% probability or circles with 20% probability. Clutter rectangles are sized randomly with length of max(0. [ ~ (0.2, 0.05)) and height of max(0, h (V.0.8, 0.2)). Color of clutter rectangles is randomly chosen with RGB of (0, 204, 204) or (245, 87, 77). Clutter circles are sized randomly with radius of max r ~ (0.1,, 0.1)) and colored randomly with RGB of (204, 204, 0) or (96, 217, 63). Size and color distributions were chosen to ensure clutter visually resembles the double pendulum parts. The position of each clutter geometry was randomly initialized within 1.5x the extent of the image boundary.
The training and validation datasets were distributed evenly among clutter ratios of [0. 0 0.04, and 0.04 - 0.1]. For the training/validation sequences that included clutter, the number of clutters rendered beneath and on top of the double pendulum was individually randomly sampled from independent Binomial distributions using = 15, p = 0.3. To generate the test set. which was uniformly distributed among clutter ratios as described Section 5.1, rejection sampling was used with variable numbers of rendered geometries.
A.4 ARTICULATED SPIDER MODEL.
Data for the articulated spider tracking task of Section 5.5 was simulated using the Pillow Clark (2015) image processing library. Three pevslute-prismatic joints are all centrally located and treated
15

---

Under review as a conference paper at ICLR 2022
as the root of the spider's kinematic tree. The remaining three revolute joints are attached to pairs of links, forming three distinct 'arms' of the spider. Each joint is rendered as a yellow circle while the six rigid-body links are rendered as distinct red, green or blue rectangles respectively. Size param- eters that follow are with respect to rendered image size of 500x500px. The three inner revolute- prismatic joints include rotational constraints limiting each non-overlapping 120" range of artic- ulation as well as prismatic constraints limiting the extension to within [20, 80] pixels of translation. The three purely revolute joints are constrained to rotations between =35 with respect their local origins. Each rigid-bodi -ody link has width of 20px and height of 80px pixels while each joint has radius of 10px.
For every simulated sequence, the spider is initialized with uniformly random root position within the central 1 180x window and uniformly random root orientation from [0, 2m.. Furthermore, each joint state is initialized uniformly at random within its particular articulation constraints. The spider is simulated with dynamics using randomized, constant root, and joint velocities with respect to time step (dt) of 0.01. The root's position velocities (ž, g) are each sampled from equally weighted 2-component Gaussian mixture with means (+24, -24) and standard deviations (15, 15).
Whereas, the root's orientation velocity (0) is sampled from an equally weighted 2-component Gaus- sian mixture with means (+0.3, -0.3) and standard deviations (0.1, 0.1). Each rotational joint's velocity is sampled from an equally weighted 2-component Gaussian mixture with means (+-.33, -0.3) and standard deviations (0.1, 0.1). Similarly, each prismatic joint's velocity is sampled from an equally weighted 2-component Gaussian mixture with means (+500, -500) and standard devia- tions (60, 60). Note that if any joint reaches an articulation limit during simulation, the direction of its velocity is reversed.
A.5 ARTICULATED SPIDER CLUTTER
Clutter generation for the articulated spider tracking task follows similar generation process as was used for the double pendulum task. Clutter parameters that follow are with respect to rendered image size of 500x500px and time step (dt) of 0.01. 50% of clutter is rendered beneath and 50% is rendered on top of the spider. For dynamic clutter, each geometry simulates motion using a random, constant position velocity (d., j) and orientation velocity (0). Position velocities are sampled from NO. 3) while orientation velocities are sampled from (0, 0.05). Clutter is simulated as either a rectangle with 70% probability or circle with 30% probability. Clutter rectangles are sized randomly with length of max(0, 1 ~ (20, 3)) and height of max (0, h ~ NN (80, 5)). The color of clutter rectangles is chosen uniformly at random from the same colors as were used for the spider arms. Clutter circles are sized randomly with a radius of max(0, r ~ (100, 3)) and colored yellow to match the color of the spider's joints. The position of each clutter geometry was randomly initialized within the image boundary.
For the training/validation sequences that included clutter, the number of clutter shapes rendered beneath and on top of the double pendulum was each randomly sampled from independent Binomial distributions using n 10. p 0.5. The test set was generated with uniformly distributed clutter ratios, as described in Section 5. using rejection sampling with variable numbers of rendered geometries.
A.6 LEARNED PAIRWISE INSPECTION
As further validation of DNBP, the learned pairwise potentials are inspected in Fig. 9. The nor- malized histogram of pairwise translations computed from the training set for X, Xo (top) and X2 - -X, (bottom) are shown in the left column of Fig. 9. The middle column shows the normal- ized histogram of samples from learned pairwise sampler networks, VG, (-) and (-), Finally, the right column shows output from the learned pairwise density networks, v/6.1 (.) and (.) gen- erated with 100x100 uniform samples across pairwise translation space. The qualitative similarity between each learned potential model and the corresponding true distribution of pairwise transla- tions indicates that DNBP is successful in learning to model each pairwise potential factor. The circular pairwise relationships are explained by the fact that each pair of double pendulum keypoints is related by revolute joint. The effect of simulated gravity in the double pendulum experiment can be observed by the bias of each pairwise potential in favor of the lower half of each plot as indicated by increased likelihood.
16

---

Under review as a conference paper at ICLR 2022
Traising Set thanbuzion
Leanel Daribuiton
Leaae
shu (X
Figure 9: Inspection of learned pairwise potentials from double pendulum tracking.
Tamiing
(
Figure 10:
Inspection of DNBP's learned pairwise potentials from spider tracking. Only two of the six are shown to avoid redundancy, remaining four show very similar output.
The pairwise potential functions learned by DNBP in the spider tracking task are visualized as was done in the double pendulum task. Fig. 10 shows qualitative output from two of the six models. Only two are shown to avoid redundancy; chosen results are representative of remaining four potential functions. The left column of Fig. 10 shows the normalized histogram of pairwise translations as computed from the training set for X1 - Xo (top) and X4 - X1 (bottom). The middle column of Fig. 10 shows the normalized histogram of samples from learned pairwise sampler networks, (62(.) and (.). Finally in the right column of Fig. 10, uniformly sampled output (100x100 100 samples across pixel space) of the learned pairwise density networks is shown. Once again, the visual similarity between each learned potential function and the corresponding true distribution of pairwise translations factor. Observe that the learned potential functions for (-), which correspond to a revolute articulation, show no bias in favor of the downward configuration. This result notably different from the potential functions learned on the double pendulum task and can be explained by the absence of gravity in the spider simulation. Similarly, the learned models for 1 on the spider task exhibit torus shape due to the effect of prismatic motion associated with the corresponding joint's articulation type and constraint.
indicator that DNBP successful in learning model each pairwise
17

---

Under review as a conference paper at ICLR 2022
A.7
HAND TRACKING RESULTS
Ground Truth
Thumb Uncertainty
Index Uncertainty
Middle Uncertainty
Ring Uncertainty
Pinky Uncertainty
Estimate
(I)
(2)
(5)
(6)
Figure 11: Output from DNBP randomly sampled frames of the hand pose tracking experiment. Visualized
model uncertainty calculated from the marginal belief estimates of DNBP standard deviation in the and y dimensions respectively as calculated by estimated covariance of belief particles. Uncertainty in depth dimension is not visualized.
18


Reviews:
[{"review_id": "vNnfWib0ebH", "reviewer": "Reviewer_Rbn6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths:**\n The paper aims at a differentiable approach for supervised learning and prediction in MRFs with infinite state spaces.\n \n **Weaknesses:**\n It is well known that loopy BP inherently fails in estimating pairwise marginals for MRFs on graphs with cycles. The examples analysed in experiments are MRF on trees. The authors do not clearly state that in such cases there is no need for loopy BP.\n \n The proposed BP approach employs a series of approximations which are not concisely explained. Some of them are given in the supplements only. E.g.:\n  - what is meant by pairwise sampling network? (Sec. 4 and line 6 of the Algorithm in the supplement)\n  - The pull strategy approximation is adopted from Desingh et al. (2019). The quality of this approximation is not evaluated.\n  - Further empirical approximations of the BP approach (which is itself an approximation) are explained in the supplement even though they are part of the proposed approach. Again, their approximation quality is not evaluated. \n\nOverall, I can imagine that such \"approximations of approximations\" can be useful if the main goal is to solve a particular application and they are unavoidable. However, this is not adequate if the goal is to develop a conceptual approach.\n\nThe authors mention the work Xiong & Ruozzi, (2020), where the task of computing unary and pairwise marginals is solved approximately by variational inference and then combined with learning of MRF potentials given by neural networks. This approach has the potential to learn MRFs on graphs with cycles. I would have expected at least an experimental comparison with this method.\n\nFurthermore, it is well known that supervised learning of MRFs with finite state spaces can be approached by using the Pseudo-likelihood estimator instead of the ML estimator. This obviates the need of computing unary and pairwise marginals during learning. I would expect that it is much easier to adapt the PL estimator to the case of infinite state spaces.\n\nThe artificial examples studied in the experiments and seemingly also the human hand tracking use MRFs on trees. This would obviate the need  for loopy BP, because in this case a standard two-pass BP can be used instead. \n\nIt seems that the proposed approach is worse (w.r.t. estimation) than the neural network baseline (Fig. 8)? This holds both, for the frame by frame variant and the tracking variant of the proposed approach? It remains unclear to me how the latter tracking variant works precisely.", "summary_of_the_paper": "Supervised learning of Markov Random Fields (MRF) by MLE requires to compute pairwise and unary marginals of the current model estimate at each iteration of the likelihood maximisation. This task is not tractable except for MRFs on trees with finite hidden state spaces. The authors consider MRFs with infinite state spaces, propose to model the pairwise and unary potentials by neural nets and aim at developing an approximated belief propagation (BP) approach for learning these networks. BP is known to be exact on trees but is not tractable for infinite state spaces. The authors apply their method on a challenging task of hand pose estimation in  RGB-D image sequences taken from the first-person perspective.", "summary_of_the_review": "The proposed approach for learning MRF parameters by combining neural networks with an approximated belief propagation method is not concisely explained and involves (in my view) too many ad-hoc approximations. The experimental results are not convincing enough to compensate the conceptual weaknesses.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cYvB9kJAVSM", "reviewer": "Reviewer_yWQ2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n  - [S1] The proposed method produces meaningful uncertainty estimates,\n    improving state estimation robustness under\n    uncertainty/occlusion.\n  - [S2] The writing is good.\n  - [S3] The paper potentially opens up future avenues of\n    work that build on the proposed framework, improving\n    learning efficiency, tackling a wider range of\n    problems, etc.\n\n\n## Limitations\n\n  - [L1] Limited empirical performance - the proposed\n    approach seems to struggle to outperform a much simpler\n    LSTM baseline. While the proposed DNBP does provide\n    uncertainty estimates which are unavailable with the\n    LSTM, there are many ways to add, train, and calibrate\n    such outputs for NNs as well. However, such baselines\n    are not evaluated.\n  - [L2] The reasons why Hernando et al. outperform the\n    proposed approach are not discussed at all, in spite of\n    the large gap between the two methods.\n  - [L3] Training is not end-to-end, as backpropagation\n    does not occur through the particle resampling stage.\n  - [L4] The comparisons to related papers which may, for\n    example, also use NN factors but perform inference in a\n    different way are a bit too brief in my opinion. For\n    readers who are not experts in graphical models it may\n    be difficult to establish what this paper does\n    differently, and what its strengths and limitations are\n    w.r.t. the rest of the literature.\n\n\n## Suggestions (Per-Section)\n\n### Related Work\n\n  - Could you please elaborate on the differences between\n    the setting of the current paper and that of, e.g.,\n    Xiong & Ruozzi? That paper also seems to learn the deep\n    potentials end to end. I did not read that paper in\n    detail, but is the main idea that they use a\n    variational method rather than belief propagation?\n\n\n### Results\n\n  - Can you please make the text on Fig. 3 a bit bigger,\n    especially the legend? Currently the legend is\n    difficult to read.\n\n", "summary_of_the_paper": "  The paper proposes an extension of particle belief\n  propagation which allows the factor parameters (e.g.,\n  neural network weights for factors modeled as NNs) to be\n  learned using standard stochastic gradient descent.\n\n  This algorithm is then applied to several\n  continuous-domain state estimation tasks involving\n  articulated objects (e.g., hand pose estimation from\n  images).\n\n  The main contribution of the paper is adapting an\n  existing line of nonparametric belief propagation methods\n  to this setting, which allows (partial) end-to-end\n  learning.\n\n  The learning is only partly end-to-end as the particle\n  resampling stage is non-differentiable. Instead, the\n  proposed approach supervises the belief at every step of\n  the algorithm, but does not backpropagate through the\n  entire inference procedure opting to instead maximize the\n  belief of the GT values for the unobserved labels at each\n  step.\n\n  Experimental results on synthetic and real datasets for\n  articulated object state estimation show that the\n  proposed algorithm is able to learn meaningful NNs for\n  the unary and pairwise potentials while also producing\n  reasonable uncertainty estimates. The proposed approach\n  sometimes outperforms the baselines (e.g., an LSTM), but\n  in the task of parametric human hand tracking, it lags\n  behind the state of the art by a large margin.\n\n", "summary_of_the_review": "  While I think that there is definitely a lot of potential\n  in the paper, there are several areas that remain open to\n  improvement. First, the comparisons to related work are a\n  bit unclear to me, and explaining the trade-offs between\n  different kinds of inference methods, especially through\n  the lens of data-driven factor learning, would help\n  position the paper better in this area of ML.\n\n  Second, the empirical evaluation is not very thorough,\n  and its results don't improve much over simple\n  baselines like an LSTM. A broader evaluation, including\n  on tasks where formulating \"hand-engineered\" potentials\n  is more difficult would strengthen the paper in my\n  opinion.\n\n  Because of this I would recommend rejection for now\n  unless the above issues are fully clarified in upcoming\n  communications. That being said, I am likely to be the\n  least experienced of this paper's reviewers when it comes\n  to graphical models, so please take this into\n  consideration when evaluating my reviews (this is\n  addressed to both the authors, as well as to the ACs).\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cFGYMkpITtg", "reviewer": "Reviewer_sUCc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n1. The idea of replacing the handcrafted potential functions with feed-forward neural networks is interesting.\n\n2. The paper is well written and easy to read.\n\n3. The implementation of the gradients is not straightforward, i.e it does not simply consist of building the computational graph and back-propagating through it.\n\n## Weaknesses\n\n1. The main weakness to me, which is also pointed out in the conclusion, is the fact that the graph structure needs to be provided (i.e. it is handcrafted). Moreover, if I am not mistaken, the paper only considers trees (i.e. graphs without loops). These two limitations are (likely) the reason why DNBP is not able to outperform the considered baseline in each application (it is even significantly outperformed by the method Hernando et al. on the hand pose tracking task).\n\n2. In section 4 \"Pairwise potential functions\", it looks like the pairwise sampling network $\\psi_{sd}^{\\sim}$ output is deterministic. Shouldn't it be stochastic (since it is a \"sampling\" network)? Please provide more details about this network.\n\n3. The main benefit of DNBP over classical neural networks, is its ability to associate a measure of uncertainty to its predictions but the three applications that are considered do not accentuate this benefit. I suggest to evaluate DNBP on an application where its ability to associate a measure of uncertainty is very important/useful.", "summary_of_the_paper": "The method (DNBP) proposed in the paper considers a non-parametric belief propagation method where the unary potential functions, the pairwise potential functions and the particle diffusion function are modeled as feed-forward neural networks. It allows them to learn the parameters of these networks using labeled data which is the main contribution of the paper.\n\nDNBP is evaluated on three tasks (simulated double pendulum, simulated articulated spider, first-person hand action). In each application, DNBP is not able to outperform the considered baseline but it is able to provide measures of uncertainty associated with its predictions.\n", "summary_of_the_review": "Currently, the weaknesses outweigh the strengths. Especially the fact that the handcrafted graph structure is probably the reason why DNBP is not able to outperform the considered baselines.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Z-5XheerRcN", "reviewer": "Reviewer_Yvfb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nFor readers who aren't sufficiently knowledgeable on BP techniques, the paper does a good job of going over the basics of BP. \nThe paper is clearly written, easy to understand.\n\nWeaknesses:\nThe main issue I have with the paper is that I find the results of the paper weak:\n1) The chosen baseline is LSTM, and it is trained with an extremely small batch size (6). This doesn't provide enough confidence that the DNBP is any better than a simple neural approach. Quantitative differences are very small, as are the toy datasets. \n2) LSTM is not the natural choice for graphical models. Graph Neural Networks (GNN) would be far more appropriate. In fact, GNNs are shown to beat BP techniques significantly in this uncited paper: Yoon et al. \"Inference in Probabilistic Graphical Models by Graph Neural Networks\". \n3) Why is DNBP not compared against NBP or some other BP method? The paper mentioned above does compare against BP for instance. The paper doesn't give a good reason to choose this method over others.\n4) Qualitatively, hand tracking results are significantly worse than any other recent hand tracking method I know, so it is hard to judge how well the method works. The jitter is extremely high, which would be the opposite of what I'd expect from a method that takes uncertainties into account.\n5) I'd expect the uncertainty estimates to be elongated along the bones for the hand tracking results. For some reason DBNP seems to somehow claim that keypoints can be significantly outside of the hand area. Especially for a depth based method, where background pixels are very far away, this doesn't make any sense. Likewise for the toy dataset results with occlusion, DNBP doesn't seem to do a better job than the simplistic LSTM. \n6) Other Neural Net based methods that can measure uncertainty are not mentioned. For instance, Kumar et al.'s \"LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood\" paper uses Gaussian and Laplacian log-likelhood losses, and their uncertainty estimates align much better with the face features, e.g. elongated along the jawline. Here we don't see the same property. Also that paper has a good methodology to check how informative the uncertainties are. I don't see a similar analysis in this paper.\n\n\n\n\n", "summary_of_the_paper": "This paper enables end-to-end learning of the factors of a graphical model for nonparametric belief propagation (NBP) methods by using neural networks. It calls this method \"Differentiable nonparametric belief propagation\" (DNBP). \n\nThe aim is to replace domain-specific hand crafted factors with learned factors, by replacing each factor with a neural network. Compared to vanilla neural net based solutions, DNBP also reports uncertainty.\n\nThe method is evaluated on a couple of toy examples of articulated pose tracking, as well as using hand pose estimation on the FPHAB dataset.\n\nThe method is compared against learned, neural network based baselines.\n", "summary_of_the_review": "While I'm not an expert on BP techniques and haven't verified the math (hence the low confidence), I've found the paper to be weak on experimental results. The LSTM baselines are very weak, and some important citations are missing. Uncertainty visualizations for hands do not seem sensible, and the uncertainty estimates are not analyzed in depth. There are no comparisons with other BP methods, as well as other uncertainty prediction techniques. Even if the approach seems sound, the results don't justify adopting this challenging-to-implement method over any other regular method.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers concur that the paper has promise, but fails to deliver on that promise.  The idea of learning potentials based on DNNs is appreciated, but the evaluation of the contribution is considered lacking by all reviewers.  In addition, reviewers note that the training is not differentiable, which the rebuttal acknowledges is future work.

I do not reject the paper simply for failing to beat a deep learning baseline, but for having chosen applications which do not even test the paper's hypotheses: reviewers note that the models are tree structured, so loopy BP is not tested, despite the revised paper's claim that "the inference strategy is compatible with graphs containing cycles".
Decision: Reject