  We present a differentiable approach to learn the probabilistic factors used
for inference by a nonparametric belief propagation algorithm. Existing
nonparametric belief propagation methods rely on domain-specific features
encoded in the probabilistic factors of a graphical model. In this work, we
replace each crafted factor with a differentiable neural network enabling the
factors to be learned using an efficient optimization routine from labeled
data. By combining differentiable neural networks with an efficient belief
propagation algorithm, our method learns to maintain a set of marginal
posterior samples using end-to-end training. We evaluate our differentiable
nonparametric belief propagation (DNBP) method on a set of articulated pose
tracking tasks and compare performance with a recurrent neural network. Results
from this comparison demonstrate the effectiveness of using learned factors for
tracking and suggest the practical advantage over hand-crafted approaches. The
project webpage is available at: progress.eecs.umich.edu/projects/dnbp.
