  The use of iterative pose refinement is a critical processing step for 6D
object pose estimation, and its performance depends greatly on one's choice of
image representation. Image representations learned via deep convolutional
neural networks (CNN) are currently the method of choice as they are able to
robustly encode object keypoint locations. However, CNN-based image
representations are computational expensive to use for iterative pose
refinement, as they require that image features are extracted using a deep
network, once for the input image and multiple times for rendered images during
the refinement process. Instead of using a CNN to extract image features from a
rendered RGB image, we propose to directly render a deep feature image. We call
this deep texture rendering, where a shallow multi-layer perceptron is used to
directly regress a view invariant image representation of an object. Using an
estimate of the pose and deep texture rendering, our system can render an image
representation in under 1ms. This image representation is optimized such that
it makes it easier to perform nonlinear 6D pose estimation by adding a
differentiable Levenberg-Marquardt optimization network and back-propagating
the 6D pose alignment error. We call our method, RePOSE, a Real-time Iterative
Rendering and Refinement algorithm for 6D POSE estimation. RePOSE runs at 71
FPS and achieves state-of-the-art accuracy of 51.6% on the Occlusion LineMOD
dataset - a 4.1% absolute improvement over the prior art, and comparable
performance on the YCB-Video dataset with a much faster runtime than the other
pose refinement methods.
