  We present a new multi-stream 3D mesh reconstruction network (MSMR-Net) for
hand pose estimation from a single RGB image. Our model consists of an image
encoder followed by a mesh-convolution decoder composed of connected graph
convolution layers. In contrast to previous models that form a single mesh
decoding path, our decoder network incorporates multiple cross-resolution
trajectories that are executed in parallel. Thus, global and local information
are shared to form rich decoding representations at minor additional parameter
cost compared to the single trajectory network. We demonstrate the
effectiveness of our method in hand-hand and hand-object interaction scenarios
at various levels of interaction. To evaluate the former scenario, we propose a
method to generate RGB images of closely interacting hands. Moreoever, we
suggest a metric to quantify the degree of interaction and show that close hand
interactions are particularly challenging. Experimental results show that the
MSMR-Net outperforms existing algorithms on the hand-object FreiHAND dataset as
well as on our own hand-hand dataset.
