Learning to Estimate Pose and Shape of HandHeld Objects from RGB Images We develop a system for modeling handobject interactions in 3D from RGB images that show a hand which is holding a novel object from a known category. We design a Convolutional Neural Network (CNN) for Handheld Object Pose and Shape estimation called HOPSNet and utilize prior work to estimate the hand pose and configuration. We leverage the insight that information about the hand facilitates object pose and shape estimation by incorporating the hand into both training and inference of the object pose and shape as well as the refinement of the estimated pose. The network is trained on a large synthetic dataset of objects in interaction with a human hand. To bridge the gap between real and synthetic images, we employ an imagetoimage translation model (Augmented CycleGAN) that generates realistically textured objects given a synthetic rendering. This provides a scalable way of generating annotated data for training HOPSNet. Our quantitative experiments show that even noisy hand parameters significantly help object pose and shape estimation. The qualitative experiments show results of pose and shape estimation of objects held by a hand xe2x80x9cin the wildxe2x80x9d.

Generating Grasp Poses for a HighDOF Gripper Using Neural Networks We present a learningbased method for representing grasp poses of a highDOF hand using neural networks. Due to redundancy in such highDOF grippers, there exists a large number of equally effective grasp poses for a given target object, making it difficult for the neural network to find consistent grasp poses. We resolve this ambiguity by generating an augmented dataset that covers many possible grasps for each target object and train our neural networks using a consistency loss function to identify a onetoone mapping from objects to grasp poses. We further enhance the quality of neuralnetworkpredicted grasp poses using a collision loss function to avoid penetrations. We use an object dataset that combines the BigBIRD Database, the KIT Database, the YCB Database, and the Grasp Dataset to show that our method can generate highDOF grasp poses with higher accuracy than supervised learning baselines. The quality of the grasp poses is on par with the groundtruth poses in the dataset. In addition, our method is robust and can handle noisy object models such as those constructed from multiview depth images, allowing our method to be implemented on a 25DOF Shadow Hand hardware platform.