  Human actions involving hand manipulations are structured according to the
making and breaking of hand-object contact, and human visual understanding of
action is reliant on anticipation of contact as is demonstrated by pioneering
work in cognitive science. Taking inspiration from this, we introduce
representations and models centered on contact, which we then use in action
prediction and anticipation. We annotate a subset of the EPIC Kitchens dataset
to include time-to-contact between hands and objects, as well as segmentations
of hands and objects. Using these annotations we train the Anticipation Module,
a module producing Contact Anticipation Maps and Next Active Object
Segmentations - novel low-level representations providing temporal and spatial
characteristics of anticipated near future action. On top of the Anticipation
Module we apply Egocentric Object Manipulation Graphs (Ego-OMG), a framework
for action anticipation and prediction. Ego-OMG models longer term temporal
semantic relations through the use of a graph modeling transitions between
contact delineated action states. Use of the Anticipation Module within Ego-OMG
produces state-of-the-art results, achieving 1st and 2nd place on the unseen
and seen test sets, respectively, of the EPIC Kitchens Action Anticipation
Challenge, and achieving state-of-the-art results on the tasks of action
anticipation and action prediction over EPIC Kitchens. We perform ablation
studies over characteristics of the Anticipation Module to evaluate their
utility.
