We propose an algorithm for learning the semantics of a (motion) verb from videos depicting the action expressed by the verb, paired with sentences describing the action participants and their roles. Acknowledging that commonalities among example videos may not exist at the level of the input features, our approximation algorithm efficiently searches the space of more abstract features for a common solution. We test our algorithm by using it to learn the semantics of a sample set of verbs; results demonstrate the usefulness of the proposed framework, while identifying directions for further improvement.
An interesting in-depth study by REF automatically learnt the semantics of motion verbs as abstract features from videos.
Learning the abstract motion semantics of verbs from captioned videos
2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Computer Science