This contribution addresses generation of natural language descriptions for human actions, behaviour and their relations with other objects observed in video streams. The work starts with implementation of conventional image processing techniques to extract high level features from video. These features are converted into natural language descriptions using context free grammar. Although feature extraction processes are erroneous at various levels, we explore approaches to putting them together to produce a coherent description. Evaluation is made by calculating ROUGE scores between human annotated and machine generated descriptions. Further we introduce a task based evaluation by human subjects which provides qualitative evaluation of generated descriptions.
Gotoh et al. REF addressed generation of natural language descriptions of human actions, behavior and their relations with other things detected in video streams.
Describing Video Contents in Natural Language
Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data
Computer Science