Describing video scenarios using deep learning techniques

International Journal of Intelligent SystemsVolume 36, Issue 6 p. 2465-2490 RESEARCH ARTICLE Describing video scenarios using deep learning techniques Yin-Fu Huang, Corresponding Author Yin-Fu Huang [email protected] orcid.org/0000-0001-6665-0135 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, Taiwan Correspondence Yin-Fu Huang, Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, 123 University Rd., Section 3, 640 Touliu, Yunlin, Taiwan. Email: [email protected]Search for more papers by this authorLi-Ping Shih, Li-Ping Shih orcid.org/0000-0002-2933-3306 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, TaiwanSearch for more papers by this authorChia-Hsin Tsai, Chia-Hsin Tsai orcid.org/0000-0001-6694-4387 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, TaiwanSearch for more papers by this authorGuan-Ting Shen, Guan-Ting Shen orcid.org/0000-0001-5698-2136 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, TaiwanSearch for more papers by this author Yin-Fu Huang, Corresponding Author Yin-Fu Huang [email protected] orcid.org/0000-0001-6665-0135 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, Taiwan Correspondence Yin-Fu Huang, Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, 123 University Rd., Section 3, 640 Touliu, Yunlin, Taiwan. Email: [email protected]Search for more papers by this authorLi-Ping Shih, Li-Ping Shih orcid.org/0000-0002-2933-3306 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, TaiwanSearch for more papers by this authorChia-Hsin Tsai, Chia-Hsin Tsai orcid.org/0000-0001-6694-4387 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, TaiwanSearch for more papers by this authorGuan-Ting Shen, Guan-Ting Shen orcid.org/0000-0001-5698-2136 Department of Computer Science and Information Engineering, National Yunlin University of Science and Technology, Touliu, Yunlin, TaiwanSearch for more papers by this author First published: 25 February 2021 https://doi.org/10.1002/int.22387Citations: 2Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Abstract The combination of computer vision and natural language processing is still a very challenging issue. In contrast to previous models focusing on generating only a single sentence for a video, we think that describing a longer video is an important application. In this paper, we propose a video scenario description system that considers video genres to generate multiple sentences. First, the semantics and genres of videos are analyzed. Next, video descriptions are also analyzed. Then, relevant semantic features are selected and translated into the corresponding video descriptions through deep learning. In the experiments, we compare the generated video descriptions based on four evaluation metrics. The results reveal our method is comparable with the state-of-the-art methods. CONFLICT OF INTERESTS The author declares that there is no conflict of interests. Citing Literature Volume36, Issue6June 2021Pages 2465-2490 RelatedInformation