Overview of the M ed V id QA 2022 Shared Task on Medical Video Question-Answering (2022)

Abstract: In this paper, we present an overview of the MedVidQA 2022 shared task, collocated with the 21st BioNLP workshop at ACL 2022. The shared task addressed two of the challenges faced by medical video question answering: (I) a video classification task that explores new approaches to medical video understanding (labeling), and (ii) a visual answer localization task. Visual answer localization refers to the identification of the relevant temporal segments (start and end timestamps) in the video where the answer to the medical question is being shown or illustrated. A total of thirteen teams participated in the shared task challenges, with eleven system descriptions submitted to the workshop. The descriptions present monomodal and multi-modal approaches developed for medical video classification and visual answer localization. This paper describes the tasks, the datasets, evaluation metrics, and baseline systems for both tasks. Finally, the paper summarizes the techniques and results of the evaluation of the various approaches explored by the participating teams.

## introduction with the increasing interest in using artificial intelligence (ai) to support clinical decision-making, improving patient engagement, patient health and well-being @xcite , there is a need to explore the efficient algorithms for medical language-video understanding. further, the recent surge in availability of online educational videos on diverse medical and health-related topics demands the development of effective systems that can understand medical videos to provide the best possible answers to consumers' first aid, medical emergency, and medical educational questions. video question answering (vqa) is an emerging and challenging task that requires the understanding of video, language, and their interaction to correctly provide the answer to the question. the majority of the existing studies @xcite @xcite on video question answering are focused on open-domain videos such as movies @xcite , tv shows @xcite , and games @xcite . moreover, the primary objective of the existing vqa studies is to develop a system that can provide natural language answers to the users' questions about the video. some works, such as anne @xcite ; @xcite ; @xcite focus on natural language frame/video localization, but most of them aim to find the video segment that has semantic understanding equivalent to the natural language query. the existing vqa approaches, however, do not take into account the real-world scenarios, where people interact through natural language questions and expect relevant and concise temporal segments from the videos as answers to their questions. consider a health-related question "how can i ease my neck pain?". the textual answer (cf. fig. 1 ) to the given health-related question will be hard to understand and act upon without visual assistance. in order to provide a visual answer to the question, the first step is to identify the most relevant medical video that has a series of steps describing the detailed visual answer to the question. the second and most important step is to locate the relevant temporal segment in the video that is suitable to be a visual answer (cf. fig. 1 ) to the question. towards solving these challenges, we introduced the medvidqa 2022 shared task 1 , which aims to explore and develop efficient algorithms for video question answering that remain understudied in the medical domain. in the first task (medical video classification) of the medvidqa 2022 shared task, participants are asked to develop a system that can categorize the video into medical instructional, medical non-instructional, and non-medical. the figure 1 : an example of a health-related question, textual answer, video containing the answer, and visual answer (temporal segment) from the video. the textual answer (center-left) is retrieved from the web. it contains a series of steps to relieve neck pain by improving neck flexion. the suggested steps in textual answer might be difficult to follow for a consumer who has little or no medical knowledge. the top video (center-right) retrieved from the youtube search contains the answer; however, one has to watch the entire video to find the appropriate temporal segment from the video, which could be served as a visual answer to the question. unlike the textual and video containing the answer, locating the appropriate temporal segment (bottom) which has the visual answer is easy to follow and also eliminates the need to watch the entire video to find the answer. ## conclusion this paper describes the overview of medvidqa 2022 shared task organized as part of the bionlp 2022 workshop. we discussed the tasks, datasets, evaluation metrics, and baseline systems. we also provided a summary of the participating systems for both tasks. for the mvc task, the approach utilizing the attention-based fusion of the pre-trained language model features and video features outperformed all the competitive methods. overall, the mvc task with the coarse-grained category was relatively easy compared to the classical video classification task, where the model has to detect and learn the specific action to classify the video into the fine-grained category. we observe that video subtitles are key information to localize the visual answer in the video for the medical instructional question. we are optimistic that introducing these tasks and datasets will foster research toward designing systems that can understand medical videos and effectively provide visual answers to natural language questions.