Automatic Concept Extraction for Concept Bottleneck-based Video Classification

1 INTRODUCTION . Deep neural networks ( DNNs ) provide unparalleled performance when applied to application domains , including video classification and activity recognition . However , the inherent black-box nature of the DNNs inhibits the ability to explain the output decisions of a model . While opaque decision-making may be sufficient for certain tasks , several critical and sensitive applications force model developers to face a dilemma between selecting the best-performing solution or one that is inherently explainable . For example , in the healthcare domain ( Yeung et al . ( 2019 ) ) , a life-or-death diagnosis compels the use of the best performing model , yet accepting an automated prediction without justification is wholly insufficient . Ideally , one could take advantage of the power of deep learning while still providing a sufficient understanding of why a model is making a particular decision , especially if the situation demands trust in a decision that can have severe impacts . To address the need for model interpretability , researchers have sought to enable model intervention by leveraging concept bottleneck-based explanations . Unlike post hoc explanation methods–where techniques are used to extract an explanation for a given input for an inference by a trained black-box model ( Chakraborty et al . ( 2017 ) ; Jeyakumar et al . ( 2020 ) ) , concept bottleneck models are inherently interpretable and take a human reasoning-inspired approach to explaining a model inference based on an underlying set of concepts that define the decisions within an application . Thus far , prior works have focused on concept-based explanation models for image ( Kumar et al . ( 2009 ) ; Koh et al . ( 2020 ) ) and text classification ( Murty et al . ( 2020 ) ) . However , the concepts are assumed to be given a priori by a domain expert–a process that may not result in a necessary and sufficient set of concepts . For instance , for bird species identification , an expert may provide two redundant concepts that are possibly correlated , such as wing color and beak color . More critically , prior works have considered simple concepts with the same level of abstraction , e.g. , visual elements present in a single image . For more complex tasks such as video activity classification , a label may span multiple frames . Thus , the composing set of concepts will have various levels of abstraction representing relationships of various visual elements spanning multiple frames , e.g. , a bird flapping its wings . Unlike the prior works , we aim to exploit the complex abstractions inherent in natural language explanations to conceptualize such complex events . Research Questions . In summary , this paper seeks to answer the following research questions : • How can a machine automatically elicit the inherent complex concepts from natural language to construct a necessary and sufficient set of concepts for video classification tasks ? • Given that a machine can extract such concepts , are they informative and meaningful enough to be detected in videos by DNNs for downstream prediction tasks ? • Are the machine extracted concepts perceived by humans as good explanations for the correct classifications ? Approach . This paper introduces an automatic concept extraction module for concept bottleneckbased video classification . The bottleneck architecture equips a standard video classification model with an intermediate concept prediction layer that identifies concepts spanning multiple video frames . To compose the concepts that will be predicted by the model , we propose a natural language processing ( NLP ) based automatic Concept Discovery and Extraction module , CODEX , to extract a rich set of concepts from natural language explanations of a video classification . NLP tools are leveraged to elicit inherent complex concept abstractions in natural language . CODEX identifies and groups short textual fragments relating to events , thereby capturing the complex concepts from videos . Thus , we amortize the effort required to define and label the necessary and sufficient set of concepts . Moreover , we employ an attention mechanism to highlight and quantify which concepts are most important for a given decision . To demonstrate the efficacy of our approach , we construct two new datasets–MLB V2E ( Video to Explanations ) for baseball activity classification and MSR-V2E for video category classification– that combine complex video classification datasets with short , crowd-sourced natural language explanations for their corresponding labels . We first compare our model against the existing standard end-to-end deep-learning methods for video classification and show that our architecture provides additional benefits of an inherently interpretable model with a marginal impact on performance ( less than 0.3 % accuracy loss on classification tasks ) . A subsequent user study showed that the extracted concepts were perceived by humans as good explanations for the classification on both the MLB-V2E and MSR-V2E datasets . Contributions . We summarize our contributions as follows . • We propose CoDEx , a concept discovery and extraction module that leverages NLP techniques to automatically extract complex concept abstractions from crowd-sourced , natural language explanations for a given video and label–obviating the need to manually define a necessary and sufficient set of concepts . • We evaluate our approach on complex video classification datasets and show that our model attains high concept accuracies while maintaining competitive task performance with standard end-to-end video classification models . • We also augment the concept-based explanation architecture to include an attention mechanism that highlights the importance of each concept for a given decision . We show that users prefer our concept extraction method over baseline methods to explain a given label . • We construct two new public datasets , MLB-V2E and MSR-V2E , that combine complex video classification datasets with short , crowd-sourced natural language explanations and labels . 2 RELATED WORK . There is a wide array of works in explainable deep learning for various applications . This work focuses on the concepts-based explanations for video classification , and this section provides an overview of the existing literature for overlapping domains . Concept-Based Explanations for Images and Text . A number of existing works consider conceptbottleneck architectures where models are trained to interact with high-level concepts . Generally , the approaches are multi-task architectures , where the model first identifies a human-understandable set of concepts and then reasons about the identified concepts . Until now , the applications have been limited to static image and text applications . Koh et al . ( 2020 ) used pre-labeled concepts provided by the dataset to train a model that predicts the concepts , which is then used to predict the final classification . However , the caveat is that the concepts had to be manually provided . Ghorbani et al . ( 2019 ) and Yeh et al . ( 2020 ) proposed approaches that automatically extract groups of pixels from the input image that represent meaningful concepts for the prediction . They were designed largely for image classification and extract concepts directly from the dataset . Kim et al . ( 2018 ) propose a post-hoc explanation method that returns the importance of user-defined concepts for a classification . In the mentioned works , the concepts have been limited to simple concepts and are not suited for complex tasks such as video classification where we have complex concepts that may span multiple frames with various levels of abstraction . Explanations for Video Classification Other approaches have been considered to explain video classification and activity recognition . Chattopadhay et al . ( 2018 ) applied GradCAM and GradCAM++ to video classification , where for each frame , the important region of the frame to the model is highlighted as a heatmap . Hiley et al . ( 2020 ) extract both spatial and temporal explanations from input videos by highlighting the relevant pixels . However , these are post-hoc techniques that focus on explaining black-box models , whereas our approach enables concept-bottleneck methods for video classification that are intended to be inherently interpretable and intervenable . Video Captioning . In recent years , there is a large number of works ( Pan et al . ( 2017 ) ; Gao et al . ( 2017 ) ; Wang et al . ( 2018 ) ; Yan et al . ( 2019 ) ; Zhou et al . ( 2018 ) ; Chen & Jiang ( 2021 ) ; Yu et al . ( 2017 ) ) on video captioning . While they also employ natural language techniques , these works are tangential to generating text explanations for classifications , since they are merely describing the video . Our model provides an explanation justifying the classification of the video . Similarly , the associated datasets such as MSR-VTT ( Xu et al . ( 2016 ) ) only have videos with ground truth captions that only describe the video without the context of a classification–which often results in concepts that do not pertain to a classification . Semantic Concept Video Classification . The closest works to this paper is the body of work in semantic concept video classification ( Fan et al . ( 2004 ; 2007 ) ) , where the concepts are defined as salient objects that are visually distinguishable video components . The concepts in these works are simple objects detected in the videos and are not complex enough to capture the semantics of events that happen over multiple frames of the videos . These works typically used traditional SVM-based video classifiers . Assari et al . ( 2014 ) represent a video category based on the co-occurrences of the semantic concepts and classify based on the co-occurrences , but their method requires a predefined set of concepts . Thus , we now present the methodology behind our automatic concept extraction for concept bottleneck video classification . 3 CONCEPT DISCOVERY AND BOTTLENECK VIDEO CLASSIFICATION . This work introduces CoDEx , an automatic concept extraction method from natural language explanations for concept-based video classification . Figure 1 depicts the overall concept-bottleneck pipeline , composed of CoDEx and the concept bottleneck model . CoDEx extracts a set of concepts from natural language explanations that will comprise the bottleneck layer for the video classification model . We first formalize the overall problem and then provide the methodology for both modules . Problem Formalization . We assume that we have a training dataset { ( xn , ln ) } Nn=1 = D of videos xn with a label ln ∈ L , where L is a predefined set of possible class labels for the video . Each video is represented as a sequence of frames f ∈ F where F is the set of video frames . Thus video xn = 〈fn0 , fn1 , . . . , fnT 〉 , where fnt represents frame t of video n. For each video xn , we form a label-explanation pair ( ln , en ) , where en is a ( short ) natural language document explaining the given label ln . If multiple annotators contribute to an explanation for video-label pair , ( xn , ln ) , then these are concatenated to form en . The full set of pairs E = { ( ln , en ) } Nn=1 is the explanation corpus . Thus , the design goals are : • Concept Discovery and Extraction ( CODEX ) Module : Given the explanation corpus , first produce an N × K concept matrix , C , where the ( n , k ) th element is 1 if the nth explanation contains discovered concept k and 0 otherwise . We call the nth row cn , the concept vector for video xn . K is the total number of discovered concepts . • Concept Bottleneck Model : Given a concept matrix , C , the second goal is to train a concept bottleneck model such that for a given video xi , we predict a concept vector ci– which indicates the presence or absence of concepts and their importance scores . The model then makes use of ci to make the final video classification . label , ln explanation , en strike The batter did not swing . The ball was in the strike zone . foul The batter hit the ball into the stands and it landed in foul territory . ball The hitter didn ’ t swing . The ball was outside the strike zone . none The video did not load . out the batter hit the ball and it was caught by the fielder label , ln concepts , cn strike { the batter did not swing , the ball was in the strike zone } foul { the batter hit the ball , it landed in foul territory } ball { the batter did not swing , the ball was outside the strike zone } out { the batter hit the ball , it was caught by the fielder } Grouped conceptsCompleted concepts Text Pruned conceptsText Removed text Explanation Corpus Ɛ Extracted Concepts Figure 2 : Running example for all six stages of the discovery pipeline module . The left table is the explanation corpus , with highlighted fragments to be modified . The right table contains the discovered concepts . The detailed step-by-step modifications are provided in Appendix A.1 .

Summary: This paper works on the interpretability of video understanding problem. With a set of textual descriptions, the authors propose a pipeline called CoDex to extract the key concepts for explaining the classification, in contrast to previous methods which use the predefined classes. The CoDex method contains clearning, extraction, completion, grouping, pruning and vectorization phases to obtain the final concept matrix. The trained final concept bottleneck model can obtain interpretable explanations aligned with human servey, while retaining the performance