Model Agnostic Interpretability for Multiple Instance Learning

1 INTRODUCTION . In Multiple Instance Learning ( MIL ) , data is organised into bags of instances , and each bag is given a single label . This reduces the burden of labelling , as each instance does not have to be assigned a label , making it useful in applications where labelling is expensive , such as healthcare ( Carbonneau et al. , 2018 ) . However , there are often only a few key instances within a bag that determine the bag label , so it is necessary to identify these key instances in order to interpret a model ’ s decision-making ( Liu et al. , 2012 ) . Directly interpreting the decision-making process of machine learning methods is difficult due to the complexity of the models and the scale of the data on which they trained , so there is a need for methods that allow insights into the decision-making processes ( Gilpin et al. , 2018 ) . In MIL problems , only some of the instances in each bag will be discriminatory , i.e. , a significant number of the instances in a bag will be non-discriminatory or even related to other bag classes ( Amores , 2013 ) . Identifying the important instances and presenting those as the interpretation of model decision-making filters out the non-discriminatory information and provides the explainee with a reduced and relevant interpretation , which will avoid overloading the user ( Li et al. , 2015 ) . Our work provides the following novel contributions : 1 . MIL interpretability definition We identify two questions that interpretability methods for MIL should be able to answer : 1 ) Which are the key instances for a bag ? 2 ) What outcome ( class/value ) does each key instance support ? In the rest of this work , we will refer to these two questions as the which and the what questions respectively . As we explore in Section 2 , existing interpretability methods are able to answer which questions with varying degrees of accuracy , but can only answer what questions under certain assumptions . 2 . MIL model-agnostic interpretability methods Existing interpretability methods are often model-specific , i.e. , they can only be applied to certain types of MIL models . To this end , we build upon the state-of-the-art MIL inherent interpretability methods by developing model-agnostic interpretability methods that are up to 30 % more more accurate . The model-agnostic interpretability methods that we propose can be applied to any MIL model , and are able to answer both which and what questions . 3 . MIL interpretability method comparison We compare existing model-specific methods with our model-agnostic methods on several MIL datasets . Our experiments are also carried out on four types of MIL model , giving a comprehensive comparison of interpretability performance . To the best of the authors ’ knowledge , this is one of the first studies to compare different interpretability methods for MIL . The remainder of this work is laid out as follows . Section 2 provides relevant background knowledge and reviews existing MIL interpretability methods . Section 3 outlines the requirements for MIL interpretability and details our approaches that meet these requirements . Next , Section 4 provides our results and experiments . Section 5 discusses our findings , and Section 6 concludes . 2 BACKGROUND AND RELATED WORK . The standard MIL assumption ( SMIL ) is a binary problem with positive and negative bags ( Dietterich et al. , 1997 ; Maron & Lozano-Pérez , 1998 ) . A bag is positive if any of its instances are positive , otherwise it is negative . The assumptions of SMIL can be relaxed to allow more generalised versions of MIL , e.g. , through extensions that include additional positive classes ( Scott et al. , 2005 ; Weidmann et al. , 2003 ) . SMIL also assumes there is no interaction between instances . However , recent work has highlighted that modelling relationships between instances is beneficial for performance ( Tu et al. , 2019 ; Zhou et al. , 2009 ) . Existing methods for interpreting MIL models often rely on the SMIL assumption , so can not generalise to other MIL problems . In addition , existing methods are often model-specific , i.e. , they only work for certain types of MIL models , which constrains the choice of model ( Ribeiro et al. , 2016a ) . Identifying the key instances in MIL bags is a form of local interpretability ( Molnar , 2020 ) , as the key instances are detected for a particular input to the model . Two of the key motivators for interpreting the decision-making process of a MIL system are reliability and trust — identifying the key instances allows an evaluation of the reliability of the system , which increases trust as the decision-making process becomes more transparent . In this work , we use the term interpretability rather than explainability to convey that the analysis remains tied to the models , i.e. , these methods do not provide non-technical explanations in human terms . Under the SMIL assumption , which and what questions are equivalent — there are only two classes ( positive and negative ) , and the only key instances are the instances that are positive , therefore once the key instances are identified , it is also known what outcome they support . However , when there are multiple positive classes , if instances from different classes co-occur in the same bags , answering which and what questions becomes two distinct problems . For example , some key instances will support one positive class , and some key instances will support another . Therefore , solely identifying which are the key instances does not answer the second question of what class they support . Existing methods , such as key instance detection ( Liu et al. , 2012 ) , MIL attention ( Ilse et al. , 2018 ) , and MIL graph neural networks ( GNNs ; Tu et al . ( 2019 ) ) do not condition their output on a particular class , so it is not apparent what class each instance supports , i.e. , they can only answer which questions . One existing method that can answer what questions is mi-Net ( Wang et al. , 2018 ) , as it produces instance-level predictions as part of its processing . However , these instance-level predictions do not take account of interactions between the instances , so are often inaccurate . A related piece of work on MIL interpretability is Tibo et al . ( 2020 ) , which considers interpretability within the scope of multi-multi-instance learning ( MMIL ; Tibo et al . ( 2017 ) ; Fuster et al . ( 2021 ) ) . In MMIL , the instances within a bag are arranged into into further bags , giving a hierarchical bags-of-bags structure . The interpretability techniques presented by Tibo et al . ( 2020 ) are model-specific as they are only designed for MMIL networks . In this work , we aim to overcome the limitations of existing methods by developing model-agnostic methods that can answer both which and what questions . In single instance supervised learning , model-agnostic techniques have been developed to interpret models . Post-hoc local interpretability methods , such as Local Interpretable Model-agnostic Explanations ( LIME ; Ribeiro et al . ( 2016b ) ) and SHapley Additive exPlanations ( SHAP ; Lundberg & Lee ( 2017 ) ) , work by approximating the original predictive model with a locally faithful surrogate model that is inherently interpretable . The surrogate model learns from simplified inputs that represent perturbations of the original input that is being analysed . In this work , one of our proposed methods is a MIL-specific version of this approach . 3 METHODOLOGY . At the start of this section , we outline the requirements for MIL interpretability ( Section 3.1 ) . In Section 3.2 , we propose three methods that meet these requirements under the assumption that there are no interactions between instances ( independent-instance methods ) . In Section 3.3 we remove this assumption and propose our local surrogate model-agnostic interpretability method for MIL . 3.1 MIL INTERPRETABILITY REQUIREMENTS . A general MIL classification problem has C possible classes , with one class being negative and the rest being positive . This is a generalisation of the SMIL assumption , which is a special case when C = 2 . An interpretability method that can only provide the general importance of an instance ( without associating it with a particular class ) can only answer which questions . In order to answer what questions , the method needs to state which classes each instance supports and refutes . Formally , for a bag of instances X = { x1 , . . . , xk } and a bag classification function F , we want to assign a value to each instance that represents whether it supports or refutes a class c : I ( X , F , c ) = { ϕ1 , . . . , ϕk } where I is the interpretability function and ϕi ∈ R is the interpretability value for instance i with respect to class c. Here , we assume that ϕi > 0 means instance i supports class c , ϕi < 0 implies instance i refutes class c , and the greater |ϕi| , the greater the importance of instance i . For some existing methods , such as attention , ϕi is the same for all classes , and ϕi ≥ 0 in all cases , so these requirements are not met . We later demonstrate these limitations in Section 5 . In the next two sections we propose several model-agnostic methods that satisfy these requirements . 3.2 DETERMINING INSTANCE ATTRIBUTIONS . In this section , we propose three model-agnostic methods for interpreting MIL models under the assumption that the instances are independent . This means we can observe the effects of each instance in isolation without worrying about interactions between the instances . We exploit a property of MIL models : the ability to deal with different sized bags . As MIL models are able to process bags of different sizes , it is possible to remove instances from the bags and observe any changes in prediction , allowing us to understand what instances are responsible for the model ’ s prediction . Below , we propose three methods that use this property to interpret a model ’ s decision making . Single Given a bag of instances X = { x1 , . . . , xk } and a bag classification function F , we can take each instance in turn and form a single instance bag : Xi = { xi } for i ∈ { 1 , . . . , k } . We then observe the model ’ s prediction on each single instance bag ϕi = Fc ( Xi ) , where Fc is the output of F for class c ( i.e. , the cth entry in the output vector of F ( Xi ) ) . A large value for ϕi suggests instance xi supports c , and a value close to zero suggest it has no effect with respect to class c. If we repeat this over all instances and all classes , we can build a picture of the classes that each instance supports , allowing us to answer what questions . However , this method can not refute classes ( i.e. , ϕi ≥ 0 in all cases ) . It should be noted that this method gives the same outputs as the inherent interpretability of mi-Net ( Wang et al. , 2018 ) , but here it is a model-agnostic rather than a model-specific method . One Removed A natural counterpart to the Single method is the One Removed method , where each instance is removed from the complete bag in turn , i.e. , we form bags Xi = X \ { xi } . For a particular class c , we can then observe the change in the model ’ s prediction caused by removing xi from the bag : ϕi = Fc ( X ) −Fc ( Xi ) . If the prediction decreases , xi supports c , and if it increases , xi refutes c , i.e. , this method is able to both support and refute different classes . However , if there are other instances in the bag that support or refute class c , we may not observe a change in prediction when xi is removed , even if xi is a key instance . Combined In order to access the benefits of both the Single and One Removed methods , we can combine their outputs . A simple approach is to take the mean , i.e. , ϕi = 12 [ Fc ( { xi } ) + Fc ( X ) − Fc ( X \ { xi } ) ] . This method can identify the important instances revealed by the Single method , and also refute outcomes as revealed by the One Removed method . With these three methods , it is assumed that there are no interactions between the instances . This is not true for all datasets , therefore , in the next section , we remove this assumption and propose a further method that is able to deal with the interactions between instances .

Summary: The manuscript aims at the design of a model-agnostic method of the the interpretability of models and methods addressing multiple instance learning (MIL) problem. Towards this goal, six explanation methods are proposed, three of which are based on kernel-SHAP. The proposed method is validated on the SIVAL, ColoRectal Cancer and a variant of the MNIST-Bags dataset.