Deep Multiple Instance Learning with Gaussian Weighting

1 INTRODUCTION . Multiple instance learning ( MIL ) [ 7 ] is a form of weakly supervised learning that have been widely applied in drug activity prediction [ 7 ] , medical imaging [ 14 ] , text analysis and natural language processing [ 2 ] and computer vision [ 18 ] . In contrast to the standard supervised learning where every training instance is provided with a label , in MIL , training instances are grouped in sets , called bags , and labels are provided at bag-level ( i.e . one label for entire bag ) . In the case of binary classification , a bag is labeled as positive , if there is at-least one positive sample , otherwise it is negative . As the instance labels are only indirectly accessible through labels attached to bags , the ambiguity of not knowing which instances are positive makes the problem significantly more challenging than the standard supervised classification . As the ratio of negatives to the positives increases in bags , the ambiguity increases and this renders the problem of bag classification harder . Our hypothesis is that the ability of robustly detecting positive instances in a bag is essential for successful MIL methods . There is a rich body of literature in MIL which can broadly be grouped in two main categories , bag-level [ 17 ] and instance-level approaches [ 1 , 22 ] . The first group maps instances of a bag into a single embedding ( e.g . Fisher Vector [ 17 ] ) and trains bag-level predictors to classify each bag in a similar manner to the standard classification . The second approach learns instance-level classifiers and aggregates the scores of individual instances from each bag . Though the former can be thought to provide a more unbiased solution to the problem of bag classification , the instance-level approach has recently been shown to outperform the bag-level one in [ 16 , 10 ] under the assumption that bag labels can be predicted from individual instances . However , predicting instance labels from baglevel ones is an ill-posed problem . Thus , previous work proposes constrained solutions to tackle the ambiguity . Liu et al . [ 13 ] propose a voting framework to capture neighborhood relationship between instances from different bags to identify key instances . To this end , the authors use the voting framework to obtain a weighted nearest neighbor ( NN ) classifier where its weights globally generated from a kNN graph . Conditional random fields ( CRF ) are used to model relationships between bags in [ 6 ] . This models bags as nodes in a CRF with instances as their states . However , these methods are not learned end-to-end and thus relies on the feature space metric to obtain meaningful weights for each instance . Deep neural networks are also widely applied to solve MIL problem [ 15 , 22 , 16 ] . Zhou et al . [ 22 ] aggregate the outputs of instance level classifier by the max operator to ensure that if at least one instance of a positive training bag is predicted as positive , then the concerned bag is assumed as positive . Similarly , if all instances of a negative bag are predicted as negative then the bag is negative . However , this method does not explicitly model for identifying the key instances in the neural network architecture but rather hope that feature representation of the instances would adapt ( network would learn ) such that instance level classifier can solely learn to give higher scores for positive instances from this loss function . Ramon and De Raedt [ 15 ] introduce log-sum-exp as the convex max to calculate bag probabilities from instance probabilities . Wang et al . [ 16 ] introduce differential MIL pooling layer to enforce MIL constraints such as that each positive bag should have at-least one positive instance . Recently , attention-based deep multiple instance learning was proposed where attention is applied over the features using Softmax attention before obtaining a bag level representation [ 10 ] . We also build on the idea of instance level classification , however , we take a different approach from all these methods in three aspects : First , we propose to take a two stream approach for instance classification and instance weighting ( see fig . 1 ) . Specifically , we have two dedicated paths or streams in our neural network , one dedicated for instance classification by returning a score and the other stream dedicated for learning normalized weights . We show empirically that this is crucial . Second , we do not specifically enforce any MIL constraint ( e.g . number of positives in bag , certain graph structure ) but rather choose a Gaussian activation function that allows us to learn a soft constraint to weight instances by comparing instances within bags and/or across bags . We investigate two variants of Gaussian activated weighting functions that employ bag-level or cross-bag statistics to assign weights to each instance . By comparing instances within a bag or across bags , our Gaussian activated weighting function learns meaningful weights to detect key instances within a bag . It assigns a higher weight for positive instances while assigns a small value for negative instances . In-fact , the instance weighing function assigns lower weights to the instances that are far from the mean and thus encourages positive instances to form compact clusters in the feature space . Enforcing similarity among the features of positive instances through a single mean per class also has a strong regularization on the feature space which implicitly enforces constraints on MIL . Third , we simply extend our binary MIL method to a multi-class bag-classification and our method is robust against severe ambiguity levels ( very few positives in a very large bag ) . We demonstrate that our method obtain state-of-the-art results in the standard MIL instance learning benchmarks . 2 METHOD . Problem definition . In standard classification , it is assumed that we are given a set of training pairs ( x , y ) ∈ Rd × Y with the goal of learning a predictor from inputs to labels f : Rd → Y . In this paper , we focus on the more general problem of MIL and use a weaker form of supervision . Input samples are now organized into bags B1 , B2 , ... , Bm with each bag contains a variable number of samples , i.e . BI = { xi : i ∈ { 1 , · · · , |I| } where |I| is the number of instances in BI . In this setting , individual instances do not have access to a label but each bag BI is associated with a label YI ∈ Y . We consider two settings , binary and multi-class classification . In the binary case Y = { −1 , 1 } , if a bag is negative , all instances in the bag are negative . On the other hand , if a bag is positive , at least one of the instances is positive . In the multi-class case , the output space is categorical , i.e . Y = { 1 , 2 , · · · , C } where C is the number of categories . We learn a classifier g : X |I| → Y that takes in a bag of instances and outputs high score only for the target class : g ( BI ) = ρ ( f ( x1 ) , · · · , f ( x|I| ) ) ( 1 ) where ρ is a symmetric function as in [ 19 , 10 , 4 ] ( i.e . permutation invariant to the ordering of instances x in a bag ) which can be decomposed into predictions for individiual instances f ( x ) . In other words , as we do not have access to ground-truth labels y for individual samples x , instance level predictions f ( x ) are aggregated to obtain bag-level prediction g ( B ) . Examples of the aggregation function ρ are max operator [ 1 ] , weighted sum operator [ 10 ] and log of sum of exponentials [ 15 ] . As a matter of fact , we also build our MIL method on an aggregation function takes in instance level predictions , predicts the importance of each and aggregates them by taking a weighted sum to obtain a bag level prediction . In particular , we extend the idea present in eq . ( 1 ) to weight each instance using a neural network specifically designed to select key instances [ 13 ] i.e . discriminative instances that can reliably be utilized to predict bag labels . Formally we define each key instance to be positive below : Definition . Given a bag of multiple instances , if the bag label is positive , the key instances are the instances having positive label . Property.Given a bag of multiple instances , if the bag is positive , key instances are fewer than nonkey instances . A diagram of our model is depicted in fig . 1 which consists of two streams , each stream contains multiple components . We describe them in the following . Instance classifier f . First we define the instance level prediction f ( x ) = ψθψ ◦ φθφ as a composition of a feature extractor φ : X → H where H is latent variable space and of a classifier ψ where ψ ( φ ( x ; θφ ) ; θψ ) ∈ RC and C is the number of target categories . In fact , both φ and ψ are modeled as deep neural networks with learnable parameters θφ and θψ respectively . Instance weighting β . The bag classifier g ( BI ) takes in the instance scores f ( x1 ) , · · · , f ( x|I| ) for BI and outputs a weighted sum of them by using β ( · ; θβ ) : X → [ 0 , 1 ] C where θβ are the parameters of the mapping β . Equation ( 1 ) can be rewritten as follows g ( BI ) = ρ ( [ β ( x1 ; θβ ) · f ( x1 ) ] , · · · , [ β ( x|I| ; θβ ) · f ( x|I| ) ] ) ( 2 ) where ρ is an aggregating operator ( or the pooling function ) and θβ are the learnable parameters of β . The weighting function β = σ ◦ ϕ ◦ φ is also modeled as a composition of three functions . β shares the feature extractor φ with the instance classifier f , however , applies consecutively ϕ ( · ; θϕ ) ( drawn in green in fig . 1 ) on the features which outputs an C dimensional unnormalized vector and followed by a univariate Gaussian radial basis normalization σ ( drawn in blue in fig . 1 ) defined as follows : σ ( r ; µ , ) = exp ( −‖r − µ ‖2 ) ( 3 ) where µ and are the mean and standard deviation respectively . We propose two ways of calculating µ and . In the first case , we fit µ and to the scores that are extracted over BI , i.e . { ϕ ( φ ( x1 ) ) , · · · , ϕ ( φ ( x|I| ) ) } . µ and are simply statistics and can be calculated by the following formulas : µ = E { ∀xi∈BI } [ ϕ ( φ ( xi ) ) ] 2 = E { ∀xi∈BI } [ ‖ϕ ( φ ( xi ) ) − µ‖ 2 ] ( 4 ) In other words , weights can be normalized by using the local bag statistics . These variants are denoted by GP0T0 and GP0T1 as explained later in the experiments . Here the intuition is that by comparing each instance with local-bag statistics , our weight learning function is able to learn to detect key instances . In the second one , we set µ and as parameters of our network and jointly optimize them with the rest of the network parameters ( θφ , θψ , θϕ ) . In this case , the parameters are not local to a bag but global , learned from the updates from all the training samples and do not necessarily correspond to a statistical mean and standard deviation of the score distribution anymore . This variant is denoted by GP1T1 . Training . In this paper we focus on two kinds of MIL tasks , binary and multi-class classification . In the binary case , both the instance classifier f and weighting function β output a scalar per instance , their outputs are simply multiplied ( see eq . ( 2 ) ) . Afterwards , log-sum-exponent [ 15 ] is applied over all the instances as the aggregation function ρ and finally , fed into a sigmoid function to get the probability of being positive for a bag . We use the binary cross entropy loss to train this model . In the multi-class case , both the instance classifier f and weighting function β output a C dimensional vector , they are element-wise multiplied , then averaged over the bag and finally normalized by a softmax function over classes . Here we use the cross entropy loss to optimize the model . We train all network parameters jointly by using stochastic gradient descent or Adam . The implementation details are given in section 3 .

Summary: The paper proposes a new approach to weighting in multiple-instance learning scenario. They multiply scores for instances by Gaussian normalized weights. The hyperparameters of the Gaussian RBF are either estimated locally (i.e., within a bag) or trained across bags. Additionally, the authors verify whether it is better to have two separate neural networks (one for instance scoring and one for instance weighting) or share weights between them. Eventually, a variant with globally trained hyperparameters of the Gaussian RBF and separate weights of neural networks performs the best.