Partially Mutual Exclusive Softmax for Positive and Unlabeled data

In recent years softmax , together with its fast approximations , has become the defacto loss function for deep neural networks with multiclass predictions . However , softmax is used in many problems that do not fully fit the multiclass framework and where the softmax assumption of mutually exclusive outcomes can lead to biased results . This is often the case for applications such as language modeling , nextevent prediction , and matrix factorization , where many of the potential outcomes are not mutually exclusive , but are more likely to be conditionally independent given the state . To this end , for the set of problems with positive and unlabeled data , we propose Partially Mutual Exclusive Softmax ( PMES ) , a relaxation of the original softmax formulation , where , given the observed state , each of the outcomes are conditionally independent but share a common set of negatives . Since we operate in a regime where explicit negatives are missing , we create a cooperatively-trained model of negatives , and derive a new negative sampling and weighting scheme which we call Cooperative Importance Sampling ( CIS ) . We show empirically the advantages of our newly introduced negative sampling scheme by plugging it into the Word2Vec algorithm and evaluating it extensively against other negative sampling schemes on both language modeling and matrix factorization tasks , where we show large lifts in performance . 1 INTRODUCTION . Learning from positive and unlabeled data is a well-defined task in machine learning , known as positive-unlabeled ( PU ) learning ( Elkan & Noto ( 2008 ) ; Li & Liu ( 2003 ) ; Lee & Liu ( 2003 ) ; Ren et al . ( 2014 ) ; Paquet & Koenigstein ( 2013 ) ) . The applications of PU learning are numerous , as in many fields negative data is either too expensive to obtain or too hard to define . This is the case in language modeling , where one only observes examples of valid sentences and documents , and tries to learn a generative process . Similarly , in computer vision , with the recent work on Generative Adversarial Networks ( GANs ) , one tries to learn the underlying generative process that produces meaningful images . In both cases , the space of negatives ( e.g. , non-sentences or non-images ) is not clear . Similarly , in matrix factorization , most of the matrices contain pairs of observed interactions , and there are no available explicit negatives . In all of the applications enumerated above , it is quite common to encounter solutions that are based on a softmax loss function . In language modeling , and more recently in matrix factorization , the Word2Vec algorithm ( see Mikolov et al . ( 2013a ) ) models the conditional probability of observing all possible items in the vicinity of a context item as a categorical distribution using the softmax loss . However , modeling the probability of co-occurrence as a categorical distribution is a very biased assumption that is clearly refuted by the data , since for all context words there are multiple words that co-occur at the same time . In our paper we propose Partially Mutual Exclusive Softmax ( PMES ) , a new model that relaxes the mutual exclusivity constraint over the outcomes . PMES relaxes this constraint by splitting the set of all outcomes into a set of possible outcomes that are conditionally independent given the context , and a set of impossible outcomes which become negative examples . The context-dependent negative set is hypothesized but not known , so in our method we introduce a model for negatives that is used to weight the sampled candidate negatives . The training algorithm is based on the simultaneous training of two neural networks . The first network is a generator that fits the positives and the sampled negatives . The second network is the discriminator , which is trained to separate the true positive pairs from generated pairs , and is used as the model of the probability that an example would receive a negative label . The resulting solution has many similarities with other recent negative sampling methods for approximating full softmax . However , unlike most of the previous methods , our method is not trying to faithfully approximate the full softmax formulation , but to fix some of its over-simplifying assumptions . Furthermore , we believe that the observed lift in performance of some of the negative sampling work over the full softmax can be explained through the prism of Partially Mutual Exclusive Softmax . Our hypothesis is further confirmed by experiments on language modeling and matrix factorization , where we show a big lift in performance over previous work on negative sampling and full softmax . We validate some of our intuitions on the advantages of our sampling procedure on an artificial dataset where the support sets are known , and show that our sampling scheme correctly approximates the support , which is not the case for other softmax variants . Overall , the main contributions of this paper are the following : • We propose Partially Mutual Exclusive ( PME ) Softmax , a modified version of the softmax loss that is a better fit for problems with positive and unlabeled data . • We derive a new negative sampling scheme based on an cooperatively-trained models of negatives which we denote as Cooperative Importance Sampling ( CIS ) • We show empirically the validity of our proposed approach by plugging our new loss into the Word2Vec model , and evaluating this enhanced Word2Vec model against classical sampling schemes for Word2Vec on language modeling and matrix factorization tasks across six real-world datasets . We discuss related work on Word2Vec , negative sampling schemes for softmax and GANs in Section 2 of this paper . In Section 3 we formally introduce our PME-Softmax loss and the associated CIS negative sampling scheme , and describe the training algorithm . We highlight the performance of our method in Section 4 , and conclude with ideas and directions for future work in Section 5 . 2 RELATED WORK . 2.1 WORD2VEC , SOFTMAX AND NEGATIVE SAMPLING SCHEMES . With the introduction of Word2Vec , Mikolov et al . ( 2013b ; a ) achieved state-of-the-art results in terms of the quality of the words embeddings for various language modeling tasks . In the last couple of years , Word2Vec became a widely embedding method , that is used in various setups from language modeling to matrix factorization . Since then , it is important to note the progress made in improving words embeddings ( Bojanowski et al . ( 2016 ) ; Pennington et al . ( 2014 ) ) , especially more recently with the FastText model ( Joulin et al . ( 2016 ) ) that also leverages n-gram features . When dealing with the training of multi-class models with thousands or millions of output classes , one can replace softmax by candidate sampling algorithms . Different methods can speed up the training by considering a small randomly-chosen subset of candidates for each batch of training examples . Gutmann & Hyvärinen ( 2010 ) introduced Noise Contrastive Estimation ( NCE ) as an unbiased estimator of the softmax loss , and it has been shown to be efficient for learning word embeddings Mnih & Teh ( 2012 ) . In Mikolov et al . ( 2013b ) , the authors propose a negative sampling loss not to approximate softmax and but to learn high-quality vectors . Defined in Bengio et al . ( 2003 ) , sampled softmax can also be used to train these language models . Benchmarked in ( ( Bengio & Senécal , 2008 ; Jean et al. , 2014 ) ) , the authors have have shown state-of-the-art results in terms of performance and time complexity . Indeed , sampled softmax avoids computing scores for every possible continuation . Here , one chooses a proposal distribution from which it is cheap to sample , and performs a biased importance sampling scheme for approximating the softmax . Most previous work has focused on generic sampling methods such as uniform sampling and unigram sampling . More recently , in Chen et al . ( 2018 ) , the authors provide an insightful analysis of negative sampling . They show that negative samples that have high inner product scores with the context word are more informative in terms of gradients on the loss function . They show analytically and empirically the benefits of sampling from the unigram distribution . Leveraging this analysis , the authors propose a dynamic sampling method based on inner-product rankings . 2.2 GANS . First proposed by in 2014 in Goodfellow et al . ( 2014 ) , GANs have been quite successful at generating realistic images ( Ledig et al . ( 2016 ) ; Radford et al . ( 2015 ) ; Miyato et al . ( 2018 ) ) . GANs can be viewed as a framework for training generative models by posing the training procedure as a minimax game between the generative and discriminative models . Our approach leverages recent work on discrete GANs . Bose et al . ( 2018 ) describes an approach for adversarial contrastive estimation , where discrete GANs are leveraged to implement an adversarial negative sampler . However , our approach differs as in our method , called cooperative importance sampling , both the generator and the discriminator are trained to work cooperatively . 3 OUR APPROACH . 3.1 CONCEPTS AND NOTATION . We begin this section by formally defining the task . We denote by I and J two sets of objects . These objects can potentially represent sets of users , items or words . Given an i ∈ I sampled uniformly from I , we denote by P ( .|i ) the unknown conditional generative process that generates the set of j ∈ J and by X the resulting joint distribution over I × J . In our data , we assume that we only observe a finite sample of pairs ( i , j ) ∼ X . We denote G = { Gθ } θ∈Θ as the family of functions , with parameters Θ ⊂ Rp . The model Gθ takes an input in I × J and outputs a score Gθ ( i , j ) ∈ R. The objective is to learn a generative model Gθ that fits the conditional process observed in the data . 3.2 SOFTMAX ASSUMPTION : MODELING THE DATA AS A 1D RANDOM VECTOR . Usually , one tries to model the process X with a softmax formulation ( see eq 1 ) , defining a multinomial distribution P ( .|i ) ∈ P ( J ) . Therefore , for a given context i and any two targets ( j1 , j2 ) , P ( j1|i ) and P ( j2|i ) are not independent since the sum of all conditional probabilities over the target set must sum to 1 . That means , that under this formulation , the probability of observing j2 given we already observed j1 in the context of i goes to zero . We denote this property mutual exclusivity . ∀i ∈ I , gθ ( j|i ) = exp ( Gθ ( i , j ) ) exp ( Gθ ( i , j ) ) + ∑ j′∈J\j exp ( Gθ ( i , j ′ ) ) ( 1 ) In order to fit this model , for a given i ∈ I , one tries to perform Kullback-Leibler divergence minimization between the true distribution P ( .|i ) and the model of the distribution gθ : argmin θ∈Θ KL ( Pi||gθ ( .|i ) ) = argmax θ∈Θ Ej∼Pi [ ln ( gθ ( j|i ) ) ] ( 2 ) argmax θ∈Θ Ej∼Pi Gθ ( i , j ) − ln ∑ j′∈J exp ( Gθ ( i , j ′ ) ) ( 3 ) In the case of multi-class and single-label tasks it is natural to use the softmax formulation . However , when it comes to language modeling and sequence prediction , most of the tasks fall in the multilabeled settings . For a given context i , one does not observe one target item j but rather a subset of target items { j1 , ... , jk } ∈ Jk . For example , in a word2vec setting , the subset of targets is defined by the sliding window parameter . We generalize the current formulation to the multi-labeled settings as follows : P ( { j1 .. jk } |i ) = k∏ x=1 gθ ( jx|i ) , with ∑ j∈J gθ ( j|i ) = 1 ( 4 ) Inspired from textual analysis , Ueda & Saito ( 2003 ) ; Blei et al . ( 2003 ) suggested that words in sequences can be regarded as a mixture of distributions related to each of the different categories of the vocabulary , such as `` sports '' and `` music '' . Building upon this example , we effectively search over an enlarged class of models to better represent the multiplicity of the data . As detailed in the next subsection , we now train a product of independent distributions to learn this set generative process .

Summary: This paper presents Partially Mutual Exclusive Softmax (PMES), a relaxation of the full softmax that is commonly used for multi-class data. PMES is designed for positive-unlabeled learning, e.g., language modeling, recommender systems (implicit feedback), where we only get to observe positive examples. The basic idea behind PMES is that rather than considering all the non-positive examples as negative in a regular full softmax, it instead only considers a "relevant" subset of negatives. Since we actually don't know which of the negatives are more relevant, the authors propose to incorporate a discriminator which attempts to rate each negative by how hard it is to distinguish it from positives, and weight them by the predicted score from the discriminator when computing the normalizing constant for the multinomial probability. The motivation is that the negatives with higher weights are the ones that are closer to the decision boundary, hence will provide more informative gradient comparing to the negatives that are further away from the decision boundary. On both real-world and synthetic data, the authors demonstrate the PMES improves over some other negative sampling strategies used in the literature.