Title: LEARNING WITH REFLECTIVE LIKELIHOODS

Abstract: Models parameterized by deep neural networks have achieved state-of-the-art results in many domains. These models are usually trained using the maximum likelihood principle with a finite set of observations. However, training deep probabilistic models with maximum likelihood can lead to the issue we refer to as input forgetting. In deep generative latent-variable models, input forgetting corresponds to posterior collapse-a phenomenon in which the latent variables are driven independent from the observations. However input forgetting can happen even in the absence of latent variables. We attribute input forgetting in deep probabilistic models to the finite sample dilemma of maximum likelihood. We formalize this problem and propose a learning criterion-termed reflective likelihood-that explicitly prevents input forgetting. We empirically observe that the proposed criterion significantly outperforms the maximum likelihood objective when used in classification under a skewed class distribution. Furthermore, the reflective likelihood objective prevents posterior collapse when used to train stochastic autoencoders with amortized inference. For example in a neural topic modeling experiment, the reflective likelihood objective leads to better quantitative and qualitative results than the variational auto-encoder and the importance-weighted autoencoder.

1. INTRODUCTION:

Learning deep probabilistic models with maximum likelihood has led to many successes in density estimation, variational inference, and text and image generation (Dinh et al., 2016; Kingma & Dhariwal, 2018; Kingma & Welling, 2013; Rezende et al., 2014; Oord et al., 2016) . However, maximum likelihood learning of deep models often causes the problem of input forgetting-so called because it corresponds to ignoring the input. We refer to an input here as any variable being conditioned upon-for example a covariate in supervised learning or a latent variable in deep generative latent variable models. Input forgetting corresponds to latent variable collapse in the context of latent variable models and has been discussed in several works (Burda et al., 2015; Bowman et al., 2015; Hoffman & Johnson, 2016; Chen et al., 2016; Sønderby et al., 2016; Zhao et al., 2017; Alemi et al., 2018; Dieng et al., 2018a) . It also occurs when learning with Restricted Boltzmann Machines (RBMs) where all the hidden units of the RBM collapse by learning to only capture the bias in the visible units (Cho et al., 2011) . Input forgetting makes posterior inference in deep generative models very difficult. In this paper we study input forgetting when the input is a latent variable and when the input is a covariate.

1.1. CONTRIBUTIONS:

In this paper, we propose a learning criterion that does not suffer from the input forgetting problem induced by maximum likelihood when fitting deep probabilistic models. This criterion naturally arises from what we call the finite sample dilemma of maximum likelihood.

The finite sample dilemma of maximum likelihood. Consider a supervised learning setting where we have two random variables x and y. Our goal is to learn the relationship between these two random variables by fitting a conditional parametric model p θ (y | x). The learning problem corresponds to finding θ * that minimizes the risk function,

EQUATION

where p * denotes the population distribution and l(•, •) is a loss function that measures how different f θ (x) is from the true y. Maximum likelihood corresponds to a log loss,

l(y, f θ (x)) = -log p θ (y | x) and R mle (θ) = -E x∼p * (x) E y∼p * (y | x) [log p θ (y | x)] . (2)

In practice we approximate R mle (θ) using a finite dataset D = (x n , y n ) N n=1 that contains N realizations of the random variables x and y. The sample-based maximum likelihood objective is

EQUATION

Now consider the risk of maximum likelihood under the same conditional model p θ (y | x) when x and y are independent,

EQUATION

The corresponding sample-based maximum likelihood objective is

EQUATION

The objectives L(θ; D) and L (θ; D) in Eq. 3 and Eq. 5 are related as follows

EQUATION

Although there is no relationship between the two population risks R mle (θ) and R mle (θ), there is a relationship between their sample-based estimates.

When maximizing the likelihood under finite data-i.e. maximizing L(θ; D)-the term L (θ; D) is also being maximized. However maximizing L (θ; D) leads to input forgetting as the underlying data being used do not encode any relationship between x and y. To see this consider maximum likelihood with the same data but where each label y is also paired with all possible inputs x. This version of the data does not contain any relationship between x and y. Maximum likelihood using this data where x and y are independent corresponds to maximizing L (θ; D).

A new learning criterion for supervised learning. In light of the relationship in Eq. 6, we propose to explicitly discourage the independence behavior induced by maximizing L (θ; D). We do so by maximizing the reflective likelihood (RLL). Maximizing the RLL simultaneously maximizes the likelihood of the outputs under their true associated inputs and minimizes the average log-probability of these outputs when paired with all other possible inputs. On a classification under imbalance with the MNIST dataset, the classifiers trained with RLL outperform those trained with the usual maximum likelihood objective in all the instances of class imbalance. The performance were similar when the class distribution is uniform.

A new family of stochastic auto-encoders for unsupervised learning.

We extend the proposed RLL to unsupervised learning with stochastic auto-encoders and amortized inference. We call stochastic auto-encoders trained with the RLL objective Reflective Auto-Encoders (RAEs). RAEs do not suffer from the issue of posterior collapse-a phenomenon in which the estimated approximate posterior distribution of the latent variables reduces to the prior. We evaluate RAEs on a neural topic modelling experiment and found they outperform models learned with the maximum likelihood objective both quantitatively and qualitatively.

1.2 RELATED WORK.

Our work is closely related to two lines of work: penalized maximum likelihood and posterior inference in deep generative models.

Traditional penalized maximum likelihood methods such as the Lasso and L 2 -norm regularization directly operate on the parameters of a model (Tibshirani, 1996; Hinton, 1987; Louizos et al., 2017) .

These regularizers penalize the magnitude of the parameters and correspond to specific prior distributions on the parameters from the Bayesian perspective. In contrast several data-dependent regularizers have been proposed. As early as 1995, Bishop (1995) proposed to add noise to the input when training a neural network with stochastic gradient descent for maximum likelihood learning and showed this corresponds to a form of Tikhonov regularization. Several works have extended this to noise injection in the hidden units of a neural network (Srivastava et al., 2014; Maaten et al., 2013; Gal & Ghahramani, 2016; Wager et al., 2013; Dieng et al., 2018b) . Our work relates to those datadependent regularizers that are explicit-in the sense that the corresponding objective function can be written as the sum of the initial objective function and an additional regularization term.

Our work also relates to the literature on posterior inference in deep generative models. Latent variable models are ubiquitous in machine learning. They often involve intractable integrals that are approximated with Markov chain Monte Carlo or variational inference. Our work relates to variational methods for deep generative models (Kingma & Welling, 2013; Rezende et al., 2014) .

The problem that often occurs in these settings is the latent variables are driven independent to the observations thus rendering posterior inference meaningless. Several works have discussed this issue (Burda et al., 2015; Bowman et al., 2015; Hoffman & Johnson, 2016; Chen et al., 2016; Sønderby et al., 2016; Zhao et al., 2017; Alemi et al., 2018; Dieng et al., 2018a) . In this paper we identify a possible cause to this problem and adopt a regularization approach to fix it.

2. SUPERVISED LEARNING WITH REFLECTIVE LIKELIHOODS:

We consider the same setting as Section 1.1. We have observations (x n , y n ) N n=1 and our goal is to fit a parametric model p θ (y | x) to this data. As stated in Section 1.1, the usual maximum likelihood objective might lead to parameters θ that do not capture the true relationship between x and y.

In this section we propose the RLL criterion that encourages parameter settings under which y is more likely when conditioning on its corresponding x than under the average of all the conditional probabilities of y given all possible inputs. The RLL criterion has two forms.

Weak reflective likelihood. The weaker form of RLL minimizes L (θ; D) while maximizing the usual sample-based approximation L(θ; D),

EQUATION

where α n ≥ 0 is a hyperparameter trading off these two terms. We will discuss how to set α n at the end of this section. The objective Lweak RLL (θ; D) provides us with a way to explicitly prefer R mle (θ) in Eq. ( 1) over R mle (θ) in Eq. ( 4), over an entire training set. The objective Lweak RLL (θ; D), however, may capture the dependence between the input and the output for some examples but not necessarily for all examples. This is because the second term in Eq. 7 can be driven to its minimal value by setting the term inside the log to zero for one example. We tackle this issue by proposing a stronger version of RLL.

Strong reflective likelihood. The stronger version of RLL favors parameters that capture the relationship between x and y by discouraging the independence behavior induced by maximization of L (θ; D),

EQUATION

The second term comes from an application of Jensen's inequality,

R mle (θ) ≤ E y∼p * (y) log E x∼p * (x) p θ (y | x) ,

and using the dataset D to approximate this upper bound. The inner term of this upper bound is what we call a reflective probability,

EQUATION

This reflective probability can be used as a diagnostic measure of whether a trained model has captured the dependence between the input and output for each pair. The sample-based approximation of p refl θ (y) is what appears inside the logarithm in the second term in Eq. 8. Unlike Lweak RLL , this stronger objective function Lstrong RLL explicitly decreases the reflective probability of each individual example (x, y). From here on out we focus on Lstrong

RLL and refer to it as RLL.

Stochastic Approximation of RLL. The objective in Eq. 8 is not amenable to large datasets. First, when the sample size N is large computing Lstrong RLL (θ; D) is intractable. Second even if we use a small subset of data we are still left with approximating p refl θ (y) which is also intractable for large N . To enable maximizing RLL with large datasets, we propose to maximize

Lstrong RLL (θ; M, M ) = 1 |M| (x,y)∈M   log p θ (y | x) -α n log 1 |M )| (x ,y )∈M p θ (y | x )   , (10)

where M and M are two minibatch of data samples. In practice we improve the efficiency by constructing the second minibatch M using the first minibatch M. For each pair (x, y) ∈ M, we uniformly select K examples from M at random to construct M .

Choice of penalty α n . We investigate two strategies to set the cofficient α n . The first strategy simply sets α n to a global fixed coefficient α n = α 0 , where α 0 is a hyperparameter. The second strategy on the other hand dynamically adapts α n for each training example (x n , y n ) according to:

α n = α(x n , y n , θ) = α 0 if log p θ (y n | x n ) ≤ log prefl θ (y n ) 0 otherwise (11)

The second strategy is motivated by the intuition that the probability assigned to a correctly paired example (x n , y n ) by a model p θ should be higher than the average of the probabilities assigned to incorrectly paired examples (x, y n ) for all x ∼ p * (x). It is however unclear how much higher the probability of a correctly paired example p θ (y n | x n ) should be over the reflective probability p refl θ (y n ), and we do not want to increase the difference if the former is already greater than the latter.

3. UNSUPERVISED LEARNING WITH REFLECTIVE LIKELIHOODS:

We extend RLL to unsupervised learning. We do so using the auto-encoding framework (Hinton & Salakhutdinov, 2006; Vincent et al., 2008; Kingma & Ba, 2014; Rezende et al., 2014) . We first discuss stochastic auto-encoders and posterior collapse before proposing RAEs-a family of stochastic auto-encoders that do not suffer from the posterior collapse issue.

Stochastic auto-encoders and posterior collapse. We position ourselves in the setting where there are global parameters θ and one latent variable z for every observation y. An observation y is generated by first drawing a latent variable z from some prior distribution p(z)-that we assume fixed-and then sampling y from the conditional distribution of y given z. This conditional distribution p θ (y | z) is parameterized by θ. We are concerned with learning the parameters θ in the presence of the latent variables. Maximum likelihood corresponds to maximizing

EQUATION

)

The likelihood p θ (y | z) is represented as a powerful deep neural network that takes z as input. This objective Eq. 12 is intractable. Stochastic auto-encoders posit a distribution q φ (z | y) over the latents and either maximize (Burda et al., 2015 )

EQUATION

or its lower bound (Kingma & Ba, 2014; Rezende et al., 2014 )

EQUATION

)

The distribution q φ (z | y) is parameterized by a deep neural network that takes the observation y as input.

As discussed in several works, the objective in Eq. 14 is prone to posterior collapse. It can be rewritten as

EQUATION

What effectively happens is that the KL term quickly collapses to zero early on during optimization, leading to an approximate posterior that does not capture the data.

The importance-weighted auto-encoder (IWAE, Burda et al., 2015) was introduced to learn more expressive approximate posteriors by directly optimizing the objective in Eq. ( 13) using K samples from the approximate posterior,

EQUATION

) Burda et al. (2015) show this leads to less posterior collapse. However, the IWAE can still lead to posterior collapse due to the finite sample dilemma of maximum likelihood discussed in Section 1.

Reflective auto-encoders. We propose a new class of stochastic auto-encoders that prevent posterior collapse by maximizing the RLL. We reconsider the original population risk functions for maximum likelihood under both paired data,

R mle (θ) = -E x∼p * (x) E y∼p * (y | x) [log p θ (y | x)]

and its upper bound under independent data,

R mle (θ) ≤ E y∼p * (y) log E x∼p * (x) [p θ (y | x)] .

The RLL in Eq. 8 is a sample-based approximation of

L sup RLL (θ; p * ) = E x∼p * (x) E y∼p * (y | x) log p θ (y | x) -α log E x ∼p * (x ) [p θ (y | x )]

. Note this is simply the difference between the original maximum likelihood objective and the log reflective probability. In the case of unsupervised learning this corresponds to

EQUATION

where we define p θ (y | y ) as

p θ (y | y ) = p θ (y | z)p θ (z | y )dz = E p θ (z | y ) [p θ (y | z)].

Using this definition we rewrite the objective as

EQUATION

The expectations in Eq. 18 are intractable. In this paper, we propose to approximate them using importance weighting (Burda et al., 2015) . For that we define a parametric proposal distribution q φ (z | y)-we make the conditioning on y explicit to account for recognition networks as proposal distributions. We now write the different expectations as

E z∼p(z) [p θ (y | z)] ≈ pθ (y) = 1 K K k=1 ω(y, z k )p θ (y|z k ) where z k ∼ q φ (z | y); and E y ∼p * (y) E z∼p θ (z | y ) [p θ (y | z)] ≈ E y ∼p * (y) K k=1 v(y , z k )p θ (y | z k )

where z k ∼ q φ (z | y ). The expensive expectation over p * (y ) is dealt with as before by using a small random subset of training examples. The importance weights ω(y, z k ) and v(y , z k ) are computed as

ω(y, z k ) = exp(ω(y, z k )) and ω(y, z k ) = log p(z k ) -log q φ (z k | y). v(y , z k ) = exp(ṽ(y , z k )) K s=1 exp(ṽ(y , z s ))

and ṽ(y

, z k ) = log p θ (y | z k ) + log p(z k ) -log q φ (z k | y ).

Maximizing the objective in Eq. 18 encourages the proposal q φ to output a distinct approximate posterior distribution for each input y. We conjecture this helps avoid posterior collapse. We verify this later in our empirical study.

4. CONNECTIONS TO RANKING:

Collobert et al. ( 2011) proposed a pair-wise ranking loss for natural language processing. This ranking loss is defined as

EQUATION

where f θ is a parametric scoring function, and s pos and s neg are positive and negative examples respectively. s pos is often simply a training example such as a sentence, whereas s neg is a sample from a distribution conditioned on s pos . The hyperparameter m is the margin.

Consider the RLL objective in Eq. ( 8). Consider the choice of α n in Eq. ( 11). Using stochastic approximation with a single sample (x, y) for the likelihood term and a single sample (x , y) for the reflective probability we can rewrite the objective Lstrong RLL (θ; D) as

Lstrong RLL (θ; D) = log p θ (y | x) -α 0 log p θ (y | x ) =(1 -α 0 ) log p θ (y | x) -α 0 [0 -log p θ (y | x) + log p θ (y | x )] + .

This reveals that RLL is a convex sum between the usual maximum likelihood objective and the ranking loss in Eq. ( 19), modulated by the global coefficient α 0 . It directly tackles the finite sample dilemma discussed in Section 1 by maximizing the likelihood while ensuring that the correctly paired examples are better scored than incorrectly paired examples. This in turn maximizes the dependence between the inputs and outputs.

5. APPLICATIONS:

In this section we apply our proposed method to two different problems: MNIST digit classification under imbalance and neural topic modeling. In classification under imbalance it is hard to learn features of rare classes. Because RLL promotes a stronger dependence between inputs and outputs we expect it to perform better than the maximum likelihood objective in the presence of imbalance. We also consider neural topic models that use stochastic auto-encoders. Posterior collapse is particularly severe in text modeling. Neural topic models suffer from this problem (Srivastava & Sutton, 2017) . One manifestation of posterior collapse in neural topic modeling is that all the dimensions of the topic matrix collapse to the same topic which in turn contains words that are not necessarily related to each other (Miao et al., 2016; Srivastava & Sutton, 2017) . On both applications we found RLL yields better performance both quantitatively and qualitatively. It learns more useful features for rare classes in classification under imbalance on MNIST as evidenced by higher F1 scores (See fig.

1.) Finally it learns more meaningful latent variables as evidenced by lower perplexity on document completion (See Table 2 .) We also found that the choice of α n is dependent on the application. For classification we found a fixed schedule for alpha to perform best. For neural topic modeling we found an adaptive schedule to perform best. The reported results for RLL correspond to the best schedule for α n . We measure both F1 score and accuracy for each setting.

As a classifier, we use a deep convolutional network (LeCun et al., 1995) . It has two convolutional layers and two fully connected layers. Each convolutional layer consists of 5 × 5 convolution with stride 1 followed by 2 × 2 max pooling and a rectified linear unit (ReLU, Glorot et al., 2011) . The output from the final convolutional layer is flattened and processed by two fully-connected layers of 50 units. We train the classifier using the Adam optimizer with dropout regularization. We find the number of samples to estimate the second term in RLL by maximizing accuracy on the validation set. We found that in most cases one sample suffices. We choose α 0 to maximize accuracy on the validation set. We found α 0 = 0.5 to be the best.

In Table 1 , we see that the classifiers trained with RLL always outperform those trained with the usual maximum likelihood criterion when the class distribution is imbalanced. This is the case both in terms of F1 score and accuracy on both the original (balanced) test set as well as the matched test set. 

5.2. NEURAL TOPIC MODELLING:

We use the 20NewsGroup (Joachims, 1996) for topic modelling. 20NewGroup consists of 11,314 training and 7,531 test articles. The vocabulary size is 2,000. We follow the standard preprocessing steps which include tokenization and the removal of non-UTF-8 characters and English stop words (Miao et al., 2016; Srivastava & Sutton, 2017; Card et al., 2017) .

We train a neural topic model, which is an extension of latent Dirichlet allocation (LDA, Blei et al., 2003) with amortized inference. This is effectively a stochastic autoencoder with a bag-of-word input and output. The inference network q φ of this auto-encoder is a two-layer feed-forward neural network with 100 tanh units each. The output of this network is projected to a 15-dimensional vector normalized by softmax to compute the topic proportion. The likelihood p θ is a three-layer feedforward neural network that maps this topic proportion to a bag-of-words. Our architecture choice is based on the earlier observation by Srivastava & Sutton (2017) that this model is particularly prone to posterior collapse. We test both the VAE (see Eq. ( 15)) and the IWAE (see Eq. ( 16)) as baselines. These are two auto-encoders trained with different forms of the maximum likelihood objective. We use Adam (Kingma & Ba, 2014 ) and train each model for 200 epochs with a fixed learning rate of 0.002. For RLL, we use 5 samples to estimate the reflective probability. We choose an adaptive schedule for α n .

We measure both perplexity and the KL divergence between the estimated posterior distribution q φ and the prior distribution p(z) = N (z; 0, 1 2 ). The KL has been used as a metric for measuring posterior collapse. For perplexity, we report both the perplexity on the test set (Full PPL) and the document completion perplexity (Completion PPL, Wallach et al., 2009) . Both types of perplexities are computed using importance sampling. Document completion consists in holding out some words for each document in the test set to compute the topic proportions and then evaluating perplexity on the remaining words of each document using the topic proportions learned with the held out words.

It is a good way to assess the quality of the learned latent variables in topic modeling. We held out the first half of each document to compute the topic proportions and evaluated perplexity on the second half.

Similarly to the classification experiment, we observe that RAE-the stochastic auto-encoder trained with RLL-always outperforms the neural topic models trained with a maximum likelihood objective (VAE and IWAE) (see Table 2 .) In addition to having a lower perplexity, RLL yields slightly higher KL divergence scores-which suggests that RLL indeed improves upon VAE and IWAE in terms of posterior collapse. We verify this further by looking at the learned topics; RLL the topics learned by RLL are as good as the topics learned by Latent Dirichlet Allocation (LDA). This is not the case for the VAE. IWAE also yields good topics but they are less sharp than the ones learned by RAE. These topics can be found in the appendix.

6. CONCLUSION:

In this paper, we identified the finite sample dilemma when fitting probabilistic models with maximum likelihood using a finite set of observations. We conjectured this to be the cause of the input forgetting issue often encountered when fitting deep probabilistic models using maximum likelihood. We then proposed RLL-a new learning criterion that mitigates the input forgetting issue. RLL encourages a strong dependence between observations and their conditioning variables. This in turn results in better performance both in classification under imbalance and neural topic modeling with stochastic auto-encoders. 1e -3 0.12 0.12 0.12 0.12 0.12 0.12 0.12 0.12 D4 1e -3 1e -3 1e -3 0.14 0.14 0.14 0.14 0.14 0.14 0.14 D5 1e -3 1e -3 1e -3 1e -3 0.17 0.17 0.17 0.17 0.17 0.17 D6 1e -3 1e -3 1e -3 1e -3 1e -3 0.20 0.20 0.20 0.20 0.20 D7 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 0.25 0.25 0.25 0.25 D8 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 0.33 0.33 0.33 D9 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 0.49 0.49 D10 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 1e -3 0.99 

:



Figures and Tables:

Figure 1: Histogram of classification F1 scores for MLE and RLL. Left: Uniform distribution D1.Right: Imbalanced distribution D10. Performance of MLE and RLL on D1 is similar. However RLL outperforms MLE by a significant margin for the imbalanced distribution. This gain in performance comes from how well RLL performs on rare classes. For digits 2, 6, and 8 both MLE and RLL have 0 F1 scores.

F1 scores and accuracies on (a) the original test set and (b) matched test sets. Across the board, the classifiers trained with the proposed reflective log-likelihood perform similarly to or outperform those trained with maximum log-likelihood. We refer to this original dataset as D0. We build 9 additional training sets. Each additional training set Dk leaves only 5 training examples for each class j < k. See Table3in the Appendix for all the class distributions. The validation set in each case is constructed by collecting all the data points that are left from the training set. We train a classifier on each of these constructed training sets and test it on both the original test set, in which all the classes are uniformly represented, and the matched test set, in which the class distribution matches that of the corresponding training set.

Table content: <table><tr><td/><td/><td/><td/><td colspan="3">(a) Original Test Set</td><td/><td/><td/></tr><tr><td colspan="3">Method Metric D1</td><td>D2</td><td>D3</td><td>D4</td><td>D5</td><td>D6</td><td>D7</td><td>D8</td><td>D9</td><td>D10</td></tr><tr><td>MLE</td><td>Acc</td><td colspan="9">98.5 89.0 77.7 68.5 65.4 55.9 45.5 31.9 28.0 21.1</td></tr><tr><td>RLL</td><td>Acc</td><td colspan="9">98.6 92.0 83.3 70.2 71.7 59.1 48.8 35.6 33.8 31.1</td></tr><tr><td>MLE</td><td>F1</td><td colspan="9">98.5 84.5 69.3 57.9 57.8 43.9 32.2 19.5 20.6 17.5</td></tr><tr><td>RLL</td><td>F1</td><td colspan="9">98.6 92.6 80.9 60.7 65.9 48.4 37.2 22.8 27.4 27.0</td></tr><tr><td/><td/><td/><td/><td colspan="3">(b) Matched Test Set</td><td/><td/><td/></tr><tr><td colspan="3">Method Metric D1</td><td>D2</td><td>D3</td><td>D4</td><td>D5</td><td>D6</td><td>D7</td><td>D8</td><td>D9</td><td>D10</td></tr><tr><td>MLE</td><td>Acc</td><td colspan="9">99.2 90.9 80.8 72.4 66.1 55.7 45.1 32.6 29.2 19.5</td></tr><tr><td>RLL</td><td>Acc</td><td colspan="9">99.0 93.9 85.1 74.5 74.1 57.3 48.1 37.5 34.1 28.8</td></tr><tr><td>MLE</td><td>F1</td><td colspan="9">99.2 87.0 73.1 62.7 57.5 43.7 31.2 20.7 22.8 15.9</td></tr><tr><td>RLL</td><td>F1</td><td colspan="9">99.0 94.3 83.3 66.2 68.8 45.9 36.1 24.1 27.8 24.8</td></tr><tr><td colspan="6">5.1 CLASSIFICATION UNDER CLASS IMBALANCE</td><td/><td/><td/><td/></tr><tr><td colspan="11">We start with MNIST which has 5,000 training examples per class, totalling 50,000 training exam-</td></tr><tr><td>ples.</td><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

Perplexity and KL divergence computed from the neural topic model trained on the 20NewsGroup dataset. K denotes the number of posterior samples used during training. ( ) The difference between VAE (K = 1) and IWAE (K = 1) comes from the fact that we use an analytical solution to the KL divergence in the case of VAE and sample-based approximation in the case of IWAE.

Table content: <table><tr><td colspan="2">Criterion Autoencoder</td><td>K</td><td colspan="3">Full PPL↓ Completion PPL↓ KL(q φ (z | y) p(z))↑</td></tr><tr><td>MLE</td><td>VAE</td><td>1</td><td>841</td><td>911</td><td>2.3</td></tr><tr><td>MLE</td><td>IWAE</td><td>1</td><td>820</td><td>884</td><td>2.8</td></tr><tr><td>RLL</td><td>RAE</td><td>1</td><td>809</td><td>875</td><td>3.0</td></tr><tr><td>MLE</td><td>VAE</td><td>100</td><td>818</td><td>886</td><td>3.1</td></tr><tr><td>MLE</td><td>IWAE</td><td>100</td><td>780</td><td>838</td><td>3.5</td></tr><tr><td>RLL</td><td>RAE</td><td>100</td><td>763</td><td>822</td><td>4.0</td></tr></table>

Class distributions using the MNIST dataset. There are 10 class-one class for each of the 10 digits in MNIST. The distribution D1 is uniform and the other distributions correspond to different imbalance settings as given by the proportions in the table. Note these proportions might not sum to one exactly because of rounding.

Table content: <table><tr><td>Dist</td><td>0</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>6</td><td>7</td><td>8</td><td>9</td></tr><tr><td>D1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td><td>0.1</td></tr><tr><td colspan="11">D2 1e -3 0.11 0.11 0.11 0.11 0.11 0.11 0.11 0.11 0.11</td></tr><tr><td colspan="2">D3 1e -3</td><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

Top ten words of five randomly selected topics for different models on the 20NewsGroup dataset. Overall RAE learns topics as good as LDA-a non-neural network based model that does not suffer from posterior collapse. The VAE suffers from collapse and learns topics that are meaningless. The IWAE learns better topics than the VAE but has two topics (topics 3 and 5) that are worse than RAE. arab state peace land jewish write policy pay tax insurance money write care year article health rate car drive engine buy write speed article light dealer driver offer sale condition mouse best old excellent tape month trade game team year win play season player fan write baseball VAE write article thanks want try need help buy work really write article thanks thing really look buy drive gun problem thanks run work write problem program software drive buy computer write article buy drive thanks problem car try want help armenians armenian kill turkish say child war government live attack IWAE game play team player year write article better win baseball god christian jesus faith bible truth church christ believe christianity write article car thanks buy problem game bike team player armenians armenian turks turkish government israel state genocide attack serve thanks buy write car article phone appreciate drive sale run RAE god christian jesus life faith church believe bible christ christianity game team play player win score year season hit toronto gun weapon say kill police come carry crime health criminal card drive windows mode driver problem pc disk printer scsi government key clipper chip secure encryption escrow law enforcement security

Table content: <table><tr><td>Setting</td><td>Topics</td></tr><tr><td>israel israeli jews</td><td/></tr><tr><td>LDA</td><td/></tr></table>

Peer review (iclr2019):
 Summary: This paper proposes maximizing the “reflective likelihood,” which the authors define as: E_x E_y [log q(y|x) - .alpha log q(y)] where the expectations are taken over the data, q is the classifier, and .alpha is a weight on the log q(y) term. The paper derives the reflective likelihood for classification models and unsupervised latent variable models. Choices for .alpha are also discussed, and connections are made to ranking losses. Results show superior F1 and perplexity in MNIST classification and 20NewsGroups modeling. Pros: I like how the paper frames the reflective likelihood as a ranking loss. It does seem like subtracting off the marginal probability of y from the conditional likelihood should indeed ‘focus’ the model on the dependent relationship y|x. Can this be further formalized? I would be very interested in seeing a derivation of this kind. I like that the authors test under class imbalance and report F1 metrics in the experiments as it does seem the proposed method operates through better calibration. Cons: My biggest issue with the paper is that I find much of the discussion lacks rigor. I followed the argument through to Equation 3, but then I became confused when the discussion turned to ‘dependence paths’: “we want our learning procedure to follow the dependence path—the subspace in Θ for which inputs and outputs are dependent. However this dependence path is unknown to us; there is nothing in Eq. 1 that guides learning to follow this dependence path instead of following Eq. 3—the independence path” (p 3). What are these dependence paths? Can they be defined mathematically in a way that is more direct than switching around the KLD directions in Equations 1-3? Surely any conditional model x-->y has a ‘dependence path’ flowing from y to x, so it seems the paper is trying to make some stronger statement about the conditional structure? Moving on to the proposed reflective likelihood in Equation 4, I could see some connections to Equations 1-3, but I’m not sure how exactly that final form was settled upon. There seems to be a connection to maximum entropy methods? That is, E_x E_y [log q(y|x) - .alpha log q(y)] = E_x E_y [log q(y|x)] + .alpha E_y [ -log q(y)] .approx E_x E_y [log q(y|x)] + .alpha H[y], if we assume q(y) approximates the empirical distribution of y well. Thus, the objective can be thought of as maximizing the traditional log model probability plus an estimate of the entropy. As there is a long history of maximum entropy methods / classifiers, I’m surprised there were no mentions or references to this literature. Also, I believe there might be some connections to Bayesian loss calibration / risk by viewing .alpha as a utility function (which is easy to do when it is defined to be data dependent). I’m less sure about this connection though; see Cobb et al. (2018) (https://arxiv.org/abs/1805.03901) and its citations for references. The data sets used in the experiments are also somewhat dissatisfying as MNIST and 20NewsGroups are fairly easy to get high-performing models for. I would have liked to have seen more direct analysis / simulation of what we expect from the reflective likelihood. As I mentioned above, I suspect its really providing gains through better calibration---which the authors may recognize as F1 scores are reported and class imbalance tested---but the word ‘calibration’ is never mentioned. More direction comparison against calibration methods such as Platt scaling would be make the experiments have better focus. It would be great to show that this method provides good calibration directly during optimization and doesn’t need the post-hoc calibration steps that most methods require. Evaluation: While the paper has some interesting ideas, they are not well defined, making the paper unready for publication. Discussion of the connections to calibration and maximum entropy seems like a large piece missing from the paper’s argument.