Title: ON UNSUPERVISED-SUPERVISED RISK AND ONE-CLASS NEURAL NETWORKS

Abstract: Most unsupervised neural networks training methods concern generative models, deep clustering, pretraining or some form of representation learning. We rather deal in this work with unsupervised training of the final classification stage of a standard deep learning stack, with a focus on two types of methods: unsupervisedsupervised risk approximations and one-class models. We derive a new analytical solution for the former and identify and analyze its similarity with the latter. We apply and validate the proposed approach on multiple experimental conditions, in particular on four challenging recent Natural Language Processing tasks as well as on an anomaly detection task, where it improves over state-of-the-art models.

1. INTRODUCTION:

Machine learning systems often share the same architecture composed of two stages: the first stage computes representations of the input observations, while the second stage performs classification based on these representations. Most unsupervised training methods focus on the first stage: representation learning. This includes for instance generative models (VAE, GAN...), clustering techniques and, in the Natural Language Processing (NLP) domain, all recent contextual words embeddings (RoBERTa, XLNet, GPT-2...). This work rather deals with the final classification step, more precisely how to train neural classifiers in an unsupervised way. In contrast to unsupervised training of the first stage that aims at learning representations, unsupervised training of the final stage may rather pursue one of the following objectives, among others:

• Training one-class models for anomaly detection

• Exploiting unsupervised approximations of the classifier risk to train a model from a priori knowledge and unlabeled data instead of labeled samples

The former is a special type of binary classification task, where the positive class represents "normal" observations and the objective is to identify unknown and often rare observations that can be considered as anomalies and form the negative class.

The latter deals with training standard discriminative classifiers without labels, i.e., when assuming that the precise target classification task is not defined explicitly with sample labels, but implicitly with a priori knowledge. We review in Section 2 the family of one-class models as well as an unsupervised approximation of the risk, and explore their relation in Section 3.3, hence bridging the gap between both unsupervised discriminative classification approaches.

The main original contributions of this work are:

• We derive an exact and analytical solution (Eq 5) to the risk approximation proposed by Balasubramanian et al. (2011) • We analyze the properties of this solution, which lead to the following new results:

-We extend this solution into an end-to-end differentiable loss that can be easily integrated into any modern deep learning toolkit (Eqs 6, 7) -We propose an unsupervised training algorithm based on this analysis (Alg 1) -We propose a new posterior regularization term to improve this approach (Eq 8)

• We identify and study the similarity of this approximation with the one-class neural network anomaly detection method (Section 3.3) • We validate experimentally the unsupervised model on several datasets and tasks, including a comparison with state-of-the-art one-class neural networks (Section 4)

2. RELATED WORK:

We focus in this literature review on two unsupervised training methods for discriminative classifiers that do not aim at computing representations of the input space, but that rather exploit such representations to perform a final classification task. The first such method is an unsupervised-supervised (we have adopted the terminology of the original paper) approximation of the classifier risk that has been proposed by Balasubramanian et al. (2011) and that is detailed in Section 2.1. The second class of methods is the family of one-class models for anomaly detection, which is reviewed in Section 2.2.

2.1. RISK APPROXIMATION:

Let be given a binary linear classifier with parameters θ that computes a scalar score f (x) ∈ IR for observation x. The classifier outputs class ŷ = 0 iff f (x) <= 0, and ŷ = 1 iff f (x) > 0. The risk of this classifier with a hinge loss is (Balasubramanian et al., 2011) :

R(θ) = E p(x,y) (1 -f (x) • (2y -1)) + (1) R(θ) = P (y = 0) p(f (x) = α|y = 0)(1+α) + dα+P (y = 1) p(f (x) = α|y = 1)(1-α) + dα

(2) Balasubramanian et al. (2011) prove that this risk can be optimized in an unsupervised way, as the labels y are not required to compute Eq 1, when assuming that:

• The class-marginal prior P (y) is known;

• The class-conditional distribution of the scores p(f (x)|y) is Gaussian, which is supported by the central limit theorem -please refer to Balasubramanian et al. (2011) for further details.

The training algorithm proposed by the authors consists in the combination of (i) a gradient descent to optimize the linear classifier parameters θ; and (ii) the Expectation-Maximization (EM) algorithm, to compute the Gaussian parameters.

We derive a new formulation of this risk and study it in Section 3.1.

2.2. ONE-CLASS MODELS:

One-class models are based on the assumption that all observations belong to a single positive, "normal" class, except for (a few) outliers associated with the negative class. Given that there is no label to identify which observations are outliers, the problem can be cast as an unsupervised training problem. This class of models are typically used in anomaly detection applications.

The model at the origin of this research domain is the One-Class SVM (Schölkopf et al., 2001 ) (OC-SVM). This model projects positive observations into a feature space, and computes an hyper-plane in this feature space that separates most of these points from the region close to the origin, where outliers (noise) are assumed to be. The objective function of this model is:

min w,r,e 1 2 ||w|| 2 -r + 1 νN N i e i

under the constraints (e i are slack variables): e i ≥ 0 and w T φ(x i ) ≥ r -e i .

w corresponds to the linear classifier weights and φ is the non-linear SVM projection. w T φ(x) -r is the signed distance between any of the N samples and the decision hyperplane.

A powerful extension of the one-class SVM is the Support Vector Data Description (SVDD) model (Tax & Duin, 2004) . This model exploits an hypersphere with radius R and center c instead of an hyperplane to separate the positive and negative classes:

min R,c,e R 2 + 1 νN N i e i

under the constraints that e i ≥ 0 and ||φ(

x i ) -c|| 2 ≤ R 2 + e i

The SVDD model has been enriched by Ruff et al. (2018) to learn a representation φ W (x) computed with a deep neural network with parameters W , which gives the Deep SVDD model:

min R,W R 2 + 1 νN N max 0, ||φ W (x i ) -c|| 2 -R 2 + λ 2 l ||W l || 2 (3)

This model is trained by alternating a Stochastic Gradient Descent (SGD) step on W and computing the optimum R.

Finally, the original OC-SVM has also been extended as a deep learning model with the One-Class Neural Network (OC-NN) (Chalapathy et al., 2018) . In this model, the final linear layer w in a stack of deep neural network layers is interpreted as defining the decision hyperplane:

EQUATION

with V the previous layers that compute a representation of the input and g() the previous activation. This model is trained by alternating a SGD step to update (V, w) and computing the optimal r.

Another model of this family has recently been published: the One-class Convolutional Neural Network (Oza & Patel, 2018) , but the training objective of this model departs from the previous unsupervised training objectives, as this model is trained with the standard cross-entropy loss with negative samples that are artificially generated from a Gaussian distribution centered at the origin.

3. UNSUPERVISED SUPERVISED RISK:



3.1. EXACT RISK DERIVATION:

Starting from Eq 1, we derive1 a closed-form solution to compute the risk from the two Gaussian means µ and variances Σ that model the distribution of the score f (x) (we note P (y = 0) = p 0 ):

R(µ, Σ) = p 0 2 (1 + µ 0 ) 1 -erf -1 -µ 0 σ 0 √ 2 + (5) 1 -p 0 2 (1 -µ 1 ) 1 + erf 1 -µ 1 σ 1 √ 2 + p 0 σ 2 0 N (-1; µ 0 , σ 0 ) + (1 -p 0 )σ 2 1 N (1; µ 1 , σ 1 ) with N (α; µ, σ) = 1 √ 2πσ 2 e -(α-µ) 2 2σ 2

Balasubramanian et al. (2011) proposed to optimize the risk with finite differences. We rather propose to use the analytical solution derived in Eq 5, which has the following advantages:

• The risk value is exact and not approximated;

• Computation of the risk is much faster using Eq 5 than with numerical approximations;

• This equation is differentiable with respect to the Gaussian parameters. We derive next another function that relates the Gaussian parameters to the model parameters θ. Hence, the full risk can be directly integrated as a loss function in deep learning toolkits; • The analytical equation can be analyzed, which leads to novel insights as shown next.

Let us plot Equation 5 as a function of ( µ 0 , µ 1 ) in Figure 1 (left), for p 0 = 0.1 and σ 0 = σ 1 = 1 When we fix µ 1 , we can see in Figure 1 (right) that the risk as a function of µ 0 can be well approximated by a scaled and translated rectified linear function, as long as the variances are small enough. Furthermore, the lower σ 0 (and σ 1 ) is, the better the risk is. Varying µ 1 and σ 1 only translates this curve vertically, above the horizontal axis. So, assuming that the risk has first been minimized with respect to µ 1 , then the global minimum of the risk may be obtained by decreasing linearly µ 0 . Conversely, lower risks are obtained when µ 1 is increasing. Although we have not exploited this piece-wise linear approximation of the risk in our implementation, it is interesting to compare it to the max(0, • • • ) term in Equation 4.

Let us now make another assumption: that both modes (µ 0 , σ 0 ) and (µ 1 , σ 1 ) of the score distribution are well separated. This is a reasonable assumption when we are not too far away from the global optimum, because the previous analysis has already shown that getting close to the optimum implies that µ 0 is small, µ 1 is large and that σ 0 and σ 1 are small. Then, a good approximation of µ 0 and µ 1 can be computed by splitting all the scores f (x) according to the p 0 -quantile x p0 defined as

x p0 = arg min x p 0 -z∈X 1 f (z)<f (x) N (6)

where the set of all observations X is of size N . Let us call X -the subset of size N -of all data points that are on the left side of the p 0 -quantile:

X -= {x ∈ X s.t. f (x) < f (x p0 )}

and similarly for the other side:

X + = {x ∈ X s.t. f (x) ≥ f (x p0 )}

We can now approximate the Gaussian parameters deterministically:

µ 0 1 N - x∈X - f (x) µ 1 1 N + x∈X + f (x) (7) σ 2 0 1 N - x∈X - f (x) 2 - 1 N - x∈X - f (x) 2 σ 2 1 1 N + x∈X + f (x) 2 - 1 N + x∈X + f (x) 2

Intuitively, decreasing the risk may be achieved by decreasing µ 0 , σ 0 , σ 1 and increasing µ 1 . Plugging these equations into equation 5 gives a differentiable loss with respect to the network parameters, which can be used in every modern deep learning toolkit.

3.2. GEOMETRIC INTERPRETATION:

Following Chalapathy et al. (2018) , we can consider a deep neural network that computes some representation of its inputs. These representations are then passed to a final binary linear classification layer with a single scalar output. This final layer, and optionally the previous layers, may be trained by minimizing our unsupervised risk in Eq 5 with Stochastic Gradient Descent. As discussed in Section 2.2, this final layer actually defines an hyperplane that separates both positive and negative instances, and its output is the signed distance between each observation and this hyperplane. µ 0 is the average of these signed distances for all points that are on one side of the hyperplane (X -), and µ 1 for all points on the other side X + . So decreasing µ 0 and increasing µ 1 can be interpreted as moving away all samples in X -and in X + as far as possible from the hyperplane, as show in Figure 2 (right).

An important constraint is that the proportion of points on both sides of the hyperplane should be equal (or close) to p 0 , otherwise, an easy way to decrease the risk with unbalanced classes is to translate the hyper-plane along the vector w infinitely, moving all samples into the most frequent class. The constraint is thus

f (x) ≤ 0 ∀x ∈ X -and f (x) ≥ 0 ∀x ∈ X +

This constraint can be fulfilled by adding another term to the risk, which becomes:

EQUATION

While Balasubramanian et al. (2011) have used the class marginal only as a prior information, the additional term in Equation 8 can be seen as a posterior regularization term, which forces the posterior distribution P (y = 0|X) to match p 0 .

Algorithm 1 summarizes the training procedure.

Algorithm 1 End-to-end unsupervised training • Initialization:

-Let consider a binary classification task, for which we assume that the proportion of class-0 elements p 0 is known approximately; -Let be given a corpus of observations {x i } 1≤i≤N without labels; -Let be given a deep neural network g φ (x) with parameters φ that computes a vectorial representation of an input x, which is fed to a final linear classification layer f θ (g φ (x)) with parameters θ; φ and θ may be initialized randomly or pretrained.

• Iterate::

-Run a forward pass on the dataset {x i } 1≤i≤N with the current parameters φ, θ.

-Compute all classifier scores {s i = f θ (g φ (x i ))} 1≤i≤n over the full corpus N , or over a batch of observations n that is large enough to assume that the distribution of classes in the batch is representative of the distribution in the whole corpus. -Sort the list of scores (s i ) 1≤i≤n to compute the p 0 -quantile x p0 , following Equation 6.

-Compute the Gaussian parameters µ = (µ 0 , µ 1 ), Σ = (σ 0 , σ 1 ) with Equations 7.

-Compute the risk (Eq 8) with these Gaussian parameters.

-Apply automatic differentiation to compute ∇ θ R(µ, Σ), and optionally ∇ φ R(µ, Σ); -Run a step of SGD to update θ, and optionally φ.

3.3. RELATION WITH ONE-CLASS NEURAL NETWORKS:

The One-Class Neural Network Chalapathy et al. (2018) similarly splits the set of observations with an hyper-plane defined by the last layer of a deep neural network stack, but while Equation 8 splits the samples according to p 0 , the OC-NN splits them according to the ν-quantile of the points sorted by their signed distance to the hyper-plane, where ν controls the number of data points that are allowed to be on the negative side of the hyper-plane. This ν hyper-parameter plays the same role as our p 0 . By rewriting their distance with our notation f (x), their loss (Eq 4) becomes:

min L + 1 ν 1 N i (max(0, r -f (x i )))

where L is a term that does not depend on x i . Chalapathy et al. ( 2018) compute the optimal r as "the ν-quantile" of the scores, so we can rewrite r = f (x p0 ), and their sum as our µ 0 :

min L + 1 ν N - N (f (x p0 ) -µ 0 ) = min (L + f (x p0 ) -µ 0 )

Their objective thus aims at maximizing µ 0 , i.e., making all negative samples as close as possible to the hyperplane, as shown in Figure 2 (left). This equation strongly resembles the linear approximation of the risk that we have depicted in Figure 1 , except that the OC-NN takes into account only the negative part of the embeddings space, while our risk includes both negative and positive parts, and that the gradients are in opposite directions, as summarized in Figure 2 . are represented in the embeddings space just before the final linear layer; the hyperplane is defined by the parameters θ, and f θ (x) is the signed distance of the samples to the hyperplane. During training, the OC-NN tends to reduce the distance between the negative samples and the hyperplane, while our unsupervised loss tends to increase this distance for both negative and positive samples.

4. EXPERIMENTAL VALIDATION:

The proposed unsupervised risk is coded in pytorch (Paszke et al., 2017) and is freely distributed2 It is evaluated in various tasks: (i) on a synthetic toy classification dataset; (ii) on the Wisconsin Breast Cancer benchmark; (iii) on four NLP tasks and (iv) on a standard anomaly detection task. Following the related works, the standard unsupervised accuracy metric (Xie et al., 2016) is used for the first three cases, while the Area Under Curve (AUC) metric is used for anomaly detection.

4.1. SYNTHETIC DATASET:

We first validate our approach on a synthetic dataset, which contains 10,000 4-dimensional instances sampled from a bi-Gaussian distribution (p 0 = 0.6,

µ 0 = [1, 1, 1, 1] T , σ 0 = [1, 1, 1, 1] T ; p 1 = 0.4, µ 1 = [-2, -2, -2, -2] T , σ 1 = [1, 1, 1, 1] T ).

We train two simple models with 1,000 training epochs: one with a single layer, and another one with two layers and 2 hidden neurons (half of the input size). The accuracy per training epoch is shown on the left curve in Figure 3 . We further study the sensitivity of our algorithm to initial conditions by retraining the model 10 times with random initial parameters: the standard deviation of the accuracy is smaller than 2%. This first experiment validates that the unsupervised training algorithm is able to quickly and reliably converge towards the expected solution when the feature space explicitly encodes the class information.

4.2. WISCONSIN BREAST CANCER DATASET:

We validate next our unsupervised approach on a standard machine learning benchmark for binary classification: the Wisconsin Breast Cancer dataset (Dua & Karra Taniskidou, 2017) , composed of 569 instances with 30 dimensions each. The right curve in Figure 3 shows the accuracy of both our 1 and 2-layers models. 15 hidden neurons (half of the input dimension) are used for the 2-layers model.

The convergence of our method is also fast and stable on this more realistic dataset. The state-ofthe-art for supervised learning on this dataset is 99.1% of accuracy (Osman, 2017) . With 91% of accuracy, our approach performs relatively well given that it is purely unsupervised. As a comparison, we have run a K-Means clustering algorithm on the same dataset, which gives 85% of accuracy.

4.3. SENTEVAL TASKS:

We validate next our unsupervised approach on four recent and more difficult Natural Language Processing (NLP) binary classification datasets:

• Movie Review (MR): classification of positive vs. negative movie reviews;

• Product Review (CR): classification of positive vs. negative product reviews;

• Subjectivity status (SUBJ): classification of subjective vs. objective movie reviews; • Opinion polarity (MPQA): classification of positive vs. negative movie reviews.

These datasets as well as the experimental evaluation protocol that we have used are described in details in Conneau & Kiela (2018) . This protocol first computes a sentence representation with the state-of-the-art method InferSent (Conneau et al., 2017) , and then passes these sentence embeddings into a simple feed-forward network that is trained on each dataset.

We have adopted the same experimental protocol and the same hyper-parameters, except that we do not train the final feed-forward network with supervised labels and the cross-entropy loss, but we rather train it without any label and with our proposed unsupervised loss. Table 1 summarizes the accuracy of the state-of-the-art supervised models trained on the full corpus (InferSent sup.) and on only 100 instances (InferSent 100-ex), as well as the accuracy of the proposed unsupervised model (Unsup risk). Results in italic are taken from Conneau et al. (2017) , other results are computed. We can observe that the proposed purely unsupervised method always gives at least as good results as the state-of-the-art transfer learning model trained on 100 reviews, with a notable improvement of +20% absolute for the subjectivity classification task.

4.4. ANOMALY DETECTION:

We finally validate our approach on an anomaly detection task. We adopt the same dataset and experimental protocol than Ruff et al. (2018) and Chalapathy et al. (2018) for comparison. The tasks consists in detecting outliers in digits images, where the "normal class" is the positive class and is composed of images corresponding to a single target digit, and the outliers are randomly sampled from the other digits images. Our model is composed of a single additional feed-forward layer on top of the Ruff et al. (2018) model. This final classification layer is initialized from the Ruff et al. ( 2018) parameters, and it is then trained in an unsupervised way with the loss in Equation 8. We tune the hyper-parameters (number of epochs and learning rate) on a development corpus obtained by keeping the same positive instances from the training corpus, but adding different negative training samples. We rerun every experiment 10 times with different seeds to compute the standard deviation.

For the DeepSVDD, we report both the figures from the original paper, and the results obtained with the authors code, which may differ because of slightly varying conditions. The DeepSVDD outputs on the right are the ones that our own model is based on, and with which it should be compared to. We can note that our proposed unsupervised method always improve compared to the One-Class neural network, and is also generally better than the DeepSVDD model run on the same platform.

Compared to the one-class models, our approach exploits information from all instances instead of only the negative samples (see Figure 2 ). Furthermore, under reasonable assumptions, our loss converges towards the theoretical optimum of the classifier risk (See Eq 1).

5. CONCLUSION:

We have shown that both unsupervised-supervised classifier risk approximation and one-class neural networks lead to similar training procedures, although they optimize a slightly different objective. One of the main difference is that the former exploits all training samples, positive and negative, which should lead to better parameter estimates. This seems to be confirmed by experimental validation. Based on the similarity between both types of methods, we have also shown experimentally and through analysis that the unsupervised-supervised classifier risk approximation is a valuable method to be included in the set of approaches dedicated to anomaly detection. In future works, we plan to extend this approach for multi-class classification and few-shot learning.

p(f (x)|y = 1) ∼ N (µ 1 , σ 1 ) where N (µ, σ) is the standard normal distribution with mean µ and variance σ 2 . We can then rewrite an approximation of this risk under the previous assumption and the additional assumption that the class priors P (y) are known:

R = P (y = 0) N (α; µ 0 , σ 0 )(1 + α) + dα + P (y = 1) N (α; µ 1 , σ 1 )(1 -α) + dα

Removing the non-linearity:

R = P (y = 0) +∞ -1 N (α; µ 0 , σ 0 )(1 + α)dα + P (y = 1) 1 -∞ N (α; µ 1 , σ 1 )(1 -α)dα Distributing: R = P (y = 0) +∞ -1 N (α; µ 0 , σ 0 )dα + P (y = 0) +∞ -1 αN (α; µ 0 , σ 0 )dα + P (y = 1) 1 -∞ N (α; µ 1 , σ 1 )dα -P (y = 1) 1 -∞ αN (α; µ 1 , σ 1 )dα

We know that the cumulative distribution function of a (µ, σ) normal is:

EQUATION

We know that erf(-∞) = -1, erf(0) = 0 and erf(+∞) = 1, so

R = P (y = 0) 2 1 -erf -1 -µ 0 σ 0 √ 2 + P (y = 0) +∞ -1 αN (α; µ 0 , σ 0 )dα + P (y = 1) 2 1 + erf 1 -µ 1 σ 1 √ 2 -P (y = 1) 1 -∞ αN (α; µ 1 , σ 1 )dα

Integration by parts give: 

-µ σ √ 2 - a 2 - a 2 erf a -µ σ √ 2 + - b 2 + a 2 - b 2 erf b -µ σ √ 2 + µ 2 erf b -µ σ √ 2 + a 2 erf a -µ σ √ 2 - µ 2 erf a -µ σ √ 2 - σ √ 2π (e -

:

See full derivation in Appendix.

The code is given in the supplementary material and is further distributed with an open-source license on a public git repository.

Figures and Tables:

.

Figure 1: Risk as a function of both (µ 0 , µ 1 ) (left), and only µ 0 (right) for µ 1 = 2, σ 1 = 1 and σ 0 ∈ {0.1, 1, 3}

Figure2: Comparative illustration of OC-NN and unsupervised risk approximation: observations are represented in the embeddings space just before the final linear layer; the hyperplane is defined by the parameters θ, and f θ (x) is the signed distance of the samples to the hyperplane. During training, the OC-NN tends to reduce the distance between the negative samples and the hyperplane, while our unsupervised loss tends to increase this distance for both negative and positive samples.

Figure 3: Accuracy as a function of the number of unsupervised training epochs on the synthetic dataset (left) and on the Wisconsin Breast Cancer dataset (right).

µ, σ)dx = bF (b) -aF (a)erf(u)du = u(b)erf(u(b)) -u(a)erf(u(a)) + 1 √ π (e -u(b) 2 -e -u(a) 2 )

way to obtain this result is to use the following known formula: b a xN (x; µ, σ)dx = µ b a N (x; µ, σ)dx -σ 2 [N (x; µ, σ)] (α; µ 1 , σ 1 )dα

Unsupervised accuracy on four binary NLP tasks

Table content: <table><tr><td>System</td><td colspan="4">CR SUBJ MPQA MR</td></tr><tr><td>InferSent sup.</td><td>86.3</td><td>92.4</td><td>90.2</td><td>81.1</td></tr><tr><td colspan="2">InferSent 100-ex 63.8</td><td>62.5</td><td>70.1</td><td>53.9</td></tr><tr><td>Unsup risk</td><td>66.8</td><td>83.0</td><td>70.9</td><td>59.7</td></tr></table>

Results (AUC) on anomaly detection (

Table content: <table><tr><td>: from original papers)</td></tr></table>

Peer review (iclr2020):
The anonymous authors consider the problem of training of classifiers in an unsupervised way. They propose an extension to a one-class based approach that can do anomaly detection in an unsupervised fashion. The main contribution is a modification of the target function for the training of one-class NN. The experiments are not convincing and the modification doesn*t seem to provide much inside into representation learning and anomaly detection area. 1. Figure 3: no axis labels 2. ROC AUC is not the best quality to measure the quality of imbalanced classification problems or anomaly detection, PR AUC (average precision) is better 3. In Table 2 and other experiments, there is a comparison to only one existed method e.g. authors don*t reproduce results for OC-NN 4. Table 1: why compare a supervised method to an unsupervised one and don*t compare to other methods?