Learning with Noisy Labels for Sentence-level Sentiment Classification

Abstract: Deep neural networks (DNNs) can fit (or even over-fit) the training data very well. If a DNN model is trained using data with noisy labels and tested on data with clean labels, the model may perform poorly. This paper studies the problem of learning with noisy labels for sentence-level sentiment classification. We propose a novel DNN model called NetAb (as shorthand for convolutional neural Networks with Ab-networks) to handle noisy labels during training. NetAb consists of two convolutional neural networks, one with a noise transition layer for dealing with the input noisy labels and the other for predicting 'clean' labels. We train the two networks using their respective loss functions in a mutual reinforcement manner. Experimental results demonstrate the effectiveness of the proposed model.

It is well known that sentiment annotation or labeling is subjective BIBREF0. Annotators often have many disagreements. This is especially so for crowd-workers who are not well trained. That is why one always feels that there are many errors in an annotated dataset. In this paper, we study whether it is possible to build accurate sentiment classifiers even with noisy-labeled training data. Sentiment classification aims to classify a piece of text according to the polarity of the sentiment expressed in the text, e.g., positive or negative BIBREF1, BIBREF0, BIBREF2. In this work, we focus on sentence-level sentiment classification (SSC) with labeling errors.
As we will see in the experiment section, noisy labels in the training data can be highly damaging, especially for DNNs because they easily fit the training data and memorize their labels even when training data are corrupted with noisy labels BIBREF3. Collecting datasets annotated with clean labels is costly and time-consuming as DNN based models usually require a large number of training examples. Researchers and practitioners typically have to resort to crowdsourcing. However, as mentioned above, the crowdsourced annotations can be quite noisy. Research on learning with noisy labels dates back to 1980s BIBREF4. It is still vibrant today BIBREF5, BIBREF6, BIBREF7, BIBREF8, BIBREF9, BIBREF10, BIBREF11, BIBREF12 as it is highly challenging. We will discuss the related work in the next section.
This paper studies the problem of learning with noisy labels for SSC. Formally, we study the following problem.
Problem Definition: Given noisy labeled training sentences $S=\lbrace (x_1,y_1),...,(x_n,y_n)\rbrace $, where $x_i|_{i=1}^n$ is the $i$-th sentence and $y_i\in \lbrace 1,...,c\rbrace $ is the sentiment label of this sentence, the noisy labeled sentences are used to train a DNN model for a SSC task. The trained model is then used to classify sentences with clean labels to one of the $c$ sentiment labels.
In this paper, we propose a convolutional neural Network with Ab-networks (NetAb) to deal with noisy labels during training, as shown in Figure FIGREF2. We will introduce the details in the subsequent sections. Basically, NetAb consists of two convolutional neural networks (CNNs) (see Figure FIGREF2), one for learning sentiment scores to predict `clean' labels and the other for learning a noise transition matrix to handle input noisy labels. We call the two CNNs A-network and Ab-network, respectively. The fundamental here is that (1) DNNs memorize easy instances first and gradually adapt to hard instances as training epochs increase BIBREF3, BIBREF13; and (2) noisy labels are theoretically flipped from the clean/true labels by a noise transition matrix BIBREF14, BIBREF15, BIBREF16, BIBREF17. We motivate and propose a CNN model with a transition layer to estimate the noise transition matrix for the input noisy labels, while exploiting another CNN to predict `clean' labels for the input training (and test) sentences. In training, we pre-train A-network in early epochs and then train Ab-network and A-network with their own loss functions in an alternating manner. To our knowledge, this is the first work that addresses the noisy label problem in sentence-level sentiment analysis. Our experimental results show that the proposed model outperforms the state-of-the-art methods.
Our work is related to sentence sentiment classification (SSC). SSC has been studied extensively BIBREF18, BIBREF19, BIBREF20, BIBREF21, BIBREF22, BIBREF23, BIBREF24, BIBREF25, BIBREF26, BIBREF27, BIBREF28. None of them can handle noisy labels. Since many social media datasets are noisy, researchers have tried to build robust models BIBREF29, BIBREF30, BIBREF31. However, they treat noisy data as additional information and don't specifically handle noisy labels. A noise-aware classification model in BIBREF12 trains using data annotated with multiple labels. BIBREF32 exploited the connection of users and noisy labels of sentiments in social networks. Since the two works use multiple-labeled data or users' information (we only use single-labeled data, and we do not use any additional information), they have different settings than ours.
Our work is closely related to DNNs based approaches to learning with noisy labels. DNNs based approaches explored three main directions: (1) training DNNs on selected samples BIBREF33, BIBREF34, BIBREF35, BIBREF17, (2) modifying the loss function of DNNs with regularization biases BIBREF5, BIBREF36, BIBREF37, BIBREF38, BIBREF39, BIBREF40, and (3) plugging an extra layer into DNNs BIBREF14, BIBREF41, BIBREF15, BIBREF16. All these approaches were proposed for image classification where training images were corrupted with noisy labels. Some of them require noise rate to be known a priori in order to tune their models during training BIBREF37, BIBREF17. Our approach combines direction (1) and direction (3), and trains two networks jointly without knowing the noise rate. We have used five latest existing methods in our experiments for SSC. The experimental results show that they are inferior to our proposed method. In addition, BIBREF42, BIBREF43, BIBREF44, BIBREF45, BIBREF46, and BIBREF47 studied weakly-supervised DNNs or semi-supervised DNNs. But they still need some clean-labeled training data. We use no clean-labeled data.
Our model builds on CNN BIBREF25. The key idea is to train two CNNs alternately, one for addressing the input noisy labels and the other for predicting `clean' labels. The overall architecture of the proposed model is given in Figure FIGREF2. Before going further, we first introduce a proposition, a property, and an assumption below.
Proposition 1 Noisy labels are flipped from clean labels by an unknown noise transition matrix.
Proposition UNKREF3 is reformulated from BIBREF16 and has been investigated in BIBREF14, BIBREF15, BIBREF41. This proposition shows that if we know the noise transition matrix, we can use it to recover the clean labels. In other words, we can put noise transition matrix on clean labels to deal with noisy labels. Given these, we ask the following question: How to estimate such an unknown noise transition matrix?
Below we give a solution to this question based on the following property of DNNs.
Property 1 DNNs tend to prioritize memorization of simple instances first and then gradually memorize hard instances BIBREF3.
BIBREF13 further investigated this property of DNNs. Our setting is that simple instances are sentences of clean labels and hard instances are those with noisy labels. We also have the following assumption.
Assumption 1 The noise rate of the training data is less than $50\%$.
This assumption is usually satisfied in practice because without it, it is hard to tackle the input noisy labels during training.
Based on the above preliminaries, we need to estimate the noisy transition matrix $Q\in \mathbb {R}^{c\times c}$ ($c=2$ in our case, i.e., positive and negative), and train two classifiers $\ddot{y}\sim P(\ddot{y}|x,\theta )$ and $\widehat{y}\sim \ P(\widehat{y}|x,\vartheta )$, where $x$ is an input sentence, $\ddot{y}$ is its noisy label, $\widehat{y}$ is its `clean' label, $\theta $ and $\vartheta $ are the parameters of two classifiers. Note that both $\ddot{y}$ and $\widehat{y}$ here are the prediction results from our model, not the input labels. We propose to formulate the probability of the sentence $x$ labeled as $j$ with
where $P(\ddot{y}=j|\widehat{y}=i)$ is an item (the $ji$-th item) in the noisy transition matrix $Q$. We can see that the noisy transition matrix $Q$ is exploited on the `clean' scores $P(\widehat{y}|x,\vartheta )$ to tackle noisy labels.
We now present our model NetAb and introduce how NetAb performs Eq. (DISPLAY_FORM6). As shown in Figure FIGREF2, NetAb consists of two CNNs. The intuition here is that we use one CNN to perform $P(\widehat{y}=i|x,\vartheta )$ and use another CNN to perform $P(\ddot{y}=j|x,\theta )$. Meanwhile, the CNN performing $P(\ddot{y}=j|x,\theta )$ estimates the noise transition matrix $Q$ to deal with noisy labels. Thus we add a transition layer into this CNN.
More precisely, in Figure FIGREF2, the CNN with a clean loss performs $P(\widehat{y}=i|x,\vartheta )$. We call this CNN the A-network. The other CNN with a noisy loss performs $P(\ddot{y}=j|x,\theta )$. We call this CNN the Ab-network. Ab-network shares all the parameters of A-network except the parameters from the Gate unit and the clean loss. In addition, Ab-network has a transition layer to estimate the noisy transition matrix $Q$. In such a way, A-network predict `clean' labels, and Ab-network handles the input noisy labels.
We use cross-entropy with the predicted labels $\ddot{y}$ and the input labels $y$ (given in the dataset) to compute the noisy loss, formulated as below
where $\mathbb {I}$ is the indicator function (if $y\!==\!i$, $\mathbb {I}\!=\!1$; otherwise, $\mathbb {I}\!=\!0$), and $|\ddot{S}|$ is the number of sentences to train Ab-network in each batch.
Similarly, we use cross-entropy with the predicted labels $\widehat{y}$ and the input labels $y$ to compute the clean loss, formulated as
where $|\widehat{S}|$ is the number of sentences to train A-network in each batch.
Next we introduce how our model learns the parameters ($\vartheta $, $\theta $ and $Q$). An embedding matrix $v$ is produced for each sentence $x$ by looking up a pre-trained word embedding database (e.g., GloVe.840B BIBREF48). Then an encoding vector $h\!=\!CNN(v)$ (and $u\!=\!CNN(v)$) is produced for each embedding matrix $v$ in A-network (and Ab-network). A sofmax classifier gives us $P(\hat{y}\!=\!i|x,\vartheta )$ (i.e., `clean' sentiment scores) on the learned encoding vector $h$. As the noise transition matrix $Q$ indicates the transition values from clean labels to noisy labels, we compute $Q$ as follows
where $W_i$ is a trainable parameter matrix, $b_i$ and $f_i$ are two trainable parameter vectors. They are trained in the Ab-network. Finally, $P(\ddot{y}=j|x,\theta )$ is computed by Eq. (DISPLAY_FORM6).
In training, NetAb is trained end-to-end. Based on Proposition UNKREF3 and Property UNKREF4, we pre-train A-network in early epochs (e.g., 5 epochs). Then we train Ab-network and A-network in an alternating manner. The two networks are trained using their respective cross-entropy loss. Given a batch of sentences, we first train Ab-network. Then we use the scores predicted from A-network to select some possibly clean sentences from this batch and train A-network on the selected sentences. Specifically speaking, we use the predicted scores to compute sentiment labels by $\arg \max _i \lbrace \ddot{y}=i|\ddot{y}\sim P(\ddot{y}|x,\theta )\rbrace $. Then we select the sentences whose resulting sentiment label equals to the input label. The selection process is marked by a Gate unit in Figure FIGREF2. When testing a sentence, we use A-network to produce the final classification result.
In this section, we evaluate the performance of the proposed NetAb model. we conduct two types of experiments. (1) We corrupt clean-labeled datasets to produce noisy-labeled datasets to show the impact of noises on sentiment classification accuracy. (2) We collect some real noisy data and use them to train models to evaluate the performance of NetAb.
Clean-labeled Datasets. We use three clean labeled datasets. The first one is the movie sentence polarity dataset from BIBREF19. The other two datasets are laptop and restaurant datasets collected from SemEval-2016 . The former consists of laptop review sentences and the latter consists of restaurant review sentences. The original datasets (i.e., Laptop and Restaurant) were annotated with aspect polarity in each sentence. We used all sentences with only one polarity (positive or negative) for their aspects. That is, we only used sentences with aspects having the same sentiment label in each sentence. Thus, the sentiment of each aspect gives the ground-truth as the sentiments of all aspects are the same.
For each clean-labeled dataset, the sentences are randomly partitioned into training set and test set with $80\%$ and $20\%$, respectively. Following BIBREF25, We also randomly select $10\%$ of the test data for validation to check the model during training. Summary statistics of the training, validation, and test data are shown in Table TABREF9.
Noisy-labeled Training Datasets. For the above three domains (movie, laptop, and restaurant), we collected 2,000 reviews for each domain from the same review source. We extracted sentences from each review and assigned review's label to its sentences. Like previous work, we treat 4 or 5 stars as positive and 1 or 2 stars as negative. The data is noisy because a positive (negative) review can contain negative (positive) sentences, and there are also neutral sentences. This gives us three noisy-labeled training datasets. We still use the same test sets as those for the clean-labeled datasets. Summary statistics of all the datasets are shown in Table TABREF9.
Experiment 1: Here we use the clean-labeled data (i.e., the last three columns in Table TABREF9). We corrupt the clean training data by switching the labels of some random instances based on a noise rate parameter. Then we use the corrupted data to train NetAb and CNN BIBREF25.
The test accuracy curves with the noise rates [0, $0.1$, $0.2$, $0.3$, $0.4$, $0.5$] are shown in Figure FIGREF13. From the figure, we can see that the test accuracy drops from around 0.8 to 0.5 when the noise rate increases from 0 to 0.5, but our NetAb outperforms CNN. The results clearly show that the performance of the CNN drops quite a lot with the noise rate increasing.
Experiment 2: Here we use the real noisy-labeled training data to train our model and the baselines, and then test on the test data in Table TABREF9. Our goal is two fold. First, we want to evaluate NetAb using real noisy data. Second, we want to see whether sentences with review level labels can be used to build effective SSC models.
Baselines. We use one strong non-DNN baseline, NBSVM (with unigrams or bigrams features) BIBREF23 and six DNN baselines. The first DNN baseline is CNN BIBREF25, which does not handle noisy labels. The other five were designed to handle noisy labels.
The comparison results are shown in Table TABREF12. From the results, we can make the following observations. (1) Our NetAb model achieves the best ACC and F1 on all datasets except for F1 of negative class on Laptop. The results demonstrate the superiority of NetAb. (2) NetAb outperforms the baselines designed for learning with noisy labels. These baselines are inferior to ours as they were tailored for image classification. Note that we found no existing method to deal with noisy labels for SSC. Training Details. We use the publicly available pre-trained embedding GloVe.840B BIBREF48 to initialize the word vectors and the embedding dimension is 300.
For each baseline, we obtain the system from its author and use its default parameters. As the DNN baselines (except CNN) were proposed for image classification, we change the input channels from 3 to 1. For our NetAb, we follow BIBREF25 to use window sizes of 3, 4 and 5 words with 100 feature maps per window size, resulting in 300-dimensional encoding vectors. The input length of sentence is set to 40. The network parameters are updated using the Adam optimizer BIBREF49 with a learning rate of 0.001. The learning rate is clipped gradually using a norm of 0.96 in performing the Adam optimization. The dropout rate is 0.5 in the input layer. The number of epochs is 200 and batch size is 50.
This paper proposed a novel CNN based model for sentence-level sentiment classification learning for data with noisy labels. The proposed model learns to handle noisy labels during training by training two networks alternately. The learned noisy transition matrices are used to tackle noisy labels. Experimental results showed that the proposed model outperforms a wide range of baselines markedly. We believe that learning with noisy labels is a promising direction as it is often easy to collect noisy-labeled training data.
Hao Wang and Yan Yang's work was partially supported by a grant from the National Natural Science Foundation of China (No. 61572407).

Figure and table captions:
Figure 1: The proposed NETAB model (left) and its training method (right). Components in light gray color denote that these components are deactivated during training in that stage. (Color online)
Table 1: Summary statistics of the datasets. Number of positive (P) and negative (N) sentences in (noisy and clean) training data, validation data, and test data. The second column shows the statistics of sentences extracted from the 2,000 reviews of each dataset. The last three columns show the statistics of the sentences in three clean-labeled datasets, see “Clean-labeled Datasets”.
Table 2: Accuracy (ACC) of both classes, F1 (F1 pos) of positive class and F1 (F1 neg) of negative class on clean test data/sentences. Training data are real noisy-labeled sentences.
Figure 2: Accuracy (ACC) on clean test data. For training, the labels of clean data are flipped with the noise rates [0, 0.1, 0.2, 0.3, 0.4, 0.5]. For example, 0.1means that 10% of the labels are flipped. (Color online)

Questions about this paper:
What is the performance of the model?

Answers:
Experiment 1: ACC around 0.5 with 50% noise rate in worst case - clearly higher than baselines for all noise rates
Experiment 2: ACC on real noisy datasets: 0.7 on Movie, 0.79 on Laptop, 0.86 on Restaurant (clearly higher than baselines in almost all cases)