Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding
Abstract
Dialogue understanding tasks often necessitate abundant annotated data to achieve good performance and that presents challenges in low-resource settings.
To alleviate this barrier, we explore few-shot data augmentation for dialogue understanding by prompting large pre-trained language models and present a novel approach that iterates on augmentation quality by applying weakly-supervised filters.
We evaluate our methods on the emotion and act classification tasks in D AILY D IALOG and the intent classification task in F ACEBOOK M ULTILINGUAL T ASK -O RIENTED
D IALOGUE .
Models fine-tuned on our augmented data mixed with few-shot ground truth data are able to approach or surpass existing state-of-the-art performance on both datasets.
For D AILY D IALOG specifically, using 10% of the ground truth data we outperform the current state-of-the-art model which uses 100% of the data.
Introduction & Related Work
Most common ways of automatic data augmentation in natural language tasks include simple perturbations [Wei and Zou, 2019, Karimi et al., 2021, Xie et al., 2020] and generative approaches [Kim et al., 2021, Sahu et al., 2022, Edunov et al., 2018].
However, these methods do not utilize intersentential context, which is essential to encode for both dialogue understanding and generation.
On the other hand modern pre-trained language models (PLMs) can be prompted to complete dialogues using prefix prompts [Liu et al., 2021] which naturally encode conversational context.
PLMs also have shown impressive zero- and few-shot capabilities [Brown et al., 2020, Bommasani et al., 2021] in dialogue tasks and have been successfully used in generative augmentation frameworks for tasks such as intent classification [Sahu et al., 2022, Li et al., 2021], commonsense reasoning
[Yang et al., 2020], and response generation [Kulhánek et al., 2021, Gao et al., 2020b].
Several studies examine in-context learning, which involves including training examples as part of a prompt [Wei et al., 2022, Min et al., 2022, Chen et al., 2022, Lu et al., 2022].
In this work, we take the first step towards applying few-shot prompting to augmenting dialogue datasets.
We focus on low-resourcesettings, contributing an empirical account of augmenting turn-level dialogue understanding tasks using discrete prompting which encodes dialogue history as in-context examples.
One challenge with zero- and few-shot prompting with PLMs is that the outputs may exhibit more diversity than one would expect for a specific task, which confounds model training [Perez et al., 2021, Zhao et al., 2021].
Specifically, PLMs often synthesize data points which lie outside of the data
7/15/22, 5:20 PM
Weak Supervision.drawio 1/
(neutral)
Alice: You're going to set up your own law oﬃce, aren't you?
(neutral)
Bob
: Yes.
After so many years of hard work, I'd rather I had an oﬃce of my own.
(happy)
Alice:
If you need help, don't hesitate to ask me.
(happy)
Bob: I'll be very glad if you would help.
(happy)
Alice: I'd like to wish you every success in your new venture.
(happy)
Bob: Thank you.
I wish I would.
(happy)
Alice:
Generated Responses: 1.
Good luck to you.
Let's do lunch soon, Bob. 2.
It's such a rare pleasure to meet such an ideal partner in your work.
You know, you seem quite diﬀerent.
manifold 2 of a given task, instead following the distribution of the generic pretraining corpora. Due to their distance from the target task’s distribution, these augmented samples may be considered low quality.
We thus propose W EAK DAP (Weakly supervised Data Augmentation through Prompting), a framework that iteratively improves the quality of augmented data in dialogue classification tasks by introducing a weakly supervised labeler to filter prospective data points.
Figure 1 demonstrates
W EAK DAP filtering out a low-quality synthetic utterance.
We demonstrate the effectiveness of
W EAK DAP on emotion and dialogue act classification in D AILY D IALOG [Li et al., 2017], showing on-par or better performance compared to state-of-the-art full-shot results by augmenting only 10% of the original data.
We additionally examine the robustness of W EAK DAP using a separate task: cross-lingual augmentation for Spanish intent detection in FBTOD [Schuster et al., 2019].
Data Augmentation Methods
Our approach consists of two parts: prompting PLMs using dialogue context, and applying weak supervision to improve prompt-augmented datasets.
2.1 Constructing Dialogue Prompts
Dialogue contexts can be used to form prefix prompts which serve as the input to a PLM 3 .
We augment the data by replacing dialogue turns, which are selected using the dialogue context construction strategies below.
We illustrate specific examples of each in Figure 2 and Section E in the Appendix.
Each generated utterance can be prescribed a randomly sampled or ground truth reference label.
Conversation Trajectory Augmentation ( CTA ).
Taking the first turn from each speaker in the data as context, we iteratively replace the next turn with a generated utterance, and autoregressively use the generated utterances as context for the next turn.
All-Turn Augmentation ( ATA ).
ATA iteratively replaces each turn in the conversation with a generated utterance, but uses the ground truth context instead of the generated context.
Last-Turn Augmentation ( LTA ).
This is a special case of ATA where we simply choose the last turn of the conversation to replace with a generated utterance.
This results in the largest conversational context, helping guide the conditional output closer to the ground truth language manifold.
2.2 Augmentation with Weak Supervision
While prompting large PLMs provides a convenient, powerful way to bridge the gap between inadequate training data and data-hungry conversational models, there is a caveat: those PLMs are trained on generic corpora ( i.e. , web crawls, books, etc .), whose distribution may considerably differ
7/8/22, 2:51 PM
Weak Supervision.drawio 1/ [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
from the data needed to train task-specific models ( e.g. , see Figure 5). This motivates post-hoc adjustments to make our prompted augmentations more task-aware.
Weak supervision has been proposed for finding a “useful representation” for a task [Robinson et al., 2020].
Intuitively, naive prompted augmentations are less potent because they lack task-knowledge 4 , which can be distilled from ground-truth (“gold”) samples by training an auxiliary model.
We can then use that model to filter out inconsistent generated utterances.
We propose W EAK DAP, a framework generalizeable to any prompt-based augmentation task.
In this work, we prompt GPT-J 6B
[Wang and Komatsuzaki, 2021] and the Alexa Teacher Model (ATM) 20B [Soltan et al., 2022].
As Figure 3 illustrates, W EAK DAP consists of three parts.
We first augment the “gold” data and train a task classifier on the gold and “silver” data.
Then, we iteratively re-augment the data and re-train the classifier.
For the augmentation step on each iteration, we use the classifier trained during the previous iteration to create a weak silver label for each generated instance, and filter out instances where the silver label does not match the prescribed label with high confidence, i.e., low entropy.
We reason that data points which a weak labeler thinks are labeled incorrectly with low confidence could still be useful for learning during training (further discussion in Section G in the Appendix).
Moreover, this indicates that their labels may be in fact be correct.
To this end, we filter out incorrect instances classified in the bottom 80th percentile of entropy, computed as in the equation below, where C is the number of classes and p i is the probability of class i .
[Equation]
This weakly guarantees that the generated data is not of low-quality.
This continues until the classifier’s performance doesn’t improve by at least ϵ for k rounds.
Here, we fix ϵ = 0 .
005 , k = 3 .
Other Task-Aware Augmentation Approaches.
Similar task-aware generative augmentation approaches typically distill task-knowledge into the generator.
Yang et al.
[2020] proposes augmentation for commonsense reasoning by fine-tuning two generators (for answering and distracting) and relabelling synthetic data points using a task model, while Papangelis et al.
[2021] fine-tunes a generator using reinforcement learning.
With large PLMs, these methods are costly and less practical.
While few-shot prompting is a cheaper solution, it is less effective at encoding lots of task knowledge, as
7/15/22, 3:57 PM
Weak Supervision.drawio 1/
[Figure]
in-context example capacity is limited.
W EAK DAP bridges the gap between prompt-based augmentation with little task-knowledge and complex mechanisms with higher computational costs; it does not need to fine-tune the generator, as we prompt it using dialogue context as in-context utterance examples.
Experiments
We benchmark various augmentation methods on the classification tasks in D AILY D IALOG , a highquality open-domain dialogue dataset, and the intent detection task of FBTOD, a task-oriented dialogue dataset (dataset details in Figure C).
D AILY D IALOG Emotion Classification We first conduct a thorough evaluation of our augmentation methods using the emotion classification task in D AILY D IALOG as a case study, in the full and few-shot settings 6 .
For our augmentation model, we use GPT-J 6B
[Wang and Komatsuzaki, 2021], which is one of the largest causal language models publicly available, and has been able to achieve performance competitive to GPT-3 on many tasks [Wang, 2021, Black et al., 2022].
For all D AILY D IALOG experiments we use the Speaker Turn Model (STM)
[He et al., 2021], a RoBERTa [Liu et al., 2019]-based classification model with speaker turn awareness 8 , as the classification task model and weak labeler.
There are seven emotion labels: neutral, anger, disgust, fear, happiness, sadness , and surprise .
Each label is a rich, descriptive token on its own, so in constructing a prompt, we directly use it as an adjective (e.g., “Alice in a happy mood:”).
Additionally, we conjecture that directly using conversation history forms the best set of in-context examples to generate utterances which convey a prescribed emotion while remaining within the gold data manifold.
Example prompts provided in Figure 4 ( LTA ) and Section E in the Appendix ( ATA , CTA ).
In Figure 5, we compare the data manifold of the synthetic data resulting from LTA and CTA with a maximum augmentation size of 2x that of the amount of original data used, and ATA which results in a size between 5 .
7 x and 6 .
2 x (see Appendix Table 3).
We can see that CTA results in a separate cluster of data, likely due to the underlying distribution of GPT-J’s pretraining data.
ATA ’s distribution has a clear distinct cluster as with CTA , but also posseses some overlap with the training distribution.
In contrast, we see that synthetic data from LTA lies within the training data manifold.
This is likely due to the in-distribution [Figure] context provided as conditional input to GPT-J.
We hypothesize that this context most closely guides synthetic data towards the original data distribution.
This may be more beneficial in tasks where we do not expect distribution shift from training to inference.
In tasks such as response generation, where diverse output is desirable, it may be more beneficial to have training data that falls into an expanded manifold as with CTA or ATA .
Comparing the three methods, we find that CTA and ATA (Appendix
Figure 6, Table 3) are beneficial, but underperform LTA (Figure 4) for this task.
Thus, we primarily experiment using LTA .
In low-resource settings, it is important to quantify how much ground truth data is available.
When augmenting seed data, it is essential to quantify how much data is being generated (i.e., by providing a Size Multiplier relative to the amount of seed data used; Feng et al.
Figure 4 shows the performance of STM in each data setting with varying amounts of synthetic data resulting from LTA .
Following existing work, we report micro F1 ignoring the majority label, due to a heavy imbalance towards the neutral emotion [Liang et al., 2021, Lee and Choi, 2021, Wang et al., 2020].
We see that
STM with augmented 10% data reaches an F1 score of 0 . 686 and 0 .
70 with augmented full-shot data, surpassing the existing state-of-the-art of 0 .
641 set by S+PAGE
[Liang et al., 2021].
We observe that adding too much augmented data eventually hurts performance in each setting, further suggesting that prompting alone has inconsistent quality.
2x is the best performing multiplier and we use that to conduct all other experiments.
Figure 4 also shows that W EAK DAP improved STM’s performance in each few-shot setting.
On average, the labeler reduced the augmented data size from 2x to 1.8x, indicating that it may foster more efficient learning.
Prior work has not found large differences in performance between backtranslation and other perturbation methods despite higher computational overhead [Xie et al., 2020, Kim et al., 2021].
Thus, we primarily compare against EDA [Wei and Zou, 2019] 9 and AEDA [Karimi et al., 2021] 10 as noise injection baselines.
As seen in Figure 4, using noise perturbed data underperforms normal data, an outcome corroborated by Kumar et al.
[2020], Chen et al.
Finally, we examined the importance of context by experimenting with a standard in-context learning prompt with 10 randomly sampled utterances of the same emotion label [Sahu et al., 2022, Brown et al., 2020], finding that dialogue context consistently outperforms random sampling (e.g. 0.624 versus 0.600 F1 using 5% data).
D AILY D IALOG Act
Classification There are four dialogue act labels: inform, question, directive , and commissive .
While these individual terms are less descriptive than emotion labels, they form descriptive tokens if used as active verbs (e.g., “Alice directs Bob:”).
See Section E in the Appendix for an example.
D AILY D IALOG
[Table]
FBTOD ES
Acc LR
F1 LR Acc HR F1 HR
[Table]
(fewshot matching the Spanish percentage) English Training data.
Following our findings on Emotion Classification, we augment the few-shot D AILY D IALOG Act
Classification task with LTA while providing full-shot performance for reference. Our results in Table 1 indicate that W EAK DAP results in the highest accuracy across all few-shot settings, although using STM on the unaugmented data achieves the highest F1 in the 5% setting. Noticeably, the performance improvements on the act classification are not as drastic as in the emotion classification task.
This is likely because the baseline performance on the act classification task was already competitive.
Cross-lingual Augmentation To assess the robustness of W EAK DAP, we additionally evaluated its performance on a completely different setting: low-resource cross-lingual augmentation for intent classification on FBTOD.
For augmentation, we used ATM [Soltan et al., 2022], a 20B multilingual seq2seq language model pre-trained on a text de-noising objective similar to Liu et al.
[2020] using large publicly available datasets including mC4 [Xue et al., 2021] and Wikipedia.
For intent classification, we fine-tuned XLMRoBERTa (XLMR; Conneau et al.
Full-shot fine-tuning of XLMR yields an accuracy of 98 .
8% on the Spanish test set, however we focus on the few-shot Spanish setting, with few- and full-shot cross-lingual augmentation from English.
For all few-shot datasets, we use 1% , 5% , and 10% of the original training data for that language (see Section F).
FBTOD is a single-turn dataset, so we adapt the in-context learning prompt in Sahu et al.
[2022], mixing the reference Spanish instance with randomly sampled English instances of the same label for in-context learning (example in Section E in the Appendix).
We do not examine Thai because the model has not been pre-trained on Thai data.
We augment the few-shot Spanish setting using both low-resource and high-resource English data.
With low-resource English, we use the same percentage of data as the ground-truth Spanish data.
Table 2 indicates that fine-tuning XLMR on W EAK DAP outperforms prompt-based augmentation in all settings both in terms of accuracy and F1 score.
Conclusion
We contribute significant progress towards few-shot prompt-based augmentation for dialogue tasks.
We introduce W EAK DAP and demonstrate its augmentation quality filtering capabilities by surpassing full-shot state-of-the-art performance with few-shot examples on D AILY D IALOG and achieving strong few-shot performance on FBTOD.
In the future, we will examine ways to integrate soft prompting into W EAK DAP as well as identify an appropriate feedback mechanism for response generation.
A Ethical Considerations
Pre-trained Language Model Biases In this work, we directly use only two datasets: D AILYD IALOG and FBTOD.
However, large pre-trained language models like GPT-J have already been pre-trained on massive corpora such as The Pile [Gao et al., 2020a], which is a webcrawl of much of the internet.
While this promotes diverse language, this also provides no guarantee over the types of content that the model is capable of producing.
It is possible that the model could generate negative, hurtful, or offensive content [Gao et al., 2020a, Christian, 2020].
Prompting methods as the ones proposed in this work fundamentally rely on large pre-trained language models.
As such, it is possible that this hurtful content could leak into augmented datasets, if left unchecked.
While in this work we only discuss dialogue understanding tasks, this may present a bigger issue in dialogue generation where hurtful utterances may actually be relayed to users.
However, it should be noted that in our experiences, this has not been an issue because we only generate one synthetic utterance at a time.
This does not overly grant the language model freedom in any particular call.
Data Biases In a similar vein, every dataset, including D AILY D IALOG and FBTOD has its own biases.
While introducing language model output from another language manifold (i.e., a pre-trained language model’s training corpora) will help to lessen some of the biases present in unaugmented datasets, generation conditioned solely on existing dialogue contexts may continue to reinforce some of these existing biases.
Reproducibility Upon acceptance, we plan to release all of the augmentation code, as well as the seed data (i.e. each few-shot setting) used in all experiments.
B Limitations
In this work, we consider two different dialogue contexts — social chit-chat in D AILY D IALOG and multi-lingual task-oriented dialogue in F ACEBOOK M ULTILINGUAL T ASK -O RIENTED D IALOGUE .
Due to computational constraints, it is difficult to consider several dialogue contexts with all of our experimental settings.
However, we believe that we present a set of experiments representative of the scope of our method’s generalizability to different dialogue understanding tasks and integratability with different pre-trained language models.
[Table]
Weak Supervision.drawio 1/
[Table]
[Figure]
Prompt Selection
Much recent research focuses on how to design prompts to optimize performance on a variety of different tasks Liu et al.
In this work, we did not apply such prompt engineering methods.
Instead, we focused on the conversational nature of prompting for augmentation in dialogue tasks.
Our prompt primarily consists of dialogue context.
Design decisions included the use of the names “Alice” and “Bob,” as well as the choice to encode instance labels in natural language form.
We did not formally evaluate these decisions, but we subjectively saw that the use of other names can yield similar performance.
We also noticed that using names generally provided better results than generic speaker tags such as “Speaker 1” and “Speaker 2.”
C Datasets
D AILY D IALOG [Li et al., 2017] is a high-quality open-domain conversation dataset.
The official
D AILY D IALOG training set contains 11,118 dialogues, while the validation and test sets each have 1,000 dialogues.
Each conversation is annotated with a topic label and has on average eight turns.
Additionally, each turn is annotated with a dialogue act and an emotion label.
Facebook Multilingual Task-Oriented Dialogue (FBTOD; Schuster et al.
[2019]) is a dataset which contains single-turn task-oriented utterances in English, Spanish, and Thai.
Each utterance is annotated with one of twelve intent labels.
D D AILY D IALOG Experiments
In order to measure and isolate the effect of prompting methods, we hold many of the experimental settings fixed.
For data generation with GPT-J, we use top- p sampling with p = 0 . 92 .
The resulting generated data is parsed from the decoded language model outputs.
All of our downstream classification experiments are performed using STM.
In order to isolate the effects of augmentation and due to computational limitations, we fix the tunable STM parameters with an initial learning rate of . 0001 ,
[-Augmented full-shot STM reaches an F
-]
7/8/22, 2:50 PM
Weak Supervision.drawio 1/
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
recurrent layers and a 50% dropout layer.
We let STM fine-tune for up to 100 epochs; we use early stopping with a patience of 10 epochs.
All experiments are implemented with PyTorch [Paszke et al., 2019] and HuggingFace’s
Transformers [Wolf et al., 2020], and run on AWS p3.16xlarge EC2 instances.
D.1 Conversation Trajectory Augmentation
In Figure 6, we see that we are able to reach state-of-the-art performance on the emotion classification task using conversation trajectory augmentation as well.
Augmented full-shot STM reaches an F score of 0.695.
In all except the 1% ground-truth setting, Conversation Trajectory Augmentation underperforms Last Turn Augmentation.
This is likely due to the fact that Conversation Trajectory
Augmentation departs from the ground-truth data distribution, in contrast to Last Turn Augmentation as displayed in Figure 5.
This is not necessarily a downside of generating new trajectories - it just indicates that it does not perform as strongly on this particular classification task.
By definition, training a model on data that closely matches the testing distribution will be advantageous during evaluation.
However, beyond classification tasks, it is possible that this diverse data distribution will be more favorable, e.g., in response and dialogue generation, where one would want to see more diverse responses.
D.2 All-Turn Augmentation
As displayed in Table 3, All-Turn Augmentation generally does not outperform either Conversation
Trajectory Augmentation nor Last-Turn Augmentation.
Moreover, we see that the resulting augmented sizes are roughly six times that of the original data.
The closest comparable augmented size used
[Figure]
[Table]
[Figure]
GPT
[Figure]
J
[Figure]
informs
[Table]
[Figure]
[Table] [Table] [Table]
[Table] [Table]
was
[Table]
[Table]
[Table]
[Table]
name, please
[Table] [Table]
[Table]
Bob questions
[Table]
[Table]
[Table]
[Table]
[Table]
Alice
[Table]
[Table]
[Table]
with Last-Turn and Conversation Trajectory Augmentation is a multiplier of five, but at that multiple, either Last Turn or Conversation Trajectory Augmentation yields the strongest performance for each data setting.
E Example Prompts
We present several examples of prompts corresponding to the different methods of conversation augmentation.
Figure 2 shows an example of Last Turn Augmentation and demonstrates that by omitting the last turn of the original conversation, we can form a prefix prompt that allows GPT-J to generate an utterance according to a prescribed emotion, whether it be the ground truth emotion from the training data or a user-defined emotion.
The ground truth turns given as context combined with the generated utterance constitute a new augmented conversation.
Figure 7 shows an example of Conversation Trajectory Augmentation.
In order to set the context for each speaker, we always include the first ground truth turn for each speaker in the context.
For each subsequent turn, we autoregressively let the generation model generate the utterance by feeding in the previously generated utterance.
This process continues until the number of turns in the new augmented conversation reaches the length of the original ground truth conversation.
We show an example of All Turn Augmentation in Figure 8.
We again include the first ground truth turn for each speaker in the conversation.
However, instead of autoregressively feeding in previously generated utterances, the context of the generated turn i is always the ground truth turns 1 through i − 1 .
Since the generated turn i is not guaranteed to form a coherent conversation when with ground truth turns i + 1 through n (where n is the length of the ground truth conversation), we consider each generated turn to be an endpoint for a new conversation. 7/8/22, 5:43 PM Weak Supervision.drawio 1/
[Figure] Finally, we show an example using Last Turn Augmentation while prescribing a D AILY D IALOG act label in Figure 9.
We see that GPT-J correctly generates an utterance that follows the act “inform,” and is even able to pick up on some of the speaker’s tendencies (e.g., using the onomatopoeia “Erm”).
F Cross-lingual Augmentation Experimental Setup
The original FBTOD dataset contains an English dataset with 30,521 training instances, 4, evaluation instances, and 8,621 testing instances.[-It also contains a Spanish dataset with 3,
 7/8/22, 5:43 PM-]
It also contains a Spanish dataset with 3, training instances, 1,983 evaluation instances, and 3,043 testing instances.
For both languages, we created versions of the training set that had 1% , 5% , and 10% of the original number of training instances.
For each percentage, we randomly sampled that proportion of training examples for each intent label.
In cases where that proportion would result in a number smaller than 1 . 0 , we ensured that there would be at least one training instance.
For augmentation, we use the prompt given in Figure 10.
We perform beam search, taking up to three sequences as augmented data and rejecting any duplicate examples.
For classification, we fine-tune
XLMRoBERTa with a maximum sequence length of 128, 80 training epochs, and an initial learning rate of 5e-5.
G Entropy-based Weak Filtering
We hypothesized that two categories of synthetic data could improve to eventual task performance.
The first is simply correct data — synthetic training instances which match the intended instance label.
However, we determine this correctness using an imperfect classifier.
Thus, the second category is “hard to learn” data.
This follows from early work [Guo and Viktor, 2004] finding that classification performance can be improved by focusing on augmenting datasets with “hard to learn” examples.
However Guo and Viktor [2004] identify such difficult examples post hoc through classifier performance.
Solely evaluating classifier performance for each example is prohibitively expensive for large models on large datasets.
Thus, in our case, we make our decisions based on uncertainty.
Out of those instances that the classifier states are incorrect, we do not filter out the ones for which the classification is made with high uncertainty, hypothesizing that uncertainty is a strong proxy for data points’ learning difficulty, which may be more useful in training the next iteration of the classifier.
There are several ways to consider how to quantify uncertainty.
Ott et al.
[2018] quantified uncertainty for neural machine translation post hoc by comparing generation methods’ sequence-level probability mass coverage of ground-truth translations.
While these methods may be appropriate for translation or even response generation-related tasks, there is no reference point for what the ideal augmented data is.
Other approaches have looked specifically at quantifying dataset uncertainty using Bayesian Neural
Networks [Xiao and Wang, 2019, Chen et al., 2020].
However, dataset level approaches are not necessarily appropriate for augmentation, where one needs to make inferences about data quality at the instance level.
Moreover, approaches requiring Bayesian Neural Networks do not achieve state-of-the-art performance on the D AILY D IALOG classification tasks.
On the other hand, entropy maximization has long been used as an information theoretic technique for maximizing uncertainty [Seidenfeld, 1986, Wang, 2008].
Even in natural language processing,
Csáky et al.
[2019] performs data filtering using entropy to improve diversity by specifically removing generic data points.
They achieve promising results in dialogue generation, and moreover, entropy computation is directly applicable at the instance level.
Therefore, we adopt entropy into our framework to identify uncertain and difficult data points.