Contrastive Novelty Learning: Anticipating Outliers with Large Language Models
Authors: Albert Xu
Xiang Ren
Robin Jia
Venue: Submitted to ICLR 2023 (2023)
Keywords: selective prediction
open-set classification
large language models
NLP

Abstract: In many task settings, text classification models are likely to encounter examples from novel classes on which they cannot predict correctly. Selective prediction, in which models abstain on low-confidence examples, provides a possible solution, but existing models are often overly confident on OOD examples. To remedy this overconfidence, we introduce Contrastive Novelty Learning (CNL), a two-step method that generates OOD examples representative of novel classes, then trains to decrease confidence on them. First, we generate OOD examples by prompting a large language model twice: we prompt it to enumerate novel classes relevant to the label set, then generate examples from each novel class matching the task format. Second, we train our classifier with a novel contrastive objective that encourages lower confidence on generated OOD examples than training examples. When trained with CNL, classifiers improve in their ability to detect and abstain on OOD examples over prior methods by an average of 2.3% AUAC and 5.5% AUROC across 4 NLP datasets, with no cost to in-distribution accuracy.

Under review as a conference paper at ICLR 2023
CONTRASTIVE NOVELTY LEARNING: ANTICIPATING OUTLIERS WITH LARGE LANGUAGE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
In many task settings, text classification models are likely to encounter examples from novel classes on which they cannot predict correctly. Selective prediction, which models abstain low-confidence examples, provides possible solution, but existing models are often overly confident on OOD examples. To remedy this overconfidence, we introduce Contrastive Novelty Learning (CNL), two-step method that generates OOD examples representative of novel classes, then trains decrease confidence them. First, we generate OOD examples by prompting a large language model twice: we prompt it to enumerate novel classes relevant the label set, then generate examples from each novel class matching the task format. Second, we train our classifier with novel contrastive objective that encourages lower confidence on generated OOD examples than training examples. When trained with CNL,, classifiers improve in their ability to detect and abstain on OOD examples over prior methods by an average of 2.3% AUAC and 5.5% AUROC across 4 NLP datasets, with no cost to in-distribution accuracy
INTRODUCTION
Recent progress in NLP has led to text classification models that are accurate not only in-distribution, but also on some out-of-domain data (Arora et al. 2021). Nonetheless, some categories of real- world distribution shift still pose serious challenges. For instance, in open-set label shift, the test data includes examples from novel classes not present in the training data, making it impossible for standard classifier predict correctly (Scheirer et al. 2013)) Moreover, novel class examples can be difficult to detect with conventional OOD detection methods, as they typically bear strong surface resemblance to training examples (Tifrea et al.) (02T)). In this paper, we frame open- set label shift as selective prediction problem Yan- & Wiener 20100) Geifman & El- El-Yaniv] (0017) that we call open-set selective classification (OSSC), OSSC requires text classifiers to predict correctly on set examples while abstaining novel class examples.
To perform well on OSSC, a classifier must have lower confidence on novel class examples than closed-set examples by learning features which differentiate novel classes from closed-set classes (Perera et al., 2020). In order to supervise this representation learning, it is useful to identify what examples from novel classes might look like. Prior work has explored automatically generat- ing OOD images by adding random perturbations to ID examples (Setlur et al., 2022). Text inputs, however, are composed of discrete tokens, and modifying even a single token can unpredictably alter the meaning of a sentence. We seek an automatic generation method that addresses these limi- tations, leveraging the generative ability of large language models (LLMs) like GPT-3 (Brown al., 2020). LLMs are a desirable source for novelty, as their generation is informed by a broad corpus of examples seen during pretraining, allowing them to reliably generate from classes outside a dataset.
We present Contrastive Novelty Learning (CNL), method to improve the OSSC ability of clas- sifier by automatically generating OOD examples and then training to abstain on them. To generate diverse set of OOD examples that anticipate different potential test-time shifts, we introduce Nov- elty Prompting, a method that augments a source dataset with novel class examples generated by a LLM. We first perform label generation, prompting our LLM to extend the closed-set labels with novel labels. We then prompt the LLM to generate new examples conditioned on each novel label
Cooe and data have heen uploaded and will he released.

---

Under review as a conference paper at ICLR 2023
categoris:
Yankees Unlikely to Get DD-Rays...
Sports World Business
Kyria Irving returns to play
World sparti, Staamee Laak
000 1ick
Pio
food
Ecample Comenstio
corrasponding
A new from the Canadian Food...
Sports World Business
The Cartwhrel Galary the...
Dyun
Minerneet
Novelty Prompting
CCL Training
Selective Prediction
Contrastive Novelty Learning pipeline. We Novelty Prompt a generator model to produce novel set DOoD, then train with contrastive confidence loss (CCL) our original train set DID and DOOD ensuring that our classifier is less confident on generated novel examples than closed-set examples. Finally, we abstain based on model logits.
Figure/ I:
to form a large set of probable novel examples. Finally, we propose contrastive confidence loss (CCL) for training, which encourages both high accuracy on the ID training set and lower relative confidence on the generated novel examples. We show that CCL outperforms stricter losses like Outlier Exposure (Hendrycks et al. 2019), which can adversely affect ID performance. Our full pipeline is shown in Figure Our method can be viewed as form of "partial" knowledge dis- tillation: we leverage an LLM "teacher model" to improve novelty detection performance without altering the student model's strong IDD classification ability.
We evaluate CNL against state-of-the-art baselines across 14 splits of datasets (AGNews (Zhang et al. 2015), TREC-10 (Li & Roth 2002)) TACRED (Zhang al., 2017). Emotion (Saravia al. 2018 ). finding that it improves both OOD detection and OSSC, by an average of 5.5% AUROC and 2.3% AUAC over the best prior method. These improvements come no cost to ID accuracy, demonstrating that is possible to distill novelty detection alone without affecting predictive power. Finally, we analyze the settings in which CNL can improve OSSC performance. In the data dimen- sion, scale is often optional: with as few as 1000 generated examples, we find an improvement over vanilla training on 4 datasets. The same is only partially true for LLM size, as on some datasets only sufficiently large model can generate useful examples.
2
PROBLEM SETTING
2.1 OPEN-SET SELECTIVE CLASSIFICATION
In standard classification, an optimal model f should predict the ground-truth label y of an input example x from a closed set of known labels ViD. However, under a more realistic open-set setting. some test examples are drawn from unknown novel classes VoOD- Without priori knowledge of VOOD, a standard discriminative classifier will never correctly classify a novel example. Instead, an optimal open-set seleetive classifier f should predict y when y VID. and abstain otherwise.
For probabilistic model po(y x) and associated confidence metric, the corresponding prediction is given by fxx) (Ú, c). where ý arg maxye Po x)) and e denotes the model's confidence. When used as a selective classifier with threshold 7, f predicts ÿ when c > y and abstains other- wise (Geifman & El- Yaniy 2017]. This differs from OOD detection (Hendrycks & Gimpel 2017) in that must abstain on both novel examples and its own errors and must attain high ID accuracy.
2.2 EVALUATION PROTOCOL.
We holistically measure selective classification performance with the area under the accuracy- coverage curve (AUAC). The accuracy-coveraal curve plots accuracy as a function of the fraction of examples on which the model predicts (i.e., coverage) the confidence threshold y varies. For curacy computation, we treat predictions on all novel class examples as incorrect. AUAC measures the combined ability of model in ID classification accuracy, IID calibration, and OOD detection.
2

---

Under review as a conference paper at ICLR 2023
Though we deviate from prior work in reporting AUAC, to demonstrate that our method is still effective in OOD detection, we additionally compute the Area under the ROC curve (AUROC). AUROC measures model's 's abilitt detect when test example x is of a novel class (y E YOOD). Higher is better: 50% AUROC is random, and 100% is perfect.
3 METHOD: CONTRASTIVE NOVELTY LEARNING
Here we describe Contrastive Novelty Learning, a method for automatically improving OSSC. At high level, generate novel examples and then train to ensure that model is less confident on generated novel examples than closed-set examples. We first describe desiderata for useful nov- elty, then introduce two-phased novel example generation method, Novelty Prompting, and finally introduce a contrastive confidence loss for classifier training. We illustrate the method in Figure I
3.1 NOVELTY PROMPTING
Desiderata of Novelty Generation Inspired by previous work which utilize known, representative
OOD data to train selective prediction and OOD detection models (Kamath al. 2020 Hendrycks et al. 2019), we focus on creating an generated "novel set" that is representative of potential label shifts at test time. The "novel set" must be (1) plausible, meaning that it should bear surface resemblance to the training data, e.g., we should create news examples for a news dataset, and (2) semantically novel, meaning that these examples should be from new classes. For example, selecting data from an entirely separate dataset, as is done in Hendrycks et al. (2019), violates plausibility. Meanwhile simply editing surface features or recombining examples as is done in mixup (Zhang et al. 2018) might induce a distribution shift but would not result in semantic novelty.
To satisfy these desiderata, we propose a two-stage generation method called Novelty Prompting (NP). To encourage semantic novelty, we first generate novel labels given a dataset's 'ss extant labels. We then show existing examples to a language model (to encourage plausibility) and ask to gen- erate a new example conditioned on one of the new labels. Figure shows both prompt formats.
Label Generation Though prompting with large autoregressive language models (LLMs) like GPT- 3 has typically been explored in the context of few and zero-shot learning to perform standard NLP tasks [Brown et al.] (2020), we find that LLMs are also capable of "expanding" a set of topically related concepts that might realistically co-occur via sequence continuation.
We leverage this capability to generate novel labels. We prompt the largest GPT-3 model available (Davinci) with task-speciff instruction well as the concatenation of the normalized known (Vi)) labels. By taking the union over continuations of one or more novel labels N times, we obtain a diverse "novel label set." We combine multiple completions because in preliminary experiments, we observed that single completions tend to overgenerate labels from a narrow subcategory of possible novel classes. To remedy concerns about data leakage due dataset examples of the true unknown class possibly appearing in LLM pretraining, we remove instances of the gold novel label(s) from this set. In practice, predicting the true novel test-time labels is both permissible and desirable, so our experimental setup likely underestimates our method's 's performance.
Finally, we filter out generated labels that are closely related to ID labels. For example, if joy appears in the ID labels, we remove synonyms like happiness. We use a large online thesaurus" to identify synonyms from the final novel label set. We analyze the impact of filtering Appendix [ A.
Example Generation To generate each novel example, we randomly sample novel label from our set and prompt LLM (we use GPT-J) to generate an example of that label. We prime this model with one random sampled label-example pair from each ID class in the training dataset in the prompt, resulting in 3-6 in-context examples, varying based on the dataset. Providing these context pairs ensures that our generation is plausible: the model is encouraged to generate a specific style of text. We perform this generation procedure repeatedly to form novel example set. We show the prompt use for this step in Appendix A.T and several generated label-example pairs in Figure
Though this requires some human intervention, it both (1) satisfies the true zero-shot nature of test-time label shift as it requires no knowledge of the unknown labels and (2) requires minimal effort, typically only involving converting an abhreviation label such as LOC into Location. http:://mobythheauuuss.
3

---

Under review as a conference paper at ICLR 2023
3.2 CONTRASTIVE CONFIDENCE LOss TRAINING
Our second contribution is an improved loss function for training models to have lower confidence on OOD examples than ID examples. Prior work has used the Outlier Exposure (Hendrycks et al.) 2019) objective, which encourages the model f to output a uniform probability distribution over closed-set classes when given a novel example Hendrycks et al. (2019) successfully used Outlier Exposure to train models on OOD data gathered from a different dataset (e.g., Wikitext), since there was very little risk of this data overlapping with ID data. In contrast, we automatically generate plausible novel examples, which runs the risk that some of our novel examples will inevitably be in distribution. Since Outlier Exposure encourages models have the lowest possible confidence on novel examples, it can hurt predictive accuracy when some examples x actually resemble closed-set examples. Instead, we seek a solution which treats outliers more flexibly.
We propose a novel contrastive confidence loss (CCL) that encourages models to be less confident on OOD examples than ID examples. This a less strict objective as models can achieve minimum loss without predicting a perfectly uniform distribution for the generated novel examples. For an input I, let pe 2) be the model's predicted distribution over ViD. Let cyl (2) maX (0(3) r),, the Maximum Softmax Probability (MaxProb) (Hendrycks & Gimpel, 2017), which we use as our confidence metric. Finally, let & denote the cross-entropy loss with a one-hot target vector, and DID and DOOD denote the training set and novel set respectively. We define CCL as follows:
(()) Emmannna)> Le Iia), Maa] + D Doom maa (0, co (Zood) - co (a(a))) (1)
That is, we penalize the confidence of novel examples which have higher confidence than any closed- set example. While this still induces our model learn lower confidence novel examples, it simultaneously permits our model to learn that some novel examples can be more novel than others, rather than learn minimal confidence on all members of the generated novel set. In practice, we obtain an unbiased estimate of the second term by sampling batch of n ID and n OOD examples at each step and computing the second term pairwise between each of the n² ID-OOD example pairs. We arbitrarily choose x 1.0, weighting the two terms of the objective equally.
4 EXPERIMENTAL SETUP
4.1 DATASETS
We construct artificial dataset splits from 4 popular NLP classification datasets by holding out one or more labels from training and moving all examples of that label to the test split, removing classes that are too small to yield statistical significance in our evaluations. Specifically, we evaluate a question intent detection dataset, TREC-10 10 (Li & Roth 2002) and construct 5 splits. We also evaluate two popular topic classification datasets, AGNews (Zhang et al. 2015), a news classification dataset, and Emotion (Saravia et al., 2018), tweet classification dataset. We construct 4 splits for each. Finally, we evaluate TACRED (Zhang et al. 2017), a strongly class-imbalanced sentence relation- classification dataset with large set of 41 possible relations, for which we construct single split where we hold out the 35 smallest classes. Further dataset details are outlined in Appendix A.6 Results for each dataset are averaged across all splits.
4.2 EXPERIMENTAL DETAILS
For Novelty Prompting, we perform label generation using the best available GPT-3 model, GPT-3 Davinci (Brown et al. 2020) and example generation with a smaller pretrained GPT-J 6B model Ko- matsuzaki (2021). To generate a sufficiently diverse novelty set, we perform 5 label generation iter- ations, then generate 100000 examples. We analyze the impact of generation budget in Section 5.3 We train BERT-base base classifiers with CCL for 5000 steps and batch size n = 40. On TACRED, we permit only generations containing exactly two entities, one of which is subject and the other an object, filtering out roughly 90% of generations. This is a task constraint: the relation between any other number of entities is under- overspecified. We describe further details in Appendix A.7
4

---

Under review as a conference paper at ICLR 2023
AUAC (1)
TREC-10
AGNews
Emotion
TACRED
Averuge
Vanilla
89.2
87.910.8
90.3
89.610.1
89.3
kFolden
93,5
85.8
90.610.9
84.913.5
88.7
Contrastive
92.010.4
87.010.9
92.210.4
88.810.7
90.0
CCL + Wikitext
91
88.620.6
92.0+0.4
89.3+0.5
90.3
CCL + Zero-Shot
92.520.8
89. 89.1
92.61002
88.940.4
90.8
CCL + Few-Shot
93.5
89. 89.7
93.3+0.11
90.8+0.1
91.8
OE + Wikitext
92.6+0.8
88.940.4
91.640.6
89.8+0.1
90.7
OE + Novelty Prompting
83.6+0.4
90.640.2
92.440.1
91.3+0.3
89.5
Contrastive Novelty Learning
94.3
1..3
93.4
91.110.2
92.3
CCL + Gold Label f
94,8+0.3
91.44
93.7
91.040.2
92.7
CCL + Gold Data f
96.6+0.1
93.5+00.1
94.840.2
94.340.4
94.8
OE + Gold Data
96.510.2
94.810.0
95.210.0
96.210.2
95.7
Table 1: OSSC Results of Contrastive Novelty Learning. Methods listed below CNL are upper bounds. All outlier exposure (OE) methods are trained on 100K outlier generations. We average over the results of 5 seeds of all splits and report standard error of the mean in subscript. We report macro-average of all datasets in the rightmost column. Oracle methods are marked with f. We find that CNL significantly outperforms prior methods on 3 datasets, and both the Novelty Prompting and CCL loss components important for strong performance.
4.3 BASELINES
We evaluate our method against OSSC baselines from prior work, CCL baselines with other novel sets, and Outlier Exposure (Hendrycks et al. 2019). For all methods, we train a BERT-base model and use hyperparameters from the original papers unless otherwise specified. Of the basleines, only CCL and Outlier Exposure use an explicit novel set.
Vanilla We evaluate vanilla cross-entropy loss training, calculating confidence using MaxProb.
kFolden We evaluate kFolden (Li et al. 2021), a method that trains an ensemble of individual classifiers, each trained k labels. The average of the ensemble probability distributions is used for confidence computation.
contrastive objective to induce training examples of different classes to be distant and of the same class to be near. This sparsifies the embedding space. ensuring that most OOD examples are far from feature representations of ID samples. We use the supervised constrastive loss and the Mahalanobis distance metric for confidence computation, finding that this setup performed the best on our evaluation.
Contrastive We evaluate Contrastive OOD Detection (Zhou al. 2021), which
CCL + Zero/Few-Shot Data Augmentation To measure the impact of explicitly prompting for novel labels, we generate with an identical pretrained GPT-J model, but prompt with only an in- struction and one (or zero) ID training example from each class (See Appendix A. for the specific prompt format). Essentially, we perform example generation identically but skip label generation en- tirely. We perform CCL training and MaxProb inference. While some resultant generations will be useful, we expect that many will not semantically novel, resulting in strictly worse performance.
CCL + Wikitext To measure whether plausibility of examples impacts their usefulness for CCL, use an entirely different dataset, Wikitext- 103, as our novel set. Though these examples represent distribution shift, they do not accurately reflect the open-set shift the classifier will encounter.
Outlier Exposure + Novelty Prompting We pair our novel set with Outlier Exposure (OE) Hendrrycks et al. 2019) as described in Section 3.2 and compute confidence with MaxProb.
RESULTS
5.1 OSSC RESULTS
CNL outperforms prior work. We report comparisons of CNL against baselines in Table I Broadly, find that while baselines like kFolden and Contrastive training struggle to consistently

---

Under review as a conference paper at ICLR 2023
Label
Generated Example
CEL
CURIOSITY
i am still interested but more in- terested visit the pyramids and learn more
06.5
QE
60.0
DESPAIR
love my friends but sometimes feel like im not good enough
95.0
DISAPPOINTMENT
am a human nothing is going keep me from flying away
66.5
Noaing
Figure 2: Example novel generations for Emo- tion. In this split, the gold novel label is "sad- ness". Though we remove the gold novel label before example generation, many generations are still relevant to this label. More generation exam- ples are shown in Appendix A.4
Figure 3:
Noisy novel sets hurt accuracy for OE We plot the ID accuracy of classifiers trained with OE and CCL mixtures of held- out TREC-10 OOD and ID data as an novel set. ID accuracy with OE decreases we introduce more noise, while CCL stays stable.
outperform vanilla training (c.g., on TACRED), CNL improves selective classification over vanilla across all datasets. We outperform the best prior method (Contrastive) by 2.3% AUAC, and on three of four datasets, our method significantly outperforms all prior methods. Furthermore, we outperform kFolden by 3.6% AUAC despite its ensemble totaling many times the capacity of our single classifier. CNL also results in zero or insignificant accuracy drop (less than 0.2 points) for all datasets. In Appendix A.2 we show full ID accuracy results for all datasets.
Other choices of novel set for CCL training can still be beneficial. Prompting with only task- relevant instruction (zero-shot) generates sufficiently useful novel examples to slightly outperform the vanilla baseline by 1.5% AUAC. Using Wikitext as our novel set performs roughly on par with zero-shot generation: though Wikitext examples are less noisy than generations, they also tend to be less releset- Few-shot generation, which generates more plausible examples, is significantly better than all prior methods, though it performs worse than Novelty Prompting on of 4 datasets.
To further test the importance of novel set selection, we compare with two oracle methods. In the Gold Data setting, we train with CCL against held out data of the gold novel test class(es) as strict upper bound for both label and example generation. In the Gold Label setting, label generation step, fixing the gold label as the sole member of our novel label set. We perform example generation normal using this gold label alone. This setting is overly optimistic as cannot know what new labels will appear at test-time We find that CCL in the Gold Label setting slightly outperforms CNL, but paired with gold novel data can achieve significantly stronger OSSC.
climinate the
Training loss choice matters for generated data. Although OE training with Novelty Prompting data improves OOD detection over vanilla, it causes sharp decrease in accuracy on TREC-10 (96.6% - 71.3%) and an average of 0.6% accuracy on the other three datasets (see Appendix (A.2). We attribute this behavior to generation noise: some generated examples are similar ID examples, and thus greatly affect the model's 's ID predictions when training with OE, To verify this hypothesis, we conduct an experiment where we train classifiers with novel sets formed by noising heldout OOD data with various amounts of heldout ID data. In Figure we show that as the simulated ID noise ratio increases, OE training hurts accuracy, whereas CCL models retain accuracy. Though this phenomenon is most salient on TREC-10, we show in Appendix A0 that it also occurs on other small datasets, and is more severe when the novel set is larger than the training set.
When the novel set is sampled from held-out gold OOD data instead of being generated by LLM, OE outperforms CCL in AUAC and AUROC, suffering only a small accuracy drop (an average of 0.4%). In contrast, we find that CCL training maintains accuracy on all generated novel data settings (see Appendix A.2). as it does not enforce a uniform probability distribution on all novel set examples. CCL with both zero- and few-shot augmented novel sets outperforms all prior methods, and our full CNL method (CCL+NP) significantly outperforms prior methods on all but one dataset.
In practice, expert knowledge of the novelty expect to see at test-time is sometimes available, and as shown in our results, can he leveraged for better performance.

---

Under review as a conference paper at ICLR 2023
AUROC (1)
TREC-10
AGNews
Emotion
TACRED
Averuge
Vanilla
76.6144
76.411.
85.0
46.310.1
71.1
kFolden
84.7120
72.5
85.3
53.116.2
73.9
Contrastive
79.8113
76.5
89.1
45.7.112
72.3
CCL + Wikitext
81.022.6
78.1
90.3
45.2+1.2
74.1
CCL + Zero-Shot
84.8
78.8
90.74.07
44.2+1.0
74.6
CCL + Few-Shot
88.4 88.4+0.6
80.5
92.840.5
49.7+a.3
77.9
OE + Wikitext
85.0+1.7
78.3
88.8+1.1
46.2+0.5
74.6
OE + Novelty Prompting
74.2+0.6
85.5
91.0+0.3
53.5+0.7
76.0
Contrastive Novelty Learning
90.8
82.61..66
93.4
50.940.5
79.4
CCL + Gold Label f
92.0+a.8
84,9
94.2+a.3
51.2+o.a
80.6
CCL + Gold Data f
98.3+0.3
91.7+00.
98.8+a.1
63.1
88.0
OE + Gold Data
99.1
98.810.3
99.710.0
89.010.5
96.7
Table 2: OOD Detection Results of Contrastive Novelty Learning. Methods same in Table We find that CNL signficantly improves OOD detection AUROC over all prior methods on 3 of datasets. While OE training results better AUROC on some datasets, it hurts ID accuracy.
Vani (1354) GPT New
Vaalla DEIT- Eass
Voalle RellERTe
06-
Selective Prediction AUAC
30)
Selective Prediction AUAC
(2 my GPT
03-
(68) GPT-
04
CNL RLDEST
CuDCITy
92-
CNL RubEEn wlege
CNL Cattimaaagge
02-
80
88
47
TREC-10
TACRED
Figure 4: Smaller generators can also help. We perform generation with variously sized gen- erator models, from 125M to 6B parameters. Larger generators seem to yield better results, but even our smallest generator does well.
Figure 5: Larger classifiers can also ben- efit. We additionally train RoBERTa-base and RoBERTa-lane with and without Novelty Prompted training. We find that both classifiers improve over vanilla with CNL.
5.2 OOD DETECTION RESULTS
To confirm that CNL improves a classifier's ability to disambiguate novel class examples, we com- pare CNL against the same baselines on OOD detection in Table We find similar improvements, outperforming the best prior method (kFolden) by 5.5% AUROC. We interpret this result in Ap- pendix [I.II showing that CNL improves ID/OOD separability. Unlike other datasets, TACRED exhibits strong OOD overconfidence: all baselines except kFolden yield worse-than-random OOD detection (below 50% AUROC). We hypothesize that this could be due to models incorrectly as- suming that an NER tag pair seen at training time in only a single class could not belong to novel relation. OOD detection on TACRED remains a challenging goal for future work, as the strong per- formance of CCL training with gold heldout data indicates significant remaining headroom. In fact, on all three other datasets, models achieve greater than 90% AUROC when trained with gold heldout data. While OE results in better AUROC performance AGNews, ID accuracy also decreases.
5.3 PERFORMANCE ANALYSIS
Label Generator Model We investigate whether a smaller, open-source model can suffice as the label generator. Specifically, replace the label generator with GPT-J and use 100 label generation iterations. We find that GPT-J performs on-par with GPT-3 on of datasets in all metrics, except on AGNews where it performs within point AUAC. We provide full details in Appendix A.3
7

---

Under review as a conference paper at ICLR 2023
TREC-10
AGNews
Emoition
TACRED
Malla shole
Valla 1hilen
Prol 4d PALAC
10:00 10:000
130
Figure 6: Selective prediction performance is positively correlated with generation quota. We mea- sure selective prediction against the total number of examples generated, showing various baselines in horizontal lines. On most datasets, even a low quota can meaningfully improve AUAC.
Example Generator Size Since model scale often affects prompting performance Saah et al. (202 ));
Brown et al. (2020), we compare generator models ranging in size from 125M parameters to 6B parameters. For each, we generate 100K examples, and compare CNL results in Figure 4 All generators improve over the Vanilla baseline. GPT2-Neo 125M is competitive with GPT2- despite being roughly 5x smaller, suggesting that its larger pretraining corpus (the Pile) aids gener- ation ability. We observe that novel generation is easier on simpler tasks: on Emotion, where labels (or synonyms) appear directly in the example, inter-generator differences small. We posit that even larger generators such as GPT-3 could yield better performance across abstract tasks f
Other Classifier Models We investigate the generalizability of CNL to two other classifier architec- tures, RoBERTa (Liu et al. (219)) and DeBERTa-v3 (He et al. 2021), I. of both base and large sizes, with results in Figure Averaged over datasets, CNL improves AUAC for all classifiers, though these improvements are most apparent with the smaller base models. Larger classifiers are better at OSSC: vanilla RoBERTa- large improves over BERT-base by 2.8% AUAC. Vanilla RoBERTa- slightly outperforms vanilla BERT-base, but after CNL training, the two perform on-par, suggesting that learning from generated examples can make up for BERT's s smaller pretraining corpus.
Generation Quota Since large-scale LLM prompting is costly, we analyze the performance tradeoff of shrinking the generation quota, the number of novel examples that we can generate. Figure 6 we show that on some datasets, using orders of magnitude smaller novel sets can still improve se- lective prediction. For example, 1000 generations is sufficient to improve AUAC across all datasets, and for most datasets require far fewer. In such cases where low quota is sufficient, novelty prompting is nearly as efficient as vanilla classifier training.
Generation Analysis To evaluate the remaining errors in OOD generation, perform two types of manual analysis on Novelty Prompting (NP). First, we categorize the labels generated by NP after filtering, finding that 70%+ of GPT-3 generated labels are novel all datasets except TREC- 10, where only 40% are novel, and the vast majority of the others are valid closed-set examples. This highlights one source of generation noise in our pipeline. Second, we categorize the examples generated by NP and a well-performing baseline method, Few-shot data augmentation (FS). Specif- ically, for each of 4 splits of AGNews, we annotate 100 NP and 100 FS examples. We find that on average 41% of NP generations come from novel classes, compared to only 26% of FS generations, explaining CNL's stronger performance over CCL. + Few-Shot. We provide further analysis in Ap- pendix A.5 Our method performs well despite the high fraction (50.5%) of closed-set examples generated in NP, showing that CCL is robust to noise in the novel example generation process.
RELATED WORK
6.1 IDENTIFYING OOD DATA
OOD Detection. Significant prior work has focused on OOD detection, which asks model to detect when a test example originates from a new distribution (Hendrycks & Gimpel 2017). Many of these change the training objective of a model, e.g., with a contrastive training objective (Winkens et al.
'ee evaluate GPT-3 for label generation but not for example generation, as the latter requires many more API calls.
8

---

Under review as a conference paper at ICLR 2023
2020 Sehwag et al. 2021 Zhou al., 2021b. When the nature of the distribution shift is known, the model can directly be trained to be uncertain on known OOD examples (Dhamija et al. 2018 Hendrycks al., 2019). We draw on the success of these known-shift methods, but eliminate the need for known OOD data by using generative models.
Recently, many works OOD detection have explored alternative modeling paradigms. By draw- ing on assumptions about the IID nature of neural network errors, confidence can be predicted from model ensembles Tifrea al. [021 Li et al. [202]] Lakshminarayanan et al.) 2017). Simple meth- ods like deep nearest-neiighbrs Sun et al. 2022); Bergman et al. (2020) can also perform surpris- ingly well. Further performance improvements can be achieved by modifying the confidence metric. Podolskiy et al. (2021) find that Mahalanobis distance better exploits the geometry of the learned scoring
embedding space, explaining strong performance achieved by replacing probability- mechanisms (Lee et al., 2018; |Ren et al., (021). We show that standard models are sufficient: Max- Prob scoring with a standard classifier can perform well when given proper OOD demonstrations.
OOD Selective Prediction. Selective prediction work focuses a different paradigm altogether. fusing abstention (detection) with prediction (El-Yaniv & Wiener, 2010; Geifman
Yaniv
2017 External calibrators popularized by Kamath et al. (2020) have become popular as a selective prediction framework Zhang et al. (202I); Ye & Durrett (2021); Varshney et al. (2022). However, calibrators are typically smaller than classifier models Tajwar al. 202ID, limiting their capacity to learn the volume of outliers we generate and swaying us to consider using no auxiliary model to detect novel examples. Additionally, methods such as ours which train model predict their own confidence can be deployed with no additional inference overhead. For example, large generative LMs can be trained self-verify on challenging question answering tasks (Kadavath et al. (022).
6.2 OPEN-SET CLASSIFICATION
Open-set classification is well-explored in the image classification space, tasks like CIFAR-100 tend towards large label spaces (Scheirer et al. 2013 Geng et al. 2021). Some methods for detecting open-set examples build on the classifier, e.g., by classifying over the model's activations (Bendale
& Boult 2016) or adding an additional reconstruction model (Oza & Patel 2019). Our work is
most closely related to methods which generate near-OOD examples and regulariza confidence on them (Ge et al. 2017 Du et al. 2022 Kong et al. 2020: Vernekar et al., 2019: Möller et al. 2021
Setlur et al. 2022). However, methods like perturbation and embedding space sampling align poorly
with the discrete nature of text, prompting us to investigate powerful generative language models. Esmaeilpour et al. (2022) closely related our work in that they also generate novel labels, but directly use these labels as input to a classifier.
Open-set classification for text has been less explored. Early works built upon the k-way, I 1-vs-rest paradigm of SVMs, classifying an example as "novel" if all l scores fall below threshold (Fei & Liu 2016) [Shu et al. 2017 Doan & Kalita 2017). Many recent works explore similar methods as prior vision work, but focus on the intent detection setting, as task-oriented dialogue models should abstain on unknown intents (Zng et al.) [02T Zheng et al. 2020 Lin & Xu 2019). To the best of our knowledge, we are the first work to generate novel examples for open-set text classification.
7 DISCUSSION AND FUTURE WORK
In this work, we introduce Contrastive Novelty Learning. a method for generating novel examples which simulate open-set shift and training to abstain on them. Through extensive experiments, we demonstrate that by presenting generated examples classifier, we can significantly improve its ability to abstain on examples from novel classes against state-of-the-art baselines. Our work provides generalizable framework for improving OSSC and OOD detection: in fact, we show through CCL training's strong performance with gold heldout data that there remains significant headroom for novel example generation. Additionally, CNL is modular, as provides additional supervision signal but does not alter the classifier model's architecture, and thus remains extensible with other training objectives or classification metrics. Finally, given recent interest in the emergent hope that future work classification in the presence
capabilities of large language models, of distribution shifts can better leverage large language models to both directly identify shifts and improve the abstention ability of smaller classifiers.
9

---

Under review as a conference paper at ICLR 2023
8 REPRODUCIBILITY STATEMENT
We provide a full code implementation of Contrastive Novelty Learning, including Novelty Prompt- ing and CCL and OE training. We detail hyperparameters for our experiments Section 4.2, prompt formats in Appendix A.I] and data processing details in Appendices [A.6 and A.7 CNL can be run with any text dataset and discriminative classifier model. We release generated novel examples from Novelty Prompting and several generation baselines for reproducibility.
REFERENCES
Udit Arora, William Huang, and He He. Types of In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing (EMNLP), 2021.
texts and how to detect them.
Abhijit Bendale and Terrance E Boult. Towards open set deep networks, In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016.
Liron Bergman, Niv Cohen, and Yedid Hoshen. Deep nearest neighbor anomaly detection. arXiv preprint arXiv:2002.10445, 2020.
Tom Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal,
Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Advances Neural Information Processing Systems, 2020.
Akshay Raj Dhamija, Manuel Günther, and Terrance F. Boult. Reducing network agnostophobia. In Advances in Neural Information Processing Systems, 2018.
Tri Doan and Jugal Kalita. Overcoming the challenge for text classification in the open world. In 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), 2017.
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. VOS: Learning what you don't know by virtual outlier synthesis. arXiv preprint a7Xiv2220..01197, 2022.
Ran El- Yaniv and Yair Wiener. On the foundations of noise-free selective classification. In Journal of Machine Learning Research, 2010.
Sepideh Esmacilpour, Bing Liu, Eric Robertson, and Lei Shu. Zero-shot out-of-distribution detec- tion based the pretrained model CLIP. In Proceedings of the AAAI conference artificial intelligence, 2022.
Geli Fei and Bing Liu. Breaking the closed world assumption in text classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016.
Yuag Yuan Ge, Sergey Demyanov, Zetao Chen, and Rahil Garnavi. Generative openmax for multi- class open set classification. arXiv preprint arXiv:1707.00 10, 2017.
Yonatan Geifmaan and Ran El- Yaniv. Selective classification for deep neural networks. In Advances in Neural Information Processing Systems, 2017.
Chuanxing Geng. Sheng-Jun Huang, and Songcan Chen. Recent advances in open set recognition:
A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertay3: Improving deberta using electra-style pre-training with meadient-disentanlle embedding sharing. 2021.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and diitr---iittiittonon ex- amples in neural networks. In 5th International Conference Learning Representations (ICLR), 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. In 7th International Conference Learning Representations (ICLR), 2019.
10

---

Under review as a conference paper at ICLR 2023
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. Span- BERT: Improving pre- pre-training by representing and predicting spans. In Transactions of the Asso- ciation for Computational Linguistics, 2019.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language mod- els (mostly) know what they know, arXiv preprint arXiv:2207.05222 2022.
Amita Kamath, Robin Jia, and Percy Liang. Selective question answering under domain shift. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
Aran Komatsuzaki. Gpt-j-6b: 6b jax-based transformer, 2021.
URL https://
arankomatsuzzkii www.drews.ddooooooooooooooo...so com/ wtt.coom///22////452/
Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. Calibrated
language model fine-tuning for in- and out-of-distribution data. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP). 2020.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Processing Systems, 2018.
Xiaoya Li, Jiwei Li, Xiaofei Sun, Chun Fan, Tianwei Zhang, Fei Wu, Yuxian Meng, and Jun Zhang. kfolden: k-fold ensemble for out-of-distribution detection. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing (EMNLP), 2021.
Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics, 2002.
Ting- Enn--En Lin and Hua Xu. Deep unknown intent detection with margin loss. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. 2019.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Ad- vances in Neural Information Processing Systems, 2019.
Felix Möller, Diego Botache, Denis Huseljic, Florian Heidecker, Maarten Bieshaar, and Bernhard Sick. Out-of-distribution detection and generation using soft brownian offset sampling and au- toencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2021.
Poojan Oza and Vishal M Patel. C2AE: Class conditioned auto-encoder for open-set recognition. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, 2019.
Pramuditha Perera, Vlad I Morariu, Rajiv Jain, Varun Manjunatha, Curtis Wigington, Vicente Or- donez, and Vishal M Patel. Generatieediiccrininatee feature representations for open-set recog- nition. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2020.
Alexander Podolskiy, Dmitry Lipin, Andrey Bout, Ekaterina Artemova, and Irina Piontkovskaya. Revisiting mahalanobis distance for transformer-based out-of-domain detection. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshmi- narayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint 10Xi:2116600002 2021.
11

---

Under review as a conference paper at ICLR 2023
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, An- toine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero task generalization. In The Tenth International Conference on Learning Repre- sentations, 2021.
Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. CARER: Con- textualized affect representations for emotion recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018
Walter J. Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E. Boult. Toward open set recognition. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013,
Vikash Sehwag, Mung Chiang, and Prateek Mittal. SSD: A unified framework for self-supervised outlier detection, In The Ninth International Conference on Learning Representations, 2021.
Amrith Setlur, Benjamin Eysenbach, Virginia Smith, and Sergey Levine. Adversarial unlearning: Reducing confidence along adversarial directions. arXiv preprint arXiv:2206.0136; 2022.
Lei Shu, Hu Xu, and Bing Liu. DOC: Deep open classification of text documents. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2017.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribttion detection with deep nearest neighbors. In The Thirty-nimth International Conference on Machine Learning, 2022.
Fahim Tajwar, Ananya Kumar, Sang Michael Xie, and Percy Liang. No true state-of state-of-the-art? ood detection methods are inconsistent across datasets. In ICML Workshop on Uncertainty Robustness in Deep Learning, 2021.
Neeraj Varshney, Swaroop Mishra, and Chitta Baral. Investigating selective prediction approaches across several tasks in IID, OOD, and adversarial settings. In Findings of the Association for Computational Linguisties (ACL), 2022.
Sachin Vernekar, Ashish Gaurav, Vahdat Abdelzad, Taylor Denouden, Rick Salay, and Krzysztof detection in classifiers via generation. In NeurIPS 2019, Safety and Robustness in Decision Making Workshop, 2019.
Czarnecki.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R Ledsam, Patricia WacWiliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, et al. Contrastive training for improved out-of-distribuut detection. arXiv preprint arXiv:2007.05566, 2020.
Xi Ye and Greg Durrett. Can explanations be useful for calibrating black box models? In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics, 2021.
Zhiyuan Zeng, Keging He, Yuanmeng Yan. Zijun Liu, Yanan Wu, Hong Xu, Huixing Jiang, and Weiran Xu. Modeling discriminative representations for oft-onn detection with supervised contrastive learning. In Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics, 2021.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empiri- cal risk minimization. In 6th International Conference on Learning Representations, 2018. 8.
Shujian Zhang, Chengyue Gong, and Eunsol Choi. Knowing more about questions can help: Im- proving calibration in question answering. In Findings of the Association for Computational Linguistics, 2021.
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In Twenty-ninth Conference on Neural Information Processing Systems, 2015.
Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. Position- aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2017.
Yinhe Zheng, Guanyi Chen, and Minlie Huang. Out-of- detection for natural language un- derstanding in dialog systems. In JEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020.
12

---

Under review as a conference paper at ICLR 2023
Wenxuan Zhou, Fangyu Liu, and Muhao Chen. Contrastive citoo--distribution detection for pre- trained transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.
Alexandru Tifrea, Eric Stavarache, and Fanny Yang. Novelty detection using ensembles with regu- larized disagreement. arXiv preprint arXhv:2012.05823, 2021.
A APPENDIX
A.
PROMPT FORMAT
We use the same format for label generation for all datasets, shown in Figure 7 but customize the instruction for each dataset, as shown Figure
Instruction
Generate a diverse list of news genres: [World, Sports, Sei/Tech,
ID Labels
Figure Label Generation prompt for AGNews.
Dataset
Instruction
Emotion
Generate diverse list emctions
AGNews
Generate diverse list of news genres
TREC-10
Generate diverse list entity types
TACRED
Generate diverse list of relations between entities
Figure 8: Label Prompts for each Dataset.
For example generation, prompt with example sampled from each class and a random novel label. We use the same instruction for all datasets. An example prompt shown in Figure
Instruction ID Label ID Example
Given label, generate corresponding example: business
Starwood Names New Chief Executive SEPTEMBER 21, 2004 -- White Plains, NY -- Former Coca-Cola Company president Steven Heyer teday was named the chief executive of Starwood Hotels, effective Oct. 1. Heyer succeeds starwood founder Barry sports
ID Label 2 ID Example
Marino, Young Considered for Hall of Fame Dan Marino and Steve Young highlighted a list Friday of 25 candidates for the Pro Football Hall of Fame.
ID Label
world
ID Example
Afghan warlords 'threaten poll' Afghan warlords are involved in intimidation which could threaten October's s elections, Human Rights Watch says. entertainnent
Novel Label
Figure Example Generation prompt for AGNews.
Few-shot prompting is done with task-specific instruction, but does not include labels, shown in Figure TO Zero-shot prompting done with the task-specific instruction only.
A.2 FULL ACCURACY RESULTS
CNL improves AUAC on all datasets without any cost to ID accuracy, as shown in Figure II We show full ID accuracy results in Table 3 CCL training maintains accuracy across all datasets,
13

---

Under review as a conference paper at ICLR 2023
Instruction
Generate a news headlines Starwood Names New Chief Executive SEPTEMBER 21, 2004 - White Plains, NY -- Former Coca-Cola Company president Steven Heyer teday was named the chief executive of Starwood Hotels, effective Oct. 1. Heyer succeeds Starwood founder Barry Marino, Young Considered for Hall of Fame Dan Marino and Steve Young highlighted candidates for the Pro Football Hall of Fame. Afghan warlords 'threaten poll' Afghan warlords are involved in intimidas dationni which could threaten October's 8 elections, Human Rights Watch says.
ID Example
ID Example 2
list Friday of 25
ID Example
Figure 10: Few-Shot Generation prompt for AGNews.
TROG4E
Vadui
(f)
TREC-10
AGNews
Emotion
TACRED
Avg
Sdctee Proiccio RUAC
ADhen
AUAC
89.2+2.2
87.9
90.3+1.0
89.6
89.3
AUROC
76.614.4
16.4
85.0
16.3
ID Ace
96.610.3
96.1 96.1
97.710.
95.0go.1
96.4
GPT3
AUAC
94.3
90.5
93.41a.
91.1
92.3
AUROC
90.8
82.620.6
93.440.
50.9
79.4
ID Ace
96.4+0.2
96.2+0.0
97.8+0.1
94.9=0.1
96.3
AUAC
94.2
89.8
93.5
91.0go.2
92.1
AUROC
90.01m.a
80.810.
93.510.
50.41o.4
Ancurayy
ID Ace 96.42..1
96.220.0
97.9.0.1
94.920.
96.4
Figure 11: CNL training main- tains accuracy. Training with novelty prompted examples does significantly alter ID accuracy, but improves selective prediction across all datasets.
Figure 12: GPT- is also a strong label generator We com- pare label generation using GPT-3 and GPT-J, using GPT-J as the example generator for both methods. GPT-J performs within a negligible margin GPT-3 on TREC- 10 and Emo- tion, but slightly worse AGNews and TACRED.
while OE training decreases accuracy on of 4 datasets, with a very sharp drop on TREC-10. In Appendix A. show through analyses two datasets that this steep accuracy drop is anomaly: when paired with generated data, OE training is sensitive to the sizes the novel set and training set, and can signficantly hurt ID accuracy when the novel set is much larger than the training set. Additionally, despite improving selective prediction performance, training with gold held-out data curiously hurts accuracy on TACRED.
A.3 CNL PERFORMS WELL WITHOUT GPT-3
In our main experiments, we use GPT-3 as the label generator and GPT-J 6B as the example gen- erator. Section 5.3 we show that smaller models can used as example generators. Here we investigate whether smaller, open-source language model can be used as a label generator. In Table 12 we show that GPT-J 6B also performs well at label generation. We empirically observe that GPT-J generates shorter and noisier completions, requiring to increase the number of model calls from 100 and filter out all labels containing punctuation marks. We find that the differ- ence between GPT-J and GPT-3 label generation is only significant on of 4 datasets (AGNews), suggesting that CNL with GPT-J only can still work well.
A.4 GENERATION EXAMPLES
We show examples of the generations from Novelty Prompting for AGNews in Table Recall that we do not allow the gold novel label to be generated to hedge against data leakage from LLM pre- training. However, the gold novel label due to signal from similar novel labels. Despite many generations not being directly relevant the gold novel label, we observe that the generated novel labels are sufficiently
observe that our generator is still capable of producing relevant examples to
14

---

Under review as a conference paper at ICLR 2023
ID Acc (1)
TREC-10
AGNews
Emotion
TACRED
Vanilla
96.610.2
96.1
10.1
95.0
kFolden
96.510.1
96.010.1
97.210.2
88.3
Contrastive
90.3
96.0+0.0
98.0+0.1
94.8+0.2
CCL + Wikitext
96.610.1
96.1
97.610.1
94.9
CCL Zero-Shot
96.5
96.310.0
97.640.1
94.940.2
CCL Few-Shot
96.340.2
96.1
97.8+0.1
94.840.2
OE + Wikitext
96.640.2
96. 1
97.640.1
94.8+0.1
OE Novelty Prompting
71.3
95.640.0
96.4+0.2
94.840.2
Contrastive Novelty Learning
96.4
96.210.0
97.840.1
94.9
CCL Gold Label
96.5
96. 96. 10.0
97.740.1
94.9
CCL Gold Data f
96.0
95.8 95.810.1
97.610.1
OE Gold Data f
96.41p.1
95.8 95.8
97.910.1
93.610.2
Table Full Accuracy Results of Contrastive Novelty Learning
Dataset
Label Type
Frequency
Example Label
Rowerry
Oheet Sa
Paccot of Tosal Garoetturs
Implausible
7.8%
August 27
100
TREC-10
Novel
40.0%
time
80
Closed-Set
52.2%
person
60-
Implausible
14.9%
ology
AGNEWS
Novel
81.7%
food
Closed-Set
3.3%
technology
Implausible
5.1%
app
EMOTION
Novel 83.9%
serenity
Basiness
Sc/Tech
Closed-Set
11.1%
frustration
Implausible 14.3%
ualifications
Error Analysis on Exam- ple Generation. For each split of AG- News, we manually annotate 100 gen- erations each for two generation meth- ods and compute the frequency of novel class examples, closed-set class exam- ples, and implausible examples.
Figure 14:
TACRED
Novel
parent company 73.6%
Closed-Set 12.1%
current location
Figure 13: Error Analysis on Label Generation. We manually annotate generated label sets across splits of each dataset, recording the frequency of novel and plausible labels.
distinct from the closed-set labels that most generated examples still provide useful "novelty" super- vision signal to the classifier.
A.S NOVELTY PROMPTING ERROR ANALYSIS
Though CNL improves OSSC ability on all datasets, we still find headroom between Novelty Prompting generated data and gold OOD data (92.3 + 94.8) in Table To understand the remain- ing failure modes of Novelty Prompting, we manually inspect the generated labels and examples from our method. Broadly, we seek to attribute "generation noise," or the frequency with which the purported novel sets which we generate instead contain closed-set class examples.
First, we manually annotate GPT-3 generated labels from all dataset splits, categorizing a label into "implausible" if it does conform to the dataset's format, eelosed-set" (II) is synonymous with a class seen in training, and "novel" (OOD) if it describes class distinct from all closed-set classes. In Figure [3] we perform this analysis for all four datasets. Across all datasets, less than 15% of generations are implausible, suggesting that the model is usually able to generate reasonable additional labels given only 3-6 ID classes. We also observe that while on of 4 datasets less than 15% of generated classes closed-set, TREC- more than half of generated labels closed- set. One reason for this label generation noise that the TREC-16 labels are very broad (c.g., "entity" describes questions about any subcategory of entity, including all objects and events), so
15

---

Under review as a conference paper at ICLR 2023
Gold Label
Generated Label
Generated Example
HEALTH
The Top S Places Take Break From the Rat Race This sum- mer could be the best one years take break and recharge the ol' batteries.
TRAVEL
U.S. Airlines Say They Won't Raise Rates to Pay for the New Security Fee and The government, for its part, has set a deadline for some of the other fees the beginning of this month. The Real Reason Republicans Cautiously Endorse the 'Ob- scure' Paul Ryan CNN Politics If we're going to pick fights with the other party, we might as well pick fights with the right people.
WORLD
POLITICS
ENTERTAINMENT
David Beckham And Victoria Beckham Are Exiting The Lon- don Home They Have Shared With The NBA Star. LONDON (AP) - David Beckham has been kicked out the house that he shared with his British soccer team, the Los Angeles Galaxy, The couple and the couple's 's kids are moving into an apartment In search of fashion in South Africa What will the fashion world be wearing next year? Will the world need another white wed- ding dress?
SPORTS
FASHION
CRIME
The Police Are Going To Find You a Lot Easier in the Future: Security Advances of 2008 (Video) The police are going to find you lot easier in the future: Security advances of 2008
FOOD/DRINK
How to Make Fried Sausage Slider (Fast) YouTTuee A couple of weeks ago. I made fried sausage slider and they re awe- some! A few friends have asked me to go through this again but in a shorter time to make up a batch quickly for them.
WEATHER
Snowstorm Blasts Northeast with Another Round of Winter Storms to Leave Snow, Ice and Blowdowns A big nor' caster is expected to bring sleet, ice and another round of dangerous wind gusts around New Year's ' Eve. Winter storms are expected to roll through Wednesday and Thursday in the AP- Michael Phelps and his wife will be moving back to Wash- ington State from Arizona while he finishes his Olympic career.
BUSINESS
RELATIONSHIPS
LIFESTYLE
A new batch of Apple iPhone 3Gs have gone up for sale in the UK, with all six major networks now having network price. Apple unveiled the 3Gs on Wednesday, making a number of changes to the device, which is expected to be hugely popular in the market.
SCI/TECH
ENTERTAINMENT
THE FILM: JERSEY GIRL "Jersey Girl" tells the story of the love-hate relationship between an Irish-American girl from New Jersey and a native New Jerseyan. Directed by Elizabeth Swados.
TECHNOLOGY
Yahoo Japan buy a majority stake in Nikkei Corp Yahoo
Japan Corporation announced it plans buy 69.8 per cent stake of Nikkei for 1.43billion, the two companies said Friday.
Table 4: Example novel generations for AGNews.
while a generated label might differ in definition, it could still overlap with or fall into subcategory of a closed-set class.
Second, we manually annotate GPT-J generated examples to understand whether example genera- tion is source of generation noise. In Figure [4 we annotate 100 examples of each split of AGNews for both Few-shot data augmentation and Novelty Prompting- We observe that Novelty Prompting generates novel class examples significantly more frequently across 3 of 4 splits. Both methods gen- crate implausible (e.g., agrammatical, non-news) examples rarely, as ID demonstrations sufficiently prime the model to generate text in the style of news. Additionally, under Novelty Prompting, we find that the fraction of novel class examples (41.3%) is significantly lower than the fraction of novel labels generated (81.7%), suggesting that GPT-J can easily adhere to the dataset format, but struggles to extrapolate to the novel label. Future work should thus focus on better specifying the example generation step to leverage the generated labels.
16

---

Under review as a conference paper at ICLR 2023
TREC-10
AGNews
Emotion
TACRED
Avg
Vanilla
AUAC
89.212.2
87.9
90.3110
89.610.1
89.3
AUROC
76.614.4
76,411.0
85.012.4
46.310.1
71.1
ID Acc
96.610.2
96.110.0
97.7
95.010.1
96.4
AUAC
90.6
83.5
82.0
87.121.0
85.8
AUROC
80.5
72.9
75.1
41.2+22.4
67.4
ID Ace
96.7+0.2
96.2+0.D
97.74+0.1
95.04a.1
96.4
Table 5: Label Smoothing (LS) hurts AUROC and AUAC on all but one dataset.
A.6
DATASET SPLIT DETAILS
We remove the Abbreviation class as it is too small yield statistically significant metrics in our task setting, leaving 5 remaining classes.
TREC-10:
Emotion (Saravia al 2018): We remove two small classes, love and surprise, leaving remaining classes.
TACRED (Zhang et al. 2017): We process the data for training following Joshi et al. (2019). This dataset is particularly challenging due to its class-imbalanced nature. We evaluate a single split where we keep the largest classes ID data, and hold out the other 35. This is the largest class, and thus results in approximately 80% of examples being OOD at test time.
A.7 TACRED PROCESSING DETAILS
We perform label normalization, removing underscores and prefixes, c.g. converting per employee into employee of. This both helps the label generator model understand our label space and generate more relevant novel labels and ensures that generated novel labels are well-formatted for downstream example generation.
For examples, we normalize the Subject and Object token tags into standard English equivalent containing the subject or object indicator and the NER tag, egg., (subject person]. To ensure that generated examples satisfy the task format, we filter out examples that do not contain exactly one subject and one object (many generations contain partial or malformed indicator/NER spans). Finally, we denormalize tags back into original model input tokens.
A.8 LABEL FILTERING
After label generation, we perform synonym filtering reduce occurrences of ID synonyms. We find this step to have a large impact on datasets for which labels are common English words which appear in our thesaurus, and less where label names more abstract. For example, for Emotion and TREC-10 where dataset names are words such as "fear" or "human," filtering removes 21% and 20% of generated labels respectively. Meanwhile removes only 2% of labels. In the case of AGNews, news genre overlaps are not easily captured by synonyms, and even after normalization, many TACRED labels such as "employee of"" do not appear in our thesaurus.
both AGNews and TACRED, label filtering
A.9 LABEL SMOOTHING PERFORMS POORLY.
We evaluate label smoothing (LS) Müller et al., 2019) as an additional baseline for improving OSSC, which mirrors vanilla training but alters the one-hot target vector to "smoother" version, incentivizing uncertainty. Label smoothing has been shown to be effective in domain shift detec- tion (Kong et al. 2020). We use label smoothing factor a = 0.1 and calculate confidence with MaxProb. In Table 5 show that label smoothing performs poorly in setting. While it does not affect classifiers' ID accuracy, it significantly decreases AUROC on all but one dataset (TREC- 10), where it still remains worse than CNL and all of our data generation baselines.
17

---

Under review as a conference paper at ICLR 2023
875
Arcumrr (5)
000
Figure 15: Outlirr Exposure is sensitive to the size of the novel set on TREC-1O. We vary the novel set size from 0 to 100K, finding that both accuracy and AUROC decrease with as few as 100 novel generations. We indicate with a dashed line the point where the novel set and training set size are approximately equal.
CO.
Vadii
80-
D Acuuoyyy
A00CC
66-
122
20
2.7
222
127
Troining Set Sine
Traviig Sie Siee
Toaking Stze
Figure 16: Outlier Exposure disproportionately hurts smaller datasets. We subsample the training set for AGNews, use 100K novel generated examples, and vary the training loss. We find that CCL achieves similar IDD performance as Vanilla all training set sizes, but OE significantly hurts accuracy when the training set smaller than 1000 examples.
A.10 OUTLIER EXPOSURE IS SENSITIVE TO GENERATED DATA
In the setting where Outlier Exposure is originally evaluated, access to some known OOD data (e.g., Wikitext) is assumed (Hendrycks al. 2019). However in our setting, where we generate potential novel set, there is no guarantee that the generated examples are indeed OOD. For example, show in Appendix [A.5 that less than 50% of NP generations for AGNews come from novel classes. Without this guarantee, more generated data is not always better when training with OE. One risk of using more generated novel data is that the model will large number of ID examples in the novel set relative to in the training set. We conduct two experiments to analyze the impact of novel set size relative to training set size.
First, we vary the novel set size relative to the training set size. In Figure 15 we train with novel sets TREC-10 from size 0 to 100K using both OE and CCL. We observe that training with OE hurts accuracy and AUROC when the novel set is larger than 100 examples, while CCL continues to improve as the novel set size grows, and maintains accuracy for all novel set sizes. As the novel set becomes larger than the size of the training set (to the right of the dashed line), both OOD detection AUROC and ID accuracy quickly decrease. This result suggests as the IDD noise the classifier sees in OE training outsizes the training set, its ID predictive ability worsens.
Of the datasets in our experiments, TREC-10 by far the smallest, with only about 2800 training examples per split. To determine whether OE is also sensitive to the size of the ID set, we subsample the AGNews dataset into smaller training sets and perform OE and CCL training with 100K-sized novel sets. We compare the results against Vanilla training with the same ID sets in Figure 16 Although reducing the training set size decreases the ID accuracy even for vanilla training, CCL training achieves similar accuracy for all subsampling sizes. We do observe that sub-10% accuracy
18

---

Under review as a conference paper at ICLR 2023
margin appears between vanilla and CCL extremely small training set sizes, though this margin disappears at 1000 or more training examples. OE, meanwhile, decreases ID accuracy by as much as 35% when the dataset is subsampled to 30 examples, and 25%+ at 300 examples. OE-trained classifiers are also worse OOD detectors given limited training data: they underperform vanilla find that OE does yield better
classifiers for all training sets smaller than 3000 examples. Finally, This expands on our findings OOD detectors than CCL for sufficiently large AGNews training sets. in Table suggesting that when there access large amount of training data, in this case 10000 examples are more, OE can learn from noisy novel sets (though ID accuracy still decreases). Our results indicate that TREC-10 is not alone: As training set size becomes smaller, the ID classes becomes less well-specified, and ID examples present in the novel set induce the model to make incorrect predictions (and poor confidence estimates) on true ID test examples.
A.11 CNL AND SEPARABILITY
To understand why CNL improves AUROC, compare the confidence profiles of vanilla fine- tuned classifier against those of CNL trained classifier. Specifically, in Figure 17 we select 50 random ID examples and 50 random OOD examples from each dataset split and compute MaxProb confidences. We find that CNL decreases confidence on OOD examples, though not to the same ex- tent on all examples. In datasets like TREC-10 and Emotion where CNL achieves stronger AUROC gains, the decrease in OOD confidence is more pronounced. Though IDD test examples also decrease confidence on all dataset splits, this decrease is less pronounced and is likely due the confi- dence contrastive objective term incentivizing the model's confidence distributions to be generally less peaked.
The shifts reflected in the confidence distributions directly impact the separability of OOD and ID examples. On the Vanilla model confidence axis, is difficult to identify threshold above which most examples are ID and below which most examples are OOD. Given CNL confidences, OOD and ID examples are more separable. This visual separability reflected in the OOD Detection AUROC metric.
To demonstrate the strictness of the OE objective, we plot the confidences of the same examples without (Vanilla) and with OE training in Figure [8] First, observe that the vast majority of OOD examples have similar confidence after OE training, as they are all pushed towards minimum confidence (maximum entropy). Second, observe that OE significantly affects the confidence of ID test examples, decreasing the confidence of some examples lower than that of OOD test examples.
19

---

Under review as a conference paper at ICLR 2023
TRE SaR
TRECH SON
TIEC Sol
Cottaes CuL
Cord CiL
3:30
1:30
1:30
1-187
1-127
1-127
1-127
1-187
1-:27
1-304
1-20
TAORD sylvo
Cardena CR.
3.300
1:004
3:30
1-187
1-:
1-197
3-00
1-00
3-0"
(2i0imn hygh
Cordees C4
3.304
1:304
1:30
1-191
1-:20
1-:0"
1-191
1-307
3190
Aillon
Enstoe Sul
3-30
Emedee Solk
Cordinn OIL
Coridens OIL
1:004
1:00
1:34
12
3-30
1-:9
Carkinn DNL
Curiinn DKL
1400
1484
14127
1-127
Figure 17: CNL improves the separability of ID and OOD examples. We plot the confidences of 50 random ID and 50 random OOD examples on vanilla finetuned BERT classifier versus CNL. trained BERT classifier. CNL successfully decreases the confidence of OOD test examples while minimizing the impact of the confidence of test examples.
20

---

Under review as a conference paper at ICLR 2023
500
000
500
Coelbrn 0z
Corlbonn 0z
1.330
1.:30
1-:07
1.330
Go48drunn Vaalo
0.68
Ged8dren Viall
220
220
Canbbro 02
Canbbooe 05
1:00
1-330
1-10
1-07
14357
1-197
1-077
14390
1000 1-300
1007
Grekdeee Varla
Cobidoer Varda
Crekdeen Virlaa
520
000
020
Contora 02
Confeorne 02
3-00
3-30
3 207
3 307
Cookdencs Varda
Coolddnnes Varda
Coeldencs Varda
220
000
200
Catbbe 02
Cartboas 02
Creideacs Varill
Coriddancs Varia
Coelidacss Virilla
200
400
Coltura 26
Coltura 36
01108 11:34 51027 1094
Cookdenncs Viaddis
Conficeres Vantaa
Figure/ 18: OE significantly decreases the confidence of OOD examples, but unfortunately also decreases confidence ID examples. We plot the confidences of 50 random ID and 50 random OOD examples on vanilla finetuned BERT classifier versus a NP+OE trained BERT classifier. We also observe that ID examples exhibit large confidence distribution after OE training: some ID examples have similar confidence as OOD examples. Note that the axis limits on these plots differ from the axis limits on Figure 17 confidences in general much lower.
21


Reviews:
[{"review_id": "sfBln30gyY", "reviewer": "Reviewer_r6H5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "To tackle the problem of OOD detection (especially in text classification task), the authors propose a method called Contrastive Novelty Learning (CNL). The core idea is to generate OOD examples by prompting a large language model twice, first to generate label, then to generate the content. Experiments show effectiveness of the proposed method.", "summary_of_the_review": "See Strengths/Weaknesses", "strength_and_weaknesses": "**Strength**\n* The idea to use large pretrained generative model to generate OOD example is inspiring.\n* The experimental results are strong.\n\n\n**Weaknesses**\n* While the results seems promising, I'm curious whether the choice of pretrained model (e.g. replace GPT-3 with GPT-2) impact the performance much. If the proposed method only works well on very large pretrained generative models, the computational cost could be extremely high, since the authors generate 100k OOD samples.\n\n* The proposed method seems hard to be extended to other tasks like sequence labeling and other domains like computer vision.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ePTNVA5gAGK", "reviewer": "Reviewer_i6wa", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper deals with OOD examples in classification. It leverages a huge PLM and generates a few examples that distribute differently from the ID examples. Subsequently, it learns a classifier with a contrastive confidence loss. Experimental results on a few public datasets indicate the effectiveness of the proposed method on classification and OOD detection. ", "summary_of_the_review": "The paper attempts to train a classifier that can accurately detect OOD examples. The interesting part is novelty prompting in which examples that are semantically novel and bear some resemblance to ID examples are generated with prompts using huge language models. However, I have some concerns regarding to the presentation and the empirical results:\n(1) CCL+few shot data augmentation: the description is \"prompt with only an instruction and zero or more ID training examples\", then how many ID training examples are involved in CCL+few shot? What is the difference between the proposed method? \n(2) what is the classification model? \n(3) Looking at Table 1, I am curious why OE+novelty prompting performs such badly on TREC-10. If it is sensitive to noise, then why it renders descent results on the other three datasets? This is critical, as the baseline outperforms the proposed method on two of the four datasets but loses finally on the \"average score\". \n(4) the improvement over CCL+few-shot is marginal. That's why I am curious about the difference between the proposed method and CCL+few-shot. \n", "strength_and_weaknesses": "Strength:\nGenerating OOD examples with huge PLMs is an interesting idea. \n\nWeaknesses:\nSome important details are missing.\nImprovements over the baselines are not fully convincing. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Yw1uDTzJu7e", "reviewer": "Reviewer_b3bf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposed a two-step generation-based method for open set selective classification. The generation process starts with first generating a set of OOD labels then examples conditioned on these novel labels. The paper also proposed a new training objective that pushes the max probability of ID predictions above those of OOD predictions, which is less restrictive than existing proposals by avoiding global uniformity. The proposals are evaluated on a few classification tasks on NLP and empirically shown to be effective, with ablation on some selected aspects of the setup.", "summary_of_the_review": "The paper proposed a reasonable solution to an important problem with good empirical results and decent result analysis. Publication would benefit the community.", "strength_and_weaknesses": "Strength\n\n- The idea is well motivated, both wrt the novelty prompt generation and the contrastive confidence loss.\n- Empirical results are overall positive, with reasonable baselines and comparisons and some ablation studies.\n- The writing is clear and easy to follow.\n\nWeaknesses\n\n- There are some details that are not very clear\n  - S4.3 For contrastive baseline, I'm not sure how to interpret \"pairwise distant but internally compact\" in the context here. Are there novel classes and examples involved? Might be good to clarify this question for the baselines here?\n  - S5.1 it was a bit difficult to follow the various comparisons made in ref to Table 1. E.g., I had a hard time figuring out what is being compared at \"This accuracy decrease is smaller\"\n  - S5.2 \"To isolate the factors...\" what are the factors being isolated here and how is the isolation achieved by Table 2?\n  - S5.3 Generator size is concerned only with the example generator, while the novel-label generator is always GPT3? What happens if one generates novel labels with smaller models?\n- There's not much error analysis other than offhand comments with limited support.\n  - For example, when discussing vanilla outperforming OE+NP (\"We attribute this behavior to the noisiness of generation: some novelty prompted examples are similar to ID examples\"), although I intuitively agree with this sentiment, it would be great to have some empirical support by quantifying example similarity.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VUdJNSJaNFH", "reviewer": "Reviewer_KAeh", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors study out-of-distribution detection for text\nclassification.  They propose \"contrastive novelty learning,\" which\ncombines two methods: 1) data augmentation with GPT-3 for\nout-of-distribution classes; and 2) a loss function which encourages\nthe maximum probability scores to be lower on out-of-distribution\nlabels. The authors report a few accuracy points of gain compared to\nprior methods, and conclude by exploring a number of questions, e.g.,\nabout limiting GPT-3 budget, and training with larger models.", "summary_of_the_review": "Overall: the authors propose to do OOD-label detection by explicitly\ngenerating OOD data with GPT3: a new loss function seems to overcome\nnoise in this process which causes a prior loss function to fail. The\npaper is well-written, creative, and the results promising. But, I\nhave some doubts that the performance gains of the new method (a few\nAUC points) will entice folks to undertake the suggested protocol in\npractice, esp. given the GPT-3 dependence and also, the fact that\nchoosing a slightly different loss function can cause catastrophic\nfailure (e.g., on TREC-10).", "strength_and_weaknesses": "The data augmentation method is straightforward/creative, and I am\nglad to see that it works: it's a direct approach to ood detection\n(i.e., generating ood data) -- I am a fan of things that work :-) The\npaper is easy to follow, and the baselines which the authors compared\nto are thorough. I appreciated the oracle experiments which frame how\nmuch potential room for improvement there may be in follow up\nwork. The evaluation metrics made sense. The authors report averages\nover a number of runs, yielding confidence intervals.\n\nThe biggest negative of the paper, in my view, is that the performance\ngains (particularly of the new loss function) are marginal. The new\nloss function outperforms OE mostly because OE seems to fail on\nTREC-10 (else, it's competitive). The reason for OE failing is\n\"attributed to noisiness in generation\" of the OOD data --- this\ncauses some concern, because while the new loss function seems to\novercome this noise in the other dataset cases, I would be a bit\nnervous using it with any loss function given the massive performance\ndrop on TREC-10: is it worth the risk of involving GPT-3 for a few AUC\npoints? The new data augmentation, while resulting in performance\nimprovements over kFolden and Contrastive, also has more dependencies,\ni.e., access to GPT3. In more detail, a few technical concerns:\n\n- All of the out-of-label-distribution examples are synthetic,\n  whereas, all of the in-label-distribution examples are (presumably)\n  human written from the original corpus. Are models implicitly just\n  doing a \"human-authored\" vs. \"machine-authored\" classification task?\n  This question is partly addressed --- given that the test sets are\n  human-written, the method isn't only doing machine vs. human. But,\n  I'd suggest to add an additional experiment that goes the other way:\n  that is, an experiment where, at test time, the\n  out-of-label-distribution examples are human-written (from the\n  corpora), and the in-label-distribution examples are autogenerated.\n\n- The authors propose a new loss function in eq. 1 that goes beyond\n  the uniform distribution loss of Hendrycks et al. 2019. While I can\n  somewhat qualitatively see why this loss might be different than the\n  uniform loss, the authors argue that it works better because it\n  enables \"some novel examples can be more novel than others.\" Can\n  this claim be validated by examining the logits of the OE model\n  vs. this one? The uniform loss presumably (in a soft way) also\n  allows some examples to be more novel than others. The empirical\n  impact of this loss function is also not entirely clear. OE seems to\n  do badly on TREC-10 for some reason, but it generally seems to match\n  (or sometimes outperform) the new proposed loss on the other corpora.\n\n- I appreciated the oracle experiments --- but, somewhat\n  inexplicably, OE seems to do much better with the gold data\n  available vs. CCL. The reason for this is not explored deeply ---\n  why is that?\n\n- All experiments are conducted with BERT-Base, which is a 4 year old\n  model at this point. While the authors take some steps to run with\n  improved models like Roberta-Large, it would have been nice to see\n  additional scaling runs, e.g., with larger models.\n\n- The authors method can only work with text classification tasks,\n  which is great, but --- some of the competing methods are more\n  general, and the Wikitext + CCL method (which simulates the more\n  general setting of not having GPT-3 but only an OOD set) only\n  marginally outperforms prior solutions (kFolden, for example,\n  doesn't even depend on the external set).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject