MoritzLaurer HF staff commited on
Commit
6d5161a
1 Parent(s): 82f0150

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +488 -50
README.md CHANGED
@@ -1,70 +1,508 @@
1
  ---
2
- license: mit
3
- base_model: microsoft/deberta-v3-base
4
  tags:
5
- - generated_from_trainer
6
- metrics:
7
- - accuracy
8
- model-index:
9
- - name: deberta-v3-base-zeroshot-v1.1-none
10
- results: []
11
  ---
12
 
13
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
- should probably proofread and complete it, then remove this comment. -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- # deberta-v3-base-zeroshot-v1.1-none
17
 
18
- This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on an unknown dataset.
19
- It achieves the following results on the evaluation set:
20
- - Loss: 0.1987
21
- - F1 Macro: 0.6760
22
- - F1 Micro: 0.7335
23
- - Accuracy Balanced: 0.7247
24
- - Accuracy: 0.7335
25
- - Precision Macro: 0.6872
26
- - Recall Macro: 0.7247
27
- - Precision Micro: 0.7335
28
- - Recall Micro: 0.7335
29
 
30
- ## Model description
 
 
 
31
 
32
- More information needed
 
 
 
 
 
 
 
 
 
 
33
 
34
- ## Intended uses & limitations
 
35
 
36
- More information needed
 
37
 
38
- ## Training and evaluation data
39
 
40
- More information needed
 
 
 
 
41
 
42
- ## Training procedure
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
- ### Training hyperparameters
 
45
 
46
- The following hyperparameters were used during training:
47
- - learning_rate: 2e-05
48
- - train_batch_size: 32
49
- - eval_batch_size: 128
50
- - seed: 42
51
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
- - lr_scheduler_type: linear
53
- - lr_scheduler_warmup_ratio: 0.06
54
- - num_epochs: 3
55
 
56
- ### Training results
 
 
 
57
 
58
- | Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Micro | Accuracy Balanced | Accuracy | Precision Macro | Recall Macro | Precision Micro | Recall Micro |
59
- |:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:-----------------:|:--------:|:---------------:|:------------:|:---------------:|:------------:|
60
- | 0.2135 | 1.0 | 30790 | 0.3591 | 0.8469 | 0.8620 | 0.8432 | 0.8620 | 0.8511 | 0.8432 | 0.8620 | 0.8620 |
61
- | 0.1573 | 2.0 | 61580 | 0.3712 | 0.8517 | 0.8664 | 0.8478 | 0.8664 | 0.8562 | 0.8478 | 0.8664 | 0.8664 |
62
- | 0.1038 | 3.0 | 92370 | 0.4572 | 0.8569 | 0.8701 | 0.8556 | 0.8701 | 0.8583 | 0.8556 | 0.8701 | 0.8701 |
 
 
 
 
63
 
 
64
 
65
- ### Framework versions
66
 
67
- - Transformers 4.33.3
68
- - Pytorch 1.11.0+cu113
69
- - Datasets 2.14.6
70
- - Tokenizers 0.12.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
  tags:
5
+ - text-classification
6
+ - zero-shot-classification
7
+ pipeline_tag: zero-shot-classification
8
+ library_name: transformers
9
+ license: mit
 
10
  ---
11
 
12
+ # Model description: deberta-v3-base-zeroshot-v1.1-all-33
13
+ The model is designed for zero-shot classification with the Hugging Face pipeline.
14
+
15
+ The model can do one universal task: determine whether a hypothesis is `true` or `not_true`
16
+ given a text (also called `entailment` vs. `not_entailment`).
17
+ This task format is based on the Natural Language Inference task (NLI).
18
+ The task is so universal that any classification task can be reformulated into the task.
19
+
20
+ A detailed description of how the model was trained and how it can be used is available in this paper: [link to be added]
21
+
22
+ ## Training data
23
+ The model was trained on a mixture of __33 datasets and 387 classes__ that have been reformatted into this universal format.
24
+ 1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli", "ling"
25
+ 2. 28 classification tasks reformatted into the universal NLI format. ~51k cleaned texts were used to avoid overfitting:
26
+ 'amazonpolarity', 'imdb', 'appreviews', 'yelpreviews', 'rottentomatoes',
27
+ 'emotiondair', 'emocontext', 'empathetic',
28
+ 'financialphrasebank', 'banking77', 'massive',
29
+ 'wikitoxic_toxicaggregated', 'wikitoxic_obscene', 'wikitoxic_threat', 'wikitoxic_insult', 'wikitoxic_identityhate',
30
+ 'hateoffensive', 'hatexplain', 'biasframes_offensive', 'biasframes_sex', 'biasframes_intent',
31
+ 'agnews', 'yahootopics',
32
+ 'trueteacher', 'spam', 'wellformedquery',
33
+ 'manifesto', 'capsotu'.
34
 
35
+ See details on each dataset here: https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv
36
 
37
+ Note that compared to other NLI models, this model predicts two classes (`entailment` vs. `not_entailment`)
38
+ as opposed to three classes (entailment/neutral/contradiction)
 
 
 
 
 
 
 
 
 
39
 
40
+ The model was only trained on English data. For __multilingual use-cases__,
41
+ I recommend machine translating texts to English with libraries like [EasyNMT](https://github.com/UKPLab/EasyNMT).
42
+ English-only models tend to perform better than multilingual models and
43
+ validation with English data can be easier if you don't speak all languages in your corpus.
44
 
45
+ ### How to use the model
46
+ #### Simple zero-shot classification pipeline
47
+ ```python
48
+ from transformers import pipeline
49
+ text = "Angela Merkel is a politician in Germany and leader of the CDU"
50
+ hypothesis_template = "This example is about {}"
51
+ classes_verbalized = ["politics", "economy", "entertainment", "environment"]
52
+ zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33")
53
+ output = zeroshot_classifier(text, classes_verbalised, hypothesis_template=hypothesis_template, multi_label=False)
54
+ print(output)
55
+ ```
56
 
57
+ ### Details on data and training
58
+ The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
59
 
60
+ ## Limitations and bias
61
+ The model can only do text classification tasks.
62
 
63
+ Please consult the original DeBERTa paper and the papers for the different datasets for potential biases.
64
 
65
+ ## License
66
+ The base model (DeBERTa-v3) is published under the MIT license.
67
+ The datasets the model was fine-tuned on are published under a diverse set of licenses.
68
+ The following spreadsheet provides an overview of the non-NLI datasets used for fine-tuning.
69
+ The spreadsheets contains information on licenses, the underlying papers etc.: https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv
70
 
71
+ ## Citation
72
+ If you use this model academically, please cite:
73
+ ```
74
+ @article{laurer_less_2023,
75
+ title = {Less {Annotating}, {More} {Classifying}: {Addressing} the {Data} {Scarcity} {Issue} of {Supervised} {Machine} {Learning} with {Deep} {Transfer} {Learning} and {BERT}-{NLI}},
76
+ issn = {1047-1987, 1476-4989},
77
+ shorttitle = {Less {Annotating}, {More} {Classifying}},
78
+ url = {https://www.cambridge.org/core/product/identifier/S1047198723000207/type/journal_article},
79
+ doi = {10.1017/pan.2023.20},
80
+ language = {en},
81
+ urldate = {2023-06-20},
82
+ journal = {Political Analysis},
83
+ author = {Laurer, Moritz and Van Atteveldt, Wouter and Casas, Andreu and Welbers, Kasper},
84
+ month = jun,
85
+ year = {2023},
86
+ pages = {1--33},
87
+ }
88
+ ```
89
 
90
+ ### Ideas for cooperation or questions?
91
+ If you have questions or ideas for cooperation, contact me at m{dot}laurer{at}vu{dot}nl or [LinkedIn](https://www.linkedin.com/in/moritz-laurer/)
92
 
93
+ ### Debugging and issues
94
+ Note that DeBERTa-v3 was released on 06.12.21 and older versions of HF Transformers can have issues running the model (e.g. resulting in an issue with the tokenizer). Using Transformers>=4.13 might solve some issues.
 
 
 
 
 
 
 
95
 
96
+ ### Hypotheses used for classification
97
+ The hypotheses in the tables below were used to fine-tune the model.
98
+ Inspecting them can help users get a feeling for which type of hypotheses and tasks the model was trained on.
99
+ You can formulate your own hypotheses by changing the `hypothesis_template` of the zeroshot pipeline. For example:
100
 
101
+ ```python
102
+ from transformers import pipeline
103
+ text = "Angela Merkel is a politician in Germany and leader of the CDU"
104
+ hypothesis_template = "Merkel is the leader of the party: {}"
105
+ classes_verbalized = ["CDU", "SPD", "Greens"]
106
+ zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33")
107
+ output = zeroshot_classifier(text, classes_verbalised, hypothesis_template=hypothesis_template, multi_label=False)
108
+ print(output)
109
+ ```
110
 
111
+ Note that a few rows in the `massive` and `banking77` datasets contain `nan` because some classes were so ambiguous/unclear that I excluded them from the data.
112
 
 
113
 
114
+ #### wellformedquery
115
+ | label | hypothesis |
116
+ |:----------------|:-----------------------------------------------|
117
+ | not_well_formed | This example is not a well formed Google query |
118
+ | well_formed | This example is a well formed Google query. |
119
+ #### biasframes_sex
120
+ | label | hypothesis |
121
+ |:--------|:-----------------------------------------------------------|
122
+ | not_sex | This example does not contain allusions to sexual content. |
123
+ | sex | This example contains allusions to sexual content. |
124
+ #### biasframes_intent
125
+ | label | hypothesis |
126
+ |:-----------|:-----------------------------------------------------------------|
127
+ | intent | The intent of this example is to be offensive/disrespectful. |
128
+ | not_intent | The intent of this example is not to be offensive/disrespectful. |
129
+ #### biasframes_offensive
130
+ | label | hypothesis |
131
+ |:--------------|:-------------------------------------------------------------------------|
132
+ | not_offensive | This example could not be considered offensive, disrespectful, or toxic. |
133
+ | offensive | This example could be considered offensive, disrespectful, or toxic. |
134
+ #### financialphrasebank
135
+ | label | hypothesis |
136
+ |:---------|:--------------------------------------------------------------------------|
137
+ | negative | The sentiment in this example is negative from an investor's perspective. |
138
+ | neutral | The sentiment in this example is neutral from an investor's perspective. |
139
+ | positive | The sentiment in this example is positive from an investor's perspective. |
140
+ #### rottentomatoes
141
+ | label | hypothesis |
142
+ |:---------|:-----------------------------------------------------------------------|
143
+ | negative | The sentiment in this example rotten tomatoes movie review is negative |
144
+ | positive | The sentiment in this example rotten tomatoes movie review is positive |
145
+ #### amazonpolarity
146
+ | label | hypothesis |
147
+ |:---------|:----------------------------------------------------------------|
148
+ | negative | The sentiment in this example amazon product review is negative |
149
+ | positive | The sentiment in this example amazon product review is positive |
150
+ #### imdb
151
+ | label | hypothesis |
152
+ |:---------|:------------------------------------------------------------|
153
+ | negative | The sentiment in this example imdb movie review is negative |
154
+ | positive | The sentiment in this example imdb movie review is positive |
155
+ #### appreviews
156
+ | label | hypothesis |
157
+ |:---------|:------------------------------------------------------|
158
+ | negative | The sentiment in this example app review is negative. |
159
+ | positive | The sentiment in this example app review is positive. |
160
+ #### yelpreviews
161
+ | label | hypothesis |
162
+ |:---------|:-------------------------------------------------------|
163
+ | negative | The sentiment in this example yelp review is negative. |
164
+ | positive | The sentiment in this example yelp review is positive. |
165
+ #### wikitoxic_toxicaggregated
166
+ | label | hypothesis |
167
+ |:--------------------|:----------------------------------------------------------------|
168
+ | not_toxicaggregated | This example wikipedia comment does not contain toxic language. |
169
+ | toxicaggregated | This example wikipedia comment contains toxic language. |
170
+ #### wikitoxic_obscene
171
+ | label | hypothesis |
172
+ |:------------|:------------------------------------------------------------------|
173
+ | not_obscene | This example wikipedia comment does not contain obscene language. |
174
+ | obscene | This example wikipedia comment contains obscene language. |
175
+ #### wikitoxic_threat
176
+ | label | hypothesis |
177
+ |:-----------|:----------------------------------------------------------|
178
+ | not_threat | This example wikipedia comment does not contain a threat. |
179
+ | threat | This example wikipedia comment contains a threat. |
180
+ #### wikitoxic_insult
181
+ | label | hypothesis |
182
+ |:-----------|:-----------------------------------------------------------|
183
+ | insult | This example wikipedia comment contains an insult. |
184
+ | not_insult | This example wikipedia comment does not contain an insult. |
185
+ #### wikitoxic_identityhate
186
+ | label | hypothesis |
187
+ |:-----------------|:---------------------------------------------------------------|
188
+ | identityhate | This example wikipedia comment contains identity hate. |
189
+ | not_identityhate | This example wikipedia comment does not contain identity hate. |
190
+ #### hateoffensive
191
+ | label | hypothesis |
192
+ |:------------|:------------------------------------------------------------------------|
193
+ | hate_speech | This example tweet contains hate speech. |
194
+ | neither | This example tweet contains neither offensive language nor hate speech. |
195
+ | offensive | This example tweet contains offensive language without hate speech. |
196
+ #### hatexplain
197
+ | label | hypothesis |
198
+ |:------------|:-------------------------------------------------------------------------------------------|
199
+ | hate_speech | This example text from twitter or gab contains hate speech. |
200
+ | neither | This example text from twitter or gab contains neither offensive language nor hate speech. |
201
+ | offensive | This example text from twitter or gab contains offensive language without hate speech. |
202
+ #### spam
203
+ | label | hypothesis |
204
+ |:---------|:------------------------------|
205
+ | not_spam | This example sms is not spam. |
206
+ | spam | This example sms is spam. |
207
+ #### emotiondair
208
+ | label | hypothesis |
209
+ |:---------|:---------------------------------------------------|
210
+ | anger | This example tweet expresses the emotion: anger |
211
+ | fear | This example tweet expresses the emotion: fear |
212
+ | joy | This example tweet expresses the emotion: joy |
213
+ | love | This example tweet expresses the emotion: love |
214
+ | sadness | This example tweet expresses the emotion: sadness |
215
+ | surprise | This example tweet expresses the emotion: surprise |
216
+ #### emocontext
217
+ | label | hypothesis |
218
+ |:--------|:--------------------------------------------------------------------------------------|
219
+ | angry | This example tweet expresses the emotion: anger |
220
+ | happy | This example tweet expresses the emotion: happiness |
221
+ | others | This example tweet does not express any of the emotions: anger, sadness, or happiness |
222
+ | sad | This example tweet expresses the emotion: sadness |
223
+ #### empathetic
224
+ | label | hypothesis |
225
+ |:-------------|:-----------------------------------------------------------|
226
+ | afraid | The main emotion of this example dialogue is: afraid |
227
+ | angry | The main emotion of this example dialogue is: angry |
228
+ | annoyed | The main emotion of this example dialogue is: annoyed |
229
+ | anticipating | The main emotion of this example dialogue is: anticipating |
230
+ | anxious | The main emotion of this example dialogue is: anxious |
231
+ | apprehensive | The main emotion of this example dialogue is: apprehensive |
232
+ | ashamed | The main emotion of this example dialogue is: ashamed |
233
+ | caring | The main emotion of this example dialogue is: caring |
234
+ | confident | The main emotion of this example dialogue is: confident |
235
+ | content | The main emotion of this example dialogue is: content |
236
+ | devastated | The main emotion of this example dialogue is: devastated |
237
+ | disappointed | The main emotion of this example dialogue is: disappointed |
238
+ | disgusted | The main emotion of this example dialogue is: disgusted |
239
+ | embarrassed | The main emotion of this example dialogue is: embarrassed |
240
+ | excited | The main emotion of this example dialogue is: excited |
241
+ | faithful | The main emotion of this example dialogue is: faithful |
242
+ | furious | The main emotion of this example dialogue is: furious |
243
+ | grateful | The main emotion of this example dialogue is: grateful |
244
+ | guilty | The main emotion of this example dialogue is: guilty |
245
+ | hopeful | The main emotion of this example dialogue is: hopeful |
246
+ | impressed | The main emotion of this example dialogue is: impressed |
247
+ | jealous | The main emotion of this example dialogue is: jealous |
248
+ | joyful | The main emotion of this example dialogue is: joyful |
249
+ | lonely | The main emotion of this example dialogue is: lonely |
250
+ | nostalgic | The main emotion of this example dialogue is: nostalgic |
251
+ | prepared | The main emotion of this example dialogue is: prepared |
252
+ | proud | The main emotion of this example dialogue is: proud |
253
+ | sad | The main emotion of this example dialogue is: sad |
254
+ | sentimental | The main emotion of this example dialogue is: sentimental |
255
+ | surprised | The main emotion of this example dialogue is: surprised |
256
+ | terrified | The main emotion of this example dialogue is: terrified |
257
+ | trusting | The main emotion of this example dialogue is: trusting |
258
+ #### agnews
259
+ | label | hypothesis |
260
+ |:---------|:-------------------------------------------------------|
261
+ | Business | This example news text is about business news |
262
+ | Sci/Tech | This example news text is about science and technology |
263
+ | Sports | This example news text is about sports |
264
+ | World | This example news text is about world news |
265
+ #### yahootopics
266
+ | label | hypothesis |
267
+ |:-----------------------|:---------------------------------------------------------------------------------------------------|
268
+ | Business & Finance | This example question from the Yahoo Q&A forum is categorized in the topic: Business & Finance |
269
+ | Computers & Internet | This example question from the Yahoo Q&A forum is categorized in the topic: Computers & Internet |
270
+ | Education & Reference | This example question from the Yahoo Q&A forum is categorized in the topic: Education & Reference |
271
+ | Entertainment & Music | This example question from the Yahoo Q&A forum is categorized in the topic: Entertainment & Music |
272
+ | Family & Relationships | This example question from the Yahoo Q&A forum is categorized in the topic: Family & Relationships |
273
+ | Health | This example question from the Yahoo Q&A forum is categorized in the topic: Health |
274
+ | Politics & Government | This example question from the Yahoo Q&A forum is categorized in the topic: Politics & Government |
275
+ | Science & Mathematics | This example question from the Yahoo Q&A forum is categorized in the topic: Science & Mathematics |
276
+ | Society & Culture | This example question from the Yahoo Q&A forum is categorized in the topic: Society & Culture |
277
+ | Sports | This example question from the Yahoo Q&A forum is categorized in the topic: Sports |
278
+ #### massive
279
+ | label | hypothesis |
280
+ |:-------------------------|:------------------------------------------------------------------------------------------|
281
+ | alarm_query | The example utterance is a query about alarms. |
282
+ | alarm_remove | The intent of this example utterance is to remove an alarm. |
283
+ | alarm_set | The intent of the example utterance is to set an alarm. |
284
+ | audio_volume_down | The intent of the example utterance is to lower the volume. |
285
+ | audio_volume_mute | The intent of this example utterance is to mute the volume. |
286
+ | audio_volume_other | The example utterance is related to audio volume. |
287
+ | audio_volume_up | The intent of this example utterance is turning the audio volume up. |
288
+ | calendar_query | The example utterance is a query about a calendar. |
289
+ | calendar_remove | The intent of the example utterance is to remove something from a calendar. |
290
+ | calendar_set | The intent of this example utterance is to set something in a calendar. |
291
+ | cooking_query | The example utterance is a query about cooking. |
292
+ | cooking_recipe | This example utterance is about cooking recipies. |
293
+ | datetime_convert | The example utterance is related to date time changes or conversion. |
294
+ | datetime_query | The intent of this example utterance is a datetime query. |
295
+ | email_addcontact | The intent of this example utterance is adding an email address to contacts. |
296
+ | email_query | The example utterance is a query about emails. |
297
+ | email_querycontact | The intent of this example utterance is to query contact details. |
298
+ | email_sendemail | The intent of the example utterance is to send an email. |
299
+ | general_greet | This example utterance is a general greet. |
300
+ | general_joke | The intent of the example utterance is to hear a joke. |
301
+ | general_quirky | nan |
302
+ | iot_cleaning | The intent of the example utterance is for an IoT device to start cleaning. |
303
+ | iot_coffee | The intent of this example utterance is for an IoT device to make coffee. |
304
+ | iot_hue_lightchange | The intent of this example utterance is changing the light. |
305
+ | iot_hue_lightdim | The intent of the example utterance is to dim the lights. |
306
+ | iot_hue_lightoff | The example utterance is related to turning the lights off. |
307
+ | iot_hue_lighton | The example utterance is related to turning the lights on. |
308
+ | iot_hue_lightup | The intent of this example utterance is to brighten lights. |
309
+ | iot_wemo_off | The intent of this example utterance is turning an IoT device off. |
310
+ | iot_wemo_on | The intent of the example utterance is to turn an IoT device on. |
311
+ | lists_createoradd | The example utterance is related to creating or adding to lists. |
312
+ | lists_query | The example utterance is a query about a list. |
313
+ | lists_remove | The intent of this example utterance is to remove a list or remove something from a list. |
314
+ | music_dislikeness | The intent of this example utterance is signalling music dislike. |
315
+ | music_likeness | The example utterance is related to liking music. |
316
+ | music_query | The example utterance is a query about music. |
317
+ | music_settings | The intent of the example utterance is to change music settings. |
318
+ | news_query | The example utterance is a query about the news. |
319
+ | play_audiobook | The example utterance is related to playing audiobooks. |
320
+ | play_game | The intent of this example utterance is to start playing a game. |
321
+ | play_music | The intent of this example utterance is for an IoT device to play music. |
322
+ | play_podcasts | The example utterance is related to playing podcasts. |
323
+ | play_radio | The intent of the example utterance is to play something on the radio. |
324
+ | qa_currency | This example utteranceis about currencies. |
325
+ | qa_definition | The example utterance is a query about a definition. |
326
+ | qa_factoid | The example utterance is a factoid question. |
327
+ | qa_maths | The example utterance is a question about maths. |
328
+ | qa_stock | This example utterance is about stocks. |
329
+ | recommendation_events | This example utterance is about event recommendations. |
330
+ | recommendation_locations | The intent of this example utterance is receiving recommendations for good locations. |
331
+ | recommendation_movies | This example utterance is about movie recommendations. |
332
+ | social_post | The example utterance is about social media posts. |
333
+ | social_query | The example utterance is a query about a social network. |
334
+ | takeaway_order | The intent of this example utterance is to order takeaway food. |
335
+ | takeaway_query | This example utterance is about takeaway food. |
336
+ | transport_query | The example utterance is a query about transport or travels. |
337
+ | transport_taxi | The intent of this example utterance is to get a taxi. |
338
+ | transport_ticket | This example utterance is about transport tickets. |
339
+ | transport_traffic | This example utterance is about transport or traffic. |
340
+ | weather_query | This example utterance is a query about the wheather. |
341
+ #### banking77
342
+ | label | hypothesis |
343
+ |:-------------------------------------------------|:----------------------------------------------------------------------------------------------------------|
344
+ | Refund_not_showing_up | This customer example message is about a refund not showing up. |
345
+ | activate_my_card | This banking customer example message is about activating a card. |
346
+ | age_limit | This banking customer example message is related to age limits. |
347
+ | apple_pay_or_google_pay | This banking customer example message is about apple pay or google pay |
348
+ | atm_support | This banking customer example message requests ATM support. |
349
+ | automatic_top_up | This banking customer example message is about automatic top up. |
350
+ | balance_not_updated_after_bank_transfer | This banking customer example message is about a balance not updated after a transfer. |
351
+ | balance_not_updated_after_cheque_or_cash_deposit | This banking customer example message is about a balance not updated after a cheque or cash deposit. |
352
+ | beneficiary_not_allowed | This banking customer example message is related to a beneficiary not being allowed or a failed transfer. |
353
+ | cancel_transfer | This banking customer example message is related to the cancellation of a transfer. |
354
+ | card_about_to_expire | This banking customer example message is related to the expiration of a card. |
355
+ | card_acceptance | This banking customer example message is related to the scope of acceptance of a card. |
356
+ | card_arrival | This banking customer example message is about the arrival of a card. |
357
+ | card_delivery_estimate | This banking customer example message is about a card delivery estimate or timing. |
358
+ | card_linking | nan |
359
+ | card_not_working | This banking customer example message is about a card not working. |
360
+ | card_payment_fee_charged | This banking customer example message is about a card payment fee. |
361
+ | card_payment_not_recognised | This banking customer example message is about a payment the customer does not recognise. |
362
+ | card_payment_wrong_exchange_rate | This banking customer example message is about a wrong exchange rate. |
363
+ | card_swallowed | This banking customer example message is about a card swallowed by a machine. |
364
+ | cash_withdrawal_charge | This banking customer example message is about a cash withdrawal charge. |
365
+ | cash_withdrawal_not_recognised | This banking customer example message is about an unrecognised cash withdrawal. |
366
+ | change_pin | This banking customer example message is about changing a pin code. |
367
+ | compromised_card | This banking customer example message is about a compromised card. |
368
+ | contactless_not_working | This banking customer example message is about contactless not working |
369
+ | country_support | This banking customer example message is about country-specific support. |
370
+ | declined_card_payment | This banking customer example message is about a declined card payment. |
371
+ | declined_cash_withdrawal | This banking customer example message is about a declined cash withdrawal. |
372
+ | declined_transfer | This banking customer example message is about a declined transfer. |
373
+ | direct_debit_payment_not_recognised | This banking customer example message is about an unrecognised direct debit payment. |
374
+ | disposable_card_limits | This banking customer example message is about the limits of disposable cards. |
375
+ | edit_personal_details | This banking customer example message is about editing personal details. |
376
+ | exchange_charge | This banking customer example message is about exchange rate charges. |
377
+ | exchange_rate | This banking customer example message is about exchange rates. |
378
+ | exchange_via_app | nan |
379
+ | extra_charge_on_statement | This banking customer example message is about an extra charge. |
380
+ | failed_transfer | This banking customer example message is about a failed transfer. |
381
+ | fiat_currency_support | This banking customer example message is about fiat currency support |
382
+ | get_disposable_virtual_card | This banking customer example message is about getting a disposable virtual card. |
383
+ | get_physical_card | nan |
384
+ | getting_spare_card | This banking customer example message is about getting a spare card. |
385
+ | getting_virtual_card | This banking customer example message is about getting a virtual card. |
386
+ | lost_or_stolen_card | This banking customer example message is about a lost or stolen card. |
387
+ | lost_or_stolen_phone | This banking customer example message is about a lost or stolen phone. |
388
+ | order_physical_card | This banking customer example message is about ordering a card. |
389
+ | passcode_forgotten | This banking customer example message is about a forgotten passcode. |
390
+ | pending_card_payment | This banking customer example message is about a pending card payment. |
391
+ | pending_cash_withdrawal | This banking customer example message is about a pending cash withdrawal. |
392
+ | pending_top_up | This banking customer example message is about a pending top up. |
393
+ | pending_transfer | This banking customer example message is about a pending transfer. |
394
+ | pin_blocked | This banking customer example message is about a blocked pin. |
395
+ | receiving_money | This banking customer example message is about receiving money. |
396
+ | request_refund | This banking customer example message is about a refund request. |
397
+ | reverted_card_payment? | This banking customer example message is about reverting a card payment. |
398
+ | supported_cards_and_currencies | nan |
399
+ | terminate_account | This banking customer example message is about terminating an account. |
400
+ | top_up_by_bank_transfer_charge | nan |
401
+ | top_up_by_card_charge | This banking customer example message is about the charge for topping up by card. |
402
+ | top_up_by_cash_or_cheque | This banking customer example message is about topping up by cash or cheque. |
403
+ | top_up_failed | This banking customer example message is about top up issues or failures. |
404
+ | top_up_limits | This banking customer example message is about top up limitations. |
405
+ | top_up_reverted | This banking customer example message is about issues with topping up. |
406
+ | topping_up_by_card | This banking customer example message is about topping up by card. |
407
+ | transaction_charged_twice | This banking customer example message is about a transaction charged twice. |
408
+ | transfer_fee_charged | This banking customer example message is about an issue with a transfer fee charge. |
409
+ | transfer_into_account | This banking customer example message is about transfers into the customer's own account. |
410
+ | transfer_not_received_by_recipient | This banking customer example message is about a transfer that has not arrived yet. |
411
+ | transfer_timing | This banking customer example message is about transfer timing. |
412
+ | unable_to_verify_identity | This banking customer example message is about an issue with identity verification. |
413
+ | verify_my_identity | This banking customer example message is about identity verification. |
414
+ | verify_source_of_funds | This banking customer example message is about the source of funds. |
415
+ | verify_top_up | This banking customer example message is about verification and top ups |
416
+ | virtual_card_not_working | This banking customer example message is about a virtual card not working |
417
+ | visa_or_mastercard | This banking customer example message is about types of bank cards. |
418
+ | why_verify_identity | This banking customer example message questions why identity verification is necessary. |
419
+ | wrong_amount_of_cash_received | This banking customer example message is about a wrong amount of cash received. |
420
+ | wrong_exchange_rate_for_cash_withdrawal | This banking customer example message is about a wrong exchange rate for a cash withdrawal. |
421
+ #### trueteacher
422
+ | label | hypothesis |
423
+ |:-----------------------|:---------------------------------------------------------------------|
424
+ | factually_consistent | The example summary is factually consistent with the full article. |
425
+ | factually_inconsistent | The example summary is factually inconsistent with the full article. |
426
+ #### capsotu
427
+ | label | hypothesis |
428
+ |:----------------------|:----------------------------------------------------------------------------------------------------------|
429
+ | Agriculture | This example text from a US presidential speech is about agriculture |
430
+ | Civil Rights | This example text from a US presidential speech is about civil rights or minorities or civil liberties |
431
+ | Culture | This example text from a US presidential speech is about cultural policy |
432
+ | Defense | This example text from a US presidential speech is about defense or military |
433
+ | Domestic Commerce | This example text from a US presidential speech is about banking or finance or commerce |
434
+ | Education | This example text from a US presidential speech is about education |
435
+ | Energy | This example text from a US presidential speech is about energy or electricity or fossil fuels |
436
+ | Environment | This example text from a US presidential speech is about the environment or water or waste or pollution |
437
+ | Foreign Trade | This example text from a US presidential speech is about foreign trade |
438
+ | Government Operations | This example text from a US presidential speech is about government operations or administration |
439
+ | Health | This example text from a US presidential speech is about health |
440
+ | Housing | This example text from a US presidential speech is about community development or housing issues |
441
+ | Immigration | This example text from a US presidential speech is about migration |
442
+ | International Affairs | This example text from a US presidential speech is about international affairs or foreign aid |
443
+ | Labor | This example text from a US presidential speech is about employment or labour |
444
+ | Law and Crime | This example text from a US presidential speech is about law, crime or family issues |
445
+ | Macroeconomics | This example text from a US presidential speech is about macroeconomics |
446
+ | Public Lands | This example text from a US presidential speech is about public lands or water management |
447
+ | Social Welfare | This example text from a US presidential speech is about social welfare |
448
+ | Technology | This example text from a US presidential speech is about space or science or technology or communications |
449
+ | Transportation | This example text from a US presidential speech is about transportation |
450
+ #### manifesto
451
+ | label | hypothesis |
452
+ |:-------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
453
+ | Agriculture and Farmers: Positive | This example text from a political party manifesto is positive towards policies for agriculture and farmers |
454
+ | Anti-Growth Economy: Positive | This example text from a political party manifesto is in favour of anti-growth politics |
455
+ | Anti-Imperialism | This example text from a political party manifesto is anti-imperialistic, for example against controlling other countries and for greater self-government of colonies |
456
+ | Centralisation | This example text from a political party manifesto is in favour of political centralisation |
457
+ | Civic Mindedness: Positive | This example text from a political party manifesto is positive towards national solidarity, civil society or appeals for public spiritedness or against anti-social attitudes |
458
+ | Constitutionalism: Negative | This example text from a political party manifesto is positive towards constitutionalism |
459
+ | Constitutionalism: Positive | This example text from a political party manifesto is positive towards constitutionalism and the status quo of the constitution |
460
+ | Controlled Economy | This example text from a political party manifesto is supportive of direct government control of the economy, e.g. price control or minimum wages |
461
+ | Corporatism/Mixed Economy | This example text from a political party manifesto is positive towards cooperation of government, employers, and trade unions simultaneously |
462
+ | Culture: Positive | This example text from a political party manifesto is in favour of cultural policies or leisure facilities, for example museus, libraries or public sport clubs |
463
+ | Decentralization | This example text from a political party manifesto is for decentralisation or federalism |
464
+ | Democracy | This example text from a political party manifesto favourably mentions democracy or democratic procedures or institutions |
465
+ | Economic Goals | This example text from a political party manifesto is a broad/general statement on economic goals without specifics |
466
+ | Economic Growth: Positive | This example text from a political party manifesto is supportive of economic growth, for example facilitation of more production or government aid for growth |
467
+ | Economic Orthodoxy | This example text from a political party manifesto is for economic orthodoxy, for example reduction of budget deficits, thrift or a strong currency |
468
+ | Economic Planning | This example text from a political party manifesto is positive towards government economic planning, e.g. policy plans or strategies |
469
+ | Education Expansion | This example text from a political party manifesto is about the need to expand/improve policy on education |
470
+ | Education Limitation | This example text from a political party manifesto is sceptical towards state expenditure on education, for example in favour of study fees or private schools |
471
+ | Environmental Protection | This example text from a political party manifesto is in favour of environmental protection, e.g. fighting climate change or 'green' policies or preservation of natural resources or animal rights |
472
+ | Equality: Positive | This example text from a political party manifesto is positive towards equality or social justice, e.g. protection of underprivileged groups or fair distribution of resources |
473
+ | European Community/Union: Negative | This example text from a political party manifesto negatively mentions the EU or European Community |
474
+ | European Community/Union: Positive | This example text from a political party manifesto is positive towards the EU or European Community, for example EU expansion and integration |
475
+ | Foreign Special Relationships: Negative | This example text from a political party manifesto is negative towards particular countries |
476
+ | Foreign Special Relationships: Positive | This example text from a political party manifesto is positive towards particular countries |
477
+ | Free Market Economy | This example text from a political party manifesto is in favour of a free market economy and capitalism |
478
+ | Freedom and Human Rights | This example text from a political party manifesto is in favour of freedom and human rights, for example freedom of speech, assembly or against state coercion or for individualism |
479
+ | Governmental and Administrative Efficiency | This example text from a political party manifesto is in favour of efficiency in government/administration, for example by restructuring civil service or improving bureaucracy |
480
+ | Incentives: Positive | This example text from a political party manifesto is favourable towards supply side economic policies supporting businesses, for example for incentives like subsidies or tax breaks |
481
+ | Internationalism: Negative | This example text from a political party manifesto is sceptical of internationalism, for example negative towards international cooperation, in favour of national sovereignty and unilaterialism |
482
+ | Internationalism: Positive | This example text from a political party manifesto is in favour of international cooperation with other countries, for example mentions the need for aid to developing countries, or global governance |
483
+ | Keynesian Demand Management | This example text from a political party manifesto is for keynesian demand management and demand side economic policies |
484
+ | Labour Groups: Negative | This example text from a political party manifesto is negative towards labour groups and unions |
485
+ | Labour Groups: Positive | This example text from a political party manifesto is positive towards labour groups, for example for good working conditions, fair wages or unions |
486
+ | Law and Order: Positive | This example text from a political party manifesto is positive towards law and order and strict law enforcement |
487
+ | Market Regulation | This example text from a political party manifesto is supports market regulation for a fair and open market, for example for consumer protection or for increased competition or for social market economy |
488
+ | Marxist Analysis | This example text from a political party manifesto is positive towards Marxist-Leninist ideas or uses specific Marxist terminology |
489
+ | Middle Class and Professional Groups | This example text from a political party manifesto favourably references the middle class, e.g. white colar groups or the service sector |
490
+ | Military: Negative | This example text from a political party manifesto is negative towards the military, for example for decreasing military spending or disarmament |
491
+ | Military: Positive | This example text from a political party manifesto is positive towards the military, for example for military spending or rearmament or military treaty obligations |
492
+ | Multiculturalism: Negative | This example text from a political party manifesto is sceptical towards multiculturalism, or for cultural integration or appeals to cultural homogeneity in society |
493
+ | Multiculturalism: Positive | This example text from a political party manifesto favourably mentions cultural diversity, for example for freedom of religion or linguistic heritages |
494
+ | National Way of Life: Negative | This example text from a political party manifesto unfavourably mentions a country's nation and history, for example sceptical towards patriotism or national pride |
495
+ | National Way of Life: Positive | This example text from a political party manifesto is positive towards the national way of life and history, for example pride of citizenship or appeals to patriotism |
496
+ | Nationalisation | This example text from a political party manifesto is positive towards government ownership of industries or land or for economic nationalisation |
497
+ | Non-economic Demographic Groups | This example text from a political party manifesto favourably mentions non-economic demographic groups like women, students or specific age groups |
498
+ | Peace | This example text from a political party manifesto is positive towards peace and peaceful means of solving crises, for example in favour of negotiations and ending wars |
499
+ | Political Authority | This example text from a political party manifesto mentions the speaker's competence to govern or other party's lack of such competence, or favourably mentions a strong/stable government |
500
+ | Political Corruption | This example text from a political party manifesto is negative towards political corruption or abuse of political/bureaucratic power |
501
+ | Protectionism: Negative | This example text from a political party manifesto is negative towards protectionism, in favour of free trade |
502
+ | Protectionism: Positive | This example text from a political party manifesto is in favour of protectionism, for example tariffs, export subsidies |
503
+ | Technology and Infrastructure: Positive | This example text from a political party manifesto is about technology and infrastructure, e.g. the importance of modernisation of industry, or supportive of public spending on infrastructure/tech |
504
+ | Traditional Morality: Negative | This example text from a political party manifesto is negative towards traditional morality, for example against religious moral values, for divorce or abortion, for modern families or separation of church and state |
505
+ | Traditional Morality: Positive | This example text from a political party manifesto is favourable towards traditional or religious values, for example for censorship of immoral behavour, for traditional family values or religious institutions |
506
+ | Underprivileged Minority Groups | This example text from a political party manifesto favourably mentions underprivileged minorities, for example handicapped, homosexuals or immigrants |
507
+ | Welfare State Expansion | This example text from a political party manifesto is positive towards the welfare state, e.g. health care, pensions or social housing |
508
+ | Welfare State Limitation | This example text from a political party manifesto is for limiting the welfare state, for example public funding for social services or social security, e.g. private care before state care |