|
--- |
|
language: |
|
- en |
|
tags: |
|
- text-classification |
|
- zero-shot-classification |
|
pipeline_tag: zero-shot-classification |
|
library_name: transformers |
|
license: mit |
|
--- |
|
|
|
# Model description: deberta-v3-large-zeroshot-v1.1-all-33 |
|
The model is designed for zero-shot classification with the Hugging Face pipeline. |
|
|
|
The model can do one universal task: determine whether a hypothesis is `true` or `not_true` |
|
given a text (also called `entailment` vs. `not_entailment`). |
|
This task format is based on the Natural Language Inference task (NLI). |
|
The task is so universal that any classification task can be reformulated into the task. |
|
|
|
A detailed description of how the model was trained and how it can be used is available in this paper: [link to be added] |
|
|
|
## Training data |
|
The model was trained on a mixture of __33 datasets and 387 classes__ that have been reformatted into this universal format. |
|
1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli", "ling" |
|
2. 28 classification tasks reformatted into the universal NLI format. ~51k cleaned texts were used to avoid overfitting: |
|
'amazonpolarity', 'imdb', 'appreviews', 'yelpreviews', 'rottentomatoes', |
|
'emotiondair', 'emocontext', 'empathetic', |
|
'financialphrasebank', 'banking77', 'massive', |
|
'wikitoxic_toxicaggregated', 'wikitoxic_obscene', 'wikitoxic_threat', 'wikitoxic_insult', 'wikitoxic_identityhate', |
|
'hateoffensive', 'hatexplain', 'biasframes_offensive', 'biasframes_sex', 'biasframes_intent', |
|
'agnews', 'yahootopics', |
|
'trueteacher', 'spam', 'wellformedquery', |
|
'manifesto', 'capsotu'. |
|
|
|
See details on each dataset here: https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv |
|
|
|
Note that compared to other NLI models, this model predicts two classes (`entailment` vs. `not_entailment`) |
|
as opposed to three classes (entailment/neutral/contradiction) |
|
|
|
The model was only trained on English data. For __multilingual use-cases__, |
|
I recommend machine translating texts to English with libraries like [EasyNMT](https://github.com/UKPLab/EasyNMT). |
|
English-only models tend to perform better than multilingual models and |
|
validation with English data can be easier if you don't speak all languages in your corpus. |
|
|
|
### How to use the model |
|
#### Simple zero-shot classification pipeline |
|
```python |
|
from transformers import pipeline |
|
text = "Angela Merkel is a politician in Germany and leader of the CDU" |
|
hypothesis_template = "This example is about {}" |
|
classes_verbalized = ["politics", "economy", "entertainment", "environment"] |
|
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33") |
|
output = zeroshot_classifier(text, classes_verbalised, hypothesis_template=hypothesis_template, multi_label=False) |
|
print(output) |
|
``` |
|
|
|
### Details on data and training |
|
The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main |
|
|
|
## Limitations and bias |
|
The model can only do text classification tasks. |
|
|
|
Please consult the original DeBERTa paper and the papers for the different datasets for potential biases. |
|
|
|
## Metrics |
|
|
|
Balanced accuracy metrics on all datasets. `deberta-v3-large-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset. |
|
To calculate this metrics, 28 different models were trained, each with one dataset held out from training to simulate a zeroshot setup. |
|
`deberta-v3-large-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting. |
|
(The metrics in the last column are therefore not strictly zeroshot.) |
|
|
|
|
|
| | deberta-v3-large-mnli-fever-anli-ling-wanli-binary | deberta-v3-large-zeroshot-v1.1-heldout | deberta-v3-large-zeroshot-v1.1-all-33 | |
|
|:---------------------------|----------------------------:|-----------------------------------------:|----------------------------------------:| |
|
| datasets mean (w/o nli) | 64.1 | 73.4 | 85.2 | |
|
| amazonpolarity (2) | 94.7 | 96.6 | 96.8 | |
|
| imdb (2) | 90.3 | 95.2 | 95.5 | |
|
| appreviews (2) | 93.6 | 94.3 | 94.7 | |
|
| yelpreviews (2) | 98.5 | 98.4 | 98.9 | |
|
| rottentomatoes (2) | 83.9 | 90.5 | 90.8 | |
|
| emotiondair (6) | 49.2 | 42.1 | 72.1 | |
|
| emocontext (4) | 57 | 69.3 | 82.4 | |
|
| empathetic (32) | 42 | 34.4 | 58 | |
|
| financialphrasebank (3) | 77.4 | 77.5 | 91.9 | |
|
| banking77 (72) | 29.1 | 52.8 | 72.2 | |
|
| massive (59) | 47.3 | 64.7 | 77.3 | |
|
| wikitoxic_toxicaggreg (2) | 81.6 | 86.6 | 91 | |
|
| wikitoxic_obscene (2) | 85.9 | 91.9 | 93.1 | |
|
| wikitoxic_threat (2) | 77.9 | 93.7 | 97.6 | |
|
| wikitoxic_insult (2) | 77.8 | 91.1 | 92.3 | |
|
| wikitoxic_identityhate (2) | 86.4 | 89.8 | 95.7 | |
|
| hateoffensive (3) | 62.8 | 66.5 | 88.4 | |
|
| hatexplain (3) | 46.9 | 61 | 76.9 | |
|
| biasframes_offensive (2) | 62.5 | 86.6 | 89 | |
|
| biasframes_sex (2) | 87.6 | 89.6 | 92.6 | |
|
| biasframes_intent (2) | 54.8 | 88.6 | 89.9 | |
|
| agnews (4) | 81.9 | 82.8 | 90.9 | |
|
| yahootopics (10) | 37.7 | 65.6 | 74.3 | |
|
| trueteacher (2) | 51.2 | 54.9 | 86.6 | |
|
| spam (2) | 52.6 | 51.8 | 97.1 | |
|
| wellformedquery (2) | 49.9 | 40.4 | 82.7 | |
|
| manifesto (56) | 10.6 | 29.4 | 44.1 | |
|
| capsotu (21) | 23.2 | 69.4 | 74 | |
|
| mnli_m (2) | 93.1 | nan | 93.1 | |
|
| mnli_mm (2) | 93.2 | nan | 93.2 | |
|
| fevernli (2) | 89.3 | nan | 89.5 | |
|
| anli_r1 (2) | 87.9 | nan | 87.3 | |
|
| anli_r2 (2) | 76.3 | nan | 78 | |
|
| anli_r3 (2) | 73.6 | nan | 74.1 | |
|
| wanli (2) | 82.8 | nan | 82.7 | |
|
| lingnli (2) | 90.2 | nan | 89.6 | |
|
|
|
|
|
|
|
## License |
|
The base model (DeBERTa-v3) is published under the MIT license. |
|
The datasets the model was fine-tuned on are published under a diverse set of licenses. |
|
The following spreadsheet provides an overview of the non-NLI datasets used for fine-tuning. |
|
The spreadsheets contains information on licenses, the underlying papers etc.: https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv |
|
|
|
## Citation |
|
If you use this model academically, please cite: |
|
``` |
|
@article{laurer_less_2023, |
|
title = {Less {Annotating}, {More} {Classifying}: {Addressing} the {Data} {Scarcity} {Issue} of {Supervised} {Machine} {Learning} with {Deep} {Transfer} {Learning} and {BERT}-{NLI}}, |
|
issn = {1047-1987, 1476-4989}, |
|
shorttitle = {Less {Annotating}, {More} {Classifying}}, |
|
url = {https://www.cambridge.org/core/product/identifier/S1047198723000207/type/journal_article}, |
|
doi = {10.1017/pan.2023.20}, |
|
language = {en}, |
|
urldate = {2023-06-20}, |
|
journal = {Political Analysis}, |
|
author = {Laurer, Moritz and Van Atteveldt, Wouter and Casas, Andreu and Welbers, Kasper}, |
|
month = jun, |
|
year = {2023}, |
|
pages = {1--33}, |
|
} |
|
``` |
|
|
|
### Ideas for cooperation or questions? |
|
If you have questions or ideas for cooperation, contact me at m{dot}laurer{at}vu{dot}nl or [LinkedIn](https://www.linkedin.com/in/moritz-laurer/) |
|
|
|
### Debugging and issues |
|
Note that DeBERTa-v3 was released on 06.12.21 and older versions of HF Transformers can have issues running the model (e.g. resulting in an issue with the tokenizer). Using Transformers>=4.13 might solve some issues. |
|
|
|
### Hypotheses used for classification |
|
The hypotheses in the tables below were used to fine-tune the model. |
|
Inspecting them can help users get a feeling for which type of hypotheses and tasks the model was trained on. |
|
You can formulate your own hypotheses by changing the `hypothesis_template` of the zeroshot pipeline. For example: |
|
|
|
```python |
|
from transformers import pipeline |
|
text = "Angela Merkel is a politician in Germany and leader of the CDU" |
|
hypothesis_template = "Merkel is the leader of the party: {}" |
|
classes_verbalized = ["CDU", "SPD", "Greens"] |
|
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33") |
|
output = zeroshot_classifier(text, classes_verbalised, hypothesis_template=hypothesis_template, multi_label=False) |
|
print(output) |
|
``` |
|
|
|
Note that a few rows in the `massive` and `banking77` datasets contain `nan` because some classes were so ambiguous/unclear that I excluded them from the data. |
|
|
|
|
|
#### wellformedquery |
|
| label | hypothesis | |
|
|:----------------|:-----------------------------------------------| |
|
| not_well_formed | This example is not a well formed Google query | |
|
| well_formed | This example is a well formed Google query. | |
|
#### biasframes_sex |
|
| label | hypothesis | |
|
|:--------|:-----------------------------------------------------------| |
|
| not_sex | This example does not contain allusions to sexual content. | |
|
| sex | This example contains allusions to sexual content. | |
|
#### biasframes_intent |
|
| label | hypothesis | |
|
|:-----------|:-----------------------------------------------------------------| |
|
| intent | The intent of this example is to be offensive/disrespectful. | |
|
| not_intent | The intent of this example is not to be offensive/disrespectful. | |
|
#### biasframes_offensive |
|
| label | hypothesis | |
|
|:--------------|:-------------------------------------------------------------------------| |
|
| not_offensive | This example could not be considered offensive, disrespectful, or toxic. | |
|
| offensive | This example could be considered offensive, disrespectful, or toxic. | |
|
#### financialphrasebank |
|
| label | hypothesis | |
|
|:---------|:--------------------------------------------------------------------------| |
|
| negative | The sentiment in this example is negative from an investor's perspective. | |
|
| neutral | The sentiment in this example is neutral from an investor's perspective. | |
|
| positive | The sentiment in this example is positive from an investor's perspective. | |
|
#### rottentomatoes |
|
| label | hypothesis | |
|
|:---------|:-----------------------------------------------------------------------| |
|
| negative | The sentiment in this example rotten tomatoes movie review is negative | |
|
| positive | The sentiment in this example rotten tomatoes movie review is positive | |
|
#### amazonpolarity |
|
| label | hypothesis | |
|
|:---------|:----------------------------------------------------------------| |
|
| negative | The sentiment in this example amazon product review is negative | |
|
| positive | The sentiment in this example amazon product review is positive | |
|
#### imdb |
|
| label | hypothesis | |
|
|:---------|:------------------------------------------------------------| |
|
| negative | The sentiment in this example imdb movie review is negative | |
|
| positive | The sentiment in this example imdb movie review is positive | |
|
#### appreviews |
|
| label | hypothesis | |
|
|:---------|:------------------------------------------------------| |
|
| negative | The sentiment in this example app review is negative. | |
|
| positive | The sentiment in this example app review is positive. | |
|
#### yelpreviews |
|
| label | hypothesis | |
|
|:---------|:-------------------------------------------------------| |
|
| negative | The sentiment in this example yelp review is negative. | |
|
| positive | The sentiment in this example yelp review is positive. | |
|
#### wikitoxic_toxicaggregated |
|
| label | hypothesis | |
|
|:--------------------|:----------------------------------------------------------------| |
|
| not_toxicaggregated | This example wikipedia comment does not contain toxic language. | |
|
| toxicaggregated | This example wikipedia comment contains toxic language. | |
|
#### wikitoxic_obscene |
|
| label | hypothesis | |
|
|:------------|:------------------------------------------------------------------| |
|
| not_obscene | This example wikipedia comment does not contain obscene language. | |
|
| obscene | This example wikipedia comment contains obscene language. | |
|
#### wikitoxic_threat |
|
| label | hypothesis | |
|
|:-----------|:----------------------------------------------------------| |
|
| not_threat | This example wikipedia comment does not contain a threat. | |
|
| threat | This example wikipedia comment contains a threat. | |
|
#### wikitoxic_insult |
|
| label | hypothesis | |
|
|:-----------|:-----------------------------------------------------------| |
|
| insult | This example wikipedia comment contains an insult. | |
|
| not_insult | This example wikipedia comment does not contain an insult. | |
|
#### wikitoxic_identityhate |
|
| label | hypothesis | |
|
|:-----------------|:---------------------------------------------------------------| |
|
| identityhate | This example wikipedia comment contains identity hate. | |
|
| not_identityhate | This example wikipedia comment does not contain identity hate. | |
|
#### hateoffensive |
|
| label | hypothesis | |
|
|:------------|:------------------------------------------------------------------------| |
|
| hate_speech | This example tweet contains hate speech. | |
|
| neither | This example tweet contains neither offensive language nor hate speech. | |
|
| offensive | This example tweet contains offensive language without hate speech. | |
|
#### hatexplain |
|
| label | hypothesis | |
|
|:------------|:-------------------------------------------------------------------------------------------| |
|
| hate_speech | This example text from twitter or gab contains hate speech. | |
|
| neither | This example text from twitter or gab contains neither offensive language nor hate speech. | |
|
| offensive | This example text from twitter or gab contains offensive language without hate speech. | |
|
#### spam |
|
| label | hypothesis | |
|
|:---------|:------------------------------| |
|
| not_spam | This example sms is not spam. | |
|
| spam | This example sms is spam. | |
|
#### emotiondair |
|
| label | hypothesis | |
|
|:---------|:---------------------------------------------------| |
|
| anger | This example tweet expresses the emotion: anger | |
|
| fear | This example tweet expresses the emotion: fear | |
|
| joy | This example tweet expresses the emotion: joy | |
|
| love | This example tweet expresses the emotion: love | |
|
| sadness | This example tweet expresses the emotion: sadness | |
|
| surprise | This example tweet expresses the emotion: surprise | |
|
#### emocontext |
|
| label | hypothesis | |
|
|:--------|:--------------------------------------------------------------------------------------| |
|
| angry | This example tweet expresses the emotion: anger | |
|
| happy | This example tweet expresses the emotion: happiness | |
|
| others | This example tweet does not express any of the emotions: anger, sadness, or happiness | |
|
| sad | This example tweet expresses the emotion: sadness | |
|
#### empathetic |
|
| label | hypothesis | |
|
|:-------------|:-----------------------------------------------------------| |
|
| afraid | The main emotion of this example dialogue is: afraid | |
|
| angry | The main emotion of this example dialogue is: angry | |
|
| annoyed | The main emotion of this example dialogue is: annoyed | |
|
| anticipating | The main emotion of this example dialogue is: anticipating | |
|
| anxious | The main emotion of this example dialogue is: anxious | |
|
| apprehensive | The main emotion of this example dialogue is: apprehensive | |
|
| ashamed | The main emotion of this example dialogue is: ashamed | |
|
| caring | The main emotion of this example dialogue is: caring | |
|
| confident | The main emotion of this example dialogue is: confident | |
|
| content | The main emotion of this example dialogue is: content | |
|
| devastated | The main emotion of this example dialogue is: devastated | |
|
| disappointed | The main emotion of this example dialogue is: disappointed | |
|
| disgusted | The main emotion of this example dialogue is: disgusted | |
|
| embarrassed | The main emotion of this example dialogue is: embarrassed | |
|
| excited | The main emotion of this example dialogue is: excited | |
|
| faithful | The main emotion of this example dialogue is: faithful | |
|
| furious | The main emotion of this example dialogue is: furious | |
|
| grateful | The main emotion of this example dialogue is: grateful | |
|
| guilty | The main emotion of this example dialogue is: guilty | |
|
| hopeful | The main emotion of this example dialogue is: hopeful | |
|
| impressed | The main emotion of this example dialogue is: impressed | |
|
| jealous | The main emotion of this example dialogue is: jealous | |
|
| joyful | The main emotion of this example dialogue is: joyful | |
|
| lonely | The main emotion of this example dialogue is: lonely | |
|
| nostalgic | The main emotion of this example dialogue is: nostalgic | |
|
| prepared | The main emotion of this example dialogue is: prepared | |
|
| proud | The main emotion of this example dialogue is: proud | |
|
| sad | The main emotion of this example dialogue is: sad | |
|
| sentimental | The main emotion of this example dialogue is: sentimental | |
|
| surprised | The main emotion of this example dialogue is: surprised | |
|
| terrified | The main emotion of this example dialogue is: terrified | |
|
| trusting | The main emotion of this example dialogue is: trusting | |
|
#### agnews |
|
| label | hypothesis | |
|
|:---------|:-------------------------------------------------------| |
|
| Business | This example news text is about business news | |
|
| Sci/Tech | This example news text is about science and technology | |
|
| Sports | This example news text is about sports | |
|
| World | This example news text is about world news | |
|
#### yahootopics |
|
| label | hypothesis | |
|
|:-----------------------|:---------------------------------------------------------------------------------------------------| |
|
| Business & Finance | This example question from the Yahoo Q&A forum is categorized in the topic: Business & Finance | |
|
| Computers & Internet | This example question from the Yahoo Q&A forum is categorized in the topic: Computers & Internet | |
|
| Education & Reference | This example question from the Yahoo Q&A forum is categorized in the topic: Education & Reference | |
|
| Entertainment & Music | This example question from the Yahoo Q&A forum is categorized in the topic: Entertainment & Music | |
|
| Family & Relationships | This example question from the Yahoo Q&A forum is categorized in the topic: Family & Relationships | |
|
| Health | This example question from the Yahoo Q&A forum is categorized in the topic: Health | |
|
| Politics & Government | This example question from the Yahoo Q&A forum is categorized in the topic: Politics & Government | |
|
| Science & Mathematics | This example question from the Yahoo Q&A forum is categorized in the topic: Science & Mathematics | |
|
| Society & Culture | This example question from the Yahoo Q&A forum is categorized in the topic: Society & Culture | |
|
| Sports | This example question from the Yahoo Q&A forum is categorized in the topic: Sports | |
|
#### massive |
|
| label | hypothesis | |
|
|:-------------------------|:------------------------------------------------------------------------------------------| |
|
| alarm_query | The example utterance is a query about alarms. | |
|
| alarm_remove | The intent of this example utterance is to remove an alarm. | |
|
| alarm_set | The intent of the example utterance is to set an alarm. | |
|
| audio_volume_down | The intent of the example utterance is to lower the volume. | |
|
| audio_volume_mute | The intent of this example utterance is to mute the volume. | |
|
| audio_volume_other | The example utterance is related to audio volume. | |
|
| audio_volume_up | The intent of this example utterance is turning the audio volume up. | |
|
| calendar_query | The example utterance is a query about a calendar. | |
|
| calendar_remove | The intent of the example utterance is to remove something from a calendar. | |
|
| calendar_set | The intent of this example utterance is to set something in a calendar. | |
|
| cooking_query | The example utterance is a query about cooking. | |
|
| cooking_recipe | This example utterance is about cooking recipies. | |
|
| datetime_convert | The example utterance is related to date time changes or conversion. | |
|
| datetime_query | The intent of this example utterance is a datetime query. | |
|
| email_addcontact | The intent of this example utterance is adding an email address to contacts. | |
|
| email_query | The example utterance is a query about emails. | |
|
| email_querycontact | The intent of this example utterance is to query contact details. | |
|
| email_sendemail | The intent of the example utterance is to send an email. | |
|
| general_greet | This example utterance is a general greet. | |
|
| general_joke | The intent of the example utterance is to hear a joke. | |
|
| general_quirky | nan | |
|
| iot_cleaning | The intent of the example utterance is for an IoT device to start cleaning. | |
|
| iot_coffee | The intent of this example utterance is for an IoT device to make coffee. | |
|
| iot_hue_lightchange | The intent of this example utterance is changing the light. | |
|
| iot_hue_lightdim | The intent of the example utterance is to dim the lights. | |
|
| iot_hue_lightoff | The example utterance is related to turning the lights off. | |
|
| iot_hue_lighton | The example utterance is related to turning the lights on. | |
|
| iot_hue_lightup | The intent of this example utterance is to brighten lights. | |
|
| iot_wemo_off | The intent of this example utterance is turning an IoT device off. | |
|
| iot_wemo_on | The intent of the example utterance is to turn an IoT device on. | |
|
| lists_createoradd | The example utterance is related to creating or adding to lists. | |
|
| lists_query | The example utterance is a query about a list. | |
|
| lists_remove | The intent of this example utterance is to remove a list or remove something from a list. | |
|
| music_dislikeness | The intent of this example utterance is signalling music dislike. | |
|
| music_likeness | The example utterance is related to liking music. | |
|
| music_query | The example utterance is a query about music. | |
|
| music_settings | The intent of the example utterance is to change music settings. | |
|
| news_query | The example utterance is a query about the news. | |
|
| play_audiobook | The example utterance is related to playing audiobooks. | |
|
| play_game | The intent of this example utterance is to start playing a game. | |
|
| play_music | The intent of this example utterance is for an IoT device to play music. | |
|
| play_podcasts | The example utterance is related to playing podcasts. | |
|
| play_radio | The intent of the example utterance is to play something on the radio. | |
|
| qa_currency | This example utteranceis about currencies. | |
|
| qa_definition | The example utterance is a query about a definition. | |
|
| qa_factoid | The example utterance is a factoid question. | |
|
| qa_maths | The example utterance is a question about maths. | |
|
| qa_stock | This example utterance is about stocks. | |
|
| recommendation_events | This example utterance is about event recommendations. | |
|
| recommendation_locations | The intent of this example utterance is receiving recommendations for good locations. | |
|
| recommendation_movies | This example utterance is about movie recommendations. | |
|
| social_post | The example utterance is about social media posts. | |
|
| social_query | The example utterance is a query about a social network. | |
|
| takeaway_order | The intent of this example utterance is to order takeaway food. | |
|
| takeaway_query | This example utterance is about takeaway food. | |
|
| transport_query | The example utterance is a query about transport or travels. | |
|
| transport_taxi | The intent of this example utterance is to get a taxi. | |
|
| transport_ticket | This example utterance is about transport tickets. | |
|
| transport_traffic | This example utterance is about transport or traffic. | |
|
| weather_query | This example utterance is a query about the wheather. | |
|
#### banking77 |
|
| label | hypothesis | |
|
|:-------------------------------------------------|:----------------------------------------------------------------------------------------------------------| |
|
| Refund_not_showing_up | This customer example message is about a refund not showing up. | |
|
| activate_my_card | This banking customer example message is about activating a card. | |
|
| age_limit | This banking customer example message is related to age limits. | |
|
| apple_pay_or_google_pay | This banking customer example message is about apple pay or google pay | |
|
| atm_support | This banking customer example message requests ATM support. | |
|
| automatic_top_up | This banking customer example message is about automatic top up. | |
|
| balance_not_updated_after_bank_transfer | This banking customer example message is about a balance not updated after a transfer. | |
|
| balance_not_updated_after_cheque_or_cash_deposit | This banking customer example message is about a balance not updated after a cheque or cash deposit. | |
|
| beneficiary_not_allowed | This banking customer example message is related to a beneficiary not being allowed or a failed transfer. | |
|
| cancel_transfer | This banking customer example message is related to the cancellation of a transfer. | |
|
| card_about_to_expire | This banking customer example message is related to the expiration of a card. | |
|
| card_acceptance | This banking customer example message is related to the scope of acceptance of a card. | |
|
| card_arrival | This banking customer example message is about the arrival of a card. | |
|
| card_delivery_estimate | This banking customer example message is about a card delivery estimate or timing. | |
|
| card_linking | nan | |
|
| card_not_working | This banking customer example message is about a card not working. | |
|
| card_payment_fee_charged | This banking customer example message is about a card payment fee. | |
|
| card_payment_not_recognised | This banking customer example message is about a payment the customer does not recognise. | |
|
| card_payment_wrong_exchange_rate | This banking customer example message is about a wrong exchange rate. | |
|
| card_swallowed | This banking customer example message is about a card swallowed by a machine. | |
|
| cash_withdrawal_charge | This banking customer example message is about a cash withdrawal charge. | |
|
| cash_withdrawal_not_recognised | This banking customer example message is about an unrecognised cash withdrawal. | |
|
| change_pin | This banking customer example message is about changing a pin code. | |
|
| compromised_card | This banking customer example message is about a compromised card. | |
|
| contactless_not_working | This banking customer example message is about contactless not working | |
|
| country_support | This banking customer example message is about country-specific support. | |
|
| declined_card_payment | This banking customer example message is about a declined card payment. | |
|
| declined_cash_withdrawal | This banking customer example message is about a declined cash withdrawal. | |
|
| declined_transfer | This banking customer example message is about a declined transfer. | |
|
| direct_debit_payment_not_recognised | This banking customer example message is about an unrecognised direct debit payment. | |
|
| disposable_card_limits | This banking customer example message is about the limits of disposable cards. | |
|
| edit_personal_details | This banking customer example message is about editing personal details. | |
|
| exchange_charge | This banking customer example message is about exchange rate charges. | |
|
| exchange_rate | This banking customer example message is about exchange rates. | |
|
| exchange_via_app | nan | |
|
| extra_charge_on_statement | This banking customer example message is about an extra charge. | |
|
| failed_transfer | This banking customer example message is about a failed transfer. | |
|
| fiat_currency_support | This banking customer example message is about fiat currency support | |
|
| get_disposable_virtual_card | This banking customer example message is about getting a disposable virtual card. | |
|
| get_physical_card | nan | |
|
| getting_spare_card | This banking customer example message is about getting a spare card. | |
|
| getting_virtual_card | This banking customer example message is about getting a virtual card. | |
|
| lost_or_stolen_card | This banking customer example message is about a lost or stolen card. | |
|
| lost_or_stolen_phone | This banking customer example message is about a lost or stolen phone. | |
|
| order_physical_card | This banking customer example message is about ordering a card. | |
|
| passcode_forgotten | This banking customer example message is about a forgotten passcode. | |
|
| pending_card_payment | This banking customer example message is about a pending card payment. | |
|
| pending_cash_withdrawal | This banking customer example message is about a pending cash withdrawal. | |
|
| pending_top_up | This banking customer example message is about a pending top up. | |
|
| pending_transfer | This banking customer example message is about a pending transfer. | |
|
| pin_blocked | This banking customer example message is about a blocked pin. | |
|
| receiving_money | This banking customer example message is about receiving money. | |
|
| request_refund | This banking customer example message is about a refund request. | |
|
| reverted_card_payment? | This banking customer example message is about reverting a card payment. | |
|
| supported_cards_and_currencies | nan | |
|
| terminate_account | This banking customer example message is about terminating an account. | |
|
| top_up_by_bank_transfer_charge | nan | |
|
| top_up_by_card_charge | This banking customer example message is about the charge for topping up by card. | |
|
| top_up_by_cash_or_cheque | This banking customer example message is about topping up by cash or cheque. | |
|
| top_up_failed | This banking customer example message is about top up issues or failures. | |
|
| top_up_limits | This banking customer example message is about top up limitations. | |
|
| top_up_reverted | This banking customer example message is about issues with topping up. | |
|
| topping_up_by_card | This banking customer example message is about topping up by card. | |
|
| transaction_charged_twice | This banking customer example message is about a transaction charged twice. | |
|
| transfer_fee_charged | This banking customer example message is about an issue with a transfer fee charge. | |
|
| transfer_into_account | This banking customer example message is about transfers into the customer's own account. | |
|
| transfer_not_received_by_recipient | This banking customer example message is about a transfer that has not arrived yet. | |
|
| transfer_timing | This banking customer example message is about transfer timing. | |
|
| unable_to_verify_identity | This banking customer example message is about an issue with identity verification. | |
|
| verify_my_identity | This banking customer example message is about identity verification. | |
|
| verify_source_of_funds | This banking customer example message is about the source of funds. | |
|
| verify_top_up | This banking customer example message is about verification and top ups | |
|
| virtual_card_not_working | This banking customer example message is about a virtual card not working | |
|
| visa_or_mastercard | This banking customer example message is about types of bank cards. | |
|
| why_verify_identity | This banking customer example message questions why identity verification is necessary. | |
|
| wrong_amount_of_cash_received | This banking customer example message is about a wrong amount of cash received. | |
|
| wrong_exchange_rate_for_cash_withdrawal | This banking customer example message is about a wrong exchange rate for a cash withdrawal. | |
|
#### trueteacher |
|
| label | hypothesis | |
|
|:-----------------------|:---------------------------------------------------------------------| |
|
| factually_consistent | The example summary is factually consistent with the full article. | |
|
| factually_inconsistent | The example summary is factually inconsistent with the full article. | |
|
#### capsotu |
|
| label | hypothesis | |
|
|:----------------------|:----------------------------------------------------------------------------------------------------------| |
|
| Agriculture | This example text from a US presidential speech is about agriculture | |
|
| Civil Rights | This example text from a US presidential speech is about civil rights or minorities or civil liberties | |
|
| Culture | This example text from a US presidential speech is about cultural policy | |
|
| Defense | This example text from a US presidential speech is about defense or military | |
|
| Domestic Commerce | This example text from a US presidential speech is about banking or finance or commerce | |
|
| Education | This example text from a US presidential speech is about education | |
|
| Energy | This example text from a US presidential speech is about energy or electricity or fossil fuels | |
|
| Environment | This example text from a US presidential speech is about the environment or water or waste or pollution | |
|
| Foreign Trade | This example text from a US presidential speech is about foreign trade | |
|
| Government Operations | This example text from a US presidential speech is about government operations or administration | |
|
| Health | This example text from a US presidential speech is about health | |
|
| Housing | This example text from a US presidential speech is about community development or housing issues | |
|
| Immigration | This example text from a US presidential speech is about migration | |
|
| International Affairs | This example text from a US presidential speech is about international affairs or foreign aid | |
|
| Labor | This example text from a US presidential speech is about employment or labour | |
|
| Law and Crime | This example text from a US presidential speech is about law, crime or family issues | |
|
| Macroeconomics | This example text from a US presidential speech is about macroeconomics | |
|
| Public Lands | This example text from a US presidential speech is about public lands or water management | |
|
| Social Welfare | This example text from a US presidential speech is about social welfare | |
|
| Technology | This example text from a US presidential speech is about space or science or technology or communications | |
|
| Transportation | This example text from a US presidential speech is about transportation | |
|
#### manifesto |
|
| label | hypothesis | |
|
|:-------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| Agriculture and Farmers: Positive | This example text from a political party manifesto is positive towards policies for agriculture and farmers | |
|
| Anti-Growth Economy: Positive | This example text from a political party manifesto is in favour of anti-growth politics | |
|
| Anti-Imperialism | This example text from a political party manifesto is anti-imperialistic, for example against controlling other countries and for greater self-government of colonies | |
|
| Centralisation | This example text from a political party manifesto is in favour of political centralisation | |
|
| Civic Mindedness: Positive | This example text from a political party manifesto is positive towards national solidarity, civil society or appeals for public spiritedness or against anti-social attitudes | |
|
| Constitutionalism: Negative | This example text from a political party manifesto is positive towards constitutionalism | |
|
| Constitutionalism: Positive | This example text from a political party manifesto is positive towards constitutionalism and the status quo of the constitution | |
|
| Controlled Economy | This example text from a political party manifesto is supportive of direct government control of the economy, e.g. price control or minimum wages | |
|
| Corporatism/Mixed Economy | This example text from a political party manifesto is positive towards cooperation of government, employers, and trade unions simultaneously | |
|
| Culture: Positive | This example text from a political party manifesto is in favour of cultural policies or leisure facilities, for example museus, libraries or public sport clubs | |
|
| Decentralization | This example text from a political party manifesto is for decentralisation or federalism | |
|
| Democracy | This example text from a political party manifesto favourably mentions democracy or democratic procedures or institutions | |
|
| Economic Goals | This example text from a political party manifesto is a broad/general statement on economic goals without specifics | |
|
| Economic Growth: Positive | This example text from a political party manifesto is supportive of economic growth, for example facilitation of more production or government aid for growth | |
|
| Economic Orthodoxy | This example text from a political party manifesto is for economic orthodoxy, for example reduction of budget deficits, thrift or a strong currency | |
|
| Economic Planning | This example text from a political party manifesto is positive towards government economic planning, e.g. policy plans or strategies | |
|
| Education Expansion | This example text from a political party manifesto is about the need to expand/improve policy on education | |
|
| Education Limitation | This example text from a political party manifesto is sceptical towards state expenditure on education, for example in favour of study fees or private schools | |
|
| Environmental Protection | This example text from a political party manifesto is in favour of environmental protection, e.g. fighting climate change or 'green' policies or preservation of natural resources or animal rights | |
|
| Equality: Positive | This example text from a political party manifesto is positive towards equality or social justice, e.g. protection of underprivileged groups or fair distribution of resources | |
|
| European Community/Union: Negative | This example text from a political party manifesto negatively mentions the EU or European Community | |
|
| European Community/Union: Positive | This example text from a political party manifesto is positive towards the EU or European Community, for example EU expansion and integration | |
|
| Foreign Special Relationships: Negative | This example text from a political party manifesto is negative towards particular countries | |
|
| Foreign Special Relationships: Positive | This example text from a political party manifesto is positive towards particular countries | |
|
| Free Market Economy | This example text from a political party manifesto is in favour of a free market economy and capitalism | |
|
| Freedom and Human Rights | This example text from a political party manifesto is in favour of freedom and human rights, for example freedom of speech, assembly or against state coercion or for individualism | |
|
| Governmental and Administrative Efficiency | This example text from a political party manifesto is in favour of efficiency in government/administration, for example by restructuring civil service or improving bureaucracy | |
|
| Incentives: Positive | This example text from a political party manifesto is favourable towards supply side economic policies supporting businesses, for example for incentives like subsidies or tax breaks | |
|
| Internationalism: Negative | This example text from a political party manifesto is sceptical of internationalism, for example negative towards international cooperation, in favour of national sovereignty and unilaterialism | |
|
| Internationalism: Positive | This example text from a political party manifesto is in favour of international cooperation with other countries, for example mentions the need for aid to developing countries, or global governance | |
|
| Keynesian Demand Management | This example text from a political party manifesto is for keynesian demand management and demand side economic policies | |
|
| Labour Groups: Negative | This example text from a political party manifesto is negative towards labour groups and unions | |
|
| Labour Groups: Positive | This example text from a political party manifesto is positive towards labour groups, for example for good working conditions, fair wages or unions | |
|
| Law and Order: Positive | This example text from a political party manifesto is positive towards law and order and strict law enforcement | |
|
| Market Regulation | This example text from a political party manifesto is supports market regulation for a fair and open market, for example for consumer protection or for increased competition or for social market economy | |
|
| Marxist Analysis | This example text from a political party manifesto is positive towards Marxist-Leninist ideas or uses specific Marxist terminology | |
|
| Middle Class and Professional Groups | This example text from a political party manifesto favourably references the middle class, e.g. white colar groups or the service sector | |
|
| Military: Negative | This example text from a political party manifesto is negative towards the military, for example for decreasing military spending or disarmament | |
|
| Military: Positive | This example text from a political party manifesto is positive towards the military, for example for military spending or rearmament or military treaty obligations | |
|
| Multiculturalism: Negative | This example text from a political party manifesto is sceptical towards multiculturalism, or for cultural integration or appeals to cultural homogeneity in society | |
|
| Multiculturalism: Positive | This example text from a political party manifesto favourably mentions cultural diversity, for example for freedom of religion or linguistic heritages | |
|
| National Way of Life: Negative | This example text from a political party manifesto unfavourably mentions a country's nation and history, for example sceptical towards patriotism or national pride | |
|
| National Way of Life: Positive | This example text from a political party manifesto is positive towards the national way of life and history, for example pride of citizenship or appeals to patriotism | |
|
| Nationalisation | This example text from a political party manifesto is positive towards government ownership of industries or land or for economic nationalisation | |
|
| Non-economic Demographic Groups | This example text from a political party manifesto favourably mentions non-economic demographic groups like women, students or specific age groups | |
|
| Peace | This example text from a political party manifesto is positive towards peace and peaceful means of solving crises, for example in favour of negotiations and ending wars | |
|
| Political Authority | This example text from a political party manifesto mentions the speaker's competence to govern or other party's lack of such competence, or favourably mentions a strong/stable government | |
|
| Political Corruption | This example text from a political party manifesto is negative towards political corruption or abuse of political/bureaucratic power | |
|
| Protectionism: Negative | This example text from a political party manifesto is negative towards protectionism, in favour of free trade | |
|
| Protectionism: Positive | This example text from a political party manifesto is in favour of protectionism, for example tariffs, export subsidies | |
|
| Technology and Infrastructure: Positive | This example text from a political party manifesto is about technology and infrastructure, e.g. the importance of modernisation of industry, or supportive of public spending on infrastructure/tech | |
|
| Traditional Morality: Negative | This example text from a political party manifesto is negative towards traditional morality, for example against religious moral values, for divorce or abortion, for modern families or separation of church and state | |
|
| Traditional Morality: Positive | This example text from a political party manifesto is favourable towards traditional or religious values, for example for censorship of immoral behavour, for traditional family values or religious institutions | |
|
| Underprivileged Minority Groups | This example text from a political party manifesto favourably mentions underprivileged minorities, for example handicapped, homosexuals or immigrants | |
|
| Welfare State Expansion | This example text from a political party manifesto is positive towards the welfare state, e.g. health care, pensions or social housing | |
|
| Welfare State Limitation | This example text from a political party manifesto is for limiting the welfare state, for example public funding for social services or social security, e.g. private care before state care | |
