J0nasW/paperswithcode huggingface dataset:
Task: Sentiment Analysis
Area: natural-language-processing

**Sentiment Analysis** is the task of classifying the polarity of a given text. For instance, a text-based tweet can be categorized into either "positive", "negative", or "neutral". Given the text and accompanying labels, a model can be trained to predict the correct sentiment. 

**Sentiment Analysis** techniques can be categorized into machine learning approaches, lexicon-based approaches, and even hybrid methods. Some subcategories of research in sentiment analysis include: multimodal sentiment analysis, aspect-based sentiment analysis, fine-grained opinion analysis, language specific sentiment analysis.

More recently, deep learning techniques, such as RoBERTa and T5, are used to train high-performing sentiment classifiers that are evaluated using metrics like F1, recall, and precision. To evaluate sentiment analysis systems, benchmark datasets like SST, GLUE, and IMDB movie reviews are used.

Further readings:

- [Sentiment Analysis Based on Deep Learning: A Comparative Study](https://paperswithcode.com/paper/sentiment-analysis-based-on-deep-learning-a)
Task: Hyper-Relational Extraction
Area: natural-language-processing

HyperRED is a dataset for the new task of hyper-relational extraction, which extracts relation triplets together with qualifier information such as time, quantity or location. For example, the relation triplet (Leonard Parker, Educated At, Harvard University) can be factually enriched by including the qualifier (End Time, 1967).
Task: Word Sense Disambiguation
Area: natural-language-processing

The task of Word Sense Disambiguation (WSD) consists of associating words in context with their most suitable entry in a pre-defined sense inventory. The de-facto sense inventory for English in WSD is [WordNet](https://wordnet.princeton.edu).
For example, given the word “mouse” and the following sentence:

“A mouse consists of an object held in one's hand, with one or more buttons.” 

we would assign “mouse”  with its electronic device sense ([the 4th sense in the WordNet sense inventory](http://wordnetweb.princeton.edu/perl/webwn?c=8&sub=Change&o2=&o0=1&o8=1&o1=1&o7=&o5=&o9=&o6=&o3=&o4=&i=-1&h=000000&s=mouse)).
Task: Text Augmentation
Area: natural-language-processing

You can read these blog posts to get an overview of the approaches.  

- [**A Visual Survey of Data Augmentation in NLP**](https://amitness.com/2020/05/data-augmentation-for-nlp/)
Task: Temporal Relation Extraction
Area: natural-language-processing

Temporal relation extraction systems aim to identify and classify the temporal relation between a pair of entities provided in a text. For instance, in the sentence "Bob sent a message to Alice while she was leaving her birthday party." one can infer that the actions "sent" and "leaving" entails a temporal relation that can be described as "simultaneous".
Task: Toponym Resolution
Area: natural-language-processing

The goal is to find a mapping from a toponym (a location mention) in the text to a spatial footprint.
Task: Word Alignment
Area: natural-language-processing

**Word Alignment** is the task of finding the correspondence between source and target words in a pair of sentences that are translations of each other.


<span class="description-source">Source: [Neural Network-based Word Alignment through Score Aggregation ](https://arxiv.org/abs/1606.09560)</span>
Task: Dependency Grammar Induction
Area: natural-language-processing

Also known as "unsupervised dependency parsing"
Task: Relational Reasoning
Area: natural-language-processing

The goal of **Relational Reasoning** is to figure out the relationships among different entities, such as image pixels, words or sentences, human skeletons or interactive moving agents.


<span class="description-source">Source: [Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network ](https://arxiv.org/abs/2002.06241)</span>
Task: Event Extraction
Area: natural-language-processing

Determine the extent of the events in a text.

Other names: Event Tagging; Event Identification
Task: Commonsense Causal Reasoning
Area: natural-language-processing

"Commonsense Causal Reasoning is the process of capturing and understanding the causal dependencies amongst events and actions." Luo, Zhiyi, et al. "Commonsense causal reasoning between short texts." Fifteenth International Conference on the Principles of Knowledge Representation and Reasoning. 2016.
Task: Cross-Lingual Entity Linking
Area: natural-language-processing

Cross-lingual entity linking is the task of using data and models available for one language for which ample such resources are available (e.g., English) to solve entity linking tasks (i.e., assigning a unique identity to entities in a text) in another, commonly low-resource, language.

Image Source: [Towards Zero-resource Cross-lingual Entity Linking](https://www.aclweb.org/anthology/D19-6127.pdf)
Task: Vietnamese Aspect-Based Sentiment Analysis
Area: natural-language-processing

UIT-ViSFD: A Vietnamese Smartphone Feedback Dataset for Aspect-Based Sentiment Analysis 


In this paper, we present a process of building a social listening system based on aspect-based sentiment analysis in Vietnamese from creating a dataset to building a real application. Firstly, we create UIT-ViSFD, a Vietnamese Smartphone Feedback Dataset as a new benchmark corpus built based on strict annotation schemes for evaluating aspect-based sentiment analysis, consisting of 11,122 human-annotated comments for mobile e-commerce, which is freely available for research purposes. We also present a proposed approach based on the Bi-LSTM architecture with the fastText word embeddings for the Vietnamese aspect-based sentiment task. Our experiments show that our approach achieves the best performances with the F1-score of 84.48% for the aspect task and 63.06% for the sentiment task, which performs several conventional machine learning and deep learning systems. Last but not least, we build SA2SL, a social listening system based on the best performance model on our dataset, which will inspire more social listening systems in the future. Dataset download: https://www.facebook.com/ViDataset

Paper: Phan, Luong Luc, Phuc Huynh Pham, Kim Thi-Thanh Nguyen, Tham Thi Nguyen, Sieu Khai Huynh, Luan Thanh Nguyen, Tin Van Huynh, and Kiet Van Nguyen. "SA2SL: From Aspect-Based Sentiment Analysis to Social Listening System for Business Intelligence." arXiv preprint arXiv:2105.15079 (2021).
Task: Dialogue Generation
Area: natural-language-processing

Dialogue generation is the task of "understanding" natural language inputs - within natural language processing in order to produce output. The systems are usually intended for conversing with humans, for instance back and forth dialogue with a conversation agent like a chatbot. Some example benchmarks for this task (see others such as Natural Language Understanding) include FusedChat and Ubuntu DIalogue Corpus (UDC). Models can be evaluated via metrics such as BLEU, ROUGE, and METEOR albeit with challenges in terms of weak correlation with human judgement, that may be addressed by new ones like UnSupervised and Reference-free (USR) and Metric for automatic Unreferenced dialog evaluation (MaUde).
Task: Constituency Parsing
Area: natural-language-processing

Constituency parsing aims to extract a constituency-based parse tree from a sentence that
represents its syntactic structure according to a [phrase structure grammar](https://en.wikipedia.org/wiki/Phrase_structure_grammar).

Example:

                 Sentence (S)
                     |
       +-------------+------------+
       |                          |
     Noun (N)                Verb Phrase (VP)
       |                          |
     John                 +-------+--------+
                          |                |
                        Verb (V)         Noun (N)
                          |                |
                        sees              Bill

[Recent approaches](https://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf)
convert the parse tree into a sequence following a depth-first traversal in order to
be able to apply sequence-to-sequence models to it. The linearized version of the
above parse tree looks as follows: (S (N) (VP V N)).
Task: Sign Language Production
Area: natural-language-processing

Sign Language Production (SLP) is the automatically translation from spoken language sentences into sign language sequences. Whilst Sign language Translation translates from sign to text, SLP is the opposite task from text to sign.
Task: Machine Translation
Area: natural-language-processing

**Machine translation** is the task of translating a sentence in a source language to a different target language.

Approaches for machine translation can range from rule-based to statistical to neural-based. More recently, encoder-decoder attention-based architectures like BERT have attained major improvements in machine translation. 

One of the most popular datasets used to benchmark machine translation systems is the WMT family of datasets. Some of the most commonly used evaluation metrics for machine translation systems include BLEU, METEOR, NIST, and others. 

<span style="color:grey; opacity: 0.6">( Image credit: [Google seq2seq](https://github.com/google/seq2seq) )</span>
Task: Goal-Oriented Dialog
Area: natural-language-processing

Achieving a pre-defined goal through a dialog.
Task: Aspect-Sentiment-Opinion Triplet Extraction
Area: natural-language-processing

Aspect-Sentiment-Opinion Triplet Extraction (ASOTE) extracts aspect term, sentiment and opinion term triplets from sentences. In the triplet extracted by ASOTE the sentiment is the sentiment of the aspect term and opinion term pair.
Task: Web Page Tagging
Area: natural-language-processing

Assigning appropriate tags to a web page.
Task: Lemmatization
Area: natural-language-processing

**Lemmatization** is a process of determining a base or dictionary form (lemma) for a given surface form. Especially for languages with rich morphology it is important to be able to normalize words into their base forms to better support for example search engines and linguistic studies. Main difficulties in Lemmatization arise from encountering previously unseen words during inference time as well as disambiguating ambiguous surface forms which can be inflected variants of several different base forms depending on the context.


<span class="description-source">Source: [Universal Lemmatizer: A Sequence to Sequence Model for Lemmatizing Universal Dependencies Treebanks ](https://arxiv.org/abs/1902.00972)</span>
Task: Topic coverage
Area: natural-language-processing

A prevalent use case of topic models is that of topic discovery.
However, most of the topic model evaluation methods rely on abstract metrics such as perplexity or topic coherence. The topic coverage approach is to measure the models' performance by matching model-generated topics to a fixed set of reference topics - topics discovered by humans and represented in a machine-readable format. This way, the models are evaluated in the context of their use, by essentially simulating topic modeling in a fixed setting defined by a text collection and a set of reference topics.
Reference topics represent a ground truth that can be used to evaluate both topic models and other measures of model performance. This coverage approach enables large-scale automatic evaluation of existing and future topic models.
Task: Morphological Analysis
Area: natural-language-processing

**Morphological Analysis** is a central task in language processing that can take a word as input and detect the various morphological entities in the word and provide a morphological representation of it.


<span class="description-source">Source: [Towards Finite-State Morphology of Kurdish ](https://arxiv.org/abs/2005.10652)</span>
Task: Phrase Ranking
Area: natural-language-processing

This task aims to evaluate the “global” rank list of phrases that a method finds from the input corpus.
Task: Zero-Shot Out-of-Domain Detection
Area: natural-language-processing

Tasks for simultaneously learning few-shot In-Domain text classification and zero-shot Out-of-Domain detection on sentiment classification and intent classification.
Task: Romanian Text Diacritization
Area: natural-language-processing

Addition of diacritics for undiacritized Romanian Wikipedia texts.
Task: Table-to-Text Generation
Area: natural-language-processing

**Table-to-Text Generation** is to generate a description from the structured table.


<span class="description-source">Source: [Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation ](https://arxiv.org/abs/1908.03067)</span>
Task: Multiple Choice Question Answering (MCQA)
Area: natural-language-processing

A multiple-choice question (MCQ) is composed of two parts: a stem that identifies the question or problem, and a set of alternatives or possible answers that contain a key that is the best answer to the question, and a number of distractors that are plausible but incorrect answers to the question.

In a k-way MCQA task, a model is provided with a question q, a set of candidate options O = {O1, . . . , Ok}, and a supporting context for each option C = {C1, . . . , Ck}. The model needs to predict the correct answer option that is best supported by the given contexts.
Task: Memex Question Answering
Area: natural-language-processing

Question answering with real-world multi-modal personal collections, e.g., photo albums with visual, text, time and location information.
Task: Cross-Domain Text Classification
Area: natural-language-processing

Learning an accurate model for the new unlabeled target domain given labeled data from multiple source domains where all domains have (possibly) different label sets. (Source: https://www.aclweb.org/anthology/P16-1155.pdf)
Task: Clinical Language Translation
Area: natural-language-processing

Translating clinical texts to layperson-understandable language.
Task: Spelling Correction
Area: natural-language-processing

Spelling correction is the task of detecting and correcting spelling mistakes.
Task: Meme Classification
Area: natural-language-processing

Meme classification refers to the task of classifying internet memes.
Task: Turkish Text Diacritization
Area: natural-language-processing

Addition of diacritics for undiacritized Turkish Wikipedia texts.
Task: Text Matching
Area: natural-language-processing

Matching a target text to a source text based on their meaning.
Task: Entity Linking
Area: natural-language-processing

Assigning a unique identity to entities (such as famous individuals, locations, or companies) mentioned in text (Source: Wikipedia).
Task: Document Classification
Area: natural-language-processing

**Document Classification** is a procedure of assigning one or more labels to a document from a predetermined set of labels.


<span class="description-source">Source: [Long-length Legal Document Classification ](https://arxiv.org/abs/1912.06905)</span>
Task: Email Thread Summarization
Area: natural-language-processing

Image credit: [EmailSum: Abstractive Email Thread Summarization](https://paperswithcode.com/paper/emailsum-abstractive-email-thread)
Task: Speech-to-Text Translation
Area: natural-language-processing

Translate audio signals of speech in one language into text in a foreign language, either in an end-to-end or cascade manner.
Task: Automatic Post-Editing
Area: natural-language-processing

Automatic post-editing (APE) is used to correct errors in the translation made by the machine translation systems.
Task: Punctuation Restoration
Area: natural-language-processing

Punctuation Restoration
Task: Logical Reasoning Question Answering
Area: natural-language-processing

Introduced by ReClor (ICLR 2020), logical reasoning is to evaluate the logical reasoning ability of models for question answering.
Task: Cross-Lingual Document Classification
Area: natural-language-processing

Cross-lingual document classification refers to the task of using data and models available for one language for which ample such resources are available (e.g., English) to solve classification tasks in another, commonly low-resource, language.
Task: Language Modelling
Area: natural-language-processing

**Language Modeling** is the task of predicting the next word or character in a document. This technique can be used to train language models that can further be applied to a wide range of natural language tasks like text generation, text classification, and question answering. 

The common types of language modeling techniques involve:

- N-gram Language Models
- Neural Langauge Models

A model's language modeling capability is measured using cross-entropy and perplexity. Some datasets to evaluate language modeling are WikiText-103, One Billion Word, Text8, C4, among others. 

One of the most recent popular benchmarks to evaluate language modeling capabilities is called SuperGLUE.

Some popular and notable state-of-the-art language models, include:

- [GPT-3](/method/gpt-3)
- Megatron-LM
- [BERT](/method/bert)

Check below for all state-of-the-art models. 

Here are some additional readings to go deeper on the task:

- [Language Modeling](https://lena-voita.github.io/nlp_course/language_modeling.html) - Lena Voita

<span style="color:grey; opacity: 0.6">( Image credit: [Exploring the Limits of Language Modeling](https://arxiv.org/pdf/1602.02410v2.pdf) )</span>
Task: Prosody Prediction
Area: natural-language-processing

Predicting prosodic prominence from text. This is a 2-way classification task, assigning each word in a sentence a label 1 (prominent) or 0 (non-prominent).

<span style="color:grey; opacity: 0.6">( Image credit: [Helsinki Prosody Corpus](https://github.com/Helsinki-NLP/prosody) )</span>
Task: Formality Style Transfer
Area: natural-language-processing

Formality Style Transfer
Task: Response Generation
Area: natural-language-processing

A task where an agent should play the $DE$ role and generate a text to respond to a $P$ message.
Task: incongruity detection
Area: natural-language-processing

Incongruity detection is the task of identifying statements in a text that are inconsistent with each other.
Task: Blackout Poetry Generation
Area: natural-language-processing

Blackout poetry is a form of poetry in which words in a passage are masked, except for a few which when combined together in order to convey some meaning.
Task: Information Extraction
Area: natural-language-processing

Information extraction is the task of automatically extracting structured information from unstructured and / or semi-structured machine-readable documents and other electronically represented sources (Source: Wikipedia).
Task: Sarcasm Detection
Area: natural-language-processing

The goal of **Sarcasm Detection** is to determine whether a sentence is sarcastic or non-sarcastic. Sarcasm is a type of phenomenon with specific perlocutionary effects on the hearer, such as to break their pattern of expectation. Consequently, correct understanding of sarcasm often requires a deep understanding of multiple sources of information, including the utterance, the conversational context, and, frequently some real world facts.


<span class="description-source">Source: [Attentional Multi-Reading Sarcasm Detection ](https://arxiv.org/abs/1809.03051)</span>
Task: Open Information Extraction
Area: natural-language-processing

In natural language processing, open information extraction is the task of generating a structured, machine-readable representation of the information in text, usually in the form of triples or n-ary propositions (Source: Wikipedia).
Task: Cloze Test
Area: natural-language-processing

The cloze task refers to infilling individual words.
Task: Math Information Retrieval
Area: natural-language-processing

Information Retrieval on Math Contents
Task: Claim Extraction with Stance Classification (CESC)
Area: natural-language-processing

Since claims stand at a clear position towards a given topic, the sentences with clear stances should have a higher possibility to be the claims. Hence, identifying the stances of the claims is supposed to benefit the claim extraction task. By combining the claim extraction and stance classification subtasks, we define this integrated task as: given a specific topic and relevant articles, extract the claims from the articles and also identify the stance of the claims towards the topic.
Task: Offline Handwritten Chinese Character Recognition
Area: natural-language-processing

Handwritten Chinese characters recognition is the task of detecting and interpreting the components of Chinese characters (i.e. radicals and two-dimensional structures).
Task: Sentence Compression
Area: natural-language-processing

**Sentence Compression** is the task of reducing the length of text by removing non-essential content while preserving important facts and grammaticality.
Task: Medical question pair similarity computation
Area: natural-language-processing

Predicting whether to questions on medical topics have the same meaning.
Task: ValNov
Area: natural-language-processing

Given a textual premise and conclusion candidate, the Argument-Validity-and-Novelty-Prediction-Shared-Task ValNov consists in predicting two aspects of a conclusion: its validity and novelty.

Validity is defined as the degree to which the conclusion is justified with respect to the given premise. A conclusion is considered to be valid if it is supported by inferences that link the premise to the conclusion, based on logical principles or commonsense or world knowledge, which may be defeasible. A conclusion will be trivially considered valid if it repeats or summarizes the premise – in which case it can hardly be considered as novel.

Novelty defines the degree to which the conclusion contains content that is new in relation to the premise. As extreme cases, a conclusion candidate that repeats or summarizes the premise or is unrelated to the premise will not be considered novel.
Task: Legal Document Translation
Area: natural-language-processing

Legal document translation is the task of translating legal documents between languages.
Task: Dialogue
Area: natural-language-processing

Dialogue is notoriously hard to evaluate. Past approaches have used human evaluation.
Task: Document-level Relation Extraction
Area: natural-language-processing

Document-level RE aim to identify the relations of various entity pairs expressed across multiple sentences.
Task: Word Sense Induction
Area: natural-language-processing

Word sense induction (WSI) is widely known as the “unsupervised version” of WSD. The problem states as: Given a target word (e.g., “cold”) and a collection of sentences (e.g., “I caught a cold”, “The weather is cold”) that use the word, cluster the sentences according to their different senses/meanings. We do not need to know the sense/meaning of each cluster, but sentences inside a cluster should have used the target words with the same sense.

Description from [NLP Progress](http://nlpprogress.com/english/word_sense_disambiguation.html)
Task: Word Similarity
Area: natural-language-processing

Calculate a numerical score for the semantic similarity between two words.
Task: Long Form Question Answering
Area: natural-language-processing

Long-form question answering is a task requiring elaborate and in-depth answers to open-ended questions.
Task: Dialogue State Tracking
Area: natural-language-processing

Dialogue state tacking consists of determining at each turn of a dialogue the
full representation of what the user wants at that point in the dialogue,
which contains a goal constraint, a set of requested slots, and the user's dialogue act.
Task: Action Parsing
Area: natural-language-processing

Action parsing is the task of, given a video or still image, assigning each frame or image a label describing the action in that frame or image.
Task: Hope Speech Detection
Area: natural-language-processing

Detecting speech associated with positive, uplifting,
promise, potential, support, reassurance, suggestions, or inspiration.