3497hyy/evaluation-tables huggingface dataset:
Task: Sar Image Despeckling
Categories: Computer Vision

Despeckling is the task of suppressing speckle from Synthetic Aperture Radar (SAR) acquisitions.

Image credits: GRD Sentinel-1 SAR image despeckled with [SAR2SAR-GRD](https://arxiv.org/abs/2102.00692)
Task: Image Inpainting
Categories: Computer Vision

Remove the preview tag from template
Task: Image Dehazing
Categories: Computer Vision

( Image credit: [Densely Connected Pyramid Dehazing Network](https://github.com/hezhangsprinter/DCPDN) )
Task: Image Stitching
Categories: Computer Vision

**Image Stitching** is a process of composing multiple images with narrow but overlapping fields of view to create a larger image with a wider field of view.

Source: [Single-Perspective Warps in Natural Image Stitching ](https://arxiv.org/abs/1802.04645)

( Image credit: [Kornia](https://github.com/kornia/kornia) )
Task: Image Steganography
Categories: Computer Vision

**Image Steganography** is the main content of information hiding. The sender conceal a secret message into a cover image, then get the container image called stego, and finish the secret message’s transmission on the public channel by transferring the stego image. Then the receiver part of the transmission can reveal the secret message out. Steganalysis is an attack to the steganography algorithm. The listener on the public channel intercept the image and analyze whether the image contains secret information.

Source: [Invisible Steganography via Generative Adversarial Networks ](https://arxiv.org/abs/1807.08571)
Task: Texture Classification
Categories: Computer Vision

**Texture Classification** is a fundamental issue in computer vision and image processing, playing a significant role in many applications such as medical image analysis, remote sensing, object recognition, document analysis, environment modeling, content-based image retrieval and many more.

Source: [Improving Texture Categorization with Biologically Inspired Filtering ](https://arxiv.org/abs/1312.0072)
Task: Coreference Resolution
Categories: Natural Language Processing

Coreference resolution is the task of clustering mentions in text that refer to the same underlying real world entities.

Example:

```
+-----------+
| |
I voted for Obama because he was most aligned with my values", she said.
| | |
+-------------------------------------------------+------------+
```

"I", "my", and "she" belong to the same cluster and "Obama" and "he" belong to the same cluster.
Task: Entity Resolution
Categories: Natural Language Processing

**Entity resolution** (also known as entity matching, record linkage, or duplicate detection) is the task of finding records that refer to the same real-world entity across different data sources (e.g., data files, books, websites, and databases). (Source: Wikipedia)

Surveys on entity resolution:

- [Christophides et al.: End-to-End Entity Resolution for Big Data: A Survey](https://arxiv.org/pdf/1905.06397.pdf), 2020.

- [Barlaug and Gulla: Neural Networks for Entity Matching: A Survey](https://arxiv.org/pdf/2010.11075.pdf), 2021.

The task of entity resolution is closely related to the task of [entity alignment](https://paperswithcode.com/task/entity-alignment) which focuses on matching entities between knowledge bases. The task of [entity linking](https://paperswithcode.com/task/entity-linking) differs from entity resolution as entity linking focuses on identifying entity mentions in free text.
Task: Image Compression
Categories: Computer Vision

**Image Compression** is an application of data compression for digital images to lower their storage and/or transmission requirements.

Source: [Variable Rate Deep Image Compression With a Conditional Autoencoder ](https://arxiv.org/abs/1909.04802)
Task: Model Compression
Categories: Methodology
Miscellaneous

**Model Compression** is an actively pursued area of research over the last few years with the goal of deploying state-of-the-art deep networks in low-power and resource limited devices without significant drop in accuracy. Parameter pruning, low-rank factorization and weight quantization are some of the proposed methods to compress the size of deep networks.

Source: [KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow ](https://arxiv.org/abs/2004.05319)
Task: Commonsense Causal Reasoning
Categories: Natural Language Processing

"Commonsense Causal Reasoning is the process of capturing and understanding the causal dependencies amongst events and actions." Luo, Zhiyi, et al. "Commonsense causal reasoning between short texts." Fifteenth International Conference on the Principles of Knowledge Representation and Reasoning. 2016.
Task: Optical Flow Estimation
Categories: Computer Vision

**Optical Flow Estimation** is a computer vision task that involves computing the motion of objects in an image or a video sequence. The goal of optical flow estimation is to determine the movement of pixels or features in the image, which can be used for various applications such as object tracking, motion analysis, and video compression.

Approaches for optical flow estimation include correlation-based, block-matching, feature tracking, energy-based, and more recently gradient-based.

Further readings:

- [Optical Flow Estimation](https://www.cs.toronto.edu/~fleet/research/Papers/flowChapter05.pdf)
- [Performance of Optical Flow Techniques](https://www.cs.toronto.edu/~fleet/research/Papers/ijcv-94.pdf)

Definition source: [Devon: Deformable Volume Network for Learning Optical Flow ](https://arxiv.org/abs/1802.07351)

Image credit: [Optical Flow Estimation](https://www.cs.toronto.edu/~fleet/research/Papers/flowChapter05.pdf)
Task: Data Augmentation
Categories: Methodology
Natural Language Processing
Computer Vision

Data augmentation involves techniques used for increasing the amount of data, based on different modifications, to expand the amount of examples in the original dataset. Data augmentation not only helps to grow the dataset but it also increases the diversity of the dataset. When training machine learning models, data augmentation acts as a regularizer and helps to avoid overfitting.

Data augmentation techniques have been found useful in domains like NLP and computer vision. In computer vision, transformations like cropping, flipping, and rotation are used. In NLP, data augmentation techniques can include swapping, deletion, random insertion, among others.

Further readings:

- [A Survey of Data Augmentation Approaches for NLP](https://paperswithcode.com/paper/a-survey-of-data-augmentation-approaches-for)
- [A survey on Image Data Augmentation for Deep Learning](https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0197-0)

( Image credit: [Albumentations](https://github.com/albumentations-team/albumentations) )
Task: Object Counting
Categories: Computer Vision

The goal of **Object Counting** task is to count the number of object instances in a single image or video sequence. It has many real-world applications such as traffic flow monitoring, crowdedness estimation, and product counting.

Source: [Learning to Count Objects with Few Exemplar Annotations ](https://arxiv.org/abs/1905.07898)
Task: Twitter Event Detection
Categories: Natural Language Processing

Detection of worldwide events from categories like Sports, Politics, Entertainment, Science & Technology, etc. by analyzing Twitter Tweets.
Task: Abstract Argumentation
Categories: Natural Language Processing
Reasoning

Identifying argumentative statements from natural language dialogs.
Task: Gait Recognition
Categories: Computer Vision

( Image credit: [GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition](https://github.com/AbnerHqC/GaitSet) )
Task: Pedestrian Attribute Recognition
Categories: Computer Vision

Pedestrian attribution recognition is the task of recognizing pedestrian features - such as whether they are talking on a phone, whether they have a backpack, and so on.

( Image credit: [HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis](https://arxiv.org/pdf/1709.09930v1.pdf) )
Task: Motion Segmentation
Categories: Computer Vision

**Motion Segmentation** is an essential task in many applications in Computer Vision and Robotics, such as surveillance, action recognition and scene understanding. The classic way to state the problem is the following: given a set of feature points that are tracked through a sequence of images, the goal is to cluster those trajectories according to the different motions they belong to. It is assumed that the scene contains multiple objects that are moving rigidly and independently in 3D-space.

Source: [Robust Motion Segmentation from Pairwise Matches ](https://arxiv.org/abs/1905.09043)
Task: Motion Detection
Categories: Computer Vision

**Motion Detection** is a process to detect the presence of any moving entity in an area of interest. Motion Detection is of great importance due to its application in various areas such as surveillance and security, smart homes, and health monitoring.

Source: [Different Approaches for Human Activity Recognition– A Survey ](https://arxiv.org/abs/1906.05074)
Task: Motion Estimation
Categories: Computer Vision

**Motion Estimation** is used to determine the block-wise or pixel-wise motion vectors between two frames.

Source: [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement ](https://arxiv.org/abs/1810.08768)
Task: Computational Phenotyping
Categories: Medical

**Computational Phenotyping** is the process of transforming the noisy, massive Electronic Health Record (EHR) data into meaningful medical concepts that can be used to predict the risk of disease for an individual, or the response to drug therapy.

Source: [Privacy-Preserving Tensor Factorization for Collaborative Health Data Analysis ](https://arxiv.org/abs/1908.09888)
Task: Eeg Decoding
Categories: Medical
Time Series

**EEG Decoding** - extracting useful information directly from EEG data.
Task: Denoising
Categories: Computer Vision

**Denoising** is a task in image processing and computer vision that aims to remove or reduce noise from an image. Noise can be introduced into an image due to various reasons, such as camera sensor limitations, lighting conditions, and compression artifacts. The goal of denoising is to recover the original image, which is considered to be noise-free, from a noisy observation.

( Image credit: [Beyond a Gaussian Denoiser](https://arxiv.org/pdf/1608.03981v1.pdf) )
Task: Clustering
Categories: Methodology

Clustering is the task of grouping unlabeled data point into disjoint subsets. Each data point is labeled with a single class. The number of classes is not known a priori. The grouping criteria is typically based on the similarity of data points to each other.
Task: Multi-Grained Named Entity Recognition
Categories: Natural Language Processing

Multi-Grained Named Entity Recognition aims to detect and recognize entities on multiple granularities, without explicitly assuming non-overlapping or totally nested structures.
Task: 3D Action Recognition
Categories: Natural Language Processing
Computer Vision

Image: [Rahmani et al](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Rahmani_3D_Action_Recognition_CVPR_2016_paper.pdf)
Task: Vector Quantization (k-means problem)
Categories: Miscellaneous

Given a data set $X$ of d-dimensional numeric vectors and a number $k$ find a codebook $C$ of $k$ d-dimensional vectors such that the sum of square distances of each $x \in X$ to the respective nearest $c \in C$ is as small as possible. This is also known as the k-means problem and is known to be NP-hard.
Task: Chunking
Categories: Natural Language Processing

Chunking, also known as shallow parsing, identifies continuous spans of tokens that form syntactic units such as noun phrases or verb phrases.

Example:

| Vinken | , | 61 | years | old |
| --- | ---| --- | --- | --- |
| B-NLP| I-NP | I-NP | I-NP | I-NP |
Task: AMR Parsing
Categories: Natural Language Processing

Each AMR is a single rooted, directed graph. AMRs include PropBank semantic roles, within-sentence coreference, named entities and types, modality, negation, questions, quantities, and so on. [See](https://amr.isi.edu/index.html).
Task: Constituency Parsing
Categories: Natural Language Processing

Constituency parsing aims to extract a constituency-based parse tree from a sentence that
represents its syntactic structure according to a [phrase structure grammar](https://en.wikipedia.org/wiki/Phrase_structure_grammar).

Example:

Sentence (S)
|
+-------------+------------+
| |
Noun (N) Verb Phrase (VP)
| |
John +-------+--------+
| |
Verb (V) Noun (N)
| |
sees Bill

[Recent approaches](https://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf)
convert the parse tree into a sequence following a depth-first traversal in order to
be able to apply sequence-to-sequence models to it. The linearized version of the
above parse tree looks as follows: (S (N) (VP V N)).
Task: Action Parsing
Categories: Natural Language Processing

Action parsing is the task of, given a video or still image, assigning each frame or image a label describing the action in that frame or image.
Task: Automated Theorem Proving
Categories: Miscellaneous

The goal of **Automated Theorem Proving** is to automatically generate a proof, given a conjecture (the target theorem) and a knowledge base of known facts, all expressed in a formal language. Automated Theorem Proving is useful in a wide range of applications, including the verification and synthesis of software and hardware systems.

Source: [Learning to Prove Theorems by Learning to Generate Theorems ](https://arxiv.org/abs/2002.07019)
Task: Graph Matching
Categories: Graphs

**Graph Matching** is the problem of finding correspondences between two sets of vertices while preserving complex relational information among them. Since the graph structure has a strong capacity to represent objects and robustness to severe deformation and outliers, it is frequently adopted to formulate various correspondence problems in the field of computer vision. Theoretically, the Graph Matching problem can be solved by exhaustively searching the entire solution space. However, this approach is infeasible in practice because the solution space expands exponentially as the size of input data increases. For that reason, previous studies have attempted to solve the problem by using various approximation techniques.

Source: [Consistent Multiple Graph Matching with Multi-layer Random Walks Synchronization ](https://arxiv.org/abs/1712.02575)
Task: Graph Embedding
Categories: Graphs

Graph embeddings learn a mapping from a network to a vector space, while preserving relevant network properties.

( Image credit: [GAT](https://github.com/PetarV-/GAT) )
Task: Graph Clustering
Categories: Graphs

**Graph Clustering** is the process of grouping the nodes of the graph into clusters, taking into account the edge structure of the graph in such a way that there are several edges within each cluster and very few between clusters. Graph Clustering intends to partition the nodes in the graph into disjoint groups.

Source: [Clustering for Graph Datasets via Gumbel Softmax ](https://arxiv.org/abs/2005.02372)
Task: Saliency Prediction
Categories: Computer Vision

A saliency map is a model that predicts eye fixations on a visual scene. Saliency prediction is informed by the human visual attention mechanism and predicts the possibility of the human eyes to stay in a certain position in the scene.
Task: Gaze Estimation
Categories: Computer Vision

**Gaze Estimation** is a task to predict where a person is looking at given the person’s full face. The task contains two directions: 3-D gaze vector and 2-D gaze position estimation. 3-D gaze vector estimation is to predict the gaze vector, which is usually used in the automotive safety. 2-D gaze position estimation is to predict the horizontal and vertical coordinates on a 2-D screen, which allows utilizing gaze point to control a cursor for human-machine interaction.

Source: [A Generalized and Robust Method Towards Practical Gaze Estimation on Smart Phone ](https://arxiv.org/abs/1910.07331)
Task: Disparity Estimation
Categories: Computer Vision

The Disparity Estimation is the task of finding the pixels in the multiscopic views that correspond to the same 3D point in the scene.
Task: Source Code Summarization
Categories: Natural Language Processing
Computer Code

**Code Summarization** is a task that tries to comprehend code and automatically generate descriptions directly from the source code.

Source: [Improving Automatic Source Code Summarization via Deep Reinforcement Learning ](https://arxiv.org/abs/1811.07234)
Task: Video Summarization
Categories: Computer Vision

**Video Summarization** aims to generate a short synopsis that summarizes the video content by selecting its most informative and important parts. The produced summary is usually composed of a set of representative video frames (a.k.a. *video key-frames*), or video fragments (a.k.a. *video key-fragments*) that have been stitched in chronological order to form a shorter video. The former type of a video summary is known as **video storyboard**, and the latter type is known as **video skim**.

Source: [Video Summarization Using Deep Neural Networks: A Survey](https://arxiv.org/abs/2101.06072)

Image credit: [iJRASET](https://www.ijraset.com/fileserve.php?FID=12932)
Task: Text Summarization
Categories: Knowledge Base
Natural Language Processing

**Text Summarization** is a natural language processing (NLP) task that involves condensing a lengthy text document into a shorter, more compact version while still retaining the most important information and meaning. The goal is to produce a summary that accurately represents the content of the original text in a concise form.

There are different approaches to text summarization, including extractive methods that identify and extract important sentences or phrases from the text, and abstractive methods that generate new text based on the content of the original text.
Task: Brain Decoding
Categories: Medical
Miscellaneous

**Motor Brain Decoding** is fundamental task for building motor brain computer interfaces (BCI).

Progress in predicting finger movements based on brain activity allows us to restore motor functions and improve rehabilitation process of patients.
Task: Colorization
Categories: Computer Vision

**Colorization** is the process of adding plausible color information to monochrome photographs or videos. Colorization is a highly undetermined problem, requiring mapping a real-valued luminance image to a three-dimensional color-valued one, that has not a unique solution.

Source: [ChromaGAN: An Adversarial Approach for Picture Colorization ](https://arxiv.org/abs/1907.09837)
Task: Lipreading
Categories: Computer Vision

Lipreading is a process of extracting speech by watching lip movements of a speaker in the absence of sound. Humans lipread all the time without even noticing. It is a big part in communication albeit not as dominant as audio. It is a very helpful skill to learn especially for those who are hard of hearing.

Deep Lipreading is the process of extracting speech from a video of a silent talking face using deep neural networks. It is also known by few other names: Visual Speech Recognition (VSR), Machine Lipreading, Automatic Lipreading etc.

The primary methodology involves two stages: i) Extracting visual and temporal features from a sequence of image frames from a silent talking video ii) Processing the sequence of features into units of speech e.g. characters, words, phrases etc. We can find several implementations of this methodology either done in two separate stages or trained end-to-end in one go.
Task: Atari Games
Categories: Playing Games

The Atari 2600 Games task (and dataset) involves training an agent to achieve high game scores.

( Image credit: [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/pdf/1312.5602v1.pdf) )
Task: Rumour Detection
Categories: Natural Language Processing

Rumor detection is the task of identifying rumors, i.e. statements whose veracity is not quickly or ever confirmed, in utterances on social media platforms.
Task: Representation Learning
Categories: Methodology
Natural Language Processing
Computer Vision

**Representation Learning** is a process in machine learning where algorithms extract meaningful patterns from raw data to create representations that are easier to understand and process. These representations can be designed for interpretability, reveal hidden features, or be used for transfer learning. They are valuable across many fundamental machine learning tasks like [image classification](/task/image-classification) and [retrieval](/task/image-retrieval).

Deep neural networks can be considered representation learning models that typically encode information which is projected into a different subspace. These representations are then usually passed on to a linear classifier to, for instance, train a classifier.

Representation learning can be divided into:

- **Supervised representation learning**: learning representations on task A using annotated data and used to solve task B
- **Unsupervised representation learning**: learning representations on a task in an unsupervised way (label-free data). These are then used to address downstream tasks and reducing the need for annotated data when learning news tasks. Powerful models like [GPT](/method/gpt) and [BERT](/method/bert) leverage unsupervised representation learning to tackle language tasks.

More recently, [self-supervised learning (SSL)](/task/self-supervised-learning) is one of the main drivers behind unsupervised representation learning in fields like computer vision and NLP.

Here are some additional readings to go deeper on the task:

- [Representation Learning: A Review and New Perspectives](/paper/representation-learning-a-review-and-new) - Bengio et al. (2012)
- [A Few Words on Representation Learning](https://sthalles.github.io/a-few-words-on-representation-learning/) - Thalles Silva

( Image credit: [Visualizing and Understanding Convolutional Networks](https://arxiv.org/pdf/1311.2901.pdf) )
Task: Word Embeddings
Categories: Methodology
Natural Language Processing

Word embedding is the collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from the vocabulary are mapped to vectors of real numbers.

Techniques for learning word embeddings can include Word2Vec, GloVe, and other neural network-based approaches that train on an NLP task such as language modeling or document classification.

( Image credit: [Dynamic Word Embedding for Evolving Semantic Discovery](https://arxiv.org/pdf/1703.00607v2.pdf) )
Task: Imitation Learning
Categories: Methodology

**Imitation Learning** is a framework for learning a behavior policy from demonstrations. Usually, demonstrations are presented in the form of state-action trajectories, with each pair indicating the action to take at the state being visited. In order to learn the behavior policy, the demonstrated actions are usually utilized in two ways. The first, known as Behavior Cloning (BC), treats the action as the target label for each state, and then learns a generalized mapping from states to actions in a supervised manner. Another way, known as Inverse Reinforcement Learning (IRL), views the demonstrated actions as a sequence of decisions, and aims at finding a reward/cost function under which the demonstrated decisions are optimal.

Finally, a newer methodology, Inverse Q-Learning aims at directly learning Q-functions from expert data, implicitly representing rewards, under which the optimal policy can be given as a Boltzmann distribution similar to soft Q-learning

Source: [Learning to Imitate ](https://ai.stanford.edu/blog/learning-to-imitate)
Task: Talking Head Generation
Categories: Computer Vision

Talking head generation is the task of generating a talking face from a set of images of a person.

( Image credit: [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://arxiv.org/pdf/1905.08233v2.pdf) )