3497hyy/evaluation-tables huggingface dataset:
Task: Instance Segmentation
Categories: Computer Vision

**Instance Segmentation** is a computer vision task that involves identifying and separating individual objects within an image, including detecting the boundaries of each object and assigning a unique label to each object. The goal of instance segmentation is to produce a pixel-wise segmentation map of the image, where each pixel is assigned to a specific object instance.

Image Credit: [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers, CVPR'21](https://github.com/lkeab/BCNet)
Task: Event-based vision
Categories: Computer Vision

An event camera, also known as a neuromorphic camera, silicon retina or dynamic vision sensor, is an imaging sensor that responds to local changes in brightness. Event cameras do not capture images using a shutter as conventional cameras do. Instead, each pixel inside an event camera operates independently and asynchronously, reporting changes in brightness as they occur and staying silent otherwise. Modern event cameras have microsecond temporal resolution, 120 dB dynamic range, and less under/overexposure and motion blur than frame cameras.
Task: Text Effects Transfer
Categories: Natural Language Processing

Text effects transfer refers to the task of transferring typography styles (e.g., color, texture) to an input image of a text element.
Task: Document Layout Analysis
Categories: Computer Vision

"**Document Layout Analysis** is performed to determine physical structure of a document, that is, to determine document components. These document components can consist of single connected components-regions [...] of
pixels that are adjacent to form single regions [...] , or group
of text lines. A text line is a group of characters, symbols,
and words that are adjacent, “relatively close” to each other
and through which a straight line can be drawn (usually with
horizontal or vertical orientation)." L. O'Gorman, "The document spectrum for page layout analysis," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 11, pp. 1162-1173, Nov. 1993.

Image credit: [PubLayNet: largest dataset ever for document layout analysis](https://arxiv.org/pdf/1908.07836v1.pdf)
Task: Pose Tracking
Categories: Computer Vision

**Pose Tracking** is the task of estimating multi-person human poses in videos and assigning unique instance IDs for each keypoint across frames. Accurate estimation of human keypoint-trajectories is useful for human action recognition, human interaction understanding, motion capture and animation.

Source: [LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking ](https://arxiv.org/abs/1905.02822)
Task: Direction of Arrival Estimation
Categories: Audio

Estimating the direction-of-arrival (DOA) of a sound source from multi-channel recordings.
Task: Polyphone disambiguation
Categories: Natural Language Processing

A part of the TTS-front end framework which serves to predict the correct pronunciation for the input polyphone characters.
Task: Few-Shot Semantic Segmentation
Categories: Computer Vision

Few-shot semantic segmentation (FSS) learns to segment target objects in query image given few pixel-wise annotated support image.
Task: Safety Perception Recognition
Categories: Computer Vision

City safety perception recognition
Task: Unsupervised Image-To-Image Translation
Categories: Computer Vision

Unsupervised image-to-image translation is the task of doing image-to-image translation without ground truth image-to-image pairings.

( Image credit: [Unpaired Image-to-Image Translation
using Cycle-Consistent Adversarial Networks](https://arxiv.org/pdf/1703.10593v6.pdf) )
Task: Vision-based navigation with language-based assistance
Categories: Robots

A grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to the target objects and thus makes requests by only specifying high-level endgoals, and (b) the agent is capable of sensing when it is lost and querying an advisor, who is more qualified at the task, to obtain language subgoals to make progress.
Task: Scene Flow Estimation
Categories: Computer Vision

Optical flow is a two-dimensional motion field in the image plane. It is the projection of the three-dimensional motion of the world. If the world is completely non-rigid, the motions of the points in the scene may all be indepen- dent of each other. One representation of the scene motion is therefore a dense **three-dimensional vector field** defined for every point on every surface in the scene. By analogy with optical flow, we refer to this three-dimensional motion field as **scene flow**.

Source: Vedula, Sundar, et al. "Three-dimensional scene flow." IEEE transactions on pattern analysis and machine intelligence 27.3 (2005): 475-480. [pdf](https://ieeexplore.ieee.org/document/1388274)
Task: Query Wellformedness
Categories: Natural Language Processing

Assessing whether a query is grammatically correct, contains no spelling mistakes, and asks an explicit question.

Image Source: [Identifying Well-formed Natural Language Questions](https://arxiv.org/pdf/1808.09419.pdf)
Task: Automatic Writing
Categories: Natural Language Processing

Generating text based on internal machine representations.
Task: Metric Learning
Categories: Methodology
Computer Vision

The goal of **Metric Learning** is to learn a representation function that maps objects into an embedded space. The distance in the embedded space should preserve the objects’ similarity — similar objects get close and dissimilar objects get far away. Various loss functions have been developed for Metric Learning. For example, the **contrastive loss** guides the objects from the same class to be mapped to the same point and those from different classes to be mapped to different points whose distances are larger than a margin. **Triplet loss** is also popular, which requires the distance between the anchor sample and the positive sample to be smaller than the distance between the anchor sample and the negative sample.

Source: [Road Network Metric Learning for Estimated Time of Arrival ](https://arxiv.org/abs/2006.13477)
Task: Phrase Vector Embedding
Categories: Natural Language Processing

Just like the generation of word (1-gram) vector embedding, this task is for phrase (n-gram) vector embedding.
Task: Bayesian Optimisation
Categories: Methodology

Expensive black-box functions are a common problem in many disciplines, including tuning the parameters of machine learning algorithms, robotics, and other engineering design problems. **Bayesian Optimisation** is a principled and efficient technique for the global optimisation of these functions. The idea behind Bayesian Optimisation is to place a prior distribution over the target function and then update that prior with a set of “true” observations of the target function by expensively evaluating it in order to produce a posterior predictive distribution. The posterior then informs where to make the next observation of the target function through the use of an acquisition function, which balances the exploitation of regions known to have good performance with the exploration of regions where there is little information about the function’s response.

Source: [A Bayesian Approach for the Robust Optimisation of Expensive-to-Evaluate Functions ](https://arxiv.org/abs/1904.11416)
Task: Surface Normals Estimation
Categories: Computer Vision

Surface normal estimation deals with the task of predicting the surface orientation of the objects present inside a scene. Refer to [Designing Deep Networks for Surface Normal Estimation (Wang et al.)](https://www.cs.cmu.edu/~xiaolonw/papers/deep3d.pdf) to get a good overview of several design choices that led to the development of a CNN-based surface normal estimator.
Task: Table Detection
Categories: Miscellaneous

Image credit:[Table Detection in the Wild: A Novel Diverse Table Detection Dataset and Method](https://paperswithcode.com/paper/table-detection-in-the-wild-a-novel-diverse)
Task: Text Matching
Categories: Natural Language Processing

Matching a target text to a source text based on their meaning.
Task: Question Similarity
Categories: Natural Language Processing

This is the problem of detecting duplicate questions in forums, which is an important step towards automating the process of answering new questions
Task: Video Object Segmentation
Categories: Computer Vision

Video object segmentation is a binary labeling problem aiming to separate foreground object(s) from the background region of a video.

For leaderboards please refer to the different subtasks.
Task: Participant Intervention Comparison Outcome Extraction
Categories: Medical

PICO recognition is an information extraction task for identifying Participant, Intervention, Comparator, and Outcome (PICO elements) information from clinical literature.
Task: Music Genre Recognition
Categories: Music

Recognizing the genre (e.g. rock, pop, jazz, etc.) of a piece of music.
Task: Morphological Inflection
Categories: Natural Language Processing

**Morphological Inflection** is the task of generating a target (inflected form) word from a source word (base form), given a morphological attribute, e.g. number, tense, and person etc. It is useful for alleviating data sparsity issues in translating morphologically rich languages. The transformation from a base form to an inflected form usually includes concatenating the base form with a prefix or a suffix and substituting some characters. For example, the inflected form of a Finnish stem eläkeikä (retirement age) is eläkeiittä when the case is abessive and the number is plural.

Source: [Tackling Sequence to Sequence Mapping Problems with Neural Networks ](https://arxiv.org/abs/1810.10802)
Task: Quantization
Categories: Methodology
Computer Vision

**Quantization** is a promising technique to reduce the computation cost of neural network training, which can replace high-cost floating-point numbers (e.g., float32) with low-cost fixed-point numbers (e.g., int8/int16).

Source: [Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers ](https://arxiv.org/abs/1911.00361)
Task: Privacy Preserving Deep Learning
Categories: Methodology
Miscellaneous
Computer Vision

The goal of privacy-preserving (deep) learning is to train a model while preserving privacy of the training dataset. Typically, it is understood that the trained model should be privacy-preserving (e.g., due to the training algorithm being differentially private).
Task: Image Imputation
Categories: Computer Vision

Image imputation is the task of creating plausible images from low-resolution images or images with missing data.

( Image credit: [NASA](https://www.jpl.nasa.gov/edu/news/2019/4/19/how-scientists-captured-the-first-image-of-a-black-hole/) )
Task: Image Matting
Categories: Computer Vision

**Image Matting** is the process of accurately estimating the foreground object in images and videos. It is a very important technique in image and video editing applications, particularly in film production for creating visual effects. In case of image segmentation, we segment the image into foreground and background by labeling the pixels. Image segmentation generates a binary image, in which a pixel either belongs to foreground or background. However, Image Matting is different from the image segmentation, wherein some pixels may belong to foreground as well as background, such pixels are called partial or mixed pixels. In order to fully separate the foreground from the background in an image, accurate estimation of the alpha values for partial or mixed pixels is necessary.

Source: [Automatic Trimap Generation for Image Matting ](https://arxiv.org/abs/1707.00333)

Image Source: [Real-Time High-Resolution Background Matting](https://arxiv.org/pdf/2012.07810v1.pdf)
Task: Density Estimation
Categories: Methodology

The goal of **Density Estimation** is to give an accurate description of the underlying probabilistic density distribution of an observable data set with unknown density.

Source: [Contrastive Predictive Coding Based Feature for Automatic Speaker Verification ](https://arxiv.org/abs/1904.01575)
Task: Lemmatization
Categories: Natural Language Processing

**Lemmatization** is a process of determining a base or dictionary form (lemma) for a given surface form. Especially for languages with rich morphology it is important to be able to normalize words into their base forms to better support for example search engines and linguistic studies. Main difficulties in Lemmatization arise from encountering previously unseen words during inference time as well as disambiguating ambiguous surface forms which can be inflected variants of several different base forms depending on the context.

Source: [Universal Lemmatizer: A Sequence to Sequence Model for Lemmatizing Universal Dependencies Treebanks ](https://arxiv.org/abs/1902.00972)
Task: Knowledge Base Population
Categories: Knowledge Base
Natural Language Processing

Knowledge base population is the task of filling the incomplete elements of a given knowledge base by automatically processing a large corpus of text.
Task: Image Cropping
Categories: Computer Vision

**Image Cropping** is a common photo manipulation process, which improves the overall composition by removing unwanted regions. Image Cropping is widely used in photographic, film processing, graphic design, and printing businesses.

Source: [Listwise View Ranking for Image Cropping ](https://arxiv.org/abs/1905.05352)
Task: Document Ranking
Categories: Natural Language Processing

Sort documents according to some criterion so that the "best" results appear early in the result list displayed to the user (Source: Wikipedia).
Task: Conversational Response Selection
Categories: Natural Language Processing

Conversational response selection refers to the task of identifying the most relevant response to a given input sentence from a collection of sentences.
Task: Sign Language Translation
Categories: Computer Vision

Given a video containing sign language, the task is to predict the translation into (written) spoken language.

Image credit: [How2Sign](https://how2sign.github.io/)
Task: Weakly Supervised Action Localization
Categories: Computer Vision

In this task, the training data consists of videos with a list of activities in them without any temporal boundary annotations. However, while testing, given a video, the algorithm should recognize the activities in the video and also provide the start and end time.
Task: Birds Eye View Object Detection
Categories: Computer Vision

KITTI birds eye view detection task
Task: Fine-Grained Image Classification
Categories: Computer Vision

**Fine-Grained Image Classification** is a task in computer vision where the goal is to classify images into subcategories within a larger category. For example, classifying different species of birds or different types of flowers. This task is considered to be fine-grained because it requires the model to distinguish between subtle differences in visual appearance and patterns, making it more challenging than regular image classification tasks.

( Image credit: [Looking for the Devil in the Details](https://arxiv.org/pdf/1903.06150v2.pdf) )
Task: Web Credibility
Categories: Methodology

Define the level of credibility of web sources
Task: Keyword Extraction
Categories: Natural Language Processing

Keyword extraction is tasked with the automatic identification of terms that best describe the subject of a document (Source: Wikipedia).
Task: Meeting Summarization
Categories: Natural Language Processing

Generating a summary from meeting transcriptions.
A survey for this task: [Abstractive Meeting Summarization: A Survey](https://paperswithcode.com/paper/abstractive-meeting-summarization-a-survey)
Task: 3D Object Classification
Categories: Computer Vision

3D Object Classification is the task of predicting the class of a 3D object point cloud. It is a voxel level prediction where each voxel is classified into a category. The popular benchmark for this task is the ModelNet dataset. The models for this task are usually evaluated with the Classification Accuracy metric.

Image: [Sedaghat et al](https://arxiv.org/pdf/1604.03351v2.pdf)
Task: Chatbot
Categories: Methodology
Speech
Natural Language Processing

**Chatbot** or conversational AI is a language model designed and implemented to have conversations with humans.

Source: [Open Data Chatbot ](https://arxiv.org/abs/1909.03653)

[Image source](https://arxiv.org/pdf/2006.16779v3.pdf)
Task: 3D Multi-Person Pose Estimation
Categories: Computer Vision

This task aims to solve root-relative 3D multi-person pose estimation. No human bounding box and root joint coordinate groundtruth are used in testing time.

( Image credit: [RootNet](https://github.com/mks0601/3DMPPE_ROOTNET_RELEASE) )
Task: Table-to-Text Generation
Categories: Natural Language Processing

**Here is the provided data converted into a table format for clarity:
COUNTRIES	1971-2010	2011	2012	2013	2014	2015	2016	2017	2018

Saudi Arabia	2742962	222247	358560	270502	312489	522750	462598	143363	100910
U.A.E	1595574	156353	182630	273234	350522	326986	295647	275436	208635
Oman	394436	53525	69407	47794	39793	47788	45085	42362	27202
Qatar	82043	5121	7320	8119	10042	12741	9706	11592	20993
Bahrain	94599	10641	10530	9600	9226	9029	8226	7919	5745
Kuwait	180755	173	5	229	132	164	770	773	493
South Korea	15343	12	7	12	46	13	17	9	13
Malaysia	23410	2092	1309	2031	20577	20216	10625	7174	9881
China	1717	180	220	155	254	355	482	457	854
Algeria	878	7	2	7	36	211	259	461	213
Angola	601	8	6	8	1	22	22	12	11
Azerbaijan	51	0	3	98	22	8	8	8	20
Brunei	998	79	74	67	48	85	85	212	225
Cameroon	48	15	0	0	3	2	0	1	4
Croatia	44	1	0	0	0	0	0	0	0
Cyprus	922	71	129	111	278	500	990	1729	1644
Gabon	299	2	4	1	8	0	0	2	0
Gen-Island	195	0	0	0	0	2	0	0	0
Germany	187	11	23	26	23	43	38	64	103
Greece	542	0	0	0	0	2	3	2	3
Guinea	144	15	12	13	6	10	11	6	11
Hong Kong	252	26	17	20	38	29	38	54	57
Iran	12586	14	3	26	5	65	37	100	20
Iraq	68135	0	32	951	1041	709	543	599	756
Italy	17763	2875	3361	2068	1563	431	242	141	86
Japan	380	48	62	44	69	82	102	153	258
Jordan	5341	178	279	345	328	321	282	285	170
Kenya	67	11	8	6	3	11	15	8	17
Lebanon	432	30	23	15	57	33	42	24	27
Libya	72112	490	1872	4543	2121	8	0	4	8
Morocco	44	0	0	0	2	0	0	1	5
Nigeria	2665	166	142	117	113	106	104	75	115
Task: 3D Face Animation
Categories: Playing Games
Computer Vision

Image: [Cudeiro et al](https://arxiv.org/pdf/1905.03079v1.pdf)
Task: Feature Engineering
Categories: Methodology
Miscellaneous

Feature engineering is the process of taking a dataset and constructing explanatory variables — features — that can be used to train a machine learning model for a prediction problem. Often, data is spread across multiple tables and must be gathered into a single table with rows containing the observations and features in the columns.

The traditional approach to feature engineering is to build features one at a time using domain knowledge, a tedious, time-consuming, and error-prone process known as manual feature engineering. The code for manual feature engineering is problem-dependent and must be re-written for each new dataset.
Task: 3D Semantic Segmentation
Categories: Computer Vision

**3D Semantic Segmentation** is a computer vision task that involves dividing a 3D point cloud or 3D mesh into semantically meaningful parts or regions. The goal of 3D semantic segmentation is to identify and label different objects and parts within a 3D scene, which can be used for applications such as robotics, autonomous driving, and augmented reality.
Task: 4D Spatio Temporal Semantic Segmentation
Categories: Computer Vision

Image: [Choy et al](https://paperswithcode.com/paper/4d-spatio-temporal-convnets-minkowski)
Task: Epidemiology
Categories: Medical

**Epidemiology** is a scientific discipline that provides reliable knowledge for clinical medicine focusing on prevention, diagnosis and treatment of diseases. Research in Epidemiology aims at characterizing risk factors for the outbreak of diseases and at evaluating the efficiency of certain treatment strategies, e.g., to compare a new treatment with an established gold standard. This research is strongly hypothesis-driven and statistical analysis is the major tool for epidemiologists so far. Correlations between genetic factors, environmental factors, life style-related parameters, age and diseases are analyzed.

Source: [Visual Analytics of Image-Centric Cohort Studies in Epidemiology ](https://arxiv.org/abs/1501.04009)
Task: incongruity detection
Categories: Natural Language Processing

Incongruity detection is the task of identifying statements in a text that are inconsistent with each other.
Task: Image Super-Resolution
Categories: Computer Vision

**Image Super-Resolution** is a machine learning task where the goal is to increase the resolution of an image, often by a factor of 4x or more, while maintaining its content and details as much as possible. The end result is a high-resolution version of the original image. This task can be used for various applications such as improving image quality, enhancing visual detail, and increasing the accuracy of computer vision algorithms.
Task: Text Attribute Transfer
Categories: Natural Language Processing

The goal of the **Text Attribute Transfer** task is to change an input text such that the value of a particular linguistic attribute of interest (e.g. language = English, sentiment = Positive) is transferred to a different desired value (e.g. language = French, sentiment = Negative). This task needs approaches that can disentangle the content from other linguistic attributes of the text.

Source: [Improved Neural Text Attribute Transfer with Non-parallel Data ](https://arxiv.org/abs/1711.09395)
Task: Relation Mention Extraction
Categories: Natural Language Processing

Extracting phrases representative for a specific relation.
Task: Bayesian Inference
Categories: Methodology

Bayesian Inference is a methodology that employs Bayes Rule to estimate parameters (and their full posterior).
Task: Action Recognition
Categories: Time Series
Computer Vision

**Action Recognition** is a computer vision task that involves recognizing human actions in videos or images. The goal is to classify and categorize the actions being performed in the video or image into a predefined set of action classes.

In the video domain, it is an open question whether training an action classification network on a sufficiently large dataset, will give a similar boost in performance when applied to a different temporal task or dataset. The challenges of building video datasets has meant that most popular benchmarks for action recognition are small, having on the order of 10k videos.

Please note some benchmarks may be located in the [Action Classification](https://paperswithcode.com/task/action-classification) or [Video Classification](https://paperswithcode.com/task/video-classification) tasks, e.g. Kinetics-400.
Task: Text-To-SQL
Categories: Natural Language Processing
Computer Code

**Text-to-SQL** is a task in natural language processing (NLP) where the goal is to automatically generate SQL queries from natural language text. The task involves converting the text input into a structured representation and then using this representation to generate a semantically correct SQL query that can be executed on a database.

( Image credit: [SyntaxSQLNet](https://arxiv.org/pdf/1810.05237v2.pdf) )
Task: Image-To-Gps Verification
Categories: Computer Vision

The image-to-GPS verification task asks whether a given image is taken at a claimed GPS location.

( Image credit: [Image-to-GPS Verification Through A Bottom-Up Pattern Matching Network](https://arxiv.org/pdf/1811.07288v1.pdf) )