3497hyy/evaluation-tables huggingface dataset:
Task: Reflection Removal
Categories: Computer Vision

Remove the spots from mirror and clear the picture
Task: Template Matching
Categories: Computer Vision

Template matching is a technique that is used to find a subimage or a patch (called the template) within a larger image. The basic idea behind template matching is to slide the template image over the larger image and compare the template to each portion of the larger image to determine the similarity between the template and the corresponding portion of the larger image.
Task: Voice Anti-spoofing
Categories: Audio

Discriminate genuine speech and spoofing attacks
Task: Model extraction
Categories: Methodology
Adversarial

Model extraction attacks, aka model stealing attacks, are used to extract the parameters from the target model. Ideally, the adversary will be able to steal and replicate a model that will have a very similar performance to the target model.
Task: Audio declipping
Categories: Audio

Audio declipping is the task of estimating the original audio signal given its clipped measurements.
Task: Long-tail Learning
Categories: Methodology

Long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing models from a large number of images that follow a long-tailed class distribution.
Task: Triple Classification
Categories: Graphs

Triple classification aims to judge whether a given triple (h, r, t) is correct or not with respect to the knowledge graph.
Task: Video Retrieval
Categories: Computer Vision

The objective of video retrieval is as follows: given a text query and a pool of candidate videos, select the video which corresponds to the text query. Typically, the videos are returned as a ranked list of candidates and scored via document retrieval metrics.
Task: Event Extraction
Categories: Natural Language Processing

Determine the extent of the events in a text.

Other names: Event Tagging; Event Identification
Task: Unsupervised Landmark Detection
Categories: Computer Vision

The discovery of object landmarks on a set of images depicting objects of the same category, directly from raw images without using any manual annotations.
Task: Question Generation
Categories: Natural Language Processing

The goal of **Question Generation** is to generate a valid and fluent question according to a given passage and the target answer. Question Generation can be used in many scenarios, such as automatic tutoring systems, improving the performance of Question Answering models and enabling chatbots to lead a conversation.

Source: [Generating Highly Relevant Questions ](https://arxiv.org/abs/1910.03401)
Task: Influence Approximation
Categories: Methodology

Estimating the influence of training triples on the behavior of a machine learning model.
Task: Handwriting generation
Categories: Computer Vision

The inverse of handwriting recognition. From text generate and image of handwriting (offline) of trajectory of handwriting (online).
Task: Extractive Tags Summarization
Categories: Natural Language Processing

The goal of Extractive Tags Summarization (ETS) task is to shorten the list of tags corresponding to a digital image while keeping the representativity; i.e., is to extract important tags from the context lying in an image and its corresponding tags.
Task: Handwriting Verification
Categories: Natural Language Processing
Computer Vision

The goal of handwriting verification is to find a measure of confidence whether the given handwritten samples are written by the same or different writer.
Task: Intent Discovery
Categories: Natural Language Processing

Given a set of labelled and unlabelled utterances, the idea is to identify existing (known) intents and potential (new intents) intents. This method can be utilised in conversational system setting.
Task: Audio Dequantization
Categories: Audio

Audio Dequantization is a process of estimating the original signal from its quantized counterpart.
Task: Face Selection
Categories: Natural Language Processing

A task where an agent should select at most two sentences from the paper as argumentative facts.
Task: Graph Representation Learning
Categories: Graphs

The goal of **Graph Representation Learning** is to construct a set of features (‘embeddings’) representing the structure of the graph and the data thereon. We can distinguish among Node-wise embeddings, representing each node of the graph, Edge-wise embeddings, representing each edge in the graph, and Graph-wise embeddings representing the graph as a whole.

Source: [SIGN: Scalable Inception Graph Neural Networks ](https://arxiv.org/abs/2004.11198)
Task: Brain Visual Reconstruction
Categories: Computer Vision

Reconstruct viewed or imagined visual stimuli, including but not limited to images and videos from brain activities patterns recorded with neuoimaging technoligies.
Task: Logical Reasoning Reading Comprehension
Categories: Natural Language Processing

Logical reasoning reading comprehension is a task proposed by the paper ReClor (ICLR 2020), which is to evaluate the logical reasoning ability of machine reading comprehension models. ReClor is the first dataset for logical reasoning reading comprehension.
Task: Video Matting
Categories: Computer Vision

Image credit: [https://arxiv.org/pdf/2012.07810v1.pdf](https://arxiv.org/pdf/2012.07810v1.pdf)
Task: Table annotation
Categories: Knowledge Base
Natural Language Processing

**Table annotation** is the task of annotating a table with terms/concepts from knowledge graph or database schema. Table annotation is typically broken down into the following five subtasks:

1. Cell Entity Annotation ([CEA](https://paperswithcode.com/task/cell-entity-annotation))
2. Column Type Annotation ([CTA](https://paperswithcode.com/task/column-type-annotation))
3. Column Property Annotation ([CPA](https://paperswithcode.com/task/columns-property-annotation))
4. [Table Type Detection](https://paperswithcode.com/task/table-type-detection)
5. [Row Annotation](https://paperswithcode.com/task/row-annotation)

The [SemTab](http://www.cs.ox.ac.uk/isg/challenges/sem-tab/) challenge is closely related to the Table Annotation problem. It is a yearly challenge which focuses on the first three tasks of table annotation and its purpose is to benchmark different table annotation systems.
Task: SVBRDF Estimation
Categories: Computer Vision

SVBRDF Estimation
Task: Speaker Orientation
Categories: Audio

Direction of Voice or speaker orientation of the person with respect to the target device.
Task: Protein Structure Prediction
Categories: Medical
Miscellaneous

Image credit: [FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours](https://arxiv.org/pdf/2203.00854v1.pdf)
Task: Surface Normals Estimation from Point Clouds
Categories: Computer Vision

Parent task: 3d Point Clouds Analysis
Task: Medical Image Denoising
Categories: Computer Vision

Image credit: [Learning Medical Image Denoising with Deep Dynamic Residual Attention Network](https://paperswithcode.com/paper/learning-medical-image-denoising-with-deep)
Task: Fact Verification
Categories: Natural Language Processing

Fact verification, also called "fact checking", is a process of verifying facts in natural text against a database of facts.
Task: Natural Language Understanding
Categories: Natural Language Processing

**Natural Language Understanding** is an important field of Natural Language Processing which contains various tasks such as text classification, natural language inference and story comprehension. Applications enabled by natural language understanding range from question answering to automated reasoning.

Source: [Find a Reasonable Ending for Stories: Does Logic Relation Help the Story Cloze Test?](https://arxiv.org/abs/1812.05411)
Task: 3D Object Tracking
Categories: Computer Vision

3D Object Tracking is a computer vision task dedicated to monitoring and precisely locating objects as they navigate within a three-dimensional environment. It frequently utilizes 3D object detection techniques to pinpoint the objects and establish unique identifications that persist across multiple frames.
Task: 3D Multi-Object Tracking
Categories: Computer Vision

Image: [Weng et al](https://arxiv.org/pdf/1907.03961v4.pdf)
Task: Visual Dialog
Categories: Computer Vision

Visual Dialog requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content. Specifically, given an image, a dialog history, and a follow-up question about the image, the task is to answer the question.
Task: Social Media Popularity Prediction
Categories: Miscellaneous
Time Series

Social Media Popularity Prediction (SMPP) aims to predict the future popularity (e.g., clicks, views, likes, etc.) of online posts automatically via plenty of social media data from public platforms. It is a crucial problem for social media learning and forecasting and one of the most challenging problems in the field. With the ever-changing user interests and public attention on social media platforms, how to predict popularity accurately becomes more challenging than before. This task is valuable to content providers, marketers, or consumers in a range of real-world applications, including multimedia advertising, recommendation system, or trend analysis.
Task: Scene Change Detection
Categories: Computer Vision

Scene change detection (SCD) refers to the task of localizing changes and identifying change-categories given two scenes. A scene can be either an RGB (+D) image or a 3D reconstruction (point cloud). If the scene is an image, SCD is a form of pixel-level prediction because each pixel in the image is classified according to a category. On the other hand, if the scene is point cloud, SCD is a form of point-level prediction because each point in the cloud is classified according to a category.

Some example benchmarks for this task are VL-CMU-CD, PCD, and CD2014. Recently, more complicated benchmarks such as ChangeSim, HDMap, and Mallscape are released.

Models are usually evaluated with the Mean Intersection-Over-Union (Mean IoU), Pixel Accuracy, or F1 metrics.
Task: Table Recognition
Categories: Computer Vision

Table recognition refers to the process of automatically identifying and extracting tabular structures from unstructured data sources such as text documents, images, or scanned documents. The goal of table recognition is to accurately detect the presence of tables within the data and extract their contents, including rows, columns, headers, and cell values.
Task: Few-Shot Object Detection
Categories: Computer Vision

**Few-Shot Object Detection** is a computer vision task that involves detecting objects in images with limited training data. The goal is to train a model on a few examples of each object class and then use the model to detect objects in new images.
Task: Vehicle Re-Identification
Categories: Computer Vision

Vehicle re-identification is the task of identifying the same vehicle across multiple cameras.

( Image credit: [A Two-Stream Siamese Neural Network for Vehicle Re-Identification by Using Non-Overlapping Cameras](https://github.com/icarofua/siamese-two-stream) )
Task: Open-Domain Question Answering
Categories: Natural Language Processing

Open-domain question answering is the task of question answering on open-domain datasets such as Wikipedia.
Task: Music Transcription
Categories: Music

Music transcription is the task of converting an acoustic musical signal into some form of music notation.

( Image credit: [ISMIR 2015 Tutorial - Automatic Music Transcription](http://c4dm.eecs.qmul.ac.uk/ismir15-amt-tutorial/AMT_tutorial_ISMIR_2015.pdf) )
Task: Video Instance Segmentation
Categories: Computer Vision

The goal of video instance segmentation is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain.

To facilitate research on this new task, a large-scale benchmark called YouTube-VIS, which consists of 2,883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks is built.
Task: Speech Denoising
Categories: Speech

Obtain the clean speech of the target speaker by suppressing the background noise. Recent representative github platform [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio)
Task: Clinical Assertion Status Detection
Categories: Natural Language Processing

Classifying the assertions made on given medical concepts as being present, absent, or possible in the patient, conditionally present in the patient under certain circumstances, hypothetically present in the patient at some future point, and mentioned in the patient report but associated with someoneelse. (e.g. clinical finding pertains to the patient by assigning a label such as present (”patient is diabetic”), absent (”patient denies nausea”), conditional (”dyspnea while climbing stairs”), or associated with someone else (”family history of depression”))

( [Source](https://arxiv.org/pdf/2012.04005v1.pdf) )
Task: Clinical Concept Extraction
Categories: Medical
Natural Language Processing

Automatic extraction of clinical named entities such as clinical problems, treatments, tests and anatomical parts from clinical notes.

( [Source](https://arxiv.org/pdf/2012.04005v1.pdf) )
Task: Recognizing Emotion Cause in Conversations
Categories: Natural Language Processing

Given an utterance U, labeled with emotion E, the task is to extract the causal spans S from the conversational history H (including utterance U) that sufficiently represent the causes of emotion E.
Task: Document Summarization
Categories: Natural Language Processing

Automatic **Document Summarization** is the task of rewriting a document into its shorter form while still retaining its important content. The most popular two paradigms are extractive approaches and abstractive approaches. Extractive approaches generate summaries by extracting parts of the original document (usually sentences), while abstractive methods may generate new words or phrases which are not in the original document.

Source: [HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization ](https://arxiv.org/abs/1905.06566)
Task: Argument Mining
Categories: Natural Language Processing

**Argument Mining** is a field of corpus-based discourse analysis that involves the automatic identification of argumentative structures in text.

Source: [AMPERSAND: Argument Mining for PERSuAsive oNline Discussions ](https://arxiv.org/abs/2004.14677)
Task: Deep Learning
Categories: Methodology
Natural Language Processing
Computer Code
Computer Vision

forecast
Task: 2D Human Pose Estimation
Categories: Knowledge Base
Reasoning
Computer Vision

What is Human Pose Estimation?
Human pose estimation is the process of estimating the configuration of the body (pose) from a single, typically monocular, image. Background. Human pose estimation is one of the key problems in computer vision that has been studied for well over 15 years. The reason for its importance is the
abundance of applications that can benefit from such a technology. For example,
human pose estimation allows for higher-level reasoning in the context of human-computer interaction and activity recognition; it is also one of the basic building blocks for marker-less motion capture (MoCap) technology. MoCap technology is useful for applications ranging from character animation to clinical analysis of gait pathologies.
Task: Fact Selection
Categories: Natural Language Processing

A task where an agent should select at most two sentences from the paper as argumentative facts.
Task: Automatic Modulation Recognition
Categories: Time Series

Automatic modulation recognition/classification identifies the modulation pattern of communication signals received from wireless or wired networks.
Task: Neural Rendering
Categories: Computer Vision

Given a representation of a 3D scene of some kind (point cloud, mesh, voxels, etc.), the task is to create an algorithm that can produce photorealistic renderings of this scene from an arbitrary viewpoint. Sometimes, the task is accompanied by image/scene appearance manipulation.
Task: 3D Medical Imaging Segmentation
Categories: Medical

3D medical imaging segmentation is the task of segmenting medical objects of interest from 3D medical imaging.

( Image credit: [Elastic Boundary Projection for 3D Medical Image Segmentation](https://github.com/twni2016/Elastic-Boundary-Projection) )
Task: Code Documentation Generation
Categories: Computer Code

Code Documentation Generation is a supervised task where a code function is the input to the model, and the model generates the documentation for this function.

Description from: [CodeTrans: Towards Cracking the Language of Silicone's Code Through Self-Supervised Deep Learning and High Performance Computing](https://arxiv.org/pdf/2104.02443.pdf)
Task: 3D Pose Estimation
Categories: Computer Vision

Image credit: [GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision
, ECCV'20](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123600511.pdf)
Task: Mathematical Question Answering
Categories: Knowledge Base
Time Series
Reasoning

Building systems that automatically answer mathematical questions.
Task: Speech-to-Text Translation
Categories: Natural Language Processing

Translate audio signals of speech in one language into text in a foreign language, either in an end-to-end or cascade manner.
Task: Seismic source localization
Categories: Time Series

Locating a seismic source using seismometer recordings
Task: Turning Point Identification
Categories: Natural Language Processing

Identification of key events in a narrative (such as movie or TV episode). The task is supported by screenwriting theory, according to which there are 5 different types of key events in a movie. These key events (e.g., change of plans, major setback, climax) are crucial narrative moments: they define the plot structure and determine its progression and thematic units (e.g., setup, complications, aftermath).
Task: Persian Sentiment Analysis
Categories: Natural Language Processing

Persian Sentiment analysis is the task of classifying the polarity of a given text.
Task: Lip Reading
Categories: Time Series

**Lip Reading** is a task to infer the speech content in a video by using only the visual information, especially the lip movements. It has many crucial applications in practice, such as assisting audio-based speech recognition, biometric authentication and aiding hearing-impaired people.

Source: [Mutual Information Maximization for Effective Lip Reading ](https://arxiv.org/abs/2003.06439)
Task: Edge-computing
Categories: Computer Code
Time Series

Deep Learning on EDGE devices
Task: Speaker Profiling
Categories: Speech

Estimation of Physical parameters from Speech data
Task: Medical Procedure
Categories: Medical

Predicting medical procedures performed during a hospital admission.
Task: Explainable artificial intelligence
Categories: Methodology
Computer Vision

XAI refers to methods and techniques in the application of artificial intelligence (AI) such that the results of the solution can be understood by humans. It contrasts with the concept of the "black box" in machine learning where even its designers cannot explain why an AI arrived at a specific decision. XAI may be an implementation of the social right to explanation. XAI is relevant even if there is no legal right or regulatory requirement—for example, XAI can improve the user experience of a product or service by helping end users trust that the AI is making good decisions. This way the aim of XAI is to explain what has been done, what is done right now, what will be done next and unveil the information the actions are based on. These characteristics make it possible (i) to confirm existing knowledge (ii) to challenge existing knowledge and (iii) to generate new assumptions.
Task: Log Parsing
Categories: Computer Code

**Log Parsing** is the task of transforming unstructured log data into a structured format that can be used to train machine learning algorithms. The structured log data is then used to identify patterns, trends, and anomalies, which can support decision-making and improve system performance, security, and reliability. The log parsing process involves the extraction of relevant information from log files, the conversion of this information into a standardized format, and the storage of the structured data in a database or other data repository.
Task: Structual Feature Correlation
Categories: Graphs

Expressive Power of GNN to predict structural feature's correlation mutually.
Task: Taxonomy Learning
Categories: Natural Language Processing

Taxonomy learning is the task of hierarchically classifying concepts in an automatic manner from text corpora. The process of building taxonomies is usually divided into two main steps: (1) extracting hypernyms for concepts, which may constitute a field of research in itself (see Hypernym Discovery below) and (2) refining the structure into a taxonomy.

Description from [NLP Progress](http://nlpprogress.com/english/taxonomy_learning.html)
Task: Pulse wave simulation
Categories: Medical

Simulating arterial pulse waves
Task: Photoplethysmogram simulation
Categories: Medical

Simulating photoplethysmogram (PPG) signals
Task: Physiological Computing
Categories: Medical
Computer Vision

**Physiological computing** is an interdisciplinary field that focuses on the development of computational systems and technologies that interact with and respond to the physiological signals of the human body. These systems use sensors and algorithms to detect, analyze, and interpret physiological signals in real-time, allowing for a more natural and intuitive interaction between humans and computers.

*Main goal:*
To create intelligent systems that can adapt to the user's physiological state, enhancing user experience, performance, and well-being. This field draws on knowledge from various disciplines, including computer science, engineering, psychology, neuroscience, and human-computer interaction.

*Key components include:*
- Physiological Sensors: To capture physiological signals from the human body. Examples include electrocardiogram (ECG) sensors, electroencephalogram (EEG) sensors, electromyogram (EMG) sensors, and galvanic skin response (GSR) sensors.
- Signal Processing and Analysis: Physiological signals are processed and analyzed using computational techniques to extract meaningful information about the user's physiological state. This may involve filtering, feature extraction, pattern recognition, and machine learning algorithms.
- Adaptive Systems: Physiological computing systems use the information obtained from physiological signals to adapt their behavior in real-time. For example, a computer interface may adjust its presentation based on the user's level of attention, stress, or cognitive workload.

*Applications:*
Physiological computing has applications in various domains, including healthcare, education, entertainment, gaming, virtual reality, and human-computer interaction. For example, physiological computing technologies can be used to develop biofeedback systems for stress management, adaptive learning environments, and immersive gaming experiences.
Task: Semi-Supervised Object Detection
Categories: Computer Vision

Semi-supervised object detection uses both labeled data and unlabeled data for training. It not only reduces the annotation burden for training high-performance object detectors but also further improves the object detector by using a large number of unlabeled data.
Task: Twitter Sentiment Analysis
Categories: Natural Language Processing

Twitter sentiment analysis is the task of performing sentiment analysis on tweets from Twitter.
Task: Sign Language Production
Categories: Natural Language Processing

Sign Language Production (SLP) is the automatically translation from spoken language sentences into sign language sequences. Whilst Sign language Translation translates from sign to text, SLP is the opposite task from text to sign.
Task: Diabetic Retinopathy Grading
Categories: Medical

Grading the severity of diabetic retinopathy from (ophthalmic) fundus images