3497hyy/evaluation-tables huggingface dataset:
Task: Unsupervised Domain Adaptation

**Unsupervised Domain Adaptation** is a learning framework to transfer knowledge learned from source domains with a large number of annotated training examples to target domains with unlabeled data only.

Source: [Domain-Specific Batch Normalization for Unsupervised Domain Adaptation ](https://arxiv.org/abs/1906.03950)
Task: Operator learning
Categories: Miscellaneous

Learn an operator between infinite dimensional Hilbert spaces or Banach spaces
Task: Video Frame Interpolation
Categories: Computer Vision

The goal of **Video Frame Interpolation** is to synthesize several frames in the middle of two adjacent frames of the original video. Video Frame Interpolation can be applied to generate slow motion video, increase video frame rate, and frame recovery in video streaming.

Source: [Reducing the X-ray radiation exposure frequency in cardio-angiography via deep-learning based video interpolation ](https://arxiv.org/abs/2006.00781)
Task: Word Similarity
Categories: Natural Language Processing

Calculate a numerical score for the semantic similarity between two words.
Task: Hand Detection
Categories: Computer Vision

As an important subject in the field of computer vision, hand detection plays an important role in many tasks such as human-computer interaction, automatic driving, virtual reality and so on.
Task: Action Segmentation
Categories: Computer Vision

**Action Segmentation** is a challenging problem in high-level video understanding. In its simplest form, Action Segmentation aims to segment a temporally untrimmed video by time and label each segmented part with one of pre-defined action labels. The results of Action Segmentation can be further used as input to various applications, such as video-to-text and action localization.

Source: [TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentation ](https://arxiv.org/abs/1705.07818)
Task: Counterfactual Explanation
Categories: Miscellaneous

Returns a contrastive argument that permits to achieve the desired class, e.g., “to obtain this loan, you need XXX of annual
revenue instead of the current YYY”
Task: Image Denoising
Categories: Medical
Computer Vision

**Image Denoising** is a computer vision task that involves removing noise from an image. Noise can be introduced into an image during acquisition or processing, and can reduce image quality and make it difficult to interpret. Image denoising techniques aim to restore an image to its original quality by reducing or removing the noise, while preserving the important features of the image.

( Image credit: [Wide Inference Network for Image Denoising via
Learning Pixel-distribution Prior](https://arxiv.org/pdf/1707.05414v5.pdf) )
Task: Service Composition
Categories: Miscellaneous

Let T be the task that the service composition needs to accomplish. The task T can be granulated to T 1 , T 2 , T 3 , T 4 , … , T n . i.e. T =
{T 1 , T 2 , T 3 , T 4 , … , T n } . For each task T i , a set of service S i = S i 1 , S i 2 , S i 3 , … , S i m is discovered during the service discovery process such that all
services in a set S i perform the same function and have the same input and output parameters (See Figure 2). S 1 = {S 11 , S 12 , S 13 , … , S 1m } , S 2 =
{S 21 , S 22 , S 23 , … , S 2m } , S 3 = {S 31 , S 32 , S 33 , … , S 3m } , … , S n = {S n 1 , S n 2 , S n 3 , … , S n m }
We need to select one service from each set S i in order to compose the big service such that the overall QoS attributes of the big service
are optimal. The total number of the possible distinct service composition is n m . Let k be the the number of QoS attributes. Then the total num-
ber of comparisons required are kn m . We need at least kn m comparisons to find whether the solution is optimal, thus making the problem as
NP-Hard.
Task: Extreme Summarization
Categories: Natural Language Processing

Image credit: [TLDR: Extreme Summarization of Scientific Documents](https://arxiv.org/pdf/2004.15011v3.pdf)
Task: Anxiety Detection
Categories: Medical

Detect anxiety distress of human beings / animals
Task: Control with Prametrised Actions
Categories: Playing Games

Most reinforcement learning research papers focus on environments where the agent’s actions are either discrete or continuous. However, when training an agent to play a video game, it is common to encounter situations where actions have both discrete and continuous components. For example, a set of high-level discrete actions (ex: move, jump, fire), each of them being associated with continuous parameters (ex: target coordinates for the move action, direction for the jump action, aiming angle for the fire action). These kinds of tasks are included in Control with Parameterised Actions.
Task: 3D Character Animation From A Single Photo
Categories: Computer Vision

Image: [Weng et al](https://arxiv.org/pdf/1812.02246v1.pdf)
Task: Sound Event Localization and Detection
Categories: Audio

Given multichannel audio input, a sound event detection and localization (SELD) system outputs a temporal activation track for each of the target sound classes, along with one or more corresponding spatial trajectories when the track indicates activity. This results in a spatio-temporal characterization of the acoustic scene that can be used in a wide range of machine cognition tasks, such as inference on the type of environment, self-localization, navigation without visual input or with occluded targets, tracking of specific types of sound sources, smart-home applications, scene visualization systems, and audio surveillance, among others.
Task: Retrieval
Categories: Methodology
Natural Language Processing
Computer Vision

A methodology that involves selecting relevant data or examples from a large dataset to support tasks like prediction, learning, or inference. It enhances models by providing context or additional information, often used in systems like retrieval-augmented generation or in-context learning.
Task: Affordance Detection
Categories: Computer Vision

Affordance detection refers to identifying the potential action possibilities of objects in an image, which is an important ability for robot perception and manipulation.

Image source: [Object-Based Affordances Detection with Convolutional Neural Networks and Dense Conditional Random Fields](https://dkanou.github.io/publ/P15__Nguyen_Kanoulas_Caldwell_Tsagarakis__2017__Object-Based_Affordances_Detection_with_Convolutional_Neural_Networks_and__Dense_Conditional_Random_Fields.pdf)

Unlike other visual or physical properties that mainly describe the object alone, affordances indicate functional interactions of object parts with humans.
Task: Formation Energy
Categories: Miscellaneous

On the QM9 dataset the numbers reported in the table are the mean absolute error in eV on the target variable U0 divided by U0's chemical accuracy, which is equal to 0.043.
Task: KG-to-Text Generation
Categories: Natural Language Processing

Knowledge-graph-to-text (KG-to-text) generation aims to generate high-quality texts which are consistent with input graphs.

Description from: [JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs](https://arxiv.org/pdf/2106.10502v1.pdf)
Task: Learning Theory
Categories: Miscellaneous

Learning theory
Task: Multimodal Text Prediction
Categories: Natural Language Processing

**Multimodal text prediction** is a type of natural language processing that involves predicting the next word or sequence of words in a sentence, given multiple modalities or types of input. In traditional text prediction, the prediction is based solely on the context of the sentence, such as the words that precede the target word. In multimodal text prediction, additional modalities, such as images, audio, or user behavior, are also used to inform the prediction.

For example, in a multimodal text prediction system for captioning images, the system may use both the content of the image and the words that have been typed so far to generate the next word in the caption. The image may provide additional context or information about the content of the caption, while the typed words may provide information about the style or tone of the caption.

Multimodal text prediction can be achieved using a variety of techniques, including deep learning models and statistical models. These models can be trained on large datasets of text and multimodal inputs to learn the relationships between the different types of data and improve the accuracy of the predictions.

Multimodal text prediction has many applications, including chatbots, virtual assistants, and predictive text input for mobile devices. By incorporating additional modalities into the prediction process, multimodal text prediction systems can provide more accurate and useful predictions, improving the overall user experience.
Task: Circulatory Failure
Categories: Medical

Continuous prediction of onset of circulatory failure in the next 12h, given the patient is not in failure now.
Task: Kidney Function
Categories: Medical

Continuous prediction of urine production in the next 2h as an average rate in ml/kg/h. The task is predicted at irregular intervals.
Task: Remaining Useful Lifetime Estimation
Categories: Time Series

Estimating the number of machine operation cycles until breakdown from the time series of previous cycles.
Task: Visual Grounding
Categories: Computer Vision

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in
VG:

* What is the main focus in a query?
* How to understand an image?
* How to locate an object?
Task: Point Cloud Segmentation
Categories: Computer Vision

3D point cloud segmentation is the process of
classifying point clouds into multiple homogeneous regions, the
points in the same region will have the same properties. The
segmentation is challenging because of high redundancy, uneven
sampling density, and lack explicit structure of point cloud
data. This problem has many applications in robotics such as
intelligent vehicles, autonomous mapping and navigation.

Source: [3D point cloud segmentation: A survey](https://doi.org/10.1109/RAM.2013.6758588)
Task: Few-Shot Text Classification
Categories: Natural Language Processing

Few-shot Text Classification predicts the semantic label of a given text with a handful of supporting instances [1](https://aclanthology.org/2022.emnlp-main.87)
Task: Radar odometry
Categories: Robots

Radar odometry is the task of estimating the trajectory of the radar sensor, e.g. as presented in https://arxiv.org/abs/2105.01457.
A well established performance metric was presented by Geiger (2012) - "Are we ready for autonomous driving? the KITTI vision benchmark suite"
Task: Riemannian optimization

Optimization methods on Riemannian manifolds.
Task: Role-filler Entity Extraction
Categories: Natural Language Processing

Role-filler entity extraction task on the MUC-4 dataset.
Task: Temporal Information Extraction
Categories: Natural Language Processing

Temporal information extraction is the identification of chunks/tokens corresponding to temporal intervals, and the extraction and determination of the temporal relations between those. The entities extracted may be temporal expressions (timexes), eventualities (events), or auxiliary signals that support the interpretation of an entity or relation. Relations may be temporal links (tlinks), describing the order of events and times, or subordinate links (slinks) describing modality and other subordinative activity, or aspectual links (alinks) around the various influences aspectuality has on event structure.

The markup scheme used for temporal information extraction is well-described in the ISO-TimeML standard, and also on [www.timeml.org](http://www.timeml.org).

```

PRI20001020.2000.0127
NEWS STORY
10/20/2000 20:02:07.85

The Navy has changed its account of the attack on the USS Cole in Yemen.
Officials now say the ship was hit nearly two hours after it had docked.
Initially the Navy said the explosion occurred while several boats were helping
the ship to tie up. The change raises new questions about how the attackers
were able to get past the Navy security.

10/20/2000 20:02:28.05

```

To avoid leaking knowledge about temporal structure, train, dev and test splits must be made at document level for temporal information extraction.
Task: Knowledge Distillation
Categories: Natural Language Processing
Computer Vision

Knowledge distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models) have higher knowledge capacity than small models, this capacity might not be fully utilized.
Task: Audio Signal Processing
Categories: Audio

This is a general task that covers transforming audio inputs into audio outputs, not limited to existing PaperWithCode categories of Source Separation, Denoising, Classification, Recognition, etc.
Task: Table Extraction
Categories: Miscellaneous

Table extraction involves detecting and recognizing a table's logical structure and content from its unstructured presentation within a document
Task: Czech Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Czech Wikipedia texts.
Task: Model Discovery
Categories: Miscellaneous

discovering PDEs from spatiotemporal data
Task: Spike Sorting

Spike sorting is a class of techniques used in the analysis of electrophysiological data. Spike sorting algorithms use the shape(s) of waveforms collected with one or more electrodes in the brain to distinguish the activity of one or more neurons from background electrical noise.
Task: Crowdsourced Text Aggregation
Categories: Natural Language Processing

One of the most important parts of processing responses from crowd workers is **aggregation**: given several conflicting opinions, a method should extract the truth. This problem is also known as *truth-inference* in crowdsourcing. Text aggregation problem is dedicated to extracting the correct information from crowd workers' responses for a crowdsourcing task where the output is a *text*: audio transcription, translation, character recognition, etc.
Task: Record linking
Categories: Natural Language Processing

The task of finding records in a data set that refer to the same entity across different data sources.

Record linking is also called *entity resolution* or *entity matching*. Further material about this task is collected at [entity resolution](https://paperswithcode.com/task/entity-resolution).
Task: Dialogue Generation
Categories: Natural Language Processing

Dialogue generation is the task of "understanding" natural language inputs - within natural language processing in order to produce output. The systems are usually intended for conversing with humans, for instance back and forth dialogue with a conversation agent like a chatbot. Some example benchmarks for this task (see others such as Natural Language Understanding) include FusedChat and Ubuntu DIalogue Corpus (UDC). Models can be evaluated via metrics such as BLEU, ROUGE, and METEOR albeit with challenges in terms of weak correlation with human judgement, that may be addressed by new ones like UnSupervised and Reference-free (USR) and Metric for automatic Unreferenced dialog evaluation (MaUde).
Task: Vehicle Speed Estimation
Categories: Computer Vision

Vehicle speed estimation is the task of detecting and tracking vehicles whose real-world speeds are then estimated. The task is usually evaluated with recall and precision of the detected vehicle tracks as well as the mean or median errors of the estimated vehicle speeds.
Task: Multiple Object Tracking
Categories: Computer Vision

**Multiple Object Tracking** is the problem of automatically identifying multiple objects in a video and representing them as a set of trajectories with high accuracy.

Source: [SOT for MOT ](https://arxiv.org/abs/1712.01059)
Task: parameter-efficient fine-tuning
Categories: Methodology

Parameter-Efficient Fine-Tuning (PEFT) is a technique used to adapt pre-trained models to new tasks with minimal changes to the model's parameters. This approach is particularly useful in scenarios where computational resources are limited or when it is desirable to maintain the original model's performance on the initial task.
Task: Algorithmic Trading
Categories: Time Series

An algorithmic trading system is a software that is used for trading in the stock market.
Task: Emotion Classification
Categories: Natural Language Processing
Computer Vision

Emotion classification, or emotion categorization, is the task of recognising emotions to classify them into the corresponding category. Given an input, classify it as 'neutral or no emotion' or as one, or more, of several given emotions that best represent the mental state of the subject's facial expression, words, and so on. Some example benchmarks include ROCStories, Many Faces of Anger (MFA), and GoEmotions. Models can be evaluated using metrics such as the Concordance Correlation Coefficient (CCC) and the Mean Squared Error (MSE).
Task: Moment Retrieval
Categories: Computer Vision

Moment retrieval can de defined as the task of "localizing moments in a video given a user query".

Description from: [QVHIGHLIGHTS: Detecting Moments and Highlights in Videos via Natural Language Queries](https://arxiv.org/pdf/2107.09609v1.pdf)

Image credit: [QVHIGHLIGHTS: Detecting Moments and Highlights in Videos via Natural Language Queries](https://arxiv.org/pdf/2107.09609v1.pdf)
Task: Parking Space Occupancy
Categories: Computer Vision

Image credit: [https://github.com/martin-marek/parking-space-occupancy](https://github.com/martin-marek/parking-space-occupancy)
Task: Image Similarity Detection
Categories: Computer Vision

A fundamental computer vision task to determine whether a part of an image has been copied from another image.

Description from: [The 2021 Image Similarity Dataset and Challenge](https://paperswithcode.com/paper/the-2021-image-similarity-dataset-and)

Image credit: [The 2021 Image Similarity Dataset and Challenge](https://paperswithcode.com/paper/the-2021-image-similarity-dataset-and)
Task: EditCompletion
Categories: Computer Code

Given a code snippet that is partially edited, the goal is to predict a completion of the edit for the rest of the snippet.
Task: Referring expression generation
Categories: Computer Vision

Generate referring expressions
Task: Respiratory Failure
Categories: Medical

Continuous prediction of onset of respiratory failure in the next 12h given the patient is not in failure now.
Task: Dynamic Community Detection
Categories: Graphs

community detection in dynamic networks
Task: Density Ratio Estimation
Categories: Methodology

Estimating the ratio of one density function to the other.
Task: Multi-Person Pose Estimation
Categories: Computer Vision

Multi-person pose estimation is the task of estimating the pose of multiple people in one frame.

( Image credit: [Human Pose Estimation with TensorFlow
](https://github.com/eldar/pose-tensorflow) )
Task: Approximating Betweenness-Centrality ranking
Categories: Graphs

Betweenness-centrality is a popular measure in network analysis that aims to describe the importance of nodes in a graph. It accounts for the fraction of shortest paths passing through that node and is a key measure in many applications including community detection and network dismantling.
Task: Point Cloud Quality Assessment

### Background
A large and dense collection of points in three-dimensional space, collected by sensors such as LiDAR, is known as a point cloud. Points in the point cloud consist of geometric properties, such as three-dimensional spatial coordinates (x, y, z), and other attributes like color, reflectance, opacity, etc., represented by feature vectors. Since point clouds can directly represent the 3D world, they are widely employed in various fields, such as photogrammetry, power monitoring, architectural surveying, digital manufacturing, autonomous driving, gaming, cultural heritage
reservation, and more.
### Significance
Interactive point clouds typically contain millions of colored points and may possess complex attributes. To address the substantial transmission bandwidth and storage space required by point clouds,
esearchers have developed various point cloud compression (PCC) techniques. However, point cloud compression may introduce significant visual distortions. In addition, deformations and distortions frequently occur during the acquisition, processing, transmission, rendering, and interaction of point clouds, all of which degrade the visual quality of the point cloud, ultimately impacting the application’s user experience. Therefore, effective methods
for quantifying the quality of compressed point clouds are needed. More generally, point cloud quality assessment (PCQA) is crucial for optimizing and evaluating point cloud processing algorithms, such as encoding, denoising, and super-resolution.
Task: 3D Bin Packing
Categories: Miscellaneous

As a classic NP-hard problem, the bin packing problem (1D-BPP) seeks for an assignment of a collection of items with various weights to bins. The optimal assignment houses all the items with the fewest bins such that the total weight of items in a bin is below the bin’s capacity. In its 3D version (3D-BPP), an item has a 3D “weight” corresponding
to its length, width and height.
Task: 3D Surface Generation
Categories: Computer Vision

Image: [AtlasNet](https://arxiv.org/pdf/1802.05384v3.pdf)
Task: Vietnamese Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Vietnamese Wikipedia texts.
Task: Romanian Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Romanian Wikipedia texts.
Task: Slovak Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Slovak Wikipedia texts.
Task: Latvian Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Latvian Wikipedia texts.
Task: Polish Text Diacritization

Addition of diacritics for undiacritized Polish Wikipedia texts.
Task: Irish Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Irish Wikipedia texts.
Task: Hungarian Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Hungarian Wikipedia texts.
Task: French Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized French Wikipedia texts.
Task: Turkish Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Turkish Wikipedia texts.
Task: Spanish Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Spanish Wikipedia texts.
Task: Croatian Text Diacritization
Categories: Natural Language Processing

Addition of diacritics for undiacritized Croatian Wikipedia texts.
Task: Pedestrian Detection
Categories: Computer Vision

Pedestrian detection is the task of detecting pedestrians from a camera.

Further state-of-the-art results (e.g. on the KITTI dataset) can be found at [3D Object Detection](https://paperswithcode.com/task/object-detection).

( Image credit: [High-level Semantic Feature Detection: A New Perspective for Pedestrian Detection](https://github.com/liuwei16/CSP) )
Task: Hyperedge Prediction
Categories: Graphs

Hyperlink/hyperedge prediction, targets to find missing hyperedges in a hypergraph.
Task: Audio inpainting
Categories: Audio

Filling in holes in audio data
Task: Literature Mining
Categories: Knowledge Base
Natural Language Processing

The task where the publication texts are used to mine knowledge using NLP
Task: Ancestor-descendant prediction
Categories: Graphs

Given two entities, make a binary prediction if they have ancestor-descendant relationship, based on existing and missing hierarchical edges in the graph.