J0nasW/paperswithcode huggingface dataset:
Task: Generalized Referring Expression Segmentation
Area: computer-vision

Generalized Referring Expression Segmentation (GRES), introduced by [Liu et al in CVPR 2023](https://henghuiding.github.io/GRES/), allows expressions indicating any number of target objects. GRES takes an image and a referring expression as input, and requires mask prediction of the target object(s).
Task: Disparity Estimation
Area: computer-vision

The Disparity Estimation is the task of finding the pixels in the multiscopic views that correspond to the same 3D point in the scene.
Task: Repetitive Action Counting
Area: computer-vision

Repetitive action counting aims to count the number of repetitive actions in a video.
Task: Hybrid Positioning
Area: computer-vision

Hybrid Positioning using CV and dead reckoning
Task: Indoor Localization
Area: computer-vision

Indoor localization is a fundamental problem in indoor location-based applications.
Task: Face Sketch Synthesis
Area: computer-vision

Face sketch synthesis is the task of generating a sketch from an input face photo.

<span style="color:grey; opacity: 0.6">( Image credit: [High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks](https://arxiv.org/pdf/1710.10182v2.pdf) )</span>
Task: 3D Scene Reconstruction
Area: computer-vision

Creating 3D scene either using conventional SFM pipelines or latest deep learning approaches.
Task: Lipreading
Area: computer-vision

Lipreading is a process of extracting speech by watching lip movements of a speaker in the absence of sound. Humans lipread all the time without even noticing. It is a big part in communication albeit not as dominant as audio. It is a very helpful skill to learn especially for those who are hard of hearing. 

Deep Lipreading is the process of extracting speech from a video of a silent talking face using deep neural networks.  It is also known by few other names: Visual Speech Recognition (VSR), Machine Lipreading, Automatic Lipreading etc. 

The primary methodology involves two stages: i) Extracting visual and temporal features from a sequence of image frames from a silent talking video ii) Processing the sequence of features into units of speech e.g. characters, words, phrases etc. We can find several implementations of this methodology either done in two separate stages or trained end-to-end in one go.
Task: 3D Point Cloud Matching
Area: computer-vision

Image: [Gojic et al](https://openaccess.thecvf.com/content_CVPR_2019/papers/Gojcic_The_Perfect_Match_3D_Point_Cloud_Matching_With_Smoothed_Densities_CVPR_2019_paper.pdf)
Task: HD semantic map learning
Area: computer-vision

The goal of task is to generate map elements in a vectorized form using data from onboard sensors, e.g., RGB cameras and/or LiDARs. These map elements include but are not limited to : Road boundaries, boundaries of roads that split roads and sidewalks.
Task: 3D Volumetric Reconstruction
Area: computer-vision

Image: [Grinvald et al](https://arxiv.org/pdf/1903.00268.pdf)
Task: Face Swapping
Area: computer-vision

Face swapping refers to the task of swapping faces between images or in an video, while maintaining the rest of the body and environment context.

<span style="color:grey; opacity: 0.6">( Image credit: [Swapped Face Detection using Deep Learning and Subjective Assessment](https://arxiv.org/pdf/1909.04217v1.pdf) )</span>
Task: Change Detection
Area: computer-vision

**Change Detection** is a computer vision task that involves detecting changes in an image or video sequence over time. The goal is to identify areas in the image or video that have undergone changes, such as appearance changes, object disappearance or appearance, or even changes in the scene's background.

Image credit: ["A TRANSFORMER-BASED SIAMESE NETWORK FOR CHANGE DETECTION"](https://arxiv.org/pdf/2201.01293v1.pdf)
Task: Gesture Recognition
Area: computer-vision

**Gesture Recognition** is an active field of research with applications such as automatic recognition of sign language, interaction of humans and robots or for new ways of controlling video games.


<span class="description-source">Source: [Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping ](https://arxiv.org/abs/1906.12171)</span>
Task: Aesthetics Quality Assessment
Area: computer-vision

Automatic assessment of aesthetic-related subjective ratings.
Task: Metric Learning
Area: computer-vision

The goal of **Metric Learning** is to learn a representation function that maps objects into an embedded space. The distance in the embedded space should preserve the objects’ similarity — similar objects get close and dissimilar objects get far away. Various loss functions have been developed for Metric Learning. For example, the **contrastive loss** guides the objects from the same class to be mapped to the same point and those from different classes to be mapped to different points whose distances are larger than a margin. **Triplet loss** is also popular, which requires the distance between the anchor sample and the positive sample to be smaller than the distance between the anchor sample and the negative sample.


<span class="description-source">Source: [Road Network Metric Learning for Estimated Time of Arrival ](https://arxiv.org/abs/2006.13477)</span>
Task: Handwriting Recognition
Area: computer-vision

Image source: [Handwriting Recognition of Historical Documents with few labeled data](https://arxiv.org/pdf/1811.07768v1.pdf)
Task: Hurricane Forecasting
Area: computer-vision

Tropical Cyclone Forecasting using Computer Vision, Deep Learning, and Time-Series methods
Task: Saliency Detection
Area: computer-vision

**Saliency Detection** is a preprocessing step in computer vision which aims at finding salient objects in an image.


<span class="description-source">Source: [An Unsupervised Game-Theoretic Approach to Saliency Detection ](https://arxiv.org/abs/1708.02476)</span>
Task: Dense Video Captioning
Area: computer-vision

Most natural videos contain numerous events. For example, in a video of a “man playing a piano”, the video might also contain “another man dancing” or “a crowd clapping”. The task of dense video captioning involves both detecting and describing events in a video.
Task: cross-domain few-shot learning
Area: computer-vision

Its essence is transfer learning. The model needs to be trained in the source domain and then migrated to the target domain. Compliant with (1) the category in the target domain has never appeared in the source domain (2) the data distribution of the target domain is inconsistent with the source domain (3) each class in the target domain has very few labels
Task: Age And Gender Classification
Area: computer-vision

Age and gender classification is a dual-task of identifying the age and gender of a person from an image or video.

<span style="color:grey; opacity: 0.6">( Image credit: [Multi-Expert Gender Classification on Age Group by Integrating Deep Neural Networks](https://arxiv.org/pdf/1809.01990v2.pdf) )</span>
Task: Drone navigation
Area: computer-vision

(Satellite -> Drone) Given one satellite-view image, the drone intends to find the most relevant place (drone-view images) that it has passed by. According to its flight history, the drone could be navigated back to the target place.
Task: Layout-to-Image Generation
Area: computer-vision

Layout-to-image generation its the task to generate a scene based on the given layout. The layout describes the location of the objects to be included in the output image.
In this section, you can find state-of-the-art leaderboards for Layout-to-image generation.
Task: Image Stitching
Area: computer-vision

**Image Stitching** is a process of composing multiple images with narrow but overlapping fields of view to create a larger image with a wider field of view.


<span class="description-source">Source: [Single-Perspective Warps in Natural Image Stitching ](https://arxiv.org/abs/1802.04645)</span>

( Image credit: [Kornia](https://github.com/kornia/kornia) )
Task: Optical Flow Estimation
Area: computer-vision

**Optical Flow Estimation** is a computer vision task that involves computing the motion of objects in an image or a video sequence. The goal of optical flow estimation is to determine the movement of pixels or features in the image, which can be used for various applications such as object tracking, motion analysis, and video compression.

Approaches for optical flow estimation include correlation-based, block-matching, feature tracking, energy-based, and more recently gradient-based.

Further readings:

- [Optical Flow Estimation](https://www.cs.toronto.edu/~fleet/research/Papers/flowChapter05.pdf)
- [Performance of Optical Flow Techniques](https://www.cs.toronto.edu/~fleet/research/Papers/ijcv-94.pdf)

Definition source: [Devon: Deformable Volume Network for Learning Optical Flow ](https://arxiv.org/abs/1802.07351)

Image credit: [Optical Flow Estimation](https://www.cs.toronto.edu/~fleet/research/Papers/flowChapter05.pdf)
Task: Image Clustering
Area: computer-vision

Models that partition the dataset into semantically meaningful clusters without having access to the ground truth labels. 

<span style="color:grey; opacity: 0.6"> Image credit: ImageNet clustering results of [SCAN: Learning to Classify Images without Labels (ECCV 2020)](https://arxiv.org/abs/2005.12320) </span>
Task: Image Classification
Area: computer-vision

**Image Classification** is a fundamental task that attempts to comprehend an entire image as a whole. The goal is to classify the image by assigning it to a specific label. Typically, Image Classification refers to images in which only one object appears and is analyzed. In contrast, object detection involves both classification and localization tasks, and is used to analyze more realistic cases in which multiple objects may exist in an image.


<span class="description-source">Source: [Metamorphic Testing for Object Detection Systems ](https://arxiv.org/abs/1912.12162)</span>
Task: Referring Image Matting
Area: computer-vision

Extracting the meticulous alpha matte of the specific object from the image that can best match the given natural language description, e.g., a keyword or a expression.
Task: Moment Retrieval
Area: computer-vision

Moment retrieval can de defined as the task of "localizing moments in a video given a user query".

Description from: [QVHIGHLIGHTS: Detecting Moments and Highlights in Videos via Natural Language Queries](https://arxiv.org/pdf/2107.09609v1.pdf)

Image credit: [QVHIGHLIGHTS: Detecting Moments and Highlights in Videos via Natural Language Queries](https://arxiv.org/pdf/2107.09609v1.pdf)
Task: 3D Depth Estimation
Area: computer-vision

Image: [monodepth2](https://github.com/nianticlabs/monodepth2)
Task: Reference-based Video Super-Resolution
Area: computer-vision

Reference-based video super-resolution (RefVSR) is an expansion of reference-based super-resolution (RefSR) to the video super-resolution (VSR). RefVSR inherits the objectives of both RefSR and VSR tasks and utilizes a Ref video for reconstructing an HR video from an LR video
video from an LR video.
Task: Multi class one-shot image synthesis
Area: computer-vision

The goal of Multi-class one-shot image synthesis is to learn a generative model that can generate samples with visual attributes from as few as one or more images of at least 2 related classes.
Task: Image Enhancement
Area: computer-vision

**Image Enhancement** is basically improving the interpretability or perception of information in images for human viewers and providing ‘better’ input for other automated image processing techniques. The principal objective of Image Enhancement is to modify attributes of an image to make it more suitable for a given task and a specific observer.


<span class="description-source">Source: [A Comprehensive Review of Image Enhancement Techniques ](https://arxiv.org/abs/1003.4053)</span>
Task: Video-Text Retrieval
Area: computer-vision

Video-Text retrieval requires understanding of both video and language together. Therefore it's different to video retrieval task.
Task: Video Description
Area: computer-vision

The goal of automatic **Video Description** is to tell a story about events happening in a video. While early Video Description methods produced captions for short clips that were manually segmented to contain a single event of interest, more recently dense video captioning has been proposed to both segment distinct events in time and describe them in a series of coherent sentences. This problem is a generalization of dense image region captioning and has many practical applications, such as generating textual summaries for the visually impaired, or detecting and describing important events in surveillance footage.


<span class="description-source">Source: [Joint Event Detection and Description in Continuous Video Streams ](https://arxiv.org/abs/1802.10250)</span>
Task: Partial Video Copy Detection
Area: computer-vision

The PVCD goal is identifying and locating if one or more segments of a long testing video have been copied (transformed) from the reference videos dataset.
Task: Class-agnostic Object Detection
Area: computer-vision

Class-agnostic object detection aims to localize objects in images without specifying their categories.
Task: Scene Classification
Area: computer-vision

**Scene Classification** is a task in which scenes from photographs are categorically classified. Unlike object classification, which focuses on classifying prominent objects in the foreground, Scene Classification uses the layout of objects within the scene, in addition to the ambient context, for classification.


<span class="description-source">Source: [Scene classification with Convolutional Neural Networks ](http://cs231n.stanford.edu/reports/2017/pdfs/102.pdf)</span>
Task: One-Shot Segmentation
Area: computer-vision

<span style="color:grey; opacity: 0.6">( Image credit: [One-Shot Learning for Semantic
Segmentation](https://arxiv.org/pdf/1709.03410v1.pdf) )</span>
Task: Human Instance Segmentation
Area: computer-vision

Instance segmentation is the task of detecting and delineating each distinct object of interest appearing in an image.

Image Credit: [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://arxiv.org/abs/2103.12340)
Task: Generalized Zero-Shot Learning - Unseen
Area: computer-vision

The average of the normalized top-1 prediction scores of unseen classes in the generalized zero-shot learning setting, where the label of a test sample is predicted among all (seen + unseen) classes.
Task: Head Pose Estimation
Area: computer-vision

Estimating the head pose of a person is a crucial problem that has a large amount of applications such as aiding in gaze estimation, modeling attention, fitting 3D models to video and performing face alignment.

<span style="color:grey; opacity: 0.6">( Image credit: [FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose
Estimation from a Single Image](http://openaccess.thecvf.com/content_CVPR_2019/papers/Yang_FSA-Net_Learning_Fine-Grained_Structure_Aggregation_for_Head_Pose_Estimation_From_CVPR_2019_paper.pdf) )</span>
Task: 3D Object Super-Resolution
Area: computer-vision

3D object super-resolution is the task of up-sampling 3D objects.

<span style="color:grey; opacity: 0.6">( Image credit: [Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation](https://github.com/EdwardSmith1884/Multi-View-Silhouette-and-Depth-Decomposition-for-High-Resolution-3D-Object-Representation) )</span>
Task: Few-Shot Point Cloud Classification
Area: computer-vision

Few-Shot Learning on point cloud classification task
Task: Referring Expression Segmentation
Area: computer-vision

The task aims at labeling the pixels of an image or video that represent an object instance referred by a linguistic expression. In particular, the referring expression (RE) must allow the identification of an individual object in a discourse or scene (the referent). REs unambiguously identify the target instance.
Task: User Constrained Thumbnail Generation
Area: computer-vision

Thumbnail generation is the task of generating image thumbnails from an input image.

<span style="color:grey; opacity: 0.6">( Image credit: [User Constrained Thumbnail Generation using Adaptive Convolutions](https://arxiv.org/pdf/1810.13054v3.pdf) )</span>
Task: Unsupervised Image-To-Image Translation
Area: computer-vision

Unsupervised image-to-image translation is the task of doing image-to-image translation without ground truth image-to-image pairings.

<span style="color:grey; opacity: 0.6">( Image credit: [Unpaired Image-to-Image Translation
using Cycle-Consistent Adversarial Networks](https://arxiv.org/pdf/1703.10593v6.pdf) )</span>
Task: Video Object Detection
Area: computer-vision

Video object detection is the task of detecting objects from a video as opposed to images.

<span style="color:grey; opacity: 0.6">( Image credit: [Learning Motion Priors for Efficient Video Object Detection](https://arxiv.org/pdf/1911.05253v1.pdf) )</span>
Task: Gaze Estimation
Area: computer-vision

**Gaze Estimation** is a task to predict where a person is looking at given the person’s full face. The task contains two directions: 3-D gaze vector and 2-D gaze position estimation. 3-D gaze vector estimation is to predict the gaze vector, which is usually used in the automotive safety. 2-D gaze position estimation is to predict the horizontal and vertical coordinates on a 2-D screen, which allows utilizing gaze point to control a cursor for human-machine interaction.


<span class="description-source">Source: [A Generalized and Robust Method Towards Practical Gaze Estimation on Smart Phone ](https://arxiv.org/abs/1910.07331)</span>
Task: Action Classification
Area: computer-vision

Image source: [The Kinetics Human Action Video Dataset](https://arxiv.org/pdf/1705.06950.pdf)
Task: Flare Removal
Area: computer-vision

When a camera is pointed at a strong light source, the resulting photograph may contain lens flare artifacts. Flares appear in a wide variety of patterns (halos, streaks, color bleeding, haze, etc.) and this diversity in appearance makes flare removal challenging.
Task: Motion Magnification
Area: computer-vision

Motion magnification is a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, we need to accurately measure visual motions, and group the pixels to be modified.

There are different approaches to motion magnification, such as Lagrangian and Eulerian methods. Lagrangian methods track the trajectories of moving objects and exaggerate them, while Eulerian methods manipulate the motions at fixed positions. Eulerian methods can be further divided into linear and phase-based methods. Linear methods apply a temporal bandpass filter to boost the linear term of a Taylor series expansion of the displacement function, while phase-based methods use complex wavelet transforms to manipulate the phase of the signal.

Motion magnification has various applications, such as measuring the human pulse, visualizing the heat plume of candles, revealing the oscillations of a wine glass, and detecting structural defects.
Task: Color Constancy
Area: computer-vision

**Color Constancy** is the ability of the human vision system to perceive the colors of the objects in the scene largely invariant to the color of the light source. The task of computational Color Constancy is to estimate the scene illumination and then perform the chromatic adaptation in order to remove the influence of the illumination color on the colors of the objects in the scene.


<span class="description-source">Source: [CroP: Color Constancy Benchmark Dataset Generator ](https://arxiv.org/abs/1903.12581)</span>
Task: Visual Relationship Detection
Area: computer-vision

Visual relationship detection (VRD) is one newly developed computer vision task aiming to recognize relations or interactions between objects in an image. It is a further learning task after object recognition and is essential for fully understanding images, even the visual world.
Task: Unsupervised Facial Landmark Detection
Area: computer-vision

Facial landmark detection in the unsupervised setting popularized by [1].  The evaluation occurs in two stages:
(1) Embeddings are first learned in an unsupervised manner (i.e. without labels);
(2) A simple regressor is trained to regress landmarks from the unsupervised embedding.

[1]  Thewlis, James, Hakan Bilen, and Andrea Vedaldi. "Unsupervised learning of object landmarks by factorized spatial embeddings." Proceedings of the IEEE International Conference on Computer Vision. 2017.

<span style="color:grey; opacity: 0.6">( Image credit: [Unsupervised learning of object landmarks by factorized spatial embeddings](https://www.robots.ox.ac.uk/~vedaldi/assets/pubs/thewlis17unsupervised.pdf) )</span>
Task: Generative Visual Question Answering
Area: computer-vision

Generating answers in free form to questions posed about images.
Task: Supervised Anomaly Detection
Area: computer-vision

In the training set, the amount of abnormal samples is limited and significant fewer than normal samples, producing data distributions that lead to a naturally imbalanced learning problem.
Task: Personalized Segmentation
Area: computer-vision

Given a one-shot image with a reference mask, the models are required to segment the indicated target object in any other images.
Task: 3D Human Pose Estimation
Area: computer-vision

**3D Human Pose Estimation** is a computer vision task that involves estimating the 3D positions and orientations of body joints and bones from 2D images or videos. The goal is to reconstruct the 3D pose of a person in real-time, which can be used in a variety of applications, such as virtual reality, human-computer interaction, and motion analysis.
Task: Chat-based Image Retrieval
Area: computer-vision

ChatIR: a Chat-based Image Retrieval system that engages in a conversation with the user to elicit information, in addition to an initial query, in order to clarify the user’s search intent.
Task: 3D human pose and shape estimation
Area: computer-vision

Estimate 3D human pose and shape (e.g. SMPL) from images
Task: Conditional Text-to-Image Synthesis
Area: computer-vision

Introducing extra conditions based on the text-to-image generation process, similar to the paradigm of ControlNet.
Task: Human action generation
Area: computer-vision

Yan et al. (2019) CSGN:

"When the dancer is stepping, jumping and spinning on the
stage, attentions of all audiences are attracted by the streamof the fluent and graceful movements. Building a  model that is capable of dancing is as fascinating a task as appreciating the performance itself. In this paper, we aim to generate long-duration human actions represented as skeleton sequences, e.g. those that cover the entirety of a dance, with hundreds of moves and countless possible combinations."


<span style="color:grey; opacity: 0.6">( Image credit: [Convolutional Sequence Generation for Skeleton-Based Action Synthesis](http://www.dahualin.org/publications/dhl19_csgn.pdf) )</span>
Task: Unconstrained Lip-synchronization
Area: computer-vision

Given a video of an arbitrary person, and an arbitrary driving speech, the task is to generate a lip-synced video that matches the given speech. 

This task requires the approach to not be constrained by identity, voice, or language.
Task: Face Alignment
Area: computer-vision

Face alignment is the task of identifying the geometric structure of faces in digital images, and attempting to obtain a canonical alignment of the face based on translation, scale, and rotation.

<span style="color:grey; opacity: 0.6">( Image credit: [3DDFA_V2](https://github.com/cleardusk/3DDFA_V2) )</span>
Task: Semi-Supervised Object Detection
Area: computer-vision

Semi-supervised object detection uses both labeled data and unlabeled data for training. It not only reduces the annotation burden for training high-performance object detectors but also further improves the object detector by using a large number of unlabeled data.