J0nasW/paperswithcode huggingface dataset:
Task: Combinatorial Optimization
Area: methodology

**Combinatorial Optimization** is a category of problems which requires optimizing a function over a combination of discrete objects and the solutions are constrained. Examples include finding shortest paths in a graph, maximizing value in the Knapsack problem and finding boolean settings that satisfy a set of constraints. Many of these problems are NP-Hard, which means that no polynomial time solution can be developed for them. Instead, we can only produce approximations in polynomial time that are guaranteed to be some factor worse than the true optimal solution.


<span class="description-source">Source: [Recent Advances in Neural Program Synthesis ](https://arxiv.org/abs/1802.02353)</span>
Task: Influence Approximation
Area: methodology

Estimating the influence of training triples on the behavior of a machine learning model.
Task: Auxiliary Learning
Area: methodology

Auxiliary learning aims to find or design auxiliary tasks which can improve the performance on one or some primary tasks.

<span style="color:grey; opacity: 0.6">( Image credit: [Self-Supervised Generalisation with Meta Auxiliary Learning](https://arxiv.org/pdf/1901.08933v3.pdf) )</span>
Task: Quantum Circuit Equivalence Checking
Area: methodology

Equivalence Checking of Quantum Circuits
Task: RGB+3D Anomaly Detection and Segmentation
Area: methodology

RGB+3D Anomaly Detection and Segmentation
Task: Density Ratio Estimation
Area: methodology

Estimating the ratio of one density function to the other.
Task: Bayesian Inference
Area: methodology

Bayesian Inference is a methodology that employs Bayes Rule to estimate parameters (and their full posterior).
Task: Low-Rank Matrix Completion
Area: methodology

**Low-Rank Matrix Completion** is an important problem with several applications in areas such as recommendation systems, sketching, and quantum tomography. The goal in matrix completion is to recover a low rank matrix, given a small number of entries of the matrix.


<span class="description-source">Source: [Universal Matrix Completion ](https://arxiv.org/abs/1402.2324)</span>
Task: Contrastive Learning
Area: methodology

**Contrastive Learning** is a deep learning technique for unsupervised representation learning. The goal is to learn a representation of data such that similar instances are close together in the representation space, while dissimilar instances are far apart.

It has been shown to be effective in various computer vision and natural language processing tasks, including image retrieval, zero-shot learning, and cross-modal retrieval. In these tasks, the learned representations can be used as features for downstream tasks such as classification and clustering.

<span style="color:grey; opacity: 0.6">(Image credit: [Schroff et al. 2015](https://arxiv.org/abs/1503.03832))</span>
Task: Domain Generalization
Area: methodology

The idea of **Domain Generalization** is to learn from one or multiple training domains, to extract a domain-agnostic model which can be applied to an unseen domain


<span class="description-source">Source: [Diagram Image Retrieval using Sketch-Based Deep Learning and Transfer Learning ](https://arxiv.org/abs/2004.10780)</span>
Task: Thompson Sampling
Area: methodology

Thompson sampling, named after William R. Thompson, is a heuristic for choosing actions that addresses the exploration-exploitation dilemma in the multi-armed bandit problem. It consists of choosing the action that maximizes the expected reward with respect to a randomly drawn belief.
Task: Meta-Learning
Area: methodology

Meta-learning is a methodology considered with "learning to learn" machine learning algorithms.

<span style="color:grey; opacity: 0.6">( Image credit: [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://arxiv.org/pdf/1703.03400v3.pdf) )</span>
Task: Anomaly Detection
Area: methodology

**Anomaly Detection** is a binary classification identifying unusual or unexpected patterns in a dataset, which deviate significantly from the majority of the data. The goal of anomaly detection is to identify such anomalies, which could represent errors, fraud, or other types of unusual events, and flag them for further investigation.

[Image source]: [GAN-based Anomaly Detection in Imbalance Problems](https://paperswithcode.com/paper/gan-based-anomaly-detection-in-imbalance)
Task: Dimensionality Reduction
Area: methodology

Dimensionality reduction is the task of reducing the dimensionality of a dataset.

<span style="color:grey; opacity: 0.6">( Image credit: [openTSNE](https://github.com/pavlin-policar/openTSNE) )</span>
Task: Efficient Exploration
Area: methodology

**Efficient Exploration** is one of the main obstacles in scaling up modern deep reinforcement learning algorithms. The main challenge in Efficient Exploration is the balance between exploiting current estimates, and gaining information about poorly understood states and actions.


<span class="description-source">Source: [Randomized Value Functions via Multiplicative Normalizing Flows ](https://arxiv.org/abs/1806.02315)</span>
Task: Network Pruning
Area: methodology

**Network Pruning** is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.


<span class="description-source">Source: [Ensemble Knowledge Distillation for Learning Improved and Efficient Networks ](https://arxiv.org/abs/1909.08097)</span>
Task: Data Augmentation
Area: methodology

Data augmentation involves techniques used for increasing the amount of data, based on different modifications, to expand the amount of examples in the original dataset. Data augmentation not only helps to grow the dataset but it also increases the diversity of the dataset. When training machine learning models, data augmentation acts as a regularizer and helps to avoid overfitting. 

Data augmentation techniques have been found useful in domains like NLP and computer vision. In computer vision, transformations like cropping, flipping, and rotation are used. In NLP, data augmentation techniques can include swapping, deletion, random insertion, among others. 

Further readings:

- [A Survey of Data Augmentation Approaches for NLP](https://paperswithcode.com/paper/a-survey-of-data-augmentation-approaches-for)
- [A survey on Image Data Augmentation for Deep Learning](https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0197-0)

<span style="color:grey; opacity: 0.6">( Image credit: [Albumentations](https://github.com/albumentations-team/albumentations) )</span>
Task: Two-sample testing
Area: methodology

In statistical hypothesis testing, a two-sample test is a test performed on the data of two random samples, each independently obtained from a different given population. The purpose of the test is to determine whether the difference between these two populations is statistically significant. The statistics used in two-sample tests can be used to solve many machine learning problems, such as domain adaptation, covariate shift and generative adversarial networks.
Task: Quantization
Area: methodology

**Quantization** is a promising technique to reduce the computation cost of neural network training, which can replace high-cost floating-point numbers (e.g., float32) with low-cost fixed-point numbers (e.g., int8/int16).


<span class="description-source">Source: [Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers ](https://arxiv.org/abs/1911.00361)</span>
Task: Matrix Completion
Area: methodology

**Matrix Completion** is a method for recovering lost information. It originates from machine learning and usually deals with highly sparse matrices. Missing or unknown data is estimated using the low-rank matrix of the known data.


<span class="description-source">Source: [A Fast Matrix-Completion-Based Approach for Recommendation Systems ](https://arxiv.org/abs/1912.00600)</span>
Task: Representation Learning
Area: methodology

**Representation Learning** is concerned with training machine learning algorithms to learn useful representations, e.g. those that are interpretable, have latent features, or can be used for transfer learning.

Deep neural networks can be considered representation learning models that typically encode information which is projected into a different subspace. These representations are then usually passed on to a linear classifier to, for instance, train a classifier. 

Representation learning can be divided into:

-  **Supervised representation learning**: learning representations on task A using annotated data and used to solve task B
- **Unsupervised representation learning**: learning representations on a task in an unsupervised way (label-free data). These are then used to address downstream tasks and reducing the need for annotated data when learning news tasks. Powerful models like [GPT](/method/gpt) and [BERT](/method/bert) leverage unsupervised representation learning to tackle language tasks.  

More recently, [self-supervised learning (SSL)](/task/self-supervised-learning) is one of the main drivers behind unsupervised representation learning in fields like computer vision and NLP. 

Here are some additional readings to go deeper on the task:

- [Representation Learning: A Review and New Perspectives](/paper/representation-learning-a-review-and-new) - Bengio et al. (2012)
- [A Few Words on Representation Learning](https://sthalles.github.io/a-few-words-on-representation-learning/) - Thalles Silva

<span style="color:grey; opacity: 0.6">( Image credit: [Visualizing and Understanding Convolutional Networks](https://arxiv.org/pdf/1311.2901.pdf) )</span>
Task: Computed Tomography (CT)
Area: methodology

The term “computed tomography”, or CT, refers to a computerized x-ray imaging procedure in which a narrow beam of x-rays is aimed at a patient and quickly rotated around the body, producing signals that are processed by the machine's computer to generate cross-sectional images—or “slices”—of the body.

<span style="color:grey; opacity: 0.6">( Image credit: [Liver Lesion Detection from Weakly-labeled Multi-phase CT Volumes with a Grouped Single Shot MultiBox Detector](https://github.com/L0SG/grouped-ssd-pytorch) )</span>
Task: Generative Semantic Nursing
Area: methodology

**Generative Semantic Nursing** is a task of intervening the generative process on the fly during inference time to improve the faithfulness of the generated images. It works by carefully manipulating of latents during the denoising process of a pre-trained text-to-image diffusion model.

Source: [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://paperswithcode.com/paper/attend-and-excite-attention-based-semantic)
Task: Domain Adaptation
Area: methodology

**Domain Adaptation** is the task of adapting models across domains. This is motivated by the challenge where the test and training datasets fall from different data distributions due to some factor. Domain adaptation aims to build machine learning models that can be generalized into a target domain and dealing with the discrepancy across domain distributions. 

Further readings:

- [A Brief Review of Domain Adaptation](https://paperswithcode.com/paper/a-brief-review-of-domain-adaptation)

<span style="color:grey; opacity: 0.6">( Image credit: [Unsupervised Image-to-Image Translation Networks](https://arxiv.org/pdf/1703.00848v6.pdf) )</span>
Task: Multi-Label Text Classification
Area: methodology

According to Wikipedia "In machine learning, multi-label classification and the strongly related problem of multi-output classification are variants of the classification problem where multiple labels may be assigned to each instance. Multi-label classification is a generalization of multiclass classification, which is the single-label problem of categorizing instances into precisely one of more than two classes; in the multi-label problem there is no constraint on how many of the classes the instance can be assigned to."
Task: Active Learning
Area: methodology

**Active Learning** is a paradigm in supervised machine learning which uses fewer training examples to achieve better optimization by iteratively training a predictor, and using the predictor in each iteration to choose the training examples which will increase its chances of finding better configurations and at the same time improving the accuracy of the prediction model


<span class="description-source">Source: [Polystore++: Accelerated Polystore System for Heterogeneous Workloads ](https://arxiv.org/abs/1905.10336)</span>
Task: Network Embedding
Area: methodology

**Network Embedding**, also known as "Network Representation Learning", is a collective term for techniques for mapping graph nodes to vectors of real numbers in a multidimensional space. To be useful, a good embedding should preserve the structure of the graph. The vectors can then be used as input to various network and graph analysis tasks, such as link prediction


<span class="description-source">Source: [Tutorial on NLP-Inspired Network Embedding ](https://arxiv.org/abs/1910.07212)</span>
Task: Mutual Information Estimation
Area: methodology

To estimate mutual information from samples, specially for high-dimensional variables.
Task: Electroencephalogram (EEG)
Area: methodology

**Electroencephalogram (EEG)** is a method of recording brain activity using electrophysiological indexes. When the brain is active, a large number of postsynaptic potentials generated synchronously by neurons are formed after summation. It records the changes of electric waves during brain activity and is the overall reflection of the electrophysiological activities of brain nerve cells on the surface of cerebral cortex or scalp. Brain waves originate from the postsynaptic potential of the apical dendrites of pyramidal cells. The formation of synchronous rhythm of EEG is also related to the activity of nonspecific projection system of cortex and thalamus. EEG is the basic theoretical research of brain science. EEG monitoring is widely used in its clinical application.
Task: Structured Prediction
Area: methodology

**Structured Prediction** is an area of machine learning focusing on representations of spaces with combinatorial structure, and algorithms for inference and parameter estimation over these structures. Core methods include both tractable exact approaches like dynamic programming and spanning tree algorithms as well as heuristic techniques such as linear programming relaxations and greedy search.


<span class="description-source">Source: [Torch-Struct: Deep Structured Prediction Library ](https://arxiv.org/abs/2002.00876)</span>
Task: BIG-bench Machine Learning
Area: methodology

This branch include most common machine learning fundamental algorithms.
Task: 3D Anomaly Detection and Segmentation
Area: methodology

3D-Only Anomaly Detection and Segmentation
Task: Zero-Shot Learning
Area: methodology

**Zero-shot learning (ZSL)** is a model's ability to detect classes never seen during training. The condition is that the classes are not known during supervised learning. 

Earlier work in zero-shot learning use attributes in a two-step approach to infer unknown classes. In the computer vision context, more recent advances learn mappings from image feature space to semantic space. Other approaches learn non-linear multimodal embeddings. In the modern NLP context, language models can be evaluated on downstream tasks without fine tuning. 

Benchmark datasets for zero-shot learning include [aPY](/dataset/apy), [AwA](/dataset/awa2-1), and [CUB](/dataset/cub-200-2011), among others. 

( Image credit: [Prototypical Networks for Few shot Learning in PyTorch
](https://github.com/orobix/Prototypical-Networks-for-Few-shot-Learning-PyTorch) )

Further readings:  

- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://paperswithcode.com/paper/zero-shot-learning-a-comprehensive-evaluation)
- [Zero-Shot Learning in Modern NLP](https://joeddav.github.io/blog/2020/05/29/ZSL.html)
- [Zero-Shot Learning for Text Classification](https://amitness.com/2020/05/zero-shot-text-classification/)
Task: Metric Learning
Area: methodology

The goal of **Metric Learning** is to learn a representation function that maps objects into an embedded space. The distance in the embedded space should preserve the objects’ similarity — similar objects get close and dissimilar objects get far away. Various loss functions have been developed for Metric Learning. For example, the **contrastive loss** guides the objects from the same class to be mapped to the same point and those from different classes to be mapped to different points whose distances are larger than a margin. **Triplet loss** is also popular, which requires the distance between the anchor sample and the positive sample to be smaller than the distance between the anchor sample and the negative sample.


<span class="description-source">Source: [Road Network Metric Learning for Estimated Time of Arrival ](https://arxiv.org/abs/2006.13477)</span>
Task: 3D + RGB Anomaly Detection
Area: methodology

3D + RGB Anomaly Detection
Task: Probabilistic Programming
Area: methodology

Probabilistic programming languages are designed to describe probabilistic models and then perform inference in those models. PPLs are closely related to graphical models and Bayesian networks, but are more expressive and flexible.

<span style="color:grey; opacity: 0.6">( Image credit: [Michael Betancourt](https://betanalpha.github.io/writing/) )</span>
Task: Depth Anomaly Detection and Segmentation
Area: methodology

Depth-only Anomaly Detection and Segmentation
Task: DepthAnomaly Detection
Area: methodology

Depth-only Anomaly Detection
Task: Model Selection
Area: methodology

Given a set of candidate models, the goal of **Model Selection** is to select the model that best approximates the observed data and captures its underlying regularities. Model Selection criteria are defined such that they strike a balance between the goodness of fit, and the generalizability or complexity of the models.


<span class="description-source">Source: [Kernel-based Information Criterion ](https://arxiv.org/abs/1408.5810)</span>
Task: Continuously Indexed Domain Adaptation
Area: methodology

Continuously indexed domain adaptation adapts across continuously indexed domains, e.g., across patients of different ages, where 'age' is a continuous notion.
Task: 3D + RGB Anomaly Segmentation
Area: methodology

3D + RGB Anomaly Segmentation
Task: Core set discovery
Area: methodology

A core set in machine learning is defined as the minimal set of training samples that allows a supervised algorithm to deliver a result as good as the one obtained when the whole set is used.
Task: Explanation Fidelity Evaluation
Area: methodology

Evaluation of explanation fidelity with respect to the underlying model.
Task: Unsupervised Anomaly Detection
Area: methodology

The objective of **Unsupervised Anomaly Detection** is to detect previously unseen rare objects or events without any prior knowledge about these. The only information available is that the percentage of anomalies in the dataset is small, usually less than 1%. Since anomalies are rare and unknown to the user at training time, anomaly detection in most cases boils down to the problem of modelling the normal data distribution and defining a measurement in this space in order to classify samples as anomalous or normal. In high-dimensional data such as images, distances in the original space quickly lose descriptive power (curse of dimensionality) and a mapping to some more suitable space is required.


<span class="description-source">Source: [Unsupervised Learning of Anomaly Detection from Contaminated Image Data using Simultaneous Encoder Training ](https://arxiv.org/abs/1905.11034)</span>
Task: Continual Learning
Area: methodology

**Continual Learning** (also known as **Incremental Learning**, **Life-long Learning**) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.  
If not mentioned, the benchmarks here are **Task-CL**, where task-id is provided on validation.

Source:  
[Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation](https://arxiv.org/abs/1908.02984)  
[Three scenarios for continual learning](https://arxiv.org/abs/1904.07734)  
[Lifelong Machine Learning](https://books.google.ca/books/about/Lifelong_Machine_Learning.html?id=JQ5pDwAAQBAJ&redir_esc=y)  
[Continual lifelong learning with neural networks: A review](https://www.sciencedirect.com/science/article/pii/S0893608019300231)
Task: Q-Learning
Area: methodology

The goal of Q-learning is to learn a policy, which tells an agent what action to take under what circumstances.

<span style="color:grey; opacity: 0.6">( Image credit: [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/pdf/1312.5602v1.pdf) )</span>
Task: Extreme Multi-Label Classification
Area: methodology

Extreme Multi-Label Classification is a supervised learning problem where an instance may be associated with multiple labels. The two main problems are the unbalanced labels in the dataset and the amount of different labels.
Task: Incremental Learning
Area: methodology

Incremental learning aims to develop artificially intelligent systems that can continuously learn to address new tasks from new data while preserving knowledge learned from previously learned tasks.
Task: 3D Anomaly Detection
Area: methodology

3D-only Anomaly Detection
Task: Federated Learning
Area: methodology

**Federated Learning** is a machine learning approach that allows multiple devices or entities to collaboratively train a shared model without exchanging their data with each other. Instead of sending data to a central server for training, the model is trained locally on each device, and only the model updates are sent to the central server, where they are aggregated to improve the shared model. 

This approach allows for privacy-preserving machine learning, as each device keeps its data locally and only shares the information needed to improve the model.
Task: Automated Feature Engineering
Area: methodology

Automated feature engineering improves upon the traditional approach to feature engineering by automatically extracting useful and meaningful features from a set of related data tables with a framework that can be applied to any problem.
Task: Depth + RGB Anomaly Detection
Area: methodology

Depth + RGB Anomaly Detection
Task: Stochastic Optimization
Area: methodology

**Stochastic Optimization** is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.


<span class="description-source">Source: [ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables ](https://arxiv.org/abs/1506.08004)</span>
Task: Distributional Reinforcement Learning
Area: methodology

Value distribution is the distribution of the random return received by a reinforcement learning agent.  it been used for a specific purpose such as implementing risk-aware behaviour. 

We have random return Z whose expectation is the value Q. This random return is also described by a recursive equation, but one of a distributional nature