pwc-archive/papers-with-abstracts huggingface dataset:
Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation
Authors: Payam Jome Yazdian, Mo Chen, Angelica Lim
Date: 2021-09-29 00:00:00
Tasks: Clustering, Decoder, Diversity, Gesture Generation, Machine Translation, Representation Learning, Translation

Co-speech gestures are a principal component in conveying messages and enhancing interaction experiences between humans. Similarly, the co-speech gesture is a key ingredient in human-agent interaction including both virtual agents and robots. Existing machine learning approaches have yielded only marginal success in learning speech-to-motion at the frame level. Current methods generate repetitive gesture sequences that lack appropriateness with respect to the speech context. In this paper, we propose a Gesture2Vec model using representation learning methods to learn the relationship between semantic features and corresponding gestures. We propose a vector-quantized variational autoencoder structure as well as training techniques to learn a rigorous representation of gesture sequences. Furthermore, we use a machine translation model that takes input text and translates it into a discrete sequence of associated gesture chunks in the learned gesture space. Ultimately, we use translated quantized gestures from the input text as an input to the autoencoder’s decoder to produce gesture sequences. The resulting gestures can be applied to both virtual agents and humanoid robots. Subjective and objective evaluations confirm the success of our approach in terms of appropriateness, human-likeness, and diversity.
Semi-relaxed Gromov-Wasserstein divergence and applications on graphs
Authors: Cédric Vincent-Cuaz, Rémi Flamary, Marco Corneli, Titouan Vayer, Nicolas Courty
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Dictionary Learning

Comparing structured objects such as graphs is a fundamental operation
involved in many learning tasks. To this end, the Gromov-Wasserstein (GW)
distance, based on Optimal Transport (OT), has proven to be successful in
handling the specific nature of the associated objects. More specifically,
through the nodes connectivity relations, GW operates on graphs, seen as
probability measures over specific spaces. At the core of OT is the idea of
conservation of mass, which imposes a coupling between all the nodes from
the two considered graphs. We argue in this paper that this property can be
detrimental for tasks such as graph dictionary or partition learning, and we
relax it by proposing a new semi-relaxed Gromov-Wasserstein divergence.
Aside from immediate computational benefits, we discuss its properties, and
show that it can lead to an efficient graph dictionary learning algorithm.
We empirically demonstrate its relevance for complex tasks on graphs such as
partitioning, clustering and completion.
Revisiting Layer-wise Sampling in Fast Training for Graph Convolutional Networks
Authors: Yifan Chen, Tianning Xu, Dilek Hakkani-Tur, Di Jin, Yun Yang, Ruoqing Zhu
Date: 2021-09-29 00:00:00

To accelerate the training of graph convolutional networks (GCN), many sampling-based methods have been developed for approximating the embedding aggregation. Among them, a layer-wise approach recursively performs importance sampling to select neighbors jointly for existing nodes in each layer. This paper revisits the approach from a matrix approximation perspective. We identify two issues in the existing layer-wise sampling methods: sub-optimal sampling probabilities and the approximation bias induced by sampling without replacement. We thus propose remedies to address these issues. The improvements are demonstrated by extensive analyses and experiments on common benchmarks.
Composing Features: Compositional Model Augmentation for Steerability of Music Transformers
Authors: Halley Young, Vincent Dumoulin, Pablo Samuel Castro, Jesse Engel, Cheng-Zhi Anna Huang
Date: 2021-09-29 00:00:00
Tasks: Rhythm

Music is a combinatorial art. Given a starting sequence, many continuations are possible, yet often only one is written down. With generative models, we can explore many. However, finding a continuation with specific combinations of features (such as rising pitches, with block chords played in syncopated rhythm) can take many trials.
To tackle the combinatorial nature of composing features, we propose a compositional approach to steering music transformers, building on lightweight fine-tuning methods such as prefix tuning and bias tuning. We introduce a novel contrastive loss function that enables us to steer compositional models over logical features using supervised learning. We examine the difficulty in steering based on whether features musically follow a prime or not, using existing music as a proxy. We show that with a relatively small number of extra parameters, our method allows bias tuning to perform successful fine-tuning in both the single-feature and compositional setting.
Towards Feature Overcorrelation in Deeper Graph Neural Networks
Authors: Wei Jin, Xiaorui Liu, Yao Ma, Charu Aggarwal, Jiliang Tang
Date: 2021-09-29 00:00:00
Tasks: Feature Correlation, Graph Representation Learning, Representation Learning

Graph neural networks (GNNs) have achieved great success in graph representation learning, which has tremendously facilitated various real-world applications. Nevertheless, the performance of GNNs significantly deteriorates when the depth increases. Recent researches have attributed this phenomenon to the oversmoothing issue, which indicates that the learned node representations are highly indistinguishable. In this paper, we observe a new issue in deeper GNNs, i.e., feature overcorrelation, and perform a thorough study to deepen our understanding on this issue. In particular, we demonstrate the existence of feature overcorrelation in deeper GNNs, reveal potential reasons leading to this issue, and validate that overcorrelation and oversmoothing present different patterns though they are related. Since feature overcorrelation indicates that GNNs encode less information and can harm the downstream tasks, it is of great significance to mitigate this issue. Therefore, we propose the DeCorr, a general framework to effectively reduce feature correlation for deeper GNNs. Experimental results on various datasets demonstrate that DeCorr can help train deeper GNNs effectively and is complementary to methods tackling oversmoothing.
Dataset Bias Prediction for Few-Shot Image Classification
Authors: Jangwook Kim, Kyung-Ah Sohn
Date: 2021-09-29 00:00:00
Tasks: Classification, Few-Shot Image Classification, Few-Shot Learning, image-classification, Image Classification, Prediction

One of the obstacles which negatively affect the image classification performance is dataset bias. In particular, if each class has only a few training data samples, the data are highly likely to have dataset bias. Therefore, dataset bias can be a serious issue in few-shot learning, but has rarely been studied so far. To address this issue, we propose a bias prediction network to help improve the performance of few-shot image classification models. Once the features are extracted from an image data, the bias prediction network tries to recover the bias of the raw image such as color from the features. However, if the bias prediction network can recover it easily, we can assume that the extracted features also contain the color bias. Therefore, in our proposed framework, the full model tries to extract features that are difficult for the bias prediction network to recover from. We validate our method by adding the bias prediction network to several existing models and evaluating the performance improvement. Our experimental results show that the bias prediction network can suppress the negative effect of the dataset color bias, resulting in the substantial improvements in existing few-shot classification models. The proposed bias prediction network, which can be integrated with other models very easily, could potentially benefit many existing models for various tasks.
A Risk-Sensitive Policy Gradient Method
Authors: Jared Markowitz, Ryan Gardner, Ashley Llorens, Raman Arora, I-Jeng Wang
Date: 2021-09-29 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning

Standard deep reinforcement learning (DRL) agents aim to maximize expected reward, considering collected experiences equally in formulating a policy. This differs from human decision-making, where gains and losses are valued differently and outlying outcomes are given increased consideration. It also wastes an opportunity for the agent to modulate behavior based on distributional context. Several approaches to distributional DRL have been investigated, with one popular strategy being to evaluate the projected distribution of returns for possible actions. We propose a more direct approach, whereby the distribution of full-episode outcomes is optimized to maximize a chosen function of its cumulative distribution function (CDF). This technique allows for outcomes to be weighed based on relative quality, does not require modification of the reward function to modulate agent behavior, and may be used for both continuous and discrete action spaces. We show how to achieve an unbiased estimate of the policy gradient for a broad class of CDF-based objectives via sampling, subsequently incorporating variance reduction measures to facilitate effective on-policy learning. We use the resulting approach to train agents with different “risk profiles” in penalty-based formulations of six OpenAI Safety Gym environments, finding that moderate emphasis on improvement in training scenarios where the agent performs poorly generally improves agent behavior. We interpret and explore this observation, which leads to improved performance over the widely-used Proximal Policy Optimization algorithm in all environments tested.
Directional Domain Generalization
Authors: Wei Wang, Jiaqi Li, Ruizhi Pu, Gezheng Xu, Fan Zhou, Changjian Shui, Charles Ling, Boyu Wang
Date: 2021-09-29 00:00:00
Tasks: Domain Generalization, Meta-Learning, Relation

Domain generalization aims to learn a predictive model from multiple different but related source tasks that can generalize well to a target task without the need of accessing any target data. Existing domain generalization methods ignore the relation between tasks, implicitly assuming that all the tasks are sampled from a stationary environment. Therefore, they can fail when deployed in an evolving environment. To this end, we formulate and study the \emph{directional domain generalization} (DDG) scenario, which exploits not only the source data but also their evolving pattern to generate a model for the unseen task. Our theoretical result reveals the benefits of modeling the relation between two consecutive tasks by learning a globally consistent directional mapping function. In practice, our analysis also suggest solving the DDG problem in a meta-learning manner, which leads to \emph{directional prototypical network}, the first method for the DDG problem. Empirical evaluation on both synthetic and real-world data sets validates the effectiveness of our approach.
Meta Learning Low Rank Covariance Factors for Energy Based Deterministic Uncertainty
Authors: Jeffrey Ryan Willette, Hae Beom Lee, Juho Lee, Sung Ju Hwang
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Meta-Learning, Out of Distribution (OOD) Detection

Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer. This distance sensitivity with respect to the data aids in tasks such as uncertainty calibration and out-of-distribution (OOD) detection. In previous works, features extracted with a distance sensitive model are used to construct feature covariance matrices which are used in deterministic uncertainty estimation or OOD detection. However, in cases where there is a distribution over tasks, these methods result in covariances which are sub-optimal, as they may not leverage all of the meta information which can be shared among tasks. With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices. Additionally, we propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution which can better separate OOD data, and is well calibrated under a distributional dataset shift.
Disentangling One Factor at a Time
Authors: Vaishnavi S Patil, Matthew S Evanusa, Joseph JaJa
Date: 2021-09-29 00:00:00
Tasks: Disentanglement, Generative Adversarial Network

With the overabundance of data for machines to process in the current state of machine learning, data discovery, organization, and interpretation of the data becomes a critical need.   Specifically of need are unsupervised methods that do not require laborious labeling by human observers.  One promising approach to this enedeavour is \textit{Disentanglement}, which aims at learning the underlying generative latent factors of the data.  The factors should also be as human interpretable as possible for the purposes of data discovery. \textit{Unsupervised disentanglement} is a particularly difficult open subset of the problem, which asks the network to learn on its own the generative factors without any link to the true labels. This problem area is currently dominated by two approaches: Variational Autoencoder and Generative Adversarial Network approaches.  While GANs have good performance, they suffer from difficulty in training and mode collapse, and while VAEs are stable to train, they do not perform as well as GANs in terms of interpretability. In current state of the art versions of these approaches, the networks require the user to specify the number of factors that we expect to find in the data.  This limitation prevents "true" disentanglement, in the sense that learning how many factors is actually one of the tasks we wish the network to solve.  In this work we propose a novel network for unsupervised disentanglement that combines the stable training of the VAE with the interpretability offered by GANs without the training instabilities.   We aim to disentangle interpretable latent factors "one at a time", or OAT factor learning, making no prior assumptions about the number or distribution of factors, in a completely unsupervised manner.  We demonstrate its quantitative and qualitative effectiveness by evaluating the latent representations learned on two benchmark datasets, DSprites and CelebA.
Efficient Ensembles of Graph Neural Networks
Authors: Amrit Nagarajan, Jacob R. Stevens, Anand Raghunathan
Date: 2021-09-29 00:00:00
Tasks: Ensemble Learning, Network Pruning, Node Classification, Product Recommendation

Graph Neural Networks (GNNs) have enabled the power of deep learning to be applied to inputs beyond the Euclidean domain, with applications ranging from social networks and product recommendation engines to the life sciences. GNNs, like other classes of machine learning models, benefit from ensemble learning, wherein multiple models are combined to provide higher accuracy and robustness than single models. However, ensembles suffer from significantly higher inference processing and storage requirements, limiting their use in practical applications. In this work, we leverage the unique characteristics of GNNs to overcome these overheads, creating efficient ensemble GNNs that are faster than even single models at inference time. We observe that during message passing, nodes that are incorrectly classified (error nodes) also end up adversely affecting the representations of other nodes in their neighborhood. This error propagation also makes GNNs more difficult to approximate (e.g., through pruning) for efficient inference. We propose a technique to create ensembles of diverse models, and further propose Error Node Isolation (ENI), which prevents error nodes from sending messages to (and thereby influencing) other nodes. In addition to improving accuracy, ENI also leads to a significant reduction in the memory footprint and the number of arithmetic operations required to evaluate the computational graphs of all neighbors of error nodes. Remarkably, these savings outweigh even the overheads of using multiple models in the ensemble. A second key benefit of ENI is that it  enhances the resilience of GNNs to approximations. Consequently, we propose Edge Pruning and Network Pruning techniques that target both the input graph and the neural networks used to process the graph. Our experiments on GNNs for transductive and inductive node classification demonstrate that ensembles with ENI are simultaneously more accurate (by up to 4.6% and 3.8%) and  faster (by up to 2.8$\times$ and 5.7$\times$) when compared to the best-performing single models and ensembles without ENI, respectively. In addition, GNN ensembles with ENI are consistently more accurate than single models and ensembles without ENI when subject to pruning, leading to additional speedups of up to 5$\times$ with no loss in accuracy.
Lidar Range Image Compression with Deep Delta Encoding
Authors: Xuanyu Zhou, Charles R. Qi, Yin Zhou, Dragomir Anguelov
Date: 2021-09-29 00:00:00
Tasks: Autonomous Driving, Image Compression, Point cloud reconstruction, Quantization

Lidars are widely used in applications such as autonomous driving and augmented reality. However, the large volume of data produced by lidars can lead to high cost in data storage and transmission. Besides rising standards in point cloud compression from the MPEG (G-PCC and V-PCC), recent works also explore using deep networks to improve the compression rates. However, most prior work focus on the generic point cloud representation, neglecting the spatial patterns of the points from lidar range images. In this work, we leverage the range image representation and propose a novel deep delta encoding model to compress lidar data. Our deep model takes in local range image patches and predicts the next pixel value in a raster-scanning manner. The residuals between the prediction and the original value can be entropy encoded to achieve lossless compression under certain quantization rates. Evaluated on the Waymo Open Dataset and KITTI, our method demonstrates significant improvement compared to widely algorithms as well as recent deep methods based on the point cloud representation, in both the point cloud reconstruction quality and the downstream perception model performance.
Calibrating Probabilistic Embeddings for Cross-Modal Retrieval
Authors: Fengchun Qiao, Xi Peng
Date: 2021-09-29 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval

The core of cross-modal retrieval is to measure the content similarity between data of different modalities. The main challenge focuses on learning a  shared representation space for multiple modalities where the similarity measurement can reflect the semantic closeness.
The multiplicity of correspondences further escalates the challenge since all the possible matches should be ranked ahead of the negatives. Probabilistic embeddings are proposed to handle the multiplicity while suffering from similarity miscalibration. To address it, we propose to calibrate the similarity for probabilistic embeddings. The key idea is to estimate the density ratio between the distributions of the two modalities, and use it to calibrate the similarity measurement in the embedding space. To the best of our knowledge, we are the first to study the miscalibration in probabilistic embeddings. 
In addition, we further evaluate three pre-training tasks of language models, 
which is important for cross-modal but seldom investigated in previous studies. 
Extensive experiments as well as ablation studies on two benchmarks demonstrate its superior performance in tackling the multiplicity of cross-modal retrieval.
Modeling Label Space Interactions in Multi-label Classification using Box Embeddings
Authors: Anonymous
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022
Tasks: Classification, Hierarchical Multi-label Classification, Inductive Bias, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Structured Prediction

Multi-label classification is a challenging structured prediction task in which a set of output class labels are predicted for each input. Real-world datasets often have natural or latent taxonomic relationships between labels, making it desirable for models to employ label representations capable of capturing such taxonomies. Most existing multi-label classification methods do not do so, resulting in label predictions that are inconsistent with the taxonomic constraints, thus failing to accurately represent the fundamentals of problem setting. In this work, we introduce the multi-label box model (MBM), a multi-label classification method that combines the encoding power of neural networks with the inductive bias and probabilistic semantics of box embeddings (Vilnis, et al 2018).  Box embeddings can be understood as trainable Venn-diagrams based on hyper-rectangles.  Representing labels by boxes rather than vectors, MBM is able to capture taxonomic relations among labels.  Furthermore, since box embeddings allow these relations to be learned by stochastic gradient descent from data, and to be read as calibrated conditional probabilities, our model is endowed with a high degree of interpretability. This interpretability also facilitates the injection of partial information about label-label relationships into model training, to further improve its consistency. We provide theoretical grounding for our method and show experimentally the model's ability to learn the true latent taxonomic structure from data. Through extensive empirical evaluations on both small and large-scale multi-label classification datasets, we show that BBM can significantly improve taxonomic consistency while preserving or surpassing the state-of-the-art predictive performance.
SpaceMAP: Visualizing Any Data in 2-dimension by Space Expansion
Authors: Xinrui Zu, Qian Tao
Date: 2021-09-29 00:00:00
Tasks: Dimensionality Reduction

Dimensionality reduction (DR) and visualization of high-dimensional data is of theoretical and practical value in machine learning and related fields. In theory, there exists an intriguing, non-intuitive discrepancy between the geometry of high-dimensional space and low-dimensional space. Based on this discrepancy, we propose a novel DR and visualization method called Space-based Manifold Approximation and Projection (SpaceMAP). Our method establishes a quantitative space transformation to address the ``crowding problem" in DR; with the proposed equivalent extended distance (EED) and function distortion (FD) theory, we are able to match the capacity of high-dimensional and low-dimensional space, in a principled manner. To handle complex high-dimensional data with different manifold properties, SpaceMAP makes distinctions between the near field, middle field, and far field of data distribution in a data-specific, hierarchical manner. We evaluated SpaceMAP on a range of artificial and real datasets with different manifold properties, and demonstrated its excellent performance in comparison with classical and state-of-the-art DR methods. In addition, the concept of space distortion provides a generic framework for understanding nonlinear DR methods such as t-distributed Stochastic Neighbor Embedding (tSNE) and Uniform Manifold Approximation and Projection (UMAP).