pwc-archive/papers-with-abstracts huggingface dataset:
Predicting Atlantic Multidecadal Variability
Authors: Glenn Liu, Peidong Wang, Matthew Beveridge, Young-Oh Kwon, Iddo Drori
Date: 2021-10-29 00:00:00

Atlantic Multidecadal Variability (AMV) describes variations of North Atlantic sea surface temperature with a typical cycle of between 60 and 70 years. AMV strongly impacts local climate over North America and Europe, therefore prediction of AMV, especially the extreme values, is of great societal utility for understanding and responding to regional climate change. This work tests multiple machine learning models to improve the state of AMV prediction from maps of sea surface temperature, salinity, and sea level pressure in the North Atlantic region. We use data from the Community Earth System Model 1 Large Ensemble Project, a state-of-the-art climate model with 3,440 years of data. Our results demonstrate that all of the models we use outperform the traditional persistence forecast baseline. Predicting the AMV is important for identifying future extreme temperatures and precipitation, as well as hurricane activity, in Europe and North America up to 25 years in advance.
Longitudinal Analysis of Mask and No-Mask on Child Face Recognition
Authors: Praveen Kumar Chandaliya, Zahid Akhtar, Neeta Nain
Date: 2021-10-29 00:00:00
Tasks: Face Recognition, Face Verification, Person Recognition

Face is one of the most widely employed traits for person recognition, even in many large-scale applications. Despite technological advancements in face recognition systems, they still face obstacles caused by pose, expression, occlusion, and aging variations. Owing to the COVID-19 pandemic, contactless identity verification has become exceedingly vital. Recently, few studies have been conducted on the effect of face mask on adult face recognition systems (FRS). However, the impact of aging with face mask on child subject recognition has not been adequately explored. Thus, the main objective of this study is analyzing the child longitudinal impact together with face mask and other covariates on FRS. Specifically, we performed a comparative investigation of three top performing publicly available face matchers and a post-COVID-19 commercial-off-the-shelf (COTS) system under child cross-age verification and identification settings using our generated synthetic mask and no-mask samples. Furthermore, we investigated the longitudinal consequence of eyeglasses with mask and no-mask. The study exploited no-mask longitudinal child face dataset (i.e., extended Indian Child Longitudinal Face Dataset) that contains 26,258 face images of 7,473 subjects in the age group of [2, 18] over an average time span of 3.35 years. Due to the combined effects of face mask and face aging, the FaceNet, PFE, ArcFace, and COTS face verification system accuracies decrease approximately 25%, 22%, 18%, 12%, respectively.
Classification of jujube fruit based on several pricing factors using machine learning methods
Authors: Abdollah Zakeri, Ruhollah Hedayati, Mohammad Khedmati, Mehran Taghipour-Gorjikolaie
Date: 2021-10-29 00:00:00
Tasks: feature selection

Jujube is a fruit mainly cultivated in India, China and Iran and has many health benefits. It is sold both fresh and dried. There are several factors in jujube pricing such as weight, wrinkles and defections. Some jujube farmers sell their product all at once, without any proper sorting or classification, for an average price. Our studies and experiences show that their profit can increase significantly if their product is sold after the sorting process. There are some traditional sorting methods for dried jujube fruit but they are costly, time consuming and can be inaccurate due to human error. Nowadays, computer vision combined with machine learning methods, is used increasingly in food industry for sorting and classification purposes and solve many of the traditional sorting methods' problems. In this paper we are proposing a computer vision-based method for grading jujube fruits using machine learning techniques which will take most of the important pricing factors into account and can be used to increase the profit of farmers. In this method we first acquire several images from different samples and then extract their visual features such as color features, shape and size features, texture features, defection and wrinkle features and then we select the most useful features using feature selection algorithms like PCA and CFS. A feature vector is obtained for each sample and we use these vectors to train our classifiers to be able to specify the corresponding pre-defined group for each of the samples. We used different classifiers and training methods in order to obtain the best result and by using decision tree we could reach 98.8% accuracy of the classification.
Fetal MRI by robust deep generative prior reconstruction and diffeomorphic registration: application to gestational age prediction
Authors: Lucilio Cordero-Grande, Juan Enrique Ortuño-Fisac, Alena Uus, Maria Deprez, Andrés Santos, Joseph V. Hajnal, María Jesús Ledesma-Carbayo
Date: 2021-10-29 00:00:00

Magnetic resonance imaging of whole fetal body and placenta is limited by different sources of motion affecting the womb. Usual scanning techniques employ single-shot multi-slice sequences where anatomical information in different slices may be subject to different deformations, contrast variations or artifacts. Volumetric reconstruction formulations have been proposed to correct for these factors, but they must accommodate a non-homogeneous and non-isotropic sampling, so regularization becomes necessary. Thus, in this paper we propose a deep generative prior for robust volumetric reconstructions integrated with a diffeomorphic volume to slice registration method. Experiments are performed to validate our contributions and compare with a state of the art method in a cohort of $72$ fetal datasets in the range of $20-36$ weeks gestational age. Results suggest improved image resolution and more accurate prediction of gestational age at scan when comparing to a state of the art reconstruction method. In addition, gestational age prediction results from our volumetric reconstructions compare favourably with existing brain-based approaches, with boosted accuracy when integrating information of organs other than the brain. Namely, a mean absolute error of $0.618$ weeks ($R^2=0.958$) is achieved when combining fetal brain and trunk information.
Deep Deterministic Uncertainty for Semantic Segmentation
Authors: Jishnu Mukhoti, Joost van Amersfoort, Philip H. S. Torr, Yarin Gal
Date: 2021-10-29 00:00:00
Tasks: Segmentation, Semantic Segmentation

We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty estimation using feature space densities, to semantic segmentation. DDU enables quantifying and disentangling epistemic and aleatoric uncertainty in a single forward pass through the model. We study the similarity of feature representations of pixels at different locations for the same class and conclude that it is feasible to apply DDU location independently, which leads to a significant reduction in memory consumption compared to pixel dependent DDU. Using the DeepLab-v3+ architecture on Pascal VOC 2012, we show that DDU improves upon MC Dropout and Deep Ensembles while being significantly faster to compute.
Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation
Authors: Zheng Chen, Zhengming Ding, David Crandall, Lantao Liu
Date: 2021-10-29 00:00:00
Tasks: Autonomous Navigation, Representation Learning, Segmentation, Visual Navigation

Detecting navigable space is a fundamental capability for mobile robots navigating in unknown or unmapped environments. In this work, we treat visual navigable space segmentation as a scene decomposition problem and propose Polyline Segmentation Variational autoencoder Network (PSV-Net), a representation learning-based framework for learning the navigable space segmentation in a self-supervised manner. Current segmentation techniques heavily rely on fully-supervised learning strategies which demand a large amount of pixel-level annotated images. In this work, we propose a framework leveraging a Variational AutoEncoder (VAE) and an AutoEncoder (AE) to learn a polyline representation that compactly outlines the desired navigable space boundary. Through extensive experiments, we validate that the proposed PSV-Net can learn the visual navigable space with no or few labels, producing an accuracy comparable to fully-supervised state-of-the-art methods that use all available labels. In addition, we show that integrating the proposed navigable space segmentation model with a visual planner can achieve efficient mapless navigation in real environments.
Generalized Data Weighting via Class-level Gradient Manipulation
Authors: Can Chen, Shuhao Zheng, Xi Chen, Erqun Dong, Xue Liu, Hao liu, Dejing Dou
Date: 2021-10-29 00:00:00

Label noise and class imbalance are two major issues coexisting in real-world datasets. To alleviate the two issues, state-of-the-art methods reweight each instance by leveraging a small amount of clean and unbiased data. Yet, these methods overlook class-level information within each instance, which can be further utilized to improve performance. To this end, in this paper, we propose Generalized Data Weighting (GDW) to simultaneously mitigate label noise and class imbalance by manipulating gradients at the class level. To be specific, GDW unrolls the loss gradient to class-level gradients by the chain rule and reweights the flow of each gradient separately. In this way, GDW achieves remarkable performance improvement on both issues. Aside from the performance gain, GDW efficiently obtains class-level weights without introducing any extra computational cost compared with instance weighting methods. Specifically, GDW performs a gradient descent step on class-level weights, which only relies on intermediate gradients. Extensive experiments in various settings verify the effectiveness of GDW. For example, GDW outperforms state-of-the-art methods by $2.56\%$ under the $60\%$ uniform noise setting in CIFAR10. Our code is available at https://github.com/GGchen1997/GDW-NIPS2021.
CvS: Classification via Segmentation For Small Datasets
Authors: Nooshin Mojab, Philip S. Yu, Joelle A. Hallak, Darvin Yi
Date: 2021-10-29 00:00:00
Tasks: Classification

Deep learning models have shown promising results in a wide range of computer vision applications across various domains. The success of deep learning methods relies heavily on the availability of a large amount of data. Deep neural networks are prone to overfitting when data is scarce. This problem becomes even more severe for neural network with classification head with access to only a few data points. However, acquiring large-scale datasets is very challenging, laborious, or even infeasible in some domains. Hence, developing classifiers that are able to perform well in small data regimes is crucial for applications with limited data. This paper presents CvS, a cost-effective classifier for small datasets that derives the classification labels from predicting the segmentation maps. We employ the label propagation method to achieve a fully segmented dataset with only a handful of manually segmented data. We evaluate the effectiveness of our framework on diverse problems showing that CvS is able to achieve much higher classification results compared to previous methods when given only a handful of examples.
On-device Real-time Hand Gesture Recognition
Authors: George Sung, Kanstantsin Sokal, Esha Uboweja, Valentin Bazarevsky, Jonathan Baccash, Eduard Gabriel Bazavan, Chuo-Ling Chang, Matthias Grundmann
Date: 2021-10-29 00:00:00
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

We present an on-device real-time hand gesture recognition (HGR) system, which detects a set of predefined static gestures from a single RGB camera. The system consists of two parts: a hand skeleton tracker and a gesture classifier. We use MediaPipe Hands as the basis of the hand skeleton tracker, improve the keypoint accuracy, and add the estimation of 3D keypoints in a world metric space. We create two different gesture classifiers, one based on heuristics and the other using neural networks (NN).
IRA: A shape matching approach for recognition and comparison of generic atomic patterns
Authors: Miha Gunde, Nicolas Salles, Anne Hémeryck, Layla Martin-Samos
Date: 2021-10-29 00:00:00

We propose a versatile, parameter-less approach for solving the shape matching problem, specifically in the context of atomic structures when atomic assignments are not known a priori. The algorithm Iteratively suggests Rotated atom-centered reference frames and Assignments (Iterative Rotations and Assignments, IRA). The frame for which a permutationally invariant set-set distance, namely the Hausdorff distance, returns minimal value is chosen as the solution of the matching problem. IRA is able to find rigid rotations, reflections, translations, and permutations between structures with different numbers of atoms, for any atomic arrangement and pattern, periodic or not. When distortions are present between the structures, optimal rotation and translation are found by further applying a standard Singular Value Decomposition-based method. To compute the atomic assignments under the one-to-one assignment constraint, we develop our own algorithm, Constrained Shortest Distance Assignments (CShDA). The overall approach is extensively tested on several structures, including distorted structural fragments. Efficiency of the proposed algorithm is shown as a benchmark comparison against two other shape matching algorithms. We discuss the use of our approach for the identification and comparison of structures and structural fragments through two examples: a replica exchange trajectory of a cyanine molecule, in which we show how our approach could aid the exploration of relevant collective coordinates for clustering the data; and an SiO$_2$ amorphous model, in which we compute distortion scores and compare them with a classical strain-based potential. The source code and benchmark data are available at \url{https://github.com/mammasmias/IterativeRotationsAssignments}.
SVBRDF Recovery From a Single Image With Highlights using a Pretrained Generative Adversarial Network
Authors: Tao Wen, Beibei Wang, Lei Zhang, Jie Guo, Nicolas Holzschuch
Date: 2021-10-29 00:00:00
Tasks: Generative Adversarial Network

Spatially-varying bi-directional reflectance distribution functions (SVBRDFs) are crucial for designers to incorporate new materials in virtual scenes, making them look more realistic. Reconstruction of SVBRDFs is a long-standing problem. Existing methods either rely on extensive acquisition system or require huge datasets which are nontrivial to acquire. We aim to recover SVBRDFs from a single image, without any datasets. A single image contains incomplete information about the SVBRDF, making the reconstruction task highly ill-posed. It is also difficult to separate between the changes in color that are caused by the material and those caused by the illumination, without the prior knowledge learned from the dataset. In this paper, we use an unsupervised generative adversarial neural network (GAN) to recover SVBRDFs maps with a single image as input. To better separate the effects due to illumination from the effects due to the material, we add the hypothesis that the material is stationary and introduce a new loss function based on Fourier coefficients to enforce this stationarity. For efficiency, we train the network in two stages: reusing a trained model to initialize the SVBRDFs and fine-tune it based on the input image. Our method generates high-quality SVBRDFs maps from a single input photograph, and provides more vivid rendering results compared to previous work. The two-stage training boosts runtime performance, making it 8 times faster than previous work.
Domain Agnostic Few-Shot Learning For Document Intelligence
Authors: Jaya Krishna Mandivarapu, Eric Bunch, Glenn Fung
Date: 2021-10-29 00:00:00
Tasks: Classification, Cross-Domain Few-Shot, cross-domain few-shot learning, Document Classification, document-image-classification, Document Image Classification, Few-Shot Learning, image-classification, Image Classification, Meta-Learning, Metric Learning, Transfer Learning

Few-shot learning aims to generalize to novel classes with only a few samples with class labels. Research in few-shot learning has borrowed techniques from transfer learning, metric learning, meta-learning, and Bayesian methods. These methods also aim to train models from limited training samples, and while encouraging performance has been achieved, they often fail to generalize to novel domains. Many of the existing meta-learning methods rely on training data for which the base classes are sampled from the same domain as the novel classes used for meta-testing. However, in many applications in the industry, such as document classification, collecting large samples of data for meta-learning is infeasible or impossible. While research in the field of the cross-domain few-shot learning exists, it is mostly limited to computer vision. To our knowledge, no work yet exists that examines the use of few-shot learning for classification of semi-structured documents (scans of paper documents) generated as part of a business workflow (forms, letters, bills, etc.). Here the domain shift is significant, going from natural images to the semi-structured documents of interest. In this work, we address the problem of few-shot document image classification under domain shift. We evaluate our work by extensive comparisons with existing methods. Experimental results demonstrate that the proposed method shows consistent improvements on the few-shot classification performance under domain shift.
Equivariant Contrastive Learning
Authors: Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljačić
Date: 2021-10-28 00:00:00
Tasks: Contrastive Learning, Self-Supervised Learning

In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge. In fact, the property of invariance is a trivial instance of a broader class called equivariance, which can be intuitively understood as the property that representations transform according to the way the inputs transform. Here, we show that rather than using only invariance, pre-training that encourages non-trivial equivariance to some transformations, while maintaining invariance to other transformations, can be used to improve the semantic quality of representations. Specifically, we extend popular SSL methods to a more general framework which we name Equivariant Self-Supervised Learning (E-SSL). In E-SSL, a simple additional pre-training objective encourages equivariance by predicting the transformations applied to the input. We demonstrate E-SSL's effectiveness empirically on several popular computer vision benchmarks, e.g. improving SimCLR to 72.5% linear probe accuracy on ImageNet. Furthermore, we demonstrate usefulness of E-SSL for applications beyond computer vision; in particular, we show its utility on regression problems in photonics science. Our code, datasets and pre-trained models are available at https://github.com/rdangovs/essl to aid further research in E-SSL.
Multi-Attribute Balanced Sampling for Disentangled GAN Controls
Authors: Perla Doubinsky, Nicolas Audebert, Michel Crucianu, Hervé Le Borgne
Date: 2021-10-28 00:00:00
Tasks: Attribute, Disentanglement

Various controls over the generated data can be extracted from the latent space of a pre-trained GAN, as it implicitly encodes the semantics of the training data. The discovered controls allow to vary semantic attributes in the generated images but usually lead to entangled edits that affect multiple attributes at the same time. Supervised approaches typically sample and annotate a collection of latent codes, then train classifiers in the latent space to identify the controls. Since the data generated by GANs reflects the biases of the original dataset, so do the resulting semantic controls. We propose to address disentanglement by subsampling the generated data to remove over-represented co-occuring attributes thus balancing the semantics of the dataset before training the classifiers. We demonstrate the effectiveness of this approach by extracting disentangled linear directions for face manipulation on two popular GAN architectures, PGGAN and StyleGAN, and two datasets, CelebAHQ and FFHQ. We show that this approach outperforms state-of-the-art classifier-based methods while avoiding the need for disentanglement-enforcing post-processing.
MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition
Authors: Jinming Zhao, Ruichen Li, Qin Jin, Xinchao Wang, Haizhou Li
Date: 2021-10-27 00:00:00
Tasks: Diversity, Emotion Classification, Emotion Recognition, Multimodal Emotion Recognition, Self-Supervised Learning

Multimodal emotion recognition study is hindered by the lack of labelled corpora in terms of scale and diversity, due to the high annotation cost and label ambiguity. In this paper, we propose a pre-training model \textbf{MEmoBERT} for multimodal emotion recognition, which learns multimodal joint representations through self-supervised learning from large-scale unlabeled video data that come in sheer volume. Furthermore, unlike the conventional "pre-train, finetune" paradigm, we propose a prompt-based method that reformulates the downstream emotion classification task as a masked text prediction one, bringing the downstream task closer to the pre-training. Extensive experiments on two benchmark datasets, IEMOCAP and MSP-IMPROV, show that our proposed MEmoBERT significantly enhances emotion recognition performance.
A Frequency Perspective of Adversarial Robustness
Authors: Shishira R Maiya, Max Ehrlich, Vatsal Agarwal, Ser-Nam Lim, Tom Goldstein, Abhinav Shrivastava
Date: 2021-10-26 00:00:00
Tasks: Adversarial Robustness

Adversarial examples pose a unique challenge for deep learning systems. Despite recent advances in both attacks and defenses, there is still a lack of clarity and consensus in the community about the true nature and underlying properties of adversarial examples. A deep understanding of these examples can provide new insights towards the development of more effective attacks and defenses. Driven by the common misconception that adversarial examples are high-frequency noise, we present a frequency-based understanding of adversarial examples, supported by theoretical and empirical findings. Our analysis shows that adversarial examples are neither in high-frequency nor in low-frequency components, but are simply dataset dependent. Particularly, we highlight the glaring disparities between models trained on CIFAR-10 and ImageNet-derived datasets. Utilizing this framework, we analyze many intriguing properties of training robust models with frequency constraints, and propose a frequency-based explanation for the commonly observed accuracy vs. robustness trade-off.
Resource-Efficient Federated Learning
Authors: Ahmed M. Abdelmoniem, Atal Narayan Sahu, Marco Canini, Suhaib A. Fahmy
Date: 2021-11-01 00:00:00
Tasks: Fairness, Federated Learning

Federated Learning (FL) enables distributed training by learners using local data, thereby enhancing privacy and reducing communication. However, it presents numerous challenges relating to the heterogeneity of the data distribution, device capabilities, and participant availability as deployments scale, which can impact both model convergence and bias. Existing FL schemes use random participant selection to improve fairness; however, this can result in inefficient use of resources and lower quality training. In this work, we systematically address the question of resource efficiency in FL, showing the benefits of intelligent participant selection, and incorporation of updates from straggling participants. We demonstrate how these factors enable resource efficiency while also improving trained model quality.