pwc-archive/papers-with-abstracts huggingface dataset:
Analysis of Manual and Automated Skin Tone Assignments for Face Recognition Applications
Authors: KS Krishnapriya, Michael C. King, Kevin W. Bowyer
Date: 2021-04-29 00:00:00
Tasks: Face Recognition, MORPH

News reports have suggested that darker skin tone causes an increase in face recognition errors. The Fitzpatrick scale is widely used in dermatology to classify sensitivity to sun exposure and skin tone. In this paper, we analyze a set of manual Fitzpatrick skin type assignments and also employ the individual typology angle to automatically estimate the skin tone from face images. The set of manual skin tone rating experiments shows that there are inconsistencies between human raters that are difficult to eliminate. Efforts to automate skin tone rating suggest that it is particularly challenging on images collected without a calibration object in the scene. However, after the color-correction, the level of agreement between automated and manual approaches is found to be 96% or better for the MORPH images. To our knowledge, this is the first work to: (a) examine the consistency of manual skin tone ratings across observers, (b) document that there is substantial variation in the rating of the same image by different observers even when exemplar images are given for guidance and all images are color-corrected, and (c) compare manual versus automated skin tone ratings.
EagerMOT: 3D Multi-Object Tracking via Sensor Fusion
Authors: Aleksandr Kim, Aljoša Ošep, Laura Leal-Taixé
Date: 2021-04-29 00:00:00
Tasks: 3D Multi-Object Tracking, Motion Planning, Multi-Object Tracking, Multi-Object Tracking and Segmentation, Multiple Object Tracking, Object, Object Tracking, Sensor Fusion

Multi-object tracking (MOT) enables mobile robots to perform well-informed motion planning and navigation by localizing surrounding objects in 3D space and time. Existing methods rely on depth sensors (e.g., LiDAR) to detect and track targets in 3D space, but only up to a limited sensing range due to the sparsity of the signal. On the other hand, cameras provide a dense and rich visual signal that helps to localize even distant objects, but only in the image domain. In this paper, we propose EagerMOT, a simple tracking formulation that eagerly integrates all available object observations from both sensor modalities to obtain a well-informed interpretation of the scene dynamics. Using images, we can identify distant incoming objects, while depth estimates allow for precise trajectory localization as soon as objects are within the depth-sensing range. With EagerMOT, we achieve state-of-the-art results across several MOT tasks on the KITTI and NuScenes datasets. Our code is available at https://github.com/aleksandrkim61/EagerMOT.
Physically Feasible Vehicle Trajectory Prediction
Authors: Harshayu Girase, Jerrick Hoang, Sai Yalamanchi, Micol Marchetti-Bowick
Date: 2021-04-29 00:00:00
Tasks: Autonomous Driving, Prediction, Trajectory Prediction

Predicting the future motion of actors in a traffic scene is a crucial part of any autonomous driving system. Recent research in this area has focused on trajectory prediction approaches that optimize standard trajectory error metrics. In this work, we describe three important properties -- physical realism guarantees, system maintainability, and sample efficiency -- which we believe are equally important for developing a self-driving system that can operate safely and practically in the real world. Furthermore, we introduce PTNet (PathTrackingNet), a novel approach for vehicle trajectory prediction that is a hybrid of the classical pure pursuit path tracking algorithm and modern graph-based neural networks. By combining a structured robotics technique with a flexible learning approach, we are able to produce a system that not only achieves the same level of performance as other state-of-the-art methods on traditional trajectory error metrics, but also provides strong guarantees about the physical realism of the predicted trajectories while requiring half the amount of data. We believe focusing on this new class of hybrid approaches is an useful direction for developing and maintaining a safety-critical autonomous driving system.
Lung Cancer Diagnosis Using Deep Attention Based on Multiple Instance Learning and Radiomics
Authors: Junhua Chen, Haiyan Zeng, Chong Zhang, Zhenwei Shi, Andre Dekker, Leonard Wee, Inigo Bermejo
Date: 2021-04-29 00:00:00
Tasks: Deep Attention, Lung Cancer Diagnosis, Lung Nodule Classification, Multiple Instance Learning

Early diagnosis of lung cancer is a key intervention for the treatment of lung cancer computer aided diagnosis (CAD) can play a crucial role. However, most published CAD methods treat lung cancer diagnosis as a lung nodule classification problem, which does not reflect clinical practice, where clinicians diagnose a patient based on a set of images of nodules, instead of one specific nodule. Besides, the low interpretability of the output provided by these methods presents an important barrier for their adoption. In this article, we treat lung cancer diagnosis as a multiple instance learning (MIL) problem in order to better reflect the diagnosis process in the clinical setting and for the higher interpretability of the output. We chose radiomics as the source of input features and deep attention-based MIL as the classification algorithm.The attention mechanism provides higher interpretability by estimating the importance of each instance in the set for the final diagnosis.In order to improve the model's performance in a small imbalanced dataset, we introduce a new bag simulation method for MIL.The results show that our method can achieve a mean accuracy of 0.807 with a standard error of the mean (SEM) of 0.069, a recall of 0.870 (SEM 0.061), a positive predictive value of 0.928 (SEM 0.078), a negative predictive value of 0.591 (SEM 0.155) and an area under the curve (AUC) of 0.842 (SEM 0.074), outperforming other MIL methods.Additional experiments show that the proposed oversampling strategy significantly improves the model's performance. In addition, our experiments show that our method provides an indication of the importance of each nodule in determining the diagnosis, which combined with the well-defined radiomic features, make the results more interpretable and acceptable for doctors and patients.
AGORA: Avatars in Geography Optimized for Regression Analysis
Authors: Priyanka Patel, Chun-Hao P. Huang, Joachim Tesch, David T. Hoffmann, Shashank Tripathi, Michael J. Black
Date: 2021-04-29 00:00:00
Proceeding: CVPR 2021 1
Tasks: 2D Human Pose Estimation, 3D Human Pose Estimation, 3D Human Reconstruction, 3D Human Shape Estimation, 3D Multi-Person Mesh Recovery, 3D Multi-Person Pose Estimation, Monocular 3D Human Pose Estimation, Pose Estimation, regression

While the accuracy of 3D human pose estimation from images has steadily improved on benchmark datasets, the best methods still fail in many real-world scenarios. This suggests that there is a domain gap between current datasets and common scenes containing people. To obtain ground-truth 3D pose, current datasets limit the complexity of clothing, environmental conditions, number of subjects, and occlusion. Moreover, current datasets evaluate sparse 3D joint locations corresponding to the major joints of the body, ignoring the hand pose and the face shape. To evaluate the current state-of-the-art methods on more challenging images, and to drive the field to address new problems, we introduce AGORA, a synthetic dataset with high realism and highly accurate ground truth. Here we use 4240 commercially-available, high-quality, textured human scans in diverse poses and natural clothing; this includes 257 scans of children. We create reference 3D poses and body shapes by fitting the SMPL-X body model (with face and hands) to the 3D scans, taking into account clothing. We create around 14K training and 3K test images by rendering between 5 and 15 people per image using either image-based lighting or rendered 3D environments, taking care to make the images physically plausible and photoreal. In total, AGORA consists of 173K individual person crops. We evaluate existing state-of-the-art methods for 3D human pose estimation on this dataset and find that most methods perform poorly on images of children. Hence, we extend the SMPL-X model to better capture the shape of children. Additionally, we fine-tune methods on AGORA and show improved performance on both AGORA and 3DPW, confirming the realism of the dataset. We provide all the registered 3D reference training data, rendered images, and a web-based evaluation site at https://agora.is.tue.mpg.de/.
Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation
Authors: Shreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, Vincent Lepetit
Date: 2021-04-29 00:00:00
Proceeding: CVPR 2022 1
Tasks: 3D Hand Pose Estimation, 3D Interacting Hand Pose Estimation, 3D Pose Estimation, hand-object pose, Pose Estimation

We propose a robust and accurate method for estimating the 3D poses of two hands in close interaction from a single color image. This is a very challenging problem, as large occlusions and many confusions between the joints may happen. State-of-the-art methods solve this problem by regressing a heatmap for each joint, which requires solving two problems simultaneously: localizing the joints and recognizing them. In this work, we propose to separate these tasks by relying on a CNN to first localize joints as 2D keypoints, and on self-attention between the CNN features at these keypoints to associate them with the corresponding hand joint. The resulting architecture, which we call "Keypoint Transformer", is highly efficient as it achieves state-of-the-art performance with roughly half the number of model parameters on the InterHand2.6M dataset. We also show it can be easily extended to estimate the 3D pose of an object manipulated by one or two hands with high performance. Moreover, we created a new dataset of more than 75,000 images of two hands manipulating an object fully annotated in 3D and will make it publicly available.
Text2Video: Text-driven Talking-head Video Synthesis with Personalized Phoneme-Pose Dictionary
Authors: Sibo Zhang, Jiahong Yuan, Miao Liao, Liangjun Zhang
Date: 2021-04-29 00:00:00
Tasks: Face Generation, Generative Adversarial Network, Talking Face Generation, Video Generation

With the advance of deep learning technology, automatic video generation from audio or text has become an emerging and promising research topic. In this paper, we present a novel approach to synthesize video from the text. The method builds a phoneme-pose dictionary and trains a generative adversarial network (GAN) to generate video from interpolated phoneme poses. Compared to audio-driven video generation algorithms, our approach has a number of advantages: 1) It only needs a fraction of the training data used by an audio-driven approach; 2) It is more flexible and not subject to vulnerability due to speaker variation; 3) It significantly reduces the preprocessing, training and inference time. We perform extensive experiments to compare the proposed method with state-of-the-art talking face generation methods on a benchmark dataset and datasets of our own. The results demonstrate the effectiveness and superiority of our approach.
Crack Semantic Segmentation using the U-Net with Full Attention Strategy
Authors: Fangzheng Lin, Jiesheng Yang, Jiangpeng Shu, Raimar J. Scherer
Date: 2021-04-29 00:00:00
Tasks: Segmentation, Semantic Segmentation, Structural Health Monitoring

Structures suffer from the emergence of cracks, therefore, crack detection is always an issue with much concern in structural health monitoring. Along with the rapid progress of deep learning technology, image semantic segmentation, an active research field, offers another solution, which is more effective and intelligent, to crack detection Through numerous artificial neural networks have been developed to address the preceding issue, corresponding explorations are never stopped improving the quality of crack detection. This paper presents a novel artificial neural network architecture named Full Attention U-net for image semantic segmentation. The proposed architecture leverages the U-net as the backbone and adopts the Full Attention Strategy, which is a synthesis of the attention mechanism and the outputs from each encoding layer in skip connection. Subject to the hardware in training, the experiments are composed of verification and validation. In verification, 4 networks including U-net, Attention U-net, Advanced Attention U-net, and Full Attention U-net are tested through cell images for a competitive study. With respect to mean intersection-over-unions and clarity of edge identification, the Full Attention U-net performs best in verification, and is hence applied for crack semantic segmentation in validation to demonstrate its effectiveness.
Unsupervised Layered Image Decomposition into Object Prototypes
Authors: Tom Monnier, Elliot Vincent, Jean Ponce, Mathieu Aubry
Date: 2021-04-29 00:00:00
Proceeding: ICCV 2021 10
Tasks: Object, Object Discovery

We present an unsupervised learning framework for decomposing images into layers of automatically discovered object models. Contrary to recent approaches that model image layers with autoencoder networks, we represent them as explicit transformations of a small set of prototypical images. Our model has three main components: (i) a set of object prototypes in the form of learnable images with a transparency channel, which we refer to as sprites; (ii) differentiable parametric functions predicting occlusions and transformation parameters necessary to instantiate the sprites in a given image; (iii) a layered image formation model with occlusion for compositing these instances into complete images including background. By jointly learning the sprites and occlusion/transformation predictors to reconstruct images, our approach not only yields accurate layered image decompositions, but also identifies object categories and instance parameters. We first validate our approach by providing results on par with the state of the art on standard multi-object synthetic benchmarks (Tetrominoes, Multi-dSprites, CLEVR6). We then demonstrate the applicability of our model to real images in tasks that include clustering (SVHN, GTSRB), cosegmentation (Weizmann Horse) and object discovery from unfiltered social network images. To the best of our knowledge, our approach is the first layered image decomposition algorithm that learns an explicit and shared concept of object type, and is robust enough to be applied to real images.
Analytical models and estimations for a meander line turn protecting against an ultrawide band pulse
Authors: Roman Surovtsev, Alexander Nosov, Talgat Gazizov
Date: 2021-04-30 00:00:00

The paper presents analytical models for calculating the response to a pulse excitation in a turn of a meander line. One of the models was used to derive a condition for equalizing the voltage amplitudes for three pulses at the end of the turn which has different per-unit-length delays of even and odd modes. The obtained models and equalization condition were validated by quasistatic simulation. The results will allow for accelerated design and optimization of this type of structures for protection against ultrawide band pulses without costly multivariate analysis and calculation of response
End-to-End Jet Classification of Boosted Top Quarks with the CMS Open Data
Authors: Michael Andrews, Bjorn Burkle, Yi-fan Chen, Davide DiCroce, Sergei Gleyzer, Ulrich Heintz, Meenakshi Narain, Manfred Paulini, Nikolas Pervan, Yusef Shafi, Wei Sun, Emanuele Usai, Kun Yang
Date: 2021-04-19 00:00:00
Tasks: Deep Learning

We describe a novel application of the end-to-end deep learning technique to the task of discriminating top quark-initiated jets from those originating from the hadronization of a light quark or a gluon. The end-to-end deep learning technique combines deep learning algorithms and low-level detector representation of the high-energy collision event. In this study, we use low-level detector information from the simulated CMS Open Data samples to construct the top jet classifiers. To optimize classifier performance we progressively add low-level information from the CMS tracking detector, including pixel detector reconstructed hits and impact parameters, and demonstrate the value of additional tracking information even when no new spatial structures are added. Relying only on calorimeter energy deposits and reconstructed pixel detector hits, the end-to-end classifier achieves an AUC score of 0.975$\pm$0.002 for the task of classifying boosted top quark jets. After adding derived track quantities, the classifier AUC score increases to 0.9824$\pm$0.0013, serving as the first performance benchmark for these CMS Open Data samples. We additionally provide a timing performance comparison of different processor unit architectures for training the network.
Inductive Predictions of Extreme Hydrologic Events in The Wabash River Watershed
Authors: Nicholas Majeske, Bidisha Abesh, Chen Zhu, Ariful Azad
Date: 2021-04-25 00:00:00
Tasks: Time Series, Time Series Analysis

We present a machine learning method to predict extreme hydrologic events from spatially and temporally varying hydrological and meteorological data. We used a timestep reduction technique to reduce the computational and memory requirements and trained a bidirection LSTM network to predict soil water and stream flow from time series data observed and simulated over eighty years in the Wabash River Watershed. We show that our simple model can be trained much faster than complex attention networks such as GeoMAN without sacrificing accuracy. Based on the predicted values of soil water and stream flow, we predict the occurrence and severity of extreme hydrologic events such as droughts. We also demonstrate that extreme events can be predicted in geographical locations separate from locations observed during the training process. This spatially-inductive setting enables us to predict extreme events in other areas in the US and other parts of the world using our model trained with the Wabash Basin data.
Editorial: Introduction to the Issue on Deep Learning for Image/Video Restoration and Compression
Authors: A. Murat Tekalp, Michele Covell, Radu Timofte, Chao Dong
Date: 2021-02-09 00:00:00
Tasks: Image Restoration, Video Restoration

Recent works have shown that learned models can achieve significant performance gains, especially in terms of perceptual quality measures, over traditional methods. Hence, the state of the art in image restoration and compression is getting redefined. This special issue covers the state of the art in learned image/video restoration and compression to promote further progress in innovative architectures and training methods for effective and efficient networks for image/video restoration and compression.
Revisiting the dynamics of Bose-Einstein condensates in a double well by deep learning with a hybrid network
Authors: Shurui Li, Jianqin Xu, Jing Qian, Weiping Zhang
Date: 2021-04-25 00:00:00

Deep learning, accounting for the use of an elaborate neural network, has recently been developed as an efficient and powerful tool to solve diverse problems in physics and other sciences. In the present work, we propose a novel learning method based on a hybrid network integrating two different kinds of neural networks: Long Short-Term Memory(LSTM) and Deep Residual Network(ResNet), in order to overcome the difficulty met in numerically simulating strongly-oscillating dynamical evolutions of physical systems. By taking the dynamics of Bose-Einstein condensates in a double-well potential as an example, we show that our new method makes a high efficient pre-learning and a high-fidelity prediction about the whole dynamics. This benefits from the advantage of the combination of the LSTM and the ResNet and is impossibly achieved by a single network in the case of direct learning. Our method can be applied for simulating complex cooperative dynamics in a system with fast multiple-frequency oscillations with the aid of auxiliary spectrum analysis.
Random Embeddings and Linear Regression can Predict Protein Function
Authors: Tianyu Lu, Alex X. Lu, Alan M. Moses
Date: 2021-04-25 00:00:00
Tasks: Prediction, Protein Function Prediction, regression

Large self-supervised models pretrained on millions of protein sequences have recently gained popularity in generating embeddings of protein sequences for protein function prediction. However, the absence of random baselines makes it difficult to conclude whether pretraining has learned useful information for protein function prediction. Here we show that one-hot encoding and random embeddings, both of which do not require any pretraining, are strong baselines for protein function prediction across 14 diverse sequence-to-function tasks.
Assessment of the influence of features on a classification problem: an application to COVID-19 patients
Authors: L. Davila-Pena, Ignacio García-Jurado, B. Casas-Méndez
Date: 2021-04-09 00:00:00
Tasks: Classification, General Classification

This paper deals with an important subject in classification problems addressed by machine learning techniques: the evaluation of the influence of each of the features on the classification of individuals. Specifically, a measure of that influence is introduced using the Shapley value of cooperative games. In addition, an axiomatic characterisation of the proposed measure is provided based on properties of efficiency and balanced contributions. Furthermore, some experiments have been designed in order to validate the appropriate performance of such measure. Finally, the methodology introduced is applied to a sample of COVID-19 patients to study the influence of certain demographic or risk factors on various events of interest related to the evolution of the disease.
Continuous normalizing flows on manifolds
Authors: Luca Falorsi
Date: 2021-03-14 00:00:00

Normalizing flows are a powerful technique for obtaining reparameterizable samples from complex multimodal distributions. Unfortunately, current approaches are only available for the most basic geometries and fall short when the underlying space has a nontrivial topology, limiting their applicability for most real-world data. Using fundamental ideas from differential geometry and geometric control theory, we describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifolds. We propose a general methodology for parameterizing vector fields on these spaces and demonstrate how gradient-based learning can be performed. Additionally, we provide a scalable unbiased estimator for the divergence in this generalized setting. Experiments on a diverse selection of spaces empirically showcase the defined framework's ability to obtain reparameterizable samples from complex distributions.