pwc-archive/papers-with-abstracts huggingface dataset:
Foreground-aware Semantic Representations for Image Harmonization
Authors: Konstantin Sofiiuk, Polina Popenova, Anton Konushin
Date: 2020-06-01 00:00:00
Tasks: Decoder, Image Harmonization

Image harmonization is an important step in photo editing to achieve visual consistency in composite images by adjusting the appearances of foreground to make it compatible with background. Previous approaches to harmonize composites are based on training of encoder-decoder networks from scratch, which makes it challenging for a neural network to learn a high-level representation of objects. We propose a novel architecture to utilize the space of high-level features learned by a pre-trained classification network. We create our models as a combination of existing encoder-decoder architectures and a pre-trained foreground-aware deep high-resolution network. We extensively evaluate the proposed method on existing image harmonization benchmark and set up a new state-of-the-art in terms of MSE and PSNR metrics. The code and trained models are available at \url{https://github.com/saic-vul/image_harmonization}.
Real-Time Face and Landmark Localization for Eyeblink Detection
Authors: Paul Bakker, Henk-Jan Boele, Zaid Al-Ars, Christos Strydis
Date: 2020-06-01 00:00:00
Tasks: CPU, Eyeblink detection, Face Detection, GPU

Pavlovian eyeblink conditioning is a powerful experiment used in the field of neuroscience to measure multiple aspects of how we learn in our daily life. To track the movement of the eyelid during an experiment, researchers have traditionally made use of potentiometers or electromyography. More recently, the use of computer vision and image processing alleviated the need for these techniques but currently employed methods require human intervention and are not fast enough to enable real-time processing. In this work, a face- and landmark-detection algorithm have been carefully combined in order to provide fully automated eyelid tracking, and have further been accelerated to make the first crucial step towards online, closed-loop experiments. Such experiments have not been achieved so far and are expected to offer significant insights in the workings of neurological and psychiatric disorders. Based on an extensive literature search, various different algorithms for face detection and landmark detection have been analyzed and evaluated. Two algorithms were identified as most suitable for eyelid detection: the Histogram-of-Oriented-Gradients (HOG) algorithm for face detection and the Ensemble-of-Regression-Trees (ERT) algorithm for landmark detection. These two algorithms have been accelerated on GPU and CPU, achieving speedups of 1,753$\times$ and 11$\times$, respectively. To demonstrate the usefulness of our eyelid-detection algorithm, a research hypothesis was formed and a well-established neuroscientific experiment was employed: eyeblink detection. Our experimental evaluation reveals an overall application runtime of 0.533 ms per frame, which is 1,101$\times$ faster than the sequential implementation and well within the real-time requirements of eyeblink conditioning in humans, i.e. faster than 500 frames per second.
Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving
Authors: Farzeen Munir, Shoaib Azam, Muhammd Aasim Rafique, Ahmad Muqeem Sheri, Moongu Jeon, Witold Pedrycz
Date: 2020-06-01 00:00:00
Tasks: Autonomous Driving, Domain Adaptation, Generative Adversarial Network, Object, object-detection, Object Detection, Robust Object Detection, Style Transfer, Transfer Learning

Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. A thermal camera captures an image using the heat difference emitted by objects in the infrared spectrum, and object detection in thermal images becomes effective for autonomous driving in challenging conditions. Although object detection in the visible spectrum domain imaging has matured, thermal object detection lacks effectiveness. A significant challenge is scarcity of labeled data for the thermal domain which is desiderata for SOTA artificial intelligence techniques. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images. The framework uses a generative adversarial network (GAN) to transfer the low-level features from the visible spectrum domain to the thermal domain through style consistency. The efficacy of the proposed method of object detection in thermal images is evident from the improved results when used styled images from publicly available thermal image datasets (FLIR ADAS and KAIST Multi-Spectral).
Temporal Aggregate Representations for Long-Range Video Understanding
Authors: Fadime Sener, Dipika Singhania, Angela Yao
Date: 2020-06-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Action Anticipation, Action Recognition, Future prediction, Video Segmentation, Video Semantic Segmentation, Video Understanding

Future prediction, especially in long-range videos, requires reasoning from current and past observations. In this work, we address questions of temporal extent, scaling, and level of semantic abstraction with a flexible multi-granular temporal aggregation framework. We show that it is possible to achieve state of the art in both next action and dense anticipation with simple techniques such as max-pooling and attention. To demonstrate the anticipation capabilities of our model, we conduct experiments on Breakfast, 50Salads, and EPIC-Kitchens datasets, where we achieve state-of-the-art results. With minimal modifications, our model can also be extended for video segmentation and action recognition.
Multi-scale Cloud Detection in Remote Sensing Images using a Dual Convolutional Neural Network
Authors: Markku Luotamo, Sari Metsämäki, Arto Klami
Date: 2020-06-01 00:00:00
Tasks: Cloud Detection, Segmentation, Semantic Segmentation

Semantic segmentation by convolutional neural networks (CNN) has advanced the state of the art in pixel-level classification of remote sensing images. However, processing large images typically requires analyzing the image in small patches, and hence features that have large spatial extent still cause challenges in tasks such as cloud masking. To support a wider scale of spatial features while simultaneously reducing computational requirements for large satellite images, we propose an architecture of two cascaded CNN model components successively processing undersampled and full resolution images. The first component distinguishes between patches in the inner cloud area from patches at the cloud's boundary region. For the cloud-ambiguous edge patches requiring further segmentation, the framework then delegates computation to a fine-grained model component. We apply the architecture to a cloud detection dataset of complete Sentinel-2 multispectral images, approximately annotated for minimal false negatives in a land use application. On this specific task and data, we achieve a 16\% relative improvement in pixel accuracy over a CNN baseline based on patching.
LFTag: A Scalable Visual Fiducial System with Low Spatial Frequency
Authors: Ben Wang
Date: 2020-06-01 00:00:00
Tasks: Pose Estimation

Visual fiducial systems are a key component of many robotics and AR/VR applications for 6-DOF monocular relative pose estimation and target identification. This paper presents LFTag, a visual fiducial system based on topological detection and relative position data encoding which optimizes data density within spatial frequency constraints. The marker is constructed to resolve rotational ambiguity, which combined with the robust geometric and topological false positive rejection, allows all marker bits to be used for data. When compared to existing state-of-the-art square binary markers (AprilTag) and topological markers (TopoTag) in simulation, the proposed fiducial system (LFTag) offers significant advances in dictionary size and range. LFTag 3x3 achieves 546 times the dictionary size of AprilTag 25h9 and LFTag 4x4 achieves 126 thousand times the dictionary size of AprilTag 41h12 while simultaneously achieving longer detection range. LFTag 3x3 also achieves more than twice the detection range of TopoTag 4x4 at the same dictionary size.
3D Lidar Mapping Relative Accuracy Automatic Evaluation Algorithm
Authors: Guibin Chen, Jiong Deng, Dongze Huang, Shuo Zhang
Date: 2020-06-01 00:00:00
Tasks: Decision Making, Simultaneous Localization and Mapping

HD (High Definition) map based on 3D lidar plays a vital role in autonomous vehicle localization, planning, decision-making, perception, etc. Many 3D lidar mapping technologies related to SLAM (Simultaneous Localization and Mapping) are used in HD map construction to ensure its high accuracy. To evaluate the accuracy of 3D lidar mapping, the most common methods use ground truth of poses to calculate the error between estimated poses and ground truth, however it's usually so difficult to get the ground truth of poses in the actual lidar mapping for autonomous vehicle. In this paper, we proposed a relative accuracy evaluation algorithm that can automatically evaluate the accuracy of HD map built by 3D lidar mapping without ground truth. A method for detecting the degree of ghosting in point cloud map quantitatively is designed to reflect the accuracy indirectly, which takes advantage of the principle of light traveling in a straight line and the fact that light can not penetrate opaque objects. Our experimental results confirm that the proposed evaluation algorithm can automatically and efficiently detect the bad poses whose accuracy are less than the set threshold such as 0.1m, then calculate the bad poses percentage P_bad in all estimated poses to obtain the final accuracy metric P_acc = 1 - P_bad.
Bi-directional Exponential Angular Triplet Loss for RGB-Infrared Person Re-Identification
Authors: Hanrong Ye, Hong Liu, Fanyang Meng, Xia Li
Date: 2020-06-01 00:00:00
Tasks: Person Re-Identification, Triplet

RGB-Infrared person re-identification (RGB-IR Re- ID) is a cross-modality matching problem, where the modality discrepancy is a big challenge. Most existing works use Euclidean metric based constraints to resolve the discrepancy between features of images from different modalities. However, these methods are incapable of learning angularly discriminative feature embedding because Euclidean distance cannot measure the included angle between embedding vectors effectively. As an angularly discriminative feature space is important for classifying the human images based on their embedding vectors, in this paper, we propose a novel ranking loss function, named Bi-directional Exponential Angular Triplet Loss, to help learn an angularly separable common feature space by explicitly constraining the included angles between embedding vectors. Moreover, to help stabilize and learn the magnitudes of embedding vectors, we adopt a common space batch normalization layer. The quantitative and qualitative experiments on the SYSU-MM01 and RegDB dataset support our analysis. On SYSU-MM01 dataset, the performance is improved from 7.40% / 11.46% to 38.57% / 38.61% for rank-1 accuracy / mAP compared with the baseline. The proposed method can be generalized to the task of single-modality Re-ID and improves the rank-1 accuracy / mAP from 92.0% / 81.7% to 94.7% / 86.6% on the Market-1501 dataset, from 82.6% / 70.6% to 87.6% / 77.1% on the DukeMTMC-reID dataset. Code: https://github.com/prismformore/expAT
Implementing AI-powered semantic character recognition in motor racing sports
Authors: Jose David Fernández Rodríguez, David Daniel Albarracín Molina, Jesús Hormigo Cebolla
Date: 2020-06-01 00:00:00

Oftentimes TV producers of motor-racing programs overlay visual and textual media to provide on-screen context about drivers, such as a driver's name, position or photo. Typically this is accomplished by a human producer who visually identifies the drivers on screen, manually toggling the contextual media associated to each one and coordinating with cameramen and other TV producers to keep the racer in the shot while the contextual media is on screen. This labor-intensive and highly dedicated process is mostly suited to static overlays and makes it difficult to overlay contextual information about many drivers at the same time in short shots. This paper presents a system that largely automates these tasks and enables dynamic overlays using deep learning to track the drivers as they move on screen, without human intervention. This system is not merely theoretical, but an implementation has already been deployed during live races by a TV production company at Formula E races. We present the challenges faced during the implementation and discuss the implications. Additionally, we cover future applications and roadmap of this new technological development.
Multimodal grid features and cell pointers for Scene Text Visual Question Answering
Authors: Lluís Gómez, Ali Furkan Biten, Rubèn Tito, Andrés Mafla, Marçal Rusiñol, Ernest Valveny, Dimosthenis Karatzas
Date: 2020-06-01 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

This paper presents a new model for the task of scene text visual question answering, in which questions about a given image can only be answered by reading and understanding scene text that is present in it. The proposed model is based on an attention mechanism that attends to multi-modal features conditioned to the question, allowing it to reason jointly about the textual and visual modalities in the scene. The output weights of this attention module over the grid of multi-modal spatial features are interpreted as the probability that a certain spatial location of the image contains the answer text the to the given question. Our experiments demonstrate competitive performance in two standard datasets. Furthermore, this paper provides a novel analysis of the ST-VQA dataset based on a human performance study.
One Versus all for deep Neural Network Incertitude (OVNNI) quantification
Authors: Gianni Franchi, Andrei Bursuc, Emanuel Aldea, Severine Dubuisson, Isabelle Bloch
Date: 2020-06-01 00:00:00
Tasks: All

Deep neural networks (DNNs) are powerful learning models yet their results are not always reliable. This is due to the fact that modern DNNs are usually uncalibrated and we cannot characterize their epistemic uncertainty. In this work, we propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs All the other classes (OVA) with predictions from a standard DNN trained to perform All vs All (AVA) classification. On the one hand, the adjustment provided by the AVA DNN to the score of the base classifiers allows for a more fine-grained inter-class separation. On the other hand, the two types of classifiers enforce mutually their detection of out-of-distribution (OOD) samples, circumventing entirely the requirement of using such samples during training. Our method achieves state of the art performance in quantifying OOD data across multiple datasets and architectures while requiring little hyper-parameter tuning.
GoodPoint: unsupervised learning of keypoint detection and description
Authors: Anatoly Belikov, Alexey Potapov
Date: 2020-06-01 00:00:00
Tasks: Image Registration, Keypoint Detection

This paper introduces a new algorithm for unsupervised learning of keypoint detectors and descriptors, which demonstrates fast convergence and good performance across different datasets. The training procedure uses homographic transformation of images. The proposed model learns to detect points and generate descriptors on pairs of transformed images, which are easy for it to distinguish and repeatedly detect. The trained model follows SuperPoint architecture for ease of comparison, and demonstrates similar performance on natural images from HPatches dataset, and better performance on retina images from Fundus Image Registration Dataset, which contain low number of corner-like features. For HPatches and other datasets, coverage was also computed to provide better estimation of model quality.
A Smooth Representation of Belief over SO(3) for Deep Rotation Learning with Uncertainty
Authors: Valentin Peretroukhin, Matthew Giamou, David M. Rosen, W. Nicholas Greene, Nicholas Roy, Jonathan Kelly
Date: 2020-06-01 00:00:00
Tasks: Pose Estimation, Visual Odometry

Accurate rotation estimation is at the heart of robot perception tasks such as visual odometry and object pose estimation. Deep neural networks have provided a new way to perform these tasks, and the choice of rotation representation is an important part of network design. In this work, we present a novel symmetric matrix representation of the 3D rotation group, SO(3), with two important properties that make it particularly suitable for learned models: (1) it satisfies a smoothness property that improves convergence and generalization when regressing large rotation targets, and (2) it encodes a symmetric Bingham belief over the space of unit quaternions, permitting the training of uncertainty-aware models. We empirically validate the benefits of our formulation by training deep neural rotation regressors on two data modalities. First, we use synthetic point-cloud data to show that our representation leads to superior predictive accuracy over existing representations for arbitrary rotation targets. Second, we use image data collected onboard ground and aerial vehicles to demonstrate that our representation is amenable to an effective out-of-distribution (OOD) rejection technique that significantly improves the robustness of rotation estimates to unseen environmental effects and corrupted input images, without requiring the use of an explicit likelihood loss, stochastic sampling, or an auxiliary classifier. This capability is key for safety-critical applications where detecting novel inputs can prevent catastrophic failure of learned models.
Deep Generation of Face Images from Sketches
Authors: Shu-Yu Chen, Wanchao Su, Lin Gao, Shihong Xia, Hongbo Fu
Date: 2020-06-01 00:00:00
Tasks: Image-to-Image Translation, Translation

Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.
DPDnet: A Robust People Detector using Deep Learning with an Overhead Depth Camera
Authors: David Fuentes-Jimenez, Roberto Martin-Lopez, Cristina Losada-Gutierrez, David Casillas-Perez, Javier Macias-Guarasa, Daniel Pizarro, Carlos A. Luna
Date: 2020-06-01 00:00:00
Tasks: Decoder, Head Detection, Human Detection

In this paper we propose a method based on deep learning that detects multiple people from a single overhead depth image with high reliability. Our neural network, called DPDnet, is based on two fully-convolutional encoder-decoder neural blocks based on residual layers. The Main Block takes a depth image as input and generates a pixel-wise confidence map, where each detected person in the image is represented by a Gaussian-like distribution. The refinement block combines the depth image and the output from the main block, to refine the confidence map. Both blocks are simultaneously trained end-to-end using depth images and head position labels. The experimental work shows that DPDNet outperforms state-of-the-art methods, with accuracies greater than 99% in three different publicly available datasets, without retraining not fine-tuning. In addition, the computational complexity of our proposal is independent of the number of people in the scene and runs in real time using conventional GPUs.
Glaucoma Detection From Raw Circumapillary OCT Images Using Fully Convolutional Neural Networks
Authors: Gabriel García, Rocío del Amor, Adrián Colomer, Valery Naranjo
Date: 2020-05-29 00:00:00

Nowadays, glaucoma is the leading cause of blindness worldwide. We propose in this paper two different deep-learning-based approaches to address glaucoma detection just from raw circumpapillary OCT images. The first one is based on the development of convolutional neural networks (CNNs) trained from scratch. The second one lies in fine-tuning some of the most common state-of-the-art CNNs architectures. The experiments were performed on a private database composed of 93 glaucomatous and 156 normal B-scans around the optic nerve head of the retina, which were diagnosed by expert ophthalmologists. The validation results evidence that fine-tuned CNNs outperform the networks trained from scratch when small databases are addressed. Additionally, the VGG family of networks reports the most promising results, with an area under the ROC curve of 0.96 and an accuracy of 0.92, during the prediction of the independent test set.
Learning stochastic object models from medical imaging measurements using Progressively-Growing AmbientGANs
Authors: Weimin Zhou, Sayantan Bhadra, Frank J. Brooks, Hua Li, Mark A. Anastasio
Date: 2020-05-29 00:00:00
Tasks: Diagnostic

It has been advocated that medical imaging systems and reconstruction algorithms should be assessed and optimized by use of objective measures of image quality that quantify the performance of an observer at specific diagnostic tasks. One important source of variability that can significantly limit observer performance is variation in the objects to-be-imaged. This source of variability can be described by stochastic object models (SOMs). A SOM is a generative model that can be employed to establish an ensemble of to-be-imaged objects with prescribed statistical properties. In order to accurately model variations in anatomical structures and object textures, it is desirable to establish SOMs from experimental imaging measurements acquired by use of a well-characterized imaging system. Deep generative neural networks, such as generative adversarial networks (GANs) hold great potential for this task. However, conventional GANs are typically trained by use of reconstructed images that are influenced by the effects of measurement noise and the reconstruction process. To circumvent this, an AmbientGAN has been proposed that augments a GAN with a measurement operator. However, the original AmbientGAN could not immediately benefit from modern training procedures, such as progressive growing, which limited its ability to be applied to realistically sized medical image data. To circumvent this, in this work, a new Progressive Growing AmbientGAN (ProAmGAN) strategy is developed for establishing SOMs from medical imaging measurements. Stylized numerical studies corresponding to common medical imaging modalities are conducted to demonstrate and validate the proposed method for establishing SOMs.