pwc-archive/papers-with-abstracts huggingface dataset:
NeuroMask: Explaining Predictions of Deep Neural Networks through Mask Learning
Authors: Moustafa Alzantot, Amy Widdicombe, Simon Julier, Mani Srivastava
Date: 2019-08-05 00:00:00
Tasks: General Classification, image-classification, Image Classification

Deep Neural Networks (DNNs) deliver state-of-the-art performance in many image recognition and understanding applications. However, despite their outstanding performance, these models are black-boxes and it is hard to understand how they make their decisions. Over the past few years, researchers have studied the problem of providing explanations of why DNNs predicted their results. However, existing techniques are either obtrusive, requiring changes in model training, or suffer from low output quality. In this paper, we present a novel method, NeuroMask, for generating an interpretable explanation of classification model results. When applied to image classification models, NeuroMask identifies the image parts that are most important to classifier results by applying a mask that hides/reveals different parts of the image, before feeding it back into the model. The mask values are tuned by minimizing a properly designed cost function that preserves the classification result and encourages producing an interpretable mask. Experiments using state-of-the-art Convolutional Neural Networks for image recognition on different datasets (CIFAR-10 and ImageNet) show that NeuroMask successfully localizes the parts of the input image which are most relevant to the DNN decision. By showing a visual quality comparison between NeuroMask explanations and those of other methods, we find NeuroMask to be both accurate and interpretable.
Mass Estimation from Images using Deep Neural Network and Sparse Ground Truth
Authors: Muhammad K. A. Hamdan, Daine T. Rover, Matthew J. Darr, John Just
Date: 2019-08-05 00:00:00
Tasks: GPU, regression

Supervised learning is the workhorse for regression and classification tasks, but the standard approach presumes ground truth for every measurement. In real world applications, limitations due to expense or general in-feasibility due to the specific application are common. In the context of agriculture applications, yield monitoring is one such example where simple-physics based measurements such as volume or force-impact have been used to quantify mass flow, which incur error due to sensor calibration. By utilizing semi-supervised deep learning with gradient aggregation and a sequence of images, in this work we can accurately estimate a physical quantity (mass) with complex data structures and sparse ground truth. Using a vision system capturing images of a sugarcane elevator and running bamboo under controlled testing as a surrogate material to harvesting sugarcane, mass is accurately predicted from images by training a DNN using only final load weights. The DNN succeeds in capturing the complex density physics of random stacking of slender rods internally as part of the mass prediction model, and surpasses older volumetric-based methods for mass prediction. Furthermore, by incorporating knowledge about the system physics through the DNN architecture and penalty terms, improvements in prediction accuracy and stability, as well as faster learning are obtained. It is shown that the classic nonlinear regression optimization can be reformulated with an aggregation term with some independence assumptions to achieve this feat. Since the number of images for any given run are too large to fit on typical GPU vRAM, an implementation is shown that compensates for the limited memory but still achieve fast training times. The same approach presented herein could be applied to other applications like yield monitoring on grain combines or other harvesters using vision or other instrumentation.
Detection of the Group of Traffic Signs with Central Slice Theorem
Authors: Koba Natroshvili
Date: 2019-08-04 00:00:00

Our sensor system consists of a combination of Photonic Mixer Device - PMD and Mono optical cameras. Some traffic signs have stripes at 45{deg}. These traffic signs cancel different restrictions on the road. We detect this class of signs with Radon transformation. Here the Radon transformation is calculated using Central Slice Theorem. We approximate the slice of spectrum by the Discrete Cosine Transformation (DCT).
Action Recognition in Untrimmed Videos with Composite Self-Attention Two-Stream Framework
Authors: Dong Cao, Lisha Xu, HaiBo Chen
Date: 2019-08-04 00:00:00
Tasks: Action Recognition, Temporal Action Localization, Zero-Shot Action Recognition

With the rapid development of deep learning algorithms, action recognition in video has achieved many important research results. One issue in action recognition, Zero-Shot Action Recognition (ZSAR), has recently attracted considerable attention, which classify new categories without any positive examples. Another difficulty in action recognition is that untrimmed data may seriously affect model performance. We propose a composite two-stream framework with a pre-trained model. Our proposed framework includes a classifier branch and a composite feature branch. The graph network model is adopted in each of the two branches, which effectively improves the feature extraction and reasoning ability of the framework. In the composite feature branch, a 3-channel self-attention models are constructed to weight each frame in the video and give more attention to the key frames. Each self-attention models channel outputs a set of attention weights to focus on a particular aspect of the video, and a set of attention weights corresponds to a one-dimensional vector. The 3-channel self-attention models can evaluate key frames from multiple aspects, and the output sets of attention weight vectors form an attention matrix, which effectively enhances the attention of key frames with strong correlation of action. This model can implement action recognition under zero-shot conditions, and has good recognition performance for untrimmed video data. Experimental results on relevant data sets confirm the validity of our model.
Interior Object Detection and Color Harmonization
Authors: Sharmin Pathan
Date: 2019-07-31 00:00:00
Tasks: Image Segmentation, Object, object-detection, Object Detection, Semantic Segmentation, Transfer Learning

Confused about renovating your space? Choosing the perfect color for your walls is always a challenging task. One does rounds of color consultation and several patch tests. This paper proposes an AI tool to pitch paint based on attributes of your room and other furniture, and visualize it on your walls. It makes the color selection process easy. It takes in images of a room, detects furniture objects using YOLO object detection. Once these objects have been detected, the tool picks out color of the object. Later this object specific information gets appended to the room attributes (room_type, room_size, preferred_tone, etc) and a deep neural net is trained to make predictions for color/texture/wallpaper for the walls. Finally, these predictions are visualized on the walls from the images provided. The idea is to take the knowledge of a color consultant and pitch colors that suit the walls and provide a good contrast with the furniture and harmonize with different colors in the room. Transfer learning for YOLO object detection from the COCO dataset was used as a starting point and the weights were later fine-tuned by training on additional images. The model was trained on 1000 records listing the room and furniture attributes, to predict colors. Given the room image, this method finds the best color scheme for the walls. These predictions are then visualized on the walls in the image using image segmentation. The results are visually appealing and automatically enhance the color look-and-feel.
What's in the box? Explaining the black-box model through an evaluation of its interpretable features
Authors: Francesco Ventura, Tania Cerquitelli
Date: 2019-07-31 00:00:00

Algorithms are powerful and necessary tools behind a large part of the information we use every day. However, they may introduce new sources of bias, discrimination and other unfair practices that affect people who are unaware of it. Greater algorithm transparency is indispensable to provide more credible and reliable services. Moreover, requiring developers to design transparent algorithm-driven applications allows them to keep the model accessible and human understandable, increasing the trust of end users. In this paper we present EBAnO, a new engine able to produce prediction-local explanations for a black-box model exploiting interpretable feature perturbations. EBAnO exploits the hypercolumns representation together with the cluster analysis to identify a set of interpretable features of images. Furthermore two indices have been proposed to measure the influence of input features on the final prediction made by a CNN model. EBAnO has been preliminarily tested on a set of heterogeneous images. The results highlight the effectiveness of EBAnO in explaining the CNN classification through the evaluation of interpretable features influence.
Challenge of Spatial Cognition for Deep Learning
Authors: Xi Zhang, Xiaolin Wu, Jun Du
Date: 2019-07-30 00:00:00
Tasks: Deep Learning, Spatial Reasoning

Given the success of the deep convolutional neural networks (DCNNs) in applications of visual recognition and classification, it would be tantalizing to test if DCNNs can also learn spatial concepts, such as straightness, convexity, left/right, front/back, relative size, aspect ratio, polygons, etc., from varied visual examples of these concepts that are simple and yet vital for spatial reasoning. Much to our dismay, extensive experiments of the type of cognitive psychology demonstrate that the data-driven deep learning (DL) cannot see through superficial variations in visual representations and grasp the spatial concept in abstraction. The root cause of failure turns out to be the learning methodology, not the computational model of the neural network itself. By incorporating task-specific convolutional kernels, we are able to construct DCNNs for spatial cognition tasks that can generalize to input images not drawn from the same distribution of the training set. This work raises a precaution that without manually-incorporated priors or features DCCNs may fail spatial cognitive tasks at rudimentary level.
Face Recognition in Unconstrained Conditions: A Systematic Review
Authors: Andrew Jason Shepley
Date: 2019-07-12 00:00:00
Tasks: Face Recognition

Face recognition is a biometric which is attracting significant research, commercial and government interest, as it provides a discreet, non-intrusive way of detecting, and recognizing individuals, without need for the subject's knowledge or consent. This is due to reduced cost, and evolution in hardware and algorithms which have improved their ability to handle unconstrained conditions. Evidently affordable and efficient applications are required. However, there is much debate over which methods are most appropriate, particularly in the context of the growing importance of deep neural network-based face recognition systems. This systematic review attempts to provide clarity on both issues by organizing the plethora of research and data in this field to clarify current research trends, state-of-the-art methods, and provides an outline of their benefits and shortcomings. Overall, this research covered 1,330 relevant studies, showing an increase of over 200% in research interest in the field of face recognition over the past 6 years. Our results also demonstrated that deep learning methods are the prime focus of modern research due to improvements in hardware databases and increasing understanding of neural networks. In contrast, traditional methods have lost favor amongst researchers due to their inherent limitations in accuracy, and lack of efficiency when handling large amounts of data.
Gait recognition via deep learning of the center-of-pressure trajectory
Authors: Philippe Terrier
Date: 2019-07-24 00:00:00
Tasks: Gait Recognition, Transfer Learning

The fact that every human has a distinctive walking style has prompted a proposal to use gait recognition as an identification criterion. Using end-to-end learning, I investigated whether the center-of-pressure trajectory is sufficiently unique to identify a person with a high certainty. Thirty-six adults walked on a treadmill equipped with a force platform that recorded the positions of the center of pressure. The raw two-dimensional signals were sliced into segments of two gait cycles. A set of 20,250 segments from 30 subjects was used to configure and train convolutional neural networks (CNNs). The best CNN classified a separate set containing 2,250 segments with 99.9% overall accuracy. A second set of 4,500 segments from the six remaining subjects was then used for transfer learning. Several small subsamples of this set were selected randomly and used for fine tuning. Training with two segments per subject was sufficient to achieve 100% accuracy. The results suggest that every person produces a unique trajectory of underfoot pressures and that CNNs can learn the distinctive features of these trajectories. Using transfer learning, a few strides could be sufficient to learn and identify new gaits.
Reproduction study using public data of: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs
Authors: Mike Voets, Kajsa Møllersen, Lars Ailo Bongo
Date: 2019-06-06 00:00:00
Proceeding: PLOS ONE 2019 6
Tasks: Deep Learning, Diabetic Retinopathy Grading, Medical Image Analysis

We have attempted to reproduce the results in Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs, published in JAMA 2016; 316(22), using publicly available data sets. We re-implemented the main method in the original study since the source code is not available. The original study used non-public fundus images from EyePACS and three hospitals in India for training. We used a different EyePACS data set from Kaggle. The original study used the benchmark data set Messidor-2 to evaluate the algorithm’s performance. We used another distribution of the Messidor-2 data set, since the original data set is no longer available. In the original study, ophthalmologists re-graded all images for diabetic retinopathy, macular edema, and image gradability. We have one diabetic retinopathy grade per image for our data sets, and we assessed image gradability ourselves. We were not able to reproduce the original study’s results with publicly available data. Our algorithm’s area under the receiver operating characteristic curve (AUC) of 0.951 (95% CI, 0.947-0.956) on the Kaggle EyePACS test set and 0.853 (95% CI, 0.835-0.871) on Messidor-2 did not come close to the reported AUC of 0.99 on both test sets in the original study. This may be caused by the use of a single grade per image, or different data. This study shows the challenges of reproducing deep learning method results, and the need for more replication and reproduction studies to validate deep learning methods, especially for medical image analysis. Our source code and instructions are available at: https://github.com/mikevoets/jama16-retina-replication.
User mode selection of NOMA based D2D communication for maximum sum-revenue
Authors: Linglin Kong
Date: 2019-06-10 00:00:00

Device to Device (D2D) communication underlying cellular communication can improve the utilization efficiency of resources, and the cellular user equipment (C-UE) can utilize the rental resources for D2D user equipment (D-UE) to obtain revenue. In 5g communication, non-orthogonal multiple access (NOMA) is another promising technology. In this paper, cellular users can choose to apply orthogonal multiple access (OMA) mode or NOMA mode to rent their own resources to get the maximum benefit. We proposed a dual-arm bandit machine model to solve this mode selection game and proved the effectiveness of our scheme through simulation results.
Towards Debiasing Fact Verification Models
Authors: Tal Schuster, Darsh J Shah, Yun Jie Serene Yeo, Daniel Filizzola, Enrico Santus, Regina Barzilay
Date: 2019-08-14 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Fact Verification

Fact verification requires validating a claim in the context of evidence. We show, however, that in the popular FEVER dataset this might not necessarily be the case. Claim-only classifiers perform competitively with top evidence-aware models. In this paper, we investigate the cause of this phenomenon, identifying strong cues for predicting labels solely based on the claim, without considering any evidence. We create an evaluation set that avoids those idiosyncrasies. The performance of FEVER-trained models significantly drops when evaluated on this test set. Therefore, we introduce a regularization method which alleviates the effect of bias in the training data, obtaining improvements on the newly created test set. This work is a step towards a more sound evaluation of reasoning capabilities in fact verification models.
Skill Transfer in Deep Reinforcement Learning under Morphological Heterogeneity
Authors: Yang Hu, Giovanni Montana
Date: 2019-08-14 00:00:00
Tasks: Decoder, Deep Reinforcement Learning, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Transfer learning methods for reinforcement learning (RL) domains facilitate the acquisition of new skills using previously acquired knowledge. The vast majority of existing approaches assume that the agents have the same design, e.g. same shape and action spaces. In this paper we address the problem of transferring previously acquired skills amongst morphologically different agents (MDAs). For instance, assuming that a bipedal agent has been trained to move forward, could this skill be transferred on to a one-leg hopper so as to make its training process for the same task more sample efficient? We frame this problem as one of subspace learning whereby we aim to infer latent factors representing the control mechanism that is common between MDAs. We propose a novel paired variational encoder-decoder model, PVED, that disentangles the control of MDAs into shared and agent-specific factors. The shared factors are then leveraged for skill transfer using RL. Theoretically, we derive a theorem indicating how the performance of PVED depends on the shared factors and agent morphologies. Experimentally, PVED has been extensively validated on four MuJoCo environments. We demonstrate its performance compared to a state-of-the-art approach and several ablation cases, visualize and interpret the hidden factors, and identify avenues for future improvements.
AutoCorrect: Deep Inductive Alignment of Noisy Geometric Annotations
Authors: Honglie Chen, Weidi Xie, Andrea Vedaldi, Andrew Zisserman
Date: 2019-08-14 00:00:00
Tasks: Object

We propose AutoCorrect, a method to automatically learn object-annotation alignments from a dataset with annotations affected by geometric noise. The method is based on a consistency loss that enables deep neural networks to be trained, given only noisy annotations as input, to correct the annotations. When some noise-free annotations are available, we show that the consistency loss reduces to a stricter self-supervised loss. We also show that the method can implicitly leverage object symmetries to reduce the ambiguity arising in correcting noisy annotations. When multiple object-annotation pairs are present in an image, we introduce a spatial memory map that allows the network to correct annotations sequentially, one at a time, while accounting for all other annotations in the image and corrections performed so far. Through ablation, we show the benefit of these contributions, demonstrating excellent results on geo-spatial imagery. Specifically, we show results using a new Railway tracks dataset as well as the public INRIA Buildings benchmarks, achieving new state-of-the-art results for the latter.
Few-Shot Learning with Global Class Representations
Authors: Tiange Luo, Aoxue Li, Tao Xiang, Weiran Huang, Li-Wei Wang
Date: 2019-08-14 00:00:00
Proceeding: ICCV 2019 10
Tasks: Few-Shot Image Classification, Few-Shot Learning, Generalized Few-Shot Classification, Meta-Learning

In this paper, we propose to tackle the challenging few-shot learning (FSL) problem by learning global class representations using both base and novel class training samples. In each training episode, an episodic class mean computed from a support set is registered with the global representation via a registration module. This produces a registered global class representation for computing the classification loss using a query set. Though following a similar episodic training pipeline as existing meta learning based approaches, our method differs significantly in that novel class training samples are involved in the training from the beginning. To compensate for the lack of novel class training samples, an effective sample synthesis strategy is developed to avoid overfitting. Importantly, by joint base-novel class training, our approach can be easily extended to a more practical yet challenging FSL setting, i.e., generalized FSL, where the label space of test data is extended to both base and novel classes. Extensive experiments show that our approach is effective for both of the two FSL settings.
Continuous Control for High-Dimensional State Spaces: An Interactive Learning Approach
Authors: Rodrigo Pérez-Dattari, Carlos Celemin, Javier Ruiz-del-Solar, Jens Kober
Date: 2019-08-14 00:00:00
Tasks: continuous-control, Continuous Control, Decision Making, Deep Reinforcement Learning, Reinforcement Learning, Vocal Bursts Intensity Prediction

Deep Reinforcement Learning (DRL) has become a powerful methodology to solve complex decision-making problems. However, DRL has several limitations when used in real-world problems (e.g., robotics applications). For instance, long training times are required and cannot be accelerated in contrast to simulated environments, and reward functions may be hard to specify/model and/or to compute. Moreover, the transfer of policies learned in a simulator to the real-world has limitations (reality gap). On the other hand, machine learning methods that rely on the transfer of human knowledge to an agent have shown to be time efficient for obtaining well performing policies and do not require a reward function. In this context, we analyze the use of human corrective feedback during task execution to learn policies with high-dimensional state spaces, by using the D-COACH framework, and we propose new variants of this framework. D-COACH is a Deep Learning based extension of COACH (COrrective Advice Communicated by Humans), where humans are able to shape policies through corrective advice. The enhanced version of D-COACH, which is proposed in this paper, largely reduces the time and effort of a human for training a policy. Experimental results validate the efficiency of the D-COACH framework in three different problems (simulated and with real robots), and show that its enhanced version reduces the human training effort considerably, and makes it feasible to learn policies within periods of time in which a DRL agent do not reach any improvement.