pwc-archive/papers-with-abstracts huggingface dataset:
Searching to Exploit Memorization Effect in Learning from Corrupted Labels
Authors: Quanming Yao, Hansi Yang, Bo Han, Gang Niu, James Kwok
Date: 2019-11-06 00:00:00
Tasks: AutoML, Memorization

Sample selection approaches are popular in robust learning from noisy labels. However, how to properly control the selection process so that deep networks can benefit from the memorization effect is a hard problem. In this paper, motivated by the success of automated machine learning (AutoML), we model this issue as a function approximation problem. Specifically, we design a domain-specific search space based on general patterns of the memorization effect and propose a novel Newton algorithm to solve the bi-level optimization problem efficiently. We further provide theoretical analysis of the algorithm, which ensures a good approximation to critical points. Experiments are performed on benchmark data sets. Results demonstrate that the proposed method is much better than the state-of-the-art noisy-label-learning approaches, and also much more efficient than existing AutoML algorithms.
Randomized Computer Vision Approaches for Pattern Recognition in Timepix and Timepix3 Detectors
Authors: Petr Mánek, Benedikt Bergmann, Petr Burian, Lukáš Meduna, Stanislav Pospíšil, Michal Suk
Date: 2019-11-06 00:00:00

Timepix and Timepix3 are hybrid pixel detectors ($256\times 256$ pixels), capable of tracking ionizing particles as isolated clusters of pixels. To efficiently analyze such clusters at potentially high rates, we introduce multiple randomized pattern recognition algorithms inspired by computer vision. Offering desirable probabilistic bounds on accuracy and complexity, the presented methods are well-suited for use in real-time applications, and some may even be modified to tackle trans-dimensional problems. In Timepix detectors, which do not support data-driven acquisition, they have been shown to correctly separate clusters of overlapping tracks. In Timepix3 detectors, simultaneous acquisition of Time-of-Arrival (ToA) and Time-over-Threshold (ToT) pixel data enables reconstruction of the depth, transitioning from 2D to 3D point clouds. The presented algorithms have been tested on simulated inputs, test beam data from the Heidelberg Ion therapy Center and the Super Proton Synchrotron and were applied to data acquired in the MoEDAL and ATLAS experiments at CERN.
Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds
Authors: Tassilo Klein, Moin Nabi
Date: 2019-11-06 00:00:00
Tasks: Decoder, Language Modelling, Question Answering, Question Generation, Question-Generation, Small Data Image Classification

Automatic question generation aims at the generation of questions from a context, with the corresponding answers being sub-spans of the given passage. Whereas, most of the methods mostly rely on heuristic rules to generate questions, more recently also neural network approaches have been proposed. In this work, we propose a variant of the self-attention Transformer network architectures model to generate meaningful and diverse questions. To this end, we propose an easy to use model consisting of the conjunction of the Transformer decoder GPT-2 model with Transformer encoder BERT for the downstream task for question answering. The model is trained in an end-to-end fashion, where the language model is trained to produce a question-answer-aware input representation that facilitates to generate an answer focused question. Our result of neural question generation from text on the SQuAD 1.1 dataset suggests that our method can produce semantically correct and diverse questions. Additionally, we assessed the performance of our proposed method for the downstream task of question answering. The analysis shows that our proposed generation & answering collaboration framework relatively improves both tasks and is particularly powerful in the semi-supervised setup. The results further suggest a robust and comparably lean pipeline facilitating question generation in the small-data regime.
Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings
Authors: Paul Bergmann, Michael Fauser, David Sattlegger, Carsten Steger
Date: 2019-11-06 00:00:00
Proceeding: CVPR 2020 6
Tasks: Anomaly Detection, Anomaly Segmentation, Descriptive, Segmentation, Unsupervised Anomaly Detection

We introduce a powerful student-teacher framework for the challenging problem of unsupervised anomaly detection and pixel-precise anomaly segmentation in high-resolution images. Student networks are trained to regress the output of a descriptive teacher network that was pretrained on a large dataset of patches from natural images. This circumvents the need for prior data annotation. Anomalies are detected when the outputs of the student networks differ from that of the teacher network. This happens when they fail to generalize outside the manifold of anomaly-free training data. The intrinsic uncertainty in the student networks is used as an additional scoring function that indicates anomalies. We compare our method to a large number of existing deep learning based methods for unsupervised anomaly detection. Our experiments demonstrate improvements over state-of-the-art methods on a number of real-world datasets, including the recently introduced MVTec Anomaly Detection dataset that was specifically designed to benchmark anomaly segmentation algorithms.
Convolutional Neural Network for Multipath Detection in GNSS Receivers
Authors: Evgenii Munin, Antoine Blais, Nicolas Couellan
Date: 2019-11-06 00:00:00
Tasks: Anomaly Detection

Global Navigation Satellite System (GNSS) signals are subject to different kinds of events causing significant errors in positioning. This work explores the application of Machine Learning (ML) methods of anomaly detection applied to GNSS receiver signals. More specifically, our study focuses on multipath contamination, using samples of the correlator output signal. The GPS L1 C/A signal data is used and sourced directly from the correlator output. To extract the important features and patterns from such data, we use deep convolutional neural networks (CNN), which have proven to be efficient in image analysis in particular. To take advantage of CNN, the correlator output signal is mapped as a 2D input image and fed to the convolutional layers of a neural network. The network automatically extracts the relevant features from the input samples and proceeds with the multipath detection. We train the CNN using synthetic signals. To optimize the model architecture with respect to the GNSS correlator complexity, the evaluation of the CNN performance is done as a function of the number of correlator output points.
Statistical physics of unsupervised learning with prior knowledge in neural networks
Authors: Tianqi Hou, Haiping Huang
Date: 2019-11-06 00:00:00

Integrating sensory inputs with prior beliefs from past experiences in unsupervised learning is a common and fundamental characteristic of brain or artificial neural computation. However, a quantitative role of prior knowledge in unsupervised learning remains unclear, prohibiting a scientific understanding of unsupervised learning. Here, we propose a statistical physics model of unsupervised learning with prior knowledge, revealing that the sensory inputs drive a series of continuous phase transitions related to spontaneous intrinsic-symmetry breaking. The intrinsic symmetry includes both reverse symmetry and permutation symmetry, commonly observed in most artificial neural networks. Compared to the prior-free scenario, the prior reduces more strongly the minimal data size triggering the reverse symmetry breaking transition, and moreover, the prior merges, rather than separates, permutation symmetry breaking phases. We claim that the prior can be learned from data samples, which in physics corresponds to a two-parameter Nishimori constraint. This work thus reveals mechanisms about the influence of the prior on unsupervised learning.
Melanoma detection with electrical impedance spectroscopy and dermoscopy using joint deep learning models
Authors: Nils Gessert, Marcel Bengs, Alexander Schlaefer
Date: 2019-11-06 00:00:00
Tasks: BIG-bench Machine Learning, Lesion Classification

The initial assessment of skin lesions is typically based on dermoscopic images. As this is a difficult and time-consuming task, machine learning methods using dermoscopic images have been proposed to assist human experts. Other approaches have studied electrical impedance spectroscopy (EIS) as a basis for clinical decision support systems. Both methods represent different ways of measuring skin lesion properties as dermoscopy relies on visible light and EIS uses electric currents. Thus, the two methods might carry complementary features for lesion classification. Therefore, we propose joint deep learning models considering both EIS and dermoscopy for melanoma detection. For this purpose, we first study machine learning methods for EIS that incorporate domain knowledge and previously used heuristics into the design process. As a result, we propose a recurrent model with state-max-pooling which automatically learns the relevance of different EIS measurements. Second, we combine this new model with different convolutional neural networks that process dermoscopic images. We study ensembling approaches and also propose a cross-attention module guiding information exchange between the EIS and dermoscopy model. In general, combinations of EIS and dermoscopy clearly outperform models that only use either EIS or dermoscopy. We show that our attention-based, combined model outperforms other models with specificities of 34.4% (CI 31.3-38.4), 34.7% (CI 31.0-38.8) and 53.7% (CI 50.1-57.6) for dermoscopy, EIS and the combined model, respectively, at a clinically relevant sensitivity of 98%.
Nonverbal Robot Feedback for Human Teachers
Authors: Sandy H. Huang, Isabella Huang, Ravi Pandya, Anca D. Dragan
Date: 2019-11-06 00:00:00

Robots can learn preferences from human demonstrations, but their success depends on how informative these demonstrations are. Being informative is unfortunately very challenging, because during teaching, people typically get no transparency into what the robot already knows or has learned so far. In contrast, human students naturally provide a wealth of nonverbal feedback that reveals their level of understanding and engagement. In this work, we study how a robot can similarly provide feedback that is minimally disruptive, yet gives human teachers a better mental model of the robot learner, and thus enables them to teach more effectively. Our idea is that at any point, the robot can indicate what it thinks the correct next action is, shedding light on its current estimate of the human's preferences. We analyze how useful this feedback is, both in theory and with two user studies---one with a virtual character that tests the feedback itself, and one with a PR2 robot that uses gaze as the feedback mechanism. We find that feedback can be useful for improving both the quality of teaching and teachers' understanding of the robot's capability.
Improving reinforcement learning algorithms: towards optimal learning rate policies
Authors: Othmane Mounjid, Charles-Albert Lehalle
Date: 2019-11-06 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper investigates to what extent one can improve reinforcement learning algorithms. Our study is split in three parts. First, our analysis shows that the classical asymptotic convergence rate $O(1/\sqrt{N})$ is pessimistic and can be replaced by $O((\log(N)/N)^{\beta})$ with $\frac{1}{2}\leq \beta \leq 1$ and $N$ the number of iterations. Second, we propose a dynamic optimal policy for the choice of the learning rate $(\gamma_k)_{k\geq 0}$ used in stochastic approximation (SA). We decompose our policy into two interacting levels: the inner and the outer level. In the inner level, we present the \nameref{Alg:v_4_s} algorithm (for "PAst Sign Search") which, based on a predefined sequence $(\gamma^o_k)_{k\geq 0}$, constructs a new sequence $(\gamma^i_k)_{k\geq 0}$ whose error decreases faster. In the outer level, we propose an optimal methodology for the selection of the predefined sequence $(\gamma^o_k)_{k\geq 0}$. Third, we show empirically that our selection methodology of the learning rate outperforms significantly standard algorithms used in reinforcement learning (RL) in the three following applications: the estimation of a drift, the optimal placement of limit orders and the optimal execution of large number of shares.
Linear Support Vector Regression with Linear Constraints
Authors: Quentin Klopfenstein, Samuel Vaiter
Date: 2019-11-06 00:00:00
Tasks: regression

This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Minimal Optimization (SMO) algorithm for solving the optimization problem with linear constraints and prove its convergence. Then, practical performances of this estimator are shown on simulated and real datasets with different settings: non negative regression, regression onto the simplex for biomedical data and isotonic regression for weather forecast.
Enriching Conversation Context in Retrieval-based Chatbots
Authors: Amir Vakili Tahami, Azadeh Shakery
Date: 2019-11-06 00:00:00
Tasks: Information Retrieval, Retrieval, Text Matching

Work on retrieval-based chatbots, like most sequence pair matching tasks, can be divided into Cross-encoders that perform word matching over the pair, and Bi-encoders that encode the pair separately. The latter has better performance, however since candidate responses cannot be encoded offline, it is also much slower. Lately, multi-layer transformer architectures pre-trained as language models have been used to great effect on a variety of natural language processing and information retrieval tasks. Recent work has shown that these language models can be used in text-matching scenarios to create Bi-encoders that perform almost as well as Cross-encoders while having a much faster inference speed. In this paper, we expand upon this work by developing a sequence matching architecture that %takes into account contexts in the training dataset at inference time. utilizes the entire training set as a makeshift knowledge-base during inference. We perform detailed experiments demonstrating that this architecture can be used to further improve Bi-encoders performance while still maintaining a relatively high inference speed.
Boosting Object Recognition in Point Clouds by Saliency Detection
Authors: Marlon Marcon, Riccardo Spezialetti, Samuele Salti, Luciano Silva, Luigi Di Stefano
Date: 2019-11-06 00:00:00
Tasks: Object Recognition, Saliency Detection

Object recognition in 3D point clouds is a challenging task, mainly when time is an important factor to deal with, such as in industrial applications. Local descriptors are an amenable choice whenever the 6 DoF pose of recognized objects should also be estimated. However, the pipeline for this kind of descriptors is highly time-consuming. In this work, we propose an update to the traditional pipeline, by adding a preliminary filtering stage referred to as saliency boost. We perform tests on a standard object recognition benchmark by considering four keypoint detectors and four local descriptors, in order to compare time and recognition performance between the traditional pipeline and the boosted one. Results on time show that the boosted pipeline could turn out up to 5 times faster, with the recognition rate improving in most of the cases and exhibiting only a slight decrease in the others. These results suggest that the boosted pipeline can speed-up processing time substantially with limited impacts or even benefits in recognition accuracy.
Spatial Feature Extraction in Airborne Hyperspectral Images Using Local Spectral Similarity
Authors: Anand S Sahadevan, Arundhati Misra, Praveen Gupta
Date: 2019-11-06 00:00:00
Tasks: Clustering, Edge Detection

Local spectral similarity (LSS) algorithm has been developed for detecting homogeneous areas and edges in hyperspectral images (HSIs). The proposed algorithm transforms the 3-D data cube (within a spatial window) into a spectral similarity matrix by calculating the vector-similarity between the center pixel-spectrum and the neighborhood spectra. The final edge intensity is derived upon order statistics of the similarity matrix or spatial convolution of the similarity matrix with the spatial kernels. The LSS algorithm facilitates simultaneous use of spectral-spatial information for the edge detection by considering the spatial pattern of similar spectra within a spatial window. The proposed edge-detection method is tested on benchmark HSIs as well as the image obtained from Airborne-Visible-and-Infra-RedImaging-Spectrometer-Next-Generation (AVIRIS-NG). Robustness of the LSS method against multivariate Gaussian noise and low spatial resolution scenarios were also verified with the benchmark HSIs. Figure-of-merit, false-alarm-count and miss-count were applied to evaluate the performance of edge detection methods. Results showed that Fractional distance measure and Euclidean distance measure were able to detect the edges in HSIs more precisely as compared to other spectral similarity measures. The proposed method can be applied to radiance and reflectance data (whole spectrum) and it has shown good performance on principal component images as well. In addition, the proposed algorithm outperforms the traditional multichannel edge detectors in terms of both fastness, accuracy and the robustness. The experimental results also confirm that LSS can be applied as a pre-processing approach to reduce the errors in clustering as well as classification outputs.
Optimization with soft Dice can lead to a volumetric bias
Authors: Jeroen Bertels, David Robben, Dirk Vandermeulen, Paul Suetens
Date: 2019-11-06 00:00:00
Tasks: Medical Image Analysis, Segmentation

Segmentation is a fundamental task in medical image analysis. The clinical interest is often to measure the volume of a structure. To evaluate and compare segmentation methods, the similarity between a segmentation and a predefined ground truth is measured using metrics such as the Dice score. Recent segmentation methods based on convolutional neural networks use a differentiable surrogate of the Dice score, such as soft Dice, explicitly as the loss function during the learning phase. Even though this approach leads to improved Dice scores, we find that, both theoretically and empirically on four medical tasks, it can introduce a volumetric bias for tasks with high inherent uncertainty. As such, this may limit the method's clinical applicability.
Conditional Mutual Information Neural Estimator
Authors: Sina Molavipour, Germán Bassi, Mikael Skoglund
Date: 2019-11-06 00:00:00

Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on aggregated samples from the channel. Motivated by the fact that, in many communication schemes, the achievable transmission rate is determined by a conditional mutual information term, this paper focuses on neural-based estimators for this information-theoretic quantity. Our results are based on variational bounds for the KL-divergence and, in contrast to some previous works, we provide a mathematically rigorous lower bound. However, additional challenges with respect to the unconditional mutual information emerge due to the presence of a conditional density function which we address here.
Where is the Fake? Patch-Wise Supervised GANs for Texture Inpainting
Authors: Ahmed Ben Saad, Youssef Tamaazousti, Josselin Kherroubi, Alexis He
Date: 2019-11-06 00:00:00
Tasks: Image Inpainting, Missing Values

We tackle the problem of texture inpainting where the input images are textures with missing values along with masks that indicate the zones that should be generated. Many works have been done in image inpainting with the aim to achieve global and local consistency. But these works still suffer from limitations when dealing with textures. In fact, the local information in the image to be completed needs to be used in order to achieve local continuities and visually realistic texture inpainting. For this, we propose a new segmentor discriminator that performs a patch-wise real/fake classification and is supervised by input masks. During training, it aims to locate the fake and thus backpropagates consistent signal to the generator. We tested our approach on the publicly available DTD dataset and showed that it achieves state-of-the-art performances and better deals with local consistency than existing methods.
Robot navigation and target capturing using nature-inspired approaches in a dynamic environment
Authors: Devansh Verma, Priyansh Saxena, Ritu Tiwari
Date: 2019-11-06 00:00:00
Tasks: Robot Navigation

Path Planning and target searching in a three-dimensional environment is a challenging task in the field of robotics. It is an optimization problem as the path from source to destination has to be optimal. This paper aims to generate a collision-free trajectory in a dynamic environment. The path planning problem has sought to be of extreme importance in the military, search and rescue missions and in life-saving tasks. During its operation, the unmanned air vehicle operates in a hostile environment, and faster replanning is needed to reach the target as optimally as possible. This paper presents a novel approach of hierarchical planning using multiresolution abstract levels for faster replanning. Economic constraints like path length, total path planning time and the number of turns are taken into consideration that mandate the use of cost functions. Experimental results show that the hierarchical version of GSO gives better performance compared to the BBO, IWO and their hierarchical versions.
Predictive modeling of brain tumor: A Deep learning approach
Authors: Priyansh Saxena, Akshat Maheshwari, Saumil Maheshwari
Date: 2019-11-06 00:00:00
Tasks: Anatomy, Deep Learning, Transfer Learning

Image processing concepts can visualize the different anatomy structure of the human body. Recent advancements in the field of deep learning have made it possible to detect the growth of cancerous tissue just by a patient's brain Magnetic Resonance Imaging (MRI) scans. These methods require very high accuracy and meager false negative rates to be of any practical use. This paper presents a Convolutional Neural Network (CNN) based transfer learning approach to classify the brain MRI scans into two classes using three pre-trained models. The performances of these models are compared with each other. Experimental results show that the Resnet-50 model achieves the highest accuracy and least false negative rates as 95% and zero respectively. It is followed by VGG-16 and Inception-V3 model with an accuracy of 90% and 55% respectively.