pwc-archive/papers-with-abstracts huggingface dataset:
AutoBlock: A Hands-off Blocking Framework for Entity Matching
Authors: Wei Zhang, Hao Wei, Bunyamin Sisman, Xin Luna Dong, Christos Faloutsos, David Page
Date: 2019-12-07 00:00:00
Tasks: Blocking, Entity Resolution, Representation Learning

Entity matching seeks to identify data records over one or multiple data sources that refer to the same real-world entity. Virtually every entity matching task on large datasets requires blocking, a step that reduces the number of record pairs to be matched. However, most of the traditional blocking methods are learning-free and key-based, and their successes are largely built on laborious human effort in cleaning data and designing blocking keys. In this paper, we propose AutoBlock, a novel hands-off blocking framework for entity matching, based on similarity-preserving representation learning and nearest neighbor search. Our contributions include: (a) Automation: AutoBlock frees users from laborious data cleaning and blocking key tuning. (b) Scalability: AutoBlock has a sub-quadratic total time complexity and can be easily deployed for millions of records. (c) Effectiveness: AutoBlock outperforms a wide range of competitive baselines on multiple large-scale, real-world datasets, especially when datasets are dirty and/or unstructured.
Principal Component Properties of Adversarial Samples
Authors: Malhar Jere, Sandro Herbig, Christine Lind, Farinaz Koushanfar
Date: 2019-12-07 00:00:00
Tasks: image-classification, Image Classification

Deep Neural Networks for image classification have been found to be vulnerable to adversarial samples, which consist of sub-perceptual noise added to a benign image that can easily fool trained neural networks, posing a significant risk to their commercial deployment. In this work, we analyze adversarial samples through the lens of their contributions to the principal components of each image, which is different than prior works in which authors performed PCA on the entire dataset. We investigate a number of state-of-the-art deep neural networks trained on ImageNet as well as several attacks for each of the networks. Our results demonstrate empirically that adversarial samples across several attacks have similar properties in their contributions to the principal components of neural network inputs. We propose a new metric for neural networks to measure their robustness to adversarial samples, termed the (k,p) point. We utilize this metric to achieve 93.36% accuracy in detecting adversarial samples independent of architecture and attack type for models trained on ImageNet.
Information-geometric optimization with natural selection
Authors: Jakub Otwinowski, Colin LaMont
Date: 2019-12-06 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary algorithms, inspired by natural evolution, aim to optimize difficult objective functions without computing derivatives. Here we detail the relationship between population genetics and evolutionary optimization and formulate a new evolutionary algorithm. Optimization of a continuous objective function is analogous to searching for high fitness phenotypes on a fitness landscape. We summarize how natural selection moves a population along the non-euclidean gradient that is induced by the population on the fitness landscape (the natural gradient). Under normal approximations common in quantitative genetics, we show how selection is related to Newton's method in optimization. We find that intermediate selection is most informative of the fitness landscape. We describe the generation of new phenotypes and introduce an operator that recombines the whole population to generate variants that preserve normal statistics. Finally, we introduce a proof-of-principle algorithm that combines natural selection, our recombination operator, and an adaptive method to increase selection. Our algorithm is similar to covariance matrix adaptation and natural evolutionary strategies in optimization, and has similar performance. The algorithm is extremely simple in implementation with no matrix inversion or factorization, does not require storing a covariance matrix, and may form the basis of more general model-based optimization algorithms with natural gradient updates.
Re-Translation Strategies For Long Form, Simultaneous, Spoken Language Translation
Authors: Naveen Arivazhagan, Colin Cherry, Te I, Wolfgang Macherey, Pallavi Baljekar, George Foster
Date: 2019-12-06 00:00:00
Tasks: Form, Machine Translation, speech-recognition, Speech Recognition, Speech-to-Text, Translation

We investigate the problem of simultaneous machine translation of long-form speech content. We target a continuous speech-to-text scenario, generating translated captions for a live audio feed, such as a lecture or play-by-play commentary. As this scenario allows for revisions to our incremental translations, we adopt a re-translation approach to simultaneous translation, where the source is repeatedly translated from scratch as it grows. This approach naturally exhibits very low latency and high final quality, but at the cost of incremental instability as the output is continuously refined. We experiment with a pipeline of industry-grade speech recognition and translation tools, augmented with simple inference heuristics to improve stability. We use TED Talks as a source of multilingual test data, developing our techniques on English-to-German spoken language translation. Our minimalist approach to simultaneous translation allows us to easily scale our final evaluation to six more target languages, dramatically improving incremental stability for all of them.
Deep Distance Transform for Tubular Structure Segmentation in CT Scans
Authors: Yan Wang, Xu Wei, Fengze Liu, Jieneng Chen, Yuyin Zhou, Wei Shen, Elliot K. Fishman, Alan L. Yuille
Date: 2019-12-06 00:00:00
Proceeding: CVPR 2020 6
Tasks: Segmentation

Tubular structure segmentation in medical images, e.g., segmenting vessels in CT scans, serves as a vital step in the use of computers to aid in screening early stages of related diseases. But automatic tubular structure segmentation in CT scans is a challenging problem, due to issues such as poor contrast, noise and complicated background. A tubular structure usually has a cylinder-like shape which can be well represented by its skeleton and cross-sectional radii (scales). Inspired by this, we propose a geometry-aware tubular structure segmentation method, Deep Distance Transform (DDT), which combines intuitions from the classical distance transform for skeletonization and modern deep segmentation networks. DDT first learns a multi-task network to predict a segmentation mask for a tubular structure and a distance map. Each value in the map represents the distance from each tubular structure voxel to the tubular structure surface. Then the segmentation mask is refined by leveraging the shape prior reconstructed from the distance map. We apply our DDT on six medical image datasets. The experiments show that (1) DDT can boost tubular structure segmentation performance significantly (e.g., over 13% improvement measured by DSC for pancreatic duct segmentation), and (2) DDT additionally provides a geometrical measurement for a tubular structure, which is important for clinical diagnosis (e.g., the cross-sectional scale of a pancreatic duct can be an indicator for pancreatic cancer).
Sparse and redundant signal representations for x-ray computed tomography
Authors: Davood Karimi
Date: 2019-12-06 00:00:00
Tasks: Computed Tomography (CT)

Image models are central to all image processing tasks. The great advancements in digital image processing would not have been made possible without powerful models which, themselves, have evolved over time. In the past decade, patch-based models have emerged as one of the most effective models for natural images. Patch-based methods have outperformed other competing methods in many image processing tasks. These developments have come at a time when greater availability of powerful computational resources and growing concerns over the health risks of the ionizing radiation encourage research on image processing algorithms for computed tomography (CT). The goal of this paper is to explain the principles of patch-based methods and to review some of their recent applications in CT. We review the central concepts in patch-based image processing and explain some of the state-of-the-art algorithms, with a focus on aspects that are more relevant to CT. Then, we review some of the recent application of patch-based methods in CT.
Med2Meta: Learning Representations of Medical Concepts with Meta-Embeddings
Authors: Shaika Chowdhury, Chenwei Zhang, Philip S. Yu, Yuan Luo
Date: 2019-12-06 00:00:00
Tasks: Decision Making

Distributed representations of medical concepts have been used to support downstream clinical tasks recently. Electronic Health Records (EHR) capture different aspects of patients' hospital encounters and serve as a rich source for augmenting clinical decision making by learning robust medical concept embeddings. However, the same medical concept can be recorded in different modalities (e.g., clinical notes, lab results)-with each capturing salient information unique to that modality-and a holistic representation calls for relevant feature ensemble from all information sources. We hypothesize that representations learned from heterogeneous data types would lead to performance enhancement on various clinical informatics and predictive modeling tasks. To this end, our proposed approach makes use of meta-embeddings, embeddings aggregated from learned embeddings. Firstly, modality-specific embeddings for each medical concept is learned with graph autoencoders. The ensemble of all the embeddings is then modeled as a meta-embedding learning problem to incorporate their correlating and complementary information through a joint reconstruction. Empirical results of our model on both quantitative and qualitative clinical evaluations have shown improvements over state-of-the-art embedding models, thus validating our hypothesis.
Audio-attention discriminative language model for ASR rescoring
Authors: Ankur Gandhe, Ariya Rastrow
Date: 2019-12-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, model, speech-recognition, Speech Recognition

End-to-end approaches for automatic speech recognition (ASR) benefit from directly modeling the probability of the word sequence given the input audio stream in a single neural network. However, compared to conventional ASR systems, these models typically require more data to achieve comparable results. Well-known model adaptation techniques, to account for domain and style adaptation, are not easily applicable to end-to-end systems. Conventional HMM-based systems, on the other hand, have been optimized for various production environments and use cases. In this work, we propose to combine the benefits of end-to-end approaches with a conventional system using an attention-based discriminative language model that learns to rescore the output of a first-pass ASR system. We show that learning to rescore a list of potential ASR outputs is much simpler than learning to generate the hypothesis. The proposed model results in 8% improvement in word error rate even when the amount of training data is a fraction of data used for training the first-pass system.
Bilinear Models for Machine Learning
Authors: Tayssir Doghri, Leszek Szczecinski, Jacob Benesty, Amar Mitiche
Date: 2019-12-06 00:00:00
Tasks: BIG-bench Machine Learning

In this work we define and analyze the bilinear models which replace the conventional linear operation used in many building blocks of machine learning (ML). The main idea is to devise the ML algorithms which are adapted to the objects they treat. In the case of monochromatic images, we show that the bilinear operation exploits better the structure of the image than the conventional linear operation which ignores the spatial relationship between the pixels. This translates into significantly smaller number of parameters required to yield the same performance. We show numerical examples of classification in the MNIST data set.
The surprising little effectiveness of cooperative algorithms in parallel problem solving
Authors: Sandro M. Reia, Larissa F. Aquino, José F. Fontanari
Date: 2019-12-06 00:00:00
Tasks: Evolutionary Algorithms

Biological and cultural inspired optimization algorithms are nowadays part of the basic toolkit of a great many research domains. By mimicking processes in nature and animal societies, these general-purpose search algorithms promise to deliver optimal or near-optimal solutions using hardly any information on the optimization problems they are set to tackle. Here we study the performances of a cultural-inspired algorithm -- the imitative learning search -- as well as of asexual and sexual variants of evolutionary algorithms in finding the global maxima of NK-fitness landscapes. The main performance measure is the total number of agent updates required by the algorithms to find those global maxima and the baseline performance, which establishes the effectiveness of the cooperative algorithms, is set by the blind search in which the agents explore the problem space (binary strings) by flipping bits at random. We find that even for smooth landscapes that exhibit a single maximum, the evolutionary algorithms do not perform much better than the blind search due to the stochastic effects of the genetic roulette. The imitative learning is immune to this effect thanks to the deterministic choice of the fittest string in the population, which is used as a model for imitation. The tradeoff is that for rugged landscapes the imitative learning search is more prone to be trapped in local maxima than the evolutionary algorithms. In fact, in the case of rugged landscapes with a mild density of local maxima, the blind search either beats or matches the cooperative algorithms regardless of whether the task is to find the global maximum or to find the fittest state within a given runtime.
Deep Reinforcement Learning for Routing a Heterogeneous Fleet of Vehicles
Authors: Jose Manuel Vera, Andres G. Abad
Date: 2019-12-06 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Motivated by the promising advances of deep-reinforcement learning (DRL) applied to cooperative multi-agent systems we propose a model and learning procedure to solve the Capacitated Multi-Vehicle Routing Problem (CMVRP) with fixed fleet size. Our learning procedure follows a centralized-training and decentralized-execution paradigm. We empirically test our model and showed its capability for producing near-optimal solutions through cooperative actions. In large instances, our model generates better solutions than other commonly used heuristics. Additionally, our model can solve arbitrary instances of the CMVRP without requiring re-training.
Explaining Sequence-Level Knowledge Distillation as Data-Augmentation for Neural Machine Translation
Authors: Mitchell A. Gordon, Kevin Duh
Date: 2019-12-06 00:00:00
Tasks: Data Augmentation, Knowledge Distillation, Machine Translation, Model Compression, Translation

Sequence-level knowledge distillation (SLKD) is a model compression technique that leverages large, accurate teacher models to train smaller, under-parameterized student models. Why does pre-processing MT data with SLKD help us train smaller models? We test the common hypothesis that SLKD addresses a capacity deficiency in students by "simplifying" noisy data points and find it unlikely in our case. Models trained on concatenations of original and "simplified" datasets generalize just as well as baseline SLKD. We then propose an alternative hypothesis under the lens of data augmentation and regularization. We try various augmentation strategies and observe that dropout regularization can become unnecessary. Our methods achieve BLEU gains of 0.7-1.2 on TED Talks.
ClusterFit: Improving Generalization of Visual Representations
Authors: Xueting Yan, Ishan Misra, Abhinav Gupta, Deepti Ghadiyaram, Dhruv Mahajan
Date: 2019-12-06 00:00:00
Proceeding: CVPR 2020 6
Tasks: Action Classification, Clustering, Image Classification, Transfer Learning

Pre-training convolutional neural networks with weakly-supervised and self-supervised strategies is becoming increasingly popular for several computer vision tasks. However, due to the lack of strong discriminative signals, these learned representations may overfit to the pre-training objective (e.g., hashtag prediction) and not generalize well to downstream tasks. In this work, we present a simple strategy - ClusterFit (CF) to improve the robustness of the visual representations learned during pre-training. Given a dataset, we (a) cluster its features extracted from a pre-trained network using k-means and (b) re-train a new network from scratch on this dataset using cluster assignments as pseudo-labels. We empirically show that clustering helps reduce the pre-training task-specific information from the extracted features thereby minimizing overfitting to the same. Our approach is extensible to different pre-training frameworks -- weak- and self-supervised, modalities -- images and videos, and pre-training tasks -- object and action classification. Through extensive transfer learning experiments on 11 different target datasets of varied vocabularies and granularities, we show that ClusterFit significantly improves the representation quality compared to the state-of-the-art large-scale (millions / billions) weakly-supervised image and video models and self-supervised image models.
Robust Deep Graph Based Learning for Binary Classification
Authors: Minxiang Ye, Vladimir Stankovic, Lina Stankovic, Gene Cheung
Date: 2019-12-06 00:00:00
Tasks: Binary Classification, Classification, General Classification

Convolutional neural network (CNN)-based feature learning has become state of the art, since given sufficient training data, CNN can significantly outperform traditional methods for various classification tasks. However, feature learning becomes more difficult if some training labels are noisy. With traditional regularization techniques, CNN often overfits to the noisy training labels, resulting in sub-par classification performance. In this paper, we propose a robust binary classifier, based on CNNs, to learn deep metric functions, which are then used to construct an optimal underlying graph structure used to clean noisy labels via graph Laplacian regularization (GLR). GLR is posed as a convex maximum a posteriori (MAP) problem solved via convex quadratic programming (QP). To penalize samples around the decision boundary, we propose two regularized loss functions for semi-supervised learning. The binary classification experiments on three datasets, varying in number and type of features, demonstrate that given a noisy training dataset, our proposed networks outperform several state-of-the-art classifiers, including label-noise robust support vector machine, CNNs with three different robust loss functions, model-based GLR, and dynamic graph CNN classifiers.
MRI correlates of chronic symptoms in mild traumatic brain injury
Authors: Cailey I. Kerley, Kurt G. Schilling, Justin Blaber, Beth Miller, Allen Newton, Adam W. Anderson, Bennett A. Landman, Tonia S. Rex
Date: 2019-12-06 00:00:00

Veterans with mild traumatic brain injury (mTBI) have reported auditory and visual dysfunction that persists beyond the acute incident. The etiology behind these symptoms is difficult to characterize with current clinical imaging. These functional deficits may be caused by shear injury or micro-bleeds, which can be detected with special imaging modalities. We explore these hypotheses in a pilot study of multi-parametric MRI. We extract over 1,000 imaging and clinical metrics and project them to a low-dimensional space, where we can discriminate between healthy controls and patients with mTBI. We also show correlations between the metric representations and patient symptoms.
Deep Bayesian Recurrent Neural Networks for Somatic Variant Calling in Cancer
Authors: Geoffroy Dubourg-Felonneau, Omar Darwish, Christopher Parsons, Dami Rebergen, John W Cassidy, Nirmesh Patel, Harry W Clifford
Date: 2019-12-06 00:00:00

The emerging field of precision oncology relies on the accurate pinpointing of alterations in the molecular profile of a tumor to provide personalized targeted treatments. Current methodologies in the field commonly include the application of next generation sequencing technologies to a tumor sample, followed by the identification of mutations in the DNA known as somatic variants. The differentiation of these variants from sequencing error poses a classic classification problem, which has traditionally been approached with Bayesian statistics, and more recently with supervised machine learning methods such as neural networks. Although these methods provide greater accuracy, classic neural networks lack the ability to indicate the confidence of a variant call. In this paper, we explore the performance of deep Bayesian neural networks on next generation sequencing data, and their ability to give probability estimates for somatic variant calls. In addition to demonstrating similar performance in comparison to standard neural networks, we show that the resultant output probabilities make these better suited to the disparate and highly-variable sequencing data-sets these models are likely to encounter in the real world. We aim to deliver algorithms to oncologists for which model certainty better reflects accuracy, for improved clinical application. By moving away from point estimates to reliable confidence intervals, we expect the resultant clinical and treatment decisions to be more robust and more informed by the underlying reality of the tumor molecular profile.
Community Detection and Matrix Completion with Social and Item Similarity Graphs
Authors: Qiaosheng Zhang, Vincent Y. F. Tan, Changho Suh
Date: 2019-12-06 00:00:00
Tasks: Community Detection, Matrix Completion, Stochastic Block Model

We consider the problem of recovering a binary rating matrix as well as clusters of users and items based on a partially observed matrix together with side-information in the form of social and item similarity graphs. These two graphs are both generated according to the celebrated stochastic block model (SBM). We develop lower and upper bounds on sample complexity that match for various scenarios. Our information-theoretic results quantify the benefits of the availability of the social and item similarity graphs. Further analysis reveals that under certain scenarios, the social and item similarity graphs produce an interesting synergistic effect. This means that observing two graphs is strictly better than observing just one in terms of reducing the sample complexity.
Visualizing Deep Neural Networks for Speech Recognition with Learned Topographic Filter Maps
Authors: Andreas Krug, Sebastian Stober
Date: 2019-12-06 00:00:00
Tasks: speech-recognition, Speech Recognition

The uninformative ordering of artificial neurons in Deep Neural Networks complicates visualizing activations in deeper layers. This is one reason why the internal structure of such models is very unintuitive. In neuroscience, activity of real brains can be visualized by highlighting active regions. Inspired by those techniques, we train a convolutional speech recognition model, where filters are arranged in a 2D grid and neighboring filters are similar to each other. We show, how those topographic filter maps visualize artificial neuron activations more intuitively. Moreover, we investigate, whether this causes phoneme-responsive neurons to be grouped in certain regions of the topographic map.