pwc-archive/papers-with-abstracts huggingface dataset:
Universal Sound Separation
Authors: Ilya Kavalerov, Scott Wisdom, Hakan Erdogan, Brian Patton, Kevin Wilson, Jonathan Le Roux, John R. Hershey
Date: 2019-05-08 00:00:00
Tasks: Speech Enhancement, Speech Separation

Recent deep learning approaches have achieved impressive performance on speech enhancement and separation tasks. However, these approaches have not been investigated for separating mixtures of arbitrary sounds of different types, a task we refer to as universal sound separation, and it is unknown how performance on speech tasks carries over to non-speech tasks. To study this question, we develop a dataset of mixtures containing arbitrary sounds, and use it to investigate the space of mask-based separation architectures, varying both the overall network architecture and the framewise analysis-synthesis basis for signal transformations. These network architectures include convolutional long short-term memory networks and time-dilated convolution stacks inspired by the recent success of time-domain enhancement networks like ConvTasNet. For the latter architecture, we also propose novel modifications that further improve separation performance. In terms of the framewise analysis-synthesis basis, we explore both a short-time Fourier transform (STFT) and a learnable basis, as used in ConvTasNet. For both of these bases, we also examine the effect of window size. In particular, for STFTs, we find that longer windows (25-50 ms) work best for speech/non-speech separation, while shorter windows (2.5 ms) work best for arbitrary sounds. For learnable bases, shorter windows (2.5 ms) work best on all tasks. Surprisingly, for universal sound separation, STFTs outperform learnable bases. Our best methods produce an improvement in scale-invariant signal-to-distortion ratio of over 13 dB for speech/non-speech separation and close to 10 dB for universal sound separation.
Regression from Dependent Observations
Authors: Constantinos Daskalakis, Nishanth Dikkala, Ioannis Panageas
Date: 2019-05-08 00:00:00
Tasks: regression

The standard linear and logistic regression models assume that the response variables are independent, but share the same linear relationship to their corresponding vectors of covariates. The assumption that the response variables are independent is, however, too strong. In many applications, these responses are collected on nodes of a network, or some spatial or temporal domain, and are dependent. Examples abound in financial and meteorological applications, and dependencies naturally arise in social networks through peer effects. Regression with dependent responses has thus received a lot of attention in the Statistics and Economics literature, but there are no strong consistency results unless multiple independent samples of the vectors of dependent responses can be collected from these models. We present computationally and statistically efficient methods for linear and logistic regression models when the response variables are dependent on a network. Given one sample from a networked linear or logistic regression model and under mild assumptions, we prove strong consistency results for recovering the vector of coefficients and the strength of the dependencies, recovering the rates of standard regression under independent observations. We use projected gradient descent on the negative log-likelihood, or negative log-pseudolikelihood, and establish their strong convexity and consistency using concentration of measure for dependent random variables.
QSMGAN: Improved Quantitative Susceptibility Mapping using 3D Generative Adversarial Networks with Increased Receptive Field
Authors: Yi-Cheng Chen, Angela Jakary, Sivakami Avadiappan, Christopher P. Hess, Janine M. Lupo
Date: 2019-05-08 00:00:00

Quantitative susceptibility mapping (QSM) is a powerful MRI technique that has shown great potential in quantifying tissue susceptibility in numerous neurological disorders. However, the intrinsic ill-posed dipole inversion problem greatly affects the accuracy of the susceptibility map. We propose QSMGAN: a 3D deep convolutional neural network approach based on a 3D U-Net architecture with increased receptive field of the input phase compared to the output and further refined the network using the WGAN with gradient penalty training strategy. Our method generates accurate QSM maps from single orientation phase maps efficiently and performs significantly better than traditional non-learning-based dipole inversion algorithms. The generalization capability was verified by applying the algorithm to an unseen pathology--brain tumor patients with radiation-induced cerebral microbleeds.
Embarrassingly Shallow Autoencoders for Sparse Data
Authors: Harald Steck
Date: 2019-05-08 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Combining simple elements from the literature, we define a linear model that is geared toward sparse data, in particular implicit feedback data for recommender systems. We show that its training objective has a closed-form solution, and discuss the resulting conceptual insights. Surprisingly, this simple model achieves better ranking accuracy than various state-of-the-art collaborative-filtering approaches, including deep non-linear models, on most of the publicly available data-sets used in our experiments.
Multi-fidelity classification using Gaussian processes: accelerating the prediction of large-scale computational models
Authors: Francisco Sahli Costabal, Paris Perdikaris, Ellen Kuhl, Daniel E. Hurtado
Date: 2019-05-09 00:00:00
Tasks: Active Learning, BIG-bench Machine Learning, Gaussian Processes, General Classification

Machine learning techniques typically rely on large datasets to create accurate classifiers. However, there are situations when data is scarce and expensive to acquire. This is the case of studies that rely on state-of-the-art computational models which typically take days to run, thus hindering the potential of machine learning tools. In this work, we present a novel classifier that takes advantage of lower fidelity models and inexpensive approximations to predict the binary output of expensive computer simulations. We postulate an autoregressive model between the different levels of fidelity with Gaussian process priors. We adopt a fully Bayesian treatment for the hyper-parameters and use Markov Chain Mont Carlo samplers. We take advantage of the probabilistic nature of the classifier to implement active learning strategies. We also introduce a sparse approximation to enhance the ability of themulti-fidelity classifier to handle large datasets. We test these multi-fidelity classifiers against their single-fidelity counterpart with synthetic data, showing a median computational cost reduction of 23% for a target accuracy of 90%. In an application to cardiac electrophysiology, the multi-fidelity classifier achieves an F1 score, the harmonic mean of precision and recall, of 99.6% compared to 74.1% of a single-fidelity classifier when both are trained with 50 samples. In general, our results show that the multi-fidelity classifiers outperform their single-fidelity counterpart in terms of accuracy in all cases. We envision that this new tool will enable researchers to study classification problems that would otherwise be prohibitively expensive. Source code is available at https://github.com/fsahli/MFclass.
Learning Erdős-Rényi Random Graphs via Edge Detecting Queries
Authors: Zihan Li, Matthias Fresacher, Jonathan Scarlett
Date: 2019-05-09 00:00:00

In this paper, we consider the problem of learning an unknown graph via queries on groups of nodes, with the result indicating whether or not at least one edge is present among those nodes. While learning arbitrary graphs with $n$ nodes and $k$ edges is known to be hard in the sense of requiring $\Omega( \min\{ k^2 \log n, n^2\})$ tests (even when a small probability of error is allowed), we show that learning an Erd\H{o}s-R\'enyi random graph with an average of $\bar{k}$ edges is much easier; namely, one can attain asymptotically vanishing error probability with only $O(\bar{k}\log n)$ tests. We establish such bounds for a variety of algorithms inspired by the group testing problem, with explicit constant factors indicating a near-optimal number of tests, and in some cases asymptotic optimality including constant factors. In addition, we present an alternative design that permits a near-optimal sublinear decoding time of $O(\bar{k} \log^2 \bar{k} + \bar{k} \log n)$.
A deep learning approach for analyzing the composition of chemometric data
Authors: Muhammad Bilal, Mohib Ullah
Date: 2019-05-07 00:00:00
Tasks: regression

We propose novel deep learning based chemometric data analysis technique. We trained L2 regularized sparse autoencoder end-to-end for reducing the size of the feature vector to handle the classic problem of the curse of dimensionality in chemometric data analysis. We introduce a novel technique of automatic selection of nodes inside the hidden layer of an autoencoder through Pareto optimization. Moreover, Gaussian process regressor is applied on the reduced size feature vector for the regression. We evaluated our technique on orange juice and wine dataset and results are compared against 3 state-of-the-art methods. Quantitative results are shown on Normalized Mean Square Error (NMSE) and the results show considerable improvement in the state-of-the-art.
Adversarial Image Translation: Unrestricted Adversarial Examples in Face Recognition Systems
Authors: Kazuya Kakizaki, Kosuke Yoshida
Date: 2019-05-09 00:00:00
Tasks: Face Recognition, Translation

Thanks to recent advances in deep neural networks (DNNs), face recognition systems have become highly accurate in classifying a large number of face images. However, recent studies have found that DNNs could be vulnerable to adversarial examples, raising concerns about the robustness of such systems. Adversarial examples that are not restricted to small perturbations could be more serious since conventional certified defenses might be ineffective against them. To shed light on the vulnerability to such adversarial examples, we propose a flexible and efficient method for generating unrestricted adversarial examples using image translation techniques. Our method enables us to translate a source image into any desired facial appearance with large perturbations to deceive target face recognition systems. Our experimental results indicate that our method achieved about $90$ and $80\%$ attack success rates under white- and black-box settings, respectively, and that the translated images are perceptually realistic and maintain the identifiability of the individual while the perturbations are large enough to bypass certified defenses.
Bidirectional RNN-based Few-shot Training for Detecting Multi-stage Attack
Authors: Di Zhao, Jiqiang Liu, Jialin Wang, Wenjia Niu, Endong Tong, Tong Chen, Gang Li
Date: 2019-05-09 00:00:00
Tasks: Attribute, Clustering

"Feint Attack", as a new type of APT attack, has become the focus of attention. It adopts a multi-stage attacks mode which can be concluded as a combination of virtual attacks and real attacks. Under the cover of virtual attacks, real attacks can achieve the real purpose of the attacker, as a result, it often caused huge losses inadvertently. However, to our knowledge, all previous works use common methods such as Causal-Correlation or Cased-based to detect outdated multi-stage attacks. Few attentions have been paid to detect the "Feint Attack", because the difficulty of detection lies in the diversification of the concept of "Feint Attack" and the lack of professional datasets, many detection methods ignore the semantic relationship in the attack. Aiming at the existing challenge, this paper explores a new method to solve the problem. In the attack scenario, the fuzzy clustering method based on attribute similarity is used to mine multi-stage attack chains. Then we use a few-shot deep learning algorithm (SMOTE&CNN-SVM) and bidirectional Recurrent Neural Network model (Bi-RNN) to obtain the "Feint Attack" chains. "Feint Attack" is simulated by the real attack inserted in the normal causal attack chain, and the addition of the real attack destroys the causal relationship of the original attack chain. So we used Bi-RNN coding to obtain the hidden feature of "Feint Attack" chain. In the end, our method achieved the goal to detect the "Feint Attack" accurately by using the LLDoS1.0 and LLDoS2.0 of DARPA2000 and CICIDS2017 of Canadian Institute for Cybersecurity.
Unified Adversarial Invariance
Authors: Ayush Jaiswal, Yue Wu, Wael Abd-Almageed, Premkumar Natarajan
Date: 2019-05-07 00:00:00
Tasks: Disentanglement, Fairness

We present a unified invariance framework for supervised neural networks that can induce independence to nuisance factors of data without using any nuisance annotations, but can additionally use labeled information about biasing factors to force their removal from the latent embedding for making fair predictions. Invariance to nuisance is achieved by learning a split representation of data through competitive training between the prediction task and a reconstruction task coupled with disentanglement, whereas that to biasing factors is brought about by penalizing the network if the latent embedding contains any information about them. We describe an adversarial instantiation of this framework and provide analysis of its working. Our model outperforms previous works at inducing invariance to nuisance factors without using any labeled information about such variables, and achieves state-of-the-art performance at learning independence to biasing factors in fairness settings.
Stein Point Markov Chain Monte Carlo
Authors: Wilson Ye Chen, Alessandro Barp, François-Xavier Briol, Jackson Gorham, Mark Girolami, Lester Mackey, Chris. J. Oates
Date: 2019-05-09 00:00:00
Tasks: Bayesian Inference

An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target and, hence, require the solution of a non-convex optimisation problem to obtain each new point. This paper removes the need to solve this optimisation problem by, instead, selecting each new point based on a Markov chain sample path. This significantly reduces the computational cost of Stein Points and leads to a suite of algorithms that are straightforward to implement. The new algorithms are illustrated on a set of challenging Bayesian inference problems, and rigorous theoretical guarantees of consistency are established.
Proportionally Fair Clustering
Authors: Xingyu Chen, Brandon Fain, Liang Lyu, Kamesh Munagala
Date: 2019-05-09 00:00:00
Tasks: Clustering, Fairness

We extend the fair machine learning literature by considering the problem of proportional centroid clustering in a metric context. For clustering $n$ points with $k$ centers, we define fairness as proportionality to mean that any $n/k$ points are entitled to form their own cluster if there is another center that is closer in distance for all $n/k$ points. We seek clustering solutions to which there are no such justified complaints from any subsets of agents, without assuming any a priori notion of protected subsets. We present and analyze algorithms to efficiently compute, optimize, and audit proportional solutions. We conclude with an empirical examination of the tradeoff between proportional solutions and the $k$-means objective.
Adversarial Defense Framework for Graph Neural Network
Authors: Shen Wang, Zhengzhang Chen, Jingchao Ni, Xiao Yu, Zhichun Li, Haifeng Chen, Philip S. Yu
Date: 2019-05-09 00:00:00
Tasks: Adversarial Defense, Contrastive Learning, Graph Neural Network, Representation Learning

Graph neural network (GNN), as a powerful representation learning model on graph data, attracts much attention across various disciplines. However, recent studies show that GNN is vulnerable to adversarial attacks. How to make GNN more robust? What are the key vulnerabilities in GNN? How to address the vulnerabilities and defense GNN against the adversarial attacks? In this paper, we propose DefNet, an effective adversarial defense framework for GNNs. In particular, we first investigate the latent vulnerabilities in every layer of GNNs and propose corresponding strategies including dual-stage aggregation and bottleneck perceptron. Then, to cope with the scarcity of training data, we propose an adversarial contrastive learning method to train the GNN in a conditional GAN manner by leveraging the high-level graph representation. Extensive experiments on three public datasets demonstrate the effectiveness of DefNet in improving the robustness of popular GNN variants, such as Graph Convolutional Network and GraphSAGE, under various types of adversarial attacks.
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
Authors: Colin Wei, Tengyu Ma
Date: 2019-05-09 00:00:00
Proceeding: NeurIPS 2019 12

Existing Rademacher complexity bounds for neural networks rely only on norm control of the weight matrices and depend exponentially on depth via a product of the matrix norms. Lower bounds show that this exponential dependence on depth is unavoidable when no additional properties of the training data are considered. We suspect that this conundrum comes from the fact that these bounds depend on the training data only through the margin. In practice, many data-dependent techniques such as Batchnorm improve the generalization performance. For feedforward neural nets as well as RNNs, we obtain tighter Rademacher complexity bounds by considering additional data-dependent properties of the network: the norms of the hidden layers of the network, and the norms of the Jacobians of each layer with respect to all previous layers. Our bounds scale polynomially in depth when these empirical quantities are small, as is usually the case in practice. To obtain these bounds, we develop general tools for augmenting a sequence of functions to make their composition Lipschitz and then covering the augmented functions. Inspired by our theory, we directly regularize the network's Jacobians during training and empirically demonstrate that this improves test performance.
Alignment-Free Cross-Sensor Fingerprint Matching based on the Co-Occurrence of Ridge Orientations and Gabor-HoG Descriptor
Authors: Helala AlShehri, Muhammad Hussain, Hatim AboAlSamh, Qazi Emad-ul-Haq, Aqil M. Azmi
Date: 2019-04-30 00:00:00
Tasks: fingerprint verification

The existing automatic fingerprint verification methods are designed to work
under the assumption that the same sensor is installed for enrollment and
authentication (regular matching). There is a remarkable decrease in efficiency
when one type of contact-based sensor is employed for enrolment and another
type of contact-based sensor is used for authentication (cross-matching or
fingerprint sensor interoperability problem,). The ridge orientation patterns
in a fingerprint are invariant to sensor type. Based on this observation, we
propose a robust fingerprint descriptor called the co-occurrence of ridge
orientations (Co-Ror), which encodes the spatial distribution of ridge
orientations. Employing this descriptor, we introduce an efficient automatic
fingerprint verification method for cross-matching problem. Further, to enhance
the robustness of the method, we incorporate scale based ridge orientation
information through Gabor-HoG descriptor. The two descriptors are fused with
canonical correlation analysis (CCA), and the matching score between two
fingerprints is calculated using city-block distance. The proposed method is
alignment-free and can handle the matching process without the need for a
registration step. The intensive experiments on two benchmark databases
(FingerPass and MOLF) show the effectiveness of the method and reveal its
significant enhancement over the state-of-the-art methods such as VeriFinger (a
commercial SDK), minutia cylinder-code (MCC), MCC with scale, and the
thin-plate spline (TPS) model. The proposed research will help security
agencies, service providers and law-enforcement departments to overcome the
interoperability problem of contact sensors of different technology and
interaction types.
OpenEDS: Open Eye Dataset
Authors: Stephan J. Garbin, Yiru Shen, Immo Schuetz, Robert Cavin, Gregory Hughes, Sachin S. Talathi
Date: 2019-04-30 00:00:00
Tasks: Semantic Segmentation

We present a large scale data set, OpenEDS: Open Eye Dataset, of eye-images captured using a virtual-reality (VR) head mounted display mounted with two synchronized eyefacing cameras at a frame rate of 200 Hz under controlled illumination. This dataset is compiled from video capture of the eye-region collected from 152 individual participants and is divided into four subsets: (i) 12,759 images with pixel-level annotations for key eye-regions: iris, pupil and sclera (ii) 252,690 unlabelled eye-images, (iii) 91,200 frames from randomly selected video sequence of 1.5 seconds in duration and (iv) 143 pairs of left and right point cloud data compiled from corneal topography of eye regions collected from a subset, 143 out of 152, participants in the study. A baseline experiment has been evaluated on OpenEDS for the task of semantic segmentation of pupil, iris, sclera and background, with the mean intersectionover-union (mIoU) of 98.3 %. We anticipate that OpenEDS will create opportunities to researchers in the eye tracking community and the broader machine learning and computer vision community to advance the state of eye-tracking for VR applications. The dataset is available for download upon request at https://research.fb.com/programs/openeds-challenge
Human Activity Recognition Using Visual Object Detection
Authors: Schalk Wilhelm Pienaar, Reza Malekian
Date: 2019-05-02 00:00:00
Tasks: Activity Recognition, BIG-bench Machine Learning, Human Activity Recognition, Object, object-detection, Object Detection

Visual Human Activity Recognition (HAR) and data fusion with other sensors
can help us at tracking the behavior and activity of underground miners with
little obstruction. Existing models, such as Single Shot Detector (SSD),
trained on the Common Objects in Context (COCO) dataset is used in this paper
to detect the current state of a miner, such as an injured miner vs a
non-injured miner. Tensorflow is used for the abstraction layer of implementing
machine learning algorithms, and although it uses Python to deal with nodes and
tensors, the actual algorithms run on C++ libraries, providing a good balance
between performance and speed of development. The paper further discusses
evaluation methods for determining the accuracy of the machine-learning and an
approach to increase the accuracy of the detected activity/state of people in a
mining environment, by means of data fusion.
Machine Vision in the Context of Robotics: A Systematic Literature Review
Authors: Javad Ghofrani, Robert Kirschne, Daniel Rossburg, Dirk Reichelt, Tom Dimter
Date: 2019-05-03 00:00:00
Tasks: Systematic Literature Review

Machine vision is critical to robotics due to a wide range of applications which rely on input from visual sensors such as autonomous mobile robots and smart production systems. To create the smart homes and systems of tomorrow, an overview about current challenges in the research field would be of use to identify further possible directions, created in a systematic and reproducible manner. In this work a systematic literature review was conducted covering research from the last 10 years. We screened 172 papers from four databases and selected 52 relevant papers. While robustness and computation time were improved greatly, occlusion and lighting variance are still the biggest problems faced. From the number of recent publications, we conclude that the observed field is of relevance and interest to the research community. Further challenges arise in many areas of the field.