pwc-archive/papers-with-abstracts huggingface dataset:
A new simple and effective measure for bag-of-word inter-document similarity measurement
Authors: Sunil Aryal, Kai Ming Ting, Takashi Washio, Gholamreza Haffari
Date: 2019-02-09 00:00:00

To measure the similarity of two documents in the bag-of-words (BoW) vector
representation, different term weighting schemes are used to improve the
performance of cosine similarity---the most widely used inter-document
similarity measure in text mining. In this paper, we identify the shortcomings
of the underlying assumptions of term weighting in the inter-document
similarity measurement task; and provide a more fit-to-the-purpose alternative.
Based on this new assumption, we introduce a new simple but effective
similarity measure which does not require explicit term weighting. The proposed
measure employs a more nuanced probabilistic approach than those used in term
weighting to measure the similarity of two documents w.r.t each term occurring
in the two documents. Our empirical comparison with the existing similarity
measures using different term weighting schemes shows that the new measure
produces (i) better results in the binary BoW representation; and (ii)
competitive and more consistent results in the term-frequency-based BoW
representation.
A stochastic version of Stein Variational Gradient Descent for efficient sampling
Authors: Lei Li, Yingzhou Li, Jian-Guo Liu, Zibu Liu, Jianfeng Lu
Date: 2019-02-09 00:00:00
Tasks: Bayesian Inference

We propose in this work RBM-SVGD, a stochastic version of Stein Variational
Gradient Descent (SVGD) method for efficiently sampling from a given
probability measure and thus useful for Bayesian inference. The method is to
apply the Random Batch Method (RBM) for interacting particle systems proposed
by Jin et al to the interacting particle systems in SVGD. While keeping the
behaviors of SVGD, it reduces the computational cost, especially when the
interacting kernel has long range. Numerical examples verify the efficiency of
this new version of SVGD.
Improved Knowledge Distillation via Teacher Assistant
Authors: Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, Hassan Ghasemzadeh
Date: 2019-02-09 00:00:00
Tasks: Knowledge Distillation

Despite the fact that deep neural networks are powerful models and achieve appealing results on many tasks, they are too large to be deployed on edge devices like smartphones or embedded sensor nodes. There have been efforts to compress these networks, and a popular method is knowledge distillation, where a large (teacher) pre-trained network is used to train a smaller (student) network. However, in this paper, we show that the student network performance degrades when the gap between student and teacher is large. Given a fixed student network, one cannot employ an arbitrarily large teacher, or in other words, a teacher can effectively transfer its knowledge to students up to a certain size, not smaller. To alleviate this shortcoming, we introduce multi-step knowledge distillation, which employs an intermediate-sized network (teacher assistant) to bridge the gap between the student and the teacher. Moreover, we study the effect of teacher assistant size and extend the framework to multi-step distillation. Theoretical analysis and extensive experiments on CIFAR-10,100 and ImageNet datasets and on CNN and ResNet architectures substantiate the effectiveness of our proposed approach.
Generative Moment Matching Network-based Random Modulation Post-filter for DNN-based Singing Voice Synthesis and Neural Double-tracking
Authors: Hiroki Tamaru, Yuki Saito, Shinnosuke Takamichi, Tomoki Koriyama, Hiroshi Saruwatari
Date: 2019-02-09 00:00:00
Tasks: Singing Voice Synthesis

This paper proposes a generative moment matching network (GMMN)-based
post-filter that provides inter-utterance pitch variation for deep neural
network (DNN)-based singing voice synthesis. The natural pitch variation of a
human singing voice leads to a richer musical experience and is used in
double-tracking, a recording method in which two performances of the same
phrase are recorded and mixed to create a richer, layered sound. However,
singing voices synthesized using conventional DNN-based methods never vary
because the synthesis process is deterministic and only one waveform is
synthesized from one musical score. To address this problem, we use a GMMN to
model the variation of the modulation spectrum of the pitch contour of natural
singing voices and add a randomized inter-utterance variation to the pitch
contour generated by conventional DNN-based singing voice synthesis.
Experimental evaluations suggest that 1) our approach can provide perceptible
inter-utterance pitch variation while preserving speech quality. We extend our
approach to double-tracking, and the evaluation demonstrates that 2) GMMN-based
neural double-tracking is perceptually closer to natural double-tracking than
conventional signal processing-based artificial double-tracking is.
When Causal Intervention Meets Adversarial Examples and Image Masking for Deep Neural Networks
Authors: Chao-Han Huck Yang, Yi-Chieh Liu, Pin-Yu Chen, Xiaoli Ma, Yi-Chang James Tsai
Date: 2019-02-09 00:00:00
Tasks: Causal Inference, Visual Reasoning

Discovering and exploiting the causality in deep neural networks (DNNs) are crucial challenges for understanding and reasoning causal effects (CE) on an explainable visual model. "Intervention" has been widely used for recognizing a causal relation ontologically. In this paper, we propose a causal inference framework for visual reasoning via do-calculus. To study the intervention effects on pixel-level features for causal reasoning, we introduce pixel-wise masking and adversarial perturbation. In our framework, CE is calculated using features in a latent space and perturbed prediction from a DNN-based model. We further provide the first look into the characteristics of discovered CE of adversarially perturbed images generated by gradient-based methods \footnote{~~https://github.com/jjaacckkyy63/Causal-Intervention-AE-wAdvImg}. Experimental results show that CE is a competitive and robust index for understanding DNNs when compared with conventional methods such as class-activation mappings (CAMs) on the Chest X-Ray-14 dataset for human-interpretable feature(s) (e.g., symptom) reasoning. Moreover, CE holds promises for detecting adversarial examples as it possesses distinct characteristics in the presence of adversarial perturbations.
Region based Ensemble Learning Network for Fine-grained Classification
Authors: Weikuang Li, Tian Wang, Chuanyun Wang, Guangcun Shan, Mengyi Zhang, Hichem Snoussi
Date: 2019-02-09 00:00:00
Tasks: Classification, Ensemble Learning, General Classification, Scene Recognition

As an important research topic in computer vision, fine-grained
classification which aims to recognition subordinate-level categories has
attracted significant attention. We propose a novel region based ensemble
learning network for fine-grained classification. Our approach contains a
detection module and a module for classification. The detection module is based
on the faster R-CNN framework to locate the semantic regions of the object. The
classification module using an ensemble learning method, which trains a set of
sub-classifiers for different semantic regions and combines them together to
get a stronger classifier. In the evaluation, we implement experiments on the
CUB-2011 dataset and the result of experiments proves our method s efficient
for fine-grained classification. We also extend our approach to remote scene
recognition and evaluate it on the NWPU-RESISC45 dataset.
Measuring Patient Similarities via a Deep Architecture with Medical Concept Embedding
Authors: Zihao Zhu, Changchang Yin, Buyue Qian, Yu Cheng, Jishang Wei, Fei Wang
Date: 2019-02-09 00:00:00

Evaluating the clinical similarities between pairwise patients is a
fundamental problem in healthcare informatics. A proper patient similarity
measure enables various downstream applications, such as cohort study and
treatment comparative effectiveness research. One major carrier for conducting
patient similarity research is Electronic Health Records(EHRs), which are
usually heterogeneous, longitudinal, and sparse. Though existing studies on
learning patient similarity from EHRs have shown being useful in solving real
clinical problems, their applicability is limited due to the lack of medical
interpretations. Moreover, most previous methods assume a vector-based
representation for patients, which typically requires aggregation of medical
events over a certain time period. As a consequence, temporal information will
be lost. In this paper, we propose a patient similarity evaluation framework
based on the temporal matching of longitudinal patient EHRs. Two efficient
methods are presented, unsupervised and supervised, both of which preserve the
temporal properties in EHRs. The supervised scheme takes a convolutional neural
network architecture and learns an optimal representation of patient clinical
records with medical concept embedding. The empirical results on real-world
clinical data demonstrate substantial improvement over the baselines. We make
our code and sample data available for further study.
An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity
Authors: Lijun Ding, Alp Yurtsever, Volkan Cevher, Joel A. Tropp, Madeleine Udell
Date: 2019-02-09 00:00:00

This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form. This method is particularly effective for weakly constrained SDPs. The key idea is to formulate an approximate complementarity principle: Given an approximate solution to the dual SDP, the primal SDP has an approximate solution whose range is contained in the eigenspace with small eigenvalues of the dual slack matrix. For weakly constrained SDPs, this eigenspace has very low dimension, so this observation significantly reduces the search space for the primal solution. This result suggests an algorithmic strategy that can be implemented with minimal storage: (1) Solve the dual SDP approximately; (2) compress the primal SDP to the eigenspace with small eigenvalues of the dual slack matrix; (3) solve the compressed primal SDP. The paper also provides numerical experiments showing that this approach is successful for a range of interesting large-scale SDPs.
Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
Authors: Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, Allan Halpern
Date: 2019-02-09 00:00:00
Tasks: Attribute, Lesion Segmentation, Segmentation

This work summarizes the results of the largest skin image analysis challenge
in the world, hosted by the International Skin Imaging Collaboration (ISIC), a
global partnership that has organized the world's largest public repository of
dermoscopic images of skin. The challenge was hosted in 2018 at the Medical
Image Computing and Computer Assisted Intervention (MICCAI) conference in
Granada, Spain. The dataset included over 12,500 images across 3 tasks. 900
users registered for data download, 115 submitted to the lesion segmentation
task, 25 submitted to the lesion attribute detection task, and 159 submitted to
the disease classification task. Novel evaluation protocols were established,
including a new test for segmentation algorithm performance, and a test for
algorithm ability to generalize. Results show that top segmentation algorithms
still fail on over 10% of images on average, and algorithms with equal
performance on test data can have different abilities to generalize. This is an
important consideration for agencies regulating the growing set of machine
learning tools in the healthcare domain, and sets a new standard for future
public challenges in healthcare.
Latent Representations of Dynamical Systems: When Two is Better Than One
Authors: Max Tegmark
Date: 2019-02-09 00:00:00
Tasks: Vocal Bursts Valence Prediction

A popular approach for predicting the future of dynamical systems involves
mapping them into a lower-dimensional "latent space" where prediction is
easier. We show that the information-theoretically optimal approach uses
different mappings for present and future, in contrast to state-of-the-art
machine-learning approaches where both mappings are the same. We illustrate
this dichotomy by predicting the time-evolution of coupled harmonic oscillators
with dissipation and thermal noise, showing how the optimal 2-mapping method
significantly outperforms principal component analysis and all other approaches
that use a single latent representation, and discuss the intuitive reason why
two representations are better than one. We conjecture that a single latent
representation is optimal only for time-reversible processes, not for e.g.
text, speech, music or out-of-equilibrium physical systems.
Sinogram interpolation for sparse-view micro-CT with deep learning neural network
Authors: Xu Dong, Swapnil Vekhande, Guohua Cao
Date: 2019-02-09 00:00:00
Tasks: Computed Tomography (CT), SSIM

In sparse-view Computed Tomography (CT), only a small number of projection
images are taken around the object, and sinogram interpolation method has a
significant impact on final image quality. When the amount of sparsity (the
amount of missing views in sinogram data) is not high, conventional
interpolation methods have yielded good results. When the amount of sparsity is
high, more advanced sinogram interpolation methods are needed. Recently,
several deep learning (DL) based sinogram interpolation methods have been
proposed. However, those DL-based methods have mostly tested so far on computer
simulated sinogram data rather experimentally acquired sinogram data. In this
study, we developed a sinogram interpolation method for sparse-view micro-CT
based on the combination of U-Net and residual learning. We applied the method
to sinogram data obtained from sparse-view micro-CT experiments, where the
sparsity reached 90%. The interpolated sinogram by the DL neural network was
fed to FBP algorithm for reconstruction. The result shows that both RMSE and
SSIM of CT image are greatly improved. The experimental results demonstrate
that this sinogram interpolation method produce significantly better results
over standard linear interpolation methods when the sinogram data are extremely
sparse.
Image Decomposition and Classification through a Generative Model
Authors: Houpu Yao, Malcolm Regan, Yezhou Yang, Yi Ren
Date: 2019-02-09 00:00:00
Tasks: Classification, General Classification

We demonstrate in this paper that a generative model can be designed to
perform classification tasks under challenging settings, including adversarial
attacks and input distribution shifts. Specifically, we propose a conditional
variational autoencoder that learns both the decomposition of inputs and the
distributions of the resulting components. During test, we jointly optimize the
latent variables of the generator and the relaxed component labels to find the
best match between the given input and the output of the generator. The model
demonstrates promising performance at recognizing overlapping components from
the multiMNIST dataset, and novel component combinations from a traffic sign
dataset. Experiments also show that the proposed model achieves high robustness
on MNIST and NORB datasets, in particular for high-strength gradient attacks
and non-gradient attacks.
Meta-Curvature
Authors: Eunbyung Park, Junier B. Oliva
Date: 2019-02-09 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Few-Shot Image Classification, Few-Shot Learning, General Classification, Image Classification

We propose meta-curvature (MC), a framework to learn curvature information for better generalization and fast model adaptation. MC expands on the model-agnostic meta-learner (MAML) by learning to transform the gradients in the inner optimization such that the transformed gradients achieve better generalization performance to a new task. For training large scale neural networks, we decompose the curvature matrix into smaller matrices in a novel scheme where we capture the dependencies of the model's parameters with a series of tensor products. We demonstrate the effects of our proposed method on several few-shot learning tasks and datasets. Without any task specific techniques and architectures, the proposed method achieves substantial improvement upon previous MAML variants and outperforms the recent state-of-the-art methods. Furthermore, we observe faster convergence rates of the meta-training process. Finally, we present an analysis that explains better generalization performance with the meta-trained curvature.
Forward-backward-forward methods with variance reduction for stochastic variational inequalities
Authors: Radu Ioan Bot, Panayotis Mertikopoulos, Mathias Staudigl, Phan Tu Vuong
Date: 2019-02-09 00:00:00

We develop a new stochastic algorithm with variance reduction for solving
pseudo-monotone stochastic variational inequalities. Our method builds on
Tseng's forward-backward-forward (FBF) algorithm, which is known in the
deterministic literature to be a valuable alternative to Korpelevich's
extragradient method when solving variational inequalities over a convex and
closed set governed by pseudo-monotone, Lipschitz continuous operators. The
main computational advantage of Tseng's algorithm is that it relies only on a
single projection step and two independent queries of a stochastic oracle. Our
algorithm incorporates a variance reduction mechanism and leads to almost sure
(a.s.) convergence to an optimal solution. To the best of our knowledge, this
is the first stochastic look-ahead algorithm achieving this by using only a
single projection at each iteration..
Bayesian Nonparametric Adaptive Spectral Density Estimation for Financial Time Series
Authors: Nick James, Roman Marchant, Richard Gerlach, Sally Cripps
Date: 2019-02-09 00:00:00
Tasks: Density Estimation, Time Series, Time Series Analysis

Discrimination between non-stationarity and long-range dependency is a
difficult and long-standing issue in modelling financial time series. This
paper uses an adaptive spectral technique which jointly models the
non-stationarity and dependency of financial time series in a non-parametric
fashion assuming that the time series consists of a finite, but unknown number,
of locally stationary processes, the locations of which are also unknown. The
model allows a non-parametric estimate of the dependency structure by modelling
the auto-covariance function in the spectral domain. All our estimates are made
within a Bayesian framework where we use aReversible Jump Markov Chain Monte
Carlo algorithm for inference. We study the frequentist properties of our
estimates via a simulation study, and present a novel way of generating time
series data from a nonparametric spectrum. Results indicate that our techniques
perform well across a range of data generating processes. We apply our method
to a number of real examples and our results indicate that several financial
time series exhibit both long-range dependency and non-stationarity.
Challenges in Partially-Automated Roadway Feature Mapping Using Mobile Laser Scanning and Vehicle Trajectory Data
Authors: Mohammad Billah, Farzana Rahman, Arash Maskooki, Michael Todd, Matthew Barth, Jay A. Farrell
Date: 2019-02-09 00:00:00

Connected vehicle and driver's assistance applications are greatly
facilitated by Enhanced Digital Maps (EDMs) that represent roadway features
(e.g., lane edges or centerlines, stop bars). Due to the large number of
signalized intersections and miles of roadway, manual development of EDMs on a
global basis is not feasible. Mobile Terrestrial Laser Scanning (MTLS) is the
preferred data acquisition method to provide data for automated EDM
development. Such systems provide an MTLS trajectory and a point cloud for the
roadway environment. The challenge is to automatically convert these data into
an EDM. This article presents a new processing and feature extraction method,
experimental demonstration providing SAE-J2735 map messages for eleven example
intersections, and a discussion of the results that points out remaining
challenges and suggests directions for future research.
Nonlinear Discovery of Slow Molecular Modes using State-Free Reversible VAMPnets
Authors: Wei Chen, Hythem Sidky, Andrew L. Ferguson
Date: 2019-02-09 00:00:00

The success of enhanced sampling molecular simulations that accelerate along collective variables (CVs) is predicated on the availability of variables coincident with the slow collective motions governing the long-time conformational dynamics of a system. It is challenging to intuit these slow CVs for all but the simplest molecular systems, and their data-driven discovery directly from molecular simulation trajectories has been a central focus of the molecular simulation community to both unveil the important physical mechanisms and to drive enhanced sampling. In this work, we introduce state-free reversible VAMPnets (SRV) as a deep learning architecture that learns nonlinear CV approximants to the leading slow eigenfunctions of the spectral decomposition of the transfer operator that evolves equilibrium-scaled probability distributions through time. Orthogonality of the learned CVs is naturally imposed within network training without added regularization. The CVs are inherently explicit and differentiable functions of the input coordinates making them well-suited to use in enhanced sampling calculations. We demonstrate the utility of SRVs in capturing parsimonious nonlinear representations of complex system dynamics in applications to 1D and 2D toy systems where the true eigenfunctions are exactly calculable and to molecular dynamics simulations of alanine dipeptide and the WW domain protein.
Photorealistic Image Synthesis for Object Instance Detection
Authors: Tomas Hodan, Vibhav Vineet, Ran Gal, Emanuel Shalev, Jon Hanzelka, Treb Connell, Pedro Urbina, Sudipta N. Sinha, Brian Guenter
Date: 2019-02-09 00:00:00
Tasks: 6D Pose Estimation, 6D Pose Estimation using RGB, Image Generation, Object, Object Localization

We present an approach to synthesize highly photorealistic images of 3D
object models, which we use to train a convolutional neural network for
detecting the objects in real images. The proposed approach has three key
ingredients: (1) 3D object models are rendered in 3D models of complete scenes
with realistic materials and lighting, (2) plausible geometric configuration of
objects and cameras in a scene is generated using physics simulations, and (3)
high photorealism of the synthesized images achieved by physically based
rendering. When trained on images synthesized by the proposed approach, the
Faster R-CNN object detector achieves a 24% absolute improvement of mAP@.75IoU
on Rutgers APC and 11% on LineMod-Occluded datasets, compared to a baseline
where the training images are synthesized by rendering object models on top of
random photographs. This work is a step towards being able to effectively train
object detectors without capturing or annotating any real images. A dataset of
600K synthetic images with ground truth annotations for various computer vision
tasks will be released on the project website: thodan.github.io/objectsynth.