pwc-archive/papers-with-abstracts huggingface dataset:
Neural Ideal Point Estimation Network
Authors: Kyungwoo Song, Wonsung Lee, Il-Chul Moon
Date: 2019-04-26 00:00:00

Understanding politics is challenging because the politics take the influence
from everything. Even we limit ourselves to the political context in the
legislative processes; we need a better understanding of latent factors, such
as legislators, bills, their ideal points, and their relations. From the
modeling perspective, this is difficult 1) because these observations lie in a
high dimension that requires learning on low dimensional representations, and
2) because these observations require complex probabilistic modeling with
latent variables to reflect the causalities. This paper presents a new model to
reflect and understand this political setting, NIPEN, including factors
mentioned above in the legislation. We propose two versions of NIPEN: one is a
hybrid model of deep learning and probabilistic graphical model, and the other
model is a neural tensor model. Our result indicates that NIPEN successfully
learns the manifold of the legislative bill texts, and NIPEN utilizes the
learned low-dimensional latent variables to increase the prediction performance
of legislators' votings. Additionally, by virtue of being a domain-rich
probabilistic model, NIPEN shows the hidden strength of the legislators' trust
network and their various characteristics on casting votes.
Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization
Authors: Takuya Shimada, Han Bao, Issei Sato, Masashi Sugiyama
Date: 2019-04-26 00:00:00
Tasks: Clustering, General Classification

Pairwise similarities and dissimilarities between data points might be easier
to obtain than fully labeled data in real-world classification problems, e.g.,
in privacy-aware situations. To handle such pairwise information, an empirical
risk minimization approach has been proposed, giving an unbiased estimator of
the classification risk that can be computed only from pairwise similarities
and unlabeled data. However, this direction cannot handle pairwise
dissimilarities so far. On the other hand, semi-supervised clustering is one of
the methods which can use both similarities and dissimilarities. Nevertheless,
they typically require strong geometrical assumptions on the data distribution
such as the manifold assumption, which may deteriorate the performance. In this
paper, we derive an unbiased risk estimator which can handle all of
similarities/dissimilarities and unlabeled data. We theoretically establish
estimation error bounds and experimentally demonstrate the practical usefulness
of our empirical risk minimization method.
Passive nonlinear dendritic interactions as a general computational resource in functional spiking neural networks
Authors: Andreas Stöckel, Chris Eliasmith
Date: 2019-04-26 00:00:00

Nonlinear interactions in the dendritic tree play a key role in neural computation. Nevertheless, modeling frameworks aimed at the construction of large-scale, functional spiking neural networks, such as the Neural Engineering Framework, tend to assume a linear superposition of post-synaptic currents. In this paper, we present a series of extensions to the Neural Engineering Framework that facilitate the construction of networks incorporating Dale's principle and nonlinear conductance-based synapses. We apply these extensions to a two-compartment LIF neuron that can be seen as a simple model of passive dendritic computation. We show that it is possible to incorporate neuron models with input-dependent nonlinearities into the Neural Engineering Framework without compromising high-level function and that nonlinear post-synaptic currents can be systematically exploited to compute a wide variety of multivariate, bandlimited functions, including the Euclidean norm, controlled shunting, and non-negative multiplication. By avoiding an additional source of spike noise, the function-approximation accuracy of a single layer of two-compartment LIF neurons is on a par with or even surpasses that of two-layer spiking neural networks up to a certain target function bandwidth.
Robust Metric Learning based on the Rescaled Hinge Loss
Authors: Sumia Abdulhussien Razooqi Al-Obaidi, Davood Zabihzadeh, Hamideh Hajiabadi
Date: 2019-04-26 00:00:00
Tasks: Clustering, Metric Learning, Triplet

Distance/Similarity learning is a fundamental problem in machine learning. For example, kNN classifier or clustering methods are based on a distance/similarity measure. Metric learning algorithms enhance the efficiency of these methods by learning an optimal distance function from data. Most metric learning methods need training information in the form of pair or triplet sets. Nowadays, this training information often is obtained from the Internet via crowdsourcing methods. Therefore, this information may contain label noise or outliers leading to the poor performance of the learned metric. It is even possible that the learned metric functions perform worse than the general metrics such as Euclidean distance. To address this challenge, this paper presents a new robust metric learning method based on the Rescaled Hinge loss. This loss function is a general case of the popular Hinge loss and initially introduced in (Xu et al. 2017) to develop a new robust SVM algorithm. In this paper, we formulate the metric learning problem using the Rescaled Hinge loss function and then develop an efficient algorithm based on HQ (Half-Quadratic) to solve the problem. Experimental results on a variety of both real and synthetic datasets confirm that our new robust algorithm considerably outperforms state-of-the-art metric learning methods in the presence of label noise and outliers.
General risk measures for robust machine learning
Authors: Emilie Chouzenoux, Henri Gérard, Jean-Christophe Pesquet
Date: 2019-04-26 00:00:00
Tasks: BIG-bench Machine Learning

A wide array of machine learning problems are formulated as the minimization of the expectation of a convex loss function on some parameter space. Since the probability distribution of the data of interest is usually unknown, it is is often estimated from training sets, which may lead to poor out-of-sample performance. In this work, we bring new insights in this problem by using the framework which has been developed in quantitative finance for risk measures. We show that the original min-max problem can be recast as a convex minimization problem under suitable assumptions. We discuss several important examples of robust formulations, in particular by defining ambiguity sets based on $\varphi$-divergences and the Wasserstein metric.We also propose an efficient algorithm for solving the corresponding convex optimization problems involving complex convex constraints. Through simulation examples, we demonstrate that this algorithm scales well on real data sets.
Interactive user interface based on Convolutional Auto-encoders for annotating CT-scans
Authors: Martin Längkvist, Jonas Widell, Per Thunberg, Amy Loutfi, Mats Lidén
Date: 2019-04-26 00:00:00
Tasks: Semantic Segmentation

High resolution computed tomography (HRCT) is the most important imaging
modality for interstitial lung diseases, where the radiologists are interested
in identifying certain patterns, and their volumetric and regional
distribution. The use of machine learning can assist the radiologists with both
these tasks by performing semantic segmentation. In this paper, we propose an
interactive annotation-tool for semantic segmentation that assists the
radiologist in labeling CT scans. The annotation tool is evaluated by six
radiologists and radiology residents classifying healthy lung and reticular
pattern i HRCT images. The usability of the system is evaluated with a System
Usability Score (SUS) and interaction information from the readers that used
the tool for annotating the CT volumes. It was discovered that the experienced
usability and how the users interactied with the system differed between the
users. A higher SUS-score was given by users that prioritized learning speed
over model accuracy and spent less time with manual labeling and instead
utilized the suggestions provided by the GUI. An analysis of the annotation
variations between the readers show substantial agreement (Cohen's kappa=0.69)
for classification of healthy and affected lung parenchyma in pulmonary
fibrosis. The inter-reader variation is a challenge for the definition of
ground truth.
Neural Logic Machines
Authors: Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, Denny Zhou
Date: 2019-04-26 00:00:00
Proceeding: ICLR 2019 5
Tasks: Decision Making, Inductive Learning, Inductive logic programming, Relational Reasoning

We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for
both inductive learning and logic reasoning. NLMs exploit the power of both
neural networks---as function approximators, and logic programming---as a
symbolic processor for objects with properties, relations, logic connectives,
and quantifiers. After being trained on small-scale tasks (such as sorting
short arrays), NLMs can recover lifted rules, and generalize to large-scale
tasks (such as sorting longer arrays). In our experiments, NLMs achieve perfect
generalization in a number of tasks, from relational reasoning tasks on the
family tree and general graphs, to decision making tasks including sorting
arrays, finding shortest paths, and playing the blocks world. Most of these
tasks are hard to accomplish for neural networks or inductive logic programming
alone.
Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation
Authors: Chunfeng Song, Yan Huang, Wanli Ouyang, Liang Wang
Date: 2019-04-26 00:00:00
Proceeding: CVPR 2019 6
Tasks: Semantic Segmentation, Weakly-supervised Learning, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Semantic segmentation has achieved huge progress via adopting deep Fully
Convolutional Networks (FCN). However, the performance of FCN based models
severely rely on the amounts of pixel-level annotations which are expensive and
time-consuming. To address this problem, it is a good choice to learn to
segment with weak supervision from bounding boxes. How to make full use of the
class-level and region-level supervisions from bounding boxes is the critical
challenge for the weakly supervised learning task. In this paper, we first
introduce a box-driven class-wise masking model (BCM) to remove irrelevant
regions of each class. Moreover, based on the pixel-level segment proposal
generated from the bounding box supervision, we could calculate the mean
filling rates of each class to serve as an important prior cue, then we propose
a filling rate guided adaptive loss (FR-Loss) to help the model ignore the
wrongly labeled pixels in proposals. Unlike previous methods directly training
models with the fixed individual segment proposals, our method can adjust the
model learning with global statistical information. Thus it can help reduce the
negative impacts from wrongly labeled proposals. We evaluate the proposed
method on the challenging PASCAL VOC 2012 benchmark and compare with other
methods. Extensive experimental results show that the proposed method is
effective and achieves the state-of-the-art results.
The Roadmap to 6G -- AI Empowered Wireless Networks
Authors: Khaled B. Letaief, Wei Chen, Yuanming Shi, Jun Zhang, Ying-Jun Angela Zhang
Date: 2019-04-26 00:00:00

The recent upsurge of diversified mobile applications, especially those supported by Artificial Intelligence (AI), is spurring heated discussions on the future evolution of wireless communications. While 5G is being deployed around the world, efforts from industry and academia have started to look beyond 5G and conceptualize 6G. We envision 6G to undergo an unprecedented transformation that will make it substantially different from the previous generations of wireless cellular systems. In particular, 6G will go beyond mobile Internet and will be required to support ubiquitous AI services from the core to the end devices of the network. Meanwhile, AI will play a critical role in designing and optimizing 6G architectures, protocols, and operations. In this article, we discuss potential technologies for 6G to enable mobile AI applications, as well as AI-enabled methodologies for 6G network design and optimization. Key trends in the evolution to 6G will also be discussed.
A Survey on Face Data Augmentation
Authors: Xiang Wang, Kai Wang, Shiguo Lian
Date: 2019-04-26 00:00:00
Tasks: Data Augmentation, Survey

The quality and size of training set have great impact on the results of deep
learning-based face related tasks. However, collecting and labeling adequate
samples with high quality and balanced distributions still remains a laborious
and expensive work, and various data augmentation techniques have thus been
widely used to enrich the training dataset. In this paper, we systematically
review the existing works of face data augmentation from the perspectives of
the transformation types and methods, with the state-of-the-art approaches
involved. Among all these approaches, we put the emphasis on the deep
learning-based works, especially the generative adversarial networks which have
been recognized as more powerful and effective tools in recent years. We
present their principles, discuss the results and show their applications as
well as limitations. Different evaluation metrics for evaluating these
approaches are also introduced. We point out the challenges and opportunities
in the field of face data augmentation, and provide brief yet insightful
discussions.
AutoSF: Searching Scoring Functions for Knowledge Graph Embedding
Authors: Yongqi Zhang, Quanming Yao, Wenyuan Dai, Lei Chen
Date: 2019-04-26 00:00:00
Tasks: AutoML, Graph Embedding, Knowledge Graph Embedding, Link Prediction, Link Property Prediction

Scoring functions (SFs), which measure the plausibility of triplets in knowledge graph (KG), have become the crux of KG embedding. Lots of SFs, which target at capturing different kinds of relations in KGs, have been designed by humans in recent years. However, as relations can exhibit complex patterns that are hard to infer before training, none of them can consistently perform better than others on existing benchmark data sets. In this paper, inspired by the recent success of automated machine learning (AutoML), we propose to automatically design SFs (AutoSF) for distinct KGs by the AutoML techniques. However, it is non-trivial to explore domain-specific information here to make AutoSF efficient and effective. We firstly identify a unified representation over popularly used SFs, which helps to set up a search space for AutoSF. Then, we propose a greedy algorithm to search in such a space efficiently. The algorithm is further sped up by a filter and a predictor, which can avoid repeatedly training SFs with same expressive ability and help removing bad candidates during the search before model training. Finally, we perform extensive experiments on benchmark data sets. Results on link prediction and triplets classification show that the searched SFs by AutoSF, are KG dependent, new to the literature, and outperform the state-of-the-art SFs designed by humans.
Adaptive Regret of Convex and Smooth Functions
Authors: Lijun Zhang, Tie-Yan Liu, Zhi-Hua Zhou
Date: 2019-04-26 00:00:00

We investigate online convex optimization in changing environments, and choose the adaptive regret as the performance measure. The goal is to achieve a small regret over every interval so that the comparator is allowed to change over time. Different from previous works that only utilize the convexity condition, this paper further exploits smoothness to improve the adaptive regret. To this end, we develop novel adaptive algorithms for convex and smooth functions, and establish problem-dependent regret bounds over any interval. Our regret bounds are comparable to existing results in the worst case, and become much tighter when the comparator has a small loss.
Fake News Early Detection: An Interdisciplinary Study
Authors: Xinyi Zhou, Atishay Jain, Vir V. Phoha, Reza Zafarani
Date: 2019-04-26 00:00:00
Tasks: Fake News Detection, Feature Engineering

Massive dissemination of fake news and its potential to erode democracy has increased the demand for accurate fake news detection. Recent advancements in this area have proposed novel techniques that aim to detect fake news by exploring how it propagates on social networks. Nevertheless, to detect fake news at an early stage, i.e., when it is published on a news outlet but not yet spread on social media, one cannot rely on news propagation information as it does not exist. Hence, there is a strong need to develop approaches that can detect fake news by focusing on news content. In this paper, a theory-driven model is proposed for fake news detection. The method investigates news content at various levels: lexicon-level, syntax-level, semantic-level and discourse-level. We represent news at each level, relying on well-established theories in social and forensic psychology. Fake news detection is then conducted within a supervised machine learning framework. As an interdisciplinary research, our work explores potential fake news patterns, enhances the interpretability in fake news feature engineering, and studies the relationships among fake news, deception/disinformation, and clickbaits. Experiments conducted on two real-world datasets indicate the proposed method can outperform the state-of-the-art and enable fake news early detection when there is limited content information.
Transformers with convolutional context for ASR
Authors: Abdelrahman Mohamed, Dmytro Okhonko, Luke Zettlemoyer
Date: 2019-04-26 00:00:00
Tasks: Image Classification, Machine Translation, speech-recognition, Speech Recognition, Translation

The recent success of transformer networks for neural machine translation and other NLP tasks has led to a surge in research work trying to apply it for speech recognition. Recent efforts studied key research questions around ways of combining positional embedding with speech features, and stability of optimization for large scale learning of transformer networks. In this paper, we propose replacing the sinusoidal positional embedding for transformers with convolutionally learned input representations. These contextual representations provide subsequent transformer blocks with relative positional information needed for discovering long-range relationships between local concepts. The proposed system has favorable optimization characteristics where our reported results are produced with fixed learning rate of 1.0 and no warmup steps. The proposed model achieves a competitive 4.7% and 12.9% WER on the Librispeech ``test clean'' and ``test other'' subsets when no extra LM text is provided.
DPVis: Visual Analytics with Hidden Markov Models for Disease Progression Pathways
Authors: Bum Chul Kwon, Vibha Anand, Kristen A Severson, Soumya Ghosh, Zhaonan Sun, Brigitte I Frohnert, Markus Lundgren, Kenney Ng
Date: 2019-04-26 00:00:00

Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models that both discover these states and make inferences of health states for patients. Despite the advantages of using the algorithms for discovering interesting patterns, it still remains challenging for medical experts to interpret model outputs, understand complex modeling parameters, and clinically make sense of the patterns. To tackle these problems, we conducted a design study with clinical scientists, statisticians, and visualization experts, with the goal to investigate disease progression pathways of chronic diseases, namely type 1 diabetes (T1D), Huntington's disease, Parkinson's disease, and chronic obstructive pulmonary disease (COPD). As a result, we introduce DPVis which seamlessly integrates model parameters and outcomes of HMMs into interpretable and interactive visualizations. In this study, we demonstrate that DPVis is successful in evaluating disease progression models, visually summarizing disease states, interactively exploring disease progression patterns, and building, analyzing, and comparing clinically relevant patient subgroups.
A Novel Orthogonal Direction Mesh Adaptive Direct Search Approach for SVM Hyperparameter Tuning
Authors: Alexandre Reeberg Mello, Jonathan de Matos, Marcelo R. Stemmer, Alceu de Souza Britto Jr., Alessandro Lameiras Koerich
Date: 2019-04-26 00:00:00

In this paper, we propose the use of a black-box optimization method called
deterministic Mesh Adaptive Direct Search (MADS) algorithm with orthogonal
directions (Ortho-MADS) for the selection of hyperparameters of Support Vector
Machines with a Gaussian kernel. Different from most of the methods in the
literature that exploit the properties of the data or attempt to minimize the
accuracy of a validation dataset over the first quadrant of (C, gamma), the
Ortho-MADS provides convergence proof. We present the MADS, followed by the
Ortho-MADS, the dynamic stopping criterion defined by the MADS mesh size and
two different search strategies (Nelder-Mead and Variable Neighborhood Search)
that contribute to a competitive convergence rate as well as a mechanism to
escape from undesired local minima. We have investigated the practical
selection of hyperparameters for the Support Vector Machine with a Gaussian
kernel, i.e., properly choose the hyperparameters gamma (bandwidth) and C
(trade-off) on several benchmark datasets. The experimental results have shown
that the proposed approach for hyperparameter tuning consistently finds
comparable or better solutions, when using a common configuration, than other
methods. We have also evaluated the accuracy and the number of function
evaluations of the Ortho-MADS with the Nelder-Mead search strategy and the
Variable Neighborhood Search strategy using the mesh size as a stopping
criterion, and we have achieved accuracy that no other method for
hyperparameters optimization could reach.
Structural modeling using overlapped group penalties for discovering predictive biomarkers for subgroup analysis
Authors: Chong Ma, Wenxuan Deng, Shuangge Ma, Ray Liu, Kevin Galinsky
Date: 2019-04-26 00:00:00
Tasks: regression

The identification of predictive biomarkers from a large scale of covariates
for subgroup analysis has attracted fundamental attention in medical research.
In this article, we propose a generalized penalized regression method with a
novel penalty function, for enforcing the hierarchy structure between the
prognostic and predictive effects, such that a nonzero predictive effect must
induce its ancestor prognostic effects being nonzero in the model. Our method
is able to select useful predictive biomarkers by yielding a sparse,
interpretable, and predictable model for subgroup analysis, and can deal with
different types of response variable such as continuous, categorical, and
time-to-event data. We show that our method is asymptotically consistent under
some regularized conditions. To minimize the generalized penalized regression
model, we propose a novel integrative optimization algorithm by integrating the
majorization-minimization and the alternating direction method of multipliers,
which is named after \texttt{smog}. The enriched simulation study and real case
study demonstrate that our method is very powerful for discovering the true
predictive biomarkers and identifying subgroups of patients.