pwc-archive/papers-with-abstracts huggingface dataset:
Do Subsampled Newton Methods Work for High-Dimensional Data?
Authors: Xiang Li, Shusen Wang, Zhihua Zhang
Date: 2019-02-13 00:00:00
Tasks: Distributed Optimization, Vocal Bursts Intensity Prediction

Subsampled Newton methods approximate Hessian matrices through subsampling techniques, alleviating the cost of forming Hessian matrices but using sufficient curvature information. However, previous results require $\Omega (d)$ samples to approximate Hessians, where $d$ is the dimension of data points, making it less practically feasible for high-dimensional data. The situation is deteriorated when $d$ is comparably as large as the number of data points $n$, which requires to take the whole dataset into account, making subsampling useless. This paper theoretically justifies the effectiveness of subsampled Newton methods on high dimensional data. Specifically, we prove only $\widetilde{\Theta}(d^\gamma_{\rm eff})$ samples are needed in the approximation of Hessian matrices, where $d^\gamma_{\rm eff}$ is the $\gamma$-ridge leverage and can be much smaller than $d$ as long as $n\gamma \gg 1$. Additionally, we extend this result so that subsampled Newton methods can work for high-dimensional data on both distributed optimization problems and non-smooth regularized problems.
3D Face Modeling From Diverse Raw Scan Data
Authors: Feng Liu, Luan Tran, Xiaoming Liu
Date: 2019-02-13 00:00:00
Proceeding: ICCV 2019 10
Tasks: 3D Face Modelling, 3D Face Reconstruction, Decoder, Face Model, Face Reconstruction, Weakly-supervised Learning

Traditional 3D face models learn a latent representation of faces using linear subspaces from limited scans of a single database. The main roadblock of building a large-scale face model from diverse 3D databases lies in the lack of dense correspondence among raw scans. To address these problems, this paper proposes an innovative framework to jointly learn a nonlinear face model from a diverse set of raw 3D scan databases and establish dense point-to-point correspondence among their scans. Specifically, by treating input scans as unorganized point clouds, we explore the use of PointNet architectures for converting point clouds to identity and expression feature representations, from which the decoder networks recover their 3D face shapes. Further, we propose a weakly supervised learning approach that does not require correspondence label for the scans. We demonstrate the superior dense correspondence and representation power of our proposed method, and its contribution to single-image 3D face reconstruction.
Sample Variance Decay in Randomly Initialized ReLU Networks
Authors: Kyle Luther, H. Sebastian Seung
Date: 2019-02-13 00:00:00

Before training a neural net, a classic rule of thumb is to randomly initialize the weights so the variance of activations is preserved across layers. This is traditionally interpreted using the total variance due to randomness in both weights \emph{and} samples. Alternatively, one can interpret the rule of thumb as preservation of the variance over samples for a fixed network. The two interpretations differ little for a shallow net, but the difference is shown to grow with depth for a deep ReLU net by decomposing the total variance into the network-averaged sum of the sample variance and square of the sample mean. We demonstrate that even when the total variance is preserved, the sample variance decays in the later layers through an analytical calculation in the limit of infinite network width, and numerical simulations for finite width. We show that Batch Normalization eliminates this decay and provide empirical evidence that preserving the sample variance instead of only the total variance at initialization time can have an impact on the training dynamics of a deep network.
Learning to Select Knowledge for Response Generation in Dialog Systems
Authors: Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, Hua Wu
Date: 2019-02-13 00:00:00
Tasks: Response Generation

End-to-end neural models for intelligent dialogue systems suffer from the problem of generating uninformative responses. Various methods were proposed to generate more informative responses by leveraging external knowledge. However, few previous work has focused on selecting appropriate knowledge in the learning process. The inappropriate selection of knowledge could prohibit the model from learning to make full use of the knowledge. Motivated by this, we propose an end-to-end neural model which employs a novel knowledge selection mechanism where both prior and posterior distributions over knowledge are used to facilitate knowledge selection. Specifically, a posterior distribution over knowledge is inferred from both utterances and responses, and it ensures the appropriate selection of knowledge during the training process. Meanwhile, a prior distribution, which is inferred from utterances only, is used to approximate the posterior distribution so that appropriate knowledge can be selected even without responses during the inference process. Compared with the previous work, our model can better incorporate appropriate knowledge in response generation. Experiments on both automatic and human evaluation verify the superiority of our model over previous baselines.
Super-Resolution of Brain MRI Images using Overcomplete Dictionaries and Nonlocal Similarity
Authors: Yinghua Li, Bin Song, Jie Guo, Xiaojiang Du, Mohsen Guizani
Date: 2019-02-13 00:00:00
Tasks: Compressive Sensing, Image Super-Resolution, Super-Resolution

Recently, the Magnetic Resonance Imaging (MRI) images have limited and
unsatisfactory resolutions due to various constraints such as physical,
technological and economic considerations. Super-resolution techniques can
obtain high-resolution MRI images. The traditional methods obtained the
resolution enhancement of brain MRI by interpolations, affecting the accuracy
of the following diagnose process. The requirement for brain image quality is
fast increasing. In this paper, we propose an image super-resolution (SR)
method based on overcomplete dictionaries and inherent similarity of an image
to recover the high-resolution (HR) image from a single low-resolution (LR)
image. We explore the nonlocal similarity of the image to tentatively search
for similar blocks in the whole image and present a joint reconstruction method
based on compressive sensing (CS) and similarity constraints. The sparsity and
self-similarity of the image blocks are taken as the constraints. The proposed
method is summarized in the following steps. First, a dictionary classification
method based on the measurement domain is presented. The image blocks are
classified into smooth, texture and edge parts by analyzing their features in
the measurement domain. Then, the corresponding dictionaries are trained using
the classified image blocks. Equally important, in the reconstruction part, we
use the CS reconstruction method to recover the HR brain MRI image, considering
both nonlocal similarity and the sparsity of an image as the constraints. This
method performs better both visually and quantitatively than some existing
methods.
Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps
Authors: Beomsu Kim, Junghoon Seo, SeungHyun Jeon, Jamyoung Koo, Jeongyeol Choe, Taegyun Jeon
Date: 2019-02-13 00:00:00

Saliency Map, the gradient of the score function with respect to the input, is the most basic technique for interpreting deep neural network decisions. However, saliency maps are often visually noisy. Although several hypotheses were proposed to account for this phenomenon, there are few works that provide rigorous analyses of noisy saliency maps. In this paper, we firstly propose a new hypothesis that noise may occur in saliency maps when irrelevant features pass through ReLU activation functions. Then, we propose Rectified Gradient, a method that alleviates this problem through layer-wise thresholding during backpropagation. Experiments with neural networks trained on CIFAR-10 and ImageNet showed effectiveness of our method and its superiority to other attribution methods.
Multi-views Embedding for Cattle Re-identification
Authors: Luca Bergamini, Angelo Porrello, Andrea Capobianco Dondona, Ercole Del Negro, Mauro Mattioli, Nicola D'Alterio, Simone Calderara
Date: 2019-02-13 00:00:00

People re-identification task has seen enormous improvements in the latest
years, mainly due to the development of better image features extraction from
deep Convolutional Neural Networks (CNN) and the availability of large
datasets. However, little research has been conducted on animal identification
and re-identification, even if this knowledge may be useful in a rich variety
of different scenarios. Here, we tackle cattle re-identification exploiting
deep CNN and show how this task is poorly related with the human one,
presenting unique challenges that makes it far from being solved. We present
various baselines, both based on deep architectures or on standard machine
learning algorithms, and compared them with our solution. Finally, a rich
ablation study has been conducted to further investigate the unique
peculiarities of this task.
Federated Machine Learning: Concept and Applications
Authors: Qiang Yang, Yang Liu, Tianjian Chen, Yongxin Tong
Date: 2019-02-13 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning, Transfer Learning, Vertical Federated Learning

Today's AI still faces two major challenges. One is that in most industries,
data exists in the form of isolated islands. The other is the strengthening of
data privacy and security. We propose a possible solution to these challenges:
secure federated learning. Beyond the federated learning framework first
proposed by Google in 2016, we introduce a comprehensive secure federated
learning framework, which includes horizontal federated learning, vertical
federated learning and federated transfer learning. We provide definitions,
architectures and applications for the federated learning framework, and
provide a comprehensive survey of existing works on this subject. In addition,
we propose building data networks among organizations based on federated
mechanisms as an effective solution to allow knowledge to be shared without
compromising user privacy.
Bone Texture Analysis for Prediction of Incident Radio-graphic Hip Osteoarthritis Using Machine Learning: Data from the Cohort Hip and Cohort Knee (CHECK) study
Authors: Jukka Hirvasniemi, Willem Paul Gielis, Saeed Arbabi, Rintje Agricola, Willem Evert van Spil, Vahid Arbabi, Harrie Weinans
Date: 2019-02-13 00:00:00
Tasks: Texture Classification

Our aim was to assess the ability of radiography-based bone texture
parameters in proximal femur and acetabulum to predict incident radiographic
hip osteoarthritis (rHOA) over a 10 years period. Pelvic radiographs from CHECK
(Cohort Hip and Cohort Knee) at baseline (987 hips) were analyzed for bone
texture using fractal signature analysis in proximal femur and acetabulum.
Elastic net (machine learning) was used to predict the incidence of rHOA
(Kellgren-Lawrence grade (KL) > 1 or total hip replacement (THR)), joint space
narrowing score (JSN, range 0-3), and osteophyte score (OST, range 0-3) after
10 years. Performance of prediction models was assessed using the area under
the receiver operating characteristic curve (ROC AUC). Of the 987 hips without
rHOA at baseline, 435 (44%) had rHOA at 10-year follow-up. Of the 667 hips with
JSN grade 0 at baseline, 471 (71%) had JSN grade > 0 at 10-year follow-up. Of
the 613 hips with OST grade 0 at baseline, 526 (86%) had OST grade > 0 at
10-year follow-up. AUCs for the models including age, gender, and body mass
index to predict incident rHOA, JSN, and OST were 0.59, 0.54, and 0.51,
respectively. The inclusion of bone texture parameters in the models improved
the prediction of incident rHOA (ROC AUC 0.66 and 0.71 when baseline KL was
also included in the model) and JSN (ROC AUC 0.62), but not incident OST (ROC
AUC 0.53). Bone texture analysis provides additional information for predicting
incident rHOA or THR over 10 years.
Intertemporal Connections Between Query Suggestions and Search Engine Results for Politics Related Queries
Authors: Malte Bonart, Philipp Schaer
Date: 2018-12-20 00:00:00

This short paper deals with the combination and comparison of two data
sources: Search engine results and query suggestions for 16 terms related to
political candidates and parties. The data was collected before the federal
election in Germany in September 2017 for a period of two months. The rank
biased overlap (RBO) statistic is used to measure the similarity of the
top-weighted rankings. For each search term and for both the search results and
query auto-completions we study the stability of the rankings over time.
A Survey on Session-based Recommender Systems
Authors: Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet Orgun, Defu Lian
Date: 2019-02-13 00:00:00
Tasks: Collaborative Filtering, Decision Making, Recommendation Systems, Survey

Recommender systems (RSs) have been playing an increasingly important role for informed consumption, services, and decision-making in the overloaded information era and digitized economy. In recent years, session-based recommender systems (SBRSs) have emerged as a new paradigm of RSs. Different from other RSs such as content-based RSs and collaborative filtering-based RSs which usually model long-term yet static user preferences, SBRSs aim to capture short-term but dynamic user preferences to provide more timely and accurate recommendations sensitive to the evolution of their session contexts. Although SBRSs have been intensively studied, neither unified problem statements for SBRSs nor in-depth elaboration of SBRS characteristics and challenges are available. It is also unclear to what extent SBRS challenges have been addressed and what the overall research landscape of SBRSs is. This comprehensive review of SBRSs addresses the above aspects by exploring in depth the SBRS entities (e.g., sessions), behaviours (e.g., users' clicks on items) and their properties (e.g., session length). We propose a general problem statement of SBRSs, summarize the diversified data characteristics and challenges of SBRSs, and define a taxonomy to categorize the representative SBRS research. Finally, we discuss new research opportunities in this exciting and vibrant area.
Person Re-identification in Videos by Analyzing Spatio-Temporal Tubes
Authors: Sk. Arif Ahmed, Debi Prosad Dogra, Heeseung Choi, Seungho Chae, Ig-Jae Kim
Date: 2019-02-13 00:00:00
Tasks: Person Re-Identification, Temporal Sequences

Typical person re-identification frameworks search for k best matches in a
gallery of images that are often collected in varying conditions. The gallery
may contain image sequences when re-identification is done on videos. However,
such a process is time consuming as re-identification has to be carried out
multiple times. In this paper, we extract spatio-temporal sequences of frames
(referred to as tubes) of moving persons and apply a multi-stage processing to
match a given query tube with a gallery of stored tubes recorded through other
cameras. Initially, we apply a binary classifier to remove noisy images from
the input query tube. In the next step, we use a key-pose detection-based query
minimization. This reduces the length of the query tube by removing redundant
frames. Finally, a 3-stage hierarchical re-identification framework is used to
rank the output tubes as per the matching scores. Experiments with publicly
available video re-identification datasets reveal that our framework is better
than state-of-the-art methods. It ranks the tubes with an increased CMC
accuracy of 6-8% across multiple datasets. Also, our method significantly
reduces the number of false positives. A new video re-identification dataset,
named Tube-based Reidentification Video Dataset (TRiViD), has been prepared
with an aim to help the re-identification research community
Classifying Signals on Irregular Domains via Convolutional Cluster Pooling
Authors: Angelo Porrello, Davide Abati, Simone Calderara, Rita Cucchiara
Date: 2019-02-13 00:00:00
Tasks: Clustering, General Classification

We present a novel and hierarchical approach for supervised classification of
signals spanning over a fixed graph, reflecting shared properties of the
dataset. To this end, we introduce a Convolutional Cluster Pooling layer
exploiting a multi-scale clustering in order to highlight, at different
resolutions, locally connected regions on the input graph. Our proposal
generalises well-established neural models such as Convolutional Neural
Networks (CNNs) on irregular and complex domains, by means of the exploitation
of the weight sharing property in a graph-oriented architecture. In this work,
such property is based on the centrality of each vertex within its
soft-assigned cluster. Extensive experiments on NTU RGB+D, CIFAR-10 and 20NEWS
demonstrate the effectiveness of the proposed technique in capturing both local
and global patterns in graph-structured data out of different domains.
Delog: A Privacy Preserving Log Filtering Framework for Online Compute Platforms
Authors: Amey Agrawal, Abhishek Dixit, Namrata Shettar, Darshil Kapadia, Rohit Karlupia, Vikram Agrawal, Rajat Gupta
Date: 2019-02-13 00:00:00
Tasks: Log Parsing, Privacy Preserving

In many software applications, logs serve as the only interface between the application and the developer. However, navigating through the logs of long-running applications is often challenging. Logs from previously successful application runs can be leveraged to automatically identify errors and provide users with only the logs that are relevant to the debugging process. We describe a privacy preserving framework which can be employed by Platform as a Service (PaaS) providers to utilize the user logs generated on the platform while protecting the potentially sensitive logged data. Further, in order to accurately and scalably parse log lines, we present a distributed log parsing algorithm which leverages Locality Sensitive Hashing (LSH). We outperform the state-of-the-art on multiple datasets. We further demonstrate the scalability of Delog on publicly available Thunderbird log dataset with close to 27,000 unique patterns and 211 million lines.
Relative rationality: Is machine rationality subjective?
Authors: Tshilidzi Marwala
Date: 2019-02-13 00:00:00
Tasks: Decision Making

Rational decision making in its linguistic description means making logical
decisions. In essence, a rational agent optimally processes all relevant
information to achieve its goal. Rationality has two elements and these are the
use of relevant information and the efficient processing of such information.
In reality, relevant information is incomplete, imperfect and the processing
engine, which is a brain for humans, is suboptimal. Humans are risk averse
rather than utility maximizers. In the real world, problems are predominantly
non-convex and this makes the idea of rational decision-making fundamentally
unachievable and Herbert Simon called this bounded rationality. There is a
trade-off between the amount of information used for decision-making and the
complexity of the decision model used. This explores whether machine
rationality is subjective and concludes that indeed it is.
The Odds are Odd: A Statistical Test for Detecting Adversarial Examples
Authors: Kevin Roth, Yannic Kilcher, Thomas Hofmann
Date: 2019-02-13 00:00:00

We investigate conditions under which test statistics exist that can reliably detect examples, which have been adversarially manipulated in a white-box attack. These statistics can be easily computed and calibrated by randomly corrupting inputs. They exploit certain anomalies that adversarial attacks introduce, in particular if they follow the paradigm of choosing perturbations optimally under p-norm constraints. Access to the log-odds is the only requirement to defend models. We justify our approach empirically, but also provide conditions under which detectability via the suggested test statistics is guaranteed to be effective. In our experiments, we show that it is even possible to correct test time predictions for adversarial attacks with high accuracy.
Group level MEG/EEG source imaging via optimal transport: minimum Wasserstein estimates
Authors: Hicham Janati, Thomas Bazeille, Bertrand Thirion, Marco Cuturi, Alexandre Gramfort
Date: 2019-02-13 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Magnetoencephalography (MEG) and electroencephalogra-phy (EEG) are
non-invasive modalities that measure the weak electromagnetic fields generated
by neural activity. Inferring the location of the current sources that
generated these magnetic fields is an ill-posed inverse problem known as source
imaging. When considering a group study, a baseline approach consists in
carrying out the estimation of these sources independently for each subject.
The ill-posedness of each problem is typically addressed using sparsity
promoting regularizations. A straightforward way to define a common pattern for
these sources is then to average them. A more advanced alternative relies on a
joint localization of sources for all subjects taken together, by enforcing
some similarity across all estimated sources. An important advantage of this
approach is that it consists in a single estimation in which all measurements
are pooled together, making the inverse problem better posed. Such a joint
estimation poses however a few challenges, notably the selection of a valid
regularizer that can quantify such spatial similarities. We propose in this
work a new procedure that can do so while taking into account the geometrical
structure of the cortex. We call this procedure Minimum Wasserstein Estimates
(MWE). The benefits of this model are twofold. First, joint inference allows to
pool together the data of different brain geometries, accumulating more spatial
information. Second, MWE are defined through Optimal Transport (OT) metrics
which provide a tool to model spatial proximity between cortical sources of
different subjects, hence not enforcing identical source location in the group.
These benefits allow MWE to be more accurate than standard MEG source
localization techniques. To support these claims, we perform source
localization on realistic MEG simulations based on forward operators derived
from MRI scans. On a visual task dataset, we demonstrate how MWE infer neural
patterns similar to functional Magnetic Resonance Imaging (fMRI) maps.
On Nonconvex Optimization for Machine Learning: Gradients, Stochasticity, and Saddle Points
Authors: Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M. Kakade, Michael. I. Jordan
Date: 2019-02-13 00:00:00
Tasks: BIG-bench Machine Learning

Gradient descent (GD) and stochastic gradient descent (SGD) are the workhorses of large-scale machine learning. While classical theory focused on analyzing the performance of these methods in convex optimization problems, the most notable successes in machine learning have involved nonconvex optimization, and a gap has arisen between theory and practice. Indeed, traditional analyses of GD and SGD show that both algorithms converge to stationary points efficiently. But these analyses do not take into account the possibility of converging to saddle points. More recent theory has shown that GD and SGD can avoid saddle points, but the dependence on dimension in these analyses is polynomial. For modern machine learning, where the dimension can be in the millions, such dependence would be catastrophic. We analyze perturbed versions of GD and SGD and show that they are truly efficient---their dimension dependence is only polylogarithmic. Indeed, these algorithms converge to second-order stationary points in essentially the same time as they take to converge to classical first-order stationary points.