pwc-archive/papers-with-abstracts huggingface dataset:
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs
Authors: Valentina Zantedeschi, Aurélien Bellet, Marc Tommasi
Date: 2019-01-24 00:00:00

We consider the fully decentralized machine learning scenario where many users with personal datasets collaborate to learn models through local peer-to-peer exchanges, without a central coordinator. We propose to train personalized models that leverage a collaboration graph describing the relationships between user personal tasks, which we learn jointly with the models. Our fully decentralized optimization procedure alternates between training nonlinear models given the graph in a greedy boosting manner, and updating the collaboration graph (with controlled sparsity) given the models. Throughout the process, users exchange messages only with a small number of peers (their direct neighbors when updating the models, and a few random users when updating the graph), ensuring that the procedure naturally scales with the number of users. Overall, our approach is communication-efficient and avoids exchanging personal data. We provide an extensive analysis of the convergence rate, memory and communication complexity of our approach, and demonstrate its benefits compared to competing techniques on synthetic and real datasets.
Emotion Detection and Analysis on Social Media
Authors: Bharat Gaind, Varun Syal, Sneha Padgalwar
Date: 2019-01-24 00:00:00
Tasks: General Classification, Opinion Mining, Sentiment Analysis

In this paper, we address the problem of detection, classification and quantification of emotions of text in any form. We consider English text collected from social media like Twitter, which can provide information having utility in a variety of ways, especially opinion mining. Social media like Twitter and Facebook is full of emotions, feelings and opinions of people all over the world. However, analyzing and classifying text on the basis of emotions is a big challenge and can be considered as an advanced form of Sentiment Analysis. This paper proposes a method to classify text into six different Emotion-Categories: Happiness, Sadness, Fear, Anger, Surprise and Disgust. In our model, we use two different approaches and combine them to effectively extract these emotions from text. The first approach is based on Natural Language Processing, and uses several textual features like emoticons, degree words and negations, Parts Of Speech and other grammatical analysis. The second approach is based on Machine Learning classification algorithms. We have also successfully devised a method to automate the creation of the training-set itself, so as to eliminate the need of manual annotation of large datasets. Moreover, we have managed to create a large bag of emotional words, along with their emotion-intensities. On testing, it is shown that our model provides significant accuracy in classifying tweets taken from Twitter.
Semantic Classification of Tabular Datasets via Character-Level Convolutional Neural Networks
Authors: Paul Azunre, Craig Corcoran, Numa Dhamani, Jeffrey Gleason, Garrett Honke, David Sullivan, Rebecca Ruppel, Sandeep Verma, Jonathon Morgan
Date: 2019-01-24 00:00:00
Tasks: AutoML, Classification, General Classification, Transfer Learning

A character-level convolutional neural network (CNN) motivated by
applications in "automated machine learning" (AutoML) is proposed to
semantically classify columns in tabular data. Simulated data containing a set
of base classes is first used to learn an initial set of weights. Hand-labeled
data from the CKAN repository is then used in a transfer-learning paradigm to
adapt the initial weights to a more sophisticated representation of the problem
(e.g., including more classes). In doing so, realistic data imperfections are
learned and the set of classes handled can be expanded from the base set with
reduced labeled data and computing power requirements. Results show the
effectiveness and flexibility of this approach in three diverse domains:
semantic classification of tabular data, age prediction from social media
posts, and email spam classification. In addition to providing further evidence
of the effectiveness of transfer learning in natural language processing (NLP),
our experiments suggest that analyzing the semantic structure of language at
the character level without additional metadata---i.e., network structure,
headers, etc.---can produce competitive accuracy for type classification, spam
classification, and social media age prediction. We present our open-source
toolkit SIMON, an acronym for Semantic Inference for the Modeling of
ONtologies, which implements this approach in a user-friendly and
scalable/parallelizable fashion.
Really should we pruning after model be totally trained? Pruning based on a small amount of training
Authors: Li Yue, Zhao Weibin, Shang Lin
Date: 2019-01-24 00:00:00
Tasks: Decision Making

Pre-training of models in pruning algorithms plays an important role in
pruning decision-making. We find that excessive pre-training is not necessary
for pruning algorithms. According to this idea, we propose a pruning
algorithm---Incremental pruning based on less training (IPLT). Compared with
the traditional pruning algorithm based on a large number of pre-training, IPLT
has competitive compression effect than the traditional pruning algorithm under
the same simple pruning strategy. On the premise of ensuring accuracy, IPLT can
achieve 8x-9x compression for VGG-19 on CIFAR-10 and only needs to pre-train
few epochs. For VGG-19 on CIFAR-10, we can not only achieve 10 times test
acceleration, but also about 10 times training acceleration. At present, the
research mainly focuses on the compression and acceleration in the application
stage of the model, while the compression and acceleration in the training
stage are few. We newly proposed a pruning algorithm that can compress and
accelerate in the training stage. It is novel to consider the amount of
pre-training required by pruning algorithm. Our results have implications: Too
much pre-training may be not necessary for pruning algorithms.
CT synthesis from MR images for orthopedic applications in the lower arm using a conditional generative adversarial network
Authors: Frank Zijlstra, Koen Willemsen, Mateusz C. Florkow, Ralph J. B. Sakkers, Harrie H. Weinans, Bart C. H. van der Wal, Marijn van Stralen, Peter R. Seevinck
Date: 2019-01-24 00:00:00
Tasks: Generative Adversarial Network

Purpose: To assess the feasibility of deep learning-based high resolution
synthetic CT generation from MRI scans of the lower arm for orthopedic
applications.
  Methods: A conditional Generative Adversarial Network was trained to
synthesize CT images from multi-echo MR images. A training set of MRI and CT
scans of 9 ex vivo lower arms was acquired and the CT images were registered to
the MRI images. Three-fold cross-validation was applied to generate independent
results for the entire dataset. The synthetic CT images were quantitatively
evaluated with the mean absolute error metric, and Dice similarity and surface
to surface distance on cortical bone segmentations.
  Results: The mean absolute error was 63.5 HU on the overall tissue volume and
144.2 HU on the cortical bone. The mean Dice similarity of the cortical bone
segmentations was 0.86. The average surface to surface distance between bone on
real and synthetic CT was 0.48 mm. Qualitatively, the synthetic CT images
corresponded well with the real CT scans and partially maintained high
resolution structures in the trabecular bone. The bone segmentations on
synthetic CT images showed some false positives on tendons, but the general
shape of the bone was accurately reconstructed.
  Conclusions: This study demonstrates that high quality synthetic CT can be
generated from MRI scans of the lower arm. The good correspondence of the bone
segmentations demonstrates that synthetic CT could be competitive with real CT
in applications that depend on such segmentations, such as planning of
orthopedic surgery and 3D printing.
Learning to compress and search visual data in large-scale systems
Authors: Sohrab Ferdowsi
Date: 2019-01-24 00:00:00
Tasks: Compressive Sensing, Denoising, Image Compression, Image Denoising, Retrieval

The problem of high-dimensional and large-scale representation of visual data
is addressed from an unsupervised learning perspective. The emphasis is put on
discrete representations, where the description length can be measured in bits
and hence the model capacity can be controlled. The algorithmic infrastructure
is developed based on the synthesis and analysis prior models whose
rate-distortion properties, as well as capacity vs. sample complexity
trade-offs are carefully optimized. These models are then extended to
multi-layers, namely the RRQ and the ML-STC frameworks, where the latter is
further evolved as a powerful deep neural network architecture with fast and
sample-efficient training and discrete representations. For the developed
algorithms, three important applications are developed. First, the problem of
large-scale similarity search in retrieval systems is addressed, where a
double-stage solution is proposed leading to faster query times and shorter
database storage. Second, the problem of learned image compression is targeted,
where the proposed models can capture more redundancies from the training
images than the conventional compression codecs. Finally, the proposed
algorithms are used to solve ill-posed inverse problems. In particular, the
problems of image denoising and compressive sensing are addressed with
promising results.
A XGBoost risk model via feature selection and Bayesian hyper-parameter optimization
Authors: Yan Wang, Xuelei Sherry Ni
Date: 2019-01-24 00:00:00
Tasks: Clustering, Feature Importance, feature selection, General Classification

This paper aims to explore models based on the extreme gradient boosting
(XGBoost) approach for business risk classification. Feature selection (FS)
algorithms and hyper-parameter optimizations are simultaneously considered
during model training. The five most commonly used FS methods including weight
by Gini, weight by Chi-square, hierarchical variable clustering, weight by
correlation, and weight by information are applied to alleviate the effect of
redundant features. Two hyper-parameter optimization approaches, random search
(RS) and Bayesian tree-structured Parzen Estimator (TPE), are applied in
XGBoost. The effect of different FS and hyper-parameter optimization methods on
the model performance are investigated by the Wilcoxon Signed Rank Test. The
performance of XGBoost is compared to the traditionally utilized logistic
regression (LR) model in terms of classification accuracy, area under the curve
(AUC), recall, and F1 score obtained from the 10-fold cross validation. Results
show that hierarchical clustering is the optimal FS method for LR while weight
by Chi-square achieves the best performance in XG-Boost. Both TPE and RS
optimization in XGBoost outperform LR significantly. TPE optimization shows a
superiority over RS since it results in a significantly higher accuracy and a
marginally higher AUC, recall and F1 score. Furthermore, XGBoost with TPE
tuning shows a lower variability than the RS method. Finally, the ranking of
feature importance based on XGBoost enhances the model interpretation.
Therefore, XGBoost with Bayesian TPE hyper-parameter optimization serves as an
operative while powerful approach for business risk modeling.
Provable Smoothness Guarantees for Black-Box Variational Inference
Authors: Justin Domke
Date: 2019-01-24 00:00:00
Proceeding: ICML 2020 1
Tasks: Variational Inference

Black-box variational inference tries to approximate a complex target distribution though a gradient-based optimization of the parameters of a simpler distribution. Provable convergence guarantees require structural properties of the objective. This paper shows that for location-scale family approximations, if the target is M-Lipschitz smooth, then so is the objective, if the entropy is excluded. The key proof idea is to describe gradients in a certain inner-product space, thus permitting use of Bessel's inequality. This result gives insight into how to parameterize distributions, gives bounds the location of the optimal parameters, and is a key ingredient for convergence guarantees.
Transfer-Learning Oriented Class Imbalance Learning for Cross-Project Defect Prediction
Authors: Haonan Tong, Bin Liu, Shihai Wang, Qiuying Li
Date: 2019-01-24 00:00:00
Tasks: Feature Importance, Transfer Learning

Cross-project defect prediction (CPDP) aims to predict defects of projects
lacking training data by using prediction models trained on historical defect
data from other projects. However, since the distribution differences between
datasets from different projects, it is still a challenge to build high-quality
CPDP models. Unfortunately, class imbalanced nature of software defect datasets
further increases the difficulty. In this paper, we propose a transferlearning
oriented minority over-sampling technique (TOMO) based feature weighting
transfer naive Bayes (FWTNB) approach (TOMOFWTNB) for CPDP by considering both
classimbalance and feature importance problems. Differing from traditional
over-sampling techniques, TOMO not only can balance the data but reduce the
distribution difference. And then FWTNB is used to further increase the
similarity of two distributions. Experiments are performed on 11 public defect
datasets. The experimental results show that (1) TOMO improves the average
G-Measure by 23.7\%$\sim$41.8\%, and the average MCC by 54.2\%$\sim$77.8\%. (2)
feature weighting (FW) strategy improves the average G-Measure by 11\%, and the
average MCC by 29.2\%. (3) TOMOFWTNB improves the average G-Measure value by at
least 27.8\%, and the average MCC value by at least 71.5\%, compared with
existing state-of-theart CPDP approaches. It can be concluded that (1) TOMO is
very effective for addressing class-imbalance problem in CPDP scenario; (2) our
FW strategy is helpful for CPDP; (3) TOMOFWTNB outperforms previous
state-of-the-art CPDP approaches.
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group
Authors: Mario Lezcano-Casado, David Martínez-Rubio
Date: 2019-01-24 00:00:00

We introduce a novel approach to perform first-order optimization with orthogonal and unitary constraints. This approach is based on a parametrization stemming from Lie group theory through the exponential map. The parametrization transforms the constrained optimization problem into an unconstrained one over a Euclidean space, for which common first-order optimization methods can be used. The theoretical results presented are general enough to cover the special orthogonal group, the unitary group and, in general, any connected compact Lie group. We discuss how this and other parametrizations can be computed efficiently through an implementation trick, making numerically complex parametrizations usable at a negligible runtime cost in neural networks. In particular, we apply our results to RNNs with orthogonal recurrent weights, yielding a new architecture called expRNN. We demonstrate how our method constitutes a more robust approach to optimization with orthogonal constraints, showing faster, accurate, and more stable convergence in several tasks designed to test RNNs.
Securing Tag-based recommender systems against profile injection attacks: A comparative study. (Extended Report)
Authors: Georgios K. Pitsilis, Heri Ramampiaro, Helge Langseth
Date: 2019-01-24 00:00:00
Tasks: Deep Learning, Recommendation Systems, TAG

This work addresses the challenges related to attacks on collaborative
tagging systems, which often comes in a form of malicious annotations or
profile injection attacks. In particular, we study various countermeasures
against two types of such attacks for social tagging systems, the Overload
attack and the Piggyback attack. The countermeasure schemes studied here
include baseline classifiers such as, Naive Bayes filter and Support Vector
Machine, as well as a Deep Learning approach. Our evaluation performed over
synthetic spam data generated from del.icio.us dataset, shows that in most
cases, Deep Learning can outperform the classical solutions, providing
high-level protection against threats.
A model for a Lindenmayer reconstruction algorithm
Authors: Diego Gabriel Krivochen, Beth Phillips
Date: 2019-01-24 00:00:00
Tasks: model

Given an input string s and a specific Lindenmayer system (the so-called
Fibonacci grammar), we define an automaton which is capable of (i) determining
whether s belongs to the set of strings that the Fibonacci grammar can generate
(in other words, if s corresponds to a generation of the grammar) and, if so,
(ii) reconstructing the previous generation.
Hybrid NER System for Multi-Source Offer Feeds
Authors: Anusha Holla, Bharat Gaind, Vikas Reddy Katta, Abhishek Kundu, S Kamalesh
Date: 2019-01-24 00:00:00
Tasks: NER

Data available across the web is largely unstructured. Offers published by multiple sources like banks, digital wallets, merchants, etc., are one of the most accessed advertising data in today's world. This data gets accessed by millions of people on a daily basis and is easily interpreted by humans, but since it is largely unstructured and diverse, using an algorithmic way to extract meaningful information out of these offers is hard. Identifying the essential offer entities (for instance, its amount, the product on which the offer is applicable, the merchant providing the offer, etc.) from these offers plays a vital role in targeting the right customers to improve sales. This work presents and evaluates various existing Named Entity Recognizer (NER) models which can identify the required entities from offer feeds. We also propose a novel Hybrid NER model constructed by two-level stacking of Conditional Random Field, Bidirectional LSTM and Spacy models at the first level and an SVM classifier at the second. The proposed hybrid model has been tested on offer feeds collected from multiple sources and has shown better performance in the offer domain when compared to the existing models.
To Relieve Your Headache of Training an MRF, Take AdVIL
Authors: Chongxuan Li, Chao Du, Kun Xu, Max Welling, Jun Zhu, Bo Zhang
Date: 2019-01-24 00:00:00
Proceeding: ICLR 2020 1
Tasks: Variational Inference

We propose a black-box algorithm called {\it Adversarial Variational Inference and Learning} (AdVIL) to perform inference and learning on a general Markov random field (MRF). AdVIL employs two variational distributions to approximately infer the latent variables and estimate the partition function of an MRF, respectively. The two variational distributions provide an estimate of the negative log-likelihood of the MRF as a minimax optimization problem, which is solved by stochastic gradient descent. AdVIL is proven convergent under certain conditions. On one hand, compared with contrastive divergence, AdVIL requires a minimal assumption about the model structure and can deal with a broader family of MRFs. On the other hand, compared with existing black-box methods, AdVIL provides a tighter estimate of the log partition function and achieves much better empirical results.
Self-Supervised Deep Learning on Point Clouds by Reconstructing Space
Authors: Jonathan Sauder, Bjarne Sievers
Date: 2019-01-24 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: 3D Point Cloud Linear Classification, Deep Learning, General Classification, Point Cloud Pre-training, Self-Driving Cars, Self-Supervised Learning, Semantic Segmentation, Unsupervised 3D Point Cloud Linear Evaluation

Point clouds provide a flexible and natural representation usable in countless applications such as robotics or self-driving cars. Recently, deep neural networks operating on raw point cloud data have shown promising results on supervised learning tasks such as object classification and semantic segmentation. While massive point cloud datasets can be captured using modern scanning technology, manually labelling such large 3D point clouds for supervised learning tasks is a cumbersome process. This necessitates methods that can learn from unlabelled data to significantly reduce the number of annotated samples needed in supervised learning. We propose a self-supervised learning task for deep learning on raw point cloud data in which a neural network is trained to reconstruct point clouds whose parts have been randomly rearranged. While solving this task, representations that capture semantic properties of the point cloud are learned. Our method is agnostic of network architecture and outperforms current unsupervised learning approaches in downstream object classification tasks. We show experimentally, that pre-training with our method before supervised training improves the performance of state-of-the-art models and significantly improves sample efficiency.
Application of Decision Rules for Handling Class Imbalance in Semantic Segmentation
Authors: Robin Chan, Matthias Rottmann, Fabian Hüger, Peter Schlicht, Hanno Gottschalk
Date: 2019-01-24 00:00:00
Tasks: Semantic Segmentation

As part of autonomous car driving systems, semantic segmentation is an
essential component to obtain a full understanding of the car's environment.
One difficulty, that occurs while training neural networks for this purpose, is
class imbalance of training data. Consequently, a neural network trained on
unbalanced data in combination with maximum a-posteriori classification may
easily ignore classes that are rare in terms of their frequency in the dataset.
However, these classes are often of highest interest. We approach such
potential misclassifications by weighting the posterior class probabilities
with the prior class probabilities which in our case are the inverse
frequencies of the corresponding classes in the training dataset. More
precisely, we adopt a localized method by computing the priors pixel-wise such
that the impact can be analyzed at pixel level as well. In our experiments, we
train one network from scratch using a proprietary dataset containing 20,000
annotated frames of video sequences recorded from street scenes. The evaluation
on our test set shows an increase of average recall with regard to instances of
pedestrians and info signs by $25\%$ and $23.4\%$, respectively. In addition,
we significantly reduce the non-detection rate for instances of the same
classes by $61\%$ and $38\%$.
Regret Minimisation in Multi-Armed Bandits Using Bounded Arm Memory
Authors: Arghya Roy Chaudhuri, Shivaram Kalyanakrishnan
Date: 2019-01-24 00:00:00
Tasks: Multi-Armed Bandits

In this paper, we propose a constant word (RAM model) algorithm for regret
minimisation for both finite and infinite Stochastic Multi-Armed Bandit (MAB)
instances. Most of the existing regret minimisation algorithms need to remember
the statistics of all the arms they encounter. This may become a problem for
the cases where the number of available words of memory is limited. Designing
an efficient regret minimisation algorithm that uses a constant number of words
has long been interesting to the community. Some early attempts consider the
number of arms to be infinite, and require the reward distribution of the arms
to belong to some particular family. Recently, for finitely many-armed bandits
an explore-then-commit based algorithm~\citep{Liau+PSY:2018} seems to escape
such assumption. However, due to the underlying PAC-based elimination their
method incurs a high regret. We present a conceptually simple, and efficient
algorithm that needs to remember statistics of at most $M$ arms, and for any
$K$-armed finite bandit instance it enjoys a $O(KM +K^{1.5}\sqrt{T\log
(T/MK)}/M)$ upper-bound on regret. We extend it to achieve sub-linear
\textit{quantile-regret}~\citep{RoyChaudhuri+K:2018} and empirically verify the
efficiency of our algorithm via experiments.