pwc-archive/papers-with-abstracts huggingface dataset:
Joint Optimization of Tree-based Index and Deep Model for Recommender Systems
Authors: Han Zhu, Daqing Chang, Ziru Xu, Pengye Zhang, Xiang Li, Jie He, Han Li, Jian Xu, Kun Gai
Date: 2019-02-19 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Prediction, Recommendation Systems, Retrieval

Large-scale industrial recommender systems are usually confronted with computational problems due to the enormous corpus size. To retrieve and recommend the most relevant items to users under response time limits, resorting to an efficient index structure is an effective and practical solution. The previous work Tree-based Deep Model (TDM) \cite{zhu2018learning} greatly improves recommendation accuracy using tree index. By indexing items in a tree hierarchy and training a user-node preference prediction model satisfying a max-heap like property in the tree, TDM provides logarithmic computational complexity w.r.t. the corpus size, enabling the use of arbitrary advanced models in candidate retrieval and recommendation. In tree-based recommendation methods, the quality of both the tree index and the user-node preference prediction model determines the recommendation accuracy for the most part. We argue that the learning of tree index and preference model has interdependence. Our purpose, in this paper, is to develop a method to jointly learn the index structure and user preference prediction model. In our proposed joint optimization framework, the learning of index and user preference prediction model are carried out under a unified performance measure. Besides, we come up with a novel hierarchical user preference representation utilizing the tree index hierarchy. Experimental evaluations with two large-scale real-world datasets show that the proposed method improves recommendation accuracy significantly. Online A/B test results at a display advertising platform also demonstrate the effectiveness of the proposed method in production environments.
On the consistency of supervised learning with missing values
Authors: Julie Josse, Jacob M. Chen, Nicolas Prost, Erwan Scornet, Gaël Varoquaux
Date: 2019-02-19 00:00:00
Tasks: Attribute, Imputation, Missing Values

In many application settings, the data have missing entries which make analysis challenging. An abundant literature addresses missing values in an inferential framework: estimating parameters and their variance from incomplete tables. Here, we consider supervised-learning settings: predicting a target when missing values appear in both training and testing data. We show the consistency of two approaches in prediction. A striking result is that the widely-used method of imputing with a constant, such as the mean prior to learning is consistent when missing values are not informative. This contrasts with inferential settings where mean imputation is pointed at for distorting the distribution of the data. That such a simple approach can be consistent is important in practice. We also show that a predictor suited for complete observations can predict optimally on incomplete data, through multiple imputation. Finally, to compare imputation with learning directly with a model that accounts for missing values, we analyze further decision trees. These can naturally tackle empirical risk minimization with missing values, due to their ability to handle the half-discrete nature of incomplete variables. After comparing theoretically and empirically different missing values strategies in trees, we recommend using the "missing incorporated in attribute" method as it can handle both non-informative and informative missing values.
Using an Ancillary Neural Network to Capture Weekends and Holidays in an Adjoint Neural Network Architecture for Intelligent Building Management
Authors: Zhicheng Ding, Mehmet Kerem Turkcan, Albert Boulanger
Date: 2018-12-26 00:00:00
Tasks: Management, Prediction, Time Series, Time Series Analysis, Time Series Prediction

The US EIA estimated in 2017 about 39\% of total U.S. energy consumption was
by the residential and commercial sectors. Therefore, Intelligent Building
Management (IBM) solutions that minimize consumption while maintaining tenant
comfort are an important component in addressing climate change. A forecasting
capability for accurate prediction of indoor temperatures in a planning horizon
of 24 hours is essential to IBM. It should predict the indoor temperature in
both short-term (e.g. 15 minutes) and long-term (e.g. 24 hours) periods
accurately including weekends, major holidays, and minor holidays. Other
requirements include the ability to predict the maximum and the minimum indoor
temperatures precisely and provide the confidence for each prediction. To
achieve these requirements, we propose a novel adjoint neural network
architecture for time series prediction that uses an ancillary neural network
to capture weekend and holiday information. We studied four long short-term
memory (LSTM) based time series prediction networks within this architecture.
We observed that the ancillary neural network helps to improve the prediction
accuracy, the maximum and the minimum temperature prediction and model
reliability for all networks tested.
How to Host a Data Competition: Statistical Advice for Design and Analysis of a Data Competition
Authors: Christine M. Anderson-Cook, Kary L. Myers, Lu Lu, Michael L. Fugate, Kevin R. Quinlan, Norma Pawley
Date: 2019-01-16 00:00:00

Data competitions rely on real-time leaderboards to rank competitor entries
and stimulate algorithm improvement. While such competitions have become quite
popular and prevalent, particularly in supervised learning formats, their
implementations by the host are highly variable. Without careful planning, a
supervised learning competition is vulnerable to overfitting, where the winning
solutions are so closely tuned to the particular set of provided data that they
cannot generalize to the underlying problem of interest to the host. This paper
outlines some important considerations for strategically designing relevant and
informative data sets to maximize the learning outcome from hosting a
competition based on our experience. It also describes a post-competition
analysis that enables robust and efficient assessment of the strengths and
weaknesses of solutions from different competitors, as well as greater
understanding of the regions of the input space that are well-solved. The
post-competition analysis, which complements the leaderboard, uses exploratory
data analysis and generalized linear models (GLMs). The GLMs not only expand
the range of results we can explore, they also provide more detailed analysis
of individual sub-questions including similarities and differences between
algorithms across different types of scenarios, universally easy or hard
regions of the input space, and different learning objectives. When coupled
with a strategically planned data generation approach, the methods provide
richer and more informative summaries to enhance the interpretation of results
beyond just the rankings on the leaderboard. The methods are illustrated with a
recently completed competition to evaluate algorithms capable of detecting,
identifying, and locating radioactive materials in an urban environment.
Discovery of Natural Language Concepts in Individual Units of CNNs
Authors: Seil Na, Yo Joong Choe, Dong-Hyun Lee, Gunhee Kim
Date: 2019-02-18 00:00:00
Proceeding: ICLR 2019 5
Tasks: Concept Alignment, General Classification, Translation

Although deep convolutional networks have achieved improved performance in
many natural language tasks, they have been treated as black boxes because they
are difficult to interpret. Especially, little is known about how they
represent language in their intermediate layers. In an attempt to understand
the representations of deep convolutional networks trained on language tasks,
we show that individual units are selectively responsive to specific morphemes,
words, and phrases, rather than responding to arbitrary and uninterpretable
patterns. In order to quantitatively analyze such an intriguing phenomenon, we
propose a concept alignment method based on how units respond to the replicated
text. We conduct analyses with different architectures on multiple datasets for
classification and translation tasks and provide new insights into how deep
models understand natural language.
Deep Modulation Embedding
Authors: Amin Abbasloo, Alan Salari
Date: 2019-02-17 00:00:00
Tasks: GPU

Deep neural network has recently shown very promising applications in
different research directions and attracted the industry attention as well.
Although the idea was introduced in the past but just recently the main
limitation of using this class of algorithms is solved by enabling parallel
computing on GPU hardware. Opening the possibility of hardware prototyping with
proven superiority of this class of algorithm, trigger several research
directions in communication system too. Among them cognitive radio, modulation
recognition, learning based receiver and transceiver are already given very
interesting result in simulation and real experimental evaluation implemented
on software defined radio. Specifically, modulation recognition is mostly
approached as a classification problem which is a supervised learning
framework. But it is here addressed as an unsupervised problem with introducing
new features for training, a new loss function and investigating the robustness
of the pipeline against several mismatch conditions.
A Machine Learning based Robust Prediction Model for Real-life Mobile Phone Data
Authors: Iqbal H. Sarker
Date: 2019-02-11 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

Real-life mobile phone data may contain noisy instances, which is a
fundamental issue for building a prediction model with many potential negative
consequences. The complexity of the inferred model may increase, may arise
overfitting problem, and thereby the overall prediction accuracy of the model
may decrease. In this paper, we address these issues and present a robust
prediction model for real-life mobile phone data of individual users, in order
to improve the prediction accuracy of the model. In our robust model, we first
effectively identify and eliminate the noisy instances from the training
dataset by determining a dynamic noise threshold using naive Bayes classifier
and laplace estimator, which may differ from user-to-user according to their
unique behavioral patterns. After that, we employ the most popular rule-based
machine learning classification technique, i.e., decision tree, on the
noise-free quality dataset to build the prediction model. Experimental results
on the real-life mobile phone datasets (e.g., phone call log) of individual
mobile phone users, show the effectiveness of our robust model in terms of
precision, recall and f-measure.
Robust Structured Group Local Sparse Tracker Using Deep Features
Authors: Mohammadreza Javanmardi, Amir Hossein Farzaneh, Xiaojun Qi
Date: 2019-02-18 00:00:00
Tasks: Visual Tracking

Sparse representation has recently been successfully applied in visual tracking. It utilizes a set of templates to represent target candidates and find the best one with the minimum reconstruction error as the tracking result. In this paper, we propose a robust deep features-based structured group local sparse tracker (DF-SGLST), which exploits the deep features of local patches inside target candidates and represents them by a set of templates in the particle filter framework. Unlike the conventional local sparse trackers, the proposed optimization model in DF-SGLST employs a group-sparsity regularization term to seamlessly adopt local and spatial information of the target candidates and attain the spatial layout structure among them. To solve the optimization model, we propose an efficient and fast numerical algorithm that consists of two subproblems with the closed-form solutions. Different evaluations in terms of success and precision on the benchmarks of challenging image sequences (e.g., OTB50 and OTB100) demonstrate the superior performance of the proposed tracker against several state-of-the-art trackers.
Characterization of Brain Cortical Morphology Using Localized Topology-Encoding Graphs
Authors: Sevil Maghsadhagh, Mousa Shamsi, Anders Eklund, Hamid Behjat
Date: 2018-10-17 00:00:00

The human brain cortical layer has a convoluted morphology that is unique to
each individual. Characterization of the cortical morphology is necessary in
longitudinal studies of structural brain change, as well as in discriminating
individuals in health and disease. A method for encoding the cortical
morphology in the form of a graph is presented. The design of graphs that
encode the global cerebral hemisphere cortices as well as localized cortical
regions is proposed. Spectral features of these graphs are then studied and
proposed as descriptors of cortical morphology. As proof-of-concept of their
applicability in characterizing cortical morphology, the descriptors are
studied in the context of discriminating individuals based on their sex.
Sentence Compression via DC Programming Approach
Authors: Yi-Shuai Niu, Xi-Wei Hu, Yu You, Faouzi Mohamed Benammour, Hu Zhang
Date: 2019-02-13 00:00:00
Tasks: Sentence, Sentence Compression

Sentence compression is an important problem in natural language processing.
In this paper, we firstly establish a new sentence compression model based on
the probability model and the parse tree model. Our sentence compression model
is equivalent to an integer linear program (ILP) which can both guarantee the
syntax correctness of the compression and save the main meaning. We propose
using a DC (Difference of convex) programming approach (DCA) for finding local
optimal solution of our model. Combing DCA with a parallel-branch-and-bound
framework, we can find global optimal solution. Numerical results demonstrate
the good quality of our sentence compression model and the excellent
performance of our proposed solution algorithm.
Neural Machine Translation for Cebuano to Tagalog with Subword Unit Translation
Authors: Kristine Mae M. Adlaon, Nelson Marcos
Date: 2019-02-10 00:00:00
Tasks: Diversity, Machine Translation, Translation

The Philippines is an archipelago composed of 7, 641 different islands with
more than 150 different languages. This linguistic differences and diversity,
though may be seen as a beautiful feature, have contributed to the difficulty
in the promotion of educational and cultural development of different domains
in the country. An effective machine translation system solely dedicated to
cater Philippine languages will surely help bridge this gap. In this research
work, a never before applied approach for language translation to a Philippine
language was used for a Cebuano to Tagalog translator. A Recurrent Neural
Network was used to implement the translator using OpenNMT sequence modeling
tool in TensorFlow. The performance of the translation was evaluated using the
BLEU Score metric. For the Cebuano to Tagalog translation, BLEU produced a
score of 20.01. A subword unit translation for verbs and copyable approach was
performed where commonly seen mistranslated words from the source to the target
were corrected. The BLEU score increased to 22.87. Though slightly higher, this
score still indicates that the translation is somehow understandable but is not
yet considered as a good translation.
Towards a Robust Deep Neural Network in Texts: A Survey
Authors: Wenqi Wang, Run Wang, Lina Wang, Zhibo Wang, Aoshuang Ye
Date: 2019-02-12 00:00:00
Tasks: General Classification, image-classification, Image Classification, speech-recognition, Speech Recognition, Survey

Deep neural networks (DNNs) have achieved remarkable success in various tasks (e.g., image classification, speech recognition, and natural language processing (NLP)). However, researchers have demonstrated that DNN-based models are vulnerable to adversarial examples, which cause erroneous predictions by adding imperceptible perturbations into legitimate inputs. Recently, studies have revealed adversarial examples in the text domain, which could effectively evade various DNN-based text analyzers and further bring the threats of the proliferation of disinformation. In this paper, we give a comprehensive survey on the existing studies of adversarial techniques for generating adversarial texts written by both English and Chinese characters and the corresponding defense methods. More importantly, we hope that our work could inspire future studies to develop more robust DNN-based text analyzers against known and unknown adversarial techniques. We classify the existing adversarial techniques for crafting adversarial texts based on the perturbation units, helping to better understand the generation of adversarial texts and build robust models for defense. In presenting the taxonomy of adversarial attacks and defenses in the text domain, we introduce the adversarial techniques from the perspective of different NLP tasks. Finally, we discuss the existing challenges of adversarial attacks and defenses in texts and present the future research directions in this emerging and challenging field.
Sample Efficient Imitation Learning for Continuous Control
Authors: Fumihiro Sasaki, Tetsuya Yohira, Atsuo Kawaguchi
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: continuous-control, Continuous Control, Imitation Learning

The goal of imitation learning (IL) is to enable a learner to imitate expert behavior given expert demonstrations. Recently, generative adversarial imitation learning (GAIL) has shown significant progress on IL for complex continuous tasks. However, GAIL and its extensions require a large number of environment interactions during training. In real-world environments, the more an IL method requires the learner to interact with the environment for better imitation, the more training time it requires, and the more damage it causes to the environments and the learner itself. We believe that IL algorithms could be more applicable to real-world problems if the number of interactions could be reduced. 
In this paper, we propose a model-free IL algorithm for continuous control. Our algorithm is made up mainly three changes to the existing adversarial imitation learning (AIL) methods – (a) adopting off-policy actor-critic (Off-PAC) algorithm to optimize the learner policy, (b) estimating the state-action value using off-policy samples without learning reward functions, and (c) representing the stochastic policy function so that its outputs are bounded. Experimental results show that our algorithm achieves competitive results with GAIL while significantly reducing the environment interactions.
Transfusion: Understanding Transfer Learning for Medical Imaging
Authors: Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, Samy Bengio
Date: 2019-02-14 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: image-classification, Image Classification, Transfer Learning

Transfer learning from natural image datasets, particularly ImageNet, using standard large models and corresponding pretrained weights has become a de-facto method for deep learning applications to medical imaging. However, there are fundamental differences in data sizes, features and task specifications between natural image classification and the target medical tasks, and there is little understanding of the effects of transfer. In this paper, we explore properties of transfer learning for medical imaging. A performance evaluation on two large scale medical imaging tasks shows that surprisingly, transfer offers little benefit to performance, and simple, lightweight models can perform comparably to ImageNet architectures. Investigating the learned representations and features, we find that some of the differences from transfer learning are due to the over-parametrization of standard models rather than sophisticated feature reuse. We isolate where useful feature reuse occurs, and outline the implications for more efficient model exploration. We also explore feature independent benefits of transfer arising from weight scalings.
Leveraging Communication Topologies Between Learning Agents in Deep Reinforcement Learning
Authors: Dhaval Adjodah, Dan Calacci, Abhimanyu Dubey, Anirudh Goyal, Peter Krafft, Esteban Moro, Alex Pentland
Date: 2019-02-16 00:00:00
Tasks: BIG-bench Machine Learning, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A common technique to improve learning performance in deep reinforcement learning (DRL) and many other machine learning algorithms is to run multiple learning agents in parallel. A neglected component in the development of these algorithms has been how best to arrange the learning agents involved to improve distributed search. Here we draw upon results from the networked optimization literatures suggesting that arranging learning agents in communication networks other than fully connected topologies (the implicit way agents are commonly arranged in) can improve learning. We explore the relative performance of four popular families of graphs and observe that one such family (Erdos-Renyi random graphs) empirically outperforms the de facto fully-connected communication topology across several DRL benchmark tasks. Additionally, we observe that 1000 learning agents arranged in an Erdos-Renyi graph can perform as well as 3000 agents arranged in the standard fully-connected topology, showing the large learning improvement possible when carefully designing the topology over which agents communicate. We complement these empirical results with a theoretical investigation of why our alternate topologies perform better. Overall, our work suggests that distributed machine learning algorithms could be made more effective if the communication topology between learning agents was optimized.
Forecasting the 2017-2018 Yemen Cholera Outbreak with Machine Learning
Authors: Rohil Badkundri, Victor Valbuena, Srikusmanjali Pinnamareddy, Brittney Cantrell, Janet Standeven
Date: 2019-02-16 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering

The ongoing Yemen cholera outbreak has been deemed one of the worst cholera
outbreaks in history, with over a million people impacted and thousands dead.
Triggered by a civil war, the outbreak has been shaped by various political,
environmental, and epidemiological factors and continues to worsen. While
cholera has several effective treatments, the untimely and inefficient
distribution of existing medicines has been the primary cause of cholera
mortality. With the hope of facilitating resource allocation, various
mathematical models have been created to track the Yemeni outbreak and identify
at-risk administrative divisions, called governorates. Existing models are not
powerful enough to accurately and consistently forecast cholera cases per
governorate over multiple timeframes. To address the need for a complex,
reliable model, we offer the Cholera Artificial Learning Model (CALM); a system
of 4 extreme-gradient-boosting (XGBoost) machine learning models that forecast
the number of new cholera cases a Yemeni governorate will experience from a
time range of 2 weeks to 2 months. CALM provides a novel machine learning
approach that makes use of rainfall data, past cholera cases and deaths data,
civil war fatalities, and inter-governorate interactions represented across
multiple time frames. Additionally, the use of machine learning, along with
extensive feature engineering, allows CALM to easily learn complex non-linear
relations apparent in an epidemiological phenomenon. CALM is able to forecast
cholera incidence 2 weeks to 2 months in advance within a margin of just 5
cholera cases per 10,000 people in real-world simulation.