pwc-archive/papers-with-abstracts huggingface dataset:
From open learners to open games
Authors: Jules Hedges
Date: 2019-02-22 00:00:00

The categories of open learners (due to Fong, Spivak and Tuy\'eras) and open
games (due to the present author, Ghani, Winschel and Zahn) bear a very
striking and unexpected similarity. The purpose of this short note is to prove
that there is a faithful symmetric monoidal functor from the former to the
latter, which means that any supervised neural network (without feedback or
other complicating features) can be seen as an open game in a canonical way.
Roughly, each parameter is controlled by a different player, and the game's
best response relation encodes the dynamics of gradient descent. We suggest
paths for further work exploiting the link.
Learning protein sequence embeddings using information from structure
Authors: Tristan Bepler, Bonnie Berger
Date: 2019-02-22 00:00:00
Proceeding: ICLR 2019 5
Tasks: Position, Representation Learning

Inferring the structural properties of a protein from its amino acid sequence is a challenging yet important problem in biology. Structures are not known for the vast majority of protein sequences, but structure is critical for understanding function. Existing approaches for detecting structural similarity between proteins from sequence are unable to recognize and exploit structural patterns when sequences have diverged too far, limiting our ability to transfer knowledge between structurally related proteins. We newly approach this problem through the lens of representation learning. We introduce a framework that maps any protein sequence to a sequence of vector embeddings --- one per amino acid position --- that encode structural information. We train bidirectional long short-term memory (LSTM) models on protein sequences with a two-part feedback mechanism that incorporates information from (i) global structural similarity between proteins and (ii) pairwise residue contact maps for individual proteins. To enable learning from structural similarity information, we define a novel similarity measure between arbitrary-length sequences of vector embeddings based on a soft symmetric alignment (SSA) between them. Our method is able to learn useful position-specific embeddings despite lacking direct observations of position-level correspondence between sequences. We show empirically that our multi-task framework outperforms other sequence-based methods and even a top-performing structure-based alignment method when predicting structural similarity, our goal. Finally, we demonstrate that our learned embeddings can be transferred to other protein sequence problems, improving the state-of-the-art in transmembrane domain prediction.
What makes a good conversation? How controllable attributes affect human judgments
Authors: Abigail See, Stephen Roller, Douwe Kiela, Jason Weston
Date: 2019-02-22 00:00:00
Proceeding: NAACL 2019 6
Tasks: Specificity, Text Generation

A good conversation requires balance -- between simplicity and detail;
staying on topic and changing it; asking questions and answering them. Although
dialogue agents are commonly evaluated via human judgments of overall quality,
the relationship between quality and these individual factors is less
well-studied. In this work, we examine two controllable neural text generation
methods, conditional training and weighted decoding, in order to control four
important attributes for chitchat dialogue: repetition, specificity,
response-relatedness and question-asking. We conduct a large-scale human
evaluation to measure the effect of these control parameters on multi-turn
interactive conversations on the PersonaChat task. We provide a detailed
analysis of their relationship to high-level aspects of conversation, and show
that by controlling combinations of these variables our models obtain clear
improvements in human quality judgments.
Saliency Learning: Teaching the Model Where to Pay Attention
Authors: Reza Ghaeini, Xiaoli Z. Fern, Hamed Shahbazi, Prasad Tadepalli
Date: 2019-02-22 00:00:00
Proceeding: NAACL 2019 6

Deep learning has emerged as a compelling solution to many NLP tasks with
remarkable performances. However, due to their opacity, such models are hard to
interpret and trust. Recent work on explaining deep models has introduced
approaches to provide insights toward the model's behaviour and predictions,
which are helpful for assessing the reliability of the model's predictions.
However, such methods do not improve the model's reliability. In this paper, we
aim to teach the model to make the right prediction for the right reason by
providing explanation training and ensuring the alignment of the model's
explanation with the ground truth explanation. Our experimental results on
multiple tasks and datasets demonstrate the effectiveness of the proposed
method, which produces more reliable predictions while delivering better
results compared to traditionally trained models.
Scalable Hyperbolic Recommender Systems
Authors: Benjamin Paul Chamberlain, Stephen R. Hardwick, David R. Wardrope, Fabon Dzogang, Fabio Daolio, Saúl Vargas
Date: 2019-02-22 00:00:00
Tasks: Recommendation Systems

We present a large scale hyperbolic recommender system. We discuss why
hyperbolic geometry is a more suitable underlying geometry for many
recommendation systems and cover the fundamental milestones and insights that
we have gained from its development. In doing so, we demonstrate the viability
of hyperbolic geometry for recommender systems, showing that they significantly
outperform Euclidean models on datasets with the properties of complex
networks. Key to the success of our approach are the novel choice of underlying
hyperbolic model and the use of the Einstein midpoint to define an asymmetric
recommender system in hyperbolic space. These choices allow us to scale to
millions of users and hundreds of thousands of items.
Better Algorithms for Stochastic Bandits with Adversarial Corruptions
Authors: Anupam Gupta, Tomer Koren, Kunal Talwar
Date: 2019-02-22 00:00:00
Tasks: Multi-Armed Bandits

We study the stochastic multi-armed bandits problem in the presence of
adversarial corruption. We present a new algorithm for this problem whose
regret is nearly optimal, substantially improving upon previous work. Our
algorithm is agnostic to the level of adversarial contamination and can
tolerate a significant amount of corruption with virtually no degradation in
performance.
OpenKiwi: An Open Source Framework for Quality Estimation
Authors: Fábio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, André F. T. Martins
Date: 2019-02-22 00:00:00
Proceeding: ACL 2019 7
Tasks: NMT, Sentence, Translation

We introduce OpenKiwi, a PyTorch-based open source framework for translation quality estimation. OpenKiwi supports training and testing of word-level and sentence-level quality estimation systems, implementing the winning systems of the WMT 2015-18 quality estimation campaigns. We benchmark OpenKiwi on two datasets from WMT 2018 (English-German SMT and NMT), yielding state-of-the-art performance on the word-level tasks and near state-of-the-art in the sentence-level tasks.
Learning Hash Function through Codewords
Authors: Yinjie Huang, Michael Georgiopoulos, Georgios C. Anagnostopoulos
Date: 2019-02-22 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Retrieval

In this paper, we propose a novel hash learning approach that has the
following main distinguishing features, when compared to past frameworks.
First, the codewords are utilized in the Hamming space as ancillary techniques
to accomplish its hash learning task. These codewords, which are inferred from
the data, attempt to capture grouping aspects of the data's hash codes.
Furthermore, the proposed framework is capable of addressing supervised,
unsupervised and, even, semi-supervised hash learning scenarios. Additionally,
the framework adopts a regularization term over the codewords, which
automatically chooses the codewords for the problem. To efficiently solve the
problem, one Block Coordinate Descent algorithm is showcased in the paper. We
also show that one step of the algorithms can be casted into several Support
Vector Machine problems which enables our algorithms to utilize efficient
software package. For the regularization term, a closed form solution of the
proximal operator is provided in the paper. A series of comparative experiments
focused on content-based image retrieval highlights its performance advantages.
MPP: Model Performance Predictor
Authors: Sindhu Ghanta, Sriram Subramanian, Lior Khermosh, Harshil Shah, Yakov Goldberg, Swaminathan Sundararaman, Drew Roselli, Nisha Talagala
Date: 2019-02-22 00:00:00
Tasks: Management, model

Operations is a key challenge in the domain of machine learning pipeline
deployments involving monitoring and management of real-time prediction
quality. Typically, metrics like accuracy, RMSE etc., are used to track the
performance of models in deployment. However, these metrics cannot be
calculated in production due to the absence of labels. We propose using an ML
algorithm, Model Performance Predictor (MPP), to track the performance of the
models in deployment. We argue that an ensemble of such metrics can be used to
create a score representing the prediction quality in production. This in turn
facilitates formulation and customization of ML alerts, that can be escalated
by an operations team to the data science team. Such a score automates
monitoring and enables ML deployments at scale.
Trajectories of Blocked Community Members: Redemption, Recidivism and Departure
Authors: Jonathan P. Chang, Cristian Danescu-Niculescu-Mizil
Date: 2019-02-22 00:00:00
Tasks: Blocking

Community norm violations can impair constructive communication and
collaboration online. As a defense mechanism, community moderators often
address such transgressions by temporarily blocking the perpetrator. Such
actions, however, come with the cost of potentially alienating community
members. Given this tradeoff, it is essential to understand to what extent, and
in which situations, this common moderation practice is effective in
reinforcing community rules.
  In this work, we introduce a computational framework for studying the future
behavior of blocked users on Wikipedia. After their block expires, they can
take several distinct paths: they can reform and adhere to the rules, but they
can also recidivate, or straight-out abandon the community. We reveal that
these trajectories are tied to factors rooted both in the characteristics of
the blocked individual and in whether they perceived the block to be fair and
justified. Based on these insights, we formulate a series of prediction tasks
aiming to determine which of these paths a user is likely to take after being
blocked for their first offense, and demonstrate the feasibility of these new
tasks. Overall, this work builds towards a more nuanced approach to moderation
by highlighting the tradeoffs that are in play.
Bayesian Anomaly Detection and Classification
Authors: Ethan Roberts, Bruce A. Bassett, Michelle Lochner
Date: 2019-02-22 00:00:00
Tasks: Anomaly Detection, Classification, Gaussian Processes, General Classification, Model Selection

Statistical uncertainties are rarely incorporated in machine learning
algorithms, especially for anomaly detection. Here we present the Bayesian
Anomaly Detection And Classification (BADAC) formalism, which provides a
unified statistical approach to classification and anomaly detection within a
hierarchical Bayesian framework. BADAC deals with uncertainties by
marginalising over the unknown, true, value of the data. Using simulated data
with Gaussian noise, BADAC is shown to be superior to standard algorithms in
both classification and anomaly detection performance in the presence of
uncertainties, though with significantly increased computational cost.
Additionally, BADAC provides well-calibrated classification probabilities,
valuable for use in scientific pipelines. We show that BADAC can work in online
mode and is fairly robust to model errors, which can be diagnosed through
model-selection methods. In addition it can perform unsupervised new class
detection and can naturally be extended to search for anomalous subsets of
data. BADAC is therefore ideal where computational cost is not a limiting
factor and statistical rigour is important. We discuss approximations to speed
up BADAC, such as the use of Gaussian processes, and finally introduce a new
metric, the Rank-Weighted Score (RWS), that is particularly suited to
evaluating the ability of algorithms to detect anomalies.
Fault Diagnosis Method Based on Scaling Law for On-line Refrigerant Leak Detection
Authors: Shun Takeuchi, Takahiro Saito
Date: 2019-02-22 00:00:00
Tasks: Diagnostic, Fault Detection, Fault Diagnosis, Time Series, Time Series Analysis

Early fault detection using instrumented sensor data is one of the promising
application areas of machine learning in industrial facilities. However, it is
difficult to improve the generalization performance of the trained
fault-detection model because of the complex system configuration in the target
diagnostic system and insufficient fault data. It is not trivial to apply the
trained model to other systems. Here we propose a fault diagnosis method for
refrigerant leak detection considering the physical modeling and control
mechanism of an air-conditioning system. We derive a useful scaling law related
to refrigerant leak. If the control mechanism is the same, the model can be
applied to other air-conditioning systems irrespective of the system
configuration. Small-scale off-line fault test data obtained in a laboratory
are applied to estimate the scaling exponent. We evaluate the proposed scaling
law by using real-world data. Based on a statistical hypothesis test of the
interaction between two groups, we show that the scaling exponents of different
air-conditioning systems are equivalent. In addition, we estimated the time
series of the degree of leakage of real process data based on the scaling law
and confirmed that the proposed method is promising for early leak detection
through comparison with assessment by experts.
Semi-supervised Approach to Soft Sensor Modeling for Fault Detection in Industrial Systems with Multiple Operation Modes
Authors: Shun Takeuchi, Takuya Nishino, Takahiro Saito, Isamu Watanabe
Date: 2019-02-22 00:00:00
Tasks: Fault Detection, Sensor Modeling

In industrial systems, certain process variables that need to be monitored
for detecting faults are often difficult or impossible to measure. Soft sensor
techniques are widely used to estimate such difficult-to-measure process
variables from easy-to-measure ones. Soft sensor modeling requires training
datasets including the information of various states such as operation modes,
but the fault dataset with the target variable is insufficient as the training
dataset. This paper describes a semi-supervised approach to soft sensor
modeling to incorporate an incomplete dataset without the target variable in
the training dataset. To incorporate the incomplete dataset, we consider the
properties of processes at transition points between operation modes in the
system. The regression coefficients of the operation modes are estimated under
constraint conditions obtained from the information on the mode transitions. In
a case study, this constrained soft sensor modeling was used to predict
refrigerant leaks in air-conditioning systems with heating and cooling
operation modes. The results show that this modeling method is promising for
soft sensors in a system with multiple operation modes.
Similarity Learning Networks for Animal Individual Re-Identification -- Beyond the Capabilities of a Human Observer
Authors: Stefan Schneider, Graham W. Taylor, Stefan Linquist, Stefan C. Kremer
Date: 2019-02-21 00:00:00
Tasks: Object Detection, One-Shot Learning, Triplet

Deep learning has become the standard methodology to approach computer vision tasks when large amounts of labeled data are available. One area where traditional deep learning approaches fail to perform is one-shot learning tasks where a model must correctly classify a new category after seeing only one example. One such domain is animal re-identification, an application of computer vision which can be used globally as a method to automate species population estimates from camera trap images. Our work demonstrates both the application of similarity comparison networks to animal re-identification, as well as the capabilities of deep convolutional neural networks to generalize across domains. Few studies have considered animal re-identification methods across species. Here, we compare two similarity comparison methodologies: Siamese and Triplet-Loss, based on the AlexNet, VGG-19, DenseNet201, MobileNetV2, and InceptionV3 architectures considering mean average precision (mAP)@1 and mAP@5. We consider five data sets corresponding to five different species: humans, chimpanzees, humpback whales, fruit flies, and Siberian tigers, each with their own unique set of challenges. We demonstrate that Triplet Loss outperformed its Siamese counterpart for all species. Without any species-specific modifications, our results demonstrate that similarity comparison networks can reach a performance level beyond that of humans for the task of animal re-identification. The ability for researchers to re-identify an animal individual upon re-encounter is fundamental for addressing a broad range of questions in the study of population dynamics and community/behavioural ecology. Our expectation is that similarity comparison networks are the beginning of a major trend that could stand to revolutionize animal re-identification from camera trap data.
Sparse Elasticity Reconstruction and Clustering using Local Displacement Fields
Authors: Megumi Nakao, Mitsuki Morita, Tetsuya Matsuda
Date: 2019-02-21 00:00:00
Tasks: Clustering

This paper introduces an elasticity reconstruction method based on local
displacement observations of elastic bodies. Sparse reconstruction theory is
applied to formulate the underdetermined inverse problems of elasticity
reconstruction including unobserved areas. An online local clustering scheme
called a superelement is proposed to reduce the number of dimensions of the
optimization parameters. Alternating the optimization of element boundaries and
elasticity parameters enables the elasticity distribution to be estimated with
a higher spatial resolution. The simulation experiments show that elasticity
distribution is reconstructed based on observations of approximately 10% of the
total body. The estimation error was improved when considering the sparseness
of the elasticity distribution.
Ising-Dropout: A Regularization Method for Training and Compression of Deep Neural Networks
Authors: Hojjat Salehinejad, Shahrokh Valaee
Date: 2019-02-07 00:00:00
Tasks: General Classification

Overfitting is a major problem in training machine learning models,
specifically deep neural networks. This problem may be caused by imbalanced
datasets and initialization of the model parameters, which conforms the model
too closely to the training data and negatively affects the generalization
performance of the model for unseen data. The original dropout is a
regularization technique to drop hidden units randomly during training. In this
paper, we propose an adaptive technique to wisely drop the visible and hidden
units in a deep neural network using Ising energy of the network. The
preliminary results show that the proposed approach can keep the classification
performance competitive to the original network while eliminating optimization
of unnecessary network parameters in each training cycle. The dropout state of
units can also be applied to the trained (inference) model. This technique
could compress the network in terms of number of parameters up to 41.18% and
55.86% for the classification task on the MNIST and Fashion-MNIST datasets,
respectively.
Optimizing Controller Placement for Software-Defined Networks
Authors: Victoria Huang, Gang Chen, Qiang Fu, Elliott Wen
Date: 2019-02-14 00:00:00

Controller placement problem (CPP) is a key issue for Software-Defined
Networking (SDN) with distributed controller architectures. This problem aims
to determine a suitable number of controllers deployed in important locations
so as to optimize the overall network performance. In comparison to
communication delay, existing literature on the CPP assumes that the influence
of controller workload distribution on network performance is negligible. In
this paper, we tackle the CPP that simultaneously considers the communication
delay, the control plane utilization, and the controller workload distribution.
Due to this reason, our CPP is intrinsically different from and clearly more
difficult than any previously studied CPPs that are NP-hard. To tackle this
challenging issue, we develop a new algorithm that seamlessly integrates the
genetic algorithm (GA) and the gradient descent (GD) optimization method.
Particularly, GA is used to search for suitable CPP solutions. The quality of
each solution is further evaluated through GD. Simulation results on two
representative network scenarios (small-scale and large-scale) show that our
algorithm can effectively strike the trade-off between the control plane
utilization and the network response time.
Formula of Volume of Revolution with Integration by Parts and Extension
Authors: Yi Liu, Jingwei Liu
Date: 2016-09-04 00:00:00

A calculation formula of volume of revolution with integration by parts of
definite integral is derived based on monotone function, and extended to a
general case that curved trapezoids is determined by continuous, piecewise
strictly monotone and differential function. And, two examples are given, ones
curvilinear trapezoids is determined by Kepler equation, and the other
curvilinear trapezoids is a function transmuted from Kepler equation.
SCFM: Social and crowdsourcing factorization machines for recommendation
Authors: Yue Ding a, Dong Wang b, Xin Xin b, Guoqiang Li b, ∗, Daniel Sun c, B, Xuezhi Zeng d, Rajiv Ranjan e
Date: 2017-10-14 00:00:00
Proceeding: journal 2017 10
Tasks: Recommendation Systems

With the rapid development of social networks, the exponential growth of social information has
attracted much attention. Social information has great value in recommender systems to alleviate the
sparsity and cold start problem. On the other hand, the crowd computing empowers recommender systems by utilizing human wisdom. Internal user reviews can be exploited as the wisdom of the crowd
to contribute information. In this paper, we propose social and crowdsourcing factorization machines,
called SCFM. Our approach fuses social and crowd computing into the factorization machine model. For
social computing, we calculate the influence value between users by taking users’ social information
and user similarity into account. For crowd computing, we apply LDA (Latent Dirichlet Allocation) on
people review to obtain sets of underlying topic probabilities. Furthermore, we impose two important
constraints called social regularization and domain inner regularization. The experimental results show
that our approach outperforms other state-of-the-art methods