pwc-archive/papers-with-abstracts huggingface dataset:
Constructing Parsimonious Analytic Models for Dynamic Systems via Symbolic Regression
Authors: Erik Derner, Jiří Kubalík, Nicola Ancona, Robert Babuška
Date: 2019-03-27 00:00:00
Tasks: Decision Making, regression, Reinforcement Learning, Reinforcement Learning (RL), Symbolic Regression

Developing mathematical models of dynamic systems is central to many disciplines of engineering and science. Models facilitate simulations, analysis of the system's behavior, decision making and design of automatic control algorithms. Even inherently model-free control techniques such as reinforcement learning (RL) have been shown to benefit from the use of models, typically learned online. Any model construction method must address the tradeoff between the accuracy of the model and its complexity, which is difficult to strike. In this paper, we propose to employ symbolic regression (SR) to construct parsimonious process models described by analytic equations. We have equipped our method with two different state-of-the-art SR algorithms which automatically search for equations that fit the measured data: Single Node Genetic Programming (SNGP) and Multi-Gene Genetic Programming (MGGP). In addition to the standard problem formulation in the state-space domain, we show how the method can also be applied to input-output models of the NARX (nonlinear autoregressive with exogenous input) type. We present the approach on three simulated examples with up to 14-dimensional state space: an inverted pendulum, a mobile robot, and a bipedal walking robot. A comparison with deep neural networks and local linear regression shows that SR in most cases outperforms these commonly used alternative methods. We demonstrate on a real pendulum system that the analytic model found enables a RL controller to successfully perform the swing-up task, based on a model constructed from only 100 data samples.
A Sober Look at Neural Network Initializations
Authors: Ingo Steinwart
Date: 2019-03-27 00:00:00

Initializing the weights and the biases is a key part of the training process of a neural network. Unlike the subsequent optimization phase, however, the initialization phase has gained only limited attention in the literature. In this paper we discuss some consequences of commonly used initialization strategies for vanilla DNNs with ReLU activations. Based on these insights we then develop an alternative initialization strategy. Finally, we present some large scale experiments assessing the quality of the new initialization strategy.
Tracking the Consumption Junction: Temporal Dependencies between Articles and Advertisements in Dutch Newspapers
Authors: Melvin Wevers, Jianbo Gao, Kristoffer L. Nielbo
Date: 2019-03-27 00:00:00
Tasks: Articles, Econometrics

Historians have regularly debated whether advertisements can be used as a
viable source to study the past. Their main concern centered on the question of
agency. Were advertisements a reflection of historical events and societal
debates, or were ad makers instrumental in shaping society and the ways people
interacted with consumer goods? Using techniques from econometrics (Granger
causality test) and complexity science (Adaptive Fractal Analysis), this paper
analyzes to what extent advertisements shaped or reflected society. We found
evidence that indicate a fundamental difference between the dynamic behavior of
word use in articles and advertisements published in a century of Dutch
newspapers. Articles exhibit persistent trends that are likely to be reflective
of communicative memory. Contrary to this, advertisements have a more irregular
behavior characterized by short bursts and fast decay, which, in part, mirrors
the dynamic through which advertisers introduced terms into public discourse.
On the issue of whether advertisements shaped or reflected society, we found
particular product types that seemed to be collectively driven by a causality
going from advertisements to articles. Generally, we found support for a
complex interaction pattern dubbed the consumption junction. Finally, we
discovered noteworthy patterns in terms of causality and long-range
dependencies for specific product groups. All in, this study shows how methods
from econometrics and complexity science can be applied to humanities data to
improve our understanding of complex cultural-historical phenomena such as the
role of advertising in society.
A sparse semismooth Newton based proximal majorization-minimization algorithm for nonconvex square-root-loss regression problems
Authors: Peipei Tang, Chengjing Wang, Defeng Sun, Kim-Chuan Toh
Date: 2019-03-27 00:00:00
Tasks: regression

In this paper, we consider high-dimensional nonconvex square-root-loss regression problems and introduce a proximal majorization-minimization (PMM) algorithm for these problems. Our key idea for making the proposed PMM to be efficient is to develop a sparse semismooth Newton method to solve the corresponding subproblems. By using the Kurdyka-{\L}ojasiewicz property exhibited in the underlining problems, we prove that the PMM algorithm converges to a d-stationary point. We also analyze the oracle property of the initial subproblem used in our algorithm. Extensive numerical experiments are presented to demonstrate the high efficiency of the proposed PMM algorithm.
Sensing Social Media Signals for Cryptocurrency News
Authors: Johannes Beck, Roberta Huang, David Lindner, Tian Guo, Ce Zhang, Dirk Helbing, Nino Antulov-Fantulin
Date: 2019-03-27 00:00:00
Tasks: BIG-bench Machine Learning

The ability to track and monitor relevant and important news in real-time is
of crucial interest in multiple industrial sectors. In this work, we focus on
the set of cryptocurrency news, which recently became of emerging interest to
the general and financial audience. In order to track relevant news in
real-time, we (i) match news from the web with tweets from social media, (ii)
track their intraday tweet activity and (iii) explore different machine
learning models for predicting the number of the article mentions on Twitter
within the first 24 hours after its publication. We compare several machine
learning models, such as linear extrapolation, linear and random forest
autoregressive models, and a sequence-to-sequence neural network. We find that
the random forest autoregressive model behaves comparably to more complex
models in the majority of tasks.
Multi-Species Cuckoo Search Algorithm for Global Optimization
Authors: Xin-She Yang, Suash Deb, Sudhanshu K Mishra
Date: 2019-03-27 00:00:00
Tasks: global-optimization

Many optimization problems in science and engineering are highly nonlinear,
and thus require sophisticated optimization techniques to solve. Traditional
techniques such as gradient-based algorithms are mostly local search methods,
and often struggle to cope with such challenging optimization problems. Recent
trends tend to use nature-inspired optimization algorithms. This work extends
the standard cuckoo search (CS) by using the successful features of the
cuckoo-host co-evolution with multiple interacting species, and the proposed
multi-species cuckoo search (MSCS) intends to mimic the multiple species of
cuckoos that compete for the survival of the fittest, and they co-evolve with
host species with solution vectors being encoded as position vectors. The
proposed algorithm is then validated by 15 benchmark functions as well as five
nonlinear, multimodal design case studies in practical applications. Simulation
results suggest that the proposed algorithm can be effective for finding
optimal solutions and in this case all optimal solutions are achievable. The
results for the test benchmarks are also compared with those obtained by other
methods such as the standard cuckoo search and genetic algorithm, which
demonstrated the efficiency of the present algorithm. Based on numerical
experiments and case studies, we can conclude that the proposed algorithm can
be more efficient in most cases, leading a potentially very effective tool for
solving nonlinear optimization problems.
Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving
Authors: Xinzhu Ma, Zhihui Wang, Haojie Li, Peng-Bo Zhang, Xin Fan, Wanli Ouyang
Date: 2019-03-27 00:00:00
Tasks: 3D Object Detection, 3D Reconstruction, Autonomous Driving, Monocular 3D Object Detection, object-detection, Object Detection

In this paper, we propose a monocular 3D object detection framework in the domain of autonomous driving. Unlike previous image-based methods which focus on RGB feature extracted from 2D images, our method solves this problem in the reconstructed 3D space in order to exploit 3D contexts explicitly. To this end, we first leverage a stand-alone module to transform the input data from 2D image plane to 3D point clouds space for a better input representation, then we perform the 3D detection using PointNet backbone net to obtain objects 3D locations, dimensions and orientations. To enhance the discriminative capability of point clouds, we propose a multi-modal feature fusion module to embed the complementary RGB cue into the generated point clouds representation. We argue that it is more effective to infer the 3D bounding boxes from the generated 3D scene space (i.e., X,Y, Z space) compared to the image plane (i.e., R,G,B image plane). Evaluation on the challenging KITTI dataset shows that our approach boosts the performance of state-of-the-art monocular approach by a large margin.
Using Monolingual Data in Neural Machine Translation: a Systematic Study
Authors: Franck Burlot, François Yvon
Date: 2019-03-27 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Translation

Neural Machine Translation (MT) has radically changed the way systems are
developed. A major difference with the previous generation (Phrase-Based MT) is
the way monolingual target data, which often abounds, is used in these two
paradigms. While Phrase-Based MT can seamlessly integrate very large language
models trained on billions of sentences, the best option for Neural MT
developers seems to be the generation of artificial parallel data through
\textsl{back-translation} - a technique that fails to fully take advantage of
existing datasets. In this paper, we conduct a systematic study of
back-translation, comparing alternative uses of monolingual data, as well as
multiple data generation procedures. Our findings confirm that back-translation
is very effective and give new explanations as to why this is the case. We also
introduce new data simulation techniques that are almost as effective, yet much
cheaper to implement.
Real-time data-driven detection of the rock type alteration during a directional drilling
Authors: Evgenya Romanenkova, Alexey Zaytsev, Nikita Klyuchnikov, Arseniy Gruzdev, Ksenia Antipova, Leyla Ismailova, Evgeny Burnaev, Artyom Semenikhin, Vitaliy Koryabkin, Igor Simon, Dmitry Koroteev
Date: 2019-03-27 00:00:00
Tasks: Change Detection, Vocal Bursts Type Prediction

During the directional drilling, a bit may sometimes go to a nonproductive rock layer due to the gap about 20m between the bit and high-fidelity rock type sensors. The only way to detect the lithotype changes in time is the usage of Measurements While Drilling (MWD) data. However, there are no general mathematical modeling approaches that both well reconstruct the rock type based on MWD data and correspond to specifics of the oil and gas industry. In this article, we present a data-driven procedure that utilizes MWD data for quick detection of changes in rock type. We propose the approach that combines traditional machine learning based on the solution of the rock type classification problem with change detection procedures rarely used before in the Oil\&Gas industry. The data come from a newly developed oilfield in the north of western Siberia. The results suggest that we can detect a significant part of changes in rock type reducing the change detection delay from $20$ to $1.8$ meters and the number of false-positive alarms from $43$ to $6$ per well.
Social Behavioral Phenotyping of Drosophila with a2D-3D Hybrid CNN Framework
Authors: Ziping Jiang, Paul L. Chazot, M. Emre Celebi, Danny Crookes, Richard Jiang
Date: 2019-03-27 00:00:00

Behavioural phenotyping of Drosophila is an important means in biological and
medical research to identify genetic, pathologic or psychologic impact on
animal behaviour.
Do Not Trust Additive Explanations
Authors: Alicja Gosiewska, Przemyslaw Biecek
Date: 2019-03-27 00:00:00
Tasks: Additive models, BIG-bench Machine Learning, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI)

Explainable Artificial Intelligence (XAI)has received a great deal of attention recently. Explainability is being presented as a remedy for the distrust of complex and opaque models. Model agnostic methods such as LIME, SHAP, or Break Down promise instance-level interpretability for any complex machine learning model. But how faithful are these additive explanations? Can we rely on additive explanations for non-additive models? In this paper, we (1) examine the behavior of the most popular instance-level explanations under the presence of interactions, (2) introduce a new method that detects interactions for instance-level explanations, (3) perform a large scale benchmark to see how frequently additive explanations may be misleading.
Self-Supervised Learning via Conditional Motion Propagation
Authors: Xiaohang Zhan, Xingang Pan, Ziwei Liu, Dahua Lin, Chen Change Loy
Date: 2019-03-27 00:00:00
Proceeding: CVPR 2019 6
Tasks: Human Parsing, Instance Segmentation, Self-Supervised Learning, Semantic Segmentation

Intelligent agent naturally learns from motion. Various self-supervised
algorithms have leveraged motion cues to learn effective visual
representations. The hurdle here is that motion is both ambiguous and complex,
rendering previous works either suffer from degraded learning efficacy, or
resort to strong assumptions on object motions. In this work, we design a new
learning-from-motion paradigm to bridge these gaps. Instead of explicitly
modeling the motion probabilities, we design the pretext task as a conditional
motion propagation problem. Given an input image and several sparse flow
guidance vectors on it, our framework seeks to recover the full-image motion.
Compared to other alternatives, our framework has several appealing properties:
(1) Using sparse flow guidance during training resolves the inherent motion
ambiguity, and thus easing feature learning. (2) Solving the pretext task of
conditional motion propagation encourages the emergence of kinematically-sound
representations that poss greater expressive power. Extensive experiments
demonstrate that our framework learns structural and coherent features; and
achieves state-of-the-art self-supervision performance on several downstream
tasks including semantic segmentation, instance segmentation, and human
parsing. Furthermore, our framework is successfully extended to several useful
applications such as semi-automatic pixel-level annotation. Project page:
"http://mmlab.ie.cuhk.edu.hk/projects/CMP/".
Structural Neural Encoders for AMR-to-text Generation
Authors: Marco Damonte, Shay B. Cohen
Date: 2019-03-27 00:00:00
Proceeding: NAACL 2019 6
Tasks: Abstract Meaning Representation, AMR-to-Text Generation, Graph-to-Sequence, Text Generation

AMR-to-text generation is a problem recently introduced to the NLP community, in which the goal is to generate sentences from Abstract Meaning Representation (AMR) graphs. Sequence-to-sequence models can be used to this end by converting the AMR graphs to strings. Approaching the problem while working directly with graphs requires the use of graph-to-sequence models that encode the AMR graph into a vector representation. Such encoding has been shown to be beneficial in the past, and unlike sequential encoding, it allows us to explicitly capture reentrant structures in the AMR graphs. We investigate the extent to which reentrancies (nodes with multiple parents) have an impact on AMR-to-text generation by comparing graph encoders to tree encoders, where reentrancies are not preserved. We show that improvements in the treatment of reentrancies and long-range dependencies contribute to higher overall scores for graph encoders. Our best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of the art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of the art by 1.24 points.
Deep Learning based Pedestrian Detection at Distance in Smart Cities
Authors: Ranjith K Dinakaran, Philip Easom, Ahmed Bouridane, Li Zhang, Richard Jiang, Fozia Mehboob, Abdul Rauf
Date: 2018-11-18 00:00:00
Tasks: Decoder, Deep Learning, Pedestrian Detection

Generative adversarial networks (GANs) have been promising for many computer vision problems due to their powerful capabilities to enhance the data for training and test. In this paper, we leveraged GANs and proposed a new architecture with a cascaded Single Shot Detector (SSD) for pedestrian detection at distance, which is yet a challenge due to the varied sizes of pedestrians in videos at distance. To overcome the low-resolution issues in pedestrian detection at distance, DCGAN is employed to improve the resolution first to reconstruct more discriminative features for a SSD to detect objects in images or videos. A crucial advantage of our method is that it learns a multi-scale metric to distinguish multiple objects at different distances under one image, while DCGAN serves as an encoder-decoder platform to generate parts of an image that contain better discriminative information. To measure the effectiveness of our proposed method, experiments were carried out on the Canadian Institute for Advanced Research (CIFAR) dataset, and it was demonstrated that the proposed new architecture achieved a much better detection rate, particularly on vehicles and pedestrians at distance, making it highly suitable for smart cities applications that need to discover key objects or pedestrians at distance.
Analyzing Knowledge Graph Embedding Methods from a Multi-Embedding Interaction Perspective
Authors: Hung Nghiep Tran, Atsuhiro Takasu
Date: 2019-03-27 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction, Question Answering, Recommendation Systems, World Knowledge

Knowledge graph is a popular format for representing knowledge, with many applications to semantic search engines, question-answering systems, and recommender systems. Real-world knowledge graphs are usually incomplete, so knowledge graph embedding methods, such as Canonical decomposition/Parallel factorization (CP), DistMult, and ComplEx, have been proposed to address this issue. These methods represent entities and relations as embedding vectors in semantic space and predict the links between them. The embedding vectors themselves contain rich semantic information and can be used in other applications such as data analysis. However, mechanisms in these models and the embedding vectors themselves vary greatly, making it difficult to understand and compare them. Given this lack of understanding, we risk using them ineffectively or incorrectly, particularly for complicated models, such as CP, with two role-based embedding vectors, or the state-of-the-art ComplEx model, with complex-valued embedding vectors. In this paper, we propose a multi-embedding interaction mechanism as a new approach to uniting and generalizing these models. We derive them theoretically via this mechanism and provide empirical analyses and comparisons between them. We also propose a new multi-embedding model based on quaternion algebra and show that it achieves promising results using popular benchmarks. Source code is available on GitHub at https://github.com/tranhungnghiep/AnalyzeKGE.
Learning semantic sentence representations from visually grounded language without lexical knowledge
Authors: Danny Merkx, Stefan Frank
Date: 2019-03-27 00:00:00
Tasks: Grounded language learning, Learning Semantic Representations, Retrieval, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Embeddings, Visual Grounding, Word Embeddings

Current approaches to learning semantic representations of sentences often
use prior word-level knowledge. The current study aims to leverage visual
information in order to capture sentence level semantics without the need for
word embeddings. We use a multimodal sentence encoder trained on a corpus of
images with matching text captions to produce visually grounded sentence
embeddings. Deep Neural Networks are trained to map the two modalities to a
common embedding space such that for an image the corresponding caption can be
retrieved and vice versa. We show that our model achieves results comparable to
the current state-of-the-art on two popular image-caption retrieval benchmark
data sets: MSCOCO and Flickr8k. We evaluate the semantic content of the
resulting sentence embeddings using the data from the Semantic Textual
Similarity benchmark task and show that the multimodal embeddings correlate
well with human semantic similarity judgements. The system achieves
state-of-the-art results on several of these benchmarks, which shows that a
system trained solely on multimodal data, without assuming any word
representations, is able to capture sentence level semantics. Importantly, this
result shows that we do not need prior knowledge of lexical level semantics in
order to model sentence level semantics. These findings demonstrate the
importance of visual information in semantics.
Dynamic Control of Stochastic Evolution: A Deep Reinforcement Learning Approach to Adaptively Targeting Emergent Drug Resistance
Authors: Dalit Engelhardt
Date: 2019-03-27 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

The challenge in controlling stochastic systems in which low-probability events can set the system on catastrophic trajectories is to develop a robust ability to respond to such events without significantly compromising the optimality of the baseline control policy. This paper presents CelluDose, a stochastic simulation-trained deep reinforcement learning adaptive feedback control prototype for automated precision drug dosing targeting stochastic and heterogeneous cell proliferation. Drug resistance can emerge from random and variable mutations in targeted cell populations; in the absence of an appropriate dosing policy, emergent resistant subpopulations can proliferate and lead to treatment failure. Dynamic feedback dosage control holds promise in combatting this phenomenon, but the application of traditional control approaches to such systems is fraught with challenges due to the complexity of cell dynamics, uncertainty in model parameters, and the need in medical applications for a robust controller that can be trusted to properly handle unexpected outcomes. Here, training on a sample biological scenario identified single-drug and combination therapy policies that exhibit a 100% success rate at suppressing cell proliferation and responding to diverse system perturbations while establishing low-dose no-event baselines. These policies were found to be highly robust to variations in a key model parameter subject to significant uncertainty and unpredictable dynamical changes.