pwc-archive/papers-with-abstracts huggingface dataset:
Spectral Reconstruction with Deep Neural Networks
Authors: Lukas Kades, Jan M. Pawlowski, Alexander Rothkopf, Manuel Scherzer, Julian M. Urban, Sebastian J. Wetzel, Nicolas Wink, Felix P. G. Ziegler
Date: 2019-05-10 00:00:00
Tasks: Bayesian Inference, Spectral Reconstruction

We explore artificial neural networks as a tool for the reconstruction of spectral functions from imaginary time Green's functions, a classic ill-conditioned inverse problem. Our ansatz is based on a supervised learning framework in which prior knowledge is encoded in the training data and the inverse transformation manifold is explicitly parametrised through a neural network. We systematically investigate this novel reconstruction approach, providing a detailed analysis of its performance on physically motivated mock data, and compare it to established methods of Bayesian inference. The reconstruction accuracy is found to be at least comparable, and potentially superior in particular at larger noise levels. We argue that the use of labelled training data in a supervised setting and the freedom in defining an optimisation objective are inherent advantages of the present approach and may lead to significant improvements over state-of-the-art methods in the future. Potential directions for further research are discussed in detail.
A joint text mining-rank size investigation of the rhetoric structures of the US Presidents' speeches
Authors: Valerio Ficcadenti, Roy Cerqueti, Marcel Ausloos
Date: 2019-05-09 00:00:00
Tasks: Decision Making

This work presents a text mining context and its use for a deep analysis of the messages delivered by the politicians. Specifically, we deal with an expert systems-based exploration of the rhetoric dynamics of a large collection of US Presidents' speeches, ranging from Washington to Trump. In particular, speeches are viewed as complex expert systems whose structures can be effectively analyzed through rank-size laws. The methodological contribution of the paper is twofold. First, we develop a text mining-based procedure for the construction of the dataset by using a web scraping routine on the Miller Center website -- the repository collecting the speeches. Second, we explore the implicit structure of the discourse data by implementing a rank-size procedure over the individual speeches, being the words of each speech ranked in terms of their frequencies. The scientific significance of the proposed combination of text-mining and rank-size approaches can be found in its flexibility and generality, which let it be reproducible to a wide set of expert systems and text mining contexts. The usefulness of the proposed method and the speech subsequent analysis is demonstrated by the findings themselves. Indeed, in terms of impact, it is worth noting that interesting conclusions of social, political and linguistic nature on how 45 United States Presidents, from April 30, 1789 till February 28, 2017 delivered political messages can be carried out. Indeed, the proposed analysis shows some remarkable regularities, not only inside a given speech, but also among different speeches. Moreover, under a purely methodological perspective, the presented contribution suggests possible ways of generating a linguistic decision-making algorithm.
Governance by Glass-Box: Implementing Transparent Moral Bounds for AI Behaviour
Authors: Andrea Aler Tubella, Andreas Theodorou, Virginia Dignum, Frank Dignum
Date: 2019-04-30 00:00:00

Artificial Intelligence (AI) applications are being used to predict and assess behaviour in multiple domains, such as criminal justice and consumer finance, which directly affect human well-being. However, if AI is to improve people's lives, then people must be able to trust AI, which means being able to understand what the system is doing and why. Even though transparency is often seen as the requirement in this case, realistically it might not always be possible or desirable, whereas the need to ensure that the system operates within set moral bounds remains. In this paper, we present an approach to evaluate the moral bounds of an AI system based on the monitoring of its inputs and outputs. We place a "glass box" around the system by mapping moral values into explicit verifiable norms that constrain inputs and outputs, in such a way that if these remain within the box we can guarantee that the system adheres to the value. The focus on inputs and outputs allows for the verification and comparison of vastly different intelligent systems; from deep neural networks to agent-based systems. The explicit transformation of abstract moral values into concrete norms brings great benefits in terms of explainability; stakeholders know exactly how the system is interpreting and employing relevant abstract moral human values and calibrate their trust accordingly. Moreover, by operating at a higher level we can check the compliance of the system with different interpretations of the same value. These advantages will have an impact on the well-being of AI systems users at large, building their trust and providing them with concrete knowledge on how systems adhere to moral values.
Tile Pattern KL-Divergence for Analysing and Evolving Game Levels
Authors: Simon M. Lucas, Vanessa Volz
Date: 2019-04-24 00:00:00

This paper provides a detailed investigation of using the Kullback-Leibler
(KL) Divergence as a way to compare and analyse game-levels, and hence to use
the measure as the objective function of an evolutionary algorithm to evolve
new levels. We describe the benefits of its asymmetry for level analysis and
demonstrate how (not surprisingly) the quality of the results depends on the
features used. Here we use tile-patterns of various sizes as features.
  When using the measure for evolution-based level generation, we demonstrate
that the choice of variation operator is critical in order to provide an
efficient search process, and introduce a novel convolutional mutation operator
to facilitate this. We compare the results with alternative generators,
including evolving in the latent space of generative adversarial networks, and
Wave Function Collapse. The results clearly show the proposed method to provide
competitive performance, providing reasonable quality results with very fast
training and reasonably fast generation.
Robust Metric Learning by Smooth Optimization
Authors: Kaizhu Huang, Rong Jin, Zenglin Xu, Cheng-Lin Liu
Date: 2012-03-15 00:00:00
Tasks: Articles, Combinatorial Optimization, Metric Learning, Triplet

Most existing distance metric learning methods assume perfect side
information that is usually given in pairwise or triplet constraints. Instead,
in many real-world applications, the constraints are derived from side
information, such as users' implicit feedbacks and citations among articles. As
a result, these constraints are usually noisy and contain many mistakes. In
this work, we aim to learn a distance metric from noisy constraints by robust
optimization in a worst-case scenario, to which we refer as robust metric
learning. We formulate the learning task initially as a combinatorial
optimization problem, and show that it can be elegantly transformed to a convex
programming problem. We present an efficient learning algorithm based on smooth
optimization [7]. It has a worst-case convergence rate of
O(1/{\surd}{\varepsilon}) for smooth optimization problems, where {\varepsilon}
is the desired error of the approximate solution. Finally, our empirical study
with UCI data sets demonstrate the effectiveness of the proposed method in
comparison to state-of-the-art methods.
Capturing Evolution Genes for Time Series Data
Authors: Wenjie Hu, Jianping Huang, Liang Wu, Yang Yang, Zongtao Liu, Zhanlin Sun, Bingshen Yao, Ke Chen
Date: 2019-05-10 00:00:00
Tasks: Time Series, Time Series Analysis

The modeling of time series is becoming increasingly critical in a wide variety of applications. Overall, data evolves by following different patterns, which are generally caused by different user behaviors. Given a time series, we define the evolution gene to capture the latent user behaviors and to describe how the behaviors lead to the generation of time series. In particular, we propose a uniform framework that recognizes different evolution genes of segments by learning a classifier, and adopt an adversarial generator to implement the evolution gene by estimating the segments' distribution. Experimental results based on a synthetic dataset and five real-world datasets show that our approach can not only achieve a good prediction results (e.g., averagely +10.56% in terms of F1), but is also able to provide explanations of the results.
Time-Series Event Prediction with Evolutionary State Graph
Authors: Wenjie Hu, Yang Yang, Ziqiang Cheng, Carl Yang, Xiang Ren
Date: 2019-05-10 00:00:00
Tasks: Graph Neural Network, Prediction, Time Series, Time Series Analysis, Time Series Classification, Time Series Prediction

The accurate and interpretable prediction of future events in time-series data often requires the capturing of representative patterns (or referred to as states) underpinning the observed data. To this end, most existing studies focus on the representation and recognition of states, but ignore the changing transitional relations among them. In this paper, we present evolutionary state graph, a dynamic graph structure designed to systematically represent the evolving relations (edges) among states (nodes) along time. We conduct analysis on the dynamic graphs constructed from the time-series data and show that changes on the graph structures (e.g., edges connecting certain state nodes) can inform the occurrences of events (i.e., time-series fluctuation). Inspired by this, we propose a novel graph neural network model, Evolutionary State Graph Network (EvoNet), to encode the evolutionary state graph for accurate and interpretable time-series event prediction. Specifically, Evolutionary State Graph Network models both the node-level (state-to-state) and graph-level (segment-to-segment) propagation, and captures the node-graph (state-to-segment) interactions over time. Experimental results based on five real-world datasets show that our approach not only achieves clear improvements compared with 11 baselines, but also provides more insights towards explaining the results of event predictions.
Semantic Segmentation of Seismic Images
Authors: Daniel Civitarese, Daniela Szwarcman, Emilio Vital Brazil, Bianca Zadrozny
Date: 2019-05-10 00:00:00
Tasks: Segmentation, Semantic Segmentation

Almost all work to understand Earth's subsurface on a large scale relies on the interpretation of seismic surveys by experts who segment the survey (usually a cube) into layers; a process that is very time demanding. In this paper, we present a new deep neural network architecture specially designed to semantically segment seismic images with a minimal amount of training data. To achieve this, we make use of a transposed residual unit that replaces the traditional dilated convolution for the decode block. Also, instead of using a predefined shape for up-scaling, our network learns all the steps to upscale the features from the encoder. We train our neural network using the Penobscot 3D dataset; a real seismic dataset acquired offshore Nova Scotia, Canada. We compare our approach with two well-known deep neural network topologies: Fully Convolutional Network and U-Net. In our experiments, we show that our approach can achieve more than 99 percent of the mean intersection over union (mIOU) metric, outperforming the existing topologies. Moreover, our qualitative results show that the obtained model can produce masks very close to human interpretation with very little discontinuity.
Fast and Robust Distributed Learning in High Dimension
Authors: El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
Date: 2019-05-05 00:00:00
Tasks: BIG-bench Machine Learning, Vocal Bursts Intensity Prediction

Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through multi-Bulyan. Given $n$ workers, $f$ of which are arbitrary malicious (Byzantine) and $m=n-f$ are not, we prove that multi-Bulyan can ensure a strong form of Byzantine resilience, as well as an ${\frac{m}{n}}$ slowdown, compared to averaging, the fastest (but non Byzantine resilient) rule for distributed machine learning. When $m \approx n$ (almost all workers are correct), multi-Bulyan reaches the speed of averaging. We also prove that multi-Bulyan's cost in local computation is $O(d)$ (like averaging), an important feature for ML where $d$ commonly reaches $10^9$, while robust alternatives have at least quadratic cost in $d$. Our theoretical findings are complemented with an experimental evaluation which, in addition to supporting the linear $O(d)$ complexity argument, conveys the fact that multi-Bulyan's parallelisability further adds to its efficiency.
Differential Recurrent Neural Network and its Application for Human Activity Recognition
Authors: Naifan Zhuang, Guo-Jun Qi, The Duc Kieu, Kien A. Hua
Date: 2019-05-09 00:00:00
Tasks: Activity Recognition, Human Activity Recognition, Time Series, Time Series Analysis

The Long Short-Term Memory (LSTM) recurrent neural network is capable of processing complex sequential information since it utilizes special gating schemes for learning representations from long input sequences. It has the potential to model any sequential time-series data, where the current hidden state has to be considered in the context of the past hidden states. This property makes LSTM an ideal choice to learn the complex dynamics present in long sequences. Unfortunately, the conventional LSTMs do not consider the impact of spatio-temporal dynamics corresponding to the given salient motion patterns, when they gate the information that ought to be memorized through time. To address this problem, we propose a differential gating scheme for the LSTM neural network, which emphasizes on the change in information gain caused by the salient motions between the successive video frames. This change in information gain is quantified by Derivative of States (DoS), and thus the proposed LSTM model is termed as differential Recurrent Neural Network (dRNN). In addition, the original work used the hidden state at the last time-step to model the entire video sequence. Based on the energy profiling of DoS, we further propose to employ the State Energy Profile (SEP) to search for salient dRNN states and construct more informative representations. The effectiveness of the proposed model was demonstrated by automatically recognizing human actions from the real-world 2D and 3D single-person action datasets. We point out that LSTM is a special form of dRNN. As a result, we have introduced a new family of LSTMs. Our study is one of the first works towards demonstrating the potential of learning complex time-series representations via high-order derivatives of states.
Analysis of Probabilistic multi-scale fractional order fusion-based de-hazing algorithm
Authors: U. A. Nnolim
Date: 2019-05-10 00:00:00

In this report, a de-hazing algorithm based on probability and multi-scale fractional order-based fusion is proposed. The proposed scheme improves on a previously implemented multiscale fraction order-based fusion by augmenting its local contrast and edge sharpening features. It also brightens de-hazed images, while avoiding sky region over-enhancement. The results of the proposed algorithm are analyzed and compared with existing methods from the literature and indicate better performance in most cases.
An in-network data cleaning approach for wireless sensor networks
Authors: Jianjun Leia, Haiyang Bia, Ying Xiaa, Jun Huanga and Haeyoung Baeb
Date: 2016-03-17 00:00:00
Proceeding: journal 2016 3
Tasks: Clustering Algorithms Evaluation

Wireless Sensor Networks (WSNs) are widely used for monitoring physical happenings of the
environment. However, the data gathered by the WSNs may be inaccurate and unreliable due to power
exhaustion, noise and other reasons. Unnecessary data such as erroneous data and redundant data
transmission causes a lot of extra energy consumption. To improve the data reliability and reduce the
energy consumption, we proposed an in-network processing architecture for data cleaning, which
divides the task into four stages implemented in different nodes respectively. This strategy guaranteed
the cleaning algorithms were computationally lightweight in local nodes and energy-efficient due to
almost no communication overhead. In addition, we presented the detection algorithms for data faults
and event outliers, which were conducted by utilizing the related attributes from the local sensor node
and the cooperation with its relaying neighbor. Experiment results show that our proposed approach
is accurate and energy-efficient.
MinScIE: Citation-centered Open Information Extraction
Authors: Anne Lauscher, Yide Song, Kiril Gashteovski
Date: 2019-06-01 00:00:00
Proceeding: Joint Conference on Digital Libraries (JCDL) 2019 6
Tasks: Open Information Extraction

Acknowledging the importance of citations in scientific literature,
in this work we present MinScIE, an Open Information Extraction
system which provides structured knowledge enriched with semantic information about citations. By comparing our system to it’s
original core, MinIE, we show that our approach improves extraction precision by 3 percentage points.
Automating string processing in spreadsheets using input-output examples
Authors: Sumit Gulwani
Date: 2011-01-28 00:00:00
Proceeding: POPL 2011 2011 1
Tasks: Novel Concepts

We describe the design of a string programming/expression lan- guage that supports restricted forms of regular expressions, condi- tionals and loops. The language is expressive enough to represent a wide variety of string manipulation tasks that end-users struggle with. We describe an algorithm based on several novel concepts for synthesizing a desired program in this language from input-output examples. The synthesis algorithm is very efficient taking a fraction of a second for various benchmark examples. The synthesis algo- rithm is interactive and has several desirable features: it can rank multiple solutions and has fast convergence, it can detect noise in the user input, and it supports an active interaction model wherein the user is prompted to provide outputs on inputs that may have multiple computational interpretations.
The algorithm has been implemented as an interactive add-in for Microsoft Excel spreadsheet system. The prototype tool has met the golden test - it has synthesized part of itself, and has been used to solve problems beyond author’s imagination.
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
Authors: Daniel Ho, Eric Liang, Ion Stoica, Pieter Abbeel, Xi Chen
Date: 2019-05-14 00:00:00
Tasks: Data Augmentation, Image Augmentation

A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment are computationally infeasible to run for the ordinary user. In this paper, we introduce a new data augmentation algorithm, Population Based Augmentation (PBA), which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy. We show that PBA can match the performance of AutoAugment on CIFAR-10, CIFAR-100, and SVHN, with three orders of magnitude less overall compute. On CIFAR-10 we achieve a mean test error of 1.46%, which is a slight improvement upon the current state-of-the-art. The code for PBA is open source and is available at https://github.com/arcelien/pba.
Efficient Ladder-style DenseNets for Semantic Segmentation of Large Images
Authors: Ivan Krešo, Josip Krapac, Siniša Šegvić
Date: 2019-05-14 00:00:00
Tasks: image-classification, Image Classification, Semantic Segmentation

Recent progress of deep image classification models has provided great potential to improve state-of-the-art performance in related computer vision tasks. However, the transition to semantic segmentation is hampered by strict memory limitations of contemporary GPUs. The extent of feature map caching required by convolutional backprop poses significant challenges even for moderately sized Pascal images, while requiring careful architectural considerations when the source resolution is in the megapixel range. To address these concerns, we propose a novel DenseNet-based ladder-style architecture which features high modelling power and a very lean upsampling datapath. We also propose to substantially reduce the extent of feature map caching by exploiting inherent spatial efficiency of the DenseNet feature extractor. The resulting models deliver high performance with fewer parameters than competitive approaches, and allow training at megapixel resolution on commodity hardware. The presented experimental results outperform the state-of-the-art in terms of prediction accuracy and execution speed on Cityscapes, Pascal VOC 2012, CamVid and ROB 2018 datasets. Source code will be released upon publication.
Convolutional neural networks with fractional order gradient method
Authors: Dian Sheng, Yiheng Wei, Yuquan Chen, Yong Wang
Date: 2019-05-14 00:00:00

This paper proposes a fractional order gradient method for the backward propagation of convolutional neural networks. To overcome the problem that fractional order gradient method cannot converge to real extreme point, a simplified fractional order gradient method is designed based on Caputo's definition. The parameters within layers are updated by the designed gradient method, but the propagations between layers still use integer order gradients, and thus the complicated derivatives of composite functions are avoided and the chain rule will be kept. By connecting every layers in series and adding loss functions, the proposed convolutional neural networks can be trained smoothly according to various tasks. Some practical experiments are carried out in order to demonstrate fast convergence, high accuracy and ability to escape local optimal point at last.
Learnable Triangulation of Human Pose
Authors: Karim Iskakov, Egor Burkov, Victor Lempitsky, Yury Malkov
Date: 2019-05-14 00:00:00
Proceeding: ICCV 2019 10
Tasks: 3D Human Pose Estimation, Multi-view 3D Human Pose Estimation, Pose Estimation

We present two novel solutions for multi-view 3D human pose estimation based on new learnable triangulation methods that combine 3D information from multiple 2D views. The first (baseline) solution is a basic differentiable algebraic triangulation with an addition of confidence weights estimated from the input images. The second solution is based on a novel method of volumetric aggregation from intermediate 2D backbone feature maps. The aggregated volume is then refined via 3D convolutions that produce final 3D joint heatmaps and allow modelling a human pose prior. Crucially, both approaches are end-to-end differentiable, which allows us to directly optimize the target metric. We demonstrate transferability of the solutions across datasets and considerably improve the multi-view state of the art on the Human3.6M dataset. Video demonstration, annotations and additional materials will be posted on our project page (https://saic-violet.github.io/learnable-triangulation).