pwc-archive/papers-with-abstracts huggingface dataset:
Methods of Adaptive Signal Processing on Graphs Using Vertex-Time Autoregressive Models
Authors: Thiernithi Variddhisai, Danilo Mandic
Date: 2020-03-10 00:00:00

The concept of a random process has been recently extended to graph signals, whereby random graph processes are a class of multivariate stochastic processes whose coefficients are matrices with a \textit{graph-topological} structure. The system identification problem of a random graph process therefore revolves around determining its underlying topology, or mathematically, the graph shift operators (GSOs) i.e. an adjacency matrix or a Laplacian matrix. In the same work that introduced random graph processes, a \textit{batch} optimization method to solve for the GSO was also proposed for the random graph process based on a \textit{causal} vertex-time autoregressive model. To this end, the online version of this optimization problem was proposed via the framework of adaptive filtering. The modified stochastic gradient projection method was employed on the regularized least squares objective to create the filter. The recursion is divided into 3 regularized sub-problems to address issues like multi-convexity, sparsity, commutativity and bias. A discussion on convergence analysis is also included. Finally, experiments are conducted to illustrate the performance of the proposed algorithm, from traditional MSE measure to successful recovery rate regardless correct values, all of which to shed light on the potential, the limit and the possible research attempt of this work.
KGvec2go -- Knowledge Graph Embeddings as a Service
Authors: Jan Portisch, Michael Hladik, Heiko Paulheim
Date: 2020-03-09 00:00:00
Proceeding: LREC 2020 5
Tasks: Knowledge Graph Embeddings, Knowledge Graphs

In this paper, we present KGvec2go, a Web API for accessing and consuming graph embeddings in a light-weight fashion in downstream applications. Currently, we serve pre-trained embeddings for four knowledge graphs. We introduce the service and its usage, and we show further that the trained models have semantic value by evaluating them on multiple semantic benchmarks. The evaluation also reveals that the combination of multiple models can lead to a better outcome than the best individual model.
IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control
Authors: François-Xavier Devailly, Denis Larocque, Laurent Charlin
Date: 2020-03-06 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Traffic Signal Control

Scaling adaptive traffic-signal control involves dealing with combinatorial state and action spaces. Multi-agent reinforcement learning attempts to address this challenge by distributing control to specialized agents. However, specialization hinders generalization and transferability, and the computational graphs underlying neural-networks architectures -- dominating in the multi-agent setting -- do not offer the flexibility to handle an arbitrary number of entities which changes both between road networks, and over time as vehicles traverse the network. We introduce Inductive Graph Reinforcement Learning (IG-RL) based on graph-convolutional networks which adapts to the structure of any road network, to learn detailed representations of traffic-controllers and their surroundings. Our decentralized approach enables learning of a transferable-adaptive-traffic-signal-control policy. After being trained on an arbitrary set of road networks, our model can generalize to new road networks, traffic distributions, and traffic regimes, with no additional training and a constant number of parameters, enabling greater scalability compared to prior methods. Furthermore, our approach can exploit the granularity of available data by capturing the (dynamic) demand at both the lane and the vehicle levels. The proposed method is tested on both road networks and traffic settings never experienced during training. We compare IG-RL to multi-agent reinforcement learning and domain-specific baselines. In both synthetic road networks and in a larger experiment involving the control of the 3,971 traffic signals of Manhattan, we show that different instantiations of IG-RL outperform baselines.
Explaining Away Attacks Against Neural Networks
Authors: Sean Saito, Jin Wang
Date: 2020-03-06 00:00:00

We investigate the problem of identifying adversarial attacks on image-based neural networks. We present intriguing experimental results showing significant discrepancies between the explanations generated for the predictions of a model on clean and adversarial data. Utilizing this intuition, we propose a framework which can identify whether a given input is adversarial based on the explanations given by the model. Code for our experiments can be found here: https://github.com/seansaito/Explaining-Away-Attacks-Against-Neural-Networks.
Towards Practical Lottery Ticket Hypothesis for Adversarial Training
Authors: Bai Li, Shiqi Wang, Yunhan Jia, Yantao Lu, Zhenyu Zhong, Lawrence Carin, Suman Jana
Date: 2020-03-06 00:00:00

Recent research has proposed the lottery ticket hypothesis, suggesting that for a deep neural network, there exist trainable sub-networks performing equally or better than the original model with commensurate training steps. While this discovery is insightful, finding proper sub-networks requires iterative training and pruning. The high cost incurred limits the applications of the lottery ticket hypothesis. We show there exists a subset of the aforementioned sub-networks that converge significantly faster during the training process and thus can mitigate the cost issue. We conduct extensive experiments to show such sub-networks consistently exist across various model structures for a restrictive setting of hyperparameters ($e.g.$, carefully selected learning rate, pruning ratio, and model capacity). As a practical application of our findings, we demonstrate that such sub-networks can help in cutting down the total time of adversarial training, a standard approach to improve robustness, by up to 49\% on CIFAR-10 to achieve the state-of-the-art robustness.
Technical report: Training Mixture Density Networks with full covariance matrices
Authors: Jakob Kruse
Date: 2020-03-04 00:00:00

Mixture Density Networks are a tried and tested tool for modelling conditional probability distributions. As such, they constitute a great baseline for novel approaches to this problem. In the standard formulation, an MDN takes some input and outputs parameters for a Gaussian mixture model with restrictions on the mixture components' covariance. Since covariance between random variables is a central issue in the conditional modeling problems we were investigating, I derived and implemented an MDN formulation with unrestricted covariances. It is likely that this has been done before, but I could not find any resources online. For this reason, I have documented my approach in the form of this technical report, in hopes that it may be useful to others facing a similar situation.
Investigating the influence Brexit had on Financial Markets, in particular the GBP/EUR exchange rate
Authors: Michael Filletti
Date: 2020-03-03 00:00:00
Tasks: Articles

On 23rd June 2016, 51.9% of British voters voted to leave the European Union, triggering a process and events that have led to the United Kingdom leaving the EU, an event that has become known as 'Brexit'. In this piece of research, we investigate the effects of this entire process on the currency markets, specifically the GBP/EUR exchange rate. Financial markets are known to be sensitive to news articles and media, and the aim of this research is to evaluate the magnitude of impact of relevant events, as well as whether the impact was positive or negative for the GBP.
Consistency Analysis of Replication-Based Probabilistic Key-Value Stores
Authors: Ramy E. Ali
Date: 2020-02-14 00:00:00

Partial quorum systems are widely used in distributed key-value stores due to their latency benefits at the expense of providing weaker consistency guarantees. The probabilistically bounded staleness framework (PBS) studied the latency-consistency trade-off of Dynamo-style partial quorum systems through Monte Carlo event-based simulations. In this paper, we study the latency-consistency trade-off for such systems analytically and derive a closed-form expression for the inconsistency probability. Our approach allows fine-tuning of latency and consistency guarantees in key-value stores, which is intractable using Monte Carlo event-based simulations.
Fast local linear regression with anchor regularization
Authors: Mathis Petrovich, Makoto Yamada
Date: 2020-02-21 00:00:00
Tasks: regression

Regression is an important task in machine learning and data mining. It has several applications in various domains, including finance, biomedical, and computer vision. Recently, network Lasso, which estimates local models by making clusters using the network information, was proposed and its superior performance was demonstrated. In this study, we propose a simple yet effective local model training algorithm called the fast anchor regularized local linear method (FALL). More specifically, we train a local model for each sample by regularizing it with precomputed anchor models. The key advantage of the proposed algorithm is that we can obtain a closed-form solution with only matrix multiplication; additionally, the proposed algorithm is easily interpretable, fast to compute and parallelizable. Through experiments on synthetic and real-world datasets, we demonstrate that FALL compares favorably in terms of accuracy with the state-of-the-art network Lasso algorithm with significantly smaller training time (two orders of magnitude).
A$^3$: Accelerating Attention Mechanisms in Neural Networks with Approximation
Authors: Tae Jun Ham, Sung Jun Jung, Seonghak Kim, Young H. Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W. Lee, Deog-Kyoon Jeong
Date: 2020-02-22 00:00:00
Tasks: Machine Translation, Translation

With the increasing computational demands of neural networks, many hardware accelerators for the neural networks have been proposed. Such existing neural network accelerators often focus on popular neural network types such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs); however, not much attention has been paid to attention mechanisms, an emerging neural network primitive that enables neural networks to retrieve most relevant information from a knowledge-base, external memory, or past states. The attention mechanism is widely adopted by many state-of-the-art neural networks for computer vision, natural language processing, and machine translation, and accounts for a large portion of total execution time. We observe today's practice of implementing this mechanism using matrix-vector multiplication is suboptimal as the attention mechanism is semantically a content-based search where a large portion of computations ends up not being used. Based on this observation, we design and architect A3, which accelerates attention mechanisms in neural networks with algorithmic approximation and hardware specialization. Our proposed accelerator achieves multiple orders of magnitude improvement in energy efficiency (performance/watt) as well as substantial speedup over the state-of-the-art conventional hardware.
Unsupervised Interpretable Representation Learning for Singing Voice Separation
Authors: Stylianos I. Mimilakis, Konstantinos Drossos, Gerald Schuller
Date: 2020-03-03 00:00:00
Tasks: Denoising, Music Source Separation, Representation Learning

In this work, we present a method for learning interpretable music signal representations directly from waveform signals. Our method can be trained using unsupervised objectives and relies on the denoising auto-encoder model that uses a simple sinusoidal model as decoding functions to reconstruct the singing voice. To demonstrate the benefits of our method, we employ the obtained representations to the task of informed singing voice separation via binary masking, and measure the obtained separation quality by means of scale-invariant signal to distortion ratio. Our findings suggest that our method is capable of learning meaningful representations for singing voice separation, while preserving conveniences of the the short-time Fourier transform like non-negativity, smoothness, and reconstruction subject to time-frequency masking, that are desired in audio and music source separation.
Causal Inference and Data Fusion in Econometrics
Authors: Paul Hünermund, Elias Bareinboim
Date: 2019-12-19 00:00:00
Tasks: Causal Inference, Econometrics

Learning about cause and effect is arguably the main goal in applied econometrics. In practice, the validity of these causal inferences is contingent on a number of critical assumptions regarding the type of data that has been collected and the substantive knowledge that is available. For instance, unobserved confounding factors threaten the internal validity of estimates, data availability is often limited to non-random, selection-biased samples, causal effects need to be learned from surrogate experiments with imperfect compliance, and causal knowledge has to be extrapolated across structurally heterogeneous populations. A powerful causal inference framework is required to tackle these challenges, which plague most data analysis to varying degrees. Building on the structural approach to causality introduced by Haavelmo (1943) and the graph-theoretic framework proposed by Pearl (1995), the artificial intelligence (AI) literature has developed a wide array of techniques for causal learning that allow to leverage information from various imperfect, heterogeneous, and biased data sources (Bareinboim and Pearl, 2016). In this paper, we discuss recent advances in this literature that have the potential to contribute to econometric methodology along three dimensions. First, they provide a unified and comprehensive framework for causal inference, in which the aforementioned problems can be addressed in full generality. Second, due to their origin in AI, they come together with sound, efficient, and complete algorithmic criteria for automatization of the corresponding identification task. And third, because of the nonparametric description of structural models that graph-theoretic approaches build on, they combine the strengths of both structural econometrics as well as the potential outcomes framework, and thus offer an effective middle ground between these two literature streams.
FA*IR: A Fair Top-k Ranking Algorithm
Authors: Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, Ricardo Baeza-Yates
Date: 2017-06-20 00:00:00
Tasks: Fairness

In this work, we define and solve the Fair Top-k Ranking problem, in which we want to determine a subset of k candidates from a large pool of n >> k candidates, maximizing utility (i.e., select the "best" candidates) subject to group fairness criteria. Our ranked group fairness definition extends group fairness using the standard notion of protected groups and is based on ensuring that the proportion of protected candidates in every prefix of the top-k ranking remains statistically above or indistinguishable from a given minimum. Utility is operationalized in two ways: (i) every candidate included in the top-$k$ should be more qualified than every candidate not included; and (ii) for every pair of candidates in the top-k, the more qualified candidate should be ranked above. An efficient algorithm is presented for producing the Fair Top-k Ranking, and tested experimentally on existing datasets as well as new datasets released with this paper, showing that our approach yields small distortions with respect to rankings that maximize utility without considering fairness criteria. To the best of our knowledge, this is the first algorithm grounded in statistical tests that can mitigate biases in the representation of an under-represented group along a ranked list.
Multi-timescale Nexting in a Reinforcement Learning Robot
Authors: Joseph Modayil, Adam White, Richard S. Sutton
Date: 2011-12-06 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The term "nexting" has been used by psychologists to refer to the propensity of people and many other animals to continually predict what will happen next in an immediate, local, and personal sense. The ability to "next" constitutes a basic kind of awareness and knowledge of one's environment. In this paper we present results with a robot that learns to next in real time, predicting thousands of features of the world's state, including all sensory inputs, at timescales from 0.1 to 8 seconds. This was achieved by treating each state feature as a reward-like target and applying temporal-difference methods to learn a corresponding value function with a discount rate corresponding to the timescale. We show that two thousand predictions, each dependent on six thousand state features, can be learned and updated online at better than 10Hz on a laptop computer, using the standard TD(lambda) algorithm with linear function approximation. We show that this approach is efficient enough to be practical, with most of the learning complete within 30 minutes. We also show that a single tile-coded feature representation suffices to accurately predict many different signals at a significant range of timescales. Finally, we show that the accuracy of our learned predictions compares favorably with the optimal off-line solution.
Deep-learning Architecture for Short-term Passenger Flow Forecasting in Urban Rail Transit
Authors: Jinlei Zhang, Feng Chen, Zhiyong Cui, Yinan Guo, Yadi Zhu
Date: 2019-12-29 00:00:00
Tasks: Deep Learning, Prediction

Short-term passenger flow forecasting is an essential component in urban rail transit operation. Emerging deep learning models provide good insight into improving prediction precision. Therefore, we propose a deep learning architecture combining the residual network (ResNet), graph convolutional network (GCN), and long short-term memory (LSTM) (called "ResLSTM") to forecast short-term passenger flow in urban rail transit on a network scale. First, improved methodologies of the ResNet, GCN, and attention LSTM models are presented. Then, the model architecture is proposed, wherein ResNet is used to capture deep abstract spatial correlations between subway stations, GCN is applied to extract network topology information, and attention LSTM is used to extract temporal correlations. The model architecture includes four branches for inflow, outflow, graph-network topology, as well as weather conditions and air quality. To the best of our knowledge, this is the first time that air-quality indicators have been taken into account, and their influences on prediction precision quantified. Finally, ResLSTM is applied to the Beijing subway using three time granularities (10, 15, and 30 min) to conduct short-term passenger flow forecasting. A comparison of the prediction performance of ResLSTM with those of many state-of-the-art models illustrates the advantages and robustness of ResLSTM. Moreover, a comparison of the prediction precisions obtained for time granularities of 10, 15, and 30 min indicates that prediction precision increases with increasing time granularity. This study can provide subway operators with insight into short-term passenger flow forecasting by leveraging deep learning models.
AddGraph_ Anomaly Detection in Dynamic Graph Using Attention-based Temporal GCN
Authors: Li Zheng, Zhenpeng Li, Jian Li, Zhao Li， and Jun Gao
Date: 2019-06-24 00:00:00
Proceeding: (IJCAI 2019 6
Tasks: Anomaly Detection, Edge Detection, Recommendation Systems

Anomaly detection in dynamic graphs becomes
very critical in many different application scenarios, e.g., recommender systems, while it also raises
huge challenges due to the high flexible nature of
anomaly and lack of sufficient labelled data. It is
better to learn the anomaly patterns by considering all possible hints including the structural, content and temporal features, rather than utilizing
heuristic rules over the partial features. In this paper, we propose AddGraph, a general end-to-end
anomalous edge detection framework using an extended temporal GCN (Graph Convolutional Network) with an attention model, which can capture
both long-term patterns and the short-term patterns
in dynamic graphs. In order to cope with insuffi-
cient explicit labelled data, we employ a selective
negative sampling and margin loss in training of
AddGraph in a semi-supervised fashion. We conduct extensive experiments on real-world datasets,
and illustrate that AddGraph can outperform the
state-of-the-art competitors in anomaly detection
significantly
A Uniﬁed Model for Extractive and Abstractive Summarization using Inconsistency Loss
Authors: Wan-TingHsu1, Chieh-KaiLin1, Ming-YingLee1, KeruiMin2, JingTang2, MinSun1 1 National Tsing Hua University, 2 Cheetah Mobile
Date: 2018-07-05 00:00:00
Proceeding: AAAI 2018 7
Tasks: Abstractive Text Summarization, Sentence

We propose a uniﬁed model combining the strength of extractive and abstractive summarization. On the one hand, a simple extractive model can obtain sentence-level attention with high ROUGE scores but less readable. On the other hand, a more complicated abstractive model can obtain word-level dynamic attention to generate a more readable paragraph. In our model, sentence-level attention is used to modulate the word-level attention such that words in less attended sentences are less likely to be generated. Moreover, a novel inconsistency loss function is introduced to penalize the inconsistency between two levels of attentions. By end-to-end training our model with the inconsistency loss and original losses of extractive and abstractive models, we achieve state-of-theart ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation.
Towards perspective-free object counting with deep learning
Authors: Daniel O˜noro-Rubio, Roberto J. L´opez-Sastre
Date: 2016-01-01 00:00:00
Proceeding: journal 2016 1
Tasks: Deep Learning, Object, Object Counting, regression

In this paper we address the problem of counting objects
instances in images. Our models are able to precisely estimate the number of vehicles in a traffic congestion, or to count the humans in a very
crowded scene. Our first contribution is the proposal of a novel convolutional neural network solution, named Counting CNN (CCNN). Essentially, the CCNN is formulated as a regression model where the network
learns how to map the appearance of the image patches to their corresponding object density maps. Our second contribution consists in
a scale-aware counting model, the Hydra CNN, able to estimate object
densities in different very crowded scenarios where no geometric information of the scene can be provided. Hydra CNN learns a multiscale
non-linear regression model which uses a pyramid of image patches extracted at multiple scales to perform the final density prediction. We
report an extensive experimental evaluation, using up to three different
object counting benchmarks, where we show how our solutions achieve
a state-of-the-art performance.