pwc-archive/papers-with-abstracts huggingface dataset:
Identifying DNS-tunneled traffic with predictive models
Authors: Andreas Berg, Daniel Forsberg
Date: 2019-06-26 00:00:00
Tasks: BIG-bench Machine Learning

DNS is a distributed, fault tolerant system that avoids a single point of failure. As such it is an integral part of the internet as we use it today and hence deemed a safe protocol which is let through firewalls and proxies with no or little checks. This can be exploited by malicious agents. Network forensics is effective but struggles due to size of data and manual labour. This paper explores to what extent predictive models can be used to predict network traffic, what protocols are tunneled in the DNS protocol and more specifically whether the predictive performance is enhanced when analyzing DNS-queries and responses together and which feature set that can be used for DNS-tunneled network prediction. The tested protocols are SSH, SFTP and Telnet and the machine learning models used are Multi Layered Perceptron and Random Forests. To train the models we extract the IP Packet length, Name length and Name entropy of both the queries and responses in the DNS traffic. With an experimental research strategy it is empirically shown that the performance of the models increases when training the models on the query and respose pairs rather than using only queries or responses. The accuracy of the models is >83% and reduction in data size when features are extracted is roughly 95%. Our results provides evidence that machine learning is a valuable tool in detecting network protocols in a DNS tunnel and that only an small subset of network traffic is needed to detect this anomaly.
RUSLAN: Russian Spoken Language Corpus for Speech Synthesis
Authors: Lenar Gabdrakhmanov, Rustem Garaev, Evgenii Razinkov
Date: 2019-06-26 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech

We present RUSLAN -- a new open Russian spoken language corpus for the text-to-speech task. RUSLAN contains 22200 audio samples with text annotations -- more than 31 hours of high-quality speech of one person -- being the largest annotated Russian corpus in terms of speech duration for a single speaker. We trained an end-to-end neural network for the text-to-speech task on our corpus and evaluated the quality of the synthesized speech using Mean Opinion Score test. Synthesized speech achieves 4.05 score for naturalness and 3.78 score for intelligibility on a 5-point MOS scale.
Signed Graph Attention Networks
Authors: Junjie Huang, Hua-Wei Shen, Liang Hou, Xue-Qi Cheng
Date: 2019-06-26 00:00:00
Tasks: Graph Attention, Graph Neural Network, Link Prediction, Link Sign Prediction, Network Embedding, Node Classification

Graph or network data is ubiquitous in the real world, including social networks, information networks, traffic networks, biological networks and various technical networks. The non-Euclidean nature of graph data poses the challenge for modeling and analyzing graph data. Recently, Graph Neural Network (GNNs) are proposed as a general and powerful framework to handle tasks on graph data, e.g., node embedding, link prediction and node classification. As a representative implementation of GNNs, Graph Attention Networks (GATs) are successfully applied in a variety of tasks on real datasets. However, GAT is designed to networks with only positive links and fails to handle signed networks which contain both positive and negative links. In this paper, we propose Signed Graph Attention Networks (SiGATs), generalizing GAT to signed networks. SiGAT incorporates graph motifs into GAT to capture two well-known theories in signed network research, i.e., balance theory and status theory. In SiGAT, motifs offer us the flexible structural pattern to aggregate and propagate messages on the signed network to generate node embeddings. We evaluate the proposed SiGAT method by applying it to the signed link prediction task. Experimental results on three real datasets demonstrate that SiGAT outperforms feature-based method, network embedding method and state-of-the-art GNN-based methods like signed graph convolutional network (SGCN).
A Tractable Algorithm For Finite-Horizon Continuous Reinforcement Learning
Authors: Phanideep Gampa, Sairam Satwik Kondamudi, Lakshmanan Kailasam
Date: 2019-06-26 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider the finite horizon continuous reinforcement learning problem. Our contribution is three-fold. First,we give a tractable algorithm based on optimistic value iteration for the problem. Next,we give a lower bound on regret of order $\Omega(T^{2/3})$ for any algorithm discretizes the state space, improving the previous regret bound of $\Omega(T^{1/2})$ of Ortner and Ryabko \cite{contrl} for the same problem. Next,under the assumption that the rewards and transitions are H\"{o}lder Continuous we show that the upper bound on the discretization error is $const.Ln^{-\alpha}T$. Finally,we give some simple experiments to validate our propositions.
Artificial Intelligence: the global landscape of ethics guidelines
Authors: Anna Jobin, Marcello Ienca, Effy Vayena
Date: 2019-06-24 00:00:00
Tasks: Ethics, Fairness

In the last five years, private companies, research institutions as well as public sector organisations have issued principles and guidelines for ethical AI, yet there is debate about both what constitutes "ethical AI" and which ethical requirements, technical standards and best practices are needed for its realization. To investigate whether a global agreement on these questions is emerging, we mapped and analyzed the current corpus of principles and guidelines on ethical AI. Our results reveal a global convergence emerging around five ethical principles (transparency, justice and fairness, non-maleficence, responsibility and privacy), with substantive divergence in relation to how these principles are interpreted; why they are deemed important; what issue, domain or actors they pertain to; and how they should be implemented. Our findings highlight the importance of integrating guideline-development efforts with substantive ethical analysis and adequate implementation strategies.
Repeated sequential learning increases memory capacity via effective decorrelation in a recurrent neural network
Authors: Tomoki Kurikawa, Omri Barak, Kunihiko Kaneko
Date: 2019-06-22 00:00:00

Memories in neural system are shaped through the interplay of neural and learning dynamics under external inputs. By introducing a simple local learning rule to a neural network, we found that the memory capacity is drastically increased by sequentially repeating the learning steps of input-output mappings. The origin of this enhancement is attributed to the generation of a Psuedo-inverse correlation in the connectivity. This is associated with the emergence of spontaneous activity that intermittently exhibits neural patterns corresponding to embedded memories. Stablization of memories is achieved by a distinct bifurcation from the spontaneous activity under the application of each input.
A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry
Authors: Baihan Lin, Guillermo Cecchi, Djallel Bouneffouf, Jenna Reinen, Irina Rish
Date: 2019-06-21 00:00:00
Tasks: Decision Making, Lifelong learning, Q-Learning, Recommendation Systems, Reinforcement Learning, Reinforcement Learning (RL)

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-processing biases -- an important component of human decision making which can help us better understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems, as well as various neuropsychiatric conditions associated with disruptions in normal reward processing. From the computational perspective, we observe that the proposed Split-QL model and its clinically inspired variants consistently outperform standard Q-Learning and SARSA methods, as well as recently proposed Double Q-Learning approaches, on simulated tasks with particular reward distributions, a real-world dataset capturing human decision-making in gambling tasks, and the Pac-Man game in a lifelong learning setting across different reward stationarities.
Low-dimensional Embodied Semantics for Music and Language
Authors: Francisco Afonso Raposo, David Martins de Matos, Ricardo Ribeiro
Date: 2019-06-20 00:00:00
Tasks: Topic Classification

Embodied cognition states that semantics is encoded in the brain as firing patterns of neural circuits, which are learned according to the statistical structure of human multimodal experience. However, each human brain is idiosyncratically biased, according to its subjective experience history, making this biological semantic machinery noisy with respect to the overall semantics inherent to media artifacts, such as music and language excerpts. We propose to represent shared semantics using low-dimensional vector embeddings by jointly modeling several brains from human subjects. We show these unsupervised efficient representations outperform the original high-dimensional fMRI voxel spaces in proxy music genre and language topic classification tasks. We further show that joint modeling of several subjects increases the semantic richness of the learned latent vector spaces.
Resonator Networks outperform optimization methods at solving high-dimensional vector factorization
Authors: Spencer J. Kent, E. Paxon Frady, Friedrich T. Sommer, Bruno A. Olshausen
Date: 2019-06-19 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We develop theoretical foundations of Resonator Networks, a new type of recurrent neural network introduced in Frady et al. (2020) to solve a high-dimensional vector factorization problem arising in Vector Symbolic Architectures. Given a composite vector formed by the Hadamard product between a discrete set of high-dimensional vectors, a Resonator Network can efficiently decompose the composite into these factors. We compare the performance of Resonator Networks against optimization-based methods, including Alternating Least Squares and several gradient-based algorithms, showing that Resonator Networks are superior in several important ways. This advantage is achieved by leveraging a combination of nonlinear dynamics and "searching in superposition," by which estimates of the correct solution are formed from a weighted superposition of all possible solutions. While the alternative methods also search in superposition, the dynamics of Resonator Networks allow them to strike a more effective balance between exploring the solution space and exploiting local information to drive the network toward probable solutions. Resonator Networks are not guaranteed to converge, but within a particular regime they almost always do. In exchange for relaxing this guarantee of global convergence, Resonator Networks are dramatically more effective at finding factorizations than all alternative approaches considered.
Inspirational Adversarial Image Generation
Authors: Baptiste Rozière, Morgane Riviere, Olivier Teytaud, Jérémy Rapin, Yann Lecun, Camille Couprie
Date: 2019-06-17 00:00:00
Tasks: Image Generation

The task of image generation started to receive some attention from artists and designers to inspire them in new creations. However, exploiting the results of deep generative models such as Generative Adversarial Networks can be long and tedious given the lack of existing tools. In this work, we propose a simple strategy to inspire creators with new generations learned from a dataset of their choice, while providing some control on them. We design a simple optimization method to find the optimal latent parameters corresponding to the closest generation to any input inspirational image. Specifically, we allow the generation given an inspirational image of the user choice by performing several optimization steps to recover optimal parameters from the model's latent space. We tested several exploration methods starting with classic gradient descents to gradient-free optimizers. Many gradient-free optimizers just need comparisons (better/worse than another image), so that they can even be used without numerical criterion, without inspirational image, but with only with human preference. Thus, by iterating on one's preferences we could make robust Facial Composite or Fashion Generation algorithms. High resolution of the produced design generations are obtained using progressive growing of GANs. Our results on four datasets of faces, fashion images, and textures show that satisfactory images are effectively retrieved in most cases.
Lattice Map Spiking Neural Networks (LM-SNNs) for Clustering and Classifying Image Data
Authors: Hananel Hazan, Daniel J. Saunders, Darpan T. Sanghavi, Hava Siegelmann, Robert Kozma
Date: 2019-06-04 00:00:00
Tasks: Clustering

Spiking neural networks (SNNs) with a lattice architecture are introduced in this work, combining several desirable properties of SNNs and self-organized maps (SOMs). Networks are trained with biologically motivated, unsupervised learning rules to obtain a self-organized grid of filters via cooperative and competitive excitatory-inhibitory interactions. Several inhibition strategies are developed and tested, such as (i) incrementally increasing inhibition level over the course of network training, and (ii) switching the inhibition level from low to high (two-level) after an initial training segment. During the labeling phase, the spiking activity generated by data with known labels is used to assign neurons to categories of data, which are then used to evaluate the network's classification ability on a held-out set of test data. Several biologically plausible evaluation rules are proposed and compared, including a population-level confidence rating, and an $n$-gram inspired method. The effectiveness of the proposed self-organized learning mechanism is tested using the MNIST benchmark dataset, as well as using images produced by playing the Atari Breakout game.
Enhancing temporal segmentation by nonlocal self-similarity
Authors: Mariella Dimiccoli, Herwig Wendt
Date: 2019-06-14 00:00:00
Tasks: Event Segmentation, Segmentation

Temporal segmentation of untrimmed videos and photo-streams is currently an active area of research in computer vision and image processing. This paper proposes a new approach to improve the temporal segmentation of photo-streams. The method consists in enhancing image representations by encoding long-range temporal dependencies. Our key contribution is to take advantage of the temporal stationarity assumption of photostreams for modeling each frame by its nonlocal self-similarity function. The proposed approach is put to test on the EDUB-Seg dataset, a standard benchmark for egocentric photostream temporal segmentation. Starting from seven different (CNN based) image features, the method yields consistent improvements in event segmentation quality, leading to an average increase of F-measure of 3.71% with respect to the state of the art.
Effects of rainfall on Culex mosquito population dynamics
Authors: L. D. Valdez, G. J. Sibona, L. A. Diaz, M. S. Contigiani, C. A. Condat
Date: 2017-03-27 00:00:00

The dynamics of a mosquito population depends heavily on climatic variables such as temperature and precipitation. Since climate change models predict that global warming will impact on the frequency and intensity of rainfall, it is important to understand how these variables affect the mosquito populations. We present a model of the dynamics of a {\it Culex quinquefasciatus} mosquito population that incorporates the effect of rainfall and use it to study the influence of the number of rainy days and the mean monthly precipitation on the maximum yearly abundance of mosquitoes $M_{max}$. Additionally, using a fracturing process, we investigate the influence of the variability in daily rainfall on $M_{max}$. We find that, given a constant value of monthly precipitation, there is an optimum number of rainy days for which $M_{max}$ is a maximum. On the other hand, we show that increasing daily rainfall variability reduces the dependence of $M_{max}$ on the number of rainy days, leading also to a higher abundance of mosquitoes for the case of low mean monthly precipitation. Finally, we explore the effect of the rainfall in the months preceding the wettest season, and we obtain that a regimen with high precipitations throughout the year and a higher variability tends to advance slightly the time at which the peak mosquito abundance occurs, but could significantly change the total mosquito abundance in a year.
A Bit of Progress in Language Modeling
Authors: Joshua Goodman
Date: 2001-08-09 00:00:00
Tasks: Clustering, Language Modeling, Language Modelling, Sentence

In the past several years, a number of different language modeling improvements over simple trigram models have been found, including caching, higher-order n-grams, skipping, interpolated Kneser-Ney smoothing, and clustering. We present explorations of variations on, or of the limits of, each of these techniques, including showing that sentence mixture models may have more potential. While all of these techniques have been studied separately, they have rarely been studied in combination. We find some significant interactions, especially with smoothing and clustering techniques. We compare a combination of all techniques together to a Katz smoothed trigram model with no count cutoffs. We achieve perplexity reductions between 38% and 50% (1 bit of entropy), depending on training data size, as well as a word error rate reduction of 8.9%. Our perplexity reductions are perhaps the highest reported compared to a fair baseline. This is the extended version of the paper; it contains additional details and proofs, and is designed to be a good introduction to the state of the art in language modeling.
Efficiently Indexing Large Sparse Graphs for Similarity Search
Authors: Guoren Wang, Bin Wang, Xiaochun Yang, IEEE Computer Society, and Ge Yu, Member, IEEE
Date: 2010-02-18 00:00:00

Abstract—The graph structure is a very important means to model schemaless data with complicated structures, such as protein-
protein interaction networks, chemical compounds, knowledge query inferring systems, and road networks. This paper focuses on the
index structure for similarity search on a set of large sparse graphs and proposes an efficient indexing mechanism by introducing the
Q-Gram idea. By decomposing graphs to small grams (organized by -Adjacent Tree patterns) and pairing-up on those -Adjacent
Tree patterns, the lower bound estimation of their edit distance can be calculated for candidate filtering. Furthermore, we have
developed a series of techniques for inverted index construction and online query processing. By building the candidate set for the
query graph before the exact edit distance calculation, the number of graphs need to proceed into exact matching can be greatly
reduced. Extensive experiments on real and synthetic data sets have been conducted to show the effectiveness and efficiency of the
proposed indexing mechanism.
MLFriend: Interactive Prediction Task Recommendation for Event-Driven Time-Series Data
Authors: Lei Xu, Shubhra Kanti Karmaker Santu, Kalyan Veeramachaneni
Date: 2019-06-28 00:00:00
Tasks: Model Selection, Prediction, Time Series, Time Series Analysis

Most automation in machine learning focuses on model selection and hyper parameter tuning, and many overlook the challenge of automatically defining predictive tasks. We still heavily rely on human experts to define prediction tasks, and generate labels by aggregating raw data. In this paper, we tackle the challenge of defining useful prediction problems on event-driven time-series data. We introduce MLFriend to address this challenge. MLFriend first generates all possible prediction tasks under a predefined space, then interacts with a data scientist to learn the context of the data and recommend good prediction tasks from all the tasks in the space. We evaluate our system on three different datasets and generate a total of 2885 prediction tasks and solve them. Out of these 722 were deemed useful by expert data scientists. We also show that an automatic prediction task discovery system is able to identify top 10 tasks that a user may like within a batch of 100 tasks.
Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning
Authors: Shi Pu, Alex Olshevsky, Ioannis Ch. Paschalidis
Date: 2019-06-28 00:00:00
Tasks: BIG-bench Machine Learning, Distributed Optimization, Stochastic Optimization

We provide a discussion of several recent results which, in certain scenarios, are able to overcome a barrier in distributed stochastic optimization for machine learning. Our focus is the so-called asymptotic network independence property, which is achieved whenever a distributed method executed over a network of n nodes asymptotically converges to the optimal solution at a comparable rate to a centralized method with the same computational power as the entire network. We explain this property through an example involving the training of ML models and sketch a short mathematical analysis for comparing the performance of distributed stochastic gradient descent (DSGD) with centralized stochastic gradient decent (SGD).
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
Authors: Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, Dawn Song
Date: 2019-06-28 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Anomaly Detection, Outlier Detection, Out-of-Distribution Detection, Self-Supervised Learning, Unsupervised Anomaly Detection

Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often not thought beneficial beyond obviating or reducing the need for annotations. We find that self-supervision can benefit robustness in a variety of ways, including robustness to adversarial examples, label corruption, and common input corruptions. Additionally, self-supervision greatly benefits out-of-distribution detection on difficult, near-distribution outliers, so much so that it exceeds the performance of fully supervised methods. These results demonstrate the promise of self-supervision for improving robustness and uncertainty estimation and establish these tasks as new axes of evaluation for future self-supervised learning research.