pwc-archive/papers-with-abstracts huggingface dataset:
Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks
Authors: Stefano Savazzi, Monica Nicoli, Vittorio Rampa
Date: 2019-12-27 00:00:00
Tasks: Federated Learning

Federated learning (FL) is emerging as a new paradigm to train machine learning models in distributed systems. Rather than sharing, and disclosing, the training dataset with the server, the model parameters (e.g. neural networks weights and biases) are optimized collectively by large populations of interconnected devices, acting as local learners. FL can be applied to power-constrained IoT devices with slow and sporadic connections. In addition, it does not need data to be exported to third parties, preserving privacy. Despite these benefits, a main limit of existing approaches is the centralized optimization which relies on a server for aggregation and fusion of local parameters; this has the drawback of a single point of failure and scaling issues for increasing network size. The paper proposes a fully distributed (or server-less) learning approach: the proposed FL algorithms leverage the cooperation of devices that perform data operations inside the network by iterating local computations and mutual interactions via consensus-based methods. The approach lays the groundwork for integration of FL within 5G and beyond networks characterized by decentralized connectivity and computing, with intelligence distributed over the end-devices. The proposed methodology is verified by experimental datasets collected inside an industrial IoT environment.
3D Sensing of a Moving Object with a Nodding 2D LIDAR and Reconfigurable Mirrors
Authors: Anindya Harchowdhury, Lindsay Kleeman, Leena Vachhani
Date: 2019-12-27 00:00:00
Tasks: object-detection, Object Detection

Perception in 3D has become standard practice for a large part of robotics applications. High quality 3D perception is costly. Our previous work on a nodding 2D Lidar provides high quality 3D depth information with low cost, but the sparse data generated by this sensor poses challenges in understanding the characteristics of moving objects within an uncertain environment. This paper proposes a novel design of the nodding Lidar but provides dynamic reconfigurability in terms of limiting the field of view of the sensor using a set of optical mirrors. It not only provides denser scans, but it also achieves a three times higher scan update rate. Additionally, we propose a novel calibration mechanism for this sensor and prove its effectiveness for dynamic object detection and tracking.
Amharic-Arabic Neural Machine Translation
Authors: Ibrahim Gashaw, H. L. Shashirekha
Date: 2019-12-26 00:00:00
Tasks: Decoder, Machine Translation, NMT, Translation

Many automatic translation works have been addressed between major European language pairs, by taking advantage of large scale parallel corpora, but very few research works are conducted on the Amharic-Arabic language pair due to its parallel data scarcity. Two Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU) based Neural Machine Translation (NMT) models are developed using Attention-based Encoder-Decoder architecture which is adapted from the open-source OpenNMT system. In order to perform the experiment, a small parallel Quranic text corpus is constructed by modifying the existing monolingual Arabic text and its equivalent translation of Amharic language text corpora available on Tanzile. LSTM and GRU based NMT models and Google Translation system are compared and found that LSTM based OpenNMT outperforms GRU based OpenNMT and Google Translation system, with a BLEU score of 12%, 11%, and 6% respectively.
Classifier Chains: A Review and Perspectives
Authors: Jesse Read, Bernhard Pfahringer, Geoff Holmes, Eibe Frank
Date: 2019-12-26 00:00:00
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning

The family of methods collectively known as classifier chains has become a popular approach to multi-label learning problems. This approach involves linking together off-the-shelf binary classifiers in a chain structure, such that class label predictions become features for other classifiers. Such methods have proved flexible and effective and have obtained state-of-the-art empirical performance across many datasets and multi-label evaluation metrics. This performance led to further studies of how exactly it works, and how it could be improved, and in the recent decade numerous studies have explored classifier chains mechanisms on a theoretical level, and many improvements have been made to the training and inference procedures, such that this method remains among the state-of-the-art options for multi-label learning. Given this past and ongoing interest, which covers a broad range of applications and research themes, the goal of this work is to provide a review of classifier chains, a survey of the techniques and extensions provided in the literature, as well as perspectives for this approach in the domain of multi-label classification in the future. We conclude positively, with a number of recommendations for researchers and practitioners, as well as outlining a number of areas for future research.
Text Classification for Azerbaijani Language Using Machine Learning and Embedding
Authors: Umid Suleymanov, Behnam Kiani Kalejahi, Elkhan Amrahov, Rashid Badirkhanli
Date: 2019-12-26 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Clustering, General Classification, Sentiment Analysis, text-classification, Text Classification, Text Clustering

Text classification systems will help to solve the text clustering problem in the Azerbaijani language. There are some text-classification applications for foreign languages, but we tried to build a newly developed system to solve this problem for the Azerbaijani language. Firstly, we tried to find out potential practice areas. The system will be useful in a lot of areas. It will be mostly used in news feed categorization. News websites can automatically categorize news into classes such as sports, business, education, science, etc. The system is also used in sentiment analysis for product reviews. For example, the company shares a photo of a new product on Facebook and the company receives a thousand comments for new products. The systems classify the comments into categories like positive or negative. The system can also be applied in recommended systems, spam filtering, etc. Various machine learning techniques such as Naive Bayes, SVM, Decision Trees have been devised to solve the text classification problem in Azerbaijani language.
Finite-State Extreme Effect Variable
Authors: Alexey Drutsa
Date: 2019-12-24 00:00:00

We generalize to the finite-state case the notion of the extreme effect variable $Y$ that accumulates all the effect of a variant variable $V$ observed in changes of another variable $X$. We conduct theoretical analysis and turn the problem of finding of an effect variable into a problem of a simultaneous decomposition of a set of distributions. The states of the extreme effect variable, on the one hand, are minimally affected by the variant variable $V$ and, on the other hand, are extremely different with respect to the observable variable $X$. We apply our technique to online evaluation of a web search engine through A/B testing and show its utility.
The Temporal Dynamics of Belief-based Updating of Epistemic Trust: Light at the End of the Tunnel?
Authors: Momme von Sydow, Christoph Merdes, Ulrike Hahn
Date: 2019-12-24 00:00:00

We start with the distinction of outcome- and belief-based Bayesian models of the sequential update of agents' beliefs and subjective reliability of sources (trust). We then focus on discussing the influential Bayesian model of belief-based trust update by Eric Olsson, which models dichotomic events and explicitly represents anti-reliability. After sketching some disastrous recent results for this perhaps most promising model of belief update, we show new simulation results for the temporal dynamics of learning belief with and without trust update and with and without communication. The results seem to shed at least a somewhat more positive light on the communicating-and-trust-updating agents. This may be a light at the end of the tunnel of belief-based models of trust updating, but the interpretation of the clear findings is much less clear.
Robust Group Synchronization via Cycle-Edge Message Passing
Authors: Gilad Lerman, Yunpeng Shi
Date: 2019-12-24 00:00:00

We propose a general framework for solving the group synchronization problem, where we focus on the setting of adversarial or uniform corruption and sufficiently small noise. Specifically, we apply a novel message passing procedure that uses cycle consistency information in order to estimate the corruption levels of group ratios and consequently solve the synchronization problem in our setting. We first explain why the group cycle consistency information is essential for effectively solving group synchronization problems. We then establish exact recovery and linear convergence guarantees for the proposed message passing procedure under a deterministic setting with adversarial corruption. These guarantees hold as long as the ratio of corrupted cycles per edge is bounded by a reasonable constant. We also establish the stability of the proposed procedure to sub-Gaussian noise. We further establish exact recovery with high probability under a common uniform corruption model.
Fast and Accurate Transferability Measurement for Heterogeneous Multivariate Data
Authors: Seungcheol Park, Huiwen Xu, Taehun Kim, Inhwan Hwang, Kyung-Jun Kim, U Kang
Date: 2019-12-23 00:00:00
Tasks: Decoder

Given a set of heterogeneous source datasets with their classifiers, how can we quickly find the most useful source dataset for a specific target task? We address the problem of measuring transferability between source and target datasets, where the source and the target have different feature spaces and distributions. We propose Transmeter, a fast and accurate method to estimate the transferability of two heterogeneous multivariate datasets. We address three challenges in measuring transferability between two heterogeneous multivariate datasets: reducing time, minimizing domain gap, and extracting meaningful homogeneous representations. To overcome the above issues, we utilize a pre-trained source model, an adversarial network, and an encoder-decoder architecture. Extensive experiments on heterogeneous multivariate datasets show that Transmeter gives the most accurate transferability measurement with up to 10.3 times faster performance than its competitor. We also show that selecting the best source data with Transmeter followed by a full transfer leads to the best transfer accuracy and the fastest running time.
Learning functionals via LSTM neural networks for predicting vessel dynamics in extreme sea states
Authors: José del Águila Ferrandis, Michael Triantafyllou, Chryssostomos Chryssostomidis, George Karniadakis
Date: 2019-12-23 00:00:00

Predicting motions of vessels in extreme sea states represents one of the most challenging problems in naval hydrodynamics. It involves computing complex nonlinear wave-body interactions, hence taxing heavily computational resources. Here, we put forward a new simulation paradigm by training recurrent type neural networks (RNNs) that take as input the stochastic wave elevation at a certain sea state and output the main vessel motions, e.g., pitch, heave and roll. We first compare the performance of standard RNNs versus GRU and LSTM neural networks (NNs) and show that LSTM NNs lead to the best performance. We then examine the testing error of two representative vessels, a catamaran in sea state 1 and a battleship in sea state 8. We demonstrate that good accuracy is achieved for both cases in predicting the vessel motions for unseen wave elevations. We train the NNs with expensive CFD simulations offline, but upon training, the prediction of the vessel dynamics online can be obtained at a fraction of a second. This work is motivated by the universal approximation theorem for functionals [1], and it is the first implementation of such theory to realistic engineering problems.
Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings
Authors: Maxat Tezekbayev, Zhenisbek Assylbekov, Rustem Takhanov
Date: 2019-12-23 00:00:00

We show that the skip-gram embedding of any word can be decomposed into two subvectors which roughly correspond to semantic and syntactic roles of the word.
Learning Representations by Maximizing Mutual Information in Variational Autoencoders
Authors: Ali Lotfi Rezaabad, Sriram Vishwanath
Date: 2019-12-21 00:00:00
Tasks: Representation Learning

Variational autoencoders (VAEs) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously result in meaningful learned representations. Our solution is to combine traditional VAEs with mutual information maximization, with the goal to enhance amortized inference in VAEs using Information Theoretic techniques. We call this approach InfoMax-VAE, and such an approach can significantly boost the quality of learned high-level representations. We realize this through the explicit maximization of information measures associated with the representation. Using extensive experiments on varied datasets and setups, we show that InfoMax-VAE outperforms contemporary popular approaches, including Info-VAE and $\beta$-VAE.
Data Augmentation by AutoEncoders for Unsupervised Anomaly Detection
Authors: Kasra Babaei, ZhiYuan Chen, Tomas Maul
Date: 2019-12-21 00:00:00
Tasks: Anomaly Detection, Data Augmentation, Unsupervised Anomaly Detection

This paper proposes an autoencoder (AE) that is used for improving the performance of once-class classifiers for the purpose of detecting anomalies. Traditional one-class classifiers (OCCs) perform poorly under certain conditions such as high-dimensionality and sparsity. Also, the size of the training set plays an important role on the performance of one-class classifiers. Autoencoders have been widely used for obtaining useful latent variables from high-dimensional datasets. In the proposed approach, the AE is capable of deriving meaningful features from high-dimensional datasets while doing data augmentation at the same time. The augmented data is used for training the OCC algorithms. The experimental results show that the proposed approach enhance the performance of OCC algorithms and also outperforms other well-known approaches.
AEGR: A simple approach to gradient reversal in autoencoders for network anomaly detection
Authors: Kasra Babaei, Zhi Yuan Chen, Tomas Maul
Date: 2019-12-21 00:00:00
Tasks: Anomaly Detection, Data Augmentation

Anomaly detection is referred to as a process in which the aim is to detect data points that follow a different pattern from the majority of data points. Anomaly detection methods suffer from several well-known challenges that hinder their performance such as high dimensionality. Autoencoders are unsupervised neural networks that have been used for the purpose of reducing dimensionality and also detecting network anomalies in large datasets. The performance of autoencoders debilitates when the training set contains noise and anomalies. In this paper, a new gradient-reversal method is proposed to overcome the influence of anomalies on the training phase for the purpose of detecting network anomalies. The method is different from other approaches as it does not require an anomaly-free training set and is based on reconstruction error. Once latent variables are extracted from the network, Local Outlier Factor is used to separate normal data points from anomalies. A simple pruning approach and data augmentation is also added to further improve performance. The experimental results show that the proposed model can outperform other well-know approaches.
Predictive Coding for Boosting Deep Reinforcement Learning with Sparse Rewards
Authors: Xingyu Lu, Stas Tiomkin, Pieter Abbeel
Date: 2019-12-21 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward shaping method through predictive coding to tackle sparse reward problems. By learning predictive representations offline and using these representations for reward shaping, we gain access to reward signals that understand the structure and dynamics of the environment. In particular, our method achieves better learning by providing reward signals that 1) understand environment dynamics 2) emphasize on features most useful for learning 3) resist noise in learned representations through reward accumulation. We demonstrate the usefulness of this approach in different domains ranging from robotic manipulation to navigation, and we show that reward signals produced through predictive coding are as effective for learning as hand-crafted rewards.
End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models
Authors: John Giorgi, Xindi Wang, Nicola Sahar, Won Young Shin, Gary D. Bader, Bo wang
Date: 2019-12-20 00:00:00
Tasks: Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Relation, Relation Extraction, Retrieval

Named entity recognition (NER) and relation extraction (RE) are two important tasks in information extraction and retrieval (IE \& IR). Recent work has demonstrated that it is beneficial to learn these tasks jointly, which avoids the propagation of error inherent in pipeline-based systems and improves performance. However, state-of-the-art joint models typically rely on external natural language processing (NLP) tools, such as dependency parsers, limiting their usefulness to domains (e.g. news) where those tools perform well. The few neural, end-to-end models that have been proposed are trained almost completely from scratch. In this paper, we propose a neural, end-to-end model for jointly extracting entities and their relations which does not rely on external NLP tools and which integrates a large, pre-trained language model. Because the bulk of our model's parameters are pre-trained and we eschew recurrence for self-attention, our model is fast to train. On 5 datasets across 3 domains, our model matches or exceeds state-of-the-art performance, sometimes by a large margin.
Water Supply Prediction Based on Initialized Attention Residual Network
Authors: Yuhao Long, Jingcheng Wang, Jingyi Wang
Date: 2019-12-17 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis

Real-time and accurate water supply forecast is crucial for water plant. However, most existing methods are likely affected by factors such as weather and holidays, which lead to a decline in the reliability of water supply prediction. In this paper, we address a generic artificial neural network, called Initialized Attention Residual Network (IARN), which is combined with an attention module and residual modules. Specifically, instead of continuing to use the recurrent neural network (RNN) in time-series tasks, we try to build a convolution neural network (CNN)to recede the disturb from other factors, relieve the limitation of memory size and get a more credible results. Our method achieves state-of-the-art performance on several data sets, in terms of accuracy, robustness and generalization ability.
Unsupervised Detection of Sub-events in Large Scale Disasters
Authors: Chidubem Arachie, Manas Gaur, Sam Anzaroot, William Groves, Ke Zhang, Alejandro Jaimes
Date: 2019-12-13 00:00:00

Social media plays a major role during and after major natural disasters (e.g., hurricanes, large-scale fires, etc.), as people ``on the ground'' post useful information on what is actually happening. Given the large amounts of posts, a major challenge is identifying the information that is useful and actionable. Emergency responders are largely interested in finding out what events are taking place so they can properly plan and deploy resources. In this paper we address the problem of automatically identifying important sub-events (within a large-scale emergency ``event'', such as a hurricane). In particular, we present a novel, unsupervised learning framework to detect sub-events in Tweets for retrospective crisis analysis. We first extract noun-verb pairs and phrases from raw tweets as sub-event candidates. Then, we learn a semantic embedding of extracted noun-verb pairs and phrases, and rank them against a crisis-specific ontology. We filter out noisy and irrelevant information then cluster the noun-verb pairs and phrases so that the top-ranked ones describe the most important sub-events. Through quantitative experiments on two large crisis data sets (Hurricane Harvey and the 2015 Nepal Earthquake), we demonstrate the effectiveness of our approach over the state-of-the-art. Our qualitative evaluation shows better performance compared to our baseline.
Differentiable Monte Carlo Ray Tracing through Edge Sampling
Authors: Tzu-Mao Li, Miika Aittala, Frédo Durand, Jaakko Lehtinen
Date: 2018-08-12 00:00:00
Proceeding: SIGGRAPH 2018 8
Tasks: Inverse Rendering

Gradient-based methods are becoming increasingly important for computer graphics, machine learning, and computer vision. The ability to compute gradients is crucial to optimization, inverse problems, and deep learning. In rendering, the gradient is required with respect to variables such as camera parameters, light sources, scene geometry, or material appearance. However, computing the gradient of rendering is challenging because the rendering integral includes visibility terms that are not differentiable. Previous work on differentiable rendering has focused on approximate solutions. They often do not handle secondary effects such as shadows or global illumination, or they do not provide the gradient with respect to variables other than pixel coordinates.

We introduce a general-purpose differentiable ray tracer, which, to our knowledge, is the first comprehensive solution that is able to compute derivatives of scalar functions over a rendered image with respect to arbitrary scene parameters such as camera pose, scene geometry, materials, and lighting parameters. The key to our method is a novel edge sampling algorithm that directly samples the Dirac delta functions introduced by the derivatives of the discontinuous integrand. We also develop efficient importance sampling methods based on spatial hierarchies. Our method can generate gradients in times running from seconds to minutes depending on scene complexity and desired precision.

We interface our differentiable ray tracer with the deep learning library PyTorch and show prototype applications in inverse rendering and the generation of adversarial examples for neural networks.