pwc-archive/papers-with-abstracts huggingface dataset:
DiaLex: A Benchmark for Evaluating Multidialectal Arabic Word Embeddings
Authors: Muhammad Abdul-Mageed, Shady Elbassuoni, Jad Doughman, AbdelRahim Elmadany, El Moatez Billah Nagoudi, Yorgo Zoughby, Ahmad Shaher, Iskander Gaba, Ahmed Helal, Mohammed El-Razzaz
Date: 2020-11-22 00:00:00
Proceeding: EACL (WANLP) 2021 4
Tasks: Word Embeddings

Word embeddings are a core component of modern natural language processing systems, making the ability to thoroughly evaluate them a vital task. We describe DiaLex, a benchmark for intrinsic evaluation of dialectal Arabic word embedding. DiaLex covers five important Arabic dialects: Algerian, Egyptian, Lebanese, Syrian, and Tunisian. Across these dialects, DiaLex provides a testbank for six syntactic and semantic relations, namely male to female, singular to dual, singular to plural, antonym, comparative, and genitive to past tense. DiaLex thus consists of a collection of word pairs representing each of the six relations in each of the five dialects. To demonstrate the utility of DiaLex, we use it to evaluate a set of existing and new Arabic word embeddings that we developed. Our benchmark, evaluation code, and new word embedding models will be publicly available.
FakeSafe: Human Level Data Protection by Disinformation Mapping using Cycle-consistent Adversarial Network
Authors: He Zhu, Dianbo Liu
Date: 2020-11-23 00:00:00
Tasks: Generative Adversarial Network

The concept of disinformation is to use fake messages to confuse people in order to protect the real information. This strategy can be adapted into data science to protect valuable private and sensitive data. Huge amount of private data are being generated from personal devices such as smart phone and wearable in recent years. Being able to utilize these personal data will bring big opportunities to design personalized products, conduct precision healthcare and many other tasks that were impossible in the past. However, due to privacy, safety and regulation reasons, it is often difficult to transfer or store data in its original form while keeping them safe. Building a secure data transfer and storage infrastructure to preserving privacy is costly in most cases and there is always a concern of data security due to human errors. In this study, we propose a method, named FakeSafe, to provide human level data protection using generative adversarial network with cycle consistency and conducted experiments using both benchmark and real world data sets to illustrate potential applications of FakeSafe.
Synthesis and Pruning as a Dynamic Compression Strategy for Efficient Deep Neural Networks
Authors: Alastair Finlinson, Sotiris Moschoyiannis
Date: 2020-11-23 00:00:00

The brain is a highly reconfigurable machine capable of task-specific adaptations. The brain continually rewires itself for a more optimal configuration to solve problems. We propose a novel strategic synthesis algorithm for feedforward networks that draws directly from the brain's behaviours when learning. The proposed approach analyses the network and ranks weights based on their magnitude. Unlike existing approaches that advocate random selection, we select highly performing nodes as starting points for new edges and exploit the Gaussian distribution over the weights to select corresponding endpoints. The strategy aims only to produce useful connections and result in a smaller residual network structure. The approach is complemented with pruning to further the compression. We demonstrate the techniques to deep feedforward networks. The residual sub-networks that are formed from the synthesis approaches in this work form common sub-networks with similarities up to ~90%. Using pruning as a complement to the strategic synthesis approach, we observe improvements in compression.
Semantic CPPS in Industry 4.0
Authors: Giuseppe Fenza, Mariacristina Gallo, Vincenzo Loia, Domenico Marinoand Francesco Orciuoli, Alberto Volpe
Date: 2020-11-18 00:00:00

Cyber-Physical Systems (CPS) play a crucial role in the era of the 4thIndustrial Revolution. Recently, the application of the CPS to industrial manufacturing leads to a specialization of them referred as Cyber-Physical Production Systems (CPPS). Among other challenges, CPS and CPPS should be able to address interoperability issues, since one of their intrinsic requirement is the capability to interface and cooperate with other systems. On the other hand, to fully realize theIndustry 4.0 vision, it is required to address horizontal, vertical, and end-to-end integration enabling a complete awareness through the entire supply chain. In this context, Semantic Web standards and technologies may have a promising role to represent manufacturing knowledge in a machine-interpretable way for enabling communications among heterogeneous Industrial assets. This paper proposes an integration of Semantic Web models available at state of the art for implementing a5C architecture mainly targeted to collect and process semantic data stream in a way that would unlock the potentiality of data yield in a smart manufacturing environment. The analysis of key industrial ontologies and semantic technologies allows us to instantiate an example scenario for monitoring Overall Equipment Effectiveness(OEE). The solution uses the SOSA ontology for representing the semantic datastream. Then, C-SPARQL queries are defined for periodically carrying out useful KPIs to address the proposed aim.
Consolidation via Policy Information Regularization in Deep RL for Multi-Agent Games
Authors: Tyler Malloy, Tim Klinger, Miao Liu, Matthew Riemer, Gerald Tesauro, Chris R. Sims
Date: 2020-11-23 00:00:00
Tasks: Continual Learning, continuous-control, Continuous Control, reinforcement-learning, Reinforcement Learning (RL)

This paper introduces an information-theoretic constraint on learned policy complexity in the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) reinforcement learning algorithm. Previous research with a related approach in continuous control experiments suggests that this method favors learning policies that are more robust to changing environment dynamics. The multi-agent game setting naturally requires this type of robustness, as other agents' policies change throughout learning, introducing a nonstationary environment. For this reason, recent methods in continual learning are compared to our approach, termed Capacity-Limited MADDPG. Results from experimentation in multi-agent cooperative and competitive tasks demonstrate that the capacity-limited approach is a good candidate for improving learning performance in these environments.
APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding
Authors: Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia, Qi Yang, Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang, Bowen Sun, Zhenyu Guo
Date: 2020-11-23 00:00:00
Tasks: Fraud Detection, Graph Embedding

Limited by the time complexity of querying k-hop neighbors in a graph database, most graph algorithms cannot be deployed online and execute millisecond-level inference. This problem dramatically limits the potential of applying graph algorithms in certain areas, such as financial fraud detection. Therefore, we propose Asynchronous Propagation Attention Network, an asynchronous continuous time dynamic graph algorithm for real-time temporal graph embedding. Traditional graph models usually execute two serial operations: first graph computation and then model inference. We decouple model inference and graph computation step so that the heavy graph query operations will not damage the speed of model inference. Extensive experiments demonstrate that the proposed method can achieve competitive performance and 8.7 times inference speed improvement in the meantime.
Human computation requires and enables a new approach to ethical review
Authors: Libuše Hannah Vepřek, Patricia Seymour, Pietro Michelucci
Date: 2020-11-21 00:00:00
Tasks: Ethics

With humans increasingly serving as computational elements in distributed information processing systems and in consideration of the profit-driven motives and potential inequities that might accompany the emerging thinking economy[1], we recognize the need for establishing a set of related ethics to ensure the fair treatment and wellbeing of online cognitive laborers and the conscientious use of the capabilities to which they contribute. Toward this end, we first describe human-in-the-loop computing in context of the new concerns it raises that are not addressed by traditional ethical research standards. We then describe shortcomings in the traditional approach to ethical review and introduce a dynamic approach for sustaining an ethical framework that can continue to evolve within the rapidly shifting context of disruptive new technologies.
Deep-learning based down-scaling of summer monsoon rainfall data over Indian region
Authors: Bipin Kumar, Rajib Chattopadhyay, Manmeet Singh, Niraj Chaudhari, Karthik Kodari, Amit Barve
Date: 2020-11-23 00:00:00
Tasks: Super-Resolution

Downscaling is necessary to generate high-resolution observation data to validate the climate model forecast or monitor rainfall at the micro-regional level operationally. Dynamical and statistical downscaling models are often used to get information at high-resolution gridded data over larger domains. As rainfall variability is dependent on the complex Spatio-temporal process leading to non-linear or chaotic Spatio-temporal variations, no single downscaling method can be considered efficient enough. In data with complex topographies, quasi-periodicities, and non-linearities, deep Learning (DL) based methods provide an efficient solution in downscaling rainfall data for regional climate forecasting and real-time rainfall observation data at high spatial resolutions. In this work, we employed three deep learning-based algorithms derived from the super-resolution convolutional neural network (SRCNN) methods, to precipitation data, in particular, IMD and TRMM data to produce 4x-times high-resolution downscaled rainfall data during the summer monsoon season. Among the three algorithms, namely SRCNN, stacked SRCNN, and DeepSD, employed here, the best spatial distribution of rainfall amplitude and minimum root-mean-square error is produced by DeepSD based downscaling. Hence, the use of the DeepSD algorithm is advocated for future use. We found that spatial discontinuity in amplitude and intensity rainfall patterns is the main obstacle in the downscaling of precipitation. Furthermore, we applied these methods for model data postprocessing, in particular, ERA5 data. Downscaled ERA5 rainfall data show a much better distribution of spatial covariance and temporal variance when compared with observation.
The Dynamic of Body and Brain Co-Evolution
Authors: Paolo Pagliuca, Stefano Nolfi
Date: 2020-11-23 00:00:00

We introduce a method that permits to co-evolve the body and the control properties of robots. It can be used to adapt the morphological traits of robots with a hand-designed morphological bauplan or to evolve the morphological bauplan as well. Our results indicate that robots with co-adapted body and control traits outperform robots with fixed hand-designed morphologies. Interestingly, the advantage is not due to the selection of better morphologies but rather to the mutual scaffolding process that results from the possibility to co-adapt the morphological traits to the control traits and vice versa. Our results also demonstrate that morphological variations do not necessarily have destructive effects on robot skills.
OAK: Ontology-Based Knowledge Map Model for Digital Agriculture
Authors: Quoc Hung Ngo, Tahar Kechadi, Nhien-An Le-Khac
Date: 2020-11-20 00:00:00

Nowadays, a huge amount of knowledge has been amassed in digital agriculture. This knowledge and know-how information are collected from various sources, hence the question is how to organise this knowledge so that it can be efficiently exploited. Although this knowledge about agriculture practices can be represented using ontology, rule-based expert systems, or knowledge model built from data mining processes, the scalability still remains an open issue. In this study, we propose a knowledge representation model, called an ontology-based knowledge map, which can collect knowledge from different sources, store it, and exploit either directly by stakeholders or as an input to the knowledge discovery process (Data Mining). The proposed model consists of two stages, 1) build an ontology as a knowledge base for a specific domain and data mining concepts, and 2) build the ontology-based knowledge map model for representing and storing the knowledge mined on the crop datasets. A framework of the proposed model has been implemented in agriculture domain. It is an efficient and scalable model, and it can be used as knowledge repository a digital agriculture.
Reachable Polyhedral Marching (RPM): A Safety Verification Algorithm for Robotic Systems with Deep Neural Network Components
Authors: Joseph A. Vincent, Mac Schwager
Date: 2020-11-23 00:00:00

We present a method for computing exact reachable sets for deep neural networks with rectified linear unit (ReLU) activation. Our method is well-suited for use in rigorous safety analysis of robotic perception and control systems with deep neural network components. Our algorithm can compute both forward and backward reachable sets for a ReLU network iterated over multiple time steps, as would be found in a perception-action loop in a robotic system. Our algorithm is unique in that it builds the reachable sets by incrementally enumerating polyhedral cells in the input space, rather than iterating layer-by-layer through the network as in other methods. If an unsafe cell is found, our algorithm can return this result without completing the full reachability computation, thus giving an anytime property that accelerates safety verification. In addition, our method requires less memory during execution compared to existing methods where memory can be a limiting factor. We demonstrate our algorithm on safety verification of the ACAS Xu aircraft advisory system. We find unsafe actions many times faster than the fastest existing method and certify no unsafe actions exist in about twice the time of the existing method. We also compute forward and backward reachable sets for a learned model of pendulum dynamics over a 50 time step horizon in 87s on a laptop computer. Algorithm source code: https://github.com/StanfordMSL/Neural-Network-Reach.
Toward a Next Generation Particle Precipitation Model: Mesoscale Prediction Through Machine Learning (a Case Study and Framework for Progress)
Authors: Ryan M. McGranaghan, Jack Ziegler, Téo Bloch, Spencer Hatch, Enrico Camporeale, Kristina Lynch, Mathew Owens, Jesper Gjerloev, Binzheng Zhang, Susan Skone
Date: 2020-11-19 00:00:00
Tasks: BIG-bench Machine Learning

We advance the modeling capability of electron particle precipitation from the magnetosphere to the ionosphere through a new database and use of machine learning (ML) tools to gain utility from those data. We have compiled, curated, analyzed, and made available a new and more capable database of particle precipitation data that includes 51 satellite years of Defense Meteorological Satellite Program (DMSP) observations temporally aligned with solar wind and geomagnetic activity data. The new total electron energy flux particle precipitation nowcast model, a neural network called PrecipNet, takes advantage of increased expressive power afforded by ML approaches to appropriately utilize diverse information from the solar wind and geomagnetic activity and, importantly, their time histories. With a more capable representation of the organizing parameters and the target electron energy flux observations, PrecipNet achieves a >50% reduction in errors from a current state-of-the-art model oval variation, assessment, tracking, intensity, and online nowcasting (OVATION Prime), better captures the dynamic changes of the auroral flux, and provides evidence that it can capably reconstruct mesoscale phenomena. We create and apply a new framework for space weather model evaluation that culminates previous guidance from across the solar-terrestrial research community. The research approach and results are representative of the "new frontier" of space weather research at the intersection of traditional and data science-driven discovery and provides a foundation for future efforts.
Autonomous learning of nonlocal stochastic neuron dynamics
Authors: Tyler E. Maltba, Hongli Zhao, Daniel M. Tartakovsky
Date: 2020-11-22 00:00:00

Neuronal dynamics is driven by externally imposed or internally generated random excitations/noise, and is often described by systems of random or stochastic ordinary differential equations. Such systems admit a distribution of solutions, which is (partially) characterized by the single-time joint probability density function (PDF) of system states. It can be used to calculate such information-theoretic quantities as the mutual information between the stochastic stimulus and various internal states of the neuron (e.g., membrane potential), as well as various spiking statistics. When random excitations are modeled as Gaussian white noise, the joint PDF of neuron states satisfies exactly a Fokker-Planck equation. However, most biologically plausible noise sources are correlated (colored). In this case, the resulting PDF equations require a closure approximation. We propose two methods for closing such equations: a modified nonlocal large-eddy-diffusivity closure and a data-driven closure relying on sparse regression to learn relevant features. The closures are tested for the stochastic non-spiking leaky integrate-and-fire and FitzHugh-Nagumo (FHN) neurons driven by sine-Wiener noise. Mutual information and total correlation between the random stimulus and the internal states of the neuron are calculated for the FHN neuron.
Gonogo: An R Implementation of Test Methods to Perform, Analyze and Simulate Sensitivity Experiments
Authors: Paul A. Roediger
Date: 2020-11-23 00:00:00
Tasks: Sensitivity

This work provides documentation for a suite of R functions contained in gonogo.R. The functions provide sensitivity testing practitioners and researchers with an ability to conduct, analyze and simulate various sensitivity experiments involving binary responses and a single stimulus level (e.g., drug dosage, drop height, velocity, etc.). Included are the modern Neyer and 3pod adaptive procedures, as well as the Bruceton and Langlie. The latter two benchmark procedures are capable of being performed according to generalized up-down transformed-response rules. Each procedure is designated phase-one of a three-phase experiment. The goal of phase-one is to achieve overlapping data. The two additional (and optional) refinement phases utilize the D-optimal criteria and the Robbins-Monro-Joseph procedure. The goals of the two refinement phases are to situate testing in the vicinity of the median and tails of the latent response distribution, respectively.
Concentration inequality for U-statistics of order two for uniformly ergodic Markov chains
Authors: Quentin Duchemin, Yohann de Castro, Claire Lacour
Date: 2020-11-20 00:00:00
Tasks: Blocking

We prove a new concentration inequality for U-statistics of order two for uniformly ergodic Markov chains. Working with bounded and $\pi$-canonical kernels, we show that we can recover the convergence rate of Arcones and Gin{\'e} who proved a concentration result for U-statistics of independent random variables and canonical kernels. Our result allows for a dependence of the kernels $h_{i,j}$ with the indexes in the sums, which prevents the use of standard blocking tools. Our proof relies on an inductive analysis where we use martingale techniques, uniform ergodicity, Nummelin splitting and Bernstein's type inequality. Assuming further that the Markov chain starts from its invariant distribution, we prove a Bernstein-type concentration inequality that provides sharper convergence rate for small variance terms.
On Random Matrices Arising in Deep Neural Networks: General I.I.D. Case
Authors: L. Pastur, V. Slavin
Date: 2020-11-20 00:00:00

We study the distribution of singular values of product of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the population covariance matrices assumed to be non-random or random but independent of the random data matrix in statistics and random matrix theory are now certain functions of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [25, 13] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability has to be justified. The justification has been given in [22] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper we use another, more streamlined, version of the techniques of random matrix theory to generalize the results of [22] to the case where the entries of the synaptic weight matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, extends the property of the so-called macroscopic universality on the considered random matrices.
Conditional canonical correlation estimation based on covariates with random forests
Authors: Cansu Alakus, Denis Larocque, Sebastien Jacquemont, Fanny Barlaam, Charles-Olivier Martin, Kristian Agbogba, Sarah Lippe, Aurelie Labbe
Date: 2020-11-23 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Investigating the relationships between two sets of variables helps to understand their interactions and can be done with canonical correlation analysis (CCA). However, the correlation between the two sets can sometimes depend on a third set of covariates, often subject-related ones such as age, gender, or other clinical measures. In this case, applying CCA to the whole population is not optimal and methods to estimate conditional CCA, given the covariates, can be useful. We propose a new method called Random Forest with Canonical Correlation Analysis (RFCCA) to estimate the conditional canonical correlations between two sets of variables given subject-related covariates. The individual trees in the forest are built with a splitting rule specifically designed to partition the data to maximize the canonical correlation heterogeneity between child nodes. We also propose a significance test to detect the global effect of the covariates on the relationship between two sets of variables. The performance of the proposed method and the global significance test is evaluated through simulation studies that show it provides accurate canonical correlation estimations and well-controlled Type-1 error. We also show an application of the proposed method with EEG data.
Tolerance and Prediction Intervals for Non-normal Models
Authors: Geoffrey S Johnson
Date: 2020-11-23 00:00:00
Tasks: Prediction, Prediction Intervals

A prediction interval covers a future observation from a random process in repeated sampling, and is typically constructed by identifying a pivotal quantity that is also an ancillary statistic. Analogously, a tolerance interval covers a population percentile in repeated sampling and is often based on a pivotal quantity. One approach we consider in non-normal models leverages a link function resulting in a pivotal quantity that is approximately normally distributed. In settings where this normal approximation does not hold we consider a second approach for tolerance and prediction based on a confidence interval for the mean. These methods are intuitive, simple to implement, have proper operating characteristics, and are computationally efficient compared to Bayesian, re-sampling, and machine learning methods. This is demonstrated in the context of multi-site clinical trial recruitment with staggered site initiation, real-world time on treatment, and end-of-study success for a clinical endpoint.