OpenRaiser/Intern-Atlas huggingface dataset:
System Identification via Meta-Learning in Linear Time-Varying Environments
Venue: arXiv cs.LG 2020
Year: 2020

System identification is a fundamental problem in reinforcement learning, control theory and signal processing, and the non-asymptotic analysis of the corresponding sample complexity is challenging and elusive, even for linear time-varying (LTV) systems. To tackle this challenge, we develop an episodic block model for the LTV system where the model parameters remain constant within each block but change from block to block. Based on the observation that the model parameters across different blocks are related, we treat each episodic block as a learning task and then run meta-learning over many blocks for system identification, using two steps, namely offline meta-learning and online adaptation. We carry out a comprehensive non-asymptotic analysis of the performance of meta-learning based system identification. To deal with the technical challenges rooted in the sample correlation and small sample sizes in each block, we devise a new two-scale martingale small-ball approach for offline meta-learning, for arbitrary model correlation structure across blocks. We then quantify the finite time error of online adaptation by leveraging recent advances in linear stochastic approximation with correlated samples.
Joint Path and Push Planning Among Movable Obstacles
Venue: arXiv cs.RO 2020
Year: 2020

This paper explores the Navigation Among Movable Obstacles (NAMO) problem and proposes joint path and push planning: which path to take and in what direction the obstacles should be pushed at, given a start and goal position. We present a planning algorithm for selecting a path and the obstacles to be pushed, where a Rapidly-exploring Random Tree (RRT)-based heuristic is employed to calculate a minimal collision path. When it is necessary to apply a pushing force to slide an obstacle out of the way, the planners leverage means-end analysis through a dynamic physics simulation to determine the sequence of linear pushes to clear the necessary space. Simulation experiments show that our approach finds solutions in higher clutter percentages (up to 49%) compared to the straight-line push planner (37%) and RRT without pushing (18%).
The Volctrans Machine Translation System for WMT20
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes our VolcTrans system on WMT20 shared news translation task. We participated in 8 translation directions. Our basic systems are based on Transformer, with several variants (wider or deeper Transformers, dynamic convolutions). The final system includes text pre-process, data selection, synthetic data generation, advanced model ensemble, and multilingual pre-training.
Approximate Evaluation of First-Order Counting Queries
Venue: arXiv cs.LO 2020
Year: 2020

Kuske and Schweikardt introduced the very expressive first-order counting logic FOC(P) to model database queries with counting operations. They showed that there is an efficient model-checking algorithm on graphs with bounded degree, while Grohe and Schweikardt showed that probably no such algorithm exists for trees of bounded depth. We analyze the fragment FO({>0}) of this logic. While we remove for example subtraction and comparison between two non-atomic counting terms, this logic remains quite expressive: We allow nested counting and comparison between counting terms and arbitrarily large numbers. Our main result is an approximation scheme of the model-checking problem for FO({>0}) that runs in linear fpt time on structures with bounded expansion. This scheme either gives the correct answer or says "I do not know." The latter answer may only be given if small perturbations in the number-symbols of the formula could make it both satisfied and unsatisfied. This is complemented by showing that exactly solving the model-checking problem for FO({>0}) is already hard on trees of bounded depth and just slightly increasing the expressiveness of FO({>0}) makes even approximation hard on trees.
Higher Order Linear Transformer
Venue: arXiv cs.LG 2020
Year: 2020

Following up on the linear transformer part of the article from Katharopoulos et al., that takes this idea from Shen et al., the trick that produces a linear complexity for the attention mechanism is re-used and extended to a second-order approximation of the softmax normalization.
ePython: An implementation of Python for the many-core Epiphany coprocessor
Venue: arXiv cs.DC 2020
Year: 2020

The Epiphany is a many-core, low power, low on-chip memory architecture and one can very cheaply gain access to a number of parallel cores which is beneficial for HPC education and prototyping. The very low power nature of these architectures also means that there is potential for their use in future HPC machines, however there is a high barrier to entry in programming them due to the associated complexities and immaturity of supporting tools. In this paper we present our work on ePython, a subset of Python for the Epiphany and similar many-core co-processors. Due to the limited on-chip memory per core we have developed a new Python interpreter and this, combined with additional support for parallelism, has meant that novices can take advantage of Python to very quickly write parallel codes on the Epiphany and explore concepts of HPC using a smaller scale parallel machine. The high level nature of Python opens up new possibilities on the Epiphany, we examine a computationally intensive Gauss-Seidel code from the programmability and performance perspective, discuss running Python hybrid on both the host CPU and Epiphany, and interoperability between a full Python interpreter on the CPU and ePython on the Epiphany. The result of this work is support for developing Python on the Epiphany, which can be applied to other similar architectures, that the community have already started to adopt and use to explore concepts of parallelism and HPC.
Flexible retrieval with NMSLIB and FlexNeuART
Venue: arXiv cs.IR 2020
Year: 2020

Our objective is to introduce to the NLP community an existing k-NN search library NMSLIB, a new retrieval toolkit FlexNeuART, as well as their integration capabilities. NMSLIB, while being one the fastest k-NN search libraries, is quite generic and supports a variety of distance/similarity functions. Because the library relies on the distance-based structure-agnostic algorithms, it can be further extended by adding new distances. FlexNeuART is a modular, extendible and flexible toolkit for candidate generation in IR and QA applications, which supports mixing of classic and neural ranking signals. FlexNeuART can efficiently retrieve mixed dense and sparse representations (with weights learned from training data), which is achieved by extending NMSLIB. In that, other retrieval systems work with purely sparse representations (e.g., Lucene), purely dense representations (e.g., FAISS and Annoy), or only perform mixing at the re-ranking stage.
Comparison Analysis of Tree Based and Ensembled Regression Algorithms for Traffic Accident Severity Prediction
Venue: arXiv cs.OH 2020
Year: 2020

Rapid increase of traffic volume on urban roads over time has changed the traffic scenario globally. It has also increased the ratio of road accidents that can be severe and fatal in the worst case. To improve traffic safety and its management on urban roads, there is a need for prediction of severity level of accidents. Various machine learning models are being used for accident prediction. In this study, tree based ensemble models (Random Forest, AdaBoost, Extra Tree, and Gradient Boosting) and ensemble of two statistical models (Logistic Regression Stochastic Gradient Descent) as voting classifiers are compared for prediction of road accident severity. Significant features that are strongly correlated with the accident severity are identified by Random Forest. Analysis proved Random Forest as the best performing model with highest classification results with 0.974 accuracy, 0.954 precision, 0.930 recall and 0.942 F-score using 20 most significant features as compared to other techniques classification of road accidents severity.
Predicting Engagement with the Internet Research Agency's Facebook and Instagram Campaigns around the 2016 U.S. Presidential Election
Venue: arXiv cs.SI 2020
Year: 2020

The Russian Internet Research Agency's (IRA) online interference campaign in the 2016 U.S. presidential election represents a turning point in the trajectory of democratic elections in the digital age. What can we learn about how the IRA engages U.S. audiences, ahead of the 2020 U.S. presidential election? We provide the first in-depth analysis of the relationships between IRA content characteristics and user engagement on Facebook and Instagram around the 2016 election. We find that content targeting right-wing and non-Black marginalised groups had the strongest positive association with engagement on both Facebook and Instagram, in contrast to findings from the IRA campaign on Twitter and to some previous commentary in the media. Higher engagement was associated with posting later in the 2015-2017 period and using less text on both platforms, using negative wording and not including links on Facebook, and using fewer hashtags on Instagram. The sub-audiences and sub-issues associated with most engagement differed across the platforms.
Dimensionality Reduction and Anomaly Detection for CPPS Data using Autoencoder
Venue: arXiv cs.LG 2020
Year: 2020

Unsupervised anomaly detection (AD) is a major topic in the field of Cyber-Physical Production Systems (CPPSs). A closely related concern is dimensionality reduction (DR) which is: 1) often used as a preprocessing step in an AD solution, 2) a sort of AD, if a measure of observation conformity to the learned data manifold is provided. We argue that the two aspects can be complementary in a CPPS anomaly detection solution. In this work, we focus on the nonlinear autoencoder (AE) as a DR/AD approach. The contribution of this work is: 1) we examine the suitability of AE reconstruction error as an AD decision criterion in CPPS data. 2) we analyze its relation to a potential second-phase AD approach in the AE latent space 3) we evaluate the performance of the approach on three real-world datasets. Moreover, the approach outperforms state-of-the-art techniques, alongside a relatively simple and straightforward application.
Estimating Multiplicative Relations in Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Universal approximation theorem suggests that a shallow neural network can approximate any function. The input to neurons at each layer is a weighted sum of previous layer neurons and then an activation is applied. These activation functions perform very well when the output is a linear combination of input data. When trying to learn a function which involves product of input data, the neural networks tend to overfit the data to approximate the function. In this paper we will use properties of logarithmic functions to propose a pair of activation functions which can translate products into linear expression and learn using backpropagation. We will try to generalize this approach for some complex arithmetic functions and test the accuracy on a disjoint distribution with the training set.
Forecasting Hamiltonian dynamics without canonical coordinates
Venue: arXiv cs.LG 2020
Year: 2020

Conventional neural networks are universal function approximators, but because they are unaware of underlying symmetries or physical laws, they may need impractically many training data to approximate nonlinear dynamics. Recently introduced Hamiltonian neural networks can efficiently learn and forecast dynamical systems that conserve energy, but they require special inputs called canonical coordinates, which may be hard to infer from data. Here we significantly expand the scope of such networks by demonstrating a simple way to train them with any set of generalised coordinates, including easily observable ones.
Specification description and verification of multitask hybrid systems in the OTS/CafeOBJ method
Venue: arXiv cs.SE 2020
Year: 2020

To develop IoT and/or CSP systems, we need consider both continuous data from physical world and discrete data in computer systems. Such a system is called a hybrid system. Because of density of continuous data, it is not easy to do software testing to ensure reliability of hybrid systems. Moreover, the size of the state space increases exponentially for multitask systems. Formal descriptions of hybrid systems may help us to verify desired properties of a given system formally with computer supports. In this paper, we propose a way to describe a formal specification of a given multitask hybrid system as an observational transition system in CafeOBJ algebraic specification language and verify it by the proof score method based on equational reasoning implemented in CafeOBJ interpreter.
"where is this relationship going?": Understanding Relationship Trajectories in Narrative Text
Venue: arXiv cs.CL 2020
Year: 2020

We examine a new commonsense reasoning task: given a narrative describing a social interaction that centers on two protagonists, systems make inferences about the underlying relationship trajectory. Specifically, we propose two evaluation tasks: Relationship Outlook Prediction MCQ and Resolution Prediction MCQ. In Relationship Outlook Prediction, a system maps an interaction to a relationship outlook that captures how the interaction is expected to change the relationship. In Resolution Prediction, a system attributes a given relationship outlook to a particular resolution that explains the outcome. These two tasks parallel two real-life questions that people frequently ponder upon as they navigate different social situations: "where is this relationship going?" and "how did we end up here?". To facilitate the investigation of human social relationships through these two tasks, we construct a new dataset, Social Narrative Tree, which consists of 1250 stories documenting a variety of daily social interactions. The narratives encode a multitude of social elements that interweave to give rise to rich commonsense knowledge of how relationships evolve with respect to social interactions. We establish baseline performances using language models and the accuracies are significantly lower than human performance. The results demonstrate that models need to look beyond syntactic and semantic signals to comprehend complex human relationships.
Learning Centric Wireless Resource Allocation for Edge Computing: Algorithm and Experiment
Venue: arXiv cs.IT 2020
Year: 2020

Edge intelligence is an emerging network architecture that integrates sensing, communication, computing components, and supports various machine learning applications, where a fundamental communication question is: how to allocate the limited wireless resources (such as time, energy) to the simultaneous model training of heterogeneous learning tasks? Existing methods ignore two important facts: 1) different models have heterogeneous demands on training data; 2) there is a mismatch between the simulated environment and the real-world environment. As a result, they could lead to low learning performance in practice. This paper proposes the learning centric wireless resource allocation (LCWRA) scheme that maximizes the worst learning performance of multiple tasks. Analysis shows that the optimal transmission time has an inverse power relationship with respect to the generalization error. Finally, both simulation and experimental results are provided to verify the performance of the proposed LCWRA scheme and its robustness in real implementation.
Learning Audio Embeddings with User Listening Data for Content-based Music Recommendation
Venue: arXiv cs.SD 2020
Year: 2020

Personalized recommendation on new track releases has always been a challenging problem in the music industry. To combat this problem, we first explore user listening history and demographics to construct a user embedding representing the user's music preference. With the user embedding and audio data from user's liked and disliked tracks, an audio embedding can be obtained for each track using metric learning with Siamese networks. For a new track, we can decide the best group of users to recommend by computing the similarity between the track's audio embedding and different user embeddings, respectively. The proposed system yields state-of-the-art performance on content-based music recommendation tested with millions of users and tracks. Also, we extract audio embeddings as features for music genre classification tasks. The results show the generalization ability of our audio embeddings.
FiGLearn: Filter and Graph Learning using Optimal Transport
Venue: arXiv cs.LG 2020
Year: 2020

In many applications, a dataset can be considered as a set of observed signals that live on an unknown underlying graph structure. Some of these signals may be seen as white noise that has been filtered on the graph topology by a graph filter. Hence, the knowledge of the filter and the graph provides valuable information about the underlying data generation process and the complex interactions that arise in the dataset. We hence introduce a novel graph signal processing framework for jointly learning the graph and its generating filter from signal observations. We cast a new optimisation problem that minimises the Wasserstein distance between the distribution of the signal observations and the filtered signal distribution model. Our proposed method outperforms state-of-the-art graph learning frameworks on synthetic data. We then apply our method to a temperature anomaly dataset, and further show how this framework can be used to infer missing values if only very little information is available.
Asynchronous Corner Tracking Algorithm based on Lifetime of Events for DAVIS Cameras
Venue: arXiv cs.CV 2020
Year: 2020

Event cameras, i.e., the Dynamic and Active-pixel Vision Sensor (DAVIS) ones, capture the intensity changes in the scene and generates a stream of events in an asynchronous fashion. The output rate of such cameras can reach up to 10 million events per second in high dynamic environments. DAVIS cameras use novel vision sensors that mimic human eyes. Their attractive attributes, such as high output rate, High Dynamic Range (HDR), and high pixel bandwidth, make them an ideal solution for applications that require high-frequency tracking. Moreover, applications that operate in challenging lighting scenarios can exploit the high HDR of event cameras, i.e., 140 dB compared to 60 dB of traditional cameras. In this paper, a novel asynchronous corner tracking method is proposed that uses both events and intensity images captured by a DAVIS camera. The Harris algorithm is used to extract features, i.e., frame-corners from keyframes, i.e., intensity images. Afterward, a matching algorithm is used to extract event-corners from the stream of events. Events are solely used to perform asynchronous tracking until the next keyframe is captured. Neighboring events, within a window size of 5x5 pixels around the event-corner, are used to calculate the velocity and direction of extracted event-corners by fitting the 2D planar using a randomized Hough transform algorithm. Experimental evaluation showed that our approach is able to update the location of the extracted corners up to 100 times during the blind time of traditional cameras, i.e., between two consecutive intensity images.
How do Offline Measures for Exploration in Reinforcement Learning behave?
Venue: arXiv cs.LG 2020
Year: 2020

Sufficient exploration is paramount for the success of a reinforcement learning agent. Yet, exploration is rarely assessed in an algorithm-independent way. We compare the behavior of three data-based, offline exploration metrics described in the literature on intuitive simple distributions and highlight problems to be aware of when using them. We propose a fourth metric,uniform relative entropy, and implement it using either a k-nearest-neighbor or a nearest-neighbor-ratio estimator, highlighting that the implementation choices have a profound impact on these measures.
Modulation Pattern Detection Using Complex Convolutions in Deep Learning
Venue: arXiv cs.LG 2020
Year: 2020

Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel impairments affect the signals in complicated ways such that the received signal bears little resemblance to the transmitted signal. Although deep learning approaches have shown great promise over statistical methods in this problem space, deep learning frameworks continue to lag in support for complex-valued data. To address this gap, we study the implementation and use of complex convolutions in a series of convolutional neural network architectures. Replacement of data structure and convolution operations by their complex generalization in an architecture improves performance, with statistical significance, at recognizing modulation patterns in complex-valued signals with high SNR after being trained on low SNR signals. This suggests complex-valued convolutions enables networks to learn more meaningful representations. We investigate this hypothesis by comparing the features learned in each experiment by visualizing the inputs that results in one-hot modulation pattern classification for each network.
Improving Accuracy of Federated Learning in Non-IID Settings
Venue: arXiv cs.LG 2020
Year: 2020

Federated Learning (FL) is a decentralized machine learning protocol that allows a set of participating agents to collaboratively train a model without sharing their data. This makes FL particularly suitable for settings where data privacy is desired. However, it has been observed that the performance of FL is closely tied with the local data distributions of agents. Particularly, in settings where local data distributions vastly differ among agents, FL performs rather poorly with respect to the centralized training. To address this problem, we hypothesize the reasons behind the performance degradation, and develop some techniques to address these reasons accordingly. In this work, we identify four simple techniques that can improve the performance of trained models without incurring any additional communication overhead to FL, but rather, some light computation overhead either on the client, or the server-side. In our experimental analysis, combination of our techniques improved the validation accuracy of a model trained via FL by more than 12% with respect to our baseline. This is about 5% less than the accuracy of the model trained on centralized data.
Impact of (SARS-CoV-2) COVID 19 on the indigenous language-speaking population in Mexico
Venue: arXiv cs.CY 2020
Year: 2020

The importance of the working document is that it allows the analysis of the information and the status of cases associated with (SARS-CoV-2) COVID-19 as open data at the municipal, state and national level, with a daily record of patients, according to a age, sex, comorbidities, for the condition of (SARS-CoV-2) COVID-19 according to the following characteristics: a) Positive, b) Negative, c) Suspicious. Likewise, it presents information related to the identification of an outpatient and / or hospitalized patient, attending to their medical development, identifying: a) Recovered, b) Deaths and c) Active, in Phase 3 and Phase 4, in the five main population areas speaker of indigenous language in the State of Veracruz - Mexico. The data analysis is carried out through the application of a data mining algorithm, which provides the information, fast and timely, required for the estimation of Medical Care Scenarios of (SARS-CoV-2) COVID-19, as well as for know the impact on the indigenous language-speaking population in Mexico.