OpenRaiser/Intern-Atlas huggingface dataset:
A Survey on Sampling and Profiling over Big Data (Technical Report)
Venue: arXiv cs.DB 2020
Year: 2020

Due to the development of internet technology and computer science, data is exploding at an exponential rate. Big data brings us new opportunities and challenges. On the one hand, we can analyze and mine big data to discover hidden information and get more potential value. On the other hand, the 5V characteristic of big data, especially Volume which means large amount of data, brings challenges to storage and processing. For some traditional data mining algorithms, machine learning algorithms and data profiling tasks, it is very difficult to handle such a large amount of data. The large amount of data is highly demanding hardware resources and time consuming. Sampling methods can effectively reduce the amount of data and help speed up data processing. Hence, sampling technology has been widely studied and used in big data context, e.g., methods for determining sample size, combining sampling with big data processing frameworks. Data profiling is the activity that finds metadata of data set and has many use cases, e.g., performing data profiling tasks on relational data, graph data, and time series data for anomaly detection and data repair. However, data profiling is computationally expensive, especially for large data sets. Therefore, this paper focuses on researching sampling and profiling in big data context and investigates the application of sampling in different categories of data profiling tasks. From the experimental results of these studies, the results got from the sampled data are close to or even exceed the results of the full amount of data. Therefore, sampling technology plays an important role in the era of big data, and we also have reason to believe that sampling technology will become an indispensable step in big data processing in the future.
Continual Learning Using Multi-view Task Conditional Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning change, a continual model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. The changes in goals or data are referred to as new tasks in a continual learning model. Most of the continual learning methods have a task-known setup in which the task identities are known in advance to the learning model. We propose Multi-view Task Conditional Neural Networks (Mv-TCNN) that does not require to known the reoccurring tasks in advance. We evaluate our model on standard datasets using MNIST, CIFAR10, CIFAR100, and also a real-world dataset that we have collected in a remote healthcare monitoring study (i.e. TIHM dataset). The proposed model outperforms the state-of-the-art solutions in continual learning and adapting to new tasks that are not defined in advance.
A Federated Learning Framework for Healthcare IoT devices
Venue: arXiv cs.LG 2020
Year: 2020

The Internet of Things (IoT) revolution has shown potential to give rise to many medical applications with access to large volumes of healthcare data collected by IoT devices. However, the increasing demand for healthcare data privacy and security makes each IoT device an isolated island of data. Further, the limited computation and communication capacity of wearable healthcare devices restrict the application of vanilla federated learning. To this end, we propose an advanced federated learning framework to train deep neural networks, where the network is partitioned and allocated to IoT devices and a centralized server. Then most of the training computation is handled by the powerful server. The sparsification of activations and gradients significantly reduces the communication overhead. Empirical study have suggested that the proposed framework guarantees a low accuracy loss, while only requiring 0.2% of the synchronization traffic in vanilla federated learning.
An Effective Dynamic Spatio-temporal Framework with Multi-Source Information for Traffic Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Traffic prediction is necessary not only for management departments to dispatch vehicles but also for drivers to avoid congested roads. Many traffic forecasting methods based on deep learning have been proposed in recent years, and their main aim is to solve the problem of spatial dependencies and temporal dynamics. In this paper, we propose a useful dynamic model to predict the urban traffic volume by combining fully bidirectional LSTM, the more complex attention mechanism, and the external features, including weather conditions and events. First, we adopt the bidirectional LSTM to obtain temporal dependencies of traffic volume dynamically in each layer, which is different from the hybrid methods combining bidirectional and unidirectional ones; second, we use a more elaborate attention mechanism to learn short-term and long-term periodic temporal dependencies; and finally, we collect the weather conditions and events as the external features to further improve the prediction precision. The experimental results show that the proposed model improves the prediction precision by approximately 3-7 percent on the NYC-Taxi and NYC-Bike datasets compared to the most recently developed method, being a useful tool for the urban traffic prediction.
New Ideas for Brain Modelling 6
Venue: arXiv cs.AI 2020
Year: 2020

This paper describes implementation details for a 3-level cognitive model, described in the paper series. The whole architecture is now modular, with different levels using different types of information. The ensemble-hierarchy relationship is maintained and placed in the bottom optimising and middle aggregating levels, to store memory objects and their relations. The top-level cognitive layer has been re-designed to model the Cognitive Process Language (CPL) of an earlier paper, by refactoring it into a network structure with a light scheduler. The cortex brain region is thought to be hierarchical - clustering from simple to more complex features. The refactored network might therefore challenge conventional thinking on that brain region. It is also argued that the function and structure in particular, of the new top level, is similar to the psychology theory of chunking. The model is still only a framework and does not have enough information for real intelligence. But a framework is now implemented over the whole design and so can give a more complete picture about the potential for results.
Optimal Throughput-Outage Analysis of Cache-Aided Wireless Multi-Hop D2D Networks -- Derivations of Scaling Laws
Venue: arXiv cs.IT 2020
Year: 2020

Cache-aided wireless device-to-device (D2D) networks have demonstrated promising performance improvement for video distribution compared to conventional distribution methods. Understanding the fundamental scaling behavior of such networks is thus of paramount importance. However, existing scaling laws for multi-hop networks have not been found to be optimal even for the case of Zipf popularity distributions (gaps between upper and lower bounds are not constants); furthermore, there are no scaling law results for such networks for the more practical case of a Mandelbrot-Zipf (MZipf) popularity distribution. We thus in this work investigate the throughput-outage performance for cache-aided wireless D2D networks adopting multi-hop communications, with the MZipf popularity distribution for file requests and users distributed according to Poisson point process. We propose an achievable content caching and delivery scheme and analyze its performance. By showing that the achievable performance is tight to the proposed outer bound, the optimal scaling law is obtained. Furthermore, since the Zipf distribution is a special case of the MZipf distribution, the optimal scaling law for the networks considering Zipf popularity distribution is also obtained, which closes the gap in the literature.
On the Transferability of Winning Tickets in Non-Natural Image Datasets
Venue: arXiv cs.CV 2020
Year: 2020

We study the generalization properties of pruned neural networks that are the winners of the lottery ticket hypothesis on datasets of natural images. We analyse their potential under conditions in which training data is scarce and comes from a non-natural domain. Specifically, we investigate whether pruned models that are found on the popular CIFAR-10/100 and Fashion-MNIST datasets, generalize to seven different datasets that come from the fields of digital pathology and digital heritage. Our results show that there are significant benefits in transferring and training sparse architectures over larger parametrized models, since in all of our experiments pruned networks, winners of the lottery ticket hypothesis, significantly outperform their larger unpruned counterparts. These results suggest that winning initializations do contain inductive biases that are generic to some extent, although, as reported by our experiments on the biomedical datasets, their generalization properties can be more limiting than what has been so far observed in the literature.
Reinforced Rewards Framework for Text Style Transfer
Venue: arXiv cs.CL 2020
Year: 2020

Style transfer deals with the algorithms to transfer the stylistic properties of a piece of text into that of another while ensuring that the core content is preserved. There has been a lot of interest in the field of text style transfer due to its wide application to tailored text generation. Existing works evaluate the style transfer models based on content preservation and transfer strength. In this work, we propose a reinforcement learning based framework that directly rewards the framework on these target metrics yielding a better transfer of the target style. We show the improved performance of our proposed framework based on automatic and human evaluation on three independent tasks: wherein we transfer the style of text from formal to informal, high excitement to low excitement, modern English to Shakespearean English, and vice-versa in all the three cases. Improved performance of the proposed framework over existing state-of-the-art frameworks indicates the viability of the approach.
Remote Short Blocklength Process Monitoring: Trade-off Between Resolution and Data Freshness
Venue: arXiv cs.IT 2020
Year: 2020

In cyber-physical systems, as in 5G and beyond, multiple physical processes require timely online monitoring at a remote device. There, the received information is used to estimate current and future process values. When transmitting the process data over a communication channel, source-channel coding is used in order to reduce data errors. During transmission, a high data resolution is helpful to capture the value of the process variables precisely. However, this typically comes with long transmission delays reducing the utilizability of the data, since the estimation quality gets reduced over time. In this paper, the trade-off between having recent data and precise measurements is captured for a Gauss-Markov process. An Age-of-Information (AoI) metric is used to assess data timeliness, while mean square error (MSE) is used to assess the precision of the predicted process values. AoI appears inherently within the MSE expressions, yet it can be relatively easier to optimize. Our goal is to minimize a time-averaged version of both metrics. We follow a short blocklength source-channel coding approach, and optimize the parameters of the codes being used in order to describe an achievability region between MSE and AoI.
Mobile Robot Path Planning in Dynamic Environments through Globally Guided Reinforcement Learning
Venue: arXiv cs.RO 2020
Year: 2020

Path planning for mobile robots in large dynamic environments is a challenging problem, as the robots are required to efficiently reach their given goals while simultaneously avoiding potential conflicts with other robots or dynamic objects. In the presence of dynamic obstacles, traditional solutions usually employ re-planning strategies, which re-call a planning algorithm to search for an alternative path whenever the robot encounters a conflict. However, such re-planning strategies often cause unnecessary detours. To address this issue, we propose a learning-based technique that exploits environmental spatio-temporal information. Different from existing learning-based methods, we introduce a globally guided reinforcement learning approach (G2RL), which incorporates a novel reward structure that generalizes to arbitrary environments. We apply G2RL to solve the multi-robot path planning problem in a fully distributed reactive manner. We evaluate our method across different map types, obstacle densities, and the number of robots. Experimental results show that G2RL generalizes well, outperforming existing distributed methods, and performing very similarly to fully centralized state-of-the-art benchmarks.
Delay-Aware Model-Based Reinforcement Learning for Continuous Control
Venue: arXiv cs.LG 2020
Year: 2020

Action delays degrade the performance of reinforcement learning in many real-world systems. This paper proposes a formal definition of delay-aware Markov Decision Process and proves it can be transformed into standard MDP with augmented states using the Markov reward process. We develop a delay-aware model-based reinforcement learning framework that can incorporate the multi-step delay into the learned system models without learning effort. Experiments with the Gym and MuJoCo platforms show that the proposed delay-aware model-based algorithm is more efficient in training and transferable between systems with various durations of delay compared with off-policy model-free reinforcement learning methods. Codes available at: https://github.com/baimingc/dambrl.
Age-of-Information-based Scheduling in Multiuser Uplinks with Stochastic Arrivals: A POMDP Approach
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we consider a multiuser uplink status update system, where a monitor aims to timely collect randomly generated status updates from multiple end nodes through a shared wireless channel. We adopt the recently proposed metric, termed age of information (AoI), to quantify the information timeliness and freshness. Due to the random generation of the status updates at the end node side, the monitor only grasps a partial knowledge of the status update arrivals. Under such a practical scenario, we aim to address a fundamental multiuser scheduling problem: how to schedule the end nodes to minimize the network-wide AoI? To solve this problem, we formulate it as a partially observable Markov decision process (POMDP), and develop a dynamic programming (DP) algorithm to obtain the optimal scheduling policy. By noting that the optimal policy is computationally prohibitive, we further design a low-complexity myopic policy that only minimizes the one-step expected reward. Simulation results show that the performance of the myopic policy can approach that of the optimal policy, and is better than that of the baseline policy.
Schema-Guided Natural Language Generation
Venue: arXiv cs.CL 2020
Year: 2020

Neural network based approaches to data-to-text natural language generation (NLG) have gained popularity in recent years, with the goal of generating a natural language prompt that accurately realizes an input meaning representation. To facilitate the training of neural network models, researchers created large datasets of paired utterances and their meaning representations. However, the creation of such datasets is an arduous task and they mostly consist of simple meaning representations composed of slot and value tokens to be realized. These representations do not include any contextual information that an NLG system can use when trying to generalize, such as domain information and descriptions of slots and values. In this paper, we present the novel task of Schema-Guided Natural Language Generation (SG-NLG). Here, the goal is still to generate a natural language prompt, but in SG-NLG, the input MRs are paired with rich schemata providing contextual information. To generate a dataset for SG-NLG we re-purpose an existing dataset for another task: dialog state tracking, which includes a large and rich schema spanning multiple different attributes, including information about the domain, user intent, and slot descriptions. We train different state-of-the-art models for neural natural language generation on this dataset and show that in many cases, including rich schema information allows our models to produce higher quality outputs both in terms of semantics and diversity. We also conduct experiments comparing model performance on seen versus unseen domains, and present a human evaluation demonstrating high ratings for overall output quality.
Characterizing Linear Memory-Rate Tradeoff of Coded Caching: The $(N,K)=(3,3)$ Case
Venue: arXiv cs.IT 2020
Year: 2020

We consider the cache problem introduced by Maddah-ali and Niesen [1] for the $(N,K)=(3,3)$ case, and use the computer-aided approach to derive the tight linear memory-rate trade-off. Two lower bounds $10M+6R\geq 15$ and $5M+4R\geq 9$ are proved, which are non-Shannon type. A coded linear scheme of point $(M,R)=(0.6,1.5)$ is constructed with the help of symmetry reduction and brute-force search.
Simultaneous paraphrasing and translation by fine-tuning Transformer models
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes the third place submission to the shared task on simultaneous translation and paraphrasing for language education at the 4th workshop on Neural Generation and Translation (WNGT) for ACL 2020. The final system leverages pre-trained translation models and uses a Transformer architecture combined with an oversampling strategy to achieve a competitive performance. This system significantly outperforms the baseline on Hungarian (27% absolute improvement in Weighted Macro F1 score) and Portuguese (33% absolute improvement) languages.
Existence of structured perfect Bayesian equilibrium in dynamic games of asymmetric information
Venue: arXiv cs.GT 2020
Year: 2020

In~[1],authors considered a general finite horizon model of dynamic game of asymmetric information, where N players have types evolving as independent Markovian process, where each player observes its own type perfectly and actions of all players. The authors present a sequential decomposition algorithm to find all structured perfect Bayesian equilibria of the game. The algorithm consists of solving a class of fixed-point of equations for each time $t,π_t$, whose existence was left as an open question. In this paper, we prove existence of these fixed-point equations for compact metric spaces.
Transmission Scheduling for Multi-loop Wireless Networked Control Based on LQ Cost Offset
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, transmission scheduling for multiloop wireless networked control systems sharing the wireless channel is considered. A linear quadratic cost offset has been proposed to evaluate the performance gap induced by the nonideal communication. A functional relationship between linear quadratic offset metric and Age of Information has been built up. Based on the offset metric, we come up with an age-based scheduling policy and numerical simulations show that there is a significant improvement compared to the former work.
Guarding a Polygon Without Losing Touch
Venue: arXiv cs.DC 2020
Year: 2020

We study the classical Art Gallery Problem first proposed by Klee in 1973 from a mobile multi-agents perspective. Specifically, we require an optimally small number of agents (also called guards) to navigate and position themselves in the interior of an unknown simple polygon with $n$ vertices such that the collective view of all the agents covers the polygon. We consider the visibly connected setting wherein agents must remain connected through line of sight links -- a requirement particularly relevant to multi-agent systems. We first provide a centralized algorithm for the visibly connected setting that runs in time $O(n)$, which is of course optimal. We then provide algorithms for two different distributed settings. In the first setting, agents can only perceive relative proximity (i.e., can tell which of a pair of objects is closer) whereas they can perceive exact distances in the second setting. Our distributed algorithms work despite agents having no prior knowledge of the polygon. Furthermore, we provide lower bounds to show that our distributed algorithms are near-optimal. Our visibly connected guarding ensures that (i) the guards form a connected network and (ii) the polygon is fully guarded. Consequently, this guarding provides the distributed infrastructure to execute any geometric algorithm for this polygon.
Specifying and Model Checking Workflows of Single Page Applications with TLA+
Venue: arXiv cs.SE 2020
Year: 2020

Single Page Applications (SPAs) are different than hypertext-based web applications in that their workflow is not defined by explicit links, but rather implicitly by changes of their widgets' states. The workflow may hence be hard to track. We present an approach to specifying and model checking SPAs with TLA+. Our approach makes it easier to document and to track the workflow of SPAs and to find potential design flaws.
Counting Distinct Patterns in Internal Dictionary Matching
Venue: arXiv cs.DS 2020
Year: 2020

We consider the problem of preprocessing a text $T$ of length $n$ and a dictionary $\mathcal{D}$ in order to be able to efficiently answer queries $CountDistinct(i,j)$, that is, given $i$ and $j$ return the number of patterns from $\mathcal{D}$ that occur in the fragment $T[i \mathinner{.\,.} j]$. The dictionary is internal in the sense that each pattern in $\mathcal{D}$ is given as a fragment of $T$. This way, the dictionary takes space proportional to the number of patterns $d=|\mathcal{D}|$ rather than their total length, which could be $Θ(n\cdot d)$. An $\tilde{\mathcal{O}}(n+d)$-size data structure that answers $CountDistinct(i,j)$ queries $\mathcal{O}(\log n)$-approximately in $\tilde{\mathcal{O}}(1)$ time was recently proposed in a work that introduced internal dictionary matching [ISAAC 2019]. Here we present an $\tilde{\mathcal{O}}(n+d)$-size data structure that answers $CountDistinct(i,j)$ queries $2$-approximately in $\tilde{\mathcal{O}}(1)$ time. Using range queries, for any $m$, we give an $\tilde{\mathcal{O}}(\min(nd/m,n^2/m^2)+d)$-size data structure that answers $CountDistinct(i,j)$ queries exactly in $\tilde{\mathcal{O}}(m)$ time. We also consider the special case when the dictionary consists of all square factors of the string. We design an $\mathcal{O}(n \log^2 n)$-size data structure that allows us to count distinct squares in a text fragment $T[i \mathinner{.\,.} j]$ in $\mathcal{O}(\log n)$ time.
Evaluating Ensemble Robustness Against Adversarial Attacks
Venue: arXiv cs.LG 2020
Year: 2020

Adversarial examples, which are slightly perturbed inputs generated with the aim of fooling a neural network, are known to transfer between models; adversaries which are effective on one model will often fool another. This concept of transferability poses grave security concerns as it leads to the possibility of attacking models in a black box setting, during which the internal parameters of the target model are unknown. In this paper, we seek to analyze and minimize the transferability of adversaries between models within an ensemble. To this end, we introduce a gradient based measure of how effectively an ensemble's constituent models collaborate to reduce the space of adversarial examples targeting the ensemble itself. Furthermore, we demonstrate that this measure can be utilized during training as to increase an ensemble's robustness to adversarial examples.