OpenRaiser/Intern-Atlas huggingface dataset:
Evaluating Computational Language Models with Scaling Properties of Natural Language
Venue: arXiv cs.CL 2019
Year: 2019

In this article, we evaluate computational models of natural language with respect to the universal statistical behaviors of natural language. Statistical mechanical analyses have revealed that natural language text is characterized by scaling properties, which quantify the global structure in the vocabulary population and the long memory of a text. We study whether five scaling properties (given by Zipf's law, Heaps' law, Ebeling's method, Taylor's law, and long-range correlation analysis) can serve for evaluation of computational models. Specifically, we test $n$-gram language models, a probabilistic context-free grammar (PCFG), language models based on Simon/Pitman-Yor processes, neural language models, and generative adversarial networks (GANs) for text generation. Our analysis reveals that language models based on recurrent neural networks (RNNs) with a gating mechanism (i.e., long short-term memory, LSTM; a gated recurrent unit, GRU; and quasi-recurrent neural networks, QRNNs) are the only computational models that can reproduce the long memory behavior of natural language. Furthermore, through comparison with recently proposed model-based evaluation methods, we find that the exponent of Taylor's law is a good indicator of model quality.
ZOFI: Zero-Overhead Fault Injection Tool for Fast Transient Fault Coverage Analysis
Venue: arXiv cs.DC 2019
Year: 2019

The experimental evaluation of fault-tolerance studies relies on tools that inject errors while programs are running, and then monitor the execution and the output for faulty execution. In particular, the established methodology in software-based transient-fault reliability studies, involves running each workload hundreds or thousands of times, injecting a random bit-flip in the process. The majority of such studies rely on custom-built fault-injection tools that are based on either a modified processor simulator, or a code instrumentation framework. Such tools are non-trivial to develop, and are usually orders of magnitude slower than native execution. In this paper we present ZOFI, a novel timing-based fault-injection tool that is aimed at being used in fault-coverage studies for transient faults. ZOFI is a zero-overhead tool, meaning that the analyzed workload runs at native speed. This is orders-of-magnitude faster compared to common approaches that are designed around simulators or code instrumentation tools. ZOFI is free software and is available at https://github.com/vporpo/zofi.
Efficient Peer-to-Peer Content Sharing for Learning in Virtual Worlds
Venue: arXiv cs.DC 2019
Year: 2019

Virtual world technologies provide new and immersive space for learning, training, and education. They are enabled by the content creation and content sharing function for allowing users to create and interoperate various learning objects. Unfortunately, virtual world content sharing based on persistent virtual world content storage, to the best of our knowledge, does not exist. In this paper, we address this problem by proposing a content sharing scheme based on Virtual Net, a virtual world persistency framework. For efficient content retrieval, three strategies have been proposed to reduce communication overhead and content load delay.By integrating these strategies, a virtual world content search and retrieval algorithm has been devised. The experiment results verify the effectiveness of the algorithm.
Asymmetric Random Projections
Venue: arXiv cs.LG 2019
Year: 2019

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extract statistics from the data that allows designing a data dependent RP with superior performance compared to data-oblivious RP. We tackle scenarios such as matrix multiplication and linear regression/classification in which we wish to estimate inner products between pairs of vectors from two possibly different sources. Our technique takes advantage of the difference between the sources and is provably superior to oblivious RPs. Additionally, we provide extensive experiments comparing RPs with our approach showing significant performance lifts in fast matrix multiplication, regression and classification problems.
Inferring Latent dimension of Linear Dynamical System with Minimum Description Length
Venue: arXiv cs.LG 2019
Year: 2019

Time-invariant linear dynamical system arises in many real-world applications,and its usefulness is widely acknowledged. A practical limitation with this model is that its latent dimension that has a large impact on the model capability needs to be manually specified. It can be demonstrated that a lower-order model class could be totally nested into a higher-order class, and the corresponding likelihood is nondecreasing. Hence, criterion built on the likelihood is not appropriate for model selection. This paper addresses the issue and proposes a criterion for linear dynamical system based on the principle of minimum description length. The latent structure, which is omitted in previous work, is explicitly considered in this newly proposed criterion. Our work extends the principle of minimum description length and demonstrates its effectiveness in the tasks of model training. The experiments on both univariate and multivariate sequences confirm the good performance of our newly proposed method.
Retrial Queueing Models: A Survey on Theory and Applications
Venue: arXiv cs.PF 2019
Year: 2019

Retrial phenomenon naturally arises in various systems such as call centers, cellular networks and random access protocols in local area networks. This paper gives a comprehensive survey on theory and applications of retrial queues in these systems. We investigate the state of the art of the theoretical researches including exact solutions, stability, asymptotic analyses and multidimensional models. We present an overview on retrial models arising from real world applications. Some open problems and promising research directions are also discussed.
Ego-CNN: Distributed, Egocentric Representations of Graphs for Detecting Critical Structures
Venue: arXiv cs.LG 2019
Year: 2019

We study the problem of detecting critical structures using a graph embedding model. Existing graph embedding models lack the ability to precisely detect critical structures that are specific to a task at the global scale. In this paper, we propose a novel graph embedding model, called the Ego-CNNs, that employs the ego-convolutions convolutions at each layer and stacks up layers using an ego-centric way to detects precise critical structures efficiently. An Ego-CNN can be jointly trained with a task model and help explain/discover knowledge for the task. We conduct extensive experiments and the results show that Ego-CNNs (1) can lead to comparable task performance as the state-of-the-art graph embedding models, (2) works nicely with CNN visualization techniques to illustrate the detected structures, and (3) is efficient and can incorporate with scale-free priors, which commonly occurs in social network datasets, to further improve the training efficiency.
Transfer Learning for Segmenting Dimensionally-Reduced Hyperspectral Images
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning has established the state of the art in multiple fields, including hyperspectral image analysis. However, training large-capacity learners to segment such imagery requires representative training sets. Acquiring such data is human-dependent and time-consuming, especially in Earth observation scenarios, where the hyperspectral data transfer is very costly and time-constrained. In this letter, we show how to effectively deal with a limited number and size of available hyperspectral ground-truth sets, and apply transfer learning for building deep feature extractors. Also, we exploit spectral dimensionality reduction to make our technique applicable over hyperspectral data acquired using different sensors, which may capture different numbers of hyperspectral bands. The experiments, performed over several benchmarks and backed up with statistical tests, indicated that our approach allows us to effectively train well-generalizing deep convolutional neural nets even using significantly reduced data.
Traffic Load Uniformity in Optical Packet Switched Networks
Venue: arXiv cs.NI 2019
Year: 2019

A crucial issue in Optical Packet Switched (OPS) networks is packet loss due to contentions. In this paper we study how traffic load uniformity influences the packet loss in a single OPS node. Traffic load uniformity is a measure of the load asymmetry a specific output link in an OPS node receives from its input sources. We develop analytical models based on the Engset traffic model. As a major contribution of this paper, we show that an asymmetric traffic pattern results in less packet loss compared to a symmetric traffic pattern for a single output link in an OPS node.
Quality of Uncertainty Quantification for Bayesian Neural Network Inference
Venue: arXiv cs.LG 2019
Year: 2019

Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and classification tasks. Our experiments demonstrate that commonly used metrics (e.g. test log-likelihood) can be misleading. Our experiments also indicate that inference innovations designed to capture structure in the posterior do not necessarily produce high quality posterior approximations.
EDIMA: Early Detection of IoT Malware Network Activity Using Machine Learning Techniques
Venue: arXiv cs.CR 2019
Year: 2019

The widespread adoption of Internet of Things has led to many security issues. Post the Mirai-based DDoS attack in 2016 which compromised IoT devices, a host of new malware using Mirai's leaked source code and targeting IoT devices have cropped up, e.g. Satori, Reaper, Amnesia, Masuta etc. These malware exploit software vulnerabilities to infect IoT devices instead of open TELNET ports (like Mirai) making them more difficult to block using existing solutions such as firewalls. In this research, we present EDIMA, a distributed modular solution which can be used towards the detection of IoT malware network activity in large-scale networks (e.g. ISP, enterprise networks) during the scanning/infecting phase rather than during an attack. EDIMA employs machine learning algorithms for edge devices' traffic classification, a packet traffic feature vector database, a policy module and an optional packet sub-sampling module. We evaluate the classification performance of EDIMA through testbed experiments and present the results obtained.
Blind decoding in $α$-Stable noise: An online learning approach
Venue: arXiv cs.IT 2019
Year: 2019

A novel method for performing error control coding in Symmetric $α-$Stable noise environments without any prior knowledge about the value of $α$ is introduced. We use an online learning framework which employs multiple distributions to decode the received block and then combines these results based on the past performance of each individual distributions. The proposed method is also able to handle a mixture of Symmetric $α-$Stable distributed noises. Performance results in turbo coded system highlight the utility of the work.
Learning Grasp Affordance Reasoning through Semantic Relations
Venue: arXiv cs.RO 2019
Year: 2019

Reasoning about object affordances allows an autonomous agent to perform generalised manipulation tasks among object instances. While current approaches to grasp affordance estimation are effective, they are limited to a single hypothesis. We present an approach for detection and extraction of multiple grasp affordances on an object via visual input. We define semantics as a combination of multiple attributes, which yields benefits in terms of generalisation for grasp affordance prediction. We use Markov Logic Networks to build a knowledge base graph representation to obtain a probability distribution of grasp affordances for an object. To harvest the knowledge base, we collect and make available a novel dataset that relates different semantic attributes. We achieve reliable mappings of the predicted grasp affordances on the object by learning prototypical grasping patches from several examples. We show our method's generalisation capabilities on grasp affordance prediction for novel instances and compare with similar methods in the literature. Moreover, using a robotic platform, on simulated and real scenarios, we evaluate the success of the grasping task when conditioned on the grasp affordance prediction.
The Minimum Tollbooth Problem in Atomic Network Congestion Games with Unsplittable Flows
Venue: arXiv cs.GT 2019
Year: 2019

This work analyzes the minimum tollbooth problem in atomic network congestion games with unsplittable flows. The goal is to place tolls on edges, such that there exists a pure Nash equilibrium in the tolled game that is a social optimum in the untolled one. Additionally, we require the number of tolled edges to be the minimum. This problem has been extensively studied in non-atomic games, however, to the best of our knowledge, it as not been considered for atomic games before. By a reduction from the weighted CNF SAT problem, we show both the NP-hardness of the problem and the W[2]-hardness when parameterizing the problem with the number of tolled edges. On the positive side, we present a polynomial time algorithm for networks on series-parallel graphs that turns any given state of the untolled game into a pure Nash equilibrium of the tolled game with the minimum number of tolled edges.
Internet of Autonomous Vehicles: Architecture, Features, and Socio-Technological Challenges
Venue: arXiv cs.NI 2019
Year: 2019

Mobility is the backbone of urban life and a vital economic factor in the development of the world. Rapid urbanization and the growth of mega-cities is bringing dramatic changes in the capabilities of vehicles. Innovative solutions like autonomy, electrification, and connectivity are on the horizon. How, then, we can provide ubiquitous connectivity to the legacy and autonomous vehicles? This paper seeks to answer this question by combining recent leaps of innovation in network virtualization with remarkable feats of wireless communications. To do so, this paper proposes a novel paradigm called the Internet of autonomous vehicles (IoAV). We begin painting the picture of IoAV by discussing the salient features, and applications of IoAV which is followed by a detailed discussion on the key enabling technologies. Next, we describe the proposed layered architecture of IoAV and uncover some critical functions of each layer. This is followed by the performance evaluation of IoAV which shows the significant advantage of the proposed architecture in terms of transmission time and energy consumption. Finally, to best capture the benefits of IoAV, we enumerate some social and technological challenges and explain how some unresolved issues can disrupt the widespread use of autonomous vehicles in the future.
To each route its own ETA: A generative modeling framework for ETA prediction
Venue: arXiv cs.LG 2019
Year: 2019

Accurate expected time of arrival (ETA) information is crucial in maintaining the quality of service of public transit. Recent advances in artificial intelligence (AI) has led to more effective models for ETA estimation that rely heavily on a large GPS datasets. More importantly, these are mainly cabs based datasets which may not be fit for bus-based public transport. Consequently, the latest methods may not be applicable for ETA estimation in cities with the absence of large training data set. On the other hand, the ETA estimation problem in many cities needs to be solved in the absence of big datasets that also contains outliers, anomalies and may be incomplete. This work presents a simple but robust model for ETA estimation for a bus route that only relies on the historical data of the particular route. We propose a system that generates ETA information for a trip and updates it as the trip progresses based on the real-time information. We train a deep learning based generative model that learns the probability distribution of ETA data across trips and conditional on the current trip information updates the ETA information on the go. Our plug and play model not only captures the non-linearity of the task well but that any transit agency can use without needing any other external data source. The experiments run over three routes, data collected in the city of Delhi illustrates the promise of our approach.
Intelligent Reflecting Surface Meets OFDM: Protocol Design and Rate Maximization
Venue: arXiv cs.IT 2019
Year: 2019

Intelligent reflecting surface (IRS) is a promising new technology for achieving both spectrum and energy efficient wireless communication systems in the future. However, existing works on IRS mainly consider frequency-flat channels and assume perfect knowledge of channel state information (CSI) at the transmitter. Motivated by this, in this paper we study an IRS-enhanced orthogonal frequency division multiplexing (OFDM) system under frequency-selective channels and propose a practical transmission protocol with channel estimation. First, to reduce the overhead in channel training and estimation and to exploit the channel spatial correlation, we propose a novel IRS elements grouping method, where each group consists of a set of adjacent IRS elements that share a common reflection coefficient. Based on this grouping method, we propose a practical transmission protocol where only the combined channel of each group needs to be estimated, thus substantially reducing the training overhead. Next, with any given grouping and estimated CSI, we formulate the problem to maximize the achievable rate by jointly optimizing the transmit power allocation and the IRS passive array reflection coefficients. Although the formulated problem is non-convex and thus difficult to solve, we propose an efficient algorithm to obtain a high-quality suboptimal solution for it, by alternately optimizing the power allocation and the passive array coefficients in an iterative manner, along with a customized method for the initialization. Simulation results show that the proposed design significantly improves the OFDM link rate performance as compared to the case without using IRS. Moreover, it is shown that there exists an optimal size for IRS elements grouping which achieves the maximum achievable rate due to the trade-off between the training overhead and IRS passive beamforming flexibility.
Multilingual Named Entity Recognition Using Pretrained Embeddings, Attention Mechanism and NCRF
Venue: arXiv cs.CL 2019
Year: 2019

In this paper we tackle multilingual named entity recognition task. We use the BERT Language Model as embeddings with bidirectional recurrent network, attention, and NCRF on the top. We apply multilingual BERT only as embedder without any fine-tuning. We test out model on the dataset of the BSNLP shared task, which consists of texts in Bulgarian, Czech, Polish and Russian languages.
Platform Independent Software Analysis for Near Memory Computing
Venue: arXiv cs.PF 2019
Year: 2019

Near-memory Computing (NMC) promises improved performance for the applications that can exploit the features of emerging memory technologies such as 3D-stacked memory. However, it is not trivial to find such applications and specialized tools are needed to identify them. In this paper, we present PISA-NMC, which extends a state-of-the-art hardware agnostic profiling tool with metrics concerning memory and parallelism, which are relevant for NMC. The metrics include memory entropy, spatial locality, data-level, and basic-block-level parallelism. By profiling a set of representative applications and correlating the metrics with the application's performance on a simulated NMC system, we verify the importance of those metrics. Finally, we demonstrate which metrics are useful in identifying applications suitable for NMC architectures.
Evolutionary Computation and AI Safety: Research Problems Impeding Routine and Safe Real-world Application of Evolution
Venue: arXiv cs.NE 2019
Year: 2019

Recent developments in artificial intelligence and machine learning have spurred interest in the growing field of AI safety, which studies how to prevent human-harming accidents when deploying AI systems. This paper thus explores the intersection of AI safety with evolutionary computation, to show how safety issues arise in evolutionary computation and how understanding from evolutionary computational and biological evolution can inform the broader study of AI safety.
Evaluating the Information Security Awareness of Smartphone Users
Venue: arXiv cs.CR 2019
Year: 2019

Information security awareness (ISA) is a practice focused on the set of skills, which help a user successfully mitigate a social engineering attack. Previous studies have presented various methods for evaluating the ISA of both PC and mobile users. These methods rely primarily on subjective data sources such as interviews, surveys, and questionnaires that are influenced by human interpretation and sincerity. Furthermore, previous methods for evaluating ISA did not address the differences between classes of social engineering attacks. In this paper, we present a novel framework designed for evaluating the ISA of smartphone users to specific social engineering attack classes. In addition to questionnaires, the proposed framework utilizes objective data sources: a mobile agent and a network traffic monitor; both of which are used to analyze the actual behavior of users. We empirically evaluated the ISA scores assessed from the three data sources (namely, the questionnaires, mobile agent, and network traffic monitor) by conducting a long-term user study involving 162 smartphone users. All participants were exposed to four different security challenges that resemble real-life social engineering attacks. These challenges were used to assess the ability of the proposed framework to derive a relevant ISA score. The results of our experiment show that: (1) the self-reported behavior of the users differs significantly from their actual behavior; and (2) ISA scores derived from data collected by the mobile agent or the network traffic monitor are highly correlated with the users' success in mitigating social engineering attacks.
Partial Quantifier Elimination With Learning
Venue: arXiv cs.LO 2019
Year: 2019

We consider a modification of the Quantifier Elimination (QE) problem called Partial QE (PQE). In PQE, only a small part of the formula is taken out of the scope of quantifiers. The appeal of PQE is that many verification problems, e.g. equivalence checking and model checking, reduce to PQE and the latter is much easier than complete QE. Earlier, we introduced a PQE algorithm based on the machinery of D-sequents. A D-sequent is a record stating that a clause is redundant in a quantified CNF formula in a specified subspace. To make this algorithm efficient, it is important to reuse learned D-sequents. However, reusing D-sequents is not as easy as conflict clauses in SAT-solvers because redundancy is a structural rather than a semantic property. Earlier, we modified the definition of D-sequents to enable their safe reusing. In this paper, we present a PQE algorithm based on new D-sequents. It is different from its predecessor in two aspects. First, the new algorithm can learn and reuse D-sequents. Second, it proves clauses redundant one by one and thus backtracks as soon as the current target clause is proved redundant in the current subspace. This makes the new PQE algorithm similar to a SAT-solver that backtracks as soon as just one clause is falsified. We show experimentally that the new PQE algorithm outperforms its predecessor.