OpenRaiser/Intern-Atlas huggingface dataset:
Incorporating Trip Chaining within Online Demand Estimation
Venue: arXiv cs.DS 2019
Year: 2019

Time-dependent Origin-Destination (OD) demand flows are fundamental inputs for Dynamic Traffic Assignment (DTA) systems and real-time traffic management. This work introduces a novel state-space framework to estimate these demand flows in an online context. Specifically, we propose to explicitly include trip-chaining behavior within the state-space formulation, which is solved using the well-established Kalman Filtering technique. While existing works already consider structural information and recursive behavior within the online demand estimation problem, this information has been always considered at the OD level. In this study, we introduce this structural information by explicitly representing trip-chaining within the estimation framework. The advantage is twofold. First, all trips belonging to the same tour can be jointly calibrated. Second, given the estimation during a certain time interval, a prediction of the structural deviation over the whole day can be obtained without the need to run additional simulations. The effectiveness of the proposed methodology is demonstrated first on a toy network and then on a large real-world network. Results show that the model improves the prediction performance with respect to a conventional Kalman Filtering approach. We also show that, on the basis of the estimation of the morning commute, the model can be used to predict the evening commute without need of running additional simulations.
Compositional and Abstraction-Based Approach for Synthesis of Edit Functions for Opacity Enforcement
Venue: arXiv cs.CR 2019
Year: 2019

This paper develops a novel compositional and abstraction-based approach to synthesize edit functions for opacity enforcement in modular discrete event systems. Edit functions alter the output of the system by erasing or inserting events in order to obfuscate the outside intruder, whose goal is to infer the secrets of the system from its observation. We synthesize edit functions to solve the opacity enforcement problem in a modular setting, which significantly reduces the computational complexity compared with the monolithic approach. Two abstraction methods called opaque observation equivalence and opaque bisimulation are first employed to abstract the individual components of the modular system and their observers. Subsequently, we propose a method to transform the synthesis of edit functions to the calculation of modular supremal nonblocking supervisors. We show that the edit functions synthesized in this manner correctly solve the opacity enforcement problem.
MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension
Venue: arXiv cs.CL 2019
Year: 2019

Machine Reading Comprehension (MRC) for question answering (QA), which aims to answer a question given the relevant context passages, is an important way to test the ability of intelligence systems to understand human language. Multiple-Choice QA (MCQA) is one of the most difficult tasks in MRC because it often requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations, compared to the extractive counterpart where answers are usually spans of text within given passages. Moreover, most existing MCQA datasets are small in size, making the learning task even harder. We introduce MMM, a Multi-stage Multi-task learning framework for Multi-choice reading comprehension. Our method involves two sequential stages: coarse-tuning stage using out-of-domain datasets and multi-task learning stage using a larger in-domain dataset to help model generalize better with limited data. Furthermore, we propose a novel multi-step attention network (MAN) as the top-level classifier for this task. We demonstrate MMM significantly advances the state-of-the-art on four representative MCQA datasets.
Low-Resolution Limited-Feedback NOMA for mmWave Communications
Venue: arXiv cs.IT 2019
Year: 2019

The spectrum-efficient millimeter-wave (mmWave) communications has recently attracted much attention as a viable solution to spectrum crunch problem. In this work, we propose a novel non-orthogonal multiple access (NOMA) framework, which makes use of the directional propagation characteristics of mmWave communications so as to improve the spectral efficiency through non-orthogonal signaling. In particular, we consider one-bit quantized angle information as a limited yet effective feedback scheme describing the channel quality of user equipment (UE) in mmWave bands. The UE pairs for NOMA transmission are then established using not only the one-bit distance feedback as a classical approach, but also the one-bit angle feedback. The proposed strategy is therefore referred to as two-bit NOMA. We also propose a novel hybrid strategy, called combined NOMA, for the circumstances with no UE pair through two-bit NOMA. Whenever no UE pair is available through any NOMA strategy, we resort to single user transmission (SUT) with proper UE selection schemes. The hybrid sum-rate performance is also analyzed thoroughly with the respective outage and rate expressions. The numerical results verify that the proposed strategy outperforms one-bit NOMA schemes with either angle- or distance-only feedback.
Lineage-Aware Temporal Windows: Supporting Set Operations in Temporal-Probabilistic Databases
Venue: arXiv cs.DB 2019
Year: 2019

In temporal-probabilistic (TP) databases, the combination of the temporal and the probabilistic dimension adds significant overhead to the computation of set operations. Although set queries are guaranteed to yield linearly sized output relations, existing solutions exhibit quadratic runtime complexity. They suffer from redundant interval comparisons and additional joins for the formation of lineage expressions. In this paper, we formally define the semantics of set operations in TP databases and study their properties. For their efficient computation, we introduce the lineage-aware temporal window, a mechanism that directly binds intervals with lineage expressions. We suggest the lineage-aware window advancer (LAWA) for producing the windows of two TP relations in linearithmic time, and we implement all TP set operations based on LAWA. By exploiting the flexibility of lineage-aware temporal windows, we perform direct filtering of irrelevant intervals and finalization of output lineage expressions and thus guarantee that no additional computational cost or buffer space is needed. A series of experiments over both synthetic and real-world datasets show that (a) our approach has predictable performance, depending only on the input size and not on the number of time intervals per fact or their overlap, and that (b) it outperforms state-of-the-art approaches in both temporal and probabilistic databases.
An Analysis of Malware Trends in Enterprise Networks
Venue: arXiv cs.CR 2019
Year: 2019

We present an empirical and large-scale analysis of malware samples captured from two different enterprises from 2017 to early 2018. Particularly, we perform threat vector, social-engineering, vulnerability and time-series analysis on our dataset. Unlike existing malware studies, our analysis is specifically focused on the recent enterprise malware samples. First of all, based on our analysis on the combined datasets of two enterprises, our results confirm the general consensus that AV-only solutions are not enough for real-time defenses in enterprise settings because on average 40% of the malware samples, when first appeared, are not detected by most AVs on VirusTotal or not uploaded to VT at all (i.e., never seen in the wild yet). Moreover, our analysis also shows that enterprise users transfer documents more than executables and other types of files. Therefore, attackers embed malicious codes into documents to download and install the actual malicious payload instead of sending malicious payload directly or using vulnerability exploits. Moreover, we also found that financial matters (e.g., purchase orders and invoices) are still the most common subject seen in Business Email Compromise (BEC) scams that aim to trick employees. Finally, based on our analysis on the timestamps of captured malware samples, we found that 93% of the malware samples were delivered on weekdays. Our further analysis also showed that while the malware samples that require user interaction such as macro-based malware samples have been captured during the working hours of the employees, the massive malware attacks are triggered during the off-times of the employees to be able to silently spread over the networks.
BillSum: A Corpus for Automatic Summarization of US Legislation
Venue: arXiv cs.CL 2019
Year: 2019

Automatic summarization methods have been studied on a variety of domains, including news and scientific articles. Yet, legislation has not previously been considered for this task, despite US Congress and state governments releasing tens of thousands of bills every year. In this paper, we introduce BillSum, the first dataset for summarization of US Congressional and California state bills (https://github.com/FiscalNote/BillSum). We explain the properties of the dataset that make it more challenging to process than other domains. Then, we benchmark extractive methods that consider neural sentence representations and traditional contextual features. Finally, we demonstrate that models built on Congressional bills can be used to summarize California bills, thus, showing that methods developed on this dataset can transfer to states without human-written summaries.
A Three-dimensional Convolutional-Recurrent Network for Convective Storm Nowcasting
Venue: arXiv cs.CV 2019
Year: 2019

Very short-term convective storm forecasting, termed nowcasting, has long been an important issue and has attracted substantial interest. Existing nowcasting methods rely principally on radar images and are limited in terms of nowcasting storm initiation and growth. Real-time re-analysis of meteorological data supplied by numerical models provides valuable information about three-dimensional (3D), atmospheric, boundary layer thermal dynamics, such as temperature and wind. To mine such data, we here develop a convolution-recurrent, hybrid deep-learning method with the following characteristics: (1) the use of cell-based oversampling to increase the number of training samples; this mitigates the class imbalance issue; (2) the use of both raw 3D radar data and 3D meteorological data re-analyzed via multi-source 3D convolution without any need for handcraft feature engineering; and (3) the stacking of convolutional neural networks on a long short-term memory encoder/decoder that learns the spatiotemporal patterns of convective processes. Experimental results demonstrated that our method performs better than other extrapolation methods. Qualitative analysis yielded encouraging nowcasting results.
On the Influence of Twitter Trolls during the 2016 US Presidential Election
Venue: arXiv cs.SI 2019
Year: 2019

It is a widely accepted fact that state-sponsored Twitter accounts operated during the 2016 US presidential election spreading millions of tweets with misinformation and inflammatory political content. Whether these social media campaigns of the so-called "troll" accounts were able to manipulate public opinion is still in question. Here we aim to quantify the influence of troll accounts and the impact they had on Twitter by analyzing 152.5 million tweets from 9.9 million users, including 822 troll accounts. The data collected during the US election campaign, contain original troll tweets before they were deleted by Twitter. From these data, we constructed a very large interaction graph; a directed graph of 9.3 million nodes and 169.9 million edges. Recently, Twitter released datasets on the misinformation campaigns of 8,275 state-sponsored accounts linked to Russia, Iran and Venezuela as part of the investigation on the foreign interference in the 2016 US election. These data serve as ground-truth identifier of troll users in our dataset. Using graph analysis techniques we qualify the diffusion cascades of web and media context that have been shared by the troll accounts. We present strong evidence that authentic users were the source of the viral cascades. Although the trolls were participating in the viral cascades, they did not have a leading role in them and only four troll accounts were truly influential.
End-to-end learning of energy-based representations for irregularly-sampled signals and images
Venue: arXiv cs.CV 2019
Year: 2019

For numerous domains, including for instance earth observation, medical imaging, astrophysics,..., available image and signal datasets often involve irregular space-time sampling patterns and large missing data rates. These sampling properties may be critical to apply state-of-the-art learning-based (e.g., auto-encoders, CNNs,...), fully benefit from the available large-scale observations and reach breakthroughs in the reconstruction and identification of processes of interest. In this paper, we address the end-to-end learning of representations of signals, images and image sequences from irregularly-sampled data, i.e. when the training data involved missing data. From an analogy to Bayesian formulation, we consider energy-based representations. Two energy forms are investigated: one derived from auto-encoders and one relating to Gibbs priors. The learning stage of these energy-based representations (or priors) involve a joint interpolation issue, which amounts to solving an energy minimization problem under observation constraints. Using a neural-network-based implementation of the considered energy forms, we can state an end-to-end learning scheme from irregularly-sampled data. We demonstrate the relevance of the proposed representations for different case-studies: namely, multivariate time series, 2D images and image sequences.
Unsupervised Projection Networks for Generative Adversarial Networks
Venue: arXiv cs.CV 2019
Year: 2019

We propose the use of unsupervised learning to train projection networks that project onto the latent space of an already trained generator. We apply our method to a trained StyleGAN, and use our projection network to perform image super-resolution and clustering of images into semantically identifiable groups.
Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers
Venue: arXiv cs.LG 2019
Year: 2019

We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimizing over this measure more quickly excludes reservoirs that fail to reproduce the climate. The results of optimization are surprising: the optimized parameters often specify a reservoir network with very low connectivity. Inspired by this observation, we explore reservoir designs with even simpler structure, and find well-performing reservoirs that have zero spectral radius and no recurrence. These simple reservoirs provide counterexamples to widely used heuristics in the field, and may be useful for hardware implementations of reservoir computers.
Retrieving Top Weighted Triangles in Graphs
Venue: arXiv cs.SI 2019
Year: 2019

Pattern counting in graphs is a fundamental primitive for many network analysis tasks, and a number of methods have been developed for scaling subgraph counting to large graphs. Many real-world networks carry a natural notion of strength of connection between nodes, which are often modeled by a weighted graph, but existing scalable graph algorithms for pattern mining are designed for unweighted graphs. Here, we develop a suite of deterministic and random sampling algorithms that enable the fast discovery of the 3-cliques (triangles) with the largest weight in a graph, where weight is measured by a generalized mean of a triangle's edges. For example, one of our proposed algorithms can find the top-1000 weighted triangles of a weighted graph with billions of edges in thirty seconds on a commodity server, which is orders of magnitude faster than existing "fast" enumeration schemes. Our methods thus open the door towards scalable pattern mining in weighted graphs.
NESTA: Hamming Weight Compression-Based Neural Proc. Engine
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we present NESTA, a specialized Neural engine that significantly accelerates the computation of convolution layers in a deep convolutional neural network, while reducing the computational energy. NESTA reformats Convolutions into $3 \times 3$ batches and uses a hierarchy of Hamming Weight Compressors to process each batch. Besides, when processing the convolution across multiple channels, NESTA, rather than computing the precise result of a convolution per channel, quickly computes an approximation of its partial sum, and a residual value such that if added to the approximate partial sum, generates the accurate output. Then, instead of immediately adding the residual, it uses (consumes) the residual when processing the next batch in the hamming weight compressors with available capacity. This mechanism shortens the critical path by avoiding the need to propagate carry signals during each round of computation and speeds up the convolution of each channel. In the last stage of computation, when the partial sum of the last channel is computed, NESTA terminates by adding the residual bits to the approximate output to generate a correct result.
Spherical k-Nearest Neighbors Interpolation
Venue: arXiv cs.DS 2019
Year: 2019

Geospatial interpolation is a challenging task due to real world data often being sparse, heterogeneous and inconsistent. For that matter, this work presents SkNNI, a spherical interpolation algorithm capable of working with such challenging geospatial data. This work also presents NDDNISD an accurate and efficient interpolation function for SkNNI which shines due to its spatial awareness in terms of proximity and distribution of observation neighbors. SkNNI's open source implementation is also discussed and illustrated with a simple usage example.
Toward a Secure and Decentralized Blockchain-based Ride-Hailing Platform for Autonomous Vehicles
Venue: arXiv cs.CR 2019
Year: 2019

Ride-hailing and ride-sharing applications have recently gained in popularity as a convenient alternative to traditional modes of travel. Current research into autonomous vehicles is accelerating rapidly and will soon become a critical component of a ride-hailing platform's architecture. Implementing an autonomous vehicle ride-hailing platform proves a difficult challenge due to the centralized nature of traditional ride-hailing architectures. In a traditional ride-hailing environment the drivers operate their own personal vehicles so it follows that a fleet of autonomous vehicles would be required for a centralized ride-hailing platform to succeed. Decentralization of the ride-hailing platform would remove a road block along the way to an autonomous vehicle ride-hailing platform by allowing owners of autonomous vehicles to add their vehicle to a community driven fleet when not in use. Blockchain technology is an attractive choice for this decentralized architecture due to its immutability and fault tolerance. This paper proposes a framework for developing a decentralized ride-hailing architecture implemented on the Hyperledger Fabric blockchain platform. The implementation is evaluated using a static analysis tool and performing a performance analysis under heavy network load.
State-of-the-Art Speech Recognition Using Multi-Stream Self-Attention With Dilated 1D Convolutions
Venue: arXiv cs.CL 2019
Year: 2019

Self-attention has been a huge success for many downstream tasks in NLP, which led to exploration of applying self-attention to speech problems as well. The efficacy of self-attention in speech applications, however, seems not fully blown yet since it is challenging to handle highly correlated speech frames in the context of self-attention. In this paper we propose a new neural network model architecture, namely multi-stream self-attention, to address the issue thus make the self-attention mechanism more effective for speech recognition. The proposed model architecture consists of parallel streams of self-attention encoders, and each stream has layers of 1D convolutions with dilated kernels whose dilation rates are unique given stream, followed by a self-attention layer. The self-attention mechanism in each stream pays attention to only one resolution of input speech frames and the attentive computation can be more efficient. In a later stage, outputs from all the streams are concatenated then linearly projected to the final embedding. By stacking the proposed multi-stream self-attention encoder blocks and rescoring the resultant lattices with neural network language models, we achieve the word error rate of 2.2% on the test-clean dataset of the LibriSpeech corpus, the best number reported thus far on the dataset.
A Pre-defined Sparse Kernel Based Convolution for Deep CNNs
Venue: arXiv cs.CV 2019
Year: 2019

The high demand for computational and storage resources severely impede the deployment of deep convolutional neural networks (CNNs) in limited-resource devices. Recent CNN architectures have proposed reduced complexity versions (e.g. SuffleNet and MobileNet) but at the cost of modest decreases inaccuracy. This paper proposes pSConv, a pre-defined sparse 2D kernel-based convolution, which promises significant improvements in the trade-off between complexity and accuracy for both CNN training and inference. To explore the potential of this approach, we have experimented with two widely accepted datasets, CIFAR-10 and Tiny ImageNet, in sparse variants of both the ResNet18 and VGG16 architectures. Our approach shows a parameter count reduction of up to 4.24x with modest degradation in classification accuracy relative to that of standard CNNs. Our approach outperforms a popular variant of ShuffleNet using a variant of ResNet18 with pSConv having 3x3 kernels with only four of nine elements not fixed at zero. In particular, the parameter count is reduced by 1.7x for CIFAR-10 and 2.29x for Tiny ImageNet with an increased accuracy of ~4%.
MAC Protocols for Wireless Mesh Networks with Multi-beam Antennas: A Survey
Venue: arXiv cs.NI 2019
Year: 2019

Multi-beam antenna technologies have provided lots of promising solutions to many current challenges faced in wireless mesh networks. The antenna can establish several beamformings simultaneously and initiate concurrent transmissions or receptions using multiple beams, thereby increasing the overall throughput of the network transmission. Multi-beam antenna has the ability to increase the spatial reuse, extend the transmission range, improve the transmission reliability, as well as save the power consumption. Traditional Medium Access Control (MAC) protocols for wireless network largely relied on the IEEE 802.11 Distributed Coordination Function(DCF) mechanism, however, IEEE 802.11 DCF cannot take the advantages of these unique capabilities provided by multi-beam antennas. This paper surveys the MAC protocols for wireless mesh networks with multi-beam antennas. The paper first discusses some basic information in designing multi-beam antenna system and MAC protocols, and then presents the main challenges for the MAC protocols in wireless mesh networks compared with the traditional MAC protocols. A qualitative comparison of the existing MAC protocols is provided to highlight their novel features, which provides a reference for designing the new MAC protocols. To provide some insights on future research, several open issues of MAC protocols are discussed for wireless mesh networks using multi-beam antennas.