OpenRaiser/Intern-Atlas huggingface dataset:
Mitigating the Curse of Correlation in Security Games by Entropy Maximization
Venue: arXiv cs.GT 2018
Year: 2018

In Stackelberg security games, a defender seeks to randomly allocate limited security resources to protect critical targets from an attack. In this paper, we study a fundamental, yet underexplored, phenomenon in security games, which we term the \emph{Curse of Correlation} (CoC). Specifically, we observe that there are inevitable correlations among the protection status of different targets. Such correlation is a crucial concern, especially in \emph{spatio-temporal} domains like conservation area patrolling, where attackers can surveil patrollers at certain areas and then infer their patrolling routes using such correlations. To mitigate this issue, we propose to design entropy-maximizing defending strategies for spatio-temporal security games, which frequently suffer from CoC. We prove that the problem is \#P-hard in general. However, it admits efficient algorithms in well-motivated special settings. Our experiments show significant advantages of max-entropy algorithms over previous algorithms. A scalable implementation of our algorithm is currently under pre-deployment testing for integration into FAMS software to improve the scheduling of US federal air marshals.
Virtual Reality over Wireless Networks: Quality-of-Service Model and Learning-Based Resource Management
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, the problem of resource management is studied for a network of wireless virtual reality (VR) users communicating over small cell networks (SCNs). In order to capture the VR users' quality-of-service (QoS) in SCNs, a novel VR model, based on multi-attribute utility theory, is proposed. This model jointly accounts for VR metrics such as tracking accuracy, processing delay, and transmission delay. In this model, the small base stations (SBSs) act as the VR control centers that collect the tracking information from VR users over the cellular uplink. Once this information is collected, the SBSs will then send the three dimensional images and accompanying surround stereo audio to the VR users over the downlink. Therefore, the resource allocation problem in VR wireless networks must jointly consider both the uplink and downlink. This problem is then formulated as a noncooperative game and a distributed algorithm based on the machine learning framework of echo state networks (ESNs) is proposed to find the solution of this game. The use of the proposed ESN algorithm enables the SBSs to predict the VR QoS of each SBS and guarantees the convergence to a mixed-strategy Nash equilibrium. The analytical result shows that each user's VR QoS jointly depends on both VR tracking accuracy and wireless resource allocation. Simulation results show that the proposed algorithm yields significant gains, in terms of total utility value of VR QoS, that reach up to 22.2% and 37.5%, respectively, compared to Q-learning and a baseline proportional fair algorithm. The results also show that the proposed algorithm has a faster convergence time than Q-learning and can guarantee low delays for VR services.
Separation of time scales and direct computation of weights in deep neural networks
Venue: arXiv cs.LG 2018
Year: 2018

Artificial intelligence is revolutionizing our lives at an ever increasing pace. At the heart of this revolution is the recent advancements in deep neural networks (DNN), learning to perform sophisticated, high-level tasks. However, training DNNs requires massive amounts of data and is very computationally intensive. Gaining analytical understanding of the solutions found by DNNs can help us devise more efficient training algorithms, replacing the commonly used mthod of stochastic gradient descent (SGD). We analyze the dynamics of SGD and show that, indeed, direct computation of the solutions is possible in many cases. We show that a high performing setup used in DNNs introduces a separation of time-scales in the training dynamics, allowing SGD to train layers from the lowest (closest to input) to the highest. We then show that for each layer, the distribution of solutions found by SGD can be estimated using a class-based principal component analysis (PCA) of the layer's input. This finding allows us to forgo SGD entirely and directly derive the DNN parameters using this class-based PCA, which can be well estimated using significantly less data than SGD. We implement these results on image datasets MNIST, CIFAR10 and CIFAR100 and find that, in fact, layers derived using our class-based PCA perform comparable or superior to neural networks of the same size and architecture trained using SGD. We also confirm that the class-based PCA often converges using a fraction of the data required for SGD. Thus, using our method training time can be reduced both by requiring less training data than SGD, and by eliminating layers in the costly backpropagation step of the training.
Deep learning with convolutional neural networks for EEG decoding and visualization
Venue: arXiv cs.LG 2018
Year: 2018

PLEASE READ AND CITE THE REVISED VERSION at Human Brain Mapping: http://onlinelibrary.wiley.com/doi/10.1002/hbm.23730/full Code available here: https://github.com/robintibor/braindecode
Layered black-box, behavioral interconnection perspective and applications to the problem of communication with fidelity criteria, Part I: i.i.d. sources
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, the problem of communication over an essentially unknown channel, which is known to be able to communicate a source to a destination to within a certain distortion level, is considered from a behavioral, interconnection view-point. Rates of reliable communication are derived and source-channel separation for communication with fidelity criteria is proved. The results are then generalized to the multi-user setting under certain assumptions. Other applications of this problem problem which follow from this perspective are discussed.
Layered black-box, behavioral interconnection perspective and applications to the problem of communication with fidelity criteria, Part II: stationary sources satisfying ψ-mixing criterion
Venue: arXiv cs.IT 2018
Year: 2018

Theorems from Part 1 of this paper are generalized to ψ-mixing sources in this paper. Application to Markoff chains and order m Markoff chains is presented. The main result is the generalization of Theorem 1 in Part 1.
Smart elements in combinatorial group testing problems
Venue: arXiv cs.DM 2018
Year: 2018

In combinatorial group testing problems Questioner needs to find a special element $x \in [n]$ by testing subsets of $[n]$. Tapolcai et al. introduced a new model, where each element knows the answer for those queries that contain it and each element should be able to identify the special one. Using classical results of extremal set theory we prove that if $\mathcal{F}_n \subset 2^{[n]}$ solves the non-adaptive version of this problem and has minimal cardinality, then $$\lim_{n \rightarrow \infty} \frac{|\mathcal{F}_n|}{\log_2 n} = \log_{(3/2)}2.$$ This improves results by Tapolcai et al. We also consider related models inspired by secret sharing models, where the elements should share information among them to find out the special one. Finally the adaptive versions of the different models are investigated.
Computation Peer Offloading for Energy-Constrained Mobile Edge Computing in Small-Cell Networks
Venue: arXiv cs.GT 2018
Year: 2018

The (ultra-)dense deployment of small-cell base stations (SBSs) endowed with cloud-like computing functionalities paves the way for pervasive mobile edge computing (MEC), enabling ultra-low latency and location-awareness for a variety of emerging mobile applications and the Internet of Things. To handle spatially uneven computation workloads in the network, cooperation among SBSs via workload peer offloading is essential to avoid large computation latency at overloaded SBSs and provide high quality of service to end users. However, performing effective peer offloading faces many unique challenges in small cell networks due to limited energy resources committed by self-interested SBS owners, uncertainties in the system dynamics and co-provisioning of radio access and computing services. This paper develops a novel online SBS peer offloading framework, called OPEN, by leveraging the Lyapunov technique, in order to maximize the long-term system performance while keeping the energy consumption of SBSs below individual long-term constraints. OPEN works online without requiring information about future system dynamics, yet provides provably near-optimal performance compared to the oracle solution that has the complete future information. In addition, this paper formulates a novel peer offloading game among SBSs, analyzes its equilibrium and efficiency loss in terms of the price of anarchy in order to thoroughly understand SBSs' strategic behaviors, thereby enabling decentralized and autonomous peer offloading decision making. Extensive simulations are carried out and show that peer offloading among SBSs dramatically improves the edge computing performance.
Discriminative Distance-Based Network Indices with Application to Link Prediction
Venue: arXiv cs.DS 2018
Year: 2018

In large networks, using the length of shortest paths as the distance measure has shortcomings. A well-studied shortcoming is that extending it to disconnected graphs and directed graphs is controversial. The second shortcoming is that a huge number of vertices may have exactly the same score. The third shortcoming is that in many applications, the distance between two vertices not only depends on the length of shortest paths, but also on the number of shortest paths. In this paper, first we develop a new distance measure between vertices of a graph that yields discriminative distance-based centrality indices. This measure is proportional to the length of shortest paths and inversely proportional to the number of shortest paths. We present algorithms for exact computation of the proposed discriminative indices. Second, we develop randomized algorithms that precisely estimate average discriminative path length and average discriminative eccentricity and show that they give $(ε,δ)$-approximations of these indices. Third, we perform extensive experiments over several real-world networks from different domains. In our experiments, we first show that compared to the traditional indices, discriminative indices have usually much more discriminability. Then, we show that our randomized algorithms can very precisely estimate average discriminative path length and average discriminative eccentricity, using only few samples. Then, we show that real-world networks have usually a tiny average discriminative path length, bounded by a constant (e.g., 2). Fourth, in order to better motivate the usefulness of our proposed distance measure, we present a novel link prediction method, that uses discriminative distance to decide which vertices are more likely to form a link in future, and show its superior performance compared to the well-known existing measures.
Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning
Venue: arXiv cs.CV 2018
Year: 2018

Deep convolutional neural networks (DCNNs) have been used to achieve state-of-the-art performance on many computer vision tasks (e.g., object recognition, object detection, semantic segmentation) thanks to a large repository of annotated image data. Large labeled datasets for other sensor modalities, e.g., multispectral imagery (MSI), are not available due to the large cost and manpower required. In this paper, we adapt state-of-the-art DCNN frameworks in computer vision for semantic segmentation for MSI imagery. To overcome label scarcity for MSI data, we substitute real MSI for generated synthetic MSI in order to initialize a DCNN framework. We evaluate our network initialization scheme on the new RIT-18 dataset that we present in this paper. This dataset contains very-high resolution MSI collected by an unmanned aircraft system. The models initialized with synthetic imagery were less prone to over-fitting and provide a state-of-the-art baseline for future work.
A Preferential Attachment Paradox: How Preferential Attachment Combines with Growth to Produce Networks with Log-normal In-degree Distributions
Venue: arXiv cs.DL 2018
Year: 2018

Every network scientist knows that preferential attachment combines with growth to produce networks with power-law in-degree distributions. How, then, is it possible for the network of American Physical Society journal collection citations to enjoy a log-normal citation distribution when it was found to have grown in accordance with preferential attachment? This anomalous result, which we exalt as the preferential attachment paradox, has remained unexplained since the physicist Sidney Redner first made light of it over a decade ago. Here we propose a resolution. The chief source of the mischief, we contend, lies in Redner having relied on a measurement procedure bereft of the accuracy required to distinguish preferential attachment from another form of attachment that is consistent with a log-normal in-degree distribution. There was a high-accuracy measurement procedure in use at the time, but it would have have been difficult to use it to shed light on the paradox, due to the presence of a systematic error inducing design flaw. In recent years the design flaw had been recognised and corrected. We show that the bringing of the newly corrected measurement procedure to bear on the data leads to a resolution of the paradox.
Parallel Sort-Based Matching for Data Distribution Management on Shared-Memory Multiprocessors
Venue: arXiv cs.DC 2018
Year: 2018

In this paper we consider the problem of identifying intersections between two sets of d-dimensional axis-parallel rectangles. This is a common problem that arises in many agent-based simulation studies, and is of central importance in the context of High Level Architecture (HLA), where it is at the core of the Data Distribution Management (DDM) service. Several realizations of the DDM service have been proposed; however, many of them are either inefficient or inherently sequential. These are serious limitations since multicore processors are now ubiquitous, and DDM algorithms -- being CPU-intensive -- could benefit from additional computing power. We propose a parallel version of the Sort-Based Matching algorithm for shared-memory multiprocessors. Sort-Based Matching is one of the most efficient serial algorithms for the DDM problem, but is quite difficult to parallelize due to data dependencies. We describe the algorithm and compute its asymptotic running time; we complete the analysis by assessing its performance and scalability through extensive experiments on two commodity multicore systems based on a dual socket Intel Xeon processor, and a single socket Intel Core i7 processor.
The Role of Network Analysis in Industrial and Applied Mathematics
Venue: arXiv cs.SI 2018
Year: 2018

Many problems in industry --- and in the social, natural, information, and medical sciences --- involve discrete data and benefit from approaches from subjects such as network science, information theory, optimization, probability, and statistics. The study of networks is concerned explicitly with connectivity between different entities, and it has become very prominent in industrial settings, an importance that has intensified amidst the modern data deluge. In this commentary, we discuss the role of network analysis in industrial and applied mathematics, and we give several examples of network science in industry. We focus, in particular, on discussing a physical-applied-mathematics approach to the study of networks. We also discuss several of our own collaborations with industry on projects in network analysis.
Mask R-CNN
Venue: arXiv cs.CV 2018
Year: 2018

We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without bells and whistles, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code has been made available at: https://github.com/facebookresearch/Detectron
High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks
Venue: arXiv cs.CV 2018
Year: 2018

Advances in deep learning for natural images have prompted a surge of interest in applying similar techniques to medical images. The majority of the initial attempts focused on replacing the input of a deep convolutional neural network with a medical image, which does not take into consideration the fundamental differences between these two types of images. Specifically, fine details are necessary for detection in medical images, unlike in natural images where coarse structures matter most. This difference makes it inadequate to use the existing network architectures developed for natural images, because they work on heavily downscaled images to reduce the memory requirements. This hides details necessary to make accurate predictions. Additionally, a single exam in medical imaging often comes with a set of views which must be fused in order to reach a correct conclusion. In our work, we propose to use a multi-view deep convolutional neural network that handles a set of high-resolution medical images. We evaluate it on large-scale mammography-based breast cancer screening (BI-RADS prediction) using 886,000 images. We focus on investigating the impact of the training set size and image size on the prediction accuracy. Our results highlight that performance increases with the size of training set, and that the best performance can only be achieved using the original resolution. In the reader study, performed on a random subset of the test set, we confirmed the efficacy of our model, which achieved performance comparable to a committee of radiologists when presented with the same data.
ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks
Venue: arXiv cs.NE 2018
Year: 2018

A fundamental feature of learning in animals is the "ability to forget" that allows an organism to perceive, model and make decisions from disparate streams of information and adapt to changing environments. Against this backdrop, we present a novel unsupervised learning mechanism ASP (Adaptive Synaptic Plasticity) for improved recognition with Spiking Neural Networks (SNNs) for real time on-line learning in a dynamic environment. We incorporate an adaptive weight decay mechanism with the traditional Spike Timing Dependent Plasticity (STDP) learning to model adaptivity in SNNs. The leak rate of the synaptic weights is modulated based on the temporal correlation between the spiking patterns of the pre- and post-synaptic neurons. This mechanism helps in gradual forgetting of insignificant data while retaining significant, yet old, information. ASP, thus, maintains a balance between forgetting and immediate learning to construct a stable-plastic self-adaptive SNN for continuously changing inputs. We demonstrate that the proposed learning methodology addresses catastrophic forgetting while yielding significantly improved accuracy over the conventional STDP learning method for digit recognition applications. Additionally, we observe that the proposed learning model automatically encodes selective attention towards relevant features in the input data while eliminating the influence of background noise (or denoising) further improving the robustness of the ASP learning.
Distance-Sensitive hashing
Venue: arXiv cs.DS 2018
Year: 2018

Locality-sensitive hashing (LSH) is an important tool for managing high-dimensional noisy or uncertain data, for example in connection with data cleaning (similarity join) and noise-robust search (similarity search). However, for a number of problems the LSH framework is not known to yield good solutions, and instead ad hoc solutions have been designed for particular similarity and distance measures. For example, this is true for output-sensitive similarity search/join, and for indexes supporting annulus queries that aim to report a point close to a certain given distance from the query point. In this paper we initiate the study of distance-sensitive hashing (DSH), a generalization of LSH that seeks a family of hash functions such that the probability of two points having the same hash value is a given function of the distance between them. More precisely, given a distance space $(X, \text{dist})$ and a "collision probability function" (CPF) $f\colon \mathbb{R}\rightarrow [0,1]$ we seek a distribution over pairs of functions $(h,g)$ such that for every pair of points $x, y \in X$ the collision probability is $\Pr[h(x)=g(y)] = f(\text{dist}(x,y))$. Locality-sensitive hashing is the study of how fast a CPF can decrease as the distance grows. For many spaces, $f$ can be made exponentially decreasing even if we restrict attention to the symmetric case where $g=h$. We show that the asymmetry achieved by having a pair of functions makes it possible to achieve CPFs that are, for example, increasing or unimodal, and show how this leads to principled solutions to problems not addressed by the LSH framework. This includes a novel application to privacy-preserving distance estimation. We believe that the DSH framework will find further applications in high-dimensional data management.
Dimensional-invariance principles in coupled dynamical systems-- A unified analysis and applications
Venue: arXiv cs.SY 2018
Year: 2018

In this paper we study coupled dynamical systems and investigate dimension properties of the subspace spanned by solutions of each individual system. Relevant problems on \textit{collinear dynamical systems} and their variations are discussed recently by Montenbruck et. al. in \cite{collinear2017SCL}, while in this paper we aim to provide a unified analysis to derive the dimensional-invariance principles for networked coupled systems, and to generalize the invariance principles for networked systems with more general forms of coupling terms. To be specific, we consider two types of coupled systems, one with scalar couplings and the other with matrix couplings. Via the \textit{rank-preserving flow theory}, we show that any scalar-coupled dynamical system (with constant, time-varying or state-dependent couplings) possesses the dimensional-invariance principles, in that the dimension of the subspace spanned by the individual systems' solutions remains invariant. For coupled dynamical systems with matrix coefficients/couplings, necessary and sufficient conditions (for constant, time-varying and state-dependent couplings) are given to characterize dimensional-invariance principles. The proofs via a rank-preserving matrix flow theory in this paper simplify the analysis in \cite{collinear2017SCL}, and we also extend the invariance principles to the cases of time-varying couplings and state-dependent couplings. Furthermore, subspace-preserving property and signature-preserving flows are also developed for coupled networked systems with particular coupling terms. These invariance principles provide insightful characterizations to analyze transient behaviors and solution evolutions for a large family of coupled systems, such as multi-agent consensus dynamics, distributed coordination systems, formation control systems, among others.