OpenRaiser/Intern-Atlas huggingface dataset:
DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
Venue: arXiv cs.LG 2020
Year: 2020

While it is widely known that neural networks are universal approximators of continuous functions, a less known and perhaps more powerful result is that a neural network with a single hidden layer can approximate accurately any nonlinear continuous operator. This universal approximation theorem is suggestive of the potential application of neural networks in learning nonlinear operators from data. However, the theorem guarantees only a small approximation error for a sufficient large network, and does not consider the important optimization and generalization errors. To realize this theorem in practice, we propose deep operator networks (DeepONets) to learn operators accurately and efficiently from a relatively small dataset. A DeepONet consists of two sub-networks, one for encoding the input function at a fixed number of sensors $x_i, i=1,\dots,m$ (branch net), and another for encoding the locations for the output functions (trunk net). We perform systematic simulations for identifying two types of operators, i.e., dynamic systems and partial differential equations, and demonstrate that DeepONet significantly reduces the generalization error compared to the fully-connected networks. We also derive theoretically the dependence of the approximation error in terms of the number of sensors (where the input function is defined) as well as the input function type, and we verify the theorem with computational results. More importantly, we observe high-order error convergence in our computational tests, namely polynomial rates (from half order to fourth order) and even exponential convergence with respect to the training dataset size.
A Distributed Ledger Based Infrastructure for Smart Transportation System and Social Good
Venue: arXiv cs.CR 2020
Year: 2020

This paper presents a system architecture to promote the development of smart transportation systems. Thanks to the use of distributed ledgers and related technologies, it is possible to create, store and share data generated by users through their sensors, while moving. In particular, IOTA and IPFS are used to store and certify data (and their related metadata) coming from sensors or by the users themselves. Ethereum is exploited as the smart contract platform that coordinates the data sharing and provisioning. The necessary privacy guarantees are provided by the usage of Zero Knowledge Proof. We show some results obtained from some use case scenarios that demonstrate how such technologies can be integrated to build novel smart services and to promote social good in user mobility.
Automatic Construction of Multi-layer Perceptron Network from Streaming Examples
Venue: arXiv cs.LG 2020
Year: 2020

Autonomous construction of deep neural network (DNNs) is desired for data streams because it potentially offers two advantages: proper model's capacity and quick reaction to drift and shift. While the self-organizing mechanism of DNNs remains an open issue, this task is even more challenging to be developed for standard multi-layer DNNs than that using the different-depth structures, because the addition of a new layer results in information loss of previously trained knowledge. A Neural Network with Dynamically Evolved Capacity (NADINE) is proposed in this paper. NADINE features a fully open structure where its network structure, depth and width, can be automatically evolved from scratch in an online manner and without the use of problem-specific thresholds. NADINE is structured under a standard MLP architecture and the catastrophic forgetting issue during the hidden layer addition phase is resolved using the proposal of soft-forgetting and adaptive memory methods. The advantage of NADINE, namely elastic structure and online learning trait, is numerically validated using nine data stream classification and regression problems where it demonstrates performance improvement over prominent algorithms in all problems. In addition, it is capable of dealing with data stream regression and classification problems equally well.
Differentially private anonymized histograms
Venue: arXiv cs.LG 2020
Year: 2020

For a dataset of label-count pairs, an anonymized histogram is the multiset of counts. Anonymized histograms appear in various potentially sensitive contexts such as password-frequency lists, degree distribution in social networks, and estimation of symmetric properties of discrete distributions. Motivated by these applications, we propose the first differentially private mechanism to release anonymized histograms that achieves near-optimal privacy utility trade-off both in terms of number of items and the privacy parameter. Further, if the underlying histogram is given in a compact format, the proposed algorithm runs in time sub-linear in the number of items. For anonymized histograms generated from unknown discrete distributions, we show that the released histogram can be directly used for estimating symmetric properties of the underlying distribution.
Counterexamples on the monotonicity of delay optimal strategies for energy harvesting transmitters
Venue: arXiv cs.IT 2020
Year: 2020

We consider cross-layer design of delay optimal transmission strategies for energy harvesting transmitters where the data and energy arrival processes are stochastic. Using Markov decision theory, we show that the value function is weakly increasing in the queue state and weakly decreasing in the battery state. It is natural to expect that the delay optimal policy should be weakly increasing in the queue and battery states. We show via counterexamples that this is not the case. In fact, we show that for some sample scenarios the delay optimal policy may perform 5-13% better than the best monotone policy.
Deep Latent Defence
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning methods have shown state of the art performance in a range of tasks from computer vision to natural language processing. However, it is well known that such systems are vulnerable to attackers who craft inputs in order to cause misclassification. The level of perturbation an attacker needs to introduce in order to cause such a misclassification can be extremely small, and often imperceptible. This is of significant security concern, particularly where misclassification can cause harm to humans. We thus propose Deep Latent Defence, an architecture which seeks to combine adversarial training with a detection system. At its core Deep Latent Defence has a adversarially trained neural network. A series of encoders take the intermediate layer representation of data as it passes though the network and project it to a latent space which we use for detecting adversarial samples via a $k$-nn classifier. We present results using both grey and white box attackers, as well as an adaptive $L_{\infty}$ bounded attack which was constructed specifically to try and evade our defence. We find that even under the strongest attacker model that we have investigated our defence is able to offer significant defensive benefits.
Edge crossings in random linear arrangements
Venue: arXiv cs.DM 2020
Year: 2020

In spatial networks vertices are arranged in some space and edges may cross. When arranging vertices in a 1-dimensional lattice edges may cross when drawn above the vertex sequence as it happens in linguistic and biological networks. Here we investigate the general of problem of the distribution of edge crossings in random arrangements of the vertices. We generalize the existing formula for the expectation of this number in random linear arrangements of trees to any network and derive an expression for the variance of the number of crossings in an arbitrary layout relying on a novel characterization of the algebraic structure of that variance in an arbitrary space. We provide compact formulae for the expectation and the variance in complete graphs, complete bipartite graphs, cycle graphs, one-regular graphs and various kinds of trees (star trees, quasi-star trees and linear trees). In these networks, the scaling of expectation and variance as a function of network size is asymptotically power-law-like in random linear arrangements. Our work paves the way for further research and applications in 1-dimension or investigating the distribution of the number of crossings in lattices of higher dimension or other embeddings.
Hierarchical Deep Double Q-Routing
Venue: arXiv cs.NI 2020
Year: 2020

This paper explores a deep reinforcement learning approach applied to the packet routing problem with high-dimensional constraints instigated by dynamic and autonomous communication networks. Our approach is motivated by the fact that centralized path calculation approaches are often not scalable, whereas the distributed approaches with locally acting nodes are not fully aware of the end-to-end performance. We instead hierarchically distribute the path calculation over designated nodes in the network while taking into account the end-to-end performance. Specifically, we develop a hierarchical cluster-oriented adaptive per-flow path calculation mechanism by leveraging the Deep Double Q-network (DDQN) algorithm, where the end-to-end paths are calculated by the source nodes with the assistance of cluster (group) leaders at different hierarchical levels. In our approach, a deferred composite reward is designed to capture the end-to-end performance through a feedback signal from the source nodes to the group leaders and captures the local network performance through the local resource assessments by the group leaders. This approach scales in large networks, adapts to the dynamic demand, utilizes the network resources efficiently and can be applied to segment routing.
Cost-aware Targeted Viral Marketing: Approximation with Less Samples
Venue: arXiv cs.DS 2020
Year: 2020

Cost-aware Targeted Viral Marketing (CTVM), a generalization of Influence Maximization (IM), has received a lot of attentions recently due to its commercial values. Previous approximation algorithms for this problem required a large number of samples to ensure approximate guarantee. In this paper, we propose an efficient approximation algorithm which uses fewer samples but provides the same theoretical guarantees based on generating and using important samples in its operation. Experiments on real social networks show that our proposed method outperforms the state-of-the-art algorithm which provides the same approximation ratio in terms of the number of required samples and running time.
Reversing Place Transition Nets
Venue: arXiv cs.LO 2020
Year: 2020

Petri nets are a well-known model of concurrency and provide an ideal setting for the study of fundamental aspects in concurrent systems. Despite their simplicity, they still lack a satisfactory causally reversible semantics. We develop such semantics for Place/Transitions Petri nets (P/T nets) based on two observations. Firstly, a net that explicitly expresses causality and conflict among events, for example an occurrence net, can be straightforwardly reversed by adding a reverse transition for each of its forward transitions. Secondly, given a P/T net the standard unfolding construction associates with it an occurrence net that preserves all of its computation. Consequently, the reversible semantics of a P/T net can be obtained as the reversible semantics of its unfolding. We show that such reversible behaviour can be expressed as a finite net whose tokens are coloured by causal histories. Colours in our encoding resemble the causal memories that are typical in reversible process calculi.
Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (GANs)
Venue: arXiv cs.NE 2020
Year: 2020

Recently, increasing works have proposed to drive evolutionary algorithms using machine learning models. Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models. Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality. To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial networks (GANs). At each generation of the proposed algorithm, the parent solutions are first classified into real and fake samples to train the GANs; then the offspring solutions are sampled by the trained GANs. Thanks to the powerful generative ability of the GANs, our proposed algorithm is capable of generating promising offspring solutions in high-dimensional decision space with limited training data. The proposed algorithm is tested on 10 benchmark problems with up to 200 decision variables. Experimental results on these test problems demonstrate the effectiveness of the proposed algorithm.
A Nonparametric Bayesian Model for Sparse Dynamic Multigraphs
Venue: arXiv cs.LG 2020
Year: 2020

As the availability and importance of temporal interaction data--such as email communication--increases, it becomes increasingly important to understand the underlying structure that underpins these interactions. Often these interactions form a multigraph, where we might have multiple interactions between two entities. Such multigraphs tend to be sparse yet structured, and their distribution often evolves over time. Existing statistical models with interpretable parameters can capture some, but not all, of these properties. We propose a dynamic nonparametric model for interaction multigraphs that combines the sparsity of edge-exchangeable multigraphs with dynamic clustering patterns that tend to reinforce recent behavioral patterns. We show that our method yields improved held-out likelihood over stationary variants, and impressive predictive performance against a range of state-of-the-art dynamic graph models.
Deep Learning Predictive Band Switching in Wireless Networks
Venue: arXiv cs.NI 2020
Year: 2020

In cellular systems, the user equipment (UE) can request a change in the frequency band when its rate drops below a threshold on the current band. The UE is then instructed by the base station (BS) to measure the quality of candidate bands, which requires a measurement gap in the data transmission, thus lowering the data rate. We propose an online-learning based band switching approach that does not require any measurement gap. Our proposed classifier-based band switching policy instead exploits spatial and spectral correlation between radio frequency signals in different bands based on knowledge of the UE location. We focus on switching between a lower (e.g., 3.5 GHz) band and a millimeter wave band (e.g., 28 GHz), and design and evaluate two classification models that are trained on a ray-tracing dataset. A key insight is that measurement gaps are overkill, in that only the relative order of the bands is necessary for band selection, rather than a full channel estimate. Our proposed machine learning based policies achieve roughly 30% improvement in mean effective rates over those of the industry standard policy, while achieving misclassification errors well below 0.5% and maintaining resilience against blockage uncertainty.
Spectral Efficiency of One-Bit Sigma-Delta Massive MIMO
Venue: arXiv cs.IT 2020
Year: 2020

We examine the uplink spectral efficiency of a massive MIMO base station employing a one-bit Sigma-Delta sampling scheme implemented in the spatial rather than the temporal domain. Using spatial rather than temporal oversampling, and feedback of the quantization error between adjacent antennas, the method shapes the spatial spectrum of the quantization noise away from an angular sector where the signals of interest are assumed to lie. It is shown that, while a direct Bussgang analysis of the Sigma-Delta approach is not suitable, an alternative equivalent linear model can be formulated to facilitate an analysis of the system performance. The theoretical properties of the spatial quantization noise power spectrum are derived for the Sigma-Delta array, as well as an expression for the spectral efficiency of maximum ratio combining (MRC). Simulations verify the theoretical results and illustrate the significant performance gains offered by the Sigma-Delta approach for both MRC and zero-forcing receivers.
Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction
Venue: arXiv cs.IR 2020
Year: 2020

Click-through rate (CTR) prediction is an essential task in web applications such as online advertising and recommender systems, whose features are usually in multi-field form. The key of this task is to model feature interactions among different feature fields. Recently proposed deep learning based models follow a general paradigm: raw sparse input multi-filed features are first mapped into dense field embedding vectors, and then simply concatenated together to feed into deep neural networks (DNN) or other specifically designed networks to learn high-order feature interactions. However, the simple \emph{unstructured combination} of feature fields will inevitably limit the capability to model sophisticated interactions among different fields in a sufficiently flexible and explicit fashion. In this work, we propose to represent the multi-field features in a graph structure intuitively, where each node corresponds to a feature field and different fields can interact through edges. The task of modeling feature interactions can be thus converted to modeling node interactions on the corresponding graph. To this end, we design a novel model Feature Interaction Graph Neural Networks (Fi-GNN). Taking advantage of the strong representative power of graphs, our proposed model can not only model sophisticated feature interactions in a flexible and explicit fashion, but also provide good model explanations for CTR prediction. Experimental results on two real-world datasets show its superiority over the state-of-the-arts.
Quantum-Inspired Classical Algorithms for Singular Value Transformation
Venue: arXiv cs.DS 2020
Year: 2020

A recent breakthrough by Tang (STOC 2019) showed how to "dequantize" the quantum algorithm for recommendation systems by Kerenidis and Prakash (ITCS 2017). The resulting algorithm, classical but "quantum-inspired", efficiently computes a low-rank approximation of the users' preference matrix. Subsequent works have shown how to construct efficient quantum-inspired algorithms for approximating the pseudo-inverse of a low-rank matrix as well, which can be used to (approximately) solve low-rank linear systems of equations. In the present paper, we pursue this line of research and develop quantum-inspired algorithms for a large class of matrix transformations that are defined via the singular value decomposition of the matrix. In particular, we obtain classical algorithms with complexity polynomially related (in most parameters) to the complexity of the best quantum algorithms for singular value transformation recently developed by Chakraborty, Gilyén and Jeffery (ICALP 2019) and Gilyén, Su, Low and Wiebe (STOC19).
Contour Sparse Representation with SDD Features for Object Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Slope difference distribution (SDD) is computed for the one-dimensional curve. It is not only robust to calculate the partitioning point to separate the curve logically, but also robust to calculate the clustering center of each part of the separated curve. SDD has been proposed for image segmentation and it outperforms all existing image segmentation methods. For verification purpose, we have made the Matlab codes of comparing SDD method with existing image segmentation methods freely available at Matlab Central. The contour of the object is similar to the histogram of the image. Thus, feature detection by SDD from the contour of the object is also feasible. In this letter, SDD features are defined and they form the sparse representation of the object contour. The reference model of each object is built based on the SDD features and then model matching is used for on line object recognition. The experimental results are very encouraging. For the gesture recognition, SDD achieved 100% accuracy for two public datasets: the NUS dataset and the near-infrared dataset. For the object recognition, SDD achieved 100% accuracy for the Kimia 99 dataset.
LiveGraph: A Transactional Graph Storage System with Purely Sequential Adjacency List Scans
Venue: arXiv cs.DB 2020
Year: 2020

The specific characteristics of graph workloads make it hard to design a one-size-fits-all graph storage system. Systems that support transactional updates use data structures with poor data locality, which limits the efficiency of analytical workloads or even simple edge scans. Other systems run graph analytics workloads efficiently, but cannot properly support transactions. This paper presents LiveGraph, a graph storage system that outperforms both the best graph transactional systems and the best systems for real-time graph analytics on fresh data. LiveGraph does that by ensuring that adjacency list scans, a key operation in graph workloads, are purely sequential: they never require random accesses even in presence of concurrent transactions. This is achieved by combining a novel graph-aware data structure, the Transactional Edge Log (TEL), together with a concurrency control mechanism that leverages TEL's data layout. Our evaluation shows that LiveGraph significantly outperforms state-of-the-art (graph) database solutions on both transactional and real-time analytical workloads.
Towards Efficient Discrete Integration via Adaptive Quantile Queries
Venue: arXiv cs.LG 2020
Year: 2020

Discrete integration in a high dimensional space of n variables poses fundamental challenges. The WISH algorithm reduces the intractable discrete integration problem into n optimization queries subject to randomized constraints, obtaining a constant approximation guarantee. The optimization queries are expensive, which limits the applicability of WISH. We propose AdaWISH, which is able to obtain the same guarantee but accesses only a small subset of queries of WISH. For example, when the number of function values is bounded by a constant, AdaWISH issues only O(log n) queries. The key idea is to query adaptively, taking advantage of the shape of the weight function being integrated. In general, we prove that AdaWISH has a regret of only O(log n) relative to an idealistic oracle that issues queries at data-dependent optimal points. Experimentally, AdaWISH gives precise estimates for discrete integration problems, of the same quality as that of WISH and better than several competing approaches, on a variety of probabilistic inference benchmarks. At the same time, it saves substantially on the number of optimization queries compared to WISH. On a suite of UAI inference challenge benchmarks, it saves 81.5% of WISH queries while retaining the quality of results.
DISCERN: Diversity-based Selection of Centroids for k-Estimation and Rapid Non-stochastic Clustering
Venue: arXiv cs.LG 2020
Year: 2020

One of the applications of center-based clustering algorithms such as K-Means is partitioning data points into K clusters. In some examples, the feature space relates to the underlying problem we are trying to solve, and sometimes we can obtain a suitable feature space. Nevertheless, while K-Means is one of the most efficient offline clustering algorithms, it is not equipped to estimate the number of clusters, which is useful in some practical cases. Other practical methods which do are simply too complex, as they require at least one run of K-Means for each possible K. In order to address this issue, we propose a K-Means initialization similar to K-Means++, which would be able to estimate K based on the feature space while finding suitable initial centroids for K-Means in a deterministic manner. Then we compare the proposed method, DISCERN, with a few of the most practical K estimation methods, while also comparing clustering results of K-Means when initialized randomly, using K-Means++ and using DISCERN. The results show improvement in both the estimation and final clustering performance.