OpenRaiser/Intern-Atlas huggingface dataset:
Spatial Positioning Token (SPToken) for Smart Mobility
Venue: arXiv cs.CR 2020
Year: 2020

We introduce a permissioned distributed ledger technology (DLT) design for crowdsourced smart mobility applications. This architecture is based on a directed acyclic graph architecture (similar to the IOTA tangle) and uses both Proof-of-Work and Proof-of-Position mechanisms to provide protection against spam attacks and malevolent actors. In addition to enabling individuals to retain ownership of their data and to monetize it, the architecture also is suitable for distributed privacy-preserving machine learning algorithms, is lightweight, and can be implemented in simple internet-of-things (IoT) devices. To demonstrate its efficacy, we apply this framework to reinforcement learning settings where a third party is interested in acquiring information from agents. In particular, one may be interested in sampling an unknown vehicular traffic flow in a city, using a DLT-type architecture and without perturbing the density, with the idea of realizing a set of virtual tokens as surrogates of real vehicles to explore geographical areas of interest. These tokens, whose authenticated position determines write access to the ledger, are thus used to emulate the probing actions of commanded (real) vehicles on a given planned route by "jumping" from a passing-by vehicle to another to complete the planned trajectory. Consequently, the environment stays unaffected (i.e., the autonomy of participating vehicles is not influenced by the algorithm), regardless of the number of emitted tokens. The design of such a DLT architecture is presented, and numerical results from large-scale simulations are provided to validate the proposed approach.
A type of generalization error induced by initialization in deep neural networks
Venue: arXiv cs.LG 2020
Year: 2020

How initialization and loss function affect the learning of a deep neural network (DNN), specifically its generalization error, is an important problem in practice. In this work, by exploiting the linearity of DNN training dynamics in the NTK regime \citep{jacot2018neural,lee2019wide}, we provide an explicit and quantitative answer to this problem. Focusing on regression problem, we prove that, in the NTK regime, for any loss in a general class of functions, the DNN finds the same \emph{global} minima---the one that is nearest to the initial value in the parameter space, or equivalently, the one that is closest to the initial DNN output in the corresponding reproducing kernel Hilbert space. Using these optimization problems, we quantify the impact of initial output and prove that a random non-zero one increases the generalization error. We further propose an antisymmetrical initialization (ASI) trick that eliminates this type of error and accelerates the training. To understand whether the above results hold in general, we also perform experiments for DNNs in the non-NTK regime, which demonstrate the effectiveness of our theoretical results and the ASI trick in a qualitative sense. Overall, our work serves as a baseline for the further investigation of the impact of initialization and loss function on the generalization of DNNs, which can potentially guide and improve the training of DNNs in practice.
Testing DNN Image Classifiers for Confusion & Bias Errors
Venue: arXiv cs.SE 2020
Year: 2020

Image classifiers are an important component of today's software, from consumer and business applications to safety-critical domains. The advent of Deep Neural Networks (DNNs) is the key catalyst behind such wide-spread success. However, wide adoption comes with serious concerns about the robustness of software systems dependent on DNNs for image classification, as several severe erroneous behaviors have been reported under sensitive and critical circumstances. We argue that developers need to rigorously test their software's image classifiers and delay deployment until acceptable. We present an approach to testing image classifier robustness based on class property violations. We found that many of the reported erroneous cases in popular DNN image classifiers occur because the trained models confuse one class with another or show biases towards some classes over others. These bugs usually violate some class properties of one or more of those classes. Most DNN testing techniques focus on per-image violations, so fail to detect class-level confusions or biases. We developed a testing technique to automatically detect class-based confusion and bias errors in DNN-driven image classification software. We evaluated our implementation, DeepInspect, on several popular image classifiers with precision up to 100% (avg.~72.6%) for confusion errors, and up to 84.3% (avg.~66.8%) for bias errors. DeepInspect found hundreds of classification mistakes in widely-used models, many exposing errors indicating confusion or bias.
T-EDGE: Temporal WEighted MultiDiGraph Embedding for Ethereum Transaction Network Analysis
Venue: arXiv cs.SI 2020
Year: 2020

Recently, graph embedding techniques have been widely used in the analysis of various networks, but most of the existing embedding methods omit the network dynamics and the multiplicity of edges, so it is difficult to accurately describe the detailed characteristics of the transaction networks. Ethereum is a blockchain-based platform supporting smart contracts. The open nature of blockchain makes the transaction data on Ethereum completely public, and also brings unprecedented opportunities for the transaction network analysis. By taking the realistic rules and features of transaction networks into consideration, we first model the Ethereum transaction network as a Temporal Weighted Multidigraph (TWMDG), where each node is a unique Ethereum account and each edge represents a transaction weighted by amount and assigned with timestamp. Then we define the problem of Temporal Weighted Multidigraph Embedding (T-EDGE) by incorporating both temporal and weighted information of the edges, the purpose being to capture more comprehensive properties of dynamic transaction networks. To evaluate the effectiveness of the proposed embedding method, we conduct experiments of node classification on real-world transaction data collected from Ethereum. Experimental results demonstrate that T-EDGE outperforms baseline embedding methods, indicating that time-dependent walks and multiplicity characteristic of edges are informative and essential for time-sensitive transaction networks.
Interpretable Neural Predictions with Differentiable Binary Variables
Venue: arXiv cs.CL 2020
Year: 2020

The success of neural networks comes hand in hand with a desire for more interpretability. We focus on text classifiers and make them more interpretable by having them provide a justification, a rationale, for their predictions. We approach this problem by jointly training two neural network models: a latent model that selects a rationale (i.e. a short and informative part of the input text), and a classifier that learns from the words in the rationale alone. Previous work proposed to assign binary latent masks to input positions and to promote short selections via sparsity-inducing penalties such as L0 regularisation. We propose a latent model that mixes discrete and continuous behaviour allowing at the same time for binary selections and gradient-based training without REINFORCE. In our formulation, we can tractably compute the expected value of penalties such as L0, which allows us to directly optimise the model towards a pre-specified text selection rate. We show that our approach is competitive with previous work on rationale extraction, and explore further uses in attention mechanisms.
Mutual Information Maximization in Graph Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

A variety of graph neural networks (GNNs) frameworks for representation learning on graphs have been recently developed. These frameworks rely on aggregation and iteration scheme to learn the representation of nodes. However, information between nodes is inevitably lost in the scheme during learning. In order to reduce the loss, we extend the GNNs frameworks by exploring the aggregation and iteration scheme in the methodology of mutual information. We propose a new approach of enlarging the normal neighborhood in the aggregation of GNNs, which aims at maximizing mutual information. Based on a series of experiments conducted on several benchmark datasets, we show that the proposed approach improves the state-of-the-art performance for four types of graph tasks, including supervised and semi-supervised graph classification, graph link prediction and graph edge generation and classification.
SoK: Tools for Game Theoretic Models of Security for Cryptocurrencies
Venue: arXiv cs.CR 2020
Year: 2020

Cryptocurrencies have garnered much attention in recent years, both from the academic community and industry. One interesting aspect of cryptocurrencies is their explicit consideration of incentives at the protocol level. Understanding how to incorporate this into the models used to design cryptocurrencies has motivated a large body of work, yet many open problems still exist and current systems rarely deal with incentive related problems well. This issue arises due to the gap between Cryptography and Distributed Systems security, which deals with traditional security problems that ignore the explicit consideration of incentives, and Game Theory, which deals best with situations involving incentives. With this work, we aim to offer a systematization of the work that relates to this problem, considering papers that blend Game Theory with Cryptography or Distributed systems and discussing how they can be related. This gives an overview of the available tools, and we look at their (potential) use in practice, in the context of existing blockchain based systems that have been proposed or implemented.
Learning Conserved Networks from Flows
Venue: arXiv cs.LG 2020
Year: 2020

A challenging problem in complex networks is the network reconstruction problem from data. This work deals with a class of networks denoted as conserved networks, in which a flow associated with every edge and the flows are conserved at all non-source and non-sink nodes. We propose a novel polynomial time algorithm to reconstruct conserved networks from flow data by exploiting graph theoretic properties of conserved networks combined with learning techniques. We prove that exact network reconstruction is possible for arborescence networks. We also extend the methodology for reconstructing networks from noisy data and explore the reconstruction performance on arborescence networks with different structural characteristics.
Unsupervised Linear and Nonlinear Channel Equalization and Decoding using Variational Autoencoders
Venue: arXiv cs.LG 2020
Year: 2020

A new approach for blind channel equalization and decoding, variational inference, and variational autoencoders (VAEs) in particular, is introduced. We first consider the reconstruction of uncoded data symbols transmitted over a noisy linear intersymbol interference (ISI) channel, with an unknown impulse response, without using pilot symbols. We derive an approximate maximum likelihood estimate to the channel parameters and reconstruct the transmitted data. We demonstrate significant and consistent improvements in the error rate of the reconstructed symbols, compared to existing blind equalization methods such as constant modulus, thus enabling faster channel acquisition. The VAE equalizer uses a convolutional neural network with a small number of free parameters. These results are extended to blind equalization over a noisy nonlinear ISI channel with unknown parameters. We then consider coded communication using low-density parity-check (LDPC) codes transmitted over a noisy linear or nonlinear ISI channel. The goal is to reconstruct the transmitted message from the channel observations corresponding to a transmitted codeword, without using pilot symbols. We demonstrate improvements compared to the expectation maximization (EM) algorithm using turbo equalization. Furthermore, unlike EM, the computational complexity of our method does not have exponential dependence on the size of the channel impulse response.
Neural-Symbolic Argumentation Mining: an Argument in Favor of Deep Learning and Reasoning
Venue: arXiv cs.AI 2020
Year: 2020

Deep learning is bringing remarkable contributions to the field of argumentation mining, but the existing approaches still need to fill the gap toward performing advanced reasoning tasks. In this position paper, we posit that neural-symbolic and statistical relational learning could play a crucial role in the integration of symbolic and sub-symbolic methods to achieve this goal.
Unknown mixing times in apprenticeship and reinforcement learning
Venue: arXiv cs.LG 2020
Year: 2020

We derive and analyze learning algorithms for apprenticeship learning, policy evaluation, and policy gradient for average reward criteria. Existing algorithms explicitly require an upper bound on the mixing time. In contrast, we build on ideas from Markov chain theory and derive sampling algorithms that do not require such an upper bound. For these algorithms, we provide theoretical bounds on their sample-complexity and running time.
A Smoothness Energy without Boundary Distortion for Curved Surfaces
Venue: arXiv cs.GR 2020
Year: 2020

Current quadratic smoothness energies for curved surfaces either exhibit distortions near the boundary due to zero Neumann boundary conditions, or they do not correctly account for intrinsic curvature, which leads to unnatural-looking behavior away from the boundary. This leads to an unfortunate trade-off: one can either have natural behavior in the interior, or a distortion-free result at the boundary, but not both. We introduce a generalized Hessian energy for curved surfaces, expressed in terms of the covariant one-form Dirichlet energy, the Gaussian curvature, and the exterior derivative. Energy minimizers solve the Laplace-Beltrami biharmonic equation, correctly accounting for intrinsic curvature, leading to natural-looking isolines. On the boundary, minimizers are as-linear-as-possible, which reduces the distortion of isolines at the boundary. We discretize the covariant one-form Dirichlet energy using Crouzeix-Raviart finite elements, arriving at a discrete formulation of the Hessian energy for applications on curved surfaces. We observe convergence of the discretization in our experiments.
The Effectiveness of Uniform Sampling for Center-Based Clustering with Outliers
Venue: arXiv cs.DS 2020
Year: 2020

Clustering has many important applications in computer science, but real-world datasets often contain outliers. Moreover, the presence of outliers can make the clustering problems to be much more challenging. To reduce the complexities, various sampling methods have been proposed in past years. Namely, we take a small sample (uniformly or non-uniformly) from input and run an existing approximation algorithm on the sample. Comparing with existing non-uniform sampling methods, the uniform sampling approach has several significant benefits. For example, it only needs to read the data in one-pass and is very easy to implement in practice. Thus, the effectiveness of uniform sampling for clustering with outliers is a natural and fundamental problem deserving to study in both theory and practice. In this paper, we propose a new and unified framework for analyzing the effectiveness of uniform sampling for three representative center-based clustering with outliers problems, $k$-center/median/means clustering with outliers. We introduce a "significance" criterion and prove that the performance of uniform sampling depends on the significance degree of the given instance. In particular, we show that the sample size can be independent of the ratio $n/z$ and the dimensionality. More importantly, to the best of our knowledge, our method is the first uniform sampling approach that allows to discard exactly $z$ outliers for these three center-based clustering with outliers problems. The results proposed in this paper also can be viewed as an extension of the previous sub-linear time algorithms for the ordinary clustering problems (without outliers). The experiments suggest that the uniform sampling method can achieve comparable clustering results with other existing methods, but greatly reduce the running times.
Preventing wind turbine tower natural frequency excitation with a quasi-LPV model predictive control scheme
Venue: arXiv cs.SY 2020
Year: 2020

With the ever increasing power rates of wind turbines, more advanced control techniques are needed to facilitate tall towers that are low-weight and cost effective, but in effect more flexible. Such soft-soft tower configurations generally have their fundamental side-side frequency in the below-rated operational domain. Because the turbine rotor practically has or develops a mass imbalance over time, a periodic and rotor-speed dependent side-side excitation is present during below-rated operation. Persistent operation at the coinciding tower and rotational frequency degrades the expected structural life span. To reduce this effect, earlier work has shown the effectiveness of active tower damping control strategies using collective pitch control. A more passive approach is frequency skipping by inclusion of speed exclusion zones, which avoids prolonged operation near the critical frequency. However, neither of the methods incorporate a convenient way of performing a trade-off between energy maximization and fatigue load minimization. Therefore, this paper introduces a quasi-linear parameter varying model predictive control (qLPV-MPC) scheme, exploiting the beneficial (convex) properties of a qLPV system description. The qLPV model is obtained by a demodulation transformation, and is subsequently augmented with a simple wind turbine model. Results show the effectiveness of the algorithm in synthetic and realistic simulations using the NREL 5-MW reference wind turbine in high-fidelity simulation code. Prolonged rotor speed operation at the tower side-side natural frequency is prevented, whereas when the trade-off is in favor of energy production, the algorithm decides to rapidly pass over the natural frequency to attain higher rotor speeds and power productions.
Uncertainty Estimation in One-Stage Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Environment perception is the task for intelligent vehicles on which all subsequent steps rely. A key part of perception is to safely detect other road users such as vehicles, pedestrians, and cyclists. With modern deep learning techniques huge progress was made over the last years in this field. However such deep learning based object detection models cannot predict how certain they are in their predictions, potentially hampering the performance of later steps such as tracking or sensor fusion. We present a viable approaches to estimate uncertainty in an one-stage object detector, while improving the detection performance of the baseline approach. The proposed model is evaluated on a large scale automotive pedestrian dataset. Experimental results show that the uncertainty outputted by our system is coupled with detection accuracy and the occlusion level of pedestrians.
Adversarial Learned Molecular Graph Inference and Generation
Venue: arXiv cs.LG 2020
Year: 2020

Recent methods for generating novel molecules use graph representations of molecules and employ various forms of graph convolutional neural networks for inference. However, training requires solving an expensive graph isomorphism problem, which previous approaches do not address or solve only approximately. In this work, we propose ALMGIG, a likelihood-free adversarial learning framework for inference and de novo molecule generation that avoids explicitly computing a reconstruction loss. Our approach extends generative adversarial networks by including an adversarial cycle-consistency loss to implicitly enforce the reconstruction property. To capture properties unique to molecules, such as valence, we extend the Graph Isomorphism Network to multi-graphs. To quantify the performance of models, we propose to compute the distance between distributions of physicochemical properties with the 1-Wasserstein distance. We demonstrate that ALMGIG more accurately learns the distribution over the space of molecules than all baselines. Moreover, it can be utilized for drug discovery by efficiently searching the space of molecules using molecules' continuous latent representation. Our code is available at https://github.com/ai-med/almgig
Improved object recognition using neural networks trained to mimic the brain's statistical properties
Venue: arXiv cs.CV 2020
Year: 2020

The current state-of-the-art object recognition algorithms, deep convolutional neural networks (DCNNs), are inspired by the architecture of the mammalian visual system, and are capable of human-level performance on many tasks. However, even these algorithms make errors. As they are trained for object recognition tasks, it has been shown that DCNNs develop hidden representations that resemble those observed in the mammalian visual system. Moreover, DCNNs trained on object recognition tasks are currently among the best models we have of the mammalian visual system. This led us to hypothesize that teaching DCNNs to achieve even more brain-like representations could improve their performance. To test this, we trained DCNNs on a composite task, wherein networks were trained to: a) classify images of objects; while b) having intermediate representations that resemble those observed in neural recordings from monkey visual cortex. Compared with DCNNs trained purely for object categorization, DCNNs trained on the composite task had better object recognition performance and are more robust to label corruption. Interestingly, we also found that neural data was not required, but randomized data with the same statistics as neural data also boosted performance. Our results outline a new way to train object recognition networks, using strategies in which the brain - or at least the statistical properties of its activation patterns - serves as a teacher signal for training DCNNs.
MDE: Multiple Distance Embeddings for Link Prediction in Knowledge Graphs
Venue: arXiv cs.AI 2020
Year: 2020

Over the past decade, knowledge graphs became popular for capturing structured domain knowledge. Relational learning models enable the prediction of missing links inside knowledge graphs. More specifically, latent distance approaches model the relationships among entities via a distance between latent representations. Translating embedding models (e.g., TransE) are among the most popular latent distance approaches which use one distance function to learn multiple relation patterns. However, they are mostly inefficient in capturing symmetric relations since the representation vector norm for all the symmetric relations becomes equal to zero. They also lose information when learning relations with reflexive patterns since they become symmetric and transitive. We propose the Multiple Distance Embedding model (MDE) that addresses these limitations and a framework to collaboratively combine variant latent distance-based terms. Our solution is based on two principles: 1) we use a limit-based loss instead of a margin ranking loss and, 2) by learning independent embedding vectors for each of the terms we can collectively train and predict using contradicting distance terms. We further demonstrate that MDE allows modeling relations with (anti)symmetry, inversion, and composition patterns. We propose MDE as a neural network model that allows us to map non-linear relations between the embedding vectors and the expected output of the score function. Our empirical results show that MDE performs competitively to state-of-the-art embedding models on several benchmark datasets.