OpenRaiser/Intern-Atlas huggingface dataset:
Learning Global and Local Consistent Representations for Unsupervised Image Retrieval via Deep Graph Diffusion Networks
Venue: arXiv cs.CV 2020
Year: 2020

Diffusion has shown great success in improving accuracy of unsupervised image retrieval systems by utilizing high-order structures of image manifold. However, existing diffusion methods suffer from three major limitations: 1) they usually rely on local structures without considering global manifold information; 2) they focus on improving pair-wise similarities within existing images input output transductively while lacking flexibility to learn representations for novel unseen instances inductively; 3) they fail to scale to large datasets due to prohibitive memory consumption and computational burden due to intrinsic high-order operations on the whole graph. In this paper, to address these limitations, we propose a novel method, Graph Diffusion Networks (GRAD-Net), that adopts graph neural networks (GNNs), a novel variant of deep learning algorithms on irregular graphs. GRAD-Net learns semantic representations by exploiting both local and global structures of image manifold in an unsupervised fashion. By utilizing sparse coding techniques, GRAD-Net not only preserves global information on the image manifold, but also enables scalable training and efficient querying. Experiments on several large benchmark datasets demonstrate effectiveness of our method over state-of-the-art diffusion algorithms for unsupervised image retrieval.
Scalable Gradients for Stochastic Differential Equations
Venue: arXiv cs.LG 2020
Year: 2020

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers. Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge. In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations. We use our method to fit stochastic dynamics defined by neural networks, achieving competitive performance on a 50-dimensional motion capture dataset.
Multi-Objective Genetic Programming for Manifold Learning: Balancing Quality and Dimensionality
Venue: arXiv cs.NE 2020
Year: 2020

Manifold learning techniques have become increasingly valuable as data continues to grow in size. By discovering a lower-dimensional representation (embedding) of the structure of a dataset, manifold learning algorithms can substantially reduce the dimensionality of a dataset while preserving as much information as possible. However, state-of-the-art manifold learning algorithms are opaque in how they perform this transformation. Understanding the way in which the embedding relates to the original high-dimensional space is critical in exploratory data analysis. We previously proposed a Genetic Programming method that performed manifold learning by evolving mappings that are transparent and interpretable. This method required the dimensionality of the embedding to be known a priori, which makes it hard to use when little is known about a dataset. In this paper, we substantially extend our previous work, by introducing a multi-objective approach that automatically balances the competing objectives of manifold quality and dimensionality. Our proposed approach is competitive with a range of baseline and state-of-the-art manifold learning methods, while also providing a range (front) of solutions that give different trade-offs between quality and dimensionality. Furthermore, the learned models are shown to often be simple and efficient, utilising only a small number of features in an interpretable manner.
Elastic Bulk Synchronous Parallel Model for Distributed Deep Learning
Venue: arXiv cs.LG 2020
Year: 2020

The bulk synchronous parallel (BSP) is a celebrated synchronization model for general-purpose parallel computing that has successfully been employed for distributed training of machine learning models. A prevalent shortcoming of the BSP is that it requires workers to wait for the straggler at every iteration. To ameliorate this shortcoming of classic BSP, we propose ELASTICBSP a model that aims to relax its strict synchronization requirement. The proposed model offers more flexibility and adaptability during the training phase, without sacrificing on the accuracy of the trained model. We also propose an efficient method that materializes the model, named ZIPLINE. The algorithm is tunable and can effectively balance the trade-off between quality of convergence and iteration throughput, in order to accommodate different environments or applications. A thorough experimental evaluation demonstrates that our proposed ELASTICBSP model converges faster and to a higher accuracy than the classic BSP. It also achieves comparable (if not higher) accuracy than the other sensible synchronization models.
CAE-LO: LiDAR Odometry Leveraging Fully Unsupervised Convolutional Auto-Encoder for Interest Point Detection and Feature Description
Venue: arXiv cs.CV 2020
Year: 2020

As an important technology in 3D mapping, autonomous driving, and robot navigation, LiDAR odometry is still a challenging task. Appropriate data structure and unsupervised deep learning are the keys to achieve an easy adjusted LiDAR odometry solution with high performance. Utilizing compact 2D structured spherical ring projection model and voxel model which preserves the original shape of input data, we propose a fully unsupervised Convolutional Auto-Encoder based LiDAR Odometry (CAE-LO) that detects interest points from spherical ring data using 2D CAE and extracts features from multi-resolution voxel model using 3D CAE. We make several key contributions: 1) experiments based on KITTI dataset show that our interest points can capture more local details to improve the matching success rate on unstructured scenarios and our features outperform state-of-the-art by more than 50% in matching inlier ratio; 2) besides, we also propose a keyframe selection method based on matching pairs transferring, an odometry refinement method for keyframes based on extended interest points from spherical rings, and a backward pose update method. The odometry refinement experiments verify the proposed ideas' feasibility and effectiveness.
High-speed Autonomous Drifting with Deep Reinforcement Learning
Venue: arXiv cs.RO 2020
Year: 2020

Drifting is a complicated task for autonomous vehicle control. Most traditional methods in this area are based on motion equations derived by the understanding of vehicle dynamics, which is difficult to be modeled precisely. We propose a robust drift controller without explicit motion equations, which is based on the latest model-free deep reinforcement learning algorithm soft actor-critic. The drift control problem is formulated as a trajectory following task, where the errorbased state and reward are designed. After being trained on tracks with different levels of difficulty, our controller is capable of making the vehicle drift through various sharp corners quickly and stably in the unseen map. The proposed controller is further shown to have excellent generalization ability, which can directly handle unseen vehicle types with different physical properties, such as mass, tire friction, etc.
Price Competition with LTE-U and WiFi
Venue: arXiv cs.GT 2020
Year: 2020

LTE-U is an extension of the Long Term Evolution (LTE) standard for operation in unlicensed spectrum. LTE-U differs from WiFi, the predominant technology used in unlicensed spectrum in that it utilizes a duty cycle mode for accessing the spectrum and allows for a more seamless integration with LTE deployments in licensed spectrum. There have been a number of technical studies on the co-existence of LTE-U and WiFi in unlicensed spectrum In this paper, we instead investigate the impact of such a technology from an economic perspective. We consider a model in which an incumbent service provider (SP) deploys a duty cycle-based technology like LTE-U in an unlicensed band along with operating in a licensed band and competes with one or more entrants that only operate in the unlicensed band using a different technology like WiFi. We characterize the impact of a technology like LTE-U on the market outcome and show that the welfare impacts of this technology are subtle, depending in part on the amount of unlicensed spectrum and number of entrants. The difference in spectral efficiency between LTE and WiFi also plays a role in the competition among SPs. Finally, we investigate the impact of the duty cycle and the portion of unlicensed spectrum used by the technology.
Constrained Network Slicing Games: Achieving service guarantees and network efficiency
Venue: arXiv cs.NI 2020
Year: 2020

Network slicing is a key capability for next generation mobile networks. It enables one to cost effectively customize logical networks over a shared infrastructure. A critical component of network slicing is resource allocation, which needs to ensure that slices receive the resources needed to support their mobiles/services while optimizing network efficiency. In this paper, we propose a novel approach to slice-based resource allocation named Guaranteed seRvice Efficient nETwork slicing (GREET). The underlying concept is to set up a constrained resource allocation game, where (i) slices unilaterally optimize their allocations to best meet their (dynamic) customer loads, while (ii) constraints are imposed to guarantee that, if they wish so, slices receive a pre-agreed share of the network resources. The resulting game is a variation of the well-known Fisher market, where slices are provided a budget to contend for network resources (as in a traditional Fisher market), but (unlike a Fisher market) prices are constrained for some resources to provide the desired guarantees. In this way, GREET combines the advantages of a share-based approach (high efficiency by flexible sharing) and reservation-based ones (which provide guarantees by assigning a fixed amount of resources). We characterize the Nash equilibrium, best response dynamics, and propose a practical slice strategy with provable convergence properties. Extensive simulations exhibit substantial improvements over network slicing state-of-the-art benchmarks.
Analysis of Selective-Decode and Forward Relaying Protocol Over kappa-mu Fading Channel Distribution
Venue: arXiv cs.NI 2020
Year: 2020

In this work, we examine the performance of selective-decode and forward (S-DF) relay systems over kappa-mu fading channel condition. We discuss about the probability density function (PDF), system model, and cumulative distribution function (CDF) of kappa-mu distributed envelope and signal to noise ratio (SNR) and the techniques to generate samples that follow kappa-mu distribution. Specifically, we consider the case where the source-to-relay (SR), relay-to-destination (RD) and source-to-destination (SD) link is subject to the independent and identically distributed (i.i.d.) kappa-mu fading. From the simulation results, the enhancement in the symbol error rate (SER) with a stronger line of sight (LOS) component is observed. This shows that S-DF relaying systems can perform well even in the non-fading or LOS conditions. Monte Carlo simulations are conducted for various values of fading parameters and the outcomes closely match with theoretical outcomes which validate the derivations.
UNICORN: Runtime Provenance-Based Detector for Advanced Persistent Threats
Venue: arXiv cs.CR 2020
Year: 2020

Advanced Persistent Threats (APTs) are difficult to detect due to their "low-and-slow" attack patterns and frequent use of zero-day exploits. We present UNICORN, an anomaly-based APT detector that effectively leverages data provenance analysis. From modeling to detection, UNICORN tailors its design specifically for the unique characteristics of APTs. Through extensive yet time-efficient graph analysis, UNICORN explores provenance graphs that provide rich contextual and historical information to identify stealthy anomalous activities without pre-defined attack signatures. Using a graph sketching technique, it summarizes long-running system execution with space efficiency to combat slow-acting attacks that take place over a long time span. UNICORN further improves its detection capability using a novel modeling approach to understand long-term behavior as the system evolves. Our evaluation shows that UNICORN outperforms an existing state-of-the-art APT detection system and detects real-life APT scenarios with high accuracy.
A Generalized Framework for Autonomous Calibration of Wheeled Mobile Robots
Venue: arXiv cs.RO 2020
Year: 2020

Robotic calibration allows for the fusion of data from multiple sensors such as odometers, cameras, etc., by providing appropriate transformational relationships between the corresponding reference frames. For wheeled robots equipped with exteroceptive sensors, calibration entails learning the motion model of the sensor or the robot in terms of the odometric data, and must generally be performed prior to performing tasks such as simultaneous localization and mapping (SLAM). Within this context, the current trend is to carry out simultaneous calibration of odometry and sensor without the use of any additional hardware. Building upon the existing simultaneous calibration algorithms, we put forth a generalized calibration framework that can not only handle robots operating in 2D with arbitrary or unknown motion models but also handle outliers in an automated manner. We first propose an algorithm based on the alternating minimization framework applicable to two-wheel differential drive. Subsequently, for arbitrary but known drive configurations we put forth an iteratively re-weighted least squares methodology leveraging an intelligent weighing scheme. Different from the existing works, these proposed algorithms require no manual intervention and seamlessly handle outliers that arise due to both systematic and non-systematic errors. Finally, we put forward a novel Gaussian Process-based non-parametric approach for calibrating wheeled robots with arbitrary or unknown drive configurations. Detailed experiments are performed to demonstrate the accuracy, usefulness, and flexibility of the proposed algorithms.
A Survey on Machine Reading Comprehension Systems
Venue: arXiv cs.CL 2020
Year: 2020

Machine reading comprehension is a challenging task and hot topic in natural language processing. Its goal is to develop systems to answer the questions regarding a given context. In this paper, we present a comprehensive survey on different aspects of machine reading comprehension systems, including their approaches, structures, input/outputs, and research novelties. We illustrate the recent trends in this field based on 241 reviewed papers from 2016 to 2020. Our investigations demonstrate that the focus of research has changed in recent years from answer extraction to answer generation, from single to multi-document reading comprehension, and from learning from scratch to using pre-trained embeddings. We also discuss the popular datasets and the evaluation metrics in this field. The paper ends with investigating the most cited papers and their contributions.
Identifying Historical Travelogues in Large Text Corpora Using Machine Learning
Venue: arXiv cs.DL 2020
Year: 2020

Travelogues represent an important and intensively studied source for scholars in the humanities, as they provide insights into people, cultures, and places of the past. However, existing studies rarely utilize more than a dozen primary sources, since the human capacities of working with a large number of historical sources are naturally limited. In this paper, we define the notion of travelogue and report upon an interdisciplinary method that, using machine learning as well as domain knowledge, can effectively identify German travelogues in the digitized inventory of the Austrian National Library with F1 scores between 0.94 and 1.00. We applied our method on a corpus of 161,522 German volumes and identified 345 travelogues that could not be identified using traditional search methods, resulting in the most extensive collection of early modern German travelogues ever created. To our knowledge, this is the first time such a method was implemented for the bibliographic indexing of a text corpus on this scale, improving and extending the traditional methods in the humanities. Overall, we consider our technique to be an important first step in a broader effort of developing a novel mixed-method approach for the large-scale serial analysis of travelogues.
Communication-Channel Optimized Partition
Venue: arXiv cs.IT 2020
Year: 2020

Given an original discrete source X with the distribution p_X that is corrupted by noise to produce the noisy data Y with the given joint distribution p(X, Y). A quantizer/classifier Q : Y -> Z is then used to classify/quantize the data Y to the discrete partitioned output Z with probability distribution p_Z. Next, Z is transmitted over a deterministic channel with a given channel matrix A that produces the final discrete output T. One wants to design the optimal quantizer/classifier Q^* such that the cost function F(X; T) between the input X and the final output T is minimized while the probability of the partitioned output Z satisfies a concave constraint G(p_Z) < C. Our results generalized some famous previous results. First, an iteration linear time complexity algorithm is proposed to find the local optimal quantizer. Second, we show that the optimal partition should produce a hard partition that is equivalent to the cuts by hyper-planes in the probability space of the posterior probability p(X|Y). This result finally provides a polynomial-time algorithm to find the globally optimal quantizer.
Towards a secure behavior modeling for IoT networks using Blockchain
Venue: arXiv cs.CR 2020
Year: 2020

Internet of Things (IoT) occupies a vital aspect of our everyday lives. IoT networks composed of smart-devices which communicate and transfer the information without the physical intervention of humans. Due to such proliferation and autonomous nature of IoT systems make these devices threatened and prone to a severe kind of threats. In this paper, we introduces a behavior capturing, and verification procedures in blockchain supported smart-IoT systems that can be able to show the trust-level confidence to outside networks. We defined a custom \emph{Behavior Monitor} and implement on a selected node that can extract the activity of each device and analyzes the behavior using deep machine learning strategy. Besides, we deploy Trusted Execution Technology (TEE) which can be used to provide a secure execution environment (enclave) for sensitive application code and data on the blockchain. Finally, in the evaluation phase we analyze various IoT devices data that is infected by Mirai attack. The evaluation results show the strength of our proposed method in terms of accuracy and time required for detection.
On Bounds and Closed Form Expressions for Capacities of Discrete Memoryless Channels with Invertible Positive Matrices
Venue: arXiv cs.IT 2020
Year: 2020

While capacities of discrete memoryless channels are well studied, it is still not possible to obtain a closed-form expression for the capacity of an arbitrary discrete memoryless channel. This paper describes an elementary technique based on Karush Kuhn Tucker (KKT) conditions to obtain (1) a good upper bound of a discrete memoryless channel having an invertible positive channel matrix and (2) a closed-form expression for the capacity if the channel matrix satisfies certain conditions related to its singular value and its Gershgorin disk.
Vamsa: Automated Provenance Tracking in Data Science Scripts
Venue: arXiv cs.LG 2020
Year: 2020

There has recently been a lot of ongoing research in the areas of fairness, bias and explainability of machine learning (ML) models due to the self-evident or regulatory requirements of various ML applications. We make the following observation: All of these approaches require a robust understanding of the relationship between ML models and the data used to train them. In this work, we introduce the ML provenance tracking problem: the fundamental idea is to automatically track which columns in a dataset have been used to derive the features/labels of an ML model. We discuss the challenges in capturing such information in the context of Python, the most common language used by data scientists. We then present Vamsa, a modular system that extracts provenance from Python scripts without requiring any changes to the users' code. Using 26K real data science scripts, we verify the effectiveness of Vamsa in terms of coverage, and performance. We also evaluate Vamsa's accuracy on a smaller subset of manually labeled data. Our analysis shows that Vamsa's precision and recall range from 90.4% to 99.1% and its latency is in the order of milliseconds for average size scripts. Drawing from our experience in deploying ML models in production, we also present an example in which Vamsa helps automatically identify models that are affected by data corruption issues.
Spatial Applications of Topological Data Analysis: Cities, Snowflakes, Random Structures, and Spiders Spinning Under the Influence
Venue: arXiv cs.SI 2020
Year: 2020

Spatial networks are ubiquitous in social, geographical, physical, and biological applications. To understand the large-scale structure of networks, it is important to develop methods that allow one to directly probe the effects of space on structure and dynamics. Historically, algebraic topology has provided one framework for rigorously and quantitatively describing the global structure of a space, and recent advances in topological data analysis (TDA) have given scholars a new lens for analyzing network data. In this paper, we study a variety of spatial networks -- including both synthetic and natural ones -- using novel topological methods that we recently developed for analyzing spatial networks. We demonstrate that our methods are able to capture meaningful quantities, with specifics that depend on context, in spatial networks and thereby provide useful insights into the structure of those networks, including a novel approach for characterizing them based on their topological structures. We illustrate these ideas with examples of synthetic networks and dynamics on them, street networks in cities, snowflakes, and webs spun by spiders under the influence of various psychotropic substances.
Stochastic Geometry Analysis of Sojourn Time in Multi-Tier Cellular Networks
Venue: arXiv cs.IT 2020
Year: 2020

Impact of mobility will be increasingly important in future generation wireless services and the related challenges will need to be addressed. Sojourn time, the time duration that a mobile user stays within a cell, is a mobility-aware parameter that can significantly impact the performance of mobile users and it can also be exploited to improve resource allocation and mobility management methods in the network. In this paper, we derive the distribution and mean of the sojourn time in multi-tier cellular networks, where spatial distribution of base stations (BSs) in each tier follows an independent homogeneous Poisson point process (PPP). To obtain the sojourn time distribution in multi-tier cellular networks with maximum biased averaged received power association, we derive the linear contact distribution function and chord length distribution of each tier. We also study the relation between mean sojourn time and other mobility-related performance metrics. We show that the mean sojourn time is inversely proportional to the handoff rate, and the complementary cumulative distribution function (CCDF) of sojourn time is bounded from above by the complement of the handoff probability. Moreover, we study the impact of user velocity and network parameters on the sojourn time.