OpenRaiser/Intern-Atlas huggingface dataset:
Software-Defined FPGA Accelerator Design for Mobile Deep Learning Applications
Venue: arXiv cs.CV 2019
Year: 2019

Recently, the field of deep learning has received great attention by the scientific community and it is used to provide improved solutions to many computer vision problems. Convolutional neural networks (CNNs) have been successfully used to attack problems such as object recognition, object detection, semantic segmentation, and scene understanding. The rapid development of deep learning goes hand by hand with the adaptation of GPUs for accelerating its processes, such as network training and inference. Even though FPGA design exists long before the use of GPUs for accelerating computations and despite the fact that high-level synthesis (HLS) tools are getting more attractive, the adaptation of FPGAs for deep learning research and application development is poor due to the requirement of hardware design related expertise. This work presents a workflow for deep learning mobile application acceleration on small low-cost low-power FPGA devices using HLS tools. This workflow eases the design of an improved version of the SqueezeJet accelerator used for the speedup of mobile-friendly low-parameter ImageNet class CNNs, such as the SqueezeNet v1.1 and the ZynqNet. Additionally, the workflow includes the development of an HLS-driven analytical model which is used for performance estimation of the accelerator. This model can be also used to direct the design process and lead to future design improvements and optimizations.
Challenges in Partially-Automated Roadway Feature Mapping Using Mobile Laser Scanning and Vehicle Trajectory Data
Venue: arXiv cs.CV 2019
Year: 2019

Connected vehicle and driver's assistance applications are greatly facilitated by Enhanced Digital Maps (EDMs) that represent roadway features (e.g., lane edges or centerlines, stop bars). Due to the large number of signalized intersections and miles of roadway, manual development of EDMs on a global basis is not feasible. Mobile Terrestrial Laser Scanning (MTLS) is the preferred data acquisition method to provide data for automated EDM development. Such systems provide an MTLS trajectory and a point cloud for the roadway environment. The challenge is to automatically convert these data into an EDM. This article presents a new processing and feature extraction method, experimental demonstration providing SAE-J2735 map messages for eleven example intersections, and a discussion of the results that points out remaining challenges and suggests directions for future research.
HE-SLAM: a Stereo SLAM System Based on Histogram Equalization and ORB Features
Venue: arXiv cs.RO 2019
Year: 2019

In the real-life environments, due to the sudden appearance of windows, lights, and objects blocking the light source, the visual SLAM system can easily capture the low-contrast images caused by over-exposure or over-darkness. At this time, the direct method of estimating camera motion based on pixel luminance information is infeasible, and it is often difficult to find enough valid feature points without image processing. This paper proposed HE-SLAM, a new method combining histogram equalization and ORB feature extraction, which can be robust in more scenes, especially in stages with low-contrast images. Because HE-SLAM uses histogram equalization to improve the contrast of images, it can extract enough valid feature points in low-contrast images for subsequent feature matching, keyframe selection, bundle adjustment, and loop closure detection. The proposed HE-SLAM has been tested on the popular datasets (such as KITTI and EuRoc), and the real-time performance and robustness of the system are demonstrated by comparing system runtime and the mean square root error (RMSE) of absolute trajectory error (ATE) with state-of-the-art methods like ORB-SLAM2.
Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance Kernels
Venue: arXiv cs.NE 2019
Year: 2019

In NeuroEvolution, the topologies of artificial neural networks are optimized with evolutionary algorithms to solve tasks in data regression, data classification, or reinforcement learning. One downside of NeuroEvolution is the large amount of necessary fitness evaluations, which might render it inefficient for tasks with expensive evaluations, such as real-time learning. For these expensive optimization tasks, surrogate model-based optimization is frequently applied as it features a good evaluation efficiency. While a combination of both procedures appears as a valuable solution, the definition of adequate distance measures for the surrogate modeling process is difficult. In this study, we will extend cartesian genetic programming of artificial neural networks by the use of surrogate model-based optimization. We propose different distance measures and test our algorithm on a replicable benchmark task. The results indicate that we can significantly increase the evaluation efficiency and that a phenotypic distance, which is based on the behavior of the associated neural networks, is most promising.
An Efficient Linkable Group Signature for Payer Tracing in Anonymous Cryptocurrencies
Venue: arXiv cs.CR 2019
Year: 2019

Cryptocurrencies, led by bitcoin launched in 2009, have obtained wide attention due to the emerging Blockchain in recent years. Anonymous cryptocurrencies are highly essential since users want to preserve their privacy when conducting transactions. However, some users might misbehave with the cover of anonymity such as rampant trafficking and extortion. Thus, it is important to balance anonymity and accountability of anonymous cryptocurrencies. In this paper, we solve this issue by proposing a linkable group signature (LGS) for signing cryptocurrency transactions, which can be used to trace a payer's identity in consortium blockchain based anonymous cryptocurrencies, in case the payer tries illegal activities. A payer keeps anonymous if he/she behaves honestly. We prove that the proposed scheme achieves full-anonymity, full-traceability and linkability in the random oracle. Implementation of the proposed LGS scheme demonstrates its high efficiency thus, can be adopted in anonymous cryptocurrencies in reality.
Contextual Recurrent Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

There is an implicit assumption that by unfolding recurrent neural networks (RNN) in finite time, the misspecification of choosing a zero value for the initial hidden state is mitigated by later time steps. This assumption has been shown to work in practice and alternative initialization may be suggested but often overlooked. In this paper, we propose a method of parameterizing the initial hidden state of an RNN. The resulting architecture, referred to as a Contextual RNN, can be trained end-to-end. The performance on an associative retrieval task is found to improve by conditioning the RNN initial hidden state on contextual information from the input sequence. Furthermore, we propose a novel method of conditionally generating sequences using the hidden state parameterization of Contextual RNN.
Multi-Domain Translation by Learning Uncoupled Autoencoders
Venue: arXiv cs.LG 2019
Year: 2019

Multi-domain translation seeks to learn a probabilistic coupling between marginal distributions that reflects the correspondence between different domains. We assume that data from different domains are generated from a shared latent representation based on a structural equation model. Under this assumption, we show that the problem of computing a probabilistic coupling between marginals is equivalent to learning multiple uncoupled autoencoders that embed to a given shared latent distribution. In addition, we propose a new framework and algorithm for multi-domain translation based on learning the shared latent distribution and training autoencoders under distributional constraints. A key practical advantage of our framework is that new autoencoders (i.e., new domains) can be added sequentially to the model without retraining on the other domains, which we demonstrate experimentally on image as well as genomics datasets.
Skew-Polynomial Rings and Skew-Cyclic Codes
Venue: arXiv cs.IT 2019
Year: 2019

This is a survey on the theory of skew-cyclic codes based on skew-polynomial rings of automorphism type. Skew-polynomial rings have been introduced and discussed by Ore (1933). Evaluation of skew polynomials and sets of (right) roots were first considered by Lam (1986) and studied in great detail by Lam and Leroy thereafter. After a detailed presentation of the most relevant properties of skew polynomials, we survey the algebraic theory of skew-cyclic codes as introduced by Boucher and Ulmer (2007) and studied by many authors thereafter. A crucial role will be played by skew-circulant matrices. Finally, skew-cyclic codes with designed minimum distance are discussed, and we report on two different kinds of skew-BCH codes, which were designed in 2014 and later.
Analyzing Endpoints in the Internet of Things Malware
Venue: arXiv cs.NI 2019
Year: 2019

The lack of security measures in the Internet of Things (IoT) devices and their persistent online connectivity give adversaries an opportunity to target them or abuse them as intermediary targets for larger attacks such as distributed denial-of-service (DDoS) campaigns. In this paper, we analyze IoT malware with a focus on endpoints to understand the affinity between the dropzones and their target IP addresses, and to understand the different patterns among them. Towards this goal, we reverse-engineer 2,423 IoT malware samples to obtain IP addresses. We further augment additional information about the endpoints from Internet-wide scanners, including Shodan and Censys. We then perform a deep data-driven analysis of the dropzones and their target IP addresses and further examine the attack surface of the target device space.
Set Cover in Sub-linear Time
Venue: arXiv cs.DS 2019
Year: 2019

We study the classic set cover problem from the perspective of sub-linear algorithms. Given access to a collection of $m$ sets over $n$ elements in the query model, we show that sub-linear algorithms derived from existing techniques have almost tight query complexities. On one hand, first we show an adaptation of the streaming algorithm presented in Har-Peled et al. [2016] to the sub-linear query model, that returns an $α$-approximate cover using $\tilde{O}(m(n/k)^{1/(α-1)} + nk)$ queries to the input, where $k$ denotes the value of a minimum set cover. We then complement this upper bound by proving that for lower values of $k$, the required number of queries is $\tildeΩ(m(n/k)^{1/(2α)})$, even for estimating the optimal cover size. Moreover, we prove that even checking whether a given collection of sets covers all the elements would require $Ω(nk)$ queries. These two lower bounds provide strong evidence that the upper bound is almost tight for certain values of the parameter $k$. On the other hand, we show that this bound is not optimal for larger values of the parameter $k$, as there exists a $(1+\varepsilon)$-approximation algorithm with $\tilde{O}(mn/k\varepsilon^2)$ queries. We show that this bound is essentially tight for sufficiently small constant $\varepsilon$, by establishing a lower bound of $\tildeΩ(mn/k)$ query complexity.
On modeling hard combinatorial optimization problems as linear programs: Refutations of the "unconditional impossibility" claims
Venue: arXiv cs.CC 2019
Year: 2019

There has been a series of developments in the recent literature (by essentially a same "circle" of authors) with the absolute/unconditioned (implicit or explicit) claim that there exists no abstraction of an NP-Complete combinatorial optimization problem in which the defining combinatorial configurations (such as "tours" in the case of the traveling salesman problem (TSP) for example) can be modeled by a polynomial-sized system of linear constraints. The purpose of this paper is to provide general as well as specific refutations for these recent claims.
EvalAI: Towards Better Evaluation Systems for AI Agents
Venue: arXiv cs.AI 2019
Year: 2019

We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale. EvalAI is built to provide a scalable solution to the research community to fulfill the critical need of evaluating machine learning models and agents acting in an environment against annotations or with a human-in-the-loop. This will help researchers, students, and data scientists to create, collaborate, and participate in AI challenges organized around the globe. By simplifying and standardizing the process of benchmarking these models, EvalAI seeks to lower the barrier to entry for participating in the global scientific effort to push the frontiers of machine learning and artificial intelligence, thereby increasing the rate of measurable progress in this domain.
Angle-Closure Detection in Anterior Segment OCT based on Multi-Level Deep Network
Venue: arXiv cs.CV 2019
Year: 2019

Irreversible visual impairment is often caused by primary angle-closure glaucoma, which could be detected via Anterior Segment Optical Coherence Tomography (AS-OCT). In this paper, an automated system based on deep learning is presented for angle-closure detection in AS-OCT images. Our system learns a discriminative representation from training data that captures subtle visual cues not modeled by handcrafted features. A Multi-Level Deep Network (MLDN) is proposed to formulate this learning, which utilizes three particular AS-OCT regions based on clinical priors: the global anterior segment structure, local iris region, and anterior chamber angle (ACA) patch. In our method, a sliding window based detector is designed to localize the ACA region, which addresses ACA detection as a regression task. Then, three parallel sub-networks are applied to extract AS-OCT representations for the global image and at clinically-relevant local regions. Finally, the extracted deep features of these sub-networks are concatenated into one fully connected layer to predict the angle-closure detection result. In the experiments, our system is shown to surpass previous detection methods and other deep learning systems on two clinical AS-OCT datasets.
Software Development Effort Estimation Using Regression Fuzzy Models
Venue: arXiv cs.SE 2019
Year: 2019

Software effort estimation plays a critical role in project management. Erroneous results may lead to overestimating or underestimating effort, which can have catastrophic consequences on project resources. Machine-learning techniques are increasingly popular in the field. Fuzzy logic models, in particular, are widely used to deal with imprecise and inaccurate data. The main goal of this research was to design and compare three different fuzzy logic models for predicting software estimation effort: Mamdani, Sugeno with constant output and Sugeno with linear output. To assist in the design of the fuzzy logic models, we conducted regression analysis, an approach we call regression fuzzy logic. State-of-the-art and unbiased performance evaluation criteria such as standardized accuracy, effect size and mean balanced relative error were used to evaluate the models, as well as statistical tests. Models were trained and tested using industrial projects from the International Software Benchmarking Standards Group (ISBSG) dataset. Results showed that data heteroscedasticity affected model performance. Fuzzy logic models were found to be very sensitive to outliers. We concluded that when regression analysis was used to design the model, the Sugeno fuzzy inference system with linear output outperformed the other models.
ELKI: A large open-source library for data analysis - ELKI Release 0.7.5 "Heidelberg"
Venue: arXiv cs.LG 2019
Year: 2019

This paper documents the release of the ELKI data mining framework, version 0.7.5. ELKI is an open source (AGPLv3) data mining software written in Java. The focus of ELKI is research in algorithms, with an emphasis on unsupervised methods in cluster analysis and outlier detection. In order to achieve high performance and scalability, ELKI offers data index structures such as the R*-tree that can provide major performance gains. ELKI is designed to be easy to extend for researchers and students in this domain, and welcomes contributions of additional methods. ELKI aims at providing a large collection of highly parameterizable algorithms, in order to allow easy and fair evaluation and benchmarking of algorithms. We will first outline the motivation for this release, the plans for the future, and then give a brief overview over the new functionality in this version. We also include an appendix presenting an overview on the overall implemented functionality.
An Algorithm for Approximating Continuous Functions on Compact Subsets with a Neural Network with one Hidden Layer
Venue: arXiv cs.LG 2019
Year: 2019

George Cybenko's landmark 1989 paper showed that there exists a feedforward neural network, with exactly one hidden layer (and a finite number of neurons), that can arbitrarily approximate a given continuous function $f$ on the unit hypercube. The paper did not address how to find the weight/parameters of such a network, or if finding them would be computationally feasible. This paper outlines an algorithm for a neural network with exactly one hidden layer to reconstruct any continuous scalar or vector valued continuous function.
Exploration of High-Dimensional Grids by Finite State Machines
Venue: arXiv cs.MA 2019
Year: 2019

We consider the problem of finding a treasure at an unknown point of an $n$-dimensional infinite grid, $n\geq 3$, by initially collocated finite state agents (scouts/robots). Recently, the problem has been well characterized for 2 dimensions for deterministic as well as randomized agents, both in synchronous and semi-synchronous models. It has been conjectured that $n+1$ randomized agents are necessary to solve this problem in the $n$-dimensional grid. In this paper we disprove the conjecture in a strong sense: we show that three randomized synchronous agents suffice to explore an $n$-dimensional grid for any $n$. Our algorithm is optimal in terms of the number of the agents. Our key insight is that a constant number of finite state machine agents can, by their positions and movements, implement a stack, which can store the path being explored. We also show how to implement our algorithm using: four randomized semi-synchronous agents; four deterministic synchronous agents; or five deterministic semi-synchronous agents. We give a different algorithm that uses $4$ deterministic semi-synchronous agents for the $3$-dimensional grid. This is provably optimal, and surprisingly, matches the result for $2$ dimensions. For $n\geq 4$, the time complexity of the solutions mentioned above is exponential in distance $D$ of the treasure from the starting point of the agents. We show that in the deterministic case, one additional agent brings the time down to a polynomial. Finally, we focus on algorithms that never venture much beyond the distance $D$. We describe an algorithm that uses $O(\sqrt{n})$ semi-synchronous deterministic agents that never go beyond $2D$, as well as show that any algorithm using $3$ synchronous deterministic agents in $3$ dimensions must travel beyond $Ω(D^{3/2})$ from the origin.
A Blockchain-based Self-tallying Voting Scheme in Decentralized IoT
Venue: arXiv cs.CR 2019
Year: 2019

The Internet of Things (IoT) is experiencing explosive growth and has gained extensive attention from academia and industry in recent years. Most of the existing IoT infrastructures are centralized, in which the presence of a cloud server is mandatory. However, centralized frameworks suffer from the issues of unscalability and single-point-of-failure. Consequently, decentralized IoT has been proposed by taking advantage of the emerging technology of Blockchain. Voting systems are widely adopted in IoT, such as a leader election in wireless sensor networks. Self-tallying voting systems are alternatives to traditional centralized voting systems in decentralized IoT since the traditional ones are not suitable for such scenarios. Unfortunately, self-tallying voting systems inherently suffer from fairness issues, such as adaptive and abortive issues caused by malicious voters. In this paper, we introduce a framework of self-tallying systems in decentralized IoT based on Blockchain. We propose a concrete construction and prove the proposed system satisfies all the security requirements including fairness, dispute-freeness and maximal ballot secrecy. The implementations on mobile phones demonstrate the practicability of our system.
Fast Evaluation of Low-Thrust Transfers via Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

The design of low-thrust-based multitarget interplanetary missions requires a method to quickly and accurately evaluate the low-thrust transfer between any two visiting targets. Complete evaluation of the low-thrust transfer includes not only the estimation of the optimal fuel consumption but also the judgment of transfer feasibility. In this paper, a deep neural network (DNN)-based method is proposed for quickly evaluating low-thrust transfer. An efficient database generation method is developed for obtaining both the infeasible and optimal transfers. A classification DNN and a regression DNN are trained based on the infeasible and optimal transfers to judge the transfer feasibility and estimate the optimal fuel consumption, respectively. The simulation results show that the well-trained DNNs are capable of quickly determining the transfer feasibility with a correct rate of greater than 98% and approximating the optimal transfer fuel consumption with a relative estimation error of less than 0.4%. The tests on two asteroid chains further show the superiority of the DNN-based method for application to the design of low-thrust-based multitarget interplanetary missions
Harnessing Low-Fidelity Data to Accelerate Bayesian Optimization via Posterior Regularization
Venue: arXiv cs.LG 2019
Year: 2019

Bayesian optimization (BO) is a powerful paradigm for derivative-free global optimization of a black-box objective function (BOF) that is expensive to evaluate. However, the overhead of BO can still be prohibitive for problems with highly expensive function evaluations. In this paper, we investigate how to reduce the required number of function evaluations for BO without compromise in solution quality. We explore the idea of posterior regularization to harness low fidelity (LF) data within the Gaussian process upper confidence bound (GP-UCB) framework. The LF data can arise from previous evaluations of an LF approximation of the BOF or of a related optimization task. An extra GP model called LF-GP is trained to fit the LF data. We develop an operator termed dynamic weighted product of experts (DW-POE) fusion. The regularization is induced by this operator on the posterior of the BOF. The impact of the LF GP model on the resulting regularized posterior is adaptively adjusted via Bayesian formalism. Extensive experimental results on benchmark BOF optimization tasks demonstrate the superior performance of the proposed algorithm over state-of-the-art.
Path Capsule Networks
Venue: arXiv cs.LG 2019
Year: 2019

Capsule network (CapsNet) was introduced as an enhancement over convolutional neural networks, supplementing the latter's invariance properties with equivariance through pose estimation. CapsNet achieved a very decent performance with a shallow architecture and a significant reduction in parameters count. However, the width of the first layer in CapsNet is still contributing to a significant number of its parameters and the shallowness may be limiting the representational power of the capsules. To address these limitations, we introduce Path Capsule Network (PathCapsNet), a deep parallel multi-path version of CapsNet. We show that a judicious coordination of depth, max-pooling, regularization by DropCircuit and a new fan-in routing by agreement technique can achieve better or comparable results to CapsNet, while further reducing the parameter count significantly.