OpenRaiser/Intern-Atlas huggingface dataset:
Bayes Optimal Early Stopping Policies for Black-Box Optimization
Venue: arXiv cs.LG 2019
Year: 2019

We derive an optimal policy for adaptively restarting a randomized algorithm, based on observed features of the run-so-far, so as to minimize the expected time required for the algorithm to successfully terminate. Given a suitable Bayesian prior, this result can be used to select the optimal black-box optimization algorithm from among a large family of algorithms that includes random search, Successive Halving, and Hyperband. On CIFAR-10 and ImageNet hyperparameter tuning problems, the proposed policies offer up to a factor of 13 improvement over random search in terms of expected time to reach a given target accuracy, and up to a factor of 3 improvement over a baseline adaptive policy that terminates a run whenever its accuracy is below-median.
Reduced-Rank Local Distance Metric Learning for k-NN Classification
Venue: arXiv cs.LG 2019
Year: 2019

We propose a new method for local distance metric learning based on sample similarity as side information. These local metrics, which utilize conical combinations of metric weight matrices, are learned from the pooled spatial characteristics of the data, as well as the similarity profiles between the pairs of samples, whose distances are measured. The main objective of our framework is to yield metrics, such that the resulting distances between similar samples are small and distances between dissimilar samples are above a certain threshold. For learning and inference purposes, we describe a transductive, as well as an inductive algorithm; the former approach naturally befits our framework, while the latter one is provided in the interest of faster learning. Experimental results on a collection of classification problems imply that the new methods may exhibit notable performance advantages over alternative metric learning approaches that have recently appeared in the literature.
A time-distance trade-off for GDD with preprocessing---Instantiating the DLW heuristic
Venue: arXiv cs.DS 2019
Year: 2019

For $0 \leq α\leq 1/2$, we show an algorithm that does the following. Given appropriate preprocessing $P(\mathcal{L})$ consisting of $N_α:= 2^{O(n^{1-2α} + \log n)}$ vectors in some lattice $\mathcal{L} \subset \mathbb{R}^n$ and a target vector $\boldsymbol{t}\in \mathbb{R}^n$, the algorithm finds $\boldsymbol{y} \in \mathcal{L}$ such that $\|\boldsymbol{y}- \boldsymbol{t}\| \leq n^{1/2 + α} η(\mathcal{L})$ in time $\mathrm{poly}(n) \cdot N_α$, where $η(\mathcal{L})$ is the smoothing parameter of the lattice. The algorithm itself is very simple and was originally studied by Doulgerakis, Laarhoven, and de Weger (to appear in PQCrypto, 2019), who proved its correctness under certain reasonable heuristic assumptions on the preprocessing $P(\mathcal{L})$ and target $\boldsymbol{t}$. Our primary contribution is a choice of preprocessing that allows us to prove correctness without any heuristic assumptions. Our main motivation for studying this is the recent breakthrough algorithm for IdealSVP due to Hanrot, Pellet--Mary, and Stehlé (to appear in Eurocrypt, 2019), which uses the DLW algorithm as a key subprocedure. In particular, our result implies that the HPS IdealSVP algorithm can be made to work with fewer heuristic assumptions. Our only technical tool is the discrete Gaussian distribution over $\mathcal{L}$, and in particular, a lemma showing that the one-dimensional projections of this distribution behave very similarly to the continuous Gaussian. This lemma might be of independent interest.
Hybrid Beamforming for Millimeter Wave Systems Using the MMSE Criterion
Venue: arXiv cs.IT 2019
Year: 2019

Hybrid analog and digital beamforming (HBF) has recently emerged as an attractive technique for millimeter-wave (mmWave) communication systems. It well balances the demand for sufficient beamforming gains to overcome the propagation loss and the desire to reduce the hardware cost and power consumption. In this paper, the mean square error (MSE) is chosen as the performance metric to characterize the transmission reliability. Using the minimum sum-MSE criterion, we investigate the HBF design for broadband mmWave transmissions. To overcome the difficulty of solving the multi-variable design problem, the alternating minimization method is adopted to optimize the hybrid transmit and receive beamformers alternatively. Specifically, a manifold optimization based HBF algorithm is firstly proposed, which directly handles the constant modulus constraint of the analog component. Its convergence is then proved. To reduce the computational complexity, we then propose a low-complexity general eigenvalue decomposition based HBF algorithm in the narrowband scenario and three algorithms via the eigenvalue decomposition and orthogonal matching pursuit methods in the broadband scenario. A particular innovation in our proposed alternating minimization algorithms is a carefully designed initialization method, which leads to faster convergence. {Furthermore, we extend the sum-MSE based design to that with weighted sum-MSE, which is then connected to the spectral efficiency based design.} Simulation results show that the proposed HBF algorithms achieve significant performance improvement over existing ones, and perform close to full-digital beamforming.
A laboratory-created dataset with ground-truth for hyperspectral unmixing evaluation
Venue: arXiv cs.CV 2019
Year: 2019

Spectral unmixing is an important and challenging problem in hyperspectral data processing. This topic has been extensively studied and a variety of unmixing algorithms have been proposed in the literature. However, the lack of publicly available dataset with ground-truth makes it difficult to evaluate and compare the performance of unmixing algorithms in a quantitative and objective manner. Most of the existing works rely on the use of numerical synthetic data and an intuitive inspection of the results of real data. To alleviate this dilemma, in this study, we design several experimental scenes in our laboratory, including printed checkerboards, mixed quartz sands, and reflection with a vertical board. A dataset is then created by imaging these scenes with the hyperspectral camera in our laboratory, providing 36 mixtures with more than 130, 000 pixels with 256 wavelength bands ranging from 400nm to 1000nm. The experimental settings are strictly controlled so that pure material spectral signatures and material compositions are known. To the best of our knowledge, this dataset is the first publicly available dataset created in a systematic manner with ground-truth for spectral unmixing. Some typical linear and nonlinear unmixing algorithms are also tested with this dataset and lead to meaningful results.
Generative Memory for Lifelong Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Our research is focused on understanding and applying biological memory transfers to new AI systems that can fundamentally improve their performance, throughout their fielded lifetime experience. We leverage current understanding of biological memory transfer to arrive at AI algorithms for memory consolidation and replay. In this paper, we propose the use of generative memory that can be recalled in batch samples to train a multi-task agent in a pseudo-rehearsal manner. We show results motivating the need for task-agnostic separation of latent space for the generative memory to address issues of catastrophic forgetting in lifelong learning.
Physical Adversarial Attacks Against End-to-End Autoencoder Communication Systems
Venue: arXiv cs.IT 2019
Year: 2019

We show that end-to-end learning of communication systems through deep neural network (DNN) autoencoders can be extremely vulnerable to physical adversarial attacks. Specifically, we elaborate how an attacker can craft effective physical black-box adversarial attacks. Due to the openness (broadcast nature) of the wireless channel, an adversary transmitter can increase the block-error-rate of a communication system by orders of magnitude by transmitting a well-designed perturbation signal over the channel. We reveal that the adversarial attacks are more destructive than jamming attacks. We also show that classical coding schemes are more robust than autoencoders against both adversarial and jamming attacks. The codes are available at [1].
On Transforming Functions Accessing Global Variables into Logically Constrained Term Rewriting Systems
Venue: arXiv cs.LO 2019
Year: 2019

In this paper, we show a new approach to transformations of an imperative program with function calls and global variables into a logically constrained term rewriting system. The resulting system represents transitions of the whole execution environment with a call stack. More precisely, we prepare a function symbol for the whole environment, which stores values for global variables and a call stack as its arguments. For a function call, we prepare rewrite rules to push the frame to the stack and to pop it after the execution. Any running frame is located at the top of the stack, and statements accessing global variables are represented by rewrite rules for the environment symbol. We show a precise transformation based on the approach and prove its correctness.
A Study of Newly Observed Hostnames and DNS Tunneling in the Wild
Venue: arXiv cs.CR 2019
Year: 2019

The domain name system (DNS) is a crucial backbone of the Internet and millions of new domains are created on a daily basis. While the vast majority of these domains are legitimate, adversaries also register new hostnames to carry out nefarious purposes, such as scams, phishing, or other types of attacks. In this paper, we present insights on the global utilization of DNS through a measurement study examining exclusively newly observed hostnames via passive DNS data analysis. We analyzed more than two billion such hostnames collected over a period of two months. Surprisingly, we find that only three second-level domains are responsible for more than half of all newly observed hostnames every day. More specifically, we found that Google's Accelerated Mobile Pages (AMP) project, the music streaming service Spotify, and a DNS tunnel provider generate the majority of new domains on the Internet. DNS tunneling is a covert channel technique to transfer arbitrary information over DNS via DNS queries and answers. This technique is often (ab)used by attackers to transfer data in a stealthy way, bypassing traditional network security systems. We find that potential DNS tunnels cause a significant fraction of the global DNS requests for new hostnames: our analysis reveals that nearly all resource record type NULL requests and more than a third of all TXT requests can be attributed to DNS tunnels. Motivated by these empirical measurement results, we propose and implement a method to identify DNS tunnels via a step-wise filtering approach that relies on general characteristics of such tunnels (e.g., number of subdomains or resource record type). Using our approach on empirical data, we successfully identified 273 suspicious domains related to DNS tunnels, including two known APT campaigns (Wekby and APT32).
An LSTM Network for Real-Time Odometry Estimation
Venue: arXiv cs.RO 2019
Year: 2019

The use of 2D laser scanners is attractive for the autonomous driving industry because of its accuracy, light-weight and low-cost. However, since only a 2D slice of the surrounding environment is detected at each scan, it is a challenge to execute important tasks such as the localization of the vehicle. In this paper we present a novel framework that explores the use of deep Recurrent Convolutional Neural Networks (RCNN) for odometry estimation using only 2D laser scanners. The application of RCNNs provides the tools to not only extract the features of the laser scanner data using Convolutional Neural Networks (CNNs), but in addition it models the possible connections among consecutive scans using the Long Short-Term Memory (LSTM) Recurrent Neural Network. Results on a real road dataset show that the method can run in real-time without using GPU acceleration and have competitive performance compared to other methods, being an interesting approach that could complement traditional localization systems.
Towards end-to-end pulsed eddy current classification and regression with CNN
Venue: arXiv cs.CV 2019
Year: 2019

Pulsed eddy current (PEC) is an effective electromagnetic non-destructive inspection (NDI) technique for metal materials, which has already been widely adopted in detecting cracking and corrosion in some multi-layer structures. Automatically inspecting the defects in these structures would be conducive to further analysis and treatment of them. In this paper, we propose an effective end-to-end model using convolutional neural networks (CNN) to learn effective features from PEC data. Specifically, we construct a multi-task generic model, based on 1D CNN, to predict both the class and depth of flaws simultaneously. Extensive experiments demonstrate our model is capable of handling both classification and regression tasks on PEC data. Our proposed model obtains higher accuracy and lower error compared to other standard methods.
A Family of Bayesian Cramér-Rao Bounds, and Consequences for Log-Concave Priors
Venue: arXiv cs.IT 2019
Year: 2019

Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior, we obtain a Bayesian Cramér-Rao bound which holds for any (possibly biased) estimator and, unlike the van Trees inequality, does not depend on the Fisher information of the prior.
Penetration Tests for Bluetooth Low Energy and Zigbee using the Software-Defined Radio
Venue: arXiv cs.CR 2019
Year: 2019

This article discusses the available Software Defined Radios (SDRs), compatible software, message formats, and also shows how it is possible to do penetration tests using SDR for Bluetooth Low Energy (BLE) and ZigBee technologies.
Better Algorithms for Stochastic Bandits with Adversarial Corruptions
Venue: arXiv cs.LG 2019
Year: 2019

We study the stochastic multi-armed bandits problem in the presence of adversarial corruption. We present a new algorithm for this problem whose regret is nearly optimal, substantially improving upon previous work. Our algorithm is agnostic to the level of adversarial contamination and can tolerate a significant amount of corruption with virtually no degradation in performance.
Discovering Language of the Stocks
Venue: arXiv cs.CE 2019
Year: 2019

Stock prediction has always been attractive area for researchers and investors since the financial gains can be substantial. However, stock prediction can be a challenging task since stocks are influenced by a multitude of factors whose influence vary rapidly through time. This paper proposes a novel approach (Word2Vec) for stock trend prediction combining NLP and Japanese candlesticks. First, we create a simple language of Japanese candlesticks from the source OHLC data. Then, sentences of words are used to train the NLP Word2Vec model where training data classification also takes into account trading commissions. Finally, the model is used to predict trading actions. The proposed approach was compared to three trading models Buy & Hold, MA and MACD according to the yield achieved. We first evaluated Word2Vec on three shares of Apple, Microsoft and Coca-Cola where it outperformed the comparative models. Next we evaluated Word2Vec on stocks from Russell Top 50 Index where our Word2Vec method was also very successful in test phase and only fall behind the Buy & Hold method in validation phase. Word2Vec achieved positive results in all scenarios while the average yields of MA and MACD were still lower compared to Word2Vec.
Pragmatic classification of movement primitives for stroke rehabilitation
Venue: arXiv cs.LG 2019
Year: 2019

Rehabilitation training is the primary intervention to improve motor recovery after stroke, but a tool to measure functional training does not currently exist. To bridge this gap, we previously developed an approach to classify functional movement primitives using wearable sensors and a machine learning (ML) algorithm. We found that this approach had encouraging classification performance but had computational and practical limitations, such as training time, sensor cost, and magnetic drift. Here, we sought to refine this approach and determine the algorithm, sensor configurations, and data requirements needed to maximize computational and practical performance. Motion data had been previously collected from 6 stroke patients wearing 11 inertial measurement units (IMUs) as they moved objects on a target array. To identify optimal ML performance, we evaluated 4 algorithms that are commonly used in activity recognition (linear discriminant analysis (LDA), naïve Bayes, support vector machine, and k-nearest neighbors). We compared their classification accuracy, computational complexity, and tuning requirements. To identify optimal sensor configuration, we progressively sampled fewer sensors and compared classification accuracy. To identify optimal data requirements, we compared accuracy using data from IMUs versus accelerometers. We found that LDA had the highest classification accuracy (92%) of the algorithms tested. It also was the most pragmatic, with low training and testing times and modest tuning requirements. We found that 7 sensors on the paretic arm and back resulted in the best accuracy. Using this array, accelerometers had a lower accuracy (84%). We refined strategies to accurately and pragmatically quantify functional movement primitives in stroke patients. We propose that this optimized ML-sensor approach could be a means to quantify training dose after stroke.
Allocating Limited Resources to Protect a Massive Number of Targets using a Game Theoretic Model
Venue: arXiv cs.GT 2019
Year: 2019

Resource allocation is the process of optimizing the rare resources. In the area of security, how to allocate limited resources to protect a massive number of targets is especially challenging. This paper addresses this resource allocation issue by constructing a game theoretic model. A defender and an attacker are players and the interaction is formulated as a trade-off between protecting targets and consuming resources. The action cost which is a necessary role of consuming resource, is considered in the proposed model. Additionally, a bounded rational behavior model (Quantal Response, QR), which simulates a human attacker of the adversarial nature, is introduced to improve the proposed model. To validate the proposed model, we compare the different utility functions and resource allocation strategies. The comparison results suggest that the proposed resource allocation strategy performs better than others in the perspective of utility and resource effectiveness.
Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data
Venue: arXiv cs.CV 2019
Year: 2019

Prognostic tumor growth modeling via volumetric medical imaging observations can potentially lead to better outcomes of tumor treatment and surgical planning. Recent advances of convolutional networks have demonstrated higher accuracy than traditional mathematical models in predicting future tumor volumes. This indicates that deep learning-based techniques may have great potentials on addressing such problem. However, current 2D patch-based modeling approaches cannot make full use of the spatio-temporal imaging context of the tumor's longitudinal 4D (3D + time) data. Moreover, they are incapable to predict clinically-relevant tumor properties, other than volumes. In this paper, we exploit to formulate the tumor growth process through convolutional Long Short-Term Memory (ConvLSTM) that extract tumor's static imaging appearances and capture its temporal dynamic changes within a single network. We extend ConvLSTM into the spatio-temporal domain (ST-ConvLSTM) by jointly learning the inter-slice 3D contexts and the longitudinal or temporal dynamics from multiple patient studies. Our approach can incorporate other non-imaging patient information in an end-to-end trainable manner. Experiments are conducted on the largest 4D longitudinal tumor dataset of 33 patients to date. Results validate that the ST-ConvLSTM produces a Dice score of 83.2%+-5.1% and a RVD of 11.2%+-10.8%, both significantly outperforming (p<0.05) other compared methods of linear model, ConvLSTM, and generative adversarial network (GAN) under the metric of predicting future tumor volumes. Additionally, our new method enables the prediction of both cell density and CT intensity numbers. Last, we demonstrate the generalizability of ST-ConvLSTM by employing it in 4D medical image segmentation task, which achieves an averaged Dice score of 86.3+-1.2% for left-ventricle segmentation in 4D ultrasound with 3 seconds per patient.
Slightly Superexponential Parameterized Problems
Venue: arXiv cs.CC 2019
Year: 2019

A central problem in parameterized algorithms is to obtain algorithms with running time $f(k)\cdot n^{O(1)}$ such that $f$ is as slow growing function of the parameter $k$ as possible. In particular, a large number of basic parameterized problems admit parameterized algorithms where $f(k)$ is single-exponential, that is, $c^k$ for some constant $c$, which makes aiming for such a running time a natural goal for other problems as well. However there are still plenty of problems where the $f(k)$ appearing in the best known running time is worse than single-exponential and it remained ``slightly superexponential'' even after serious attempts to bring it down. A natural question to ask is whether the $f(k)$ appearing in the running time of the best-known algorithms is optimal for any of these problems. In this paper, we examine parameterized problems where $f(k)$ is $k^{O(k)}=2^{O(k\log k)}$ in the best known running time and for a number of such problems, we show that the dependence on $k$ in the running time cannot be improved to single exponential. (See paper for the longer abstract.)
Behavioral Petri Net Mining and Automated Analysis for Human-Computer Interaction Recommendations in Multi-Application Environments
Venue: arXiv cs.HC 2019
Year: 2019

Process Mining is a famous technique which is frequently applied to Software Development Processes, while being neglected in Human-Computer Interaction (HCI) recommendation applications. Organizations usually train employees to interact with required IT systems. Often, employees, or users in general, develop their own strategies for solving repetitive tasks and processes. However, organizations find it hard to detect whether employees interact efficiently with IT systems or not. Hence, we have developed a method which detects inefficient behavior assuming that at least one optimal HCI strategy is known. This method provides recommendations to gradually adapt users' behavior towards the optimal way of interaction considering satisfaction of users. Based on users' behavior logs tracked by a Java application suitable for multi-application and multi-instance environments, we demonstrate the applicability for a specific task in a common Windows environment utilizing realistic simulated behaviors of users.