OpenRaiser/Intern-Atlas huggingface dataset:
Intelligent Reflecting Surface-assisted MU-MISO Systems with Imperfect Hardware: Channel Estimation, Beamforming Design
Venue: arXiv cs.IT 2021
Year: 2021

Most works in IRS-assisted systems have ignored the impact of the inevitable residual hardware impairments (HWIs) at both the transceiver hardware and the IRS while any relevant works have addressed only simple scenarios, e.g., with single-antenna network nodes and/or without taking the randomness of phase noise at the IRS into account. In this work, we aim at filling up this gap by considering a general IRS-assisted multi-user (MU) multiple-input single-output (MISO) system with imperfect CSI and correlated Rayleigh fading. In parallel, we present a general computationally efficient methodology for IRS reflect beamforming (RB) optimization. Specifically, we introduce an advantageous channel estimation (CE) method for such systems accounting for the HWIs. Moreover, we derive the uplink achievable spectral efficiency (SE) with maximal-ratio combining (MRC) receiver, displaying three significant advantages being: 1) its closed-form expression, 2) its dependence only on large-scale statistics, and 3) its low training overhead. Notably, by exploiting the first two benefits, we achieve to perform optimization with respect to the reflect beamforming matrix (RBM) that can take place only at every several coherence intervals, and thus, reduces significantly the computational cost compared to other methods which require frequent phase optimization. Among the insightful observations, we highlight that uncorrelated Rayleigh fading does not allow optimization of the SE, which makes the application of an IRS ineffective. Also, in the case that the phase drifts, describing the distortion of the phases in the RBM, are uniformly distributed, the presence of an IRS provides no advantage. The analytical results outperform previous works and are verified by Monte-Carlo (MC) simulations.
Enhancing Real-World Adversarial Patches through 3D Modeling of Complex Target Scenes
Venue: arXiv cs.CV 2021
Year: 2021

Adversarial examples have proven to be a concerning threat to deep learning models, particularly in the image domain. However, while many studies have examined adversarial examples in the real world, most of them relied on 2D photos of the attack scene. As a result, the attacks proposed may have limited effectiveness when implemented in realistic environments with 3D objects or varied conditions. There are few studies on adversarial learning that use 3D objects, and in many cases, other researchers are unable to replicate the real-world evaluation process. In this study, we present a framework that uses 3D modeling to craft adversarial patches for an existing real-world scene. Our approach uses a 3D digital approximation of the scene as a simulation of the real world. With the ability to add and manipulate any element in the digital scene, our framework enables the attacker to improve the adversarial patch's impact in real-world settings. We use the framework to create a patch for an everyday scene and evaluate its performance using a novel evaluation process that ensures that our results are reproducible in both the digital space and the real world. Our evaluation results show that the framework can generate adversarial patches that are robust to different settings in the real world.
Towards Feature-Based Performance Regression Using Trajectory Data
Venue: arXiv cs.NE 2021
Year: 2021

Black-box optimization is a very active area of research, with many new algorithms being developed every year. This variety is needed, on the one hand, since different algorithms are most suitable for different types of optimization problems. But the variety also poses a meta-problem: which algorithm to choose for a given problem at hand? Past research has shown that per-instance algorithm selection based on exploratory landscape analysis (ELA) can be an efficient mean to tackle this meta-problem. Existing approaches, however, require the approximation of problem features based on a significant number of samples, which are typically selected through uniform sampling or Latin Hypercube Designs. The evaluation of these points is costly, and the benefit of an ELA-based algorithm selection over a default algorithm must therefore be significant in order to pay off. One could hope to by-pass the evaluations for the feature approximations by using the samples that a default algorithm would anyway perform, i.e., by using the points of the default algorithm's trajectory. We analyze in this paper how well such an approach can work. Concretely, we test how accurately trajectory-based ELA approaches can predict the final solution quality of the CMA-ES after a fixed budget of function evaluations. We observe that the loss of trajectory-based predictions can be surprisingly small compared to the classical global sampling approach, if the remaining budget for which solution quality shall be predicted is not too large. Feature selection, in contrast, did not show any advantage in our experiments and rather led to worsened prediction accuracy. The inclusion of state variables of CMA-ES only has a moderate effect on the prediction accuracy.
Improving Aerial Instance Segmentation in the Dark with Self-Supervised Low Light Enhancement
Venue: arXiv cs.CV 2021
Year: 2021

Low light conditions in aerial images adversely affect the performance of several vision based applications. There is a need for methods that can efficiently remove the low light attributes and assist in the performance of key vision tasks. In this work, we propose a new method that is capable of enhancing the low light image in a self-supervised fashion, and sequentially apply detection and segmentation tasks in an end-to-end manner. The proposed method occupies a very small overhead in terms of memory and computational power over the original algorithm and delivers superior results. Additionally, we propose the generation of a new low light aerial dataset using GANs, which can be used to evaluate vision based networks for similar adverse conditions.
Enhancing efficiency of object recognition in different categorization levels by reinforcement learning in modular spiking neural networks
Venue: arXiv cs.CV 2021
Year: 2021

The human visual system contains a hierarchical sequence of modules that take part in visual perception at superordinate, basic, and subordinate categorization levels. During the last decades, various computational models have been proposed to mimic the hierarchical feed-forward processing of visual cortex, but many critical characteristics of the visual system, such actual neural processing and learning mechanisms, are ignored. Pursuing the line of biological inspiration, we propose a computational model for object recognition in different categorization levels, in which a spiking neural network equipped with the reinforcement learning rule is used as a module at each categorization level. Each module solves the object recognition problem at each categorization level, solely based on the earliest spike of class-specific neurons at its last layer, without using any external classifier. According to the required information at each categorization level, the relevant band-pass filtered images are utilized. The performance of our proposed model is evaluated by various appraisal criteria with three benchmark datasets and significant improvement in recognition accuracy of our proposed model is achieved in all experiments.
Capacity Optimality of AMP in Coded Systems
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies a large random matrix system (LRMS) model involving an arbitrary signal distribution and forward error control (FEC) coding. We establish an area property based on the approximate message passing (AMP) algorithm. Under the assumption that the state evolution for AMP is correct for the coded system, the achievable rate of AMP is analyzed. We prove that AMP achieves the constrained capacity of the LRMS with an arbitrary signal distribution provided that a matching condition is satisfied. We provide related numerical results of binary signaling using irregular low-density parity-check (LDPC) codes. We show that the optimized codes demonstrate significantly better performance over un-matched ones under AMP. For quadrature phase shift keying (QPSK) modulation, bit error rate (BER) performance within 1 dB from the constrained capacity limit is observed.
Epidemiological and public health requirements for COVID-19 contact tracing apps and their evaluation
Venue: arXiv cs.CY 2021
Year: 2021

Digital contact tracing is a public health intervention. It should be integrated with local health policy, provide rapid and accurate notifications to exposed individuals, and encourage high app uptake and adherence to quarantine. Real-time monitoring and evaluation of effectiveness of app-based contact tracing is key for improvement and public trust.
A Similarity-preserving Neural Network Trained on Transformed Images Recapitulates Salient Features of the Fly Motion Detection Circuit
Venue: arXiv cs.NE 2021
Year: 2021

Learning to detect content-independent transformations from data is one of the central problems in biological and artificial intelligence. An example of such problem is unsupervised learning of a visual motion detector from pairs of consecutive video frames. Rao and Ruderman formulated this problem in terms of learning infinitesimal transformation operators (Lie group generators) via minimizing image reconstruction error. Unfortunately, it is difficult to map their model onto a biologically plausible neural network (NN) with local learning rules. Here we propose a biologically plausible model of motion detection. We also adopt the transformation-operator approach but, instead of reconstruction-error minimization, start with a similarity-preserving objective function. An online algorithm that optimizes such an objective function naturally maps onto an NN with biologically plausible learning rules. The trained NN recapitulates major features of the well-studied motion detector in the fly. In particular, it is consistent with the experimental observation that local motion detectors combine information from at least three adjacent pixels, something that contradicts the celebrated Hassenstein-Reichardt model.
Breaking the Quadratic Barrier for Matroid Intersection
Venue: arXiv cs.DS 2021
Year: 2021

The matroid intersection problem is a fundamental problem that has been extensively studied for half a century. In the classic version of this problem, we are given two matroids $\mathcal{M}_1 = (V, \mathcal{I}_1)$ and $\mathcal{M}_2 = (V, \mathcal{I}_2)$ on a comment ground set $V$ of $n$ elements, and then we have to find the largest common independent set $S \in \mathcal{I}_1 \cap \mathcal{I}_2$ by making independence oracle queries of the form "Is $S \in \mathcal{I}_1$?" or "Is $S \in \mathcal{I}_2$?" for $S \subseteq V$. The goal is to minimize the number of queries. Beating the existing $\tilde O(n^2)$ bound, known as the quadratic barrier, is an open problem that captures the limits of techniques from two lines of work. The first one is the classic Cunningham's algorithm [SICOMP 1986], whose $\tilde O(n^2)$-query implementations were shown by CLS+ [FOCS 2019] and Nguyen [2019]. The other one is the general cutting plane method of Lee, Sidford, and Wong [FOCS 2015]. The only progress towards breaking the quadratic barrier requires either approximation algorithms or a more powerful rank oracle query [CLS+ FOCS 2019]. No exact algorithm with $o(n^2)$ independence queries was known. In this work, we break the quadratic barrier with a randomized algorithm guaranteeing $\tilde O(n^{9/5})$ independence queries with high probability, and a deterministic algorithm guaranteeing $\tilde O(n^{11/6})$ independence queries. Our key insight is simple and fast algorithms to solve a graph reachability problem that arose in the standard augmenting path framework [Edmonds 1968]. Combining this with previous exact and approximation algorithms leads to our results.
DANTE: Predicting Insider Threat using LSTM on system logs
Venue: arXiv cs.CR 2021
Year: 2021

Insider threat is one of the most pernicious threat vectors to information and communication technologies (ICT)across the world due to the elevated level of trust and access that an insider is afforded. This type of threat can stem from both malicious users with a motive as well as negligent users who inadvertently reveal details about trade secrets, company information, or even access information to malignant players. In this paper, we propose a novel approach that uses system logs to detect insider behavior using a special recurrent neural network (RNN) model. Ground truth is established using DANTE and used as the baseline for identifying anomalous behavior. For this, system logs are modeled as a natural language sequence and patterns are extracted from these sequences. We create workflows of sequences of actions that follow a natural language logic and control flow. These flows are assigned various categories of behaviors - malignant or benign. Any deviation from these sequences indicates the presence of a threat. We further classify threats into one of the five categories provided in the CERT insider threat dataset. Through experimental evaluation, we show that the proposed model can achieve 99% prediction accuracy.
SN4KE: Practical Mutation Testing at Binary Level
Venue: arXiv cs.SE 2021
Year: 2021

Mutation analysis is an effective technique to evaluate a test suite adequacy in terms of revealing unforeseen bugs in software. Traditional source- or IR-level mutation analysis is not applicable to the software only available in binary format. This paper proposes a practical binary mutation analysis via binary rewriting, along with a rich set of mutation operators to represent more realistic bugs. We implemented our approach using two state-of-the-art binary rewriting tools and evaluated its effectiveness and scalability by applying them to SPEC CPU benchmarks. Our analysis revealed that the richer mutation operators contribute to generating more diverse mutants, which, compared to previous works leads to a higher mutation score for the test harness. We also conclude that the reassembleable disassembly rewriting yields better scalability in comparison to lifting to an intermediate representation and performing a full translation.
Derivative-Free Reinforcement Learning: A Review
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement learning is about learning agent models that make the best sequential decisions in unknown environments. In an unknown environment, the agent needs to explore the environment while exploiting the collected information, which usually forms a sophisticated problem to solve. Derivative-free optimization, meanwhile, is capable of solving sophisticated problems. It commonly uses a sampling-and-updating framework to iteratively improve the solution, where exploration and exploitation are also needed to be well balanced. Therefore, derivative-free optimization deals with a similar core issue as reinforcement learning, and has been introduced in reinforcement learning approaches, under the names of learning classifier systems and neuroevolution/evolutionary reinforcement learning. Although such methods have been developed for decades, recently, derivative-free reinforcement learning exhibits attracting increasing attention. However, recent survey on this topic is still lacking. In this article, we summarize methods of derivative-free reinforcement learning to date, and organize the methods in aspects including parameter updating, model selection, exploration, and parallel/distributed methods. Moreover, we discuss some current limitations and possible future directions, hoping that this article could bring more attentions to this topic and serve as a catalyst for developing novel and efficient approaches.
Rate-Splitting Multiple Access for Multigateway Multibeam Satellite Systems with Feeder Link Interference
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies the precoder design problem of achieving max-min fairness (MMF) amongst users in multigateway multibeam satellite communication systems with feeder link interference. We propose a beamforming strategy based on a newly introduced transmission scheme known as rate-splitting multiple access (RSMA). RSMA relies on multi-antenna rate-splitting at the transmitter and successive interference cancellation (SIC) at the receivers, such that the intended message for a user is split into a common part and a private part and the interference is partially decoded and partially treated as noise. In this paper, we formulate the MMF problem subject to per-antenna power constraints at the satellite for the system with imperfect channel state information at the transmitter (CSIT). We also consider the case of two-stage precoding which is assisted by on-board processing (OBP) at the satellite. Numerical results obtained through simulations for RSMA and the conventional linear precoding method are compared. When RSMA is used, MMF rate gain is promised and this gain increases when OBP is used. RSMA is proven to be promising for multigateway multibeam satellite systems whereby there are various practical challenges such as feeder link interference, CSIT uncertainty, per-antenna power constraints, uneven user distribution per beam and frame-based processing.
Anomaly Detection through Transfer Learning in Agriculture and Manufacturing IoT Systems
Venue: arXiv cs.LG 2021
Year: 2021

IoT systems have been facing increasingly sophisticated technical problems due to the growing complexity of these systems and their fast deployment practices. Consequently, IoT managers have to judiciously detect failures (anomalies) in order to reduce their cyber risk and operational cost. While there is a rich literature on anomaly detection in many IoT-based systems, there is no existing work that documents the use of ML models for anomaly detection in digital agriculture and in smart manufacturing systems. These two application domains pose certain salient technical challenges. In agriculture the data is often sparse, due to the vast areas of farms and the requirement to keep the cost of monitoring low. Second, in both domains, there are multiple types of sensors with varying capabilities and costs. The sensor data characteristics change with the operating point of the environment or machines, such as, the RPM of the motor. The inferencing and the anomaly detection processes therefore have to be calibrated for the operating point. In this paper, we analyze data from sensors deployed in an agricultural farm with data from seven different kinds of sensors, and from an advanced manufacturing testbed with vibration sensors. We evaluate the performance of ARIMA and LSTM models for predicting the time series of sensor data. Then, considering the sparse data from one kind of sensor, we perform transfer learning from a high data rate sensor. We then perform anomaly detection using the predicted sensor data. Taken together, we show how in these two application domains, predictive failure classification can be achieved, thus paving the way for predictive maintenance.
Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learning
Venue: arXiv cs.LG 2021
Year: 2021

The Continual Learning (CL) problem involves performing well on a sequence of tasks under limited compute. Current algorithms in the domain are either slow, offline or sensitive to hyper-parameters. La-MAML, an optimization-based meta-learning algorithm claims to be better than other replay-based, prior-based and meta-learning based approaches. According to the MER paper [1], metrics to measure performance in the continual learning arena are Retained Accuracy (RA) and Backward Transfer-Interference (BTI). La-MAML claims to perform better in these values when compared to the SOTA in the domain. This is the main claim of the paper, which we shall be verifying in this report.
Driving Style Representation in Convolutional Recurrent Neural Network Model of Driver Identification
Venue: arXiv cs.CV 2021
Year: 2021

Identifying driving styles is the task of analyzing the behavior of drivers in order to capture variations that will serve to discriminate different drivers from each other. This task has become a prerequisite for a variety of applications, including usage-based insurance, driver coaching, driver action prediction, and even in designing autonomous vehicles; because driving style encodes essential information needed by these applications. In this paper, we present a deep-neural-network architecture, we term D-CRNN, for building high-fidelity representations for driving style, that combine the power of convolutional neural networks (CNN) and recurrent neural networks (RNN). Using CNN, we capture semantic patterns of driver behavior from trajectories (such as a turn or a braking event). We then find temporal dependencies between these semantic patterns using RNN to encode driving style. We demonstrate the effectiveness of these techniques for driver identification by learning driving style through extensive experiments conducted on several large, real-world datasets, and comparing the results with the state-of-the-art deep-learning and non-deep-learning solutions. These experiments also demonstrate a useful example of bias removal, by presenting how we preprocess the input data by sampling dissimilar trajectories for each driver to prevent spatial memorization. Finally, this paper presents an analysis of the contribution of different attributes for driver identification; we find that engine RPM, Speed, and Acceleration are the best combination of features.
Adversarial Poisoning Attacks and Defense for General Multi-Class Models Based On Synthetic Reduced Nearest Neighbors
Venue: arXiv cs.LG 2021
Year: 2021

State-of-the-art machine learning models are vulnerable to data poisoning attacks whose purpose is to undermine the integrity of the model. However, the current literature on data poisoning attacks is mainly focused on ad hoc techniques that are only applicable to specific machine learning models. Additionally, the existing data poisoning attacks in the literature are limited to either binary classifiers or to gradient-based algorithms. To address these limitations, this paper first proposes a novel model-free label-flipping attack based on the multi-modality of the data, in which the adversary targets the clusters of classes while constrained by a label-flipping budget. The complexity of our proposed attack algorithm is linear in time over the size of the dataset. Also, the proposed attack can increase the error up to two times for the same attack budget. Second, a novel defense technique based on the Synthetic Reduced Nearest Neighbor (SRNN) model is proposed. The defense technique can detect and exclude flipped samples on the fly during the training procedure. Through extensive experimental analysis, we demonstrate that (i) the proposed attack technique can deteriorate the accuracy of several models drastically, and (ii) under the proposed attack, the proposed defense technique significantly outperforms other conventional machine learning models in recovering the accuracy of the targeted model.
Explainability in CNN Models By Means of Z-Scores
Venue: arXiv cs.CV 2021
Year: 2021

This paper explores the similarities of output layers in Neural Networks (NNs) with logistic regression to explain importance of inputs by Z-scores. The network analyzed, a network for fusion of Synthetic Aperture Radar (SAR) and Microwave Radiometry (MWR) data, is applied to prediction of arctic sea ice. With the analysis the importance of MWR relative to SAR is found to favor MWR components. Further, as the model represents image features at different scales, the relative importance of these are as well analyzed. The suggested methodology offers a simple and easy framework for analyzing output layer components and can reduce the number of components for further analysis with e.g. common NN visualization methods.