OpenRaiser/Intern-Atlas huggingface dataset:
Feedback Capacity Formulas of AGN Channels Driven by Nonstationary Autoregressive Moving Average Noise
Venue: arXiv cs.IT 2021
Year: 2021

In this paper we derive closed-form formulas of feedback capacity and nonfeedback achievable rates, for Additive Gaussian Noise (AGN) channels driven by nonstationary autoregressive moving average (ARMA) noise (with unstable one poles and zeros), based on time-invariant feedback codes and channel input distributions. From the analysis and simulations follows the surprising observations, (i) the use of time-invariant channel input distributions gives rise to multiple regimes of capacity that depend on the parameters of the ARMA noise, which may or may not use feedback, (ii) the more unstable the pole (resp. zero) of the ARMA noise the higher (resp. lower) the feedback capacity, (iii) certain conditions, known as detectability and stabilizability are necessary and sufficient to ensure the feedback capacity formulas and nonfeedback achievable rates {\it are independent of the initial state of the ARMA noise}. Another surprizing observation is that Kim's \cite{kim2010} characterization of feedback capacity which is developed for stable ARMA noise, if applied to the unstable ARMA noise, gives a lower value of feedback capacity compared to our feedback capacity formula.
Benchmarking the Nvidia GPU Lineage: From Early K80 to Modern A100 with Asynchronous Memory Transfers
Venue: arXiv cs.DC 2021
Year: 2021

For many, Graphics Processing Units (GPUs) provides a source of reliable computing power. Recently, Nvidia introduced its 9th generation HPC-grade GPUs, the Ampere 100, claiming significant performance improvements over previous generations, particularly for AI-workloads, as well as introducing new architectural features such as asynchronous data movement. But how well does the A100 perform on non-AI benchmarks, and can we expect the A100 to deliver the application improvements we have grown used to with previous GPU generations? In this paper, we benchmark the A100 GPU and compare it to four previous generations of GPUs, with particular focus on empirically quantifying our derived performance expectations, and -- should those expectations be undelivered -- investigate whether the introduced data-movement features can offset any eventual loss in performance? We find that the A100 delivers less performance increase than previous generations for the well-known Rodinia benchmark suite; we show that some of these performance anomalies can be remedied through clever use of the new data-movement features, which we microbenchmark and demonstrate where (and more importantly, how) they should be used.
Energy-Based Models for Code Generation under Compilability Constraints
Venue: arXiv cs.LG 2021
Year: 2021

Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objective overlooks important global sequence-level features that are present in the data such as syntactic correctness or compilability. In this work, we pose the problem of learning to generate compilable code as constraint satisfaction. We define an Energy-Based Model (EBM) representing a pre-trained generative model with an imposed constraint of generating only compilable sequences. We then use the KL-Adaptive Distributional Policy Gradient algorithm (Khalifa et al., 2021) to train a generative model approximating the EBM. We conduct experiments showing that our proposed approach is able to improve compilability rates without sacrificing diversity and complexity of the generated samples.
Multistep Electric Vehicle Charging Station Occupancy Prediction using Hybrid LSTM Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Public charging station occupancy prediction plays key importance in developing a smart charging strategy to reduce electric vehicle (EV) operator and user inconvenience. However, existing studies are mainly based on conventional econometric or time series methodologies with limited accuracy. We propose a new mixed long short-term memory neural network incorporating both historical charging state sequences and time-related features for multistep discrete charging occupancy state prediction. Unlike the existing LSTM networks, the proposed model separates different types of features and handles them differently with mixed neural network architecture. The model is compared to a number of state-of-the-art machine learning and deep learning approaches based on the EV charging data obtained from the open data portal of the city of Dundee, UK. The results show that the proposed method produces very accurate predictions (99.99% and 81.87% for 1 step (10 minutes) and 6 steps (1 hour) ahead, respectively, and outperforms the benchmark approaches significantly (+22.4% for one-step-ahead prediction and +6.2% for 6 steps ahead). A sensitivity analysis is conducted to evaluate the impact of the model parameters on prediction accuracy.
LTrack: Stealthy Tracking of Mobile Phones in LTE
Venue: arXiv cs.CR 2021
Year: 2021

We introduce LTrack, a new tracking attack on LTE that allows an attacker to stealthily extract user devices' locations and permanent identifiers (IMSI). To remain stealthy, the localization of devices in LTrack is fully passive, relying on our new uplink/downlink sniffer. Our sniffer records both the times of arrival of LTE messages and the contents of the Timing Advance Commands, based on which LTrack calculates locations. LTrack is the first to show the feasibility of a passive localization in LTE through implementation on software-defined radio. Passive localization attacks reveal a user's location traces but can at best link these traces to a device's pseudonymous temporary identifier (TMSI), making tracking in dense areas or over a long time-period challenging. LTrack overcomes this challenge by introducing and implementing a new type of IMSI Catcher named IMSI Extractor. It extracts a device's IMSI and binds it to its current TMSI. Instead of relying on fake base stations like existing IMSI Catchers, which are detectable due to their continuous transmission, IMSI Extractor relies on our uplink/downlink sniffer enhanced with surgical message overshadowing. This makes our IMSI Extractor the stealthiest IMSI Catcher to date. We evaluate LTrack through a series of experiments and show that in line-of-sight conditions, the attacker can estimate the location of a phone with less than 6m error in 90% of the cases. We successfully tested our IMSI Extractor against a set of 17 modern smartphones connected to our industry-grade LTE testbed. We further validated our uplink/downlink sniffer and IMSI Extractor in a test facility of an operator.
Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition
Venue: arXiv cs.CV 2021
Year: 2021

With the recent surge in the research of vision transformers, they have demonstrated remarkable potential for various challenging computer vision applications, such as image recognition, point cloud classification as well as video understanding. In this paper, we present empirical results for training a stronger video vision transformer on the EPIC-KITCHENS-100 Action Recognition dataset. Specifically, we explore training techniques for video vision transformers, such as augmentations, resolutions as well as initialization, etc. With our training recipe, a single ViViT model achieves the performance of 47.4\% on the validation set of EPIC-KITCHENS-100 dataset, outperforming what is reported in the original paper by 3.4%. We found that video transformers are especially good at predicting the noun in the verb-noun action prediction task. This makes the overall action prediction accuracy of video transformers notably higher than convolutional ones. Surprisingly, even the best video transformers underperform the convolutional networks on the verb prediction. Therefore, we combine the video vision transformers and some of the convolutional video networks and present our solution to the EPIC-KITCHENS-100 Action Recognition competition.
Interactive Modelling of Volumetric Musculoskeletal Anatomy
Venue: arXiv cs.GR 2021
Year: 2021

We present a new approach for modelling musculoskeletal anatomy. Unlike previous methods, we do not model individual muscle shapes as geometric primitives (polygonal meshes, NURBS etc.). Instead, we adopt a volumetric segmentation approach where every point in our volume is assigned to a muscle, fat, or bone tissue. We provide an interactive modelling tool where the user controls the segmentation via muscle curves and we visualize the muscle shapes using volumetric rendering. Muscle curves enable intuitive yet powerful control over the muscle shapes. This representation allows us to automatically handle intersections between different tissues (musclemuscle, muscle-bone, and muscle-skin) during the modelling and automates computation of muscle fiber fields. We further introduce a novel algorithm for converting the volumetric muscle representation into tetrahedral or surface geometry for use in downstream tasks. Additionally, we introduce an interactive skeleton authoring tool that allows the users to create skeletal anatomy starting from only a skin mesh using a library of bone parts.
Single-Server Private Linear Transformation: The Individual Privacy Case
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers the single-server Private Linear Transformation (PLT) problem with individual privacy guarantees. In this problem, there is a user that wishes to obtain $L$ independent linear combinations of a $D$-subset of messages belonging to a dataset of $K$ messages stored on a single server. The goal is to minimize the download cost while keeping the identity of each message required for the computation individually private. The individual privacy requirement ensures that the identity of each individual message required for the computation is kept private. This is in contrast to the stricter notion of joint privacy that protects the entire set of identities of all messages used for the computation, including the correlations between these identities. The notion of individual privacy captures a broad set of practical applications. For example, such notion is relevant when the dataset contains information about individuals, each of them requires privacy guarantees for their data access patterns. We focus on the setting in which the required linear transformation is associated with a maximum distance separable (MDS) matrix. In particular, we require that the matrix of coefficients pertaining to the required linear combinations is the generator matrix of an MDS code. We establish lower and upper bounds on the capacity of PLT with individual privacy, where the capacity is defined as the supremum of all achievable download rates. We show that our bounds are tight under certain conditions.
Realizing GANs via a Tunable Loss Function
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a tunable GAN, called $α$-GAN, parameterized by $α\in (0,\infty]$, which interpolates between various $f$-GANs and Integral Probability Metric based GANs (under constrained discriminator set). We construct $α$-GAN using a supervised loss function, namely, $α$-loss, which is a tunable loss function capturing several canonical losses. We show that $α$-GAN is intimately related to the Arimoto divergence, which was first proposed by Österriecher (1996), and later studied by Liese and Vajda (2006). We also study the convergence properties of $α$-GAN. We posit that the holistic understanding that $α$-GAN introduces will have practical benefits of addressing both the issues of vanishing gradients and mode collapse.
Do Transformers Really Perform Bad for Graph Representation?
Venue: arXiv cs.LG 2021
Year: 2021

The Transformer architecture has become a dominant choice in many domains, such as natural language processing and computer vision. Yet, it has not achieved competitive performance on popular leaderboards of graph-level prediction compared to mainstream GNN variants. Therefore, it remains a mystery how Transformers could perform well for graph representation learning. In this paper, we solve this mystery by presenting Graphormer, which is built upon the standard Transformer architecture, and could attain excellent results on a broad range of graph representation learning tasks, especially on the recent OGB Large-Scale Challenge. Our key insight to utilizing Transformer in the graph is the necessity of effectively encoding the structural information of a graph into the model. To this end, we propose several simple yet effective structural encoding methods to help Graphormer better model graph-structured data. Besides, we mathematically characterize the expressive power of Graphormer and exhibit that with our ways of encoding the structural information of graphs, many popular GNN variants could be covered as the special cases of Graphormer.
Design and fabrication of solar powered remote controlled all terrain sprayer and mower robot
Venue: arXiv cs.RO 2021
Year: 2021

Manual spraying of pesticides and herbicides to crops and weed inhibitors onto the field are quite laborious work to humans. Manual trimming of selected unwanted plants or harvested crops from the field is also difficult. Our project proposes a multipurpose solar powered, flexible, Remote Controlled, semi-automated spraying robot with 4 Degrees of Freedom (DoF) in spatial movement, with an additional plant mowing equipment. The robot is designed to spray pesticide/insecticide directly onto individual lesions minimizing wastage or excess chemical spraying, hence making the system cost effective and also environment friendly. It is designed to cut down undesired plants selectively by remotely controlling the start and stop of the mowing system. Alternatively, it also serves the purpose of maintaining lawns and sports field made of grass. The same system can be used for water spraying and mowing the grass to desired levels, leading to proper maintenance of the field. The robot is designed to move at 1.4m/s, with an effective spraying area of 0.98 sq. m. by the nozzle and an effective cutting area of 0.3 sq. m. by the mower, when stationary. The prototype has a battery back-up of 7.2hrs under minimum load conditions.
Raman spectral analysis of mixtures with one-dimensional convolutional neural network
Venue: arXiv cs.CV 2021
Year: 2021

Recently, the combination of robust one-dimensional convolutional neural networks (1-D CNNs) and Raman spectroscopy has shown great promise in rapid identification of unknown substances with good accuracy. Using this technique, researchers can recognize a pure compound and distinguish it from unknown substances in a mixture. The novelty of this approach is that the trained neural network operates automatically without any pre- or post-processing of data. Some studies have attempted to extend this technique to the classification of pure compounds in an unknown mixture. However, the application of 1-D CNNs has typically been restricted to binary classifications of pure compounds. Here we will highlight a new approach in spectral recognition and quantification of chemical components in a multicomponent mixture. Two 1-D CNN models, RaMixNet I and II, have been developed for this purpose. The former is for rapid classification of components in a mixture while the latter is for quantitative determination of those constituents. In the proposed method, there is no limit to the number of compounds in a mixture. A data augmentation method is also introduced by adding random baselines to the Raman spectra. The experimental results revealed that the classification accuracy of RaMixNet I and II is 100% for analysis of unknown test mixtures; at the same time, the RaMixNet II model may achieve a regression accuracy of 88% for the quantification of each component.
Reinforcement Learning for Industrial Control Network Cyber Security Orchestration
Venue: arXiv cs.CR 2021
Year: 2021

Defending computer networks from cyber attack requires coordinating actions across multiple nodes based on imperfect indicators of compromise while minimizing disruptions to network operations. Advanced attacks can progress with few observable signals over several months before execution. The resulting sequential decision problem has large observation and action spaces and a long time-horizon, making it difficult to solve with existing methods. In this work, we present techniques to scale deep reinforcement learning to solve the cyber security orchestration problem for large industrial control networks. We propose a novel attention-based neural architecture with size complexity that is invariant to the size of the network under protection. A pre-training curriculum is presented to overcome early exploration difficulty. Experiments show in that the proposed approaches greatly improve both the learning sample complexity and converged policy performance over baseline methods in simulation.
Mechanisms and Attributes of Echo Chambers in Social Media
Venue: arXiv cs.SI 2021
Year: 2021

Echo chambers may exclude social media users from being exposed to other opinions, therefore, can cause rampant negative effects. Among abundant evidence are the 2016 and 2020 US presidential elections conspiracy theories and polarization, as well as the COVID-19 disinfodemic. To help better detect echo chambers and mitigate its negative effects, this paper explores the mechanisms and attributes of echo chambers in social media. In particular, we first illustrate four primary mechanisms related to three main factors: human psychology, social networks, and automatic systems. We then depict common attributes of echo chambers with a focus on the diffusion of misinformation, spreading of conspiracy theory, creation of social trends, political polarization, and emotional contagion of users. We illustrate each mechanism and attribute in a multi-perspective of sociology, psychology, and social computing with recent case studies. Our analysis suggest an emerging need to detect echo chambers and mitigate their negative effects.
Investigating Alternatives to the Root Mean Square for Adaptive Gradient Methods
Venue: arXiv cs.LG 2021
Year: 2021

Adam is an adaptive gradient method that has experienced widespread adoption due to its fast and reliable training performance. Recent approaches have not offered significant improvement over Adam, often because they do not innovate upon one of its core features: normalization by the root mean square (RMS) of recent gradients. However, as noted by Kingma and Ba (2015), any number of $L^p$ normalizations are possible, with the RMS corresponding to the specific case of $p=2$. In our work, we theoretically and empirically characterize the influence of different $L^p$ norms on adaptive gradient methods for the first time. We show mathematically how the choice of $p$ influences the size of the steps taken, while leaving other desirable properties unaffected. We evaluate Adam with various $L^p$ norms on a suite of deep learning benchmarks, and find that $p > 2$ consistently leads to improved learning speed and final performance. The choices of $p=3$ or $p=6$ also match or outperform state-of-the-art methods in all of our experiments.
VaLiPro: Linear Programming Validator for Cluster Computing Systems
Venue: arXiv cs.DC 2021
Year: 2021

The article presents and evaluates a scalable algorithm for validating solutions of linear programming problems on cluster computing systems. The main idea of the method is to generate a regular set of points (validation set) on a small-radius hypersphere centered at the point of the solution under validation. The objective function is calculated for each point of the validation set that belongs to the feasible region. If all these values are less than or equal to the value of the objective function at the point under validation, then this point is the correct solution. The parallel implementation of the VaLiPro algorithm is performed in C++ through the parallel BSF-skeleton, which encapsulates all aspects related to the MPI-based parallelization of the program. We provide the results of large-scale computational experiments on a cluster computing system to study the scalability of the VaLiPro algorithm.
Hybrid Spherical- and Planar-Wave Channel Modeling and DCNN-powered Estimation for Terahertz Ultra-massive MIMO Systems
Venue: arXiv cs.IT 2021
Year: 2021

The Terahertz band is envisioned to meet the demanding 100 Gbps data rates for 6G wireless communications. Aiming at combating the distance limitation problem with low hardware-cost, ultra-massive MIMO with hybrid beamforming is promising. However, relationships among wavelength, array size and antenna spacing give rise to the inaccuracy of planar-wave channel model (PWM), while an enlarged channel matrix dimension leads to excessive parameters of applying spherical-wave channel model (SWM). Moreover, due to the adoption of hybrid beamforming, channel estimation (CE) needs to recover high-dimensional channels from severely compressed channel observation. In this paper, a hybrid spherical- and planar-wave channel model (HSPM) is investigated and proved to be accurate and efficient by adopting PWM within subarray and SWM among subarray. Furthermore, a two-phase HSPM CE mechanism is developed. A deep convolutional-neural-network (DCNN) is designed in the first phase for parameter estimation of reference subarrays, while geometric relationships of the remaining channel parameters between reference subarrays are leveraged to complete CE in the second phase. Extensive numerical results demonstrate the HSPM is accurate at various communication distances, array sizes and carrier frequencies.The DCNN converges fast and achieves high accuracy with 5.2 dB improved normalized-mean-square-error compared to literature methods, and owns substantially low complexity.
Distance Metric Learning through Minimization of the Free Energy
Venue: arXiv cs.LG 2021
Year: 2021

Distance metric learning has attracted a lot of interest for solving machine learning and pattern recognition problems over the last decades. In this work we present a simple approach based on concepts from statistical physics to learn optimal distance metric for a given problem. We formulate the task as a typical statistical physics problem: distances between patterns represent constituents of a physical system and the objective function corresponds to energy. Then we express the problem as a minimization of the free energy of a complex system, which is equivalent to distance metric learning. Much like for many problems in physics, we propose an approach based on Metropolis Monte Carlo to find the best distance metric. This provides a natural way to learn the distance metric, where the learning process can be intuitively seen as stretching and rotating the metric space until some heuristic is satisfied. Our proposed method can handle a wide variety of constraints including those with spurious local minima. The approach works surprisingly well with stochastic nearest neighbors from neighborhood component analysis (NCA). Experimental results on artificial and real-world data sets reveal a clear superiority over a number of state-of-the-art distance metric learning methods for nearest neighbors classification.