OpenRaiser/Intern-Atlas huggingface dataset:
A Meta-Reinforcement Learning Approach to Process Control
Venue: arXiv cs.LG 2021
Year: 2021

Meta-learning is a branch of machine learning which aims to quickly adapt models, such as neural networks, to perform new tasks by learning an underlying structure across related tasks. In essence, models are being trained to learn new tasks effectively rather than master a single task. Meta-learning is appealing for process control applications because the perturbations to a process required to train an AI controller can be costly and unsafe. Additionally, the dynamics and control objectives are similar across many different processes, so it is feasible to create a generalizable controller through meta-learning capable of quickly adapting to different systems. In this work, we construct a deep reinforcement learning (DRL) based controller and meta-train the controller using a latent context variable through a separate embedding neural network. We test our meta-algorithm on its ability to adapt to new process dynamics as well as different control objectives on the same process. In both cases, our meta-learning algorithm adapts very quickly to new tasks, outperforming a regular DRL controller trained from scratch. Meta-learning appears to be a promising approach for constructing more intelligent and sample-efficient controllers.
Random restrictions and PRGs for PTFs in Gaussian Space
Venue: arXiv cs.CC 2021
Year: 2021

A polynomial threshold function (PTF) $f:\mathbb{R}^n \rightarrow \mathbb{R}$ is a function of the form $f(x) = \mathsf{sign}(p(x))$ where $p$ is a polynomial of degree at most $d$. PTFs are a classical and well-studied complexity class with applications across complexity theory, learning theory, approximation theory, quantum complexity and more. We address the question of designing pseudorandom generators (PRG) for polynomial threshold functions (PTFs) in the gaussian space: design a PRG that takes a seed of few bits of randomness and outputs a $n$-dimensional vector whose distribution is indistinguishable from a standard multivariate gaussian by a degree $d$ PTF. Our main result is a PRG that takes a seed of $d^{O(1)}\log ( n / \varepsilon)\log(1/\varepsilon)/\varepsilon^2$ random bits with output that cannot be distinguished from $n$-dimensional gaussian distribution with advantage better than $\varepsilon$ by degree $d$ PTFs. The best previous generator due to O'Donnell, Servedio, and Tan (STOC'20) had a quasi-polynomial dependence (i.e., seedlength of $d^{O(\log d)}$) in the degree $d$. Along the way we prove a few nearly-tight structural properties of restrictions of PTFs that may be of independent interest.
InversionNet3D: Efficient and Scalable Learning for 3D Full Waveform Inversion
Venue: arXiv cs.LG 2021
Year: 2021

Seismic full-waveform inversion (FWI) techniques aim to find a high-resolution subsurface geophysical model provided with waveform data. Some recent effort in data-driven FWI has shown some encouraging results in obtaining 2D velocity maps. However, due to high computational complexity and large memory consumption, the reconstruction of 3D high-resolution velocity maps via deep networks is still a great challenge. In this paper, we present InversionNet3D, an efficient and scalable encoder-decoder network for 3D FWI. The proposed method employs group convolution in the encoder to establish an effective hierarchy for learning information from multiple sources while cutting down unnecessary parameters and operations at the same time. The introduction of invertible layers further reduces the memory consumption of intermediate features during training and thus enables the development of deeper networks with more layers and higher capacity as required by different application scenarios. Experiments on the 3D Kimberlina dataset demonstrate that InversionNet3D achieves state-of-the-art reconstruction performance with lower computational cost and lower memory footprint compared to the baseline.
Internet of Things from Space: Transforming LTE Machine Type Communications for Non-terrestrial Networks
Venue: arXiv cs.IT 2021
Year: 2021

Satellite communication is experiencing a new dawn thanks to low earth orbit mega constellations being deployed at an unprecedented speed. Fueled by the renewed interest in non-terrestrial networks (NTN), the Third Generation Partnership Project (3GPP) is preparing 5G NR, NB-IoT and LTE-M for NTN operation. This article is focused on LTE-M and the essential adaptations needed for supporting satellite communication. Specifically, the major challenges facing LTE-M NTN at the physical and higher layers are discussed and potential solutions are outlined.
Understanding the Challenges and Assisting Developers with Developing Spark Applications
Venue: arXiv cs.SE 2021
Year: 2021

To process data more efficiently, big data frameworks provide data abstractions to developers. However, due to the abstraction, there may be many challenges for developers to understand and debug the data processing code. To uncover the challenges in using big data frameworks, we first conduct an empirical study on 1,000 Apache Spark-related questions on Stack Overflow. We find that most of the challenges are related to data transformation and API usage. To solve these challenges, we design an approach, which assists developers with understanding and debugging data processing in Spark. Our approach leverages statistical sampling to minimize performance overhead, and provides intermediate information and hint messages for each data processing step of a chained method pipeline. The preliminary evaluation of our approach shows that it has low performance overhead and we receive good feedback from developers.
Self-Attentive 3D Human Pose and Shape Estimation from Videos
Venue: arXiv cs.CV 2021
Year: 2021

We consider the task of estimating 3D human pose and shape from videos. While existing frame-based approaches have made significant progress, these methods are independently applied to each image, thereby often leading to inconsistent predictions. In this work, we present a video-based learning algorithm for 3D human pose and shape estimation. The key insights of our method are two-fold. First, to address the inconsistent temporal prediction issue, we exploit temporal information in videos and propose a self-attention module that jointly considers short-range and long-range dependencies across frames, resulting in temporally coherent estimations. Second, we model human motion with a forecasting module that allows the transition between adjacent frames to be smooth. We evaluate our method on the 3DPW, MPI-INF-3DHP, and Human3.6M datasets. Extensive experimental results show that our algorithm performs favorably against the state-of-the-art methods.
Infinity: A Scalable Infrastructure for In-Network Applications
Venue: arXiv cs.NI 2021
Year: 2021

Network programmability is an area of research both defined by its potential and its current limitations. While programmable hardware enables customization of device operation, tailoring processing to finely tuned objectives, limited resources stifle much of the capability and scalability desired for future technologies. Current solutions to overcome these limitations simply shift the problem, temporarily offloading memory needs or processing to other systems while incurring both round-trip time and complexity costs. To overcome these unnecessary costs, we introduce Infinity, a resource disaggregation method to move processing to capable devices while continuing to forward as the original owner, limiting unnecessary buffering and round-trip processing. By forwarding both the processing need and associated data simultaneously we are able to scale operation with minimal overhead and delay, improving both capability and performance objectives for in-network processing.
Three dimensional higher-order raypath separation in a shallow-water waveguide
Venue: arXiv cs.SD 2021
Year: 2021

Separating raypaths in a multipath shallow-water environment is a challenge problem due to the interferences between them and colored noise existing in ocean environment, especially for two raypaths arrive close to each other. Thus, in this paper, a three dimensional (3D) higher-order raypath separation in an array to array configuration is proposed. Performance tests using simulation data in a multipath environment, real data obtained in an ultrasonic waveguide and ocean shallow-water data, respectively, illustrate that the proposed algorithm achieves a higher resolution and a stronger robustness comparing to the existing algorithms.
A Survey of Microarchitectural Side-channel Vulnerabilities, Attacks and Defenses in Cryptography
Venue: arXiv cs.CR 2021
Year: 2021

Side-channel attacks have become a severe threat to the confidentiality of computer applications and systems. One popular type of such attacks is the microarchitectural attack, where the adversary exploits the hardware features to break the protection enforced by the operating system and steal the secrets from the program. In this paper, we systematize microarchitectural side channels with a focus on attacks and defenses in cryptographic applications. We make three contributions. (1) We survey past research literature to categorize microarchitectural side-channel attacks. Since these are hardware attacks targeting software, we summarize the vulnerable implementations in software, as well as flawed designs in hardware. (2) We identify common strategies to mitigate microarchitectural attacks, from the application, OS and hardware levels. (3) We conduct a large-scale evaluation on popular cryptographic applications in the real world, and analyze the severity, practicality and impact of side-channel vulnerabilities. This survey is expected to inspire side-channel research community to discover new attacks, and more importantly, propose new defense solutions against them.
Online structural health monitoring by model order reduction and deep learning algorithms
Venue: arXiv cs.LG 2021
Year: 2021

Within a structural health monitoring (SHM) framework, we propose a simulation-based classification strategy to move towards online damage localization. The procedure combines parametric Model Order Reduction (MOR) techniques and Fully Convolutional Networks (FCNs) to analyze raw vibration measurements recorded on the monitored structure. First, a dataset of possible structural responses under varying operational conditions is built through a physics-based model, allowing for a finite set of predefined damage scenarios. Then, the dataset is used for the offline training of the FCN. Because of the extremely large number of model evaluations required by the dataset construction, MOR techniques are employed to reduce the computational burden. The trained classifier is shown to be able to map unseen vibrational recordings, e.g. collected on-the-fly from sensors placed on the structure, to the actual damage state, thus providing information concerning the presence and also the location of damage. The proposed strategy has been validated by means of two case studies, concerning a 2D portal frame and a 3D portal frame railway bridge; MOR techniques have allowed us to respectively speed up the analyses about 30 and 420 times. For both the case studies, after training the classifier has attained an accuracy greater than 85%.
Hands-on Guidance for Distilling Object Detectors
Venue: arXiv cs.CV 2021
Year: 2021

Knowledge distillation can lead to deploy-friendly networks against the plagued computational complexity problem, but previous methods neglect the feature hierarchy in detectors. Motivated by this, we propose a general framework for detection distillation. Our method, called Hands-on Guidance Distillation, distills the latent knowledge of all stage features for imposing more comprehensive supervision, and focuses on the essence simultaneously for promoting more intense knowledge absorption. Specifically, a series of novel mechanisms are designed elaborately, including correspondence establishment for consistency, hands-on imitation loss measure and re-weighted optimization from both micro and macro perspectives. We conduct extensive evaluations with different distillation configurations over VOC and COCO datasets, which show better performance on accuracy and speed trade-offs. Meanwhile, feasibility experiments on different structural networks further prove the robustness of our HGD.
VDM-DA: Virtual Domain Modeling for Source Data-free Domain Adaptation
Venue: arXiv cs.CV 2021
Year: 2021

Domain adaptation aims to leverage a label-rich domain (the source domain) to help model learning in a label-scarce domain (the target domain). Most domain adaptation methods require the co-existence of source and target domain samples to reduce the distribution mismatch, however, access to the source domain samples may not always be feasible in the real world applications due to different problems (e.g., storage, transmission, and privacy issues). In this work, we deal with the source data-free unsupervised domain adaptation problem, and propose a novel approach referred to as Virtual Domain Modeling (VDM-DA). The virtual domain acts as a bridge between the source and target domains. On one hand, we generate virtual domain samples based on an approximated Gaussian Mixture Model (GMM) in the feature space with the pre-trained source model, such that the virtual domain maintains a similar distribution with the source domain without accessing to the original source data. On the other hand, we also design an effective distribution alignment method to reduce the distribution divergence between the virtual domain and the target domain by gradually improving the compactness of the target domain distribution through model learning. In this way, we successfully achieve the goal of distribution alignment between the source and target domains by training deep networks without accessing to the source domain data. We conduct extensive experiments on benchmark datasets for both 2D image-based and 3D point cloud-based cross-domain object recognition tasks, where the proposed method referred to Domain Adaptation with Virtual Domain Modeling (VDM-DA) achieves the state-of-the-art performances on all datasets.
Base Station Network Traffic Prediction Approach Based on LMA-DeepAR
Venue: arXiv cs.NI 2021
Year: 2021

Accurate network traffic prediction of base station cell is very vital for the expansion and reduction of wireless devices in base station cell. The burst and uncertainty of base station cell network traffic makes the network traffic nonlinear and non-stationary, which brings challenges to the long-term prediction of network traffic. In this paper, the traffic model LMA-DeepAR for base station network is established based on DeepAR. Acordding to the distribution characteristics of network traffic, this paper proposes an artificial feature sequence calculation method based on local moving average (LMA). The feature sequence is input into DeepAR as covariant, which makes the statistical characteristics of network traffic near a period of time in the past be considered when updating parameters, and the interference of non-stationary network traffic on model training will be reduced. Experimental results show that the proposed prediction approach (LMA-DeepAR) outperforms other methods in the overall long-term prediction performance and stability of multi cell network traffic.
Heuristics facilitates the evolution of transitive inference and social hierarchy in a large group
Venue: arXiv cs.GT 2021
Year: 2021

Transitive inference (TI) refers to social cognition that facilitates the discernment of unknown relationships between individuals using known relationships. It is extensively reported that TI evolves in animals living in a large group because TI could assess relative rank without deducing all dyadic relationships, which averts costly fights.The relationships in a large group become so complex that social cognition may not be developed adequately to handle such complexity. If all group members apply TI to all possible members in the group, TI is supposed to require extremely highly developed cognitive abilities especially in a large group. Instead of developing cognitive abilities significantly, animals may apply simplified TI we call reference TI in this study as heuristic approaches. The reference TI allows members to recognize and remember social interactions only among a set of reference members rather than all potential members. Our study assumes that information processes in the reference TI comprises 1) the number of reference members based on which individuals infer transitively, 2) the number of reference members shared by the same strategists, and 3) memory capacity. We examined how information processes evolve in a large group using evolutionary simulations in the hawk-dove game. Information processes with almost any numbers of reference members could evolve in a large group as long as the numbers of shared reference member are high because information from the others' experiences is shared. TI dominates immediate inference, which assesses relative rank on direct interactions, because TI could establish social hierarchy more rapidly applying information from others' experiences.
Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow
Venue: arXiv cs.LG 2021
Year: 2021

In the past decade, model-free reinforcement learning (RL) has provided solutions to challenging domains such as robotics. Model-based RL shows the prospect of being more sample-efficient than model-free methods in terms of agent-environment interactions, because the model enables to extrapolate to unseen situations. In the more recent past, model-based methods have shown superior results compared to model-free methods in some challenging domains with non-linear state transitions. At the same time, it has become apparent that RL is not market-ready yet and that many real-world applications are going to require model-based approaches, because model-free methods are too sample-inefficient and show poor performance in early stages of training. The latter is particularly important in industry, e.g. in production systems that directly impact a company's revenue. This demonstrates the necessity for a toolbox to push the boundaries for model-based RL. While there is a plethora of toolboxes for model-free RL, model-based RL has received little attention in terms of toolbox development. Bellman aims to fill this gap and introduces the first thoroughly designed and tested model-based RL toolbox using state-of-the-art software engineering practices. Our modular approach enables to combine a wide range of environment models with generic model-based agent classes that recover state-of-the-art algorithms. We also provide an experiment harness to compare both model-free and model-based agents in a systematic fashion w.r.t. user-defined evaluation metrics (e.g. cumulative reward). This paves the way for new research directions, e.g. investigating uncertainty-aware environment models that are not necessarily neural-network-based, or developing algorithms to solve industrially-motivated benchmarks that share characteristics with real-world problems.
Self-Supervised Learning in Multi-Task Graphs through Iterative Consensus Shift
Venue: arXiv cs.LG 2021
Year: 2021

The human ability to synchronize the feedback from all their senses inspired recent works in multi-task and multi-modal learning. While these works rely on expensive supervision, our multi-task graph requires only pseudo-labels from expert models. Every graph node represents a task, and each edge learns between tasks transformations. Once initialized, the graph learns self-supervised, based on a novel consensus shift algorithm that intelligently exploits the agreement between graph pathways to generate new pseudo-labels for the next learning cycle. We demonstrate significant improvement from one unsupervised learning iteration to the next, outperforming related recent methods in extensive multi-task learning experiments on two challenging datasets. Our code is available at https://github.com/bit-ml/cshift.
Generalized Planning as Heuristic Search
Venue: arXiv cs.AI 2021
Year: 2021

Although heuristic search is one of the most successful approaches to classical planning, this planning paradigm does not apply straightforwardly to Generalized Planning (GP). Planning as heuristic search traditionally addresses the computation of sequential plans by searching in a grounded state-space. On the other hand GP aims at computing algorithm-like plans, that can branch and loop, and that generalize to a (possibly infinite) set of classical planning instances. This paper adapts the planning as heuristic search paradigm to the particularities of GP, and presents the first native heuristic search approach to GP. First, the paper defines a novel GP solution space that is independent of the number of planning instances in a GP problem, and the size of these instances. Second, the paper defines different evaluation and heuristic functions for guiding a combinatorial search in our GP solution space. Lastly the paper defines a GP algorithm, called Best-First Generalized Planning (BFGP), that implements a best-first search in the solution space guided by our evaluation/heuristic functions.
Composable Learning with Sparse Kernel Representations
Venue: arXiv cs.RO 2021
Year: 2021

We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function (NAF). This representation of the policy enables efficiently composing multiple learned models without additional training samples or interaction with the environment. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment. We apply the composition operation to various policy combinations and test them to show that the composed policies retain the performance of their components. We also transfer the composed policy directly to a physical platform operating in an arena with obstacles in order to demonstrate a degree of generalization.
What happens when a journal converts to Open Access? A bibliometric analysis
Venue: arXiv cs.DL 2021
Year: 2021

In recent years, increased stakeholder pressure to transition research to Open Access has led to many journals converting, or 'flipping', from a closed access (CA) to an open access (OA) publishing model. Changing the publishing model can influence the decision of authors to submit their papers to a journal, and increased article accessibility may influence citation behaviour. In this paper we aimed to understand how flipping a journal to an OA model influences the journal's future publication volumes and citation impact. We analysed two independent sets of journals that had flipped to an OA model, one from the Directory of Open Access Journals (DOAJ) and one from the Open Access Directory (OAD), and compared their development with two respective control groups of similar journals. For bibliometric analyses, journals were matched to the Scopus database. We assessed changes in the number of articles published over time, as well as two citation metrics at the journal and article level: the normalised impact factor (IF) and the average relative citations (ARC), respectively. Our results show that overall, journals that flipped to an OA model increased their publication output compared to journals that remained closed. Mean normalised IF and ARC also generally increased following the flip to an OA model, at a greater rate than was observed in the control groups. However, the changes appear to vary largely by scientific discipline. Overall, these results indicate that flipping to an OA publishing model can bring positive changes to a journal.