OpenRaiser/Intern-Atlas huggingface dataset:
L2Explorer: A Lifelong Reinforcement Learning Assessment Environment
Venue: arXiv cs.LG 2022
Year: 2022

Despite groundbreaking progress in reinforcement learning for robotics, gameplay, and other complex domains, major challenges remain in applying reinforcement learning to the evolving, open-world problems often found in critical application spaces. Reinforcement learning solutions tend to generalize poorly when exposed to new tasks outside of the data distribution they are trained on, prompting an interest in continual learning algorithms. In tandem with research on continual learning algorithms, there is a need for challenge environments, carefully designed experiments, and metrics to assess research progress. We address the latter need by introducing a framework for continual reinforcement-learning development and assessment using Lifelong Learning Explorer (L2Explorer), a new, Unity-based, first-person 3D exploration environment that can be continuously reconfigured to generate a range of tasks and task variants structured into complex and evolving evaluation curricula. In contrast to procedurally generated worlds with randomized components, we have developed a systematic approach to defining curricula in response to controlled changes with accompanying metrics to assess transfer, performance recovery, and data efficiency. Taken together, the L2Explorer environment and evaluation approach provides a framework for developing future evaluation methodologies in open-world settings and rigorously evaluating approaches to lifelong learning.
Quantum Information Elicitation
Venue: arXiv cs.GT 2022
Year: 2022

In the classic scoring rule setting, a principal incentivizes an agent to truthfully report their probabilistic belief about some future outcome. This paper addresses the situation when this private belief, rather than a classical probability distribution, is instead a quantum mixed state. In the resulting quantum scoring rule setting, the principal chooses both a scoring function and a measurement function, and the agent responds with their reported density matrix. Several characterizations of quantum scoring rules are presented, which reveal a familiar structure based on convex analysis. Spectral scores, where the measurement function is given by the spectral decomposition of the reported density matrix, have particularly elegant structure and connect to quantum information theory. Turning to property elicitation, eigenvectors of the belief are elicitable, whereas eigenvalues and entropy have maximal elicitation complexity. The paper concludes with a discussion of other quantum information elicitation settings and connections to the literature.
Convolutional-Recurrent Neural Network Proxy for Robust Optimization and Closed-Loop Reservoir Management
Venue: arXiv cs.LG 2022
Year: 2022

Production optimization under geological uncertainty is computationally expensive, as a large number of well control schedules must be evaluated over multiple geological realizations. In this work, a convolutional-recurrent neural network (CNN-RNN) proxy model is developed to predict well-by-well oil and water rates, for given time-varying well bottom-hole pressure (BHP) schedules, for each realization in an ensemble. This capability enables the estimation of the objective function and nonlinear constraint values required for robust optimization. The proxy model represents an extension of a recently developed long short-term memory (LSTM) RNN proxy designed to predict well rates for a single geomodel. A CNN is introduced here to processes permeability realizations, and this provides the initial states for the RNN. The CNN-RNN proxy is trained using simulation results for 300 different sets of BHP schedules and permeability realizations. We demonstrate proxy accuracy for oil-water flow through multiple realizations of 3D multi-Gaussian permeability models. The proxy is then incorporated into a closed-loop reservoir management (CLRM) workflow, where it is used with particle swarm optimization and a filter-based method for nonlinear constraint satisfaction. History matching is achieved using an adjoint-gradient-based procedure. The proxy model is shown to perform well in this setting for five different (synthetic) `true' models. Improved net present value along with constraint satisfaction and uncertainty reduction are observed with CLRM. For the robust production optimization steps, the proxy provides O(100) runtime speedup over simulation-based optimization.
A Linearly Convergent Douglas-Rachford Splitting Solver for Markovian Information-Theoretic Optimization Problems
Venue: arXiv cs.IT 2022
Year: 2022

In this work, we propose solving the Information bottleneck (IB) and Privacy Funnel (PF) problems with Douglas-Rachford Splitting methods (DRS). We study a general Markovian information-theoretic Lagrangian that includes IB and PF into a unified framework. We prove the linear convergence of the proposed solvers using the Kurdyka-Łojasiewicz inequality. Moreover, our analysis is beyond IB and PF and applies to any convex-weakly convex pair objectives. Based on the results, we develop two types of linearly convergent IB solvers, with one improves the performance of convergence over existing solvers while the other can be independent to the relevance-compression trade-off. Moreover, our results apply to PF, yielding a new class of linearly convergent PF solvers. Empirically, the proposed IB solvers IB obtain solutions that are comparable to the Blahut-Arimoto-based benchmark and is convergent for a wider range of the penalty coefficient than existing solvers. For PF, our non-greedy solvers can characterize the privacy-utility trade-off better than the clustering-based greedy solvers.
Learning for Robot Decision Making under Distribution Shift: A Survey
Venue: arXiv cs.RO 2022
Year: 2022

With the recent advances in the field of deep learning, learning-based methods are widely being implemented in various robotic systems that help robots understand their environment and make informed decisions to achieve a wide variety of tasks or goals. However, learning-based methods have repeatedly been shown to have poor generalization when they are presented with inputs that are different from those during training leading to the problem of distribution shift. Any robotic system that employs learning-based methods is prone to distribution shift which might lead the agents to make decisions that lead to degraded performance or even catastrophic failure. In this paper, we discuss various techniques that have been proposed in the literature to aid or improve decision making under distribution shift for robotic systems. We present a taxonomy of existing literature and present a survey of existing approaches in the area based on this taxonomy. Finally, we also identify a few open problems in the area that could serve as future directions for research.
Accelerating Stochastic Probabilistic Inference
Venue: arXiv cs.LG 2022
Year: 2022

Recently, Stochastic Variational Inference (SVI) has been increasingly attractive thanks to its ability to find good posterior approximations of probabilistic models. It optimizes the variational objective with stochastic optimization, following noisy estimates of the natural gradient. However, almost all the state-of-the-art SVI algorithms are based on first-order optimization algorithm and often suffer from poor convergence rate. In this paper, we bridge the gap between second-order methods and stochastic variational inference by proposing a second-order based stochastic variational inference approach. In particular, firstly we derive the Hessian matrix of the variational objective. Then we devise two numerical schemes to implement second-order SVI efficiently. Thorough empirical evaluations are investigated on both synthetic and real dataset to backup both the effectiveness and efficiency of the proposed approach.
When Cell-Free Massive MIMO Meets OTFS Modulation: The Downlink Case
Venue: arXiv cs.IT 2022
Year: 2022

We provide a performance evaluation of orthogonal time frequency space (OTFS) modulation in cell-free massive MIMO (multiple-input multiple-output) systems. By leveraging the inherent sparsity of the delay-Doppler (DD) representation of time-varying channels, we apply the embedded pilot-aided channel estimation method with reduced guard intervals and derive the minimum mean-square error estimate of the channel gains from received uplink pilots at the access points (APs). Each AP applies conjugate beamforming to transmit data to the users. We derive a closed-form expression for the individual user downlink throughput as a function of the numbers of APs, users and DD channel estimate parameters. We compare the OTFS performance with that of orthogonal frequency division multiplexing (OFDM) at high-mobility conditions. Our findings reveal that with uncorrelated shadowing, cell-free massive MIMO with OTFS modulation achieves up to 35% gain in 95%-likely per-user throughput, compared with the OFDM counterpart. Finally, the increase in the per user throughput is more pronounced at the median rates over the correlated shadowing scenarios.
SmartValidator: A Framework for Automatic Identification and Classification of Cyber Threat Data
Venue: arXiv cs.CR 2022
Year: 2022

A wide variety of Cyber Threat Information (CTI) is used by Security Operation Centres (SOCs) to perform validation of security incidents and alerts. Security experts manually define different types of rules and scripts based on CTI to perform validation tasks. These rules and scripts need to be updated continuously due to evolving threats, changing SOCs' requirements and dynamic nature of CTI. The manual process of updating rules and scripts delays the response to attacks. To reduce the burden of human experts and accelerate response, we propose a novel Artificial Intelligence (AI) based framework, SmartValidator. SmartValidator leverages Machine Learning (ML) techniques to enable automated validation of alerts. It consists of three layers to perform the tasks of data collection, model building and alert validation. It projects the validation task as a classification problem. Instead of building and saving models for all possible requirements, we propose to automatically construct the validation models based on SOC's requirements and CTI. We built a Proof of Concept (PoC) system with eight ML algorithms, two feature engineering techniques and 18 requirements to investigate the effectiveness and efficiency of SmartValidator. The evaluation results showed that when prediction models were built automatically for classifying cyber threat data, the F1-score of 75\% of the models were above 0.8, which indicates adequate performance of the PoC for use in a real-world organization. The results further showed that dynamic construction of prediction models required 99\% less models to be built than pre-building models for all possible requirements. The framework can be followed by various industries to accelerate and automate the validation of alerts and incidents based on their CTI and SOC's preferences.
Visualization in virtual reality: a systematic review
Venue: arXiv cs.HC 2022
Year: 2022

Rapidly growing virtual reality (VR) technologies and techniques have gained importance over the past few years, and academics and practitioners have been searching for efficient visualizations in VR. To date, emphasis has been on the employment of game technologies. Despite the growing interest and discussion, visualization studies have lacked a common baseline in the transition period of 2D visualizations to immersive ones. To this end, the presented study aims to provide a systematic literature review that explains the state-of-the-art research and future trends on visualization in virtual reality. The research framework is grounded in empirical and theoretical works of visualization. We characterize the reviewed literature based on three dimensions: (a) Connection with visualization background and theory, (b) Evaluation and design considerations for virtual reality visualization, and (c) Empirical studies. The results from this systematic review suggest that: (1) There are only a few studies that focus on creating standard guidelines for virtual reality, and each study individually provides a framework or employs previous studies on traditional 2D visualizations; (2) With the myriad of advantages provided for visualization and virtual reality, most of the studies prefer to use game engines; (3) Although game engines are extensively used, they are not convenient for critical scientific studies; and (4) 3D versions of traditional statistical visualization techniques, such as bar plots and scatter plots, are still commonly used in the data visualization context. This systematic review attempts to add to the literature a clear picture of the emerging contexts, different elements, and their interdependencies.
Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue
Venue: arXiv cs.CL 2022
Year: 2022

Complex conversation settings such as persuasion involve communicating changes in attitude or behavior, so users' perspectives need to be addressed, even when not directly related to the topic. In this work, we contribute a novel modular dialogue system framework that seamlessly integrates factual information and social content into persuasive dialogue. Our framework is generalizable to any dialogue tasks that have mixed social and task contents. We conducted a study that compared user evaluations of our framework versus a baseline end-to-end generation model. We found our framework was evaluated more favorably in all dimensions including competence and friendliness, compared to the end-to-end model which does not explicitly handle social content or factual questions.
Towards Adversarial Control Loops in Sensor Attacks: A Case Study to Control the Kinematics and Actuation of Embedded Systems
Venue: arXiv cs.CR 2022
Year: 2022

Recent works investigated attacks on sensors by influencing analog sensor components with acoustic, light, and electromagnetic signals. Such attacks can have extensive security, reliability, and safety implications since many types of the targeted sensors are also widely used in critical process control, robotics, automation, and industrial control systems. While existing works advanced our understanding of the physical-level risks that are hidden from a digital-domain perspective, gaps exist in how the attack can be guided to achieve system-level control in real-time, continuous processes. This paper proposes an adversarial control loop-based approach for real-time attacks on control systems relying on sensors. We study how to utilize the system feedback extracted from physical-domain signals to guide the attacks. In the attack process, injection signals are adjusted in real time based on the extracted feedback to exert targeted influence on a victim control system that is continuously affected by the injected perturbations and applying changes to the physical environment. In our case study, we investigate how an external adversarial control system can be constructed over sensor-actuator systems and demonstrate the attacks with program-controlled processes to manipulate the victim system without accessing its internal statuses.
An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility
Venue: arXiv cs.AI 2022
Year: 2022

Many scenarios in mobility and traffic involve multiple different agents that need to cooperate to find a joint solution. Recent advances in behavioral planning use Reinforcement Learning to find effective and performant behavior strategies. However, as autonomous vehicles and vehicle-to-X communications become more mature, solutions that only utilize single, independent agents leave potential performance gains on the road. Multi-Agent Reinforcement Learning (MARL) is a research field that aims to find optimal solutions for multiple agents that interact with each other. This work aims to give an overview of the field to researchers in autonomous mobility. We first explain MARL and introduce important concepts. Then, we discuss the central paradigms that underlie MARL algorithms, and give an overview of state-of-the-art methods and ideas in each paradigm. With this background, we survey applications of MARL in autonomous mobility scenarios and give an overview of existing scenarios and implementations.
InsCon:Instance Consistency Feature Representation via Self-Supervised Learning
Venue: arXiv cs.CV 2022
Year: 2022

Feature representation via self-supervised learning has reached remarkable success in image-level contrastive learning, which brings impressive performances on image classification tasks. While image-level feature representation mainly focuses on contrastive learning in single instance, it ignores the objective differences between pretext and downstream prediction tasks such as object detection and instance segmentation. In order to fully unleash the power of feature representation on downstream prediction tasks, we propose a new end-to-end self-supervised framework called InsCon, which is devoted to capturing multi-instance information and extracting cell-instance features for object recognition and localization. On the one hand, InsCon builds a targeted learning paradigm that applies multi-instance images as input, aligning the learned feature between corresponding instance views, which makes it more appropriate for multi-instance recognition tasks. On the other hand, InsCon introduces the pull and push of cell-instance, which utilizes cell consistency to enhance fine-grained feature representation for precise boundary localization. As a result, InsCon learns multi-instance consistency on semantic feature representation and cell-instance consistency on spatial feature representation. Experiments demonstrate the method we proposed surpasses MoCo v2 by 1.1% AP^{bb} on COCO object detection and 1.0% AP^{mk} on COCO instance segmentation using Mask R-CNN R50-FPN network structure with 90k iterations, 2.1% APbb on PASCAL VOC objection detection using Faster R-CNN R50-C4 network structure with 24k iterations.
ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation
Venue: arXiv cs.CV 2022
Year: 2022

We present a GAN-based Transformer for general action-conditioned 3D human motion generation, including not only single-person actions but also multi-person interactive actions. Our approach consists of a powerful Action-conditioned motion TransFormer (ActFormer) under a GAN training scheme, equipped with a Gaussian Process latent prior. Such a design combines the strong spatio-temporal representation capacity of Transformer, superiority in generative modeling of GAN, and inherent temporal correlations from the latent prior. Furthermore, ActFormer can be naturally extended to multi-person motions by alternately modeling temporal correlations and human interactions with Transformer encoders. To further facilitate research on multi-person motion generation, we introduce a new synthetic dataset of complex multi-person combat behaviors. Extensive experiments on NTU-13, NTU RGB+D 120, BABEL and the proposed combat dataset show that our method can adapt to various human motion representations and achieve superior performance over the state-of-the-art methods on both single-person and multi-person motion generation tasks, demonstrating a promising step towards a general human motion generator.
Evaluating BERT-based Pre-training Language Models for Detecting Misinformation
Venue: arXiv cs.CL 2022
Year: 2022

It is challenging to control the quality of online information due to the lack of supervision over all the information posted online. Manual checking is almost impossible given the vast number of posts made on online media and how quickly they spread. Therefore, there is a need for automated rumour detection techniques to limit the adverse effects of spreading misinformation. Previous studies mainly focused on finding and extracting the significant features of text data. However, extracting features is time-consuming and not a highly effective process. This study proposes the BERT- based pre-trained language models to encode text data into vectors and utilise neural network models to classify these vectors to detect misinformation. Furthermore, different language models (LM) ' performance with different trainable parameters was compared. The proposed technique is tested on different short and long text datasets. The result of the proposed technique has been compared with the state-of-the-art techniques on the same datasets. The results show that the proposed technique performs better than the state-of-the-art techniques. We also tested the proposed technique by combining the datasets. The results demonstrated that the large data training and testing size considerably improves the technique's performance.
CSRS: Code Search with Relevance Matching and Semantic Matching
Venue: arXiv cs.SE 2022
Year: 2022

Developers often search and reuse existing code snippets in the process of software development. Code search aims to retrieve relevant code snippets from a codebase according to natural language queries entered by the developer. Up to now, researchers have already proposed information retrieval (IR) based methods and deep learning (DL) based methods. The IR-based methods focus on keyword matching, that is to rank codes by relevance between queries and code snippets, while DL-based methods focus on capturing the semantic correlations. However, the existing methods do not consider capturing two matching signals simultaneously. Therefore, in this paper, we propose CSRS, a code search model with relevance matching and semantic matching. CSRS comprises (1) an embedding module containing convolution kernels of different sizes which can extract n-gram embeddings of queries and codes, (2) a relevance matching module that measures lexical matching signals, and (3) a co-attention based semantic matching module to capture the semantic correlation. We train and evaluate CSRS on a dataset with 18.22M and 10k code snippets. The experimental results demonstrate that CSRS achieves an MRR of 0.614, which outperforms two state-of-the-art models DeepCS and CARLCS-CNN by 33.77% and 18.53% respectively. In addition, we also conducted several experiments to prove the effectiveness of each component of CSRS.
Social Choice Around the Block: On the Computational Social Choice of Blockchain
Venue: arXiv cs.GT 2022
Year: 2022

One of the most innovative aspects of blockchain technology consists in the introduction of an incentive layer to regulate the behavior of distributed protocols. The designer of a blockchain system faces therefore issues that are akin to those relevant for the design of economic mechanisms, and faces them in a computational setting. From this perspective the present paper argues for the importance of computational social choice in blockchain research. It identifies a few challenges at the interface of the two fields that illustrate the strong potential for cross-fertilization between them.