OpenRaiser/Intern-Atlas huggingface dataset:
Reward-Free Policy Space Compression for Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

In reinforcement learning, we encode the potential behaviors of an agent interacting with an environment into an infinite set of policies, the policy space, typically represented by a family of parametric functions. Dealing with such a policy space is a hefty challenge, which often causes sample and computation inefficiencies. However, we argue that a limited number of policies are actually relevant when we also account for the structure of the environment and of the policy parameterization, as many of them would induce very similar interactions, i.e., state-action distributions. In this paper, we seek for a reward-free compression of the policy space into a finite set of representative policies, such that, given any policy $π$, the minimum Rényi divergence between the state-action distributions of the representative policies and the state-action distribution of $π$ is bounded. We show that this compression of the policy space can be formulated as a set cover problem, and it is inherently NP-hard. Nonetheless, we propose a game-theoretic reformulation for which a locally optimal solution can be efficiently found by iteratively stretching the compressed space to cover an adversarial policy. Finally, we provide an empirical evaluation to illustrate the compression procedure in simple domains, and its ripple effects in reinforcement learning.
Differentially Private Regression with Unbounded Covariates
Venue: arXiv cs.CR 2022
Year: 2022

We provide computationally efficient, differentially private algorithms for the classical regression settings of Least Squares Fitting, Binary Regression and Linear Regression with unbounded covariates. Prior to our work, privacy constraints in such regression settings were studied under strong a priori bounds on covariates. We consider the case of Gaussian marginals and extend recent differentially private techniques on mean and covariance estimation (Kamath et al., 2019; Karwa and Vadhan, 2018) to the sub-gaussian regime. We provide a novel technical analysis yielding differentially private algorithms for the above classical regression settings. Through the case of Binary Regression, we capture the fundamental and widely-studied models of logistic regression and linearly-separable SVMs, learning an unbiased estimate of the true regression vector, up to a scaling factor.
From Resource Auction to Service Auction: An Auction Paradigm Shift in Wireless Networks
Venue: arXiv cs.NI 2022
Year: 2022

In 5G and beyond, the newly emerging services, such as edge computing/intelligence services, may demand the provision of heterogeneous communications, computing, and storage (CCS) resources on and across network entities multihop apart. In such cases, traditional resource-oriented auction schemes, where buyers place bids on resources, may not be effective in providing end-to-end (E2E) quality-of-service (QoS) guarantees. To overcome these limitations, in this article, we coin the concept of E2E service auction where the auction commodities are E2E services rather than certain resource. Under this framework, buyers simply bid for services with E2E QoS requirements without having to know the inner working (which resources are behind). To guarantee E2E QoS for winning bids while ensuring essential economic properties, E2E service auction requires addressing the joint problem of network optimization and auction design with both economical and QoS constraints. To substantiate the mechanism design, we illustrate how to devise E2E service auctions for edge computing systems under various scenarios. We also identify the research opportunities on E2E service auction mechanism design for other critical use cases, including edge intelligence.
Continual learning-based probabilistic slow feature analysis for multimode dynamic process monitoring
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, a novel multimode dynamic process monitoring approach is proposed by extending elastic weight consolidation (EWC) to probabilistic slow feature analysis (PSFA) in order to extract multimode slow features for online monitoring. EWC was originally introduced in the setting of machine learning of sequential multi-tasks with the aim of avoiding catastrophic forgetting issue, which equally poses as a major challenge in multimode dynamic process monitoring. When a new mode arrives, a set of data should be collected so that this mode can be identified by PSFA and prior knowledge. Then, a regularization term is introduced to prevent new data from significantly interfering with the learned knowledge, where the parameter importance measures are estimated. The proposed method is denoted as PSFA-EWC, which is updated continually and capable of achieving excellent performance for successive modes. Different from traditional multimode monitoring algorithms, PSFA-EWC furnishes backward and forward transfer ability. The significant features of previous modes are retained while consolidating new information, which may contribute to learning new relevant modes. Compared with several known methods, the effectiveness of the proposed method is demonstrated via a continuous stirred tank heater and a practical coal pulverizing system.
Reinforcement Learning in Practice: Opportunities and Challenges
Venue: arXiv cs.LG 2022
Year: 2022

This article is a gentle discussion about the field of reinforcement learning in practice, about opportunities and challenges, touching a broad range of topics, with perspectives and without technical details. The article is based on both historical and recent research papers, surveys, tutorials, talks, blogs, books, (panel) discussions, and workshops/conferences. Various groups of readers, like researchers, engineers, students, managers, investors, officers, and people wanting to know more about the field, may find the article interesting. In this article, we first give a brief introduction to reinforcement learning (RL), and its relationship with deep learning, machine learning and AI. Then we discuss opportunities of RL, in particular, products and services, games, bandits, recommender systems, robotics, transportation, finance and economics, healthcare, education, combinatorial optimization, computer systems, and science and engineering. Then we discuss challenges, in particular, 1) foundation, 2) representation, 3) reward, 4) exploration, 5) model, simulation, planning, and benchmarks, 6) off-policy/offline learning, 7) learning to learn a.k.a. meta-learning, 8) explainability and interpretability, 9) constraints, 10) software development and deployment, 11) business perspectives, and 12) more challenges. We conclude with a discussion, attempting to answer: "Why has RL not been widely adopted in practice yet?" and "When is RL helpful?".
Trajectory planning in Dynamics Environment : Application for Haptic Perception in Safe HumanRobot Interaction
Venue: arXiv cs.RO 2022
Year: 2022

In a human-robot interaction system, the most important thing to consider is the safety of the user. This must be guaranteed in order to implement a reliable system. The main objective of this paper is to generate a safe motion scheme that takes into account the obstacles present in a virtual reality (VR) environment. The work is developed using the MoveIt software in ROS to control an industrial robot UR5. Thanks to this, we will be able to set up the planning group, which is realized by the UR5 robot with a 6-sided prop and the base of the manipulator, in order to plan feasible trajectories that it will be able to execute in the environment. The latter is based on the interior of a vehicle, containing a person (which would be the user in this case) for which the configuration will also be made to be taken into account in the system. To do this, we first investigated the software's capabilities and options for path planning, as well as the different ways to execute the movements. We also compared the different trajectory planning algorithms that the software is capable of using in order to determine which one is best suited for the task. Finally, we proposed different mobility schemes to be executed by the robot depending on the situation it is facing. The first one is used when the robot has to plan trajectories in a safe space, where the only obstacle to avoid is the user's workspace. The second one is used when the robot has to interact with the user, where a dummy model represents the user's position as a function of time, which is the one to be avoided.
Exploratory Methods for Relation Discovery in Archival Data
Venue: arXiv cs.LG 2022
Year: 2022

In this article we propose a holistic approach to discover relations in art historical communities and enrich historians' biographies and archival descriptions with graph patterns relevant to art historiographic enquiry. We use exploratory data analysis to detect patterns, we select features, and we use them to evaluate classification models to predict new relations, to be recommended to archivists during the cataloguing phase. Results show that relations based on biographical information can be addressed with higher precision than relations based on research topics or institutional relations. Deterministic and a priori rules present better results than probabilistic methods.
Model-Driven Generation of Microservice Interfaces: From LEMMA Domain Models to Jolie APIs
Venue: arXiv cs.SE 2022
Year: 2022

We formally define and implement a translation from domain models in the LEMMA modelling framework to microservice APIs in the Jolie programming language. Our tool enables a software development process whereby microservice architectures can first be designed with the leading method of Domain-Driven Design, and then corresponding data types and service interfaces (APIs) in Jolie are automatically generated. Developers can extend and use these APIs as guides in order to produce compliant implementations. Our tool thus contributes to enhancing productivity and improving the design adherence of microservices.
Testing report of a fingerprint-based door-opening system
Venue: arXiv cs.LG 2022
Year: 2022

This paper describes the operational evaluation of a door-opening system based on a low-cost inkless fingerprint sensor. This system has been developed and installed for access control to one of our laboratories. Experimental results reveal that the system is working fine and no special cleaning requirements neither components replacement is needed. It can support more than 50 users, and an average of 74,5 access attempts per day in a 14-hour 5-day-per-week working. Emphasize is also given on some important facts to be taken into consideration when comparing and evaluating different products from different vendors.
Fairness-Aware Naive Bayes Classifier for Data with Multiple Sensitive Features
Venue: arXiv cs.LG 2022
Year: 2022

Fairness-aware machine learning seeks to maximise utility in generating predictions while avoiding unfair discrimination based on sensitive attributes such as race, sex, religion, etc. An important line of work in this field is enforcing fairness during the training step of a classifier. A simple yet effective binary classification algorithm that follows this strategy is two-naive-Bayes (2NB), which enforces statistical parity - requiring that the groups comprising the dataset receive positive labels with the same likelihood. In this paper, we generalise this algorithm into N-naive-Bayes (NNB) to eliminate the simplification of assuming only two sensitive groups in the data and instead apply it to an arbitrary number of groups. We propose an extension of the original algorithm's statistical parity constraint and the post-processing routine that enforces statistical independence of the label and the single sensitive attribute. Then, we investigate its application on data with multiple sensitive features and propose a new constraint and post-processing routine to enforce differential fairness, an extension of established group-fairness constraints focused on intersectionalities. We empirically demonstrate the effectiveness of the NNB algorithm on US Census datasets and compare its accuracy and debiasing performance, as measured by disparate impact and DF-$ε$ score, with similar group-fairness algorithms. Finally, we lay out important considerations users should be aware of before incorporating this algorithm into their application, and direct them to further reading on the pros, cons, and ethical implications of using statistical parity as a fairness criterion.
Shisha: Online scheduling of CNN pipelines on heterogeneous architectures
Venue: arXiv cs.PF 2022
Year: 2022

Chiplets have become a common methodology in modern chip design. Chiplets improve yield and enable heterogeneity at the level of cores, memory subsystem and the interconnect. Convolutional Neural Networks (CNNs) have high computational, bandwidth and memory capacity requirements owing to the increasingly large amount of weights. Thus to exploit chiplet-based architectures, CNNs must be optimized in terms of scheduling and workload distribution among computing resources. We propose Shisha, an online approach to generate and schedule parallel CNN pipelines on chiplet architectures. Shisha targets heterogeneity in compute performance and memory bandwidth and tunes the pipeline schedule through a fast online exploration technique. We compare Shisha with Simulated Annealing, Hill Climbing and Pipe-Search. On average, the convergence time is improved by ~35x in Shisha compared to other exploration algorithms. Despite the quick exploration, Shisha's solution is often better than that of other heuristic exploration algorithms.
LUCE: A Blockchain-based data sharing platform for monitoring data license accountability and compliance
Venue: arXiv cs.DC 2022
Year: 2022

Easy access to data is one of the main avenues to accelerate scientific research. As a key element of scientific innovations, data sharing allows the reproduction of results, helps prevent data fabrication, falsification, and misuse. Although the research benefits from data reuse are widely acknowledged, the data collections existing today are still kept in silos. Indeed, monitoring what happens to data once they have been handed to a third party is currently not feasible within the current data-sharing practices. We propose a blockchain-based system to trace data collections, and potentially create a more trustworthy data sharing process. In this paper, we present the LUCE (License accoUntability and CompliancE) architecture as a decentralized blockchain-based platform supporting data sharing and reuse. LUCE is designed to provide full transparency on what happens to the data after they are shared with third parties. The contributions of this work are: the definition of a generic model and an implementation for decentralized data sharing accountability and compliance and to incorporates dynamic consent and legal compliance mechanisms. We test the scalability of the platform in a real-time environment where a growing number of users access and reuse different datasets. Compared to existing data-sharing solutions, LUCE provides transparency over data sharing practices, enables data reuse and supports regulatory requirements. The experimentation shows that the platform can be scaled for a large number of users.
MuMiN: A Large-Scale Multilingual Multimodal Fact-Checked Misinformation Social Network Dataset
Venue: arXiv cs.LG 2022
Year: 2022

Misinformation is becoming increasingly prevalent on social media and in news articles. It has become so widespread that we require algorithmic assistance utilising machine learning to detect such content. Training these machine learning models require datasets of sufficient scale, diversity and quality. However, datasets in the field of automatic misinformation detection are predominantly monolingual, include a limited amount of modalities and are not of sufficient scale and quality. Addressing this, we develop a data collection and linking system (MuMiN-trawl), to build a public misinformation graph dataset (MuMiN), containing rich social media data (tweets, replies, users, images, articles, hashtags) spanning 21 million tweets belonging to 26 thousand Twitter threads, each of which have been semantically linked to 13 thousand fact-checked claims across dozens of topics, events and domains, in 41 different languages, spanning more than a decade. The dataset is made available as a heterogeneous graph via a Python package (mumin). We provide baseline results for two node classification tasks related to the veracity of a claim involving social media, and demonstrate that these are challenging tasks, with the highest macro-average F1-score being 62.55% and 61.45% for the two tasks, respectively. The MuMiN ecosystem is available at https://mumin-dataset.github.io/, including the data, documentation, tutorials and leaderboards.
On the Arithmetic Complexity of the Bandwidth of Bandlimited Signals
Venue: arXiv cs.IT 2022
Year: 2022

The bandwidth of a signal is an important physical property that is of relevance in many signal- and information-theoretic applications. In this paper we study questions related to the computability of the bandwidth of computable bandlimited signals. To this end we employ the concept of Turing computability, which exactly describes what is theoretically feasible and can be computed on a digital computer. Recently, it has been shown that there exist computable bandlimited signals with finite energy, the actual bandwidth of which is not a computable number, and hence cannot be computed on a digital computer. In this work, we consider the most general class of band-limited signals, together with different computable representations thereof. Among other things, our analysis includes a characterization of the arithmetic complexity of the bandwidth of such signals and yields a negative answer to the question of whether it is at least possible to compute non-trivial upper or lower bounds for the bandwidth of a bandlimited signal. Furthermore, we relate the problem of bandwidth computation to the theory of oracle machines. In particular, we consider halting and totality oracles, which belong to the most frequently investigated oracle machines in the theory of computation.
Stochastic Coded Offloading Scheme for Unmanned Aerial Vehicle-Assisted Edge Computing
Venue: arXiv cs.IT 2022
Year: 2022

Unmanned aerial vehicles (UAVs) have gained wide research interests due to their technological advancement and high mobility. The UAVs are equipped with increasingly advanced capabilities to run computationally intensive applications enabled by machine learning techniques. However, because of both energy and computation constraints, the UAVs face issues hovering in the sky while performing computation due to weather uncertainty. To overcome the computation constraints, the UAVs can partially or fully offload their computation tasks to the edge servers. In ordinary computation offloading operations, the UAVs can retrieve the result from the returned output. Nevertheless, if the UAVs are unable to retrieve the entire result from the edge servers, i.e., straggling edge servers, this operation will fail. In this paper, we propose a coded distributed computing approach for computation offloading to mitigate straggling edge servers. The UAVs can retrieve the returned result when the number of returned copies is greater than or equal to the recovery threshold. There is a shortfall if the returned copies are less than the recovery threshold. To minimize the cost of the network, energy consumption by the UAVs, and prevent over and under subscription of the resources, we devise a two-phase Stochastic Coded Offloading Scheme (SCOS). In the first phase, the appropriate UAVs are allocated to the charging stations amid weather uncertainty. In the second phase, we use the $z$-stage Stochastic Integer Programming (SIP) to optimize the number of computation subtasks offloaded and computed locally, while taking into account the computation shortfall and demand uncertainty. By using a real dataset, the simulation results show that our proposed scheme is fully dynamic, and minimizes the cost of the network and UAV energy consumption amid stochastic uncertainties.
Deep Reinforcement Learning based Joint Active and Passive Beamforming Design for RIS-Assisted MISO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Owing to the unique advantages of low cost and controllability, reconfigurable intelligent surface (RIS) is a promising candidate to address the blockage issue in millimeter wave (mmWave) communication systems, consequently has captured widespread attention in recent years. However, the joint active beamforming and passive beamforming design is an arduous task due to the high computational complexity and the dynamic changes of wireless environment. In this paper, we consider a RIS-assisted multi-user multiple-input single-output (MU-MISO) mmWave system and aim to develop a deep reinforcement learning (DRL) based algorithm to jointly design active hybrid beamformer at the base station (BS) side and passive beamformer at the RIS side. By employing an advanced soft actor-critic (SAC) algorithm, we propose a maximum entropy based DRL algorithm, which can explore more stochastic policies than deterministic policy, to design active analog precoder and passive beamformer simultaneously. Then, the digital precoder is determined by minimum mean square error (MMSE) method. The experimental results demonstrate that our proposed SAC algorithm can achieve better performance compared with conventional optimization algorithm and DRL algorithm.
Improving Scalability with GPU-Aware Asynchronous Tasks
Venue: arXiv cs.DC 2022
Year: 2022

Asynchronous tasks, when created with over-decomposition, enable automatic computation-communication overlap which can substantially improve performance and scalability. This is not only applicable to traditional CPU-based systems, but also to modern GPU-accelerated platforms. While the ability to hide communication behind computation can be highly effective in weak scaling scenarios, performance begins to suffer with smaller problem sizes or in strong scaling due to fine-grained overheads and reduced room for overlap. In this work, we integrate GPU-aware communication into asynchronous tasks in addition to computation-communication overlap, with the goal of reducing time spent in communication and further increasing GPU utilization. We demonstrate the performance impact of our approach using a proxy application that performs the Jacobi iterative method, Jacobi3D. In addition to optimizations to minimize synchronizations between the host and GPU devices and increase the concurrency of GPU operations, we explore techniques such as kernel fusion and CUDA Graphs to mitigate fine-grained overheads at scale.
Scalpel: The Python Static Analysis Framework
Venue: arXiv cs.SE 2022
Year: 2022

Despite being the most popular programming language, Python has not yet received enough attention from the community. To the best of our knowledge, there is no general static analysis framework proposed to facilitate the implementation of dedicated Python static analyzers. To fill this gap, we design and implement such a framework (named Scalpel) and make it publicly available as an open-source project. The Scalpel framework has already integrated a number of fundamental static analysis functions (e.g., call graph constructions, control-flow graph constructions, alias analysis, etc.) that are ready to be reused by developers to implement client applications focusing on statically resolving dedicated Python problems such as detecting bugs or fixing vulnerabilities.