OpenRaiser/Intern-Atlas huggingface dataset:
Semiparametric Language Models Are Scalable Continual Learners
Venue: arXiv cs.CL 2023
Year: 2023

Semiparametric language models (LMs) have shown promise in continuously learning from new text data by combining a parameterized neural LM with a growable non-parametric memory for memorizing new content. However, conventional semiparametric LMs will finally become prohibitive for computing and storing if they are applied to continual learning over streaming data, because the non-parametric memory grows linearly with the amount of data they learn from over time. To address the issue of scalability, we present a simple and intuitive approach called Selective Memorization (SeMem), which only memorizes difficult samples that the model is likely to struggle with. We demonstrate that SeMem improves the scalability of semiparametric LMs for continual learning over streaming data in two ways: (1) data-wise scalability: as the model becomes stronger through continual learning, it will encounter fewer difficult cases that need to be memorized, causing the growth of the non-parametric memory to slow down over time rather than growing at a linear rate with the size of training data; (2) model-wise scalability: SeMem allows a larger model to memorize fewer samples than its smaller counterpart because it is rarer for a larger model to encounter incomprehensible cases, resulting in a non-parametric memory that does not scale linearly with model size. We conduct extensive experiments in language modeling and downstream tasks to test SeMem's results, showing SeMem enables a semiparametric LM to be a scalable continual learner with little forgetting.
EPAM: A Predictive Energy Model for Mobile AI
Venue: arXiv cs.LG 2023
Year: 2023

Artificial intelligence (AI) has enabled a new paradigm of smart applications -- changing our way of living entirely. Many of these AI-enabled applications have very stringent latency requirements, especially for applications on mobile devices (e.g., smartphones, wearable devices, and vehicles). Hence, smaller and quantized deep neural network (DNN) models are developed for mobile devices, which provide faster and more energy-efficient computation for mobile AI applications. However, how AI models consume energy in a mobile device is still unexplored. Predicting the energy consumption of these models, along with their different applications, such as vision and non-vision, requires a thorough investigation of their behavior using various processing sources. In this paper, we introduce a comprehensive study of mobile AI applications considering different DNN models and processing sources, focusing on computational resource utilization, delay, and energy consumption. We measure the latency, energy consumption, and memory usage of all the models using four processing sources through extensive experiments. We explain the challenges in such investigations and how we propose to overcome them. Our study highlights important insights, such as how mobile AI behaves in different applications (vision and non-vision) using CPU, GPU, and NNAPI. Finally, we propose a novel Gaussian process regression-based general predictive energy model based on DNN structures, computation resources, and processors, which can predict the energy for each complete application cycle irrespective of device configuration and application. This study provides crucial facts and an energy prediction mechanism to the AI research community to help bring energy efficiency to mobile AI applications.
Simultaneous prediction of hand gestures, handedness, and hand keypoints using thermal images
Venue: arXiv cs.CV 2023
Year: 2023

Hand gesture detection is a well-explored area in computer vision with applications in various forms of Human-Computer Interactions. In this work, we propose a technique for simultaneous hand gesture classification, handedness detection, and hand keypoints localization using thermal data captured by an infrared camera. Our method uses a novel deep multi-task learning architecture that includes shared encoderdecoder layers followed by three branches dedicated for each mentioned task. We performed extensive experimental validation of our model on an in-house dataset consisting of 24 users data. The results confirm higher than 98 percent accuracy for gesture classification, handedness detection, and fingertips localization, and more than 91 percent accuracy for wrist points localization.
HitGNN: High-throughput GNN Training Framework on CPU+Multi-FPGA Heterogeneous Platform
Venue: arXiv cs.DC 2023
Year: 2023

As the size of real-world graphs increases, training Graph Neural Networks (GNNs) has become time-consuming and requires acceleration. While previous works have demonstrated the potential of utilizing FPGA for accelerating GNN training, few works have been carried out to accelerate GNN training with multiple FPGAs due to the necessity of hardware expertise and substantial development effort. To this end, we propose HitGNN, a framework that enables users to effortlessly map GNN training workloads onto a CPU-Multi-FPGA platform for acceleration. In particular, HitGNN takes the user-defined synchronous GNN training algorithm, GNN model, and platform metadata as input, determines the design parameters based on the platform metadata, and performs hardware mapping onto the CPU+Multi-FPGA platform, automatically. HitGNN consists of the following building blocks: (1) high-level application programming interfaces (APIs) that allow users to specify various synchronous GNN training algorithms and GNN models with only a handful of lines of code; (2) a software generator that generates a host program that performs mini-batch sampling, manages CPU-FPGA communication, and handles workload balancing among the FPGAs; (3) an accelerator generator that generates GNN kernels with optimized datapath and memory organization. We show that existing synchronous GNN training algorithms such as DistDGL and PaGraph can be easily deployed on a CPU+Multi-FPGA platform using our framework, while achieving high training throughput. Compared with the state-of-the-art frameworks that accelerate synchronous GNN training on a multi-GPU platform, HitGNN achieves up to 27.21x bandwidth efficiency, and up to 4.26x speedup using much less compute power and memory bandwidth than GPUs. In addition, HitGNN demonstrates good scalability to 16 FPGAs on a CPU+Multi-FPGA platform.
Developing a Compiler for EROP -- A Language for the Specification of Smart Contracts, An Experience Report
Venue: arXiv cs.PL 2023
Year: 2023

A smart contract is a translation of a standard paper-based contract that can be enforced and executed by a contract management system. At a high level of abstraction, a contract is only a document that describes how the signing parties are to behave in different scenarios; nevertheless, the translation of a typical paper-based contract to its electronic counterpart has proved to be both time-consuming and difficult. The requirement for a language capable of capturing the core of a contract in simple phrases and definitions has been a focus of study for many years. EROP (Events, Rights, Obligations, Prohibitions) is a contract specification language that breaks a contract down into sets of events, rights, obligations, and prohibitions.
Q2Logic: An Coarse-Grained Architecture targeting Schrödinger Quantum Circuit Simulations
Venue: arXiv cs.AR 2023
Year: 2023

Quantum computing is emerging as an important (but radical) technology that might take us beyond Moore's law for certain applications. Today, in parallel with improving quantum computers, computer scientists are relying heavily on quantum circuit simulators to develop algorithms. Most existing quantum circuit simulators run on general-purpose CPUs or GPUs. However, at the same time, quantum circuits themselves offer multiple opportunities for parallelization, some of which could map better to other architecture -- architectures such as reconfigurable systems. In this early work, we created a quantum circuit simulator system called Q2Logic. Q2Logic is a coarse-grained reconfigurable architecture (CGRA) implemented as an overlay on Field-Programmable Gate Arrays (FPGAs), but specialized towards quantum simulations. We described how Q2Logic has been created and reveal implementation details, limitations, and opportunities. We end the study by empirically comparing the performance of Q2Logic (running on a Intel Agilex FPGA) against the state-of-the-art framework SVSim (running on a modern processor), showing improvements in three large circuits (#qbit=27), where Q2Logic can be up-to ~7x faster.
Dynamic Competency Self-Assessment for Autonomous Agents
Venue: arXiv cs.RO 2023
Year: 2023

As autonomous robots are deployed in increasingly complex environments, platform degradation, environmental uncertainties, and deviations from validated operation conditions can make it difficult for human partners to understand robot capabilities and limitations. The ability for a robot to self-assess its competency in dynamic and uncertain environments will be a crucial next step in successful human-robot teaming. This work presents and evaluates an Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm for autonomous agents to dynamically assess task confidence during execution. The algorithm uses a fast online statistical test of the agent's observations and its model predictions to decide when competency assessment is needed. We provide experimental results using ET-GOA to generate competency reports during a simulated delivery task and suggest future research directions for self-assessing agents.
Ada-Grouper: Accelerating Pipeline Parallelism in Preempted Network by Adaptive Group-Scheduling for Micro-Batches
Venue: arXiv cs.DC 2023
Year: 2023

Pipeline parallelism has been demonstrated to be a remarkable approach to improve throughput for training deep neural networks with billions of parameters over heterogeneous clusters. The 1F1B scheduling plan is a widely adopted strategy for memory and performance optimization, which interchanges the forward and backward stage computations of different micro-batches. On the other hand, a common issue in using the 1F1B scheduling is that stage computation is delayed due to the data transfer when network resources are preempted by other tasks, even with the minimum communication between stages. The exclusive access of these network resources cannot be guaranteed in cloud offerings. We present a general scheduling technique to accommodate pipeline parallelism to preempted network environments at the expense of a certain amount of memory pressure. The core concept is to extend 1F1B schedule scheme to kFkB, which groups k micro-batches, and alternately executes k forward and backward computations. We propose Ada-Grouper, an adaptive kFkB scheduler which regularly adjusts the number of group members k to maintain an optimal balance between communication and computation efficiency correspond to changes in a changing network environment under the memory limit. Experimental results demonstrate that our design maintain stable performance for pipeline parallelism, yielding a performance increase of up from 4% to 30%, compared with 1F1B in preempted network scenarios.
BO-Muse: A human expert and AI teaming framework for accelerated experimental design
Venue: arXiv cs.LG 2023
Year: 2023

In this paper we introduce BO-Muse, a new approach to human-AI teaming for the optimization of expensive black-box functions. Inspired by the intrinsic difficulty of extracting expert knowledge and distilling it back into AI models and by observations of human behavior in real-world experimental design, our algorithm lets the human expert take the lead in the experimental process. The human expert can use their domain expertise to its full potential, while the AI plays the role of a muse, injecting novelty and searching for areas of weakness to break the human out of over-exploitation induced by cognitive entrenchment. With mild assumptions, we show that our algorithm converges sub-linearly, at a rate faster than the AI or human alone. We validate our algorithm using synthetic data and with human experts performing real-world experiments.
Travel Demand Forecasting: A Fair AI Approach
Venue: arXiv cs.LG 2023
Year: 2023

Artificial Intelligence (AI) and machine learning have been increasingly adopted for travel demand forecasting. The AI-based travel demand forecasting models, though generate accurate predictions, may produce prediction biases and raise fairness issues. Using such biased models for decision-making may lead to transportation policies that exacerbate social inequalities. However, limited studies have been focused on addressing the fairness issues of these models. Therefore, in this study, we propose a novel methodology to develop fairness-aware, highly-accurate travel demand forecasting models. Particularly, the proposed methodology can enhance the fairness of AI models for multiple protected attributes (such as race and income) simultaneously. Specifically, we introduce a new fairness regularization term, which is explicitly designed to measure the correlation between prediction accuracy and multiple protected attributes, into the loss function of the travel demand forecasting model. We conduct two case studies to evaluate the performance of the proposed methodology using real-world ridesourcing-trip data in Chicago, IL and Austin, TX, respectively. Results highlight that our proposed methodology can effectively enhance fairness for multiple protected attributes while preserving prediction accuracy. Additionally, we have compared our methodology with three state-of-the-art methods that adopt the regularization term approach, and the results demonstrate that our approach significantly outperforms them in both preserving prediction accuracy and enhancing fairness. This study can provide transportation professionals with a new tool to achieve fair and accurate travel demand forecasting.
EigenMPC: An Eigenmanifold-Inspired Model-Predictive Control Framework for Exciting Efficient Oscillations in Mechanical Systems
Venue: arXiv cs.RO 2023
Year: 2023

This paper proposes a Nonlinear Model-Predictive Control (NMPC) method capable of finding and converging to energy-efficient regular oscillations, which require no control action to be sustained. The approach builds up on the recently developed Eigenmanifold theory, which defines the sets of line-shaped oscillations of a robot as an invariant two-dimensional submanifold of its state space. By defining the control problem as a nonlinear program (NLP), the controller is able to deal with constraints in the state and control variables and be energy-efficient not only in its final trajectory but also during the convergence phase. An initial implementation of this approach is proposed, analyzed, and tested in simulation.
Holistic IJTAG-based External and Internal Fault Monitoring in UAVs
Venue: arXiv cs.AR 2023
Year: 2023

Cyber-Physical Systems (CPSs), such as Unmanned Aerial Vehicles (UAVs), use System-on-Chip (SoC) based computing platforms to perform multiple complex tasks in safety-critical applications that require a highly dependable operation. Due to continuous technological manufacturing miniaturization SoCs face a wide spectrum of chip-level reliability issues such as aging, soft and hard errors during the operational lifetime of a UAV. In addition, external (off-chip) faults in the sensors, actuators, and motors are another cause of UAV failures. While existing works examine either on-chip faults (internal) or sensors/actuators faults (external) separately, this research proposes a UAV health monitoring infrastructure considering both external and internal faults holistically. The proposed method relies on the IEEE 1687 standard (IJTAG) and employs on-chip embedded instruments as health monitors to instantly access external and internal sensor data. Experimental results for functional simulation of a real-life case-study design demonstrate both types of fault detection by serving only three clock cycles and the localization process using 16 and 30 clock cycles for the case of single and double faults, respectively.
Developers Need Protection, Too: Perspectives and Research Challenges for Privacy in Social Coding Platforms
Venue: arXiv cs.SE 2023
Year: 2023

Social Coding Platforms (SCPs) like GitHub have become central to modern software engineering thanks to their collaborative and version-control features. Like in mainstream Online Social Networks (OSNs) such as Facebook, users of SCPs are subjected to privacy attacks and threats given the high amounts of personal and project-related data available in their profiles and software repositories. However, unlike in OSNs, the privacy concerns and practices of SCP users have not been extensively explored nor documented in the current literature. In this work, we present the preliminary results of an online survey (N=105) addressing developers' concerns and perceptions about privacy threats steaming from SCPs. Our results suggest that, although users express concern about social and organisational privacy threats, they often feel safe sharing personal and project-related information on these platforms. Moreover, attacks targeting the inference of sensitive attributes are considered more likely than those seeking to re-identify source-code contributors. Based on these findings, we propose a set of recommendations for future investigations addressing privacy and identity management in SCPs.
Usability of Privacy Controls in Top Health Websites
Venue: arXiv cs.CR 2023
Year: 2023

With the increasing awareness and concerns around privacy, many service providers offer their users various privacy controls. Through these controls, users gain greater authority over the collection, utilisation, and dissemination of their personal information by the services. However, these controls may be buried deep within menus or settings, making them difficult for a user to access. Additionally, the terminology used to describe privacy controls can sometimes be confusing or technical, further complicating the user's ability to understand and use them effectively. This is especially true for health websites, as users often share sensitive information about their health and well-being. While many privacy controls have been proposed to protect user data on these sites, existing research focuses on individual controls (e.g., privacy policies or cookie opt-outs) rather than providing a comprehensive overview of the privacy landscape. In addition, many studies concentrate on the technical aspects of privacy controls without considering the usability of these features from a user's perspective. This paper aims to fill the gaps in the existing work by analysing four privacy controls, namely privacy nudge, privacy notice, privacy policy, and privacy setting, and evaluating their usability on the top 100 most visited health websites. First, we define usability attributes for each privacy control in three website visit scenarios; the guest, registering, and log-in visits. These attributes include awareness, efficiency, comprehension, functionality, and choice. Then, we design a survey template based on these attributes and scenarios and collect data about privacy controls. Next, we analyse the availability and usability of each privacy control on health websites. Finally, we provide suggestions for improving the design of these privacy controls based on the data analysis results.
Spectrogram Inversion for Audio Source Separation via Consistency, Mixing, and Magnitude Constraints
Venue: arXiv cs.SD 2023
Year: 2023

Audio source separation is often achieved by estimating the magnitude spectrogram of each source, and then applying a phase recovery (or spectrogram inversion) algorithm to retrieve time-domain signals. Typically, spectrogram inversion is treated as an optimization problem involving one or several terms in order to promote estimates that comply with a consistency property, a mixing constraint, and/or a target magnitude objective. Nonetheless, it is still unclear which set of constraints and problem formulation is the most appropriate in practice. In this paper, we design a general framework for deriving spectrogram inversion algorithm, which is based on formulating optimization problems by combining these objectives either as soft penalties or hard constraints. We solve these by means of algorithms that perform alternating projections on the subsets corresponding to each objective/constraint. Our framework encompasses existing techniques from the literature as well as novel algorithms. We investigate the potential of these approaches for a speech enhancement task. In particular, one of our novel algorithms outperforms other approaches in a realistic setting where the magnitudes are estimated beforehand using a neural network.
Robust Detection Outcome: A Metric for Pathology Detection in Medical Images
Venue: arXiv cs.CV 2023
Year: 2023

Detection of pathologies is a fundamental task in medical imaging and the evaluation of algorithms that can perform this task automatically is crucial. However, current object detection metrics for natural images do not reflect the specific clinical requirements in pathology detection sufficiently. To tackle this problem, we propose Robust Detection Outcome (RoDeO); a novel metric for evaluating algorithms for pathology detection in medical images, especially in chest X-rays. RoDeO evaluates different errors directly and individually, and reflects clinical needs better than current metrics. Extensive evaluation on the ChestX-ray8 dataset shows the superiority of our metrics compared to existing ones. We released the code at https://github.com/FeliMe/RoDeO and published RoDeO as pip package (rodeometric).
Intelligent O-RAN Traffic Steering for URLLC Through Deep Reinforcement Learning
Venue: arXiv cs.NI 2023
Year: 2023

The goal of Next-Generation Networks is to improve upon the current networking paradigm, especially in providing higher data rates, near-real-time latencies, and near-perfect quality of service. However, existing radio access network (RAN) architectures lack sufficient flexibility and intelligence to meet those demands. Open RAN (O-RAN) is a promising paradigm for building a virtualized and intelligent RAN architecture. This paper presents a Machine Learning (ML)-based Traffic Steering (TS) scheme to predict network congestion and then proactively steer O-RAN traffic to avoid it and reduce the expected queuing delay. To achieve this, we propose an optimized setup focusing on safeguarding both latency and reliability to serve URLLC applications. The proposed solution consists of a two-tiered ML strategy based on Naive Bayes Classifier and deep Q-learning. Our solution is evaluated against traditional reactive TS approaches that are offered as xApps in O-RAN and shows an average of 15.81 percent decrease in queuing delay across all deployed SFCs.