OpenRaiser/Intern-Atlas huggingface dataset:
Exploring a Gradient-based Explainable AI Technique for Time-Series Data: A Case Study of Assessing Stroke Rehabilitation Exercises
Venue: arXiv cs.LG 2023
Year: 2023

Explainable artificial intelligence (AI) techniques are increasingly being explored to provide insights into why AI and machine learning (ML) models provide a certain outcome in various applications. However, there has been limited exploration of explainable AI techniques on time-series data, especially in the healthcare context. In this paper, we describe a threshold-based method that utilizes a weakly supervised model and a gradient-based explainable AI technique (i.e. saliency map) and explore its feasibility to identify salient frames of time-series data. Using the dataset from 15 post-stroke survivors performing three upper-limb exercises and labels on whether a compensatory motion is observed or not, we implemented a feed-forward neural network model and utilized gradients of each input on model outcomes to identify salient frames that involve compensatory motions. According to the evaluation using frame-level annotations, our approach achieved a recall of 0.96 and an F2-score of 0.91. Our results demonstrated the potential of a gradient-based explainable AI technique (e.g. saliency map) for time-series data, such as highlighting the frames of a video that therapists should focus on reviewing and reducing the efforts on frame-level labeling for model training.
Integrating Holistic and Local Information to Estimate Emotional Reaction Intensity
Venue: arXiv cs.CV 2023
Year: 2023

Video-based Emotional Reaction Intensity (ERI) estimation measures the intensity of subjects' reactions to stimuli along several emotional dimensions from videos of the subject as they view the stimuli. We propose a multi-modal architecture for video-based ERI combining video and audio information. Video input is encoded spatially first, frame-by-frame, combining features encoding holistic aspects of the subjects' facial expressions and features encoding spatially localized aspects of their expressions. Input is then combined across time: from frame-to-frame using gated recurrent units (GRUs), then globally by a transformer. We handle variable video length with a regression token that accumulates information from all frames into a fixed-dimensional vector independent of video length. Audio information is handled similarly: spectral information extracted within each frame is integrated across time by a cascade of GRUs and a transformer with regression token. The video and audio regression tokens' outputs are merged by concatenation, then input to a final fully connected layer producing intensity estimates. Our architecture achieved excellent performance on the Hume-Reaction dataset in the ERI Esimation Challenge of the Fifth Competition on Affective Behavior Analysis in-the-Wild (ABAW5). The Pearson Correlation Coefficients between estimated and subject self-reported scores, averaged across all emotions, were 0.455 on the validation dataset and 0.4547 on the test dataset, well above the baselines. The transformer's self-attention mechanism enables our architecture to focus on the most critical video frames regardless of length. Ablation experiments establish the advantages of combining holistic/local features and of multi-modal integration. Code available at https://github.com/HKUST-NISL/ABAW5.
Sparse Stream Semantic Registers: A Lightweight ISA Extension Accelerating General Sparse Linear Algebra
Venue: arXiv cs.AR 2023
Year: 2023

Sparse linear algebra is crucial in many application domains, but challenging to handle efficiently in both software and hardware, with one- and two-sided operand sparsity handled with distinct approaches. In this work, we enhance an existing memory-streaming RISC-V ISA extension to accelerate both one- and two-sided operand sparsity on widespread sparse tensor formats like compressed sparse row (CSR) and compressed sparse fiber (CSF) by accelerating the underlying operations of streaming indirection, intersection, and union. Our extensions enable single-core speedups over an optimized RISC-V baseline of up to 7.0x, 7.7x, and 9.8x on sparse-dense multiply, sparse-sparse multiply, and sparse-sparse addition, respectively, and peak FPU utilizations of up to 80% on sparse-dense problems. On an eight-core cluster, sparse-dense and sparse-sparse matrix-vector multiply using real-world matrices are up to 4.9x and 5.9x faster and up to 2.9x and 3.0x more energy efficient. We explore further applications for our extensions, such as stencil codes and graph pattern matching. Compared to recent CPU, GPU, and accelerator approaches, our extensions enable higher flexibility on data representation, degree of sparsity, and dataflow at a minimal hardware footprint, adding only 1.8% in area to a compute cluster. A cluster with our extensions running CSR matrix-vector multiplication achieves 9.9x and 1.7x higher peak floating-point utilizations than recent highly optimized sparse data structures and libraries for CPU and GPU, respectively, even when accounting for off-chip main memory (HBM) and on-chip interconnect latency and bandwidth effects.
An Algorithm For Adversary Aware Decentralized Networked MARL
Venue: arXiv cs.LG 2023
Year: 2023

Decentralized multi-agent reinforcement learning (MARL) algorithms have become popular in the literature since it allows heterogeneous agents to have their own reward functions as opposed to canonical multi-agent Markov Decision Process (MDP) settings which assume common reward functions over all agents. In this work, we follow the existing work on collaborative MARL where agents in a connected time varying network can exchange information among each other in order to reach a consensus. We introduce vulnerabilities in the consensus updates of existing MARL algorithms where agents can deviate from their usual consensus update, who we term as adversarial agents. We then proceed to provide an algorithm that allows non-adversarial agents to reach a consensus in the presence of adversaries under a constrained setting.
Investigating the Software Engineering Roadmap for Smart City Infrastructure Development: Goals and Challenges
Venue: arXiv cs.SE 2023
Year: 2023

In today's world, many cities are embracing cutting-edge technology and transforming into "smart cities". These emerging innovations are revolutionizing the standard of living for people, and as a result, smart city infrastructure development has become a major focus for city planners and policymakers worldwide. The goal is to create more livable, sustainable, and efficient urban environments, and software engineering plays a crucial role in achieving this. In this article, we will delve into what makes a city "smart" and what it means for the future. We will explore the software engineering roadmap for smart city infrastructure development, highlighting the goals and challenges that come with this innovative approach to urban planning. Our aim is to provide valuable insights into the importance of software engineering in achieving successful smart city infrastructure development. As cities continue to grow and evolve, it is essential to adopt new technologies that can help us build smarter, more sustainable communities. Smart city initiatives are paving the way for a brighter future, and software engineering is at the forefront of this movement. By understanding the software engineering roadmap for smart city infrastructure development, we can work towards creating more livable, efficient, and sustainable urban environments for generations to come.
Description Complexity of Regular Distributions
Venue: arXiv cs.GT 2023
Year: 2023

Myerson's regularity condition of a distribution is a standard assumption in economics. In this paper, we study the complexity of describing a regular distribution within a small statistical distance. Our main result is that $\tildeΘ{(ε^{-0.5})}$ bits are necessary and sufficient to describe a regular distribution with support $[0,1]$ within $ε$ Levy distance. We prove this by showing that we can learn the regular distribution approximately with $\tilde{O}(ε^{-0.5})$ queries to the cumulative density function. As a corollary, we show that the pricing query complexity to learn the class of regular distribution with support $[0,1]$ within $ε$ Levy distance is $\tildeΘ{(ε^{-2.5})}$. To learn the mixture of two regular distributions, $\tildeΘ(ε^{-3})$ pricing queries are required.
"Alexa doesn't have that many feelings": Children's understanding of AI through interactions with smart speakers in their homes
Venue: arXiv cs.HC 2023
Year: 2023

As voice-based Conversational Assistants (CAs), including Alexa, Siri, Google Home, have become commonly embedded in households, many children now routinely interact with Artificial Intelligence (AI) systems. It is important to research children's experiences with consumer devices which use AI techniques because these shape their understanding of AI and its capabilities. We conducted a mixed-methods study (questionnaires and interviews) with primary-school children aged 6-11 in Scotland to establish children's understanding of how voice-based CAs work, how they perceive their cognitive abilities, agency and other human-like qualities, their awareness and trust of privacy aspects when using CAs and what they perceive as appropriate verbal interactions with CAs. Most children overestimated the CAs' intelligence and were uncertain about the systems' feelings or agency. They also lacked accurate understanding of data privacy and security aspects, and believed it was wrong to be rude to conversational assistants. Exploring children's current understanding of AI-supported technology has educational implications; such findings will enable educators to develop appropriate materials to address the pressing need for AI literacy.
Collaborative Chinese Text Recognition with Personalized Federated Learning
Venue: arXiv cs.CV 2023
Year: 2023

In Chinese text recognition, to compensate for the insufficient local data and improve the performance of local few-shot character recognition, it is often necessary for one organization to collect a large amount of data from similar organizations. However, due to the natural presence of private information in text data, such as addresses and phone numbers, different organizations are unwilling to share private data. Therefore, it becomes increasingly important to design a privacy-preserving collaborative training framework for the Chinese text recognition task. In this paper, we introduce personalized federated learning (pFL) into the Chinese text recognition task and propose the pFedCR algorithm, which significantly improves the model performance of each client (organization) without sharing private data. Specifically, pFedCR comprises two stages: multiple rounds of global model training stage and the the local personalization stage. During stage 1, an attention mechanism is incorporated into the CRNN model to adapt to various client data distributions. Leveraging inherent character data characteristics, a balanced dataset is created on the server to mitigate character imbalance. In the personalization phase, the global model is fine-tuned for one epoch to create a local model. Parameter averaging between local and global models combines personalized and global feature extraction capabilities. Finally, we fine-tune only the attention layers to enhance its focus on local personalized features. The experimental results on three real-world industrial scenario datasets show that the pFedCR algorithm can improve the performance of local personalized models by about 20\% while also improving their generalization performance on other client data domains. Compared to other state-of-the-art personalized federated learning methods, pFedCR improves performance by 6\% $\sim$ 8\%.
How is the speed of code review affected by activity, usage and code quality?
Venue: arXiv cs.SE 2023
Year: 2023

This paper investigates how the speed of code review is affected by the code quality, activity and usage in the context of MediaWiki extensions. The median time to merge is compared against several other variables which are collected using a variety of manual methods and APIs. The results are graphed where possible and statistical analysis is used to determine the significance of the results. The paper finds that the number of reviewers voting on code and whether the extension has a steward affects the median time to merge. Finally, conclusions are drawn and further research topics are recommended.
Modified Ring-Oscillator Physical Unclonable Function (RO-PUF) based PRBS Generation as a Device Signature in Distributed Brain Implants
Venue: arXiv cs.NI 2023
Year: 2023

In this paper, we propose and evaluate a method of generating low-cost device signatures for distributed wireless brain implants, using a Pseudo-Random Binary Sequence (PRBS) Generator that utilizes a modified Ring-Oscillator-based Physical Unclonable Function (RO-PUF). The modified RO-PUF's output is used as a seed for the PRBS generator, which creates a multi-bit output that can be mapped to a time-slot when the implant is allowed to communicate with the external world using duty-cycled time-division multiplexing. A 9-bit PRBS generator is shown in hardware (with a TSMC 65 nm test chip implementation) that demonstrates < 100 nW Power consumption in measurement (72% lower power and 78% lower area than a traditional 9-bit RO-PUF implementation), which supports 26 implants with the probability of time-slot collision being < 50%. This potentially creates a pathway for low-cost device signature generation for highly resource-constrained scenarios such as wireless, distributed neural implants.
Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization
Venue: arXiv cs.LG 2023
Year: 2023

The nuclear fuel loading pattern optimization problem belongs to the class of large-scale combinatorial optimization. It is also characterized by multiple objectives and constraints, which makes it impossible to solve explicitly. Stochastic optimization methodologies including Genetic Algorithms and Simulated Annealing are used by different nuclear utilities and vendors, but hand-designed solutions continue to be the prevalent method in the industry. To improve the state-of-the-art, Deep Reinforcement Learning (RL), in particular, Proximal Policy Optimization is leveraged. This work presents a first-of-a-kind approach to utilize deep RL to solve the loading pattern problem and could be leveraged for any engineering design optimization. This paper is also to our knowledge the first to propose a study of the behavior of several hyper-parameters that influence the RL algorithm. The algorithm is highly dependent on multiple factors such as the shape of the objective function derived for the core design that behaves as a fudge factor that affects the stability of the learning. But also, an exploration/exploitation trade-off that manifests through different parameters such as the number of loading patterns seen by the agents per episode, the number of samples collected before a policy update nsteps, and an entropy factor ent_coef that increases the randomness of the policy during training. We found that RL must be applied similarly to a Gaussian Process in which the acquisition function is replaced by a parametrized policy. Then, once an initial set of hyper-parameters is found, reducing nsteps and ent_coef until no more learning is observed will result in the highest sample efficiency robustly and stably. This resulted in an economic benefit of 535,000- 642,000 $/year/plant.
Change Detection Methods for Remote Sensing in the Last Decade: A Comprehensive Review
Venue: arXiv cs.CV 2023
Year: 2023

Change detection is an essential and widely utilized task in remote sensing that aims to detect and analyze changes occurring in the same geographical area over time, which has broad applications in urban development, agricultural surveys, and land cover monitoring. Detecting changes in remote sensing images is a complex challenge due to various factors, including variations in image quality, noise, registration errors, illumination changes, complex landscapes, and spatial heterogeneity. In recent years, deep learning has emerged as a powerful tool for feature extraction and addressing these challenges. Its versatility has resulted in its widespread adoption for numerous image-processing tasks. This paper presents a comprehensive survey of significant advancements in change detection for remote sensing images over the past decade. We first introduce some preliminary knowledge for the change detection task, such as problem definition, datasets, evaluation metrics, and transformer basics, as well as provide a detailed taxonomy of existing algorithms from three different perspectives: algorithm granularity, supervision modes, and learning frameworks in the methodology section. This survey enables readers to gain systematic knowledge of change detection tasks from various angles. We then summarize the state-of-the-art performance on several dominant change detection datasets, providing insights into the strengths and limitations of existing algorithms. Based on our survey, some future research directions for change detection in remote sensing are well identified. This survey paper will shed some light on the community and inspire further research efforts in the change detection task.
Achieving Diversity in Counterfactual Explanations: a Review and Discussion
Venue: arXiv cs.AI 2023
Year: 2023

In the field of Explainable Artificial Intelligence (XAI), counterfactual examples explain to a user the predictions of a trained decision model by indicating the modifications to be made to the instance so as to change its associated prediction. These counterfactual examples are generally defined as solutions to an optimization problem whose cost function combines several criteria that quantify desiderata for a good explanation meeting user needs. A large variety of such appropriate properties can be considered, as the user needs are generally unknown and differ from one user to another; their selection and formalization is difficult. To circumvent this issue, several approaches propose to generate, rather than a single one, a set of diverse counterfactual examples to explain a prediction. This paper proposes a review of the numerous, sometimes conflicting, definitions that have been proposed for this notion of diversity. It discusses their underlying principles as well as the hypotheses on the user needs they rely on and proposes to categorize them along several dimensions (explicit vs implicit, universe in which they are defined, level at which they apply), leading to the identification of further research challenges on this topic.
MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networks
Venue: arXiv cs.DC 2023
Year: 2023

Driven by the wide adoption of deep neural networks (DNNs) across different application domains, multi-tenancy execution, where multiple DNNs are deployed simultaneously on the same hardware, has been proposed to satisfy the latency requirements of different applications while improving the overall system utilization. However, multi-tenancy execution could lead to undesired system-level resource contention, causing quality-of-service (QoS) degradation for latency-critical applications. To address this challenge, we propose MoCA, an adaptive multi-tenancy system for DNN accelerators. Unlike existing solutions that focus on compute resource partition, MoCA dynamically manages shared memory resources of co-located applications to meet their QoS targets. Specifically, MoCA leverages the regularities in both DNN operators and accelerators to dynamically modulate memory access rates based on their latency targets and user-defined priorities so that co-located applications get the resources they demand without significantly starving their co-runners. We demonstrate that MoCA improves the satisfaction rate of the service level agreement (SLA) up to 3.9x (1.8x average), system throughput by 2.3x (1.7x average), and fairness by 1.3x (1.2x average), compared to prior work.
Optical Aberration Correction in Postprocessing using Imaging Simulation
Venue: arXiv cs.CV 2023
Year: 2023

As the popularity of mobile photography continues to grow, considerable effort is being invested in the reconstruction of degraded images. Due to the spatial variation in optical aberrations, which cannot be avoided during the lens design process, recent commercial cameras have shifted some of these correction tasks from optical design to postprocessing systems. However, without engaging with the optical parameters, these systems only achieve limited correction for aberrations.In this work, we propose a practical method for recovering the degradation caused by optical aberrations. Specifically, we establish an imaging simulation system based on our proposed optical point spread function model. Given the optical parameters of the camera, it generates the imaging results of these specific devices. To perform the restoration, we design a spatial-adaptive network model on synthetic data pairs generated by the imaging simulation system, eliminating the overhead of capturing training data by a large amount of shooting and registration. Moreover, we comprehensively evaluate the proposed method in simulations and experimentally with a customized digital-single-lens-reflex (DSLR) camera lens and HUAWEI HONOR 20, respectively. The experiments demonstrate that our solution successfully removes spatially variant blur and color dispersion. When compared with the state-of-the-art deblur methods, the proposed approach achieves better results with a lower computational overhead. Moreover, the reconstruction technique does not introduce artificial texture and is convenient to transfer to current commercial cameras. Project Page: \url{https://github.com/TanGeeGo/ImagingSimulation}.
Utility-Oriented Wireless Communications for 6G Networks: Semantic Information Transfer for IRS aided Vehicular Metaverse
Venue: arXiv cs.SI 2023
Year: 2023

This paper introduces the novel utility-oriented communications (UOC) concept and identifies its importance for 6G wireless technology. UOC encompasses existing communication paradigms and includes emerging human-centric and task-oriented communications concepts. The authors investigate semantic communications and semantic information transfer for vehicular metaverse as a case study of UOC. Consider the Internet of Vehicles (IoV) users access real-time virtual world updates from the base station (BS) wirelessly using semantic communication, and an intelligent reflecting surface (IRS) is deployed to impair co-channel interference. This paper formulates an optimization problem where a novel utility expression for semantic communications is incorporated. The proposed system model jointly considers latency and power in wireless communication and the utility of semantic communication. The proposed alternative optimization algorithm balances system efficiency and economics and outperforms existing optimization algorithms under the same channel conditions.