OpenRaiser/Intern-Atlas huggingface dataset:
User Assignment and Resource Allocation for Hierarchical Federated Learning over Wireless Networks
Venue: arXiv cs.DC 2023
Year: 2023

The large population of wireless users is a key driver of data-crowdsourced Machine Learning (ML). However, data privacy remains a significant concern. Federated Learning (FL) encourages data sharing in ML without requiring data to leave users' devices but imposes heavy computation and communications overheads on mobile devices. Hierarchical FL (HFL) alleviates this problem by performing partial model aggregation at edge servers. HFL can effectively reduce energy consumption and latency through effective resource allocation and appropriate user assignment. Nevertheless, resource allocation in HFL involves optimizing multiple variables, and the objective function should consider both energy consumption and latency, making the development of resource allocation algorithms very complicated. Moreover, it is challenging to perform user assignment, which is a combinatorial optimization problem in a large search space. This article proposes a spectrum resource optimization algorithm (SROA) and a two-stage iterative algorithm (TSIA) for HFL. Given an arbitrary user assignment pattern, SROA optimizes CPU frequency, transmit power, and bandwidth to minimize system cost. TSIA aims to find a user assignment pattern that considerably reduces the total system cost. Experimental results demonstrate the superiority of the proposed HFL framework over existing studies in energy and latency reduction.
MVP: Meta Visual Prompt Tuning for Few-Shot Remote Sensing Image Scene Classification
Venue: arXiv cs.CV 2023
Year: 2023

Vision Transformer (ViT) models have recently emerged as powerful and versatile models for various visual tasks. Recently, a work called PMF has achieved promising results in few-shot image classification by utilizing pre-trained vision transformer models. However, PMF employs full fine-tuning for learning the downstream tasks, leading to significant overfitting and storage issues, especially in the remote sensing domain. In order to tackle these issues, we turn to the recently proposed parameter-efficient tuning methods, such as VPT, which updates only the newly added prompt parameters while keeping the pre-trained backbone frozen. Inspired by VPT, we propose the Meta Visual Prompt Tuning (MVP) method. Specifically, we integrate the VPT method into the meta-learning framework and tailor it to the remote sensing domain, resulting in an efficient framework for Few-Shot Remote Sensing Scene Classification (FS-RSSC). Furthermore, we introduce a novel data augmentation strategy based on patch embedding recombination to enhance the representation and diversity of scenes for classification purposes. Experiment results on the FS-RSSC benchmark demonstrate the superior performance of the proposed MVP over existing methods in various settings, such as various-way-various-shot, various-way-one-shot, and cross-domain adaptation.
Fairness for All: Investigating Harms to Within-Group Individuals in Producer Fairness Re-ranking Optimization -- A Reproducibility Study
Venue: arXiv cs.IR 2023
Year: 2023

Recommender systems are widely used to provide personalized recommendations to users. Recent research has shown that recommender systems may be subject to different types of biases, such as popularity bias, leading to an uneven distribution of recommendation exposure among producer groups. To mitigate this, producer-centered fairness re-ranking (PFR) approaches have been proposed to ensure equitable recommendation utility across groups. However, these approaches overlook the harm they may cause to within-group individuals associated with colder items, which are items with few or no interactions. This study reproduces previous PFR approaches and shows that they significantly harm colder items, leading to a fairness gap for these items in both advantaged and disadvantaged groups. Surprisingly, the unfair base recommendation models were providing greater exposure opportunities to these individual cold items, even though at the group level, they appeared to be unfair. To address this issue, the study proposes an amendment to the PFR approach that regulates the number of colder items recommended by the system. This modification achieves a balance between accuracy and producer fairness while optimizing the selection of colder items within each group, thereby preventing or reducing harm to within-group individuals and augmenting the novelty of all recommended items. The proposed method is able to register an increase in sub-group fairness (SGF) from 0.3104 to 0.3782, 0.6156, and 0.9442 while also improving group-level fairness (GF) (112% and 37% with respect to base models and traditional PFR). Moreover, the proposed method achieves these improvements with minimal or no reduction in accuracy (or even an increase sometimes). We evaluate the proposed method on various recommendation datasets and demonstrate promising results independent of the underlying model or datasets.
An Auto-Parallelizer for Distributed Computing in Haskell
Venue: arXiv cs.DC 2023
Year: 2023

One of the main challenges in distributed computing is building interfaces and APIs that allow programmers with limited background in distributed systems to write scalable, performant, and fault-tolerant applications on large clusters. In this demonstration, we designed and implemented a Haskell auto-parallelizer with a simple yet powerful interface by taking advantage of the default purity of Haskell functions. Finally, we benchmarked our implementation on a set of examples to illustrate the potential for future work in this direction.
Privacy-Preserving Polynomial Computing Over Distributed Data
Venue: arXiv cs.CR 2023
Year: 2023

In this letter, we delve into a scenario where a user aims to compute polynomial functions using their own data as well as data obtained from distributed sources. To accomplish this, the user enlists the assistance of $N$ distributed workers, thereby defining a problem we refer to as privacy-preserving polynomial computing over distributed data. To address this challenge, we propose an approach founded upon Lagrange encoding. Our method not only possesses the ability to withstand the presence of stragglers and byzantine workers but also ensures the preservation of security. Specifically, even if a coalition of $X$ workers collude, they are unable to acquire any knowledge pertaining to the data originating from the distributed sources or the user.
How NFT Collectors Experience Online NFT Communities: A Case Study of Bored Ape
Venue: arXiv cs.HC 2023
Year: 2023

Non-fungible tokens (NFTs) are unique cryptographic assets representing the ownership of digital media. NFTs have soared in popularity and trading prices. However, there exists a large gap in the literature regarding NFTs, especially regarding the stakeholders and online communities that have formed around NFT projects. Bored Ape Yacht Club (BAYC) is one of the most influential NFT projects. Through an observational study of online BAYC communities across social media platforms and semi-structured interviews with four participants who owned BAYC NFTs, we explored the experiences of NFT collectors within the online NFT community. Positive community experiences, i.e., personal expression and identity, mutual support among BAYC holders, and exclusive access to online and offline events, were expressed. Encountered challenges included scams and "cash grab" NFT projects as well as trolling. The results of this study point towards the welcoming, positive nature of the NFT community, which is a possible causation factor of the initial rise in popularity of NFTs. Demotivators, on the other hand, countered the established trustworthiness of NFT technology among its consumers.
How People Perceive The Dynamic Zero-COVID Policy: A Retrospective Analysis From The Perspective of Appraisal Theory
Venue: arXiv cs.CY 2023
Year: 2023

The Dynamic Zero-COVID Policy in China spanned three years and diverse emotional responses have been observed at different times. In this paper, we retrospectively analyzed public sentiments and perceptions of the policy, especially regarding how they evolved over time, and how they related to people's lived experiences. Through sentiment analysis of 2,358 collected Weibo posts, we identified four representative points, i.e., policy initialization, sharp sentiment change, lowest sentiment score, and policy termination, for an in-depth discourse analysis through the lens of appraisal theory. In the end, we reflected on the evolving public sentiments toward the Dynamic Zero-COVID Policy and proposed implications for effective epidemic prevention and control measures for future crises.
Enhancing Knee Osteoarthritis severity level classification using diffusion augmented images
Venue: arXiv cs.CV 2023
Year: 2023

This research paper explores the classification of knee osteoarthritis (OA) severity levels using advanced computer vision models and augmentation techniques. The study investigates the effectiveness of data preprocessing, including Contrast-Limited Adaptive Histogram Equalization (CLAHE), and data augmentation using diffusion models. Three experiments were conducted: training models on the original dataset, training models on the preprocessed dataset, and training models on the augmented dataset. The results show that data preprocessing and augmentation significantly improve the accuracy of the models. The EfficientNetB3 model achieved the highest accuracy of 84\% on the augmented dataset. Additionally, attention visualization techniques, such as Grad-CAM, are utilized to provide detailed attention maps, enhancing the understanding and trustworthiness of the models. These findings highlight the potential of combining advanced models with augmented data and attention visualization for accurate knee OA severity classification.
Efficient Belief Road Map for Planning Under Uncertainty
Venue: arXiv cs.RO 2023
Year: 2023

Robotic systems, particularly in demanding environments like narrow corridors or disaster zones, often grapple with imperfect state estimation. Addressing this challenge requires a trajectory plan that not only navigates these restrictive spaces but also manages the inherent uncertainty of the system. We present a novel approach for graph-based belief space planning via the use of an efficient covariance control algorithm. By adaptively steering state statistics via output state feedback, we efficiently craft a belief roadmap characterized by nodes with controlled uncertainty and edges representing collision-free mean trajectories. The roadmap's structured design then paves the way for precise path searches that balance control costs and uncertainty considerations. Our numerical experiments affirm the efficacy and advantage of our method in different motion planning tasks. Our open-source implementation can be found at https://github.com/hzyu17/VIMP/tree/BRM.
Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents
Venue: arXiv cs.AI 2023
Year: 2023

Embodied agents, in the form of virtual agents or social robots, are rapidly becoming more widespread. In human-human interactions, humans use nonverbal behaviours to convey their attitudes, feelings, and intentions. Therefore, this capability is also required for embodied agents in order to enhance the quality and effectiveness of their interactions with humans. In this paper, we propose a novel framework that can generate sequences of joint angles from the speech text and speech audio utterances. Based on a conditional Generative Adversarial Network (GAN), our proposed neural network model learns the relationships between the co-speech gestures and both semantic and acoustic features from the speech input. In order to train our neural network model, we employ a public dataset containing co-speech gestures with corresponding speech audio utterances, which were captured from a single male native English speaker. The results from both objective and subjective evaluations demonstrate the efficacy of our gesture-generation framework for Robots and Embodied Agents.
Language models are susceptible to incorrect patient self-diagnosis in medical applications
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) are becoming increasingly relevant as a potential tool for healthcare, aiding communication between clinicians, researchers, and patients. However, traditional evaluations of LLMs on medical exam questions do not reflect the complexity of real patient-doctor interactions. An example of this complexity is the introduction of patient self-diagnosis, where a patient attempts to diagnose their own medical conditions from various sources. While the patient sometimes arrives at an accurate conclusion, they more often are led toward misdiagnosis due to the patient's over-emphasis on bias validating information. In this work we present a variety of LLMs with multiple-choice questions from United States medical board exams which are modified to include self-diagnostic reports from patients. Our findings highlight that when a patient proposes incorrect bias-validating information, the diagnostic accuracy of LLMs drop dramatically, revealing a high susceptibility to errors in self-diagnosis.
Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
Venue: arXiv cs.RO 2023
Year: 2023

Safe Reinforcement Learning (RL) aims to find a policy that achieves high rewards while satisfying cost constraints. When learning from scratch, safe RL agents tend to be overly conservative, which impedes exploration and restrains the overall performance. In many realistic tasks, e.g. autonomous driving, large-scale expert demonstration data are available. We argue that extracting expert policy from offline data to guide online exploration is a promising solution to mitigate the conserveness issue. Large-capacity models, e.g. decision transformers (DT), have been proven to be competent in offline policy learning. However, data collected in real-world scenarios rarely contain dangerous cases (e.g., collisions), which makes it prohibitive for the policies to learn safety concepts. Besides, these bulk policy networks cannot meet the computation speed requirements at inference time on real-world tasks such as autonomous driving. To this end, we propose Guided Online Distillation (GOLD), an offline-to-online safe RL framework. GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms. Experiments in both benchmark safe RL tasks and real-world driving tasks based on the Waymo Open Motion Dataset (WOMD) demonstrate that GOLD can successfully distill lightweight policies and solve decision-making problems in challenging safety-critical scenarios.
Scalable Label-efficient Footpath Network Generation Using Remote Sensing Data and Self-supervised Learning
Venue: arXiv cs.CV 2023
Year: 2023

Footpath mapping, modeling, and analysis can provide important geospatial insights to many fields of study, including transport, health, environment and urban planning. The availability of robust Geographic Information System (GIS) layers can benefit the management of infrastructure inventories, especially at local government level with urban planners responsible for the deployment and maintenance of such infrastructure. However, many cities still lack real-time information on the location, connectivity, and width of footpaths, and/or employ costly and manual survey means to gather this information. This work designs and implements an automatic pipeline for generating footpath networks based on remote sensing images using machine learning models. The annotation of segmentation tasks, especially labeling remote sensing images with specialized requirements, is very expensive, so we aim to introduce a pipeline requiring less labeled data. Considering supervised methods require large amounts of training data, we use a self-supervised method for feature representation learning to reduce annotation requirements. Then the pre-trained model is used as the encoder of the U-Net for footpath segmentation. Based on the generated masks, the footpath polygons are extracted and converted to footpath networks which can be loaded and visualized by geographic information systems conveniently. Validation results indicate considerable consistency when compared to manually collected GIS layers. The footpath network generation pipeline proposed in this work is low-cost and extensible, and it can be applied where remote sensing images are available. Github: https://github.com/WennyXY/FootpathSeg.
CaT: Balanced Continual Graph Learning with Graph Condensation
Venue: arXiv cs.LG 2023
Year: 2023

Continual graph learning (CGL) is purposed to continuously update a graph model with graph data being fed in a streaming manner. Since the model easily forgets previously learned knowledge when training with new-coming data, the catastrophic forgetting problem has been the major focus in CGL. Recent replay-based methods intend to solve this problem by updating the model using both (1) the entire new-coming data and (2) a sampling-based memory bank that stores replayed graphs to approximate the distribution of historical data. After updating the model, a new replayed graph sampled from the incoming graph will be added to the existing memory bank. Despite these methods are intuitive and effective for the CGL, two issues are identified in this paper. Firstly, most sampling-based methods struggle to fully capture the historical distribution when the storage budget is tight. Secondly, a significant data imbalance exists in terms of the scales of the complex new-coming graph data and the lightweight memory bank, resulting in unbalanced training. To solve these issues, a Condense and Train (CaT) framework is proposed in this paper. Prior to each model update, the new-coming graph is condensed to a small yet informative synthesised replayed graph, which is then stored in a Condensed Graph Memory with historical replay graphs. In the continual learning phase, a Training in Memory scheme is used to update the model directly with the Condensed Graph Memory rather than the whole new-coming graph, which alleviates the data imbalance problem. Extensive experiments conducted on four benchmark datasets successfully demonstrate superior performances of the proposed CaT framework in terms of effectiveness and efficiency. The code has been released on https://github.com/superallen13/CaT-CGL.
Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignment
Venue: arXiv cs.SD 2023
Year: 2023

This paper presents a novel task, zero-shot voice conversion based on face images (zero-shot FaceVC), which aims at converting the voice characteristics of an utterance from any source speaker to a newly coming target speaker, solely relying on a single face image of the target speaker. To address this task, we propose a face-voice memory-based zero-shot FaceVC method. This method leverages a memory-based face-voice alignment module, in which slots act as the bridge to align these two modalities, allowing for the capture of voice characteristics from face images. A mixed supervision strategy is also introduced to mitigate the long-standing issue of the inconsistency between training and inference phases for voice conversion tasks. To obtain speaker-independent content-related representations, we transfer the knowledge from a pretrained zero-shot voice conversion model to our zero-shot FaceVC model. Considering the differences between FaceVC and traditional voice conversion tasks, systematic subjective and objective metrics are designed to thoroughly evaluate the homogeneity, diversity and consistency of voice characteristics controlled by face images. Through extensive experiments, we demonstrate the superiority of our proposed method on the zero-shot FaceVC task. Samples are presented on our demo website.
Stochastic Performance Analysis of Phase Decomposition in Hyperledger Fabric
Venue: arXiv cs.DC 2023
Year: 2023

Hyperledger Fabric is one of the most popular permissioned blockchain platforms. Although many existing works on the overall system performance of Hyperledger Fabric are available, a decomposition of each phase in Hyperledger Fabric remains to be explored. Admittedly, the overall system performance of Hyperledger Fabric might provide an end-user with satisfied performance information when invoking a transaction; however, it is far from informative when deploying a distributed system with specific performance goals, except for understanding each phase in Hyperledger Fabric. In this paper, we develop a measurement framework to characterize each phase's transaction and block data in Hyperledger Fabric based on the Fabric SDK Nodejs, where we thoroughly analyze and open source the implementation details of the measurement framework. We evaluate the performance of Hyperledger Fabric and have some interesting observations; 1. The number of CPU cores has a linear impact on the throughput of an endorsing peer. 2. The Raft-based ordering service shows good scalability with the number of ordering service nodes. 3. The communication latencies between the client and service in Hyperledger Fabric are significant. We then identify each phase's dominant latency in Hyperledger Fabric via primitive operation analysis and propose a stochastic computation model for performance analysis. We also use the alpha-beta communication model to analyze the corresponding communication latency. Finally, we validate the accuracy of the performance model on both local and cloud clusters. The experiment results and the performance model help guide the deployment of the Hyperledger Fabric service.
Training Students' Abstraction Skills Around a CAFÉ 2.0
Venue: arXiv cs.CY 2023
Year: 2023

Shaping first year students' mind to help them master abstraction skills is as crucial as it is challenging. Although abstraction is a key competence in problem-solving (in particular in STEM disciplines), students are often found to rush that process because they find it hard and do not get any direct outcome out of it. They prefer to invest their efforts directly in a concrete ground, rather than using abstraction to create a solution. To overcome that situation, in the context of our CS1 course, we implemented a tool called CAFÉ 2.0. It allows students to actively and regularly practice (thanks to a longitudinal activity) their abstraction skills through a graphical programming methodology. Moreover, further than reviewing students' final implementation, CAFÉ 2.0 produces a personalized feedback on how students modeled their solution, and on how consistent it is with their final code. This paper describes CAFÉ 2.0 in a general setting and also provides a concrete example in our CS1 course context. This paper also assesses students' interaction with CAFÉ 2.0 through perception and participation data. Finally, we explain how CAFÉ 2.0 could extended in another context than a CS1 course.