OpenRaiser/Intern-Atlas huggingface dataset:
Boundary Unlearning
Venue: arXiv cs.CV 2023
Year: 2023

The practical needs of the ``right to be forgotten'' and poisoned data removal call for efficient \textit{machine unlearning} techniques, which enable machine learning models to unlearn, or to forget a fraction of training data and its lineage. Recent studies on machine unlearning for deep neural networks (DNNs) attempt to destroy the influence of the forgetting data by scrubbing the model parameters. However, it is prohibitively expensive due to the large dimension of the parameter space. In this paper, we refocus our attention from the parameter space to the decision space of the DNN model, and propose Boundary Unlearning, a rapid yet effective way to unlearn an entire class from a trained DNN model. The key idea is to shift the decision boundary of the original DNN model to imitate the decision behavior of the model retrained from scratch. We develop two novel boundary shift methods, namely Boundary Shrink and Boundary Expanding, both of which can rapidly achieve the utility and privacy guarantees. We extensively evaluate Boundary Unlearning on CIFAR-10 and Vggface2 datasets, and the results show that Boundary Unlearning can effectively forget the forgetting class on image classification and face recognition tasks, with an expected speed-up of $17\times$ and $19\times$, respectively, compared with retraining from the scratch.
How Interactions Influence Users' Security Perception of Virtual Reality Authentication?
Venue: arXiv cs.CR 2023
Year: 2023

Users readily embrace the rapid advancements in virtual reality (VR) technology within various everyday contexts, such as gaming, social interactions, shopping, and commerce. In order to facilitate transactions and payments, VR systems require access to sensitive user data and assets, which consequently necessitates user authentication. However, there exists a limited understanding regarding how users' unique experiences in VR contribute to their perception of security. In our study, we adopt a research approach known as ``technology probe'' to investigate this question. Specifically, we have designed probes that explore the authentication process in VR, aiming to elicit responses from participants from multiple perspectives. These probes were seamlessly integrated into the routine payment system of a VR game, thereby establishing an organic study environment. Through qualitative analysis, we uncover the interplay between participants' interaction experiences and their security perception. Remarkably, despite encountering unique challenges in usability during VR interactions, our participants found the intuitive virtualized authentication process beneficial and thoroughly enjoyed the immersive nature of VR. Furthermore, we observe how these interaction experiences influence participants' ability to transfer their pre-existing understanding of authentication into VR, resulting in a discrepancy in perceived security. Moreover, we identify users' conflicting expectations, encompassing their desire for an enjoyable VR experience alongside the assurance of secure VR authentication. Building upon our findings, we propose recommendations aimed at addressing these expectations and alleviating potential conflicts.
Uniform Risk Bounds for Learning with Dependent Data Sequences
Venue: arXiv cs.LG 2023
Year: 2023

This paper extends standard results from learning theory with independent data to sequences of dependent data. Contrary to most of the literature, we do not rely on mixing arguments or sequential measures of complexity and derive uniform risk bounds with classical proof patterns and capacity measures. In particular, we show that the standard classification risk bounds based on the VC-dimension hold in the exact same form for dependent data, and further provide Rademacher complexity-based bounds, that remain unchanged compared to the standard results for the identically and independently distributed case. Finally, we show how to apply these results in the context of scenario-based optimization in order to compute the sample complexity of random programs with dependent constraints.
A Survey on Class Imbalance in Federated Learning
Venue: arXiv cs.LG 2023
Year: 2023

Federated learning, which allows multiple client devices in a network to jointly train a machine learning model without direct exposure of clients' data, is an emerging distributed learning technique due to its nature of privacy preservation. However, it has been found that models trained with federated learning usually have worse performance than their counterparts trained in the standard centralized learning mode, especially when the training data is imbalanced. In the context of federated learning, data imbalance may occur either locally one one client device, or globally across many devices. The complexity of different types of data imbalance has posed challenges to the development of federated learning technique, especially considering the need of relieving data imbalance issue and preserving data privacy at the same time. Therefore, in the literature, many attempts have been made to handle class imbalance in federated learning. In this paper, we present a detailed review of recent advancements along this line. We first introduce various types of class imbalance in federated learning, after which we review existing methods for estimating the extent of class imbalance without the need of knowing the actual data to preserve data privacy. After that, we discuss existing methods for handling class imbalance in FL, where the advantages and disadvantages of the these approaches are discussed. We also summarize common evaluation metrics for class imbalanced tasks, and point out potential future directions.
Style Miner: Find Significant and Stable Explanatory Factors in Time Series with Constrained Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

In high-dimensional time-series analysis, it is essential to have a set of key factors (namely, the style factors) that explain the change of the observed variable. For example, volatility modeling in finance relies on a set of risk factors, and climate change studies in climatology rely on a set of causal factors. The ideal low-dimensional style factors should balance significance (with high explanatory power) and stability (consistent, no significant fluctuations). However, previous supervised and unsupervised feature extraction methods can hardly address the tradeoff. In this paper, we propose Style Miner, a reinforcement learning method to generate style factors. We first formulate the problem as a Constrained Markov Decision Process with explanatory power as the return and stability as the constraint. Then, we design fine-grained immediate rewards and costs and use a Lagrangian heuristic to balance them adaptively. Experiments on real-world financial data sets show that Style Miner outperforms existing learning-based methods by a large margin and achieves a relatively 10% gain in R-squared explanatory power compared to the industry-renowned factors proposed by human experts.
Multi-modal Prompting for Low-Shot Temporal Action Localization
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, we consider the problem of temporal action localization under low-shot (zero-shot & few-shot) scenario, with the goal of detecting and classifying the action instances from arbitrary categories within some untrimmed videos, even not seen at training time. We adopt a Transformer-based two-stage action localization architecture with class-agnostic action proposal, followed by open-vocabulary classification. We make the following contributions. First, to compensate image-text foundation models with temporal motions, we improve category-agnostic action proposal by explicitly aligning embeddings of optical flows, RGB and texts, which has largely been ignored in existing low-shot methods. Second, to improve open-vocabulary action classification, we construct classifiers with strong discriminative power, i.e., avoid lexical ambiguities. To be specific, we propose to prompt the pre-trained CLIP text encoder either with detailed action descriptions (acquired from large-scale language models), or visually-conditioned instance-specific prompt vectors. Third, we conduct thorough experiments and ablation studies on THUMOS14 and ActivityNet1.3, demonstrating the superior performance of our proposed model, outperforming existing state-of-the-art approaches by one significant margin.
Generative AI for Cyber Threat-Hunting in 6G-enabled IoT Networks
Venue: arXiv cs.CR 2023
Year: 2023

The next generation of cellular technology, 6G, is being developed to enable a wide range of new applications and services for the Internet of Things (IoT). One of 6G's main advantages for IoT applications is its ability to support much higher data rates and bandwidth as well as to support ultra-low latency. However, with this increased connectivity will come to an increased risk of cyber threats, as attackers will be able to exploit the large network of connected devices. Generative Artificial Intelligence (AI) can be used to detect and prevent cyber attacks by continuously learning and adapting to new threats and vulnerabilities. In this paper, we discuss the use of generative AI for cyber threat-hunting (CTH) in 6G-enabled IoT networks. Then, we propose a new generative adversarial network (GAN) and Transformer-based model for CTH in 6G-enabled IoT Networks. The experimental analysis results with a new cyber security dataset demonstrate that the Transformer-based security model for CTH can detect IoT attacks with a high overall accuracy of 95%. We examine the challenges and opportunities and conclude by highlighting the potential of generative AI in enhancing the security of 6G-enabled IoT networks and call for further research to be conducted in this area.
Simulating Malaria Detection in Laboratories using Deep Learning
Venue: arXiv cs.CV 2023
Year: 2023

Malaria is usually diagnosed by a microbiologist by examining a small sample of blood smear. Reducing mortality from malaria infection is possible if it is diagnosed early and followed with appropriate treatment. While the WHO has set audacious goals of reducing malaria incidence and mortality rates by 90% in 2030 and eliminating malaria in 35 countries by that time, it still remains a difficult challenge. Computer-assisted diagnostics are on the rise these days as they can be used effectively as a primary test in the absence of or providing assistance to a physician or pathologist. The purpose of this paper is to describe an approach to detecting, localizing and counting parasitic cells in blood sample images towards easing the burden on healthcare workers.
Keep Your Friends Close, but Your Routeservers Closer: Insights into RPKI Validation in the Internet
Venue: arXiv cs.NI 2023
Year: 2023

IP prefix hijacks allow adversaries to redirect and intercept traffic, posing a threat to the stability and security of the Internet. To prevent prefix hijacks, networks should deploy RPKI and filter bogus BGP announcements with invalid routes. In this work we evaluate the impact of RPKI deployments on the security and resilience of the Internet. We aim to understand which networks filter invalid routes and how effective that filtering is in blocking prefix hijacks. We extend previous data acquisition and analysis methodologies to obtain more accurate identification of networks that filter invalid routes with RPKI. We find that more than 27% of networks enforce RPKI filtering and show for the first time that deployments follow the business incentives of inter-domain routing: providers have an increased motivation to filter in order to avoid losing customers' traffic. Analyzing the effectiveness of RPKI, we find that the current trend to deploy RPKI on routeservers of Internet Exchange Points (IXPs) only provides a localized protection against hijacks but has negligible impact on preventing their spread globally. In contrast, we show that RPKI filtering in Tier-1 providers greatly benefits the security of the Internet as it limits the spread of hijacks to a localized scope. Based on our observations, we provide recommendations on the future roadmap of RPKI deployment. We make our datasets available for public use [https://sit4.me/rpki].
The MERCADO Workshop at IEEE VIS 2023: Multimodal Experiences for Remote Communication Around Data Online
Venue: arXiv cs.HC 2023
Year: 2023

We propose a half-day workshop at IEEE VIS 2023 on the topic of communication and collaboration around data. Specifically, we aim to gather researchers interested on multimodal, synchronous, and remote or hybrid forms of communication and collaboration within organizational and educational settings. This topic lies at the intersection of data visualization, human-computer interaction, and computer-supported collaborative work, and overlaps thematically with several prior seminars and workshops. Our intended outcomes for the workshop include assembling a corpus of inspiring examples and a design space, ideally consolidated into a survey paper, as well as the establishment of new collaborations and a shared research agenda. We anticipate a format comprised of short presentations and demos, an invited keynote or fireside chat, and a breakout group session organized around specific application domains. Website: https://sites.google.com/view/mercadoworkshop.
Dynamic models for Planar Peristaltic Locomotion of a Metameric Earthworm-like Robot
Venue: arXiv cs.RO 2023
Year: 2023

The development of versatile robots capable of traversing challenging and irregular environments is of increasing interest in the field of robotics, and metameric robots have been identified as a promising solution due to their slender, deformable bodies. Inspired by the effective locomotion of earthworms, earthworm-like robots capable of both rectilinear and planar locomotion have been designed and prototyped. While much research has focused on developing kinematic models to describe the planar locomotion of earthworm-like robots, the authors argue that the development of dynamic models is critical to improving the accuracy and efficiency of these robots. A comprehensive analysis of the dynamics of a metameric earthworm-like robot capable of planar motion is presented in this work. The model takes into account the complex interactions between the robot's deformable body and the forces acting on it and draws on the methods previously used to develop mathematical models of snake-like robots. The proposed model represents a significant advancement in the field of metameric robotics and has the potential to enhance the performance of earthworm-like robots in a variety of challenging environments, such as underground pipes and tunnels, and serves as a foundation for future research into the dynamics of soft-bodied robots.
LEAPS: End-to-End One-Step Person Search With Learnable Proposals
Venue: arXiv cs.CV 2023
Year: 2023

We propose an end-to-end one-step person search approach with learnable proposals, named LEAPS. Given a set of sparse and learnable proposals, LEAPS employs a dynamic person search head to directly perform person detection and corresponding re-id feature generation without non-maximum suppression post-processing. The dynamic person search head comprises a detection head and a novel flexible re-id head. Our flexible re-id head first employs a dynamic region-of-interest (RoI) operation to extract discriminative RoI features of the proposals. Then, it generates re-id features using a plain and a hierarchical interaction re-id module. To better guide discriminative re-id feature learning, we introduce a diverse re-id sample matching strategy, instead of bipartite matching in detection head. Comprehensive experiments reveal the benefit of the proposed LEAPS, achieving a favorable performance on two public person search benchmarks: CUHK-SYSU and PRW. When using the same ResNet50 backbone, our LEAPS obtains a mAP score of 55.0%, outperforming the best reported results in literature by 1.7%, while achieving around a two-fold speedup on the challenging PRW dataset. Our source code and models will be released.
Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control
Venue: arXiv cs.NE 2023
Year: 2023

Transformers are state-of-the-art networks for most sequence processing tasks. However, the self-attention mechanism often used in Transformers requires large time windows for each computation step and thus makes them less suitable for online signal processing compared to Recurrent Neural Networks (RNNs). In this paper, instead of the self-attention mechanism, we use a sliding window attention mechanism. We show that this mechanism is more efficient for continuous signals with finite-range dependencies between input and target, and that we can use it to process sequences element-by-element, this making it compatible with online processing. We test our model on a finger position regression dataset (NinaproDB8) with Surface Electromyographic (sEMG) signals measured on the forearm skin to estimate muscle activities. Our approach sets the new state-of-the-art in terms of accuracy on this dataset while requiring only very short time windows of 3.5 ms at each inference step. Moreover, we increase the sparsity of the network using Leaky-Integrate and Fire (LIF) units, a bio-inspired neuron model that activates sparsely in time solely when crossing a threshold. We thus reduce the number of synaptic operations up to a factor of $\times5.3$ without loss of accuracy. Our results hold great promises for accurate and fast online processing of sEMG signals for smooth prosthetic hand control and is a step towards Transformers and Spiking Neural Networks (SNNs) co-integration for energy efficient temporal signal processing.
Penalty-Based Imitation Learning With Cross Semantics Generation Sensor Fusion for Autonomous Driving
Venue: arXiv cs.RO 2023
Year: 2023

In recent times, there has been a growing focus on end-to-end autonomous driving technologies. This technology involves the replacement of the entire driving pipeline with a single neural network, which has a simpler structure and faster inference time. However, while this approach reduces the number of components in the driving pipeline, it also presents challenges related to interpretability and safety. For instance, the trained policy may not always comply with traffic rules, and it is difficult to determine the reason for such misbehavior due to the lack of intermediate outputs. Additionally, the successful implementation of autonomous driving technology heavily depends on the reliable and expedient processing of sensory data to accurately perceive the surrounding environment. In this paper, we provide penalty-based imitation learning approach combined with cross semantics generation sensor fusion technologies (P-CSG) to efficiently integrate multiple modalities of information and enable the autonomous agent to effectively adhere to traffic regulations. Our model undergoes evaluation within the Town 05 Long benchmark, where we observe a remarkable increase in the driving score by more than 12% when compared to the state-of-the-art (SOTA) model, InterFuser. Notably, our model achieves this performance enhancement while achieving a 7-fold increase in inference speed and reducing the model size by approximately 30%. For more detailed information, including code-based resources, they can be found at https://hk-zh.github.io/p-csg/
Deephys: Deep Electrophysiology, Debugging Neural Networks under Distribution Shifts
Venue: arXiv cs.LG 2023
Year: 2023

Deep Neural Networks (DNNs) often fail in out-of-distribution scenarios. In this paper, we introduce a tool to visualize and understand such failures. We draw inspiration from concepts from neural electrophysiology, which are based on inspecting the internal functioning of a neural networks by analyzing the feature tuning and invariances of individual units. Deep Electrophysiology, in short Deephys, provides insights of the DNN's failures in out-of-distribution scenarios by comparative visualization of the neural activity in in-distribution and out-of-distribution datasets. Deephys provides seamless analyses of individual neurons, individual images, and a set of set of images from a category, and it is capable of revealing failures due to the presence of spurious features and novel features. We substantiate the validity of the qualitative visualizations of Deephys thorough quantitative analyses using convolutional and transformers architectures, in several datasets and distribution shifts (namely, colored MNIST, CIFAR-10 and ImageNet).
High Probability Bounds for Stochastic Continuous Submodular Maximization
Venue: arXiv cs.DS 2023
Year: 2023

We consider maximization of stochastic monotone continuous submodular functions (CSF) with a diminishing return property. Existing algorithms only guarantee the performance \textit{in expectation}, and do not bound the probability of getting a bad solution. This implies that for a particular run of the algorithms, the solution may be much worse than the provided guarantee in expectation. In this paper, we first empirically verify that this is indeed the case. Then, we provide the first \textit{high-probability} analysis of the existing methods for stochastic CSF maximization, namely PGA, boosted PGA, SCG, and SCG++. Finally, we provide an improved high-probability bound for SCG, under slightly stronger assumptions, with a better convergence rate than that of the expected solution. Through extensive experiments on non-concave quadratic programming (NQP) and optimal budget allocation, we confirm the validity of our bounds and show that even in the worst-case, PGA converges to $OPT/2$, and boosted PGA, SCG, SCG++ converge to $(1 - 1/e)OPT$, but at a slower rate than that of the expected solution.
Preparing Unprepared Students For Future Learning
Venue: arXiv cs.HC 2023
Year: 2023

Based on strategy-awareness (knowing which problem-solving strategy to use) and time-awareness (knowing when to use it), students are categorized into Rote (neither type of awareness), Dabbler (strategy-aware only) or Selective (both types of awareness). It was shown that Selective is often significantly more prepared for future learning than Rote and Dabbler (Abdelshiheed et al., 2020). In this work, we explore the impact of explicit strategy instruction on Rote and Dabbler students across two domains: logic and probability. During the logic instruction, our logic tutor handles both Forward-Chaining (FC) and Backward-Chaining (BC) strategies, with FC being the default; the Experimental condition is taught how to use BC via worked examples and when to use it via prompts. Six weeks later, all students are trained on a probability tutor that supports BC only. Our results show that Experimental significantly outperforms Control in both domains, and Experimental Rote catches up with Selective.
cTBLS: Augmenting Large Language Models with Conversational Tables
Venue: arXiv cs.CL 2023
Year: 2023

Optimizing accuracy and performance while eliminating hallucinations of open-domain conversational large language models (LLMs) is an open research challenge. A particularly promising direction is to augment and ground LLMs with information from structured sources. This paper introduces Conversational Tables (cTBLS), a three-step architecture to retrieve and generate dialogue responses grounded on retrieved tabular information. cTBLS uses Transformer encoder embeddings for Dense Table Retrieval and obtains up to 125% relative improvement over the retriever in the previous state-of-the-art system on the HyrbiDialogue dataset. cTBLS then uses a shared process between encoder and decoder models to perform a coarse+fine tabular knowledge (e.g., cell) ranking combined with a GPT-3.5 LLM response generator to yield a 2x relative improvement in ROUGE scores. Finally, human evaluators prefer cTBLs +80% of the time (coherency, fluency) and judge informativeness to be 4x better than the previous state-of-the-art.