OpenRaiser/Intern-Atlas huggingface dataset:
Analysis of a Delay-Tolerant Data Harvest Architecture Leveraging Low Earth Orbit Satellite Networks
Venue: arXiv cs.IT 2024
Year: 2024

Reaching all regions of Earth, low Earth orbit (LEO) satellites can harvest delay-tolerant data from remotely located users on Earth without ground infrastructure. This work aims to assess a data harvest network architecture where users generate data and LEO satellites harvest data from users when passing by. By developing a novel stochastic geometry Cox point process model that simultaneously generates orbits and the motion of LEO satellite harvesters on them, we analyze key performance indices of such a network by deriving the following: (i) the average fraction of time that the typical user is served by LEO satellite harvesters, (ii) the average amount of data uploaded per each satellite pass, (iii) the maximum harvesting capacity of the proposed network model, and (iv) the delay distribution in the proposed network. These key metrics are given as functions of key network variables such as $λ$ the mean number of orbits and $μ$ the mean number of satellites per orbit. Providing rich comprehensive analytical results and practical interpretations of these results, this work assesses the potential of the delay-tolerant use of LEO satellites and also serves as a versatile framework to analyze, design, and optimize delay-tolerant LEO satellite networks.
Exploring Capability-Based Control Distributions of Human-Robot Teams Through Capability Deltas: Formalization and Implications
Venue: arXiv cs.RO 2024
Year: 2024

The implicit assumption that human and autonomous agents have certain capabilities is omnipresent in modern teaming concepts. However, none formalize these capabilities in a flexible and quantifiable way. In this paper, we propose Capability Deltas, which establish a quantifiable source to craft autonomous assistance systems in which one agent takes the leader and the other the supporter role. We deduct the quantification of human capabilities based on an established assessment and documentation procedure from occupational inclusion of people with disabilities. This allows us to quantify the delta, or gap, between a team's current capability and a requirement established by a work process. The concept is then extended to the multi-dimensional capability space, which then allows to formalize compensation behavior and assess required actions by the autonomous agent.
Modeling Electromagnetic Signal Injection Attacks on Camera-based Smart Systems: Applications and Mitigation
Venue: arXiv cs.CR 2024
Year: 2024

Numerous safety- or security-critical systems depend on cameras to perceive their surroundings, further allowing artificial intelligence (AI) to analyze the captured images to make important decisions. However, a concerning attack vector has emerged, namely, electromagnetic waves, which pose a threat to the integrity of these systems. Such attacks enable attackers to manipulate the images remotely, leading to incorrect AI decisions, e.g., autonomous vehicles missing detecting obstacles ahead resulting in collisions. The lack of understanding regarding how different systems react to such attacks poses a significant security risk. Furthermore, no effective solutions have been demonstrated to mitigate this threat. To address these gaps, we modeled the attacks and developed a simulation method for generating adversarial images. Through rigorous analysis, we confirmed that the effects of the simulated adversarial images are indistinguishable from those from real attacks. This method enables researchers and engineers to rapidly assess the susceptibility of various AI vision applications to these attacks, without the need for constructing complicated attack devices. In our experiments, most of the models demonstrated vulnerabilities to these attacks, emphasizing the need to enhance their robustness. Fortunately, our modeling and simulation method serves as a stepping stone toward developing more resilient models. We present a pilot study on adversarial training to improve their robustness against attacks, and our results demonstrate a significant improvement by recovering up to 91% performance, offering a promising direction for mitigating this threat.
Large Language Models and Thematic Analysis: Human-AI Synergy in Researching Hate Speech on Social Media
Venue: arXiv cs.HC 2024
Year: 2024

In the dynamic field of artificial intelligence (AI), the development and application of Large Language Models (LLMs) for text analysis are of significant academic interest. Despite the promising capabilities of various LLMs in conducting qualitative analysis, their use in the humanities and social sciences has not been thoroughly examined. This article contributes to the emerging literature on LLMs in qualitative analysis by documenting an experimental study involving GPT-4. The study focuses on performing thematic analysis (TA) using a YouTube dataset derived from an EU-funded project, which was previously analyzed by other researchers. This dataset is about the representation of Roma migrants in Sweden during 2016, a period marked by the aftermath of the 2015 refugee crisis and preceding the Swedish national elections in 2017. Our study seeks to understand the potential of combining human intelligence with AI's scalability and efficiency, examining the advantages and limitations of employing LLMs in qualitative research within the humanities and social sciences. Additionally, we discuss future directions for applying LLMs in these fields.
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
Venue: arXiv cs.LG 2024
Year: 2024

Sparse autoencoders (SAEs) are an unsupervised method for learning a sparse decomposition of a neural network's latent representations into seemingly interpretable features. Despite recent excitement about their potential, research applications outside of industry are limited by the high cost of training a comprehensive suite of SAEs. In this work, we introduce Gemma Scope, an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2 2B and 9B and select layers of Gemma 2 27B base models. We primarily train SAEs on the Gemma 2 pre-trained models, but additionally release SAEs trained on instruction-tuned Gemma 2 9B for comparison. We evaluate the quality of each SAE on standard metrics and release these results. We hope that by releasing these SAE weights, we can help make more ambitious safety and interpretability research easier for the community. Weights and a tutorial can be found at https://huggingface.co/google/gemma-scope and an interactive demo can be found at https://www.neuronpedia.org/gemma-scope
Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications
Venue: arXiv cs.DC 2024
Year: 2024

Run to run variability in parallel programs caused by floating-point non-associativity has been known to significantly affect reproducibility in iterative algorithms, due to accumulating errors. Non-reproducibility can critically affect the efficiency and effectiveness of correctness testing for stochastic programs. Recently, the sensitivity of deep learning training and inference pipelines to floating-point non-associativity has been found to sometimes be extreme. It can prevent certification for commercial applications, accurate assessment of robustness and sensitivity, and bug detection. New approaches in scientific computing applications have coupled deep learning models with high-performance computing, leading to an aggravation of debugging and testing challenges. Here we perform an investigation of the statistical properties of floating-point non-associativity within modern parallel programming models, and analyze performance and productivity impacts of replacing atomic operations with deterministic alternatives on GPUs. We examine the recently-added deterministic options in PyTorch within the context of GPU deployment for deep learning, uncovering and quantifying the impacts of input parameters triggering run to run variability and reporting on the reliability and completeness of the documentation. Finally, we evaluate the strategy of exploiting automatic determinism that could be provided by deterministic hardware, using the Groq accelerator for inference portions of the deep learning pipeline. We demonstrate the benefits that a hardware-based strategy can provide within reproducibility and correctness efforts.
Meta-Learning Guided Label Noise Distillation for Robust Signal Modulation Classification
Venue: arXiv cs.LG 2024
Year: 2024

Automatic modulation classification (AMC) is an effective way to deal with physical layer threats of the internet of things (IoT). However, there is often label mislabeling in practice, which significantly impacts the performance and robustness of deep neural networks (DNNs). In this paper, we propose a meta-learning guided label noise distillation method for robust AMC. Specifically, a teacher-student heterogeneous network (TSHN) framework is proposed to distill and reuse label noise. Based on the idea that labels are representations, the teacher network with trusted meta-learning divides and conquers untrusted label samples and then guides the student network to learn better by reassessing and correcting labels. Furthermore, we propose a multi-view signal (MVS) method to further improve the performance of hard-to-classify categories with few-shot trusted label samples. Extensive experimental results show that our methods can significantly improve the performance and robustness of signal AMC in various and complex label noise scenarios, which is crucial for securing IoT applications.
Deep-change at AXOLOTL-24: Orchestrating WSD and WSI Models for Semantic Change Modeling
Venue: arXiv cs.CL 2024
Year: 2024

This paper describes our solution of the first subtask from the AXOLOTL-24 shared task on Semantic Change Modeling. The goal of this subtask is to distribute a given set of usages of a polysemous word from a newer time period between senses of this word from an older time period and clusters representing gained senses of this word. We propose and experiment with three new methods solving this task. Our methods achieve SOTA results according to both official metrics of the first substask. Additionally, we develop a model that can tell if a given word usage is not described by any of the provided sense definitions. This model serves as a component in one of our methods, but can potentially be useful on its own.
Cell Morphology-Guided Small Molecule Generation with GFlowNets
Venue: arXiv cs.LG 2024
Year: 2024

High-content phenotypic screening, including high-content imaging (HCI), has gained popularity in the last few years for its ability to characterize novel therapeutics without prior knowledge of the protein target. When combined with deep learning techniques to predict and represent molecular-phenotype interactions, these advancements hold the potential to significantly accelerate and enhance drug discovery applications. This work focuses on the novel task of HCI-guided molecular design. Generative models for molecule design could be guided by HCI data, for example with a supervised model that links molecules to phenotypes of interest as a reward function. However, limited labeled data, combined with the high-dimensional readouts, can make training these methods challenging and impractical. We consider an alternative approach in which we leverage an unsupervised multimodal joint embedding to define a latent similarity as a reward for GFlowNets. The proposed model learns to generate new molecules that could produce phenotypic effects similar to those of the given image target, without relying on pre-annotated phenotypic labels. We demonstrate that the proposed method generates molecules with high morphological and structural similarity to the target, increasing the likelihood of similar biological activity, as confirmed by an independent oracle model.
Comment on "An Efficient Privacy-Preserving Ranked Multi-Keyword Retrieval for Multiple Data Owners in Outsourced Cloud"
Venue: arXiv cs.CR 2024
Year: 2024

Protecting the privacy of keywords in the field of search over outsourced cloud data is a challenging task. In IEEE Transactions on Services Computing (Vol. 17 No. 2, March/April 2024), Li et al. proposed PRMKR: efficient privacy-preserving ranked multi-keyword retrieval scheme, which was claimed to resist keyword guessing attack. However, we show that the scheme fails to resist keyword guessing attack, index privacy, and trapdoor privacy. Further, we propose a solution to address the above said issues by correcting the errors in the important equations of the scheme.
Early-Stage Requirements Transformation Approaches: A Systematic Review
Venue: arXiv cs.SE 2024
Year: 2024

Transformation approaches for automatically constructing analysis models from textual requirements are critical to software development, as they can bring forward the use of precise formal languages from the coding phase to the requirement analysis phase in the software development life-cycle. Over the decades, numerous transformation approaches have been developed in an attempt to fully or partially automate this initial phase. This systematic review examines transformation approaches in the early stages of software development, examining 25 studies on early-stage requirements transformation documented between 2000 and 2014. The review highlights the widespread use of natural language processing techniques, with tools like the Stanford parser and WordNet being essential. Intermediate models are often used in the transformation process to bridge the gap between textual requirements and analysis models. Significant advancements have been made in early-stage requirements transformation approaches; however, several areas require attention to enhance their effectiveness and reliability. A challenge identified is the lack of robust evaluation methods, with most approaches using simple case studies and running examples for evaluation. This makes it difficult to compare and evaluate the performance these approaches. Although most approaches can generate structural models from textual requirements, many generate incomplete models with missing elements. Furthermore, requirements traceability is largely neglected, with only two approaches addressing it and lacking explicit detail on how traceability links are maintained during the transformation process. This review emphasize the need for formalized evaluation techniques and greater transparency and accessibility of approaches used in the early-stage requirements transformation.
Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

As the performance of larger, newer Large Language Models continues to improve for strategic Theory of Mind (ToM) tasks, the demand for these state-of-the-art models increases commensurately. However, their deployment is costly both in terms of processing power and time. In this paper, we investigate the feasibility of creating smaller, highly-performing specialized algorithms by way of fine-tuning. To do this, we first present a large pre-trained model with 20 unique scenarios that combine different social contexts with games of varying social dilemmas, record its answers, and use them for Q&A fine-tuning on a smaller model of the same family. Our focus is on in-context game-theoretic decision-making, the same domain within which human interaction occurs and that requires both a theory of mind (or a semblance thereof) and an understanding of social dynamics. The smaller model is therefore trained not just on the answers provided, but also on the motivations provided by the larger model, which should contain advice and guidelines to navigate both strategic dilemmas and social cues. We find that the fine-tuned smaller language model consistently bridged the gap in performance between the smaller pre-trained version of the model and its larger relative and that its improvements extended in areas and contexts beyond the ones provided in the training examples, including on out-of-sample scenarios that include completely different game structures. On average for all games, through fine-tuning, the smaller model showed a 46% improvement measured as alignment towards the behavior of the larger model, with 100% representing indistinguishable behavior. When presented with out-of-sample social contexts and games, the fine-tuned model still displays remarkable levels of alignment, reaching an improvement of 18% and 28% respectively.
SocFedGPT: Federated GPT-based Adaptive Content Filtering System Leveraging User Interactions in Social Networks
Venue: arXiv cs.LG 2024
Year: 2024

Our study presents a multifaceted approach to enhancing user interaction and content relevance in social media platforms through a federated learning framework. We introduce personalized GPT and Context-based Social Media LLM models, utilizing federated learning for privacy and security. Four client entities receive a base GPT-2 model and locally collected social media data, with federated aggregation ensuring up-to-date model maintenance. Subsequent modules focus on categorizing user posts, computing user persona scores, and identifying relevant posts from friends' lists. A quantifying social engagement approach, coupled with matrix factorization techniques, facilitates personalized content suggestions in real-time. An adaptive feedback loop and readability score algorithm also enhance the quality and relevance of content presented to users. Our system offers a comprehensive solution to content filtering and recommendation, fostering a tailored and engaging social media experience while safeguarding user privacy.
Semi-Supervised One-Shot Imitation Learning
Venue: arXiv cs.LG 2024
Year: 2024

One-shot Imitation Learning~(OSIL) aims to imbue AI agents with the ability to learn a new task from a single demonstration. To supervise the learning, OSIL typically requires a prohibitively large number of paired expert demonstrations -- i.e. trajectories corresponding to different variations of the same semantic task. To overcome this limitation, we introduce the semi-supervised OSIL problem setting, where the learning agent is presented with a large dataset of trajectories with no task labels (i.e. an unpaired dataset), along with a small dataset of multiple demonstrations per semantic task (i.e. a paired dataset). This presents a more realistic and practical embodiment of few-shot learning and requires the agent to effectively leverage weak supervision from a large dataset of trajectories. Subsequently, we develop an algorithm specifically applicable to this semi-supervised OSIL setting. Our approach first learns an embedding space where different tasks cluster uniquely. We utilize this embedding space and the clustering it supports to self-generate pairings between trajectories in the large unpaired dataset. Through empirical results on simulated control tasks, we demonstrate that OSIL models trained on such self-generated pairings are competitive with OSIL models trained with ground-truth labels, presenting a major advancement in the label-efficiency of OSIL.
Beyond Diversity:Computing for Inclusive Software
Venue: arXiv cs.SE 2024
Year: 2024

This chapter presents, from our research on inclusive software within the context of a diversity and inclusion based STEM program at the University of Victoria, INSPIRE: STEM for Social Impact (hereafter Inspire). In a society with an ever increasing reliance on technology, we often neglect the fact that software development processes and practices unintentionally marginalize certain groups of end users. While the Inspire program and its first iteration in 2022 are described in detail in CHAPTER 26, here we describe our insights from an analysis of the development processes and practices used by the teams. We found that empathy-based requirements gathering techniques and certain influences on the software development teams' motivation levels impact the teams' ability to build inclusive software. This chapter begins with an explanation of the Inspire program and a discussion on what the term ``inclusive software'' actually means in our context before highlighting useful practices for designing inclusive software.
A Recurrent YOLOv8-based framework for Event-Based Object Detection
Venue: arXiv cs.CV 2024
Year: 2024

Object detection is crucial in various cutting-edge applications, such as autonomous vehicles and advanced robotics systems, primarily relying on data from conventional frame-based RGB sensors. However, these sensors often struggle with issues like motion blur and poor performance in challenging lighting conditions. In response to these challenges, event-based cameras have emerged as an innovative paradigm. These cameras, mimicking the human eye, demonstrate superior performance in environments with fast motion and extreme lighting conditions while consuming less power. This study introduces ReYOLOv8, an advanced object detection framework that enhances a leading frame-based detection system with spatiotemporal modeling capabilities. We implemented a low-latency, memory-efficient method for encoding event data to boost the system's performance. We also developed a novel data augmentation technique tailored to leverage the unique attributes of event data, thus improving detection accuracy. Our models outperformed all comparable approaches in the GEN1 dataset, focusing on automotive applications, achieving mean Average Precision (mAP) improvements of 5%, 2.8%, and 2.5% across nano, small, and medium scales, respectively.These enhancements were achieved while reducing the number of trainable parameters by an average of 4.43% and maintaining real-time processing speeds between 9.2ms and 15.5ms. On the PEDRo dataset, which targets robotics applications, our models showed mAP improvements ranging from 9% to 18%, with 14.5x and 3.8x smaller models and an average speed enhancement of 1.67x.
Monero Traceability Heuristics: Wallet Application Bugs and the Mordinal-P2Pool Perspective
Venue: arXiv cs.CR 2024
Year: 2024

Privacy-focused cryptoassets like Monero are intentionally difficult to trace. Over the years, several traceability heuristics have been proposed, most of which have been rendered ineffective with subsequent protocol upgrades. Between 2019 and 2023, Monero wallet application bugs "Differ By One" and "10 Block Decoy Bug" have been observed and identified and discussed in the Monero community. In addition, a decentralized mining pool named P2Pool has proliferated, and a controversial UTXO NFT imitation known as Mordinals has been tried for Monero. In this paper, we systematically describe the traceability heuristics that have emerged from these developments, and evaluate their quality based on ground truth, and through pairwise comparisons. We also explore the temporal perspective, and show which of these heuristics have been applicable over the past years, what fraction of decoys could be eliminated and what the remaining effective ring size is. Our findings illustrate that most of the heuristics have a high precision, that the "10 Block Decoy Bug" and the Coinbase decoy identification heuristics have had the most impact between 2019 and 2023, and that the former could be used to evaluate future heuristics, if they are also applicable during that time frame.