OpenRaiser/Intern-Atlas huggingface dataset:
The Ethics of AI Value Chains
Venue: arXiv cs.CY 2024
Year: 2024

Researchers, practitioners, and policymakers with an interest in AI ethics need more integrative approaches for studying and intervening in AI systems across many contexts and scales of activity. This paper presents AI value chains as an integrative concept that satisfies that need. To more clearly theorize AI value chains and conceptually distinguish them from supply chains, we review theories of value chains and AI value chains from the strategic management, service science, economic geography, industry, government, and applied research literature. We then conduct an integrative review of a sample of 67 sources that cover the ethical concerns implicated in AI value chains. Building upon the findings of our integrative review, we recommend three future directions that researchers, practitioners, and policymakers can take to advance more ethical practices across AI value chains. We urge AI ethics researchers and practitioners to move toward value chain perspectives that situate actors in context, account for the many types of resources involved in co-creating AI systems, and integrate a wider range of ethical concerns across contexts and scales.
Towards Semantically Enriched Embeddings for Knowledge Graph Completion
Venue: arXiv cs.AI 2024
Year: 2024

Embedding based Knowledge Graph (KG) Completion has gained much attention over the past few years. Most of the current algorithms consider a KG as a multidirectional labeled graph and lack the ability to capture the semantics underlying the schematic information. In a separate development, a vast amount of information has been captured within the Large Language Models (LLMs) which has revolutionized the field of Artificial Intelligence. KGs could benefit from these LLMs and vice versa. This vision paper discusses the existing algorithms for KG completion based on the variations for generating KG embeddings. It starts with discussing various KG completion algorithms such as transductive and inductive link prediction and entity type prediction algorithms. It then moves on to the algorithms utilizing type information within the KGs, LLMs, and finally to algorithms capturing the semantics represented in different description logic axioms. We conclude the paper with a critical reflection on the current state of work in the community and give recommendations for future directions.
A Sentence is Worth a Thousand Pictures: Can Large Language Models Understand Hum4n L4ngu4ge and the W0rld behind W0rds?
Venue: arXiv cs.CL 2024
Year: 2024

Modern Artificial Intelligence applications show great potential for language-related tasks that rely on next-word prediction. The current generation of Large Language Models (LLMs) have been linked to claims about human-like linguistic performance and their applications are hailed both as a step towards artificial general intelligence and as a major advance in understanding the cognitive, and even neural basis of human language. To assess these claims, first we analyze the contribution of LLMs as theoretically informative representations of a target cognitive system vs. atheoretical mechanistic tools. Second, we evaluate the models' ability to see the bigger picture, through top-down feedback from higher levels of processing, which requires grounding in previous expectations and past world experience. We hypothesize that since models lack grounded cognition, they cannot take advantage of these features and instead solely rely on fixed associations between represented words and word vectors. To assess this, we designed and ran a novel 'leet task' (l33t t4sk), which requires decoding sentences in which letters are systematically replaced by numbers. The results suggest that humans excel in this task whereas models struggle, confirming our hypothesis. We interpret the results by identifying the key abilities that are still missing from the current state of development of these models, which require solutions that go beyond increased system scaling.
Kinodynamic Motion Planning via Funnel Control for Underactuated Unmanned Surface Vehicles
Venue: arXiv cs.RO 2024
Year: 2024

We develop an algorithm to control an underactuated unmanned surface vehicle (USV) using kinodynamic motion planning with funnel control (KDF). KDF has two key components: motion planning used to generate trajectories with respect to kinodynamic constraints, and funnel control, also referred to as prescribed performance control, which enables trajectory tracking in the presence of uncertain dynamics and disturbances. We extend prescribed performance control to address the challenges posed by underactuation and control-input saturation present on the USV. The proposed scheme guarantees stability under user-defined prescribed performance functions where model parameters and exogenous disturbances are unknown. Furthermore, we present an optimization problem to obtain smooth, collision-free trajectories while respecting kinodynamic constraints. We deploy the algorithm on a USV and verify its efficiency in real-world open-water experiments.
On the Impact of the Hardware Warm-Up Time on Deep Learning-Based RF Fingerprinting
Venue: arXiv cs.CR 2024
Year: 2024

Deep learning-based RF fingerprinting offers great potential for improving the security robustness of various emerging wireless networks. Although much progress has been done in enhancing fingerprinting methods, the impact of device hardware stabilization and warm-up time on the achievable fingerprinting performances has not received adequate attention. As such, this paper focuses on addressing this gap by investigating and shedding light on what could go wrong if the hardware stabilization aspects are overlooked. Specifically, our experimental results show that when the deep learning models are trained with data samples captured after the hardware stabilizes but tested with data captured right after powering on the devices, the device classification accuracy drops below 37%. However, when both the training and testing data are captured after the stabilization period, the achievable average accuracy exceeds 99%, when the model is trained and tested on the same day, and achieves 88% and 96% when the model is trained on one day but tested on another day, for the wireless and wired scenarios, respectively. Additionally, in this work, we leverage simulation and testbed experimentation to explain the cause behind the I/Q signal behavior observed during the device hardware warm-up time that led to the RF fingerprinting performance degradation. Furthermore, we release a large WiFi dataset, containing both unstable (collected during the warm-up period) and stable (collected after the warm-up period) captures across multiple days. Our work contributes datasets, explanations, and guidelines to enhance the robustness of RF fingerprinting in securing emerging wireless networks.
Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias
Venue: arXiv cs.AI 2024
Year: 2024

Recent studies show that instruction tuning (IT) and reinforcement learning from human feedback (RLHF) improve the abilities of large language models (LMs) dramatically. While these tuning methods can help align models with human objectives and generate high-quality text, not much is known about their potential adverse effects. In this work, we investigate the effect of IT and RLHF on decision making and reasoning in LMs, focusing on three cognitive biases - the decoy effect, the certainty effect, and the belief bias - all of which are known to influence human decision-making and reasoning. Our findings highlight the presence of these biases in various models from the GPT-3, Mistral, and T5 families. Notably, we find a stronger presence of biases in models that have undergone instruction tuning, such as Flan-T5, Mistral-Instruct, GPT3.5, and GPT4. Our work constitutes a step toward comprehending cognitive biases in instruction-tuned LMs, which is crucial for the development of more reliable and unbiased language models.
COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection
Venue: arXiv cs.CV 2024
Year: 2024

DeepFakes have raised serious societal concerns, leading to a great surge in detection-based forensics methods in recent years. Face forgery recognition is a standard detection method that usually follows a two-phase pipeline. While those methods perform well in ideal experimental environment, they face challenges when dealing with DeepFakes in the wild involving complex background and multiple faces of varying sizes. Moreover, most face forgery recognition methods can only process one face at a time. One straightforward way to address this issue is to simultaneous process multi-face by integrating face extraction and forgery detection in an end-to-end fashion by adapting advanced object detection architectures. However, as these object detection architectures are designed to capture the discriminative features of different object categories rather than the subtle forgery traces among the faces, the direct adaptation suffers from limited representation ability. In this paper, we propose COMICS, an end-to-end framework for multi-face forgery detection. COMICS integrates face extraction and forgery detection in a seamless manner and adapts to advanced object detection architectures. The proposed bi-grained contrastive learning approach explores face forgery traces at both the coarse- and fine-grained levels. Specifically, coarse-grained level contrastive learning captures the discriminative features among positive and negative proposal pairs at multiple layers produced by the proposal generator, and fine-grained level contrastive learning captures the pixel-wise discrepancy between the forged and original areas of the same face and the pixel-wise content inconsistency among different faces. Extensive experiments on the OpenForensics and FFIW datasets demonstrate that our method outperforms other counterparts and shows great potential for being integrated into various architectures.
MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognition
Venue: arXiv cs.CV 2024
Year: 2024

Applying large-scale vision-language pre-trained models like CLIP to few-shot action recognition (FSAR) can significantly enhance both performance and efficiency. While several studies have recognized this advantage, most of them resort to full-parameter fine-tuning to make CLIP's visual encoder adapt to the FSAR data, which not only costs high computations but also overlooks the potential of the visual encoder to engage in temporal modeling and focus on targeted semantics directly. To tackle these issues, we introduce MA-FSAR, a framework that employs the Parameter-Efficient Fine-Tuning (PEFT) technique to enhance the CLIP visual encoder in terms of action-related temporal and semantic representations. Our solution involves a Fine-grained Multimodal Adaptation, which is different from the previous attempts of PEFT in regular action recognition. Specifically, we first insert a Global Temporal Adaptation that only receives the class token to capture global motion cues efficiently. Then these outputs integrate with visual tokens to enhance local temporal dynamics by a Local Multimodal Adaptation, which incorporates text features unique to the FSAR support set branch to highlight fine-grained semantics related to actions. In addition to these token-level designs, we propose a prototype-level text-guided construction module to further enrich the temporal and semantic characteristics of video prototypes. Extensive experiments demonstrate our superior performance in various tasks using minor trainable parameters.
End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control
Venue: arXiv cs.LG 2024
Year: 2024

(Economic) nonlinear model predictive control ((e)NMPC) requires dynamic models that are sufficiently accurate and computationally tractable. Data-driven surrogate models for mechanistic models can reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum prediction accuracy on simulation samples and perform suboptimally in (e)NMPC. We present a method for end-to-end reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We apply our method to two applications derived from an established nonlinear continuous stirred-tank reactor model. The controller performance is compared to that of (e)NMPCs utilizing models trained using system identification, and model-free neural network controllers trained using reinforcement learning. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC, and that, in contrast to the neural network controllers, the (e)NMPC controllers can react to changes in the control setting without retraining.
Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution
Venue: arXiv cs.CV 2024
Year: 2024

Visibility in hazy nighttime scenes is frequently reduced by multiple factors, including low light, intense glow, light scattering, and the presence of multicolored light sources. Existing nighttime dehazing methods often struggle with handling glow or low-light conditions, resulting in either excessively dark visuals or unsuppressed glow outputs. In this paper, we enhance the visibility from a single nighttime haze image by suppressing glow and enhancing low-light regions. To handle glow effects, our framework learns from the rendered glow pairs. Specifically, a light source aware network is proposed to detect light sources of night images, followed by the APSF (Atmospheric Point Spread Function)-guided glow rendering. Our framework is then trained on the rendered images, resulting in glow suppression. Moreover, we utilize gradient-adaptive convolution, to capture edges and textures in hazy scenes. By leveraging extracted edges and textures, we enhance the contrast of the scene without losing important structural details. To boost low-light intensity, our network learns an attention map, then adjusted by gamma correction. This attention has high values on low-light regions and low values on haze and glow regions. Extensive evaluation on real nighttime haze images, demonstrates the effectiveness of our method. Our experiments demonstrate that our method achieves a PSNR of 30.38dB, outperforming state-of-the-art methods by 13% on GTA5 nighttime haze dataset. Our data and code is available at https://github.com/jinyeying/nighttime_dehaze.
BlockChain I/O: Enabling Cross-Chain Commerce
Venue: arXiv cs.CR 2024
Year: 2024

Blockchain technology enables secure tokens transfers in digital marketplaces, and recent advances in this field provide other desirable properties such as efficiency, privacy, and price stability. However, these properties do not always generalize to a setting across multiple independent blockchains. Despite the growing number of existing blockchain platforms, there is a lack of an overarching framework whose components provide all of the necessary properties for practical cross-chain commerce. We present BlockChain I/O to provide such a framework. BlockChain I/O introduces entities called cross-chain services to relay information between different blockchains. The proposed design ensures that cross-chain services cannot violate transaction safety, and they are furthermore disincentivized from other types of misbehavior through an audit system. BlockChain I/O uses native stablecoins to mitigate price fluctuations, and a decentralized ID system to allow users to prove aspects of their identity without violating privacy. After presenting the core architecture of BlockChain I/O, we demonstrate how to use it to implement a cross-chain marketplace and discuss how its desirable properties continue to hold in the end-to-end system. Finally, we use experimental evaluations to demonstrate BlockChain I/O's practical performance.
Paired Competing Neurons Improving STDP Supervised Local Learning In Spiking Neural Networks
Venue: arXiv cs.CV 2024
Year: 2024

Direct training of Spiking Neural Networks (SNNs) on neuromorphic hardware has the potential to significantly reduce the energy consumption of artificial neural network training. SNNs trained with Spike Timing-Dependent Plasticity (STDP) benefit from gradient-free and unsupervised local learning, which can be easily implemented on ultra-low-power neuromorphic hardware. However, classification tasks cannot be performed solely with unsupervised STDP. In this paper, we propose Stabilized Supervised STDP (S2-STDP), a supervised STDP learning rule to train the classification layer of an SNN equipped with unsupervised STDP for feature extraction. S2-STDP integrates error-modulated weight updates that align neuron spikes with desired timestamps derived from the average firing time within the layer. Then, we introduce a training architecture called Paired Competing Neurons (PCN) to further enhance the learning capabilities of our classification layer trained with S2-STDP. PCN associates each class with paired neurons and encourages neuron specialization toward target or non-target samples through intra-class competition. We evaluate our methods on image recognition datasets, including MNIST, Fashion-MNIST, and CIFAR-10. Results show that our methods outperform state-of-the-art supervised STDP learning rules, for comparable architectures and numbers of neurons. Further analysis demonstrates that the use of PCN enhances the performance of S2-STDP, regardless of the hyperparameter set and without introducing any additional hyperparameters.
Sowing 'Seeds of Doubt': Cottage Industries of Election and Medical Misinformation in Brazil and the United States
Venue: arXiv cs.HC 2024
Year: 2024

We conducted ethnographic research with 31 misinformation creators and consumers in Brazil and the US before, during, and after a major election to understand the consumption and production of election and medical misinformation. This study contributes to research on misinformation ecosystems by focusing on poorly understood small players, or "micro-influencers", who create misinformation in peer-to-peer networks. We detail four key tactics that micro-influencers use. First, they typically disseminate "gray area" content rather than expert-falsified claims, using subtle aesthetic and rhetorical tactics to evade moderation. Second, they post in small, closed groups where members feel safe and predisposed to trust content. Third, they explicitly target misinformation consumers' emotional and social needs. Finally, they post a high volume of short, repetitive content to plant seeds of doubt and build trust in influencers as unofficial experts. We discuss the implications these micro-influencers have for misinformation interventions and platforms' efforts to moderate misinformation.
Label Inference Attacks against Node-level Vertical Federated GNNs
Venue: arXiv cs.LG 2024
Year: 2024

Federated learning enables collaborative training of machine learning models by keeping the raw data of the involved workers private. Three of its main objectives are to improve the models' privacy, security, and scalability. Vertical Federated Learning (VFL) offers an efficient cross-silo setting where a few parties collaboratively train a model without sharing the same features. In such a scenario, classification labels are commonly considered sensitive information held exclusively by one (active) party, while other (passive) parties use only their local information. Recent works have uncovered important flaws of VFL, leading to possible label inference attacks under the assumption that the attacker has some, even limited, background knowledge on the relation between labels and data. In this work, we are the first (to the best of our knowledge) to investigate label inference attacks on VFL using a zero-background knowledge strategy. To formulate our proposal, we focus on Graph Neural Networks (GNNs) as a target model for the underlying VFL. In particular, we refer to node classification tasks, which are widely studied, and GNNs have shown promising results. Our proposed attack, BlindSage, provides impressive results in the experiments, achieving nearly 100% accuracy in most cases. Even when the attacker has no information about the used architecture or the number of classes, the accuracy remains above 90% in most instances. Finally, we observe that well-known defenses cannot mitigate our attack without affecting the model's performance on the main classification task.
Demystifying the RSA Algorithm: An Intuitive Introduction for Novices in Cybersecurity
Venue: arXiv cs.CR 2024
Year: 2024

Given the escalating importance of cybersecurity, it becomes increasingly beneficial for a diverse community to comprehend fundamental security mechanisms. Among these, the RSA algorithm stands out as a crucial component in public-key cryptosystems. However, understanding the RSA algorithm typically entails familiarity with number theory, modular arithmetic, and related concepts, which can often exceed the knowledge base of beginners entering the field of cybersecurity. In this study, we present an intuitively crafted, student-oriented introduction to the RSA algorithm. We assume that our readers possess only a basic background in mathematics and cybersecurity. Commencing with the three essential goals of public-key cryptosystems, we provide a step-by-step elucidation of how the RSA algorithm accomplishes these objectives. Additionally, we employ a toy example to further enhance practical understanding. Our assessment of student learning outcomes, conducted across two sections of the same course, reveals a discernible improvement in grades for the students.
Affective Visualization Design: Leveraging the Emotional Impact of Data
Venue: arXiv cs.HC 2024
Year: 2024

In recent years, more and more researchers have reflected on the undervaluation of emotion in data visualization and highlighted the importance of considering human emotion in visualization design. Meanwhile, an increasing number of studies have been conducted to explore emotion-related factors. However, so far, this research area is still in its early stages and faces a set of challenges, such as the unclear definition of key concepts, the insufficient justification of why emotion is important in visualization design, and the lack of characterization of the design space of affective visualization design. To address these challenges, first, we conducted a literature review and identified three research lines that examined both emotion and data visualization. We clarified the differences between these research lines and kept 109 papers that studied or discussed how data visualization communicates and influences emotion. Then, we coded the 109 papers in terms of how they justified the legitimacy of considering emotion in visualization design (i.e., why emotion is important) and identified five argumentative perspectives. Based on these papers, we also identified 61 projects that practiced affective visualization design. We coded these design projects in three dimensions, including design fields (where), design tasks (what), and design methods (how), to explore the design space of affective visualization design.