OpenRaiser/Intern-Atlas huggingface dataset:
Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluation
Venue: arXiv cs.HC 2024
Year: 2024

Evaluating the quality of automatically generated question items has been a long standing challenge. In this paper, we leverage LLMs to simulate student profiles and generate responses to multiple-choice questions (MCQs). The generative students' responses to MCQs can further support question item evaluation. We propose Generative Students, a prompt architecture designed based on the KLI framework. A generative student profile is a function of the list of knowledge components the student has mastered, has confusion about or has no evidence of knowledge of. We instantiate the Generative Students concept on the subject domain of heuristic evaluation. We created 45 generative students using GPT-4 and had them respond to 20 MCQs. We found that the generative students produced logical and believable responses that were aligned with their profiles. We then compared the generative students' responses to real students' responses on the same set of MCQs and found a high correlation. Moreover, there was considerable overlap in the difficult questions identified by generative students and real students. A subsequent case study demonstrated that an instructor could improve question quality based on the signals provided by Generative Students.
Bioinspired Nested-Isotropic Lattices with Tunable Anisotropy for Additive Manufacturing
Venue: arXiv cs.CE 2024
Year: 2024

This study presents innovative nested-isotropic lattices for additive manufacturing, drawing inspiration from bio-architectures found in cortical bone osteons, golden spirals, and fractals. These lattices provide tunable anisotropy by integrating architectural elements like ``nesting orders (NOs)'' and corresponding ``nesting orientations (NORs),'' along with repetitive self-similar X-cross struts and three four-fold axes of symmetry, resulting in a wide spectrum of lattice designs. Nine mono-nest and twenty multi-nest lattices, along with 252 parametric variations, are realized. The relative density \( \barρ \) and surface area density \( \bar{S} \) are calculated. Employing finite element-based numerical homogenization, elastic stiffness tensors are estimated to evaluate the anisotropic measure - Zener ratio \( Z \) and elastic modulus \( \bar{E} \) for all lattice designs. The mono-nest lattices generated considering higher NOs and respective NORs exhibit a transition from shear dominant to tensile/compression dominant (TCD) anisotropic behavior and their strut size variations show a strong influence on \( \barρ \), \( \bar{S} \), and \( \bar{E} \). In contrast, multi-nest lattices exhibit isotropic and neo-isotropic characteristics, with strut size mismatch exerting more influence on \( Z \). Increasing NOs and NORs result in isotropic or TCD behavior for most multi-nest lattices, with strut size mismatch leading to many isotropic lattices. These bio-inspired nested lattices, coupled with advancements in additive manufacturing, hold potential for diverse applications.
Searching Realistic-Looking Adversarial Objects For Autonomous Driving Systems
Venue: arXiv cs.CV 2024
Year: 2024

Numerous studies on adversarial attacks targeting self-driving policies fail to incorporate realistic-looking adversarial objects, limiting real-world applicability. Building upon prior research that facilitated the transition of adversarial objects from simulations to practical applications, this paper discusses a modified gradient-based texture optimization method to discover realistic-looking adversarial objects. While retaining the core architecture and techniques of the prior research, the proposed addition involves an entity termed the 'Judge'. This agent assesses the texture of a rendered object, assigning a probability score reflecting its realism. This score is integrated into the loss function to encourage the NeRF object renderer to concurrently learn realistic and adversarial textures. The paper analyzes four strategies for developing a robust 'Judge': 1) Leveraging cutting-edge vision-language models. 2) Fine-tuning open-sourced vision-language models. 3) Pretraining neurosymbolic systems. 4) Utilizing traditional image processing techniques. Our findings indicate that strategies 1) and 4) yield less reliable outcomes, pointing towards strategies 2) or 3) as more promising directions for future research.
Inquire, Interact, and Integrate: A Proactive Agent Collaborative Framework for Zero-Shot Multimodal Medical Reasoning
Venue: arXiv cs.AI 2024
Year: 2024

The adoption of large language models (LLMs) in healthcare has attracted significant research interest. However, their performance in healthcare remains under-investigated and potentially limited, due to i) they lack rich domain-specific knowledge and medical reasoning skills; and ii) most state-of-the-art LLMs are unimodal, text-only models that cannot directly process multimodal inputs. To this end, we propose a multimodal medical collaborative reasoning framework \textbf{MultiMedRes}, which incorporates a learner agent to proactively gain essential information from domain-specific expert models, to solve medical multimodal reasoning problems. Our method includes three steps: i) \textbf{Inquire}: The learner agent first decomposes given complex medical reasoning problems into multiple domain-specific sub-problems; ii) \textbf{Interact}: The agent then interacts with domain-specific expert models by repeating the ``ask-answer'' process to progressively obtain different domain-specific knowledge; iii) \textbf{Integrate}: The agent finally integrates all the acquired domain-specific knowledge to accurately address the medical reasoning problem. We validate the effectiveness of our method on the task of difference visual question answering for X-ray images. The experiments demonstrate that our zero-shot prediction achieves state-of-the-art performance, and even outperforms the fully supervised methods. Besides, our approach can be incorporated into various LLMs and multimodal LLMs to significantly boost their performance.
Cyber Risks of Machine Translation Critical Errors : Arabic Mental Health Tweets as a Case Study
Venue: arXiv cs.CL 2024
Year: 2024

With the advent of Neural Machine Translation (NMT) systems, the MT output has reached unprecedented accuracy levels which resulted in the ubiquity of MT tools on almost all online platforms with multilingual content. However, NMT systems, like other state-of-the-art AI generative systems, are prone to errors that are deemed machine hallucinations. The problem with NMT hallucinations is that they are remarkably \textit{fluent} hallucinations. Since they are trained to produce grammatically correct utterances, NMT systems are capable of producing mistranslations that are too fluent to be recognised by both users of the MT tool, as well as by automatic quality metrics that are used to gauge their performance. In this paper, we introduce an authentic dataset of machine translation critical errors to point to the ethical and safety issues involved in the common use of MT. The dataset comprises mistranslations of Arabic mental health postings manually annotated with critical error types. We also show how the commonly used quality metrics do not penalise critical errors and highlight this as a critical issue that merits further attention from researchers.
Deep Ensemble Art Style Recognition
Venue: arXiv cs.CV 2024
Year: 2024

The massive digitization of artworks during the last decades created the need for categorization, analysis, and management of huge amounts of data related to abstract concepts, highlighting a challenging problem in the field of computer science. The rapid progress of artificial intelligence and neural networks has provided tools and technologies that seem worthy of the challenge. Recognition of various art features in artworks has gained attention in the deep learning society. In this paper, we are concerned with the problem of art style recognition using deep networks. We compare the performance of 8 different deep architectures (VGG16, VGG19, ResNet50, ResNet152, Inception-V3, DenseNet121, DenseNet201 and Inception-ResNet-V2), on two different art datasets, including 3 architectures that have never been used on this task before, leading to state-of-the-art performance. We study the effect of data preprocessing prior to applying a deep learning model. We introduce a stacking ensemble method combining the results of first-stage classifiers through a meta-classifier, with the innovation of a versatile approach based on multiple models that extract and recognize different characteristics of the input, creating a more consistent model compared to existing works and achieving state-of-the-art accuracy on the largest art dataset available (WikiArt - 68,55%). We also discuss the impact of the data and art styles themselves on the performance of our models forming a manifold perspective on the problem.
Fixed-parameter tractability of canonical polyadic decomposition over finite fields
Venue: arXiv cs.CC 2024
Year: 2024

We present a simple proof that finding a rank-$R$ canonical polyadic decomposition of a 3-dimensional tensor over a finite field $\mathbb{F}$ is fixed-parameter tractable with respect to $R$ and $\mathbb{F}$. We also show a nontrivial upper bound on the time complexity of this problem.
Trust, Because You Can't Verify:Privacy and Security Hurdles in Education Technology Acquisition Practices
Venue: arXiv cs.CY 2024
Year: 2024

The education technology (EdTech) landscape is expanding rapidly in higher education institutes (HEIs). This growth brings enormous complexity. Protecting the extensive data collected by these tools is crucial for HEIs as data breaches and misuses can have dire security and privacy consequences on the data subjects, particularly students, who are often compelled to use these tools. This urges an in-depth understanding of HEI and EdTech vendor dynamics, which is largely understudied. To address this gap, we conducted a semi-structured interview study with 13 participants who are in EdTech leadership roles at seven HEIs. Our study uncovers the EdTech acquisition process in the HEI context, the consideration of security and privacy issues throughout that process, the pain points of HEI personnel in establishing adequate protection mechanisms in service contracts, and their struggle in holding vendors accountable due to a lack of visibility into their system and power-asymmetry, among other reasons. We discuss certain observations about the status quo and conclude with recommendations for HEIs, researchers, and regulatory bodies to improve the situation.
Generalized regenerating codes and node repair on graphs
Venue: arXiv cs.IT 2024
Year: 2024

We consider regenerating codes in distributed storage systems where connections between the nodes are constrained by a graph. In this problem, the failed node downloads the information stored at a subset of vertices of the graph for the purpose of recovering the lost data. Compared to the standard setting, regenerating codes on graphs address two additional features. The repair information is moved across the network, and the cost of node repair is determined by the graphical distance from the helper nodes to the failed node. Accordingly, the helpers far away from the failed node may be expected to contribute less data for repair than the nodes in the neighborhood of that node. We analyze regenerating codes with nonuniform download for repair on graphs. Moreover, in the process of repair, the information moved from the helpers to the failed node may be combined through intermediate processing, reducing the repair bandwidth. We derive lower bounds for communication complexity of node repair on graphs, including repair schemes with nonuniform download and intermediate processing, and construct codes that attain these bounds. Additionally, some of the nodes may act as adversaries, introducing errors into the data moved in the network. For repair on graphs in the presence of adversarial nodes, we construct codes that support node repair and error correction in systematic nodes.
Versatile Teacher: A Class-aware Teacher-student Framework for Cross-domain Adaptation
Venue: arXiv cs.CV 2024
Year: 2024

Addressing the challenge of domain shift between datasets is vital in maintaining model performance. In the context of cross-domain object detection, the teacher-student framework, a widely-used semi-supervised model, has shown significant accuracy improvements. However, existing methods often overlook class differences, treating all classes equally, resulting in suboptimal results. Furthermore, the integration of instance-level alignment with a one-stage detector, essential due to the absence of a Region Proposal Network (RPN), remains unexplored in this framework. In response to these shortcomings, we introduce a novel teacher-student model named Versatile Teacher (VT). VT differs from previous works by considering class-specific detection difficulty and employing a two-step pseudo-label selection mechanism, referred to as Class-aware Pseudo-label Adaptive Selection (CAPS), to generate more reliable pseudo labels. These labels are leveraged as saliency matrices to guide the discriminator for targeted instance-level alignment. Our method demonstrates promising results on three benchmark datasets, and extends the alignment methods for widely-used one-stage detectors, presenting significant potential for practical applications. Code is available at https://github.com/RicardooYoung/VersatileTeacher.
Fed-Credit: Robust Federated Learning with Credibility Management
Venue: arXiv cs.LG 2024
Year: 2024

Aiming at privacy preservation, Federated Learning (FL) is an emerging machine learning approach enabling model training on decentralized devices or data sources. The learning mechanism of FL relies on aggregating parameter updates from individual clients. However, this process may pose a potential security risk due to the presence of malicious devices. Existing solutions are either costly due to the use of compute-intensive technology, or restrictive for reasons of strong assumptions such as the prior knowledge of the number of attackers and how they attack. Few methods consider both privacy constraints and uncertain attack scenarios. In this paper, we propose a robust FL approach based on the credibility management scheme, called Fed-Credit. Unlike previous studies, our approach does not require prior knowledge of the nodes and the data distribution. It maintains and employs a credibility set, which weighs the historical clients' contributions based on the similarity between the local models and global model, to adjust the global model update. The subtlety of Fed-Credit is that the time decay and attitudinal value factor are incorporated into the dynamic adjustment of the reputation weights and it boasts a computational complexity of O(n) (n is the number of the clients). We conducted extensive experiments on the MNIST and CIFAR-10 datasets under 5 types of attacks. The results exhibit superior accuracy and resilience against adversarial attacks, all while maintaining comparatively low computational complexity. Among these, on the Non-IID CIFAR-10 dataset, our algorithm exhibited performance enhancements of 19.5% and 14.5%, respectively, in comparison to the state-of-the-art algorithm when dealing with two types of data poisoning attacks.
DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment
Venue: arXiv cs.CV 2024
Year: 2024

Object detectors frequently encounter significant performance degradation when confronted with domain gaps between collected data (source domain) and data from real-world applications (target domain). To address this task, numerous unsupervised domain adaptive detectors have been proposed, leveraging carefully designed feature alignment techniques. However, these techniques primarily align instance-level features in a class-agnostic manner, overlooking the differences between extracted features from different categories, which results in only limited improvement. Furthermore, the scope of current alignment modules is often restricted to a limited batch of images, failing to learn the entire dataset-level cues, thereby severely constraining the detector's generalization ability to the target domain. To this end, we introduce a strong DETR-based detector named Domain Adaptive detection TRansformer (DATR) for unsupervised domain adaptation of object detection. Firstly, we propose the Class-wise Prototypes Alignment (CPA) module, which effectively aligns cross-domain features in a class-aware manner by bridging the gap between object detection task and domain adaptation task. Then, the designed Dataset-level Alignment Scheme (DAS) explicitly guides the detector to achieve global representation and enhance inter-class distinguishability of instance-level features across the entire dataset, which spans both domains, by leveraging contrastive learning. Moreover, DATR incorporates a mean-teacher based self-training framework, utilizing pseudo-labels generated by the teacher model to further mitigate domain bias. Extensive experimental results demonstrate superior performance and generalization capabilities of our proposed DATR in multiple domain adaptation scenarios. Code is released at https://github.com/h751410234/DATR.
From SHAP Scores to Feature Importance Scores
Venue: arXiv cs.AI 2024
Year: 2024

A central goal of eXplainable Artificial Intelligence (XAI) is to assign relative importance to the features of a Machine Learning (ML) model given some prediction. The importance of this task of explainability by feature attribution is illustrated by the ubiquitous recent use of tools such as SHAP and LIME. Unfortunately, the exact computation of feature attributions, using the game-theoretical foundation underlying SHAP and LIME, can yield manifestly unsatisfactory results, that tantamount to reporting misleading relative feature importance. Recent work targeted rigorous feature attribution, by studying axiomatic aggregations of features based on logic-based definitions of explanations by feature selection. This paper shows that there is an essential relationship between feature attribution and a priori voting power, and that those recently proposed axiomatic aggregations represent a few instantiations of the range of power indices studied in the past. Furthermore, it remains unclear how some of the most widely used power indices might be exploited as feature importance scores (FISs), i.e. the use of power indices in XAI, and which of these indices would be the best suited for the purposes of XAI by feature attribution, namely in terms of not producing results that could be deemed as unsatisfactory. This paper proposes novel desirable properties that FISs should exhibit. In addition, the paper also proposes novel FISs exhibiting the proposed properties. Finally, the paper conducts a rigorous analysis of the best-known power indices in terms of the proposed properties.
On the Design and Study of an Installation for Office Workers to Amplify Temporal Diversity and Connection to Nature
Venue: arXiv cs.HC 2024
Year: 2024

We present the design and user study of an installation for office workers, enabling moments of temporal diversity and connection to nature. The installation is a form of creative computing experience that departs from the traditional focus on office technologies for productivity. Drawing on neuroscience insights and the slowing effect of nature sounds on time perception, we created an immersive, slow interaction, generative AI installation that composes an audiovisual space - serving as a perceptual portal into temporal realms beyond the linear rhythm of the office. Our study investigates the lived experiences of 18 office workers, gathered via explicitation interviews, observational notes, and video recordings, analysed through an inductive thematic analysis. Key findings highlight the ephemeral qualities in creative computing experiences using generative AI, its potential to foster contemplative practices, amplify ecological temporalities, and reshape office workers' engagement with their environment. Our design and user study offer research and practical implications for utilising creative computing to enrich office experiences.
The Grandparent Scam: A Systems Perspective Case Study On Elder Fraud And The Concept Of Human Layering
Venue: arXiv cs.CY 2024
Year: 2024

In April 2024, an 81-year-old Ohio man was charged with murder, assault, and kidnapping. The man believed that he was protecting his family from scammers threatening harm. What he did not realize was that the 61-year-old Uber driver he killed, was also a victim of the same scammers. This case study examines some common variants of the Grandparent Scam from a systems perspective and how weaponization of conscience is used in these scams. Additionally, this study examines the parallels between layering in money laundering and human layering in the execution of these scams.
Counterfactual Explanation-Based Badminton Motion Guidance Generation Using Wearable Sensors
Venue: arXiv cs.HC 2024
Year: 2024

This study proposes a framework for enhancing the stroke quality of badminton players by generating personalized motion guides, utilizing a multimodal wearable dataset. These guides are based on counterfactual algorithms and aim to reduce the performance gap between novice and expert players. Our approach provides joint-level guidance through visualizable data to assist players in improving their movements without requiring expert knowledge. The method was evaluated against a traditional algorithm using metrics to assess validity, proximity, and plausibility, including arithmetic measures and motion-specific evaluation metrics. Our evaluation demonstrates that the proposed framework can generate motions that maintain the essence of original movements while enhancing stroke quality, providing closer guidance than direct expert motion replication. The results highlight the potential of our approach for creating personalized sports motion guides by generating counterfactual motion guidance for arbitrary input motion samples of badminton strokes.
Dual-sided Peltier Elements for Rapid Thermal Feedback in Wearables
Venue: arXiv cs.HC 2024
Year: 2024

This paper introduces a motor-driven Peltier device designed to deliver immediate thermal sensations within extended reality (XR) environments. The system incorporates eight motor-driven Peltier elements, facilitating swift transitions between warm and cool sensations by rotating preheated or cooled elements to opposite sides. A multi-layer structure, comprising aluminum and silicone layers, ensures user comfort and safety while maintaining optimal temperatures for thermal stimuli. Time-temperature characteristic analysis demonstrates the system's ability to provide warm and cool sensations efficiently, with a dual-sided lifetime of up to 206 seconds at a 2V input. Our system design is adaptable to various body parts and can be synchronized with corresponding visual stimuli to enhance the immersive sensation of virtual object interaction and information delivery.