OpenRaiser/Intern-Atlas huggingface dataset:
Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
Venue: arXiv cs.CL 2024
Year: 2024

The quadratic computational complexity of the attention mechanism in current Large Language Models (LLMs) renders inference with long contexts prohibitively expensive. To address this challenge, various approaches aim to retain critical portions of the context to optimally approximate Full Attention (FA) through Key-Value (KV) compression or Sparse Attention (SA), enabling the processing of virtually unlimited text lengths in a streaming manner. However, these methods struggle to achieve performance levels comparable to FA, particularly in retrieval tasks. In this paper, our analysis of attention head patterns reveals that LLMs' attention distributions show strong local correlations, naturally reflecting a chunking mechanism for input context. We propose Ltri-LLM framework, which divides KVs into spans, stores them in an offline index, and retrieves the relevant KVs into memory for various queries. Experimental results on popular long text benchmarks show that Ltri-LLM can achieve performance close to FA while maintaining efficient, streaming-based inference.
Assessing Similarity Measures for the Evaluation of Human-Robot Motion Correspondence
Venue: arXiv cs.RO 2024
Year: 2024

One key area of research in Human-Robot Interaction is solving the human-robot correspondence problem, which asks how a robot can learn to reproduce a human motion demonstration when the human and robot have different dynamics and kinematic structures. Evaluating these correspondence problem solutions often requires the use of qualitative surveys that can be time consuming to design and administer. Additionally, qualitative survey results vary depending on the population of survey participants. In this paper, we propose the use of heterogeneous time-series similarity measures as a quantitative evaluation metric for evaluating motion correspondence to complement these qualitative surveys. To assess the suitability of these measures, we develop a behavioral cloning-based motion correspondence model, and evaluate it with a qualitative survey as well as quantitative measures. By comparing the resulting similarity scores with the human survey results, we identify Gromov Dynamic Time Warping as a promising quantitative measure for evaluating motion correspondence.
Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment
Venue: arXiv cs.RO 2024
Year: 2024

Visuomotor robot policies, increasingly pre-trained on large-scale datasets, promise significant advancements across robotics domains. However, aligning these policies with end-user preferences remains a challenge, particularly when the preferences are hard to specify. While reinforcement learning from human feedback (RLHF) has become the predominant mechanism for alignment in non-embodied domains like large language models, it has not seen the same success in aligning visuomotor policies due to the prohibitive amount of human feedback required to learn visual reward functions. To address this limitation, we propose Representation-Aligned Preference-based Learning (RAPL), an observation-only method for learning visual rewards from significantly less human preference feedback. Unlike traditional RLHF, RAPL focuses human feedback on fine-tuning pre-trained vision encoders to align with the end-user's visual representation and then constructs a dense visual reward via feature matching in this aligned representation space. We first validate RAPL through simulation experiments in the X-Magical benchmark and Franka Panda robotic manipulation, demonstrating that it can learn rewards aligned with human preferences, more efficiently uses preference data, and generalizes across robot embodiments. Finally, our hardware experiments align pre-trained Diffusion Policies for three object manipulation tasks. We find that RAPL can fine-tune these policies with 5x less real human preference data, taking the first step towards minimizing human feedback while maximizing visuomotor robot policy alignment.
Rethink Deep Learning with Invariance in Data Representation
Venue: arXiv cs.AI 2024
Year: 2024

Integrating invariance into data representations is a principled design in intelligent systems and web applications. Representations play a fundamental role, where systems and applications are both built on meaningful representations of digital inputs (rather than the raw data). In fact, the proper design/learning of such representations relies on priors w.r.t. the task of interest. Here, the concept of symmetry from the Erlangen Program may be the most fruitful prior -- informally, a symmetry of a system is a transformation that leaves a certain property of the system invariant. Symmetry priors are ubiquitous, e.g., translation as a symmetry of the object classification, where object category is invariant under translation. The quest for invariance is as old as pattern recognition and data mining itself. Invariant design has been the cornerstone of various representations in the era before deep learning, such as the SIFT. As we enter the early era of deep learning, the invariance principle is largely ignored and replaced by a data-driven paradigm, such as the CNN. However, this neglect did not last long before they encountered bottlenecks regarding robustness, interpretability, efficiency, and so on. The invariance principle has returned in the era of rethinking deep learning, forming a new field known as Geometric Deep Learning (GDL). In this tutorial, we will give a historical perspective of the invariance in data representations. More importantly, we will identify those research dilemmas, promising works, future directions, and web applications.
MANTA: A Large-Scale Multi-View and Visual-Text Anomaly Detection Dataset for Tiny Objects
Venue: arXiv cs.CV 2024
Year: 2024

We present MANTA, a visual-text anomaly detection dataset for tiny objects. The visual component comprises over 137.3K images across 38 object categories spanning five typical domains, of which 8.6K images are labeled as anomalous with pixel-level annotations. Each image is captured from five distinct viewpoints to ensure comprehensive object coverage. The text component consists of two subsets: Declarative Knowledge, including 875 words that describe common anomalies across various domains and specific categories, with detailed explanations for < what, why, how>, including causes and visual characteristics; and Constructivist Learning, providing 2K multiple-choice questions with varying levels of difficulty, each paired with images and corresponded answer explanations. We also propose a baseline for visual-text tasks and conduct extensive benchmarking experiments to evaluate advanced methods across different settings, highlighting the challenges and efficacy of our dataset.
Dynamic Interference Prediction for In-X 6G Sub-networks
Venue: arXiv cs.IT 2024
Year: 2024

The sixth generation (6G) industrial Sub-networks (SNs) face several challenges in meeting extreme latency and reliability requirements in the order of 0.1-1 ms and 99.999 -to-99.99999 percentile, respectively. Interference management (IM) plays an integral role in addressing these requirements, especially in ultra-dense SN environments with rapidly varying interference induced by channel characteristics, mobility, and resource limitations. In general, IM can be achieved using resource allocation and \textit{accurate} Link adaptation (LA). In this work, we focus on the latter, where we first model interference at SN devices using the spatially consistent 3GPP channel model. Following this, we present a discrete-time dynamic state space model (DSSM) at a SN access point (AP), where interference power values (IPVs) are modeled as latent variables incorporating underlying modeling errors as well as transmission/protocol delays. Necessary approximations are then presented to simplify the DSSM and to efficiently employ the extended Kalman filter (EKF) for interference prediction. Unlike baseline methods, our proposed approach predicts IPVs solely based on the channel quality indicator (CQI) reports available at the SN AP at every transmission time interval (TTI). Numerical results demonstrate that our proposed approach clearly outperforms the conventional baseline. Furthermore, we also show that despite predicting with limited information, our proposed approach consistently achieves a comparable performance w.r.t the off-the-shelf supervised learning based baseline.
On Process Awareness in Detecting Multi-stage Cyberattacks in Smart Grids
Venue: arXiv cs.CR 2024
Year: 2024

This study delves into the role of process awareness in enhancing intrusion detection within Smart Grids, considering the increasing fusion of ICT in power systems and the associated emerging threats. The research harnesses a co-simulation environment, encapsulating IT, OT, and ET layers, to model multi-stage cyberattacks and evaluate machine learning-based IDS strategies. The key observation is that process-aware IDS demonstrate superior detection capabilities, especially in scenarios closely tied to operational processes, as opposed to IT-only IDS. This improvement is notable in distinguishing complex cyber threats from regular IT activities. The findings underscore the significance of further developing sophisticated IDS benchmarks and digital twin datasets in Smart Grid environments, paving the way for more resilient cybersecurity infrastructures.
Probing the contents of semantic representations from text, behavior, and brain data using the psychNorms metabase
Venue: arXiv cs.CL 2024
Year: 2024

Semantic representations are integral to natural language processing, psycholinguistics, and artificial intelligence. Although often derived from internet text, recent years have seen a rise in the popularity of behavior-based (e.g., free associations) and brain-based (e.g., fMRI) representations, which promise improvements in our ability to measure and model human representations. We carry out the first systematic evaluation of the similarities and differences between semantic representations derived from text, behavior, and brain data. Using representational similarity analysis, we show that word vectors derived from behavior and brain data encode information that differs from their text-derived cousins. Furthermore, drawing on our psychNorms metabase, alongside an interpretability method that we call representational content analysis, we find that, in particular, behavior representations capture unique variance on certain affective, agentic, and socio-moral dimensions. We thus establish behavior as an important complement to text for capturing human representations and behavior. These results are broadly relevant to research aimed at learning human-aligned semantic representations, including work on evaluating and aligning large language models.
Preliminary Study on Virtual Reality Framework for Effective Prospective Memory Training: Integration of Visual Imagery and Daily-life Simulations
Venue: arXiv cs.HC 2024
Year: 2024

Prospective memory (PM), defining the currently conceived intention of a future action, is crucial for daily functioning, particularly in aging populations. This study develops and validates a virtual reality prospective memory training (VR-PMT) system that integrates visual imagery training (VIT) and virtual reality training (VRT) to enhance the PM abilities of users. The framework is designed to progressively challenge users by simulating real-life PM tasks in a controlled VR environment. The VIT component is designed to improve the generation and utilization of visual imagery by users, while the VRT component provides PM tasks based on time and event cues within a virtual environment.The framework was evaluated on ten healthy adults (university students and elderly participants) over nine weeks. During the initial session, the baseline PM abilities of the participants were assessed using the memory for intentions screening test (MIST). The subsequent sessions alternated between VIT and VRT with increasing task complexity. The MIST scores were significantly positively correlated with task achievement, confirming the efficacy of the system. Imagery abilities were also strongly correlated with task performance, underscoring the importance of visual imagery in PM training.Usability and user experiences, evaluated on the Jikaku-sho Shirabe questionnaire and the user experience questionnaire, indicated an overall positive user experience but higher fatigue levels in elderly participants. This study demonstrates that the VR--PMT system effectively trains and assesses PM abilities by integrating VIT and VRT, supporting its potential for broader applications in clinical settings.
Galois self-orthogonal MDS codes with large dimensions
Venue: arXiv cs.IT 2024
Year: 2024

Let $q=p^m$ be a prime power, $e$ be an integer with $0\leq e\leq m-1$ and $s=\gcd(e,m)$. In this paper, for a vector $v$ and a $q$-ary linear code $C$, we give some necessary and sufficient conditions for the equivalent code $vC$ of $C$ and the extended code of $vC$ to be $e$-Galois self-orthogonal. From this, we directly obtain some necessary and sufficient conditions for (extended) generalized Reed-Solomon (GRS and EGRS) codes to be $e$-Galois self-orthogonal. Furthermore, for all possible $e$ satisfying $0\leq e\leq m-1$, we classify them into three cases (1) $\frac{m}{s}$ odd and $p$ even; (2) $\frac{m}{s}$ odd and $p$ odd; (3) $\frac{m}{s}$ even, and construct several new classes of $e$-Galois self-orthogonal maximum distance separable (MDS) codes. It is worth noting that our $e$-Galois self-orthogonal MDS codes can have dimensions greater than $\lfloor \frac{n+p^e-1}{p^e+1}\rfloor$, which are not covered by previously known ones. Moreover, by propagation rules, we obtain some new MDS codes with Galois hulls of arbitrary dimensions. As an application, many quantum codes can be obtained from these MDS codes with Galois hulls.
Unifying Dual-Space Embedding for Entity Alignment via Contrastive Learning
Venue: arXiv cs.CL 2024
Year: 2024

Entity alignment aims to match identical entities across different knowledge graphs (KGs). Graph neural network-based entity alignment methods have achieved promising results in Euclidean space. However, KGs often contain complex structures, including both local and hierarchical ones, which make it challenging to efficiently represent them within a single space. In this paper, we proposed a novel method UniEA, which unifies dual-space embedding to preserve the intrinsic structure of KGs. Specifically, we learn graph structure embedding in both Euclidean and hyperbolic spaces simultaneously to maximize the consistency between the embedding in both spaces. Moreover, we employ contrastive learning to mitigate the misalignment issues caused by similar entities, where embedding of similar neighboring entities within the KG become too close in distance. Extensive experiments on benchmark datasets demonstrate that our method achieves state-of-the-art performance in structure-based EA. Our code is available at https://github.com/wonderCS1213/UniEA.
Level Up or Game Over: Exploring How Dark Patterns Shape Mobile Games
Venue: arXiv cs.HC 2024
Year: 2024

This study explores the prevalence of dark patterns in mobile games that exploit players through temporal, monetary, social, and psychological means. Recognizing the ethical concerns and potential harm surrounding these manipulative strategies, we analyze user-generated data of 1496 games to identify relationships between the deployment of dark patterns within "dark" and "healthy" games. Our findings reveal that dark patterns are not only widespread in games typically seen as problematic but are also present in games that may be perceived as benign. This research contributes needed quantitative support to the broader understanding of dark patterns in games. With an emphasis on ethical design, our study highlights current problems of revenue models that can be particularly harmful to vulnerable populations. To this end, we discuss the relevance of community-based approaches to surface harmful design and the necessity for collaboration among players/users and practitioners to promote healthier gaming experiences.
Trust and distrust in electoral technologies: what can we learn from the failure of electronic voting in the Netherlands (2006/07)
Venue: arXiv cs.CY 2024
Year: 2024

This paper focuses on the complex dynamics of trust and distrust in digital government technologies by approaching the cancellation of machine voting in the Netherlands (2006-07). This case describes how a previously trusted system can collapse, how paradoxical the relationship between trust and distrust is, and how it interacts with adopting and managing electoral technologies. The analysis stresses how, although being a central component, technology's trustworthiness dialogues with the socio-technical context in which it is inserted, for example, underscoring the relevance of public administration in securing technological environments. Beyond these insights, the research offers broader reflections on trust and distrust in data-driven technologies, advocating for differentiated strategies for building trust versus managing distrust. Overall, this paper contributes to understanding trust dynamics in digital government technologies, with implications for policymaking and technology adoption strategies.
From Defects to Demands: A Unified, Iterative, and Heuristically Guided LLM-Based Framework for Automated Software Repair and Requirement Realization
Venue: arXiv cs.SE 2024
Year: 2024

This manuscript signals a new era in the integration of artificial intelligence with software engineering, placing machines at the pinnacle of coding capability. We present a formalized, iterative methodology proving that AI can fully replace human programmers in all aspects of code creation and refinement. Our approach, combining large language models with formal verification, test-driven development, and incremental architectural guidance, achieves a 38.6% improvement over the current top performer's 48.33% accuracy on the SWE-bench benchmark. This surpasses previously assumed limits, signaling the end of human-exclusive coding and the rise of autonomous AI-driven software innovation. More than a technical advance, our work challenges centuries-old assumptions about human creativity. We provide robust evidence of AI superiority, demonstrating tangible gains in practical engineering contexts and laying the foundation for a future in which computational creativity outpaces human ingenuity.
Metamemory: Exploring the Resilience of Older Internal Migrants
Venue: arXiv cs.HC 2024
Year: 2024

Immigration and aging have always been significant topics of discussion in society, concerning the stability and future development of a country and its people. Research in the field of HCI on immigration and aging has primarily focused on their practical needs but has paid less attention to the adaptability issues of older internal migrants moving with their families. In this study, we investigate the challenges older internal migrants face in adapting socially, using metadata surveys and semi-structured interviews to delve into their life struggles and resilience sources. Our findings highlight the older internal migrants' remarkable resilience, particularly evident in their reminiscences. We explore the integration of reminiscences with the metaverse, identifying the necessary conditions to create a "Metamemory". We introduce a novel design for a metaverse scene that bridges past and present experiences. This aims to encourage discussions on enhancing older internal migrants' reminiscence, leveraging the metaverse's positive potential, and devising strategies to more effectively address older internal migrants' concerns in the future.
Applying Automatic Differentiation to Optimize Differential Microphone Array Designs
Venue: arXiv cs.SD 2024
Year: 2024

This paper introduces a novel methodology leveraging differentiable programming to design efficient, constrained adaptive non-uniform Linear Differential Microphone Arrays (LDMAs) with reduced implementation costs. Utilizing an automatic differentiation framework, we propose a differentiable convex approach that enables the adaptive design of a filter with a distortionless constraint in the desired sound direction, while also imposing constraints on microphone positioning to ensure consistent performance. This approach achieves the desired Directivity Factor (DF) over a wide frequency range and facilitates effective recovery of wide-band speech signals at lower implementation costs.
Can Large Language Models Serve as Effective Classifiers for Hierarchical Multi-Label Classification of Scientific Documents at Industrial Scale?
Venue: arXiv cs.AI 2024
Year: 2024

We address the task of hierarchical multi-label classification (HMC) of scientific documents at an industrial scale, where hundreds of thousands of documents must be classified across thousands of dynamic labels. The rapid growth of scientific publications necessitates scalable and efficient methods for classification, further complicated by the evolving nature of taxonomies--where new categories are introduced, existing ones are merged, and outdated ones are deprecated. Traditional machine learning approaches, which require costly retraining with each taxonomy update, become impractical due to the high overhead of labelled data collection and model adaptation. Large Language Models (LLMs) have demonstrated great potential in complex tasks such as multi-label classification. However, applying them to large and dynamic taxonomies presents unique challenges as the vast number of labels can exceed LLMs' input limits. In this paper, we present novel methods that combine the strengths of LLMs with dense retrieval techniques to overcome these challenges. Our approach avoids retraining by leveraging zero-shot HMC for real-time label assignment. We evaluate the effectiveness of our methods on SSRN, a large repository of preprints spanning multiple disciplines, and demonstrate significant improvements in both classification accuracy and cost-efficiency. By developing a tailored evaluation framework for dynamic taxonomies and publicly releasing our code, this research provides critical insights into applying LLMs for document classification, where the number of classes corresponds to the number of nodes in a large taxonomy, at an industrial scale.