OpenRaiser/Intern-Atlas huggingface dataset:
Context-aware Inductive Knowledge Graph Completion with Latent Type Constraints and Subgraph Reasoning
Venue: arXiv cs.AI 2024
Year: 2024

Inductive knowledge graph completion (KGC) aims to predict missing triples with unseen entities. Recent works focus on modeling reasoning paths between the head and tail entity as direct supporting evidence. However, these methods depend heavily on the existence and quality of reasoning paths, which limits their general applicability in different scenarios. In addition, we observe that latent type constraints and neighboring facts inherent in KGs are also vital in inferring missing triples. To effectively utilize all useful information in KGs, we introduce CATS, a novel context-aware inductive KGC solution. With sufficient guidance from proper prompts and supervised fine-tuning, CATS activates the strong semantic understanding and reasoning capabilities of large language models to assess the existence of query triples, which consist of two modules. First, the type-aware reasoning module evaluates whether the candidate entity matches the latent entity type as required by the query relation. Then, the subgraph reasoning module selects relevant reasoning paths and neighboring facts, and evaluates their correlation to the query triple. Experiment results on three widely used datasets demonstrate that CATS significantly outperforms state-of-the-art methods in 16 out of 18 transductive, inductive, and few-shot settings with an average absolute MRR improvement of 7.2%.
Optimizing Chain-of-Thought Reasoning: Tackling Arranging Bottleneck via Plan Augmentation
Venue: arXiv cs.CL 2024
Year: 2024

Multi-step reasoning ability of large language models is crucial in tasks such as math and tool utilization. Current researches predominantly focus on enhancing model performance in these multi-step reasoning tasks through fine-tuning with Chain-of-Thought (CoT) steps, yet these methods tend to be heuristic, without exploring nor resolving the bottleneck. In this study, we subdivide CoT reasoning into two parts: arranging and executing, and identify that the bottleneck of models mainly lies in arranging rather than executing. Based on this finding, we propose a plan-based training and reasoning method that guides models to generate arranging steps through abstract plans. We experiment on both math (GSM8k) and tool utilization (ToolBench) benchmarks. Results show that compared to fine-tuning directly with CoT data, our approach achieves a better performance on alleviating arranging bottleneck, particularly excelling in long-distance reasoning generalization.
Toroidal density-equalizing map for genus-one surfaces
Venue: arXiv cs.GR 2024
Year: 2024

Density-equalizing map is a shape deformation technique originally developed for cartogram creation and sociological data visualization on planar geographical maps. In recent years, there has been an increasing interest in developing density-equalizing mapping methods for surface and volumetric domains and applying them to various problems in geometry processing and imaging science. However, the existing surface density-equalizing mapping methods are only applicable to surfaces with relatively simple topologies but not surfaces with topological holes. In this work, we develop a novel algorithm for computing density-equalizing maps for toroidal surfaces. In particular, different shape deformation effects can be easily achieved by prescribing different population functions on the torus and performing diffusion-based deformations on a planar domain with periodic boundary conditions. Furthermore, the proposed toroidal density-equalizing mapping method naturally leads to an effective method for computing toroidal parameterizations of genus-one surfaces with controllable shape changes, with the toroidal area-preserving parameterization being a prime example. Experimental results are presented to demonstrate the effectiveness of our proposed methods.
Tracing the Development of the Virtual Particle Concept Using Semantic Change Detection
Venue: arXiv cs.CL 2024
Year: 2024

Virtual particles are peculiar objects. They figure prominently in much of theoretical and experimental research in elementary particle physics. But exactly what they are is far from obvious. In particular, to what extent they should be considered "real" remains a matter of controversy in philosophy of science. Also their origin and development has only recently come into focus of scholarship in the history of science. In this study, we propose using the intriguing case of virtual particles to discuss the efficacy of Semantic Change Detection (SCD) based on contextualized word embeddings from a domain-adapted BERT model in studying specific scientific concepts. We find that the SCD metrics align well with qualitative research insights in the history and philosophy of science, as well as with the results obtained from Dependency Parsing to determine the frequency and connotations of the term "virtual." Still, the metrics of SCD provide additional insights over and above the qualitative research and the Dependency Parsing. Among other things, the metrics suggest that the concept of the virtual particle became more stable after 1950 but at the same time also more polysemous.
FlightAR: AR Flight Assistance Interface with Multiple Video Streams and Object Detection Aimed at Immersive Drone Control
Venue: arXiv cs.RO 2024
Year: 2024

The swift advancement of unmanned aerial vehicle (UAV) technologies necessitates new standards for developing human-drone interaction (HDI) interfaces. Most interfaces for HDI, especially first-person view (FPV) goggles, limit the operator's ability to obtain information from the environment. This paper presents a novel interface, FlightAR, that integrates augmented reality (AR) overlays of UAV first-person view (FPV) and bottom camera feeds with head-mounted display (HMD) to enhance the pilot's situational awareness. Using FlightAR, the system provides pilots not only with a video stream from several UAV cameras simultaneously, but also the ability to observe their surroundings in real time. User evaluation with NASA-TLX and UEQ surveys showed low physical demand ($μ=1.8$, $SD = 0.8$) and good performance ($μ=3.4$, $SD = 0.8$), proving better user assessments in comparison with baseline FPV goggles. Participants also rated the system highly for stimulation ($μ=2.35$, $SD = 0.9$), novelty ($μ=2.1$, $SD = 0.9$) and attractiveness ($μ=1.97$, $SD = 1$), indicating positive user experiences. These results demonstrate the potential of the system to improve UAV piloting experience through enhanced situational awareness and intuitive control. The code is available here: https://github.com/Sautenich/FlightAR
Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In
Venue: arXiv cs.CR 2024
Year: 2024

Following the advancement of large language models (LLMs), the development of LLM-based autonomous agents has become increasingly prevalent. As a result, the need to understand the security vulnerabilities of these agents has become a critical task. We examine how ReAct agents can be exploited using a straightforward yet effective method we refer to as the foot-in-the-door attack. Our experiments show that indirect prompt injection attacks, prompted by harmless and unrelated requests (such as basic calculations) can significantly increase the likelihood of the agent performing subsequent malicious actions. Our results show that once a ReAct agents thought includes a specific tool or action, the likelihood of executing this tool in the subsequent steps increases significantly, as the agent seldom re-evaluates its actions. Consequently, even random, harmless requests can establish a foot-in-the-door, allowing an attacker to embed malicious instructions into the agents thought process, making it more susceptible to harmful directives. To mitigate this vulnerability, we propose implementing a simple reflection mechanism that prompts the agent to reassess the safety of its actions during execution, which can help reduce the success of such attacks.
Evaluation of a Data Annotation Platform for Large, Time-Series Datasets in Intensive Care: Mixed Methods Study
Venue: arXiv cs.HC 2024
Year: 2024

Intensive Care Units are complex, data-rich environments where critically ill patients are treated using variety of clinical equipment. The data collected using this equipment can be used clinical staff to gain insight into the condition of the patients and provide adequate treatment, but it also provides ample opportunity for applications in machine learning and data science. While this data can frequently be used directly, complex problems may require additional annotations to provide context and meaning before it could be used to train the machine learning models. Annotating time-series datasets in clinical setting is a complex problem due to a large volume and complexity of the data, time-consuming nature of the process and the fact that clinicians' time is in both high demand and short supply. In this study, we present an evaluation of a bespoke tool designed to annotate large, clinical time-series datasets with staff from intensive care units. The software incorporates two modes for annotation: by annotating individual admissions and by generating rulesets which are applied to the entire dataset. Our study was split into two stages focusing on individual and semi-automated annotation and included 28 annotators across both stages who utilised 50 clinical parameters to guide their annotations. We experienced significant challenges in recruitment and engagement of the participants in the annotation activities and developed interventions which improved the participation over the course of the study. During the semi-automated annotation, we observed preferences for different parameter types (measured vs. observed), as well as relative agreement of participants across shared admissions to the decision-tree model trained using their rulesets.
IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing
Venue: arXiv cs.CL 2024
Year: 2024

Unlike professional Business-to-Consumer (B2C) e-commerce platforms (e.g., Amazon), Consumer-to-Consumer (C2C) platforms (e.g., Facebook marketplace) are mainly targeting individual sellers who usually lack sufficient experience in e-commerce. Individual sellers often struggle to compose proper descriptions for selling products. With the recent advancement of Multimodal Large Language Models (MLLMs), we attempt to integrate such state-of-the-art generative AI technologies into the product listing process. To this end, we develop IPL, an Intelligent Product Listing tool tailored to generate descriptions using various product attributes such as category, brand, color, condition, etc. IPL enables users to compose product descriptions by merely uploading photos of the selling product. More importantly, it can imitate the content style of our C2C platform Xianyu. This is achieved by employing domain-specific instruction tuning on MLLMs and adopting the multi-modal Retrieval-Augmented Generation (RAG) process. A comprehensive empirical evaluation demonstrates that the underlying model of IPL significantly outperforms the base model in domain-specific tasks while producing less hallucination. IPL has been successfully deployed in our production system, where 72% of users have their published product listings based on the generated content, and those product listings are shown to have a quality score 5.6% higher than those without AI assistance.
DIRI: Adversarial Patient Reidentification with Large Language Models for Evaluating Clinical Text Anonymization
Venue: arXiv cs.CL 2024
Year: 2024

Sharing protected health information (PHI) is critical for furthering biomedical research. Before data can be distributed, practitioners often perform deidentification to remove any PHI contained in the text. Contemporary deidentification methods are evaluated on highly saturated datasets (tools achieve near-perfect accuracy) which may not reflect the full variability or complexity of real-world clinical text and annotating them is resource intensive, which is a barrier to real-world applications. To address this gap, we developed an adversarial approach using a large language model (LLM) to re-identify the patient corresponding to a redacted clinical note and evaluated the performance with a novel De-Identification/Re-Identification (DIRI) method. Our method uses a large language model to reidentify the patient corresponding to a redacted clinical note. We demonstrate our method on medical data from Weill Cornell Medicine anonymized with three deidentification tools: rule-based Philter and two deep-learning-based models, BiLSTM-CRF and ClinicalBERT. Although ClinicalBERT was the most effective, masking all identified PII, our tool still reidentified 9% of clinical notes Our study highlights significant weaknesses in current deidentification technologies while providing a tool for iterative development and improvement.
UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs
Venue: arXiv cs.LG 2024
Year: 2024

The key components of machine learning are data samples for training, model for learning patterns, and loss function for optimizing accuracy. Analogously, unlearning can potentially be achieved through anti-data samples (or anti-samples), unlearning method, and reversed loss function. While prior research has explored unlearning methods and reversed loss functions, the potential of anti-samples remains largely untapped. In this paper, we introduce UnSTAR: Unlearning with Self-Taught Anti-Sample Reasoning for large language models (LLMs). Our contributions are threefold; first, we propose a novel concept of anti-sample-induced unlearning; second, we generate anti-samples by leveraging misleading rationales, which help reverse learned associations and accelerate the unlearning process; and third, we enable fine-grained targeted unlearning, allowing for the selective removal of specific associations without impacting related knowledge - something not achievable by previous works. Results demonstrate that anti-samples offer an efficient, targeted unlearning strategy for LLMs, opening new avenues for privacy-preserving machine learning and model modification.
Personalized Playback Technology: How Short Video Services Create Excellent User Experience
Venue: arXiv cs.MM 2024
Year: 2024

Short-form video content has become increasingly popular and influential in recent years. Its concise yet engaging format aligns well with todays' fast-paced and on-the-go lifestyles, making it a dominating trend in the digital world. As one of the front runners in the short video platform space, ByteDance has been highly successful in delivering a one-of-a-kind short video experience and attracting billions of users worldwide. One key contributing factor is its advanced end-to-end personalized short video playback technology, where we pioneered and developed the new technical field over the past five years to optimize user experience. This paper introduces the major concepts and methodologies of this personalized video playback technology that distinguish it from traditional multimedia technologies. More details, including goal setting, iterative process, modeling, experimental methods and required supporting systems, are also provided to encourage deeper research in this area.
Human-LLM Hybrid Text Answer Aggregation for Crowd Annotations
Venue: arXiv cs.CL 2024
Year: 2024

The quality is a crucial issue for crowd annotations. Answer aggregation is an important type of solution. The aggregated answers estimated from multiple crowd answers to the same instance are the eventually collected annotations, rather than the individual crowd answers themselves. Recently, the capability of Large Language Models (LLMs) on data annotation tasks has attracted interest from researchers. Most of the existing studies mainly focus on the average performance of individual crowd workers; several recent works studied the scenarios of aggregation on categorical labels and LLMs used as label creators. However, the scenario of aggregation on text answers and the role of LLMs as aggregators are not yet well-studied. In this paper, we investigate the capability of LLMs as aggregators in the scenario of close-ended crowd text answer aggregation. We propose a human-LLM hybrid text answer aggregation method with a Creator-Aggregator Multi-Stage (CAMS) crowdsourcing framework. We make the experiments based on public crowdsourcing datasets. The results show the effectiveness of our approach based on the collaboration of crowd workers and LLMs.
Permutation Picture of Graph Combinatorial Optimization Problems
Venue: arXiv cs.AI 2024
Year: 2024

This paper proposes a framework that formulates a wide range of graph combinatorial optimization problems using permutation-based representations. These problems include the travelling salesman problem, maximum independent set, maximum cut, and various other related problems. This work potentially opens up new avenues for algorithm design in neural combinatorial optimization, bridging the gap between discrete and continuous optimization techniques.
KANICE: Kolmogorov-Arnold Networks with Interactive Convolutional Elements
Venue: arXiv cs.CV 2024
Year: 2024

We introduce KANICE (Kolmogorov-Arnold Networks with Interactive Convolutional Elements), a novel neural architecture that combines Convolutional Neural Networks (CNNs) with Kolmogorov-Arnold Network (KAN) principles. KANICE integrates Interactive Convolutional Blocks (ICBs) and KAN linear layers into a CNN framework. This leverages KANs' universal approximation capabilities and ICBs' adaptive feature learning. KANICE captures complex, non-linear data relationships while enabling dynamic, context-dependent feature extraction based on the Kolmogorov-Arnold representation theorem. We evaluated KANICE on four datasets: MNIST, Fashion-MNIST, EMNIST, and SVHN, comparing it against standard CNNs, CNN-KAN hybrids, and ICB variants. KANICE consistently outperformed baseline models, achieving 99.35% accuracy on MNIST and 90.05% on the SVHN dataset. Furthermore, we introduce KANICE-mini, a compact variant designed for efficiency. A comprehensive ablation study demonstrates that KANICE-mini achieves comparable performance to KANICE with significantly fewer parameters. KANICE-mini reached 90.00% accuracy on SVHN with 2,337,828 parameters, compared to KANICE's 25,432,000. This study highlights the potential of KAN-based architectures in balancing performance and computational efficiency in image classification tasks. Our work contributes to research in adaptive neural networks, integrates mathematical theorems into deep learning architectures, and explores the trade-offs between model complexity and performance, advancing computer vision and pattern recognition. The source code for this paper is publicly accessible through our GitHub repository (https://github.com/m-ferdaus/kanice).
Faster Approximation Algorithms for Restricted Shortest Paths in Directed Graphs
Venue: arXiv cs.DS 2024
Year: 2024

In the restricted shortest paths problem, we are given a graph $G$ whose edges are assigned two non-negative weights: lengths and delays, a source $s$, and a delay threshold $D$. The goal is to find, for each target $t$, the length of the shortest $(s,t)$-path whose total delay is at most $D$. While this problem is known to be NP-hard [Garey and Johnson, 1979] $(1+\varepsilon)$-approximate algorithms running in $\tilde{O}(mn)$ time [Goel et al., INFOCOM'01; Lorenz and Raz, Oper. Res. Lett.'01] given more than twenty years ago have remained the state-of-the-art for directed graphs. An open problem posed by [Bernstein, SODA'12] -- who gave a randomized $m\cdot n^{o(1)}$ time bicriteria $(1+\varepsilon, 1+\varepsilon)$-approximation algorithm for undirected graphs -- asks if there is similarly an $o(mn)$ time approximation scheme for directed graphs. We show two randomized bicriteria $(1+\varepsilon, 1+\varepsilon)$-approximation algorithms that give an affirmative answer to the problem: one suited to dense graphs, and the other that works better for sparse graphs. On directed graphs with a quasi-polynomial weights aspect ratio, our algorithms run in time $\tilde{O}(n^2)$ and $\tilde{O}(mn^{3/5})$ or better, respectively. More specifically, the algorithm for sparse digraphs runs in time $\tilde{O}(mn^{(3 - α)/5})$ for graphs with $n^{1 + α}$ edges for any real $α\in [0,1/2]$.
DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planning
Venue: arXiv cs.RO 2024
Year: 2024

Informative path planning (IPP) is an important planning paradigm for various real-world robotic applications such as environment monitoring. IPP involves planning a path that can learn an accurate belief of the quantity of interest, while adhering to planning constraints. Traditional IPP methods typically require high computation time during execution, giving rise to reinforcement learning (RL) based IPP methods. However, the existing RL-based methods do not consider spatio-temporal environments which involve their own challenges due to variations in environment characteristics. In this paper, we propose DyPNIPP, a robust RL-based IPP framework, designed to operate effectively across spatio-temporal environments with varying dynamics. To achieve this, DyPNIPP incorporates domain randomization to train the agent across diverse environments and introduces a dynamics prediction model to capture and adapt the agent actions to specific environment dynamics. Our extensive experiments in a wildfire environment demonstrate that DyPNIPP outperforms existing RL-based IPP algorithms by significantly improving robustness and performing across diverse environment conditions.
VoiceBench: Benchmarking LLM-Based Voice Assistants
Venue: arXiv cs.CL 2024
Year: 2024

Building on the success of large language models (LLMs), recent advancements such as GPT-4o have enabled real-time speech interactions through LLM-based voice assistants, offering a significantly improved user experience compared to traditional text-based interactions. However, the absence of benchmarks designed to evaluate these speech interaction capabilities has hindered progress of LLM-based voice assistants development. Current evaluations focus primarily on automatic speech recognition (ASR) or general knowledge evaluation with clean speeches, neglecting the more intricate, real-world scenarios that involve diverse speaker characteristics, environmental and content factors. To address this, we introduce VoiceBench, the first benchmark designed to provide a multi-faceted evaluation of LLM-based voice assistants. VoiceBench also includes both real and synthetic spoken instructions that incorporate the above three key real-world variations. Extensive experiments reveal the limitations of current LLM-based voice assistant models and offer valuable insights for future research and development in this field.
On Recurrence Relations of Multi-dimensional Sequences
Venue: arXiv cs.SC 2024
Year: 2024

In this paper, we present a new algorithm for computing the linear recurrence relations of multi-dimensional sequences. Existing algorithms for computing these relations arise in computational algebra and include constructing structured matrices and computing their kernels. The challenging problem is to reduce the size of the corresponding matrices. In this paper, we show how to convert the problem of computing recurrence relations of multi-dimensional sequences into computing the orthogonal of certain ideals as subvector spaces of the dual module of polynomials. We propose an algorithm using efficient dual module computation algorithms. We present a complexity bound for this algorithm, carry on experiments using Maple implementation, and discuss the cases when using this algorithm is much faster than the existing approaches.