OpenRaiser/Intern-Atlas huggingface dataset:
Aegis:An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering
Venue: arXiv cs.MA 2024
Year: 2024

Functional safety is a critical aspect of automotive engineering, encompassing all phases of a vehicle's lifecycle, including design, development, production, operation, and decommissioning. This domain involves highly knowledge-intensive tasks. This paper introduces Aegis: An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering. Aegis is specifically designed to support complex functional safety tasks within the automotive sector. It is tailored to perform Hazard Analysis and Risk Assessment(HARA), document Functional Safety Requirements(FSR), and plan test cases for Automatic Emergency Braking(AEB) systems. The most advanced version, Aegis-Max, leverages Retrieval-Augmented Generation(RAG) and reflective mechanisms to enhance its capability in managing complex, knowledge-intensive tasks. Additionally, targeted prompt refinement by professional functional safety practitioners can significantly optimize Aegis's performance in the functional safety domain. This paper demonstrates the potential of Aegis to improve the efficiency and effectiveness of functional safety processes in automotive engineering.
Data-Driven Gyroscope Calibration
Venue: arXiv cs.LG 2024
Year: 2024

Gyroscopes are inertial sensors that measure the angular velocity of the platforms to which they are attached. To estimate the gyroscope deterministic error terms prior mission start, a calibration procedure is performed. When considering low-cost gyroscopes, the calibration requires a turntable as the gyros are incapable of sensing the Earth turn rate. In this paper, we propose a data-driven framework to estimate the scale factor and bias of a gyroscope. To train and validate our approach, a dataset of 56 minutes was recorded using a turntable. We demonstrated that our proposed approach outperforms the model-based approach, in terms of accuracy and convergence time. Specifically, we improved the scale factor and bias estimation by an average of 72% during six seconds of calibration time, demonstrating an average of 75% calibration time improvement. That is, instead of minutes, our approach requires only several seconds for the calibration.
Disentangling data distribution for Federated Learning
Venue: arXiv cs.DC 2024
Year: 2024

Federated Learning (FL) facilitates collaborative training of a global model whose performance is boosted by private data owned by distributed clients, without compromising data privacy. Yet the wide applicability of FL is hindered by entanglement of data distributions across different clients. This paper demonstrates for the first time that by disentangling data distributions FL can in principle achieve efficiencies comparable to those of distributed systems, requiring only one round of communication. To this end, we propose a novel FedDistr algorithm, which employs stable diffusion models to decouple and recover data distributions. Empirical results on the CIFAR100 and DomainNet datasets show that FedDistr significantly enhances model utility and efficiency in both disentangled and near-disentangled scenarios while ensuring privacy, outperforming traditional federated learning methods.
A Claim Decomposition Benchmark for Long-form Answer Verification
Venue: arXiv cs.CL 2024
Year: 2024

The advancement of LLMs has significantly boosted the performance of complex long-form question answering tasks. However, one prominent issue of LLMs is the generated "hallucination" responses that are not factual. Consequently, attribution for each claim in responses becomes a common solution to improve the factuality and verifiability. Existing researches mainly focus on how to provide accurate citations for the response, which largely overlook the importance of identifying the claims or statements for each response. To bridge this gap, we introduce a new claim decomposition benchmark, which requires building system that can identify atomic and checkworthy claims for LLM responses. Specifically, we present the Chinese Atomic Claim Decomposition Dataset (CACDD), which builds on the WebCPM dataset with additional expert annotations to ensure high data quality. The CACDD encompasses a collection of 500 human-annotated question-answer pairs, including a total of 4956 atomic claims. We further propose a new pipeline for human annotation and describe the challenges of this task. In addition, we provide experiment results on zero-shot, few-shot and fine-tuned LLMs as baselines. The results show that the claim decomposition is highly challenging and requires further explorations. All code and data are publicly available at \url{https://github.com/FBzzh/CACDD}.
Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving
Venue: arXiv cs.RO 2024
Year: 2024

The integration of Large Language Models (LLMs) into autonomous driving systems demonstrates strong common sense and reasoning abilities, effectively addressing the pitfalls of purely data-driven methods. Current LLM-based agents require lengthy inference times and face challenges in interacting with real-time autonomous driving environments. A key open question is whether we can effectively leverage the knowledge from LLMs to train an efficient and robust Reinforcement Learning (RL) agent. This paper introduces RAPID, a novel \underline{\textbf{R}}obust \underline{\textbf{A}}daptive \underline{\textbf{P}}olicy \underline{\textbf{I}}nfusion and \underline{\textbf{D}}istillation framework, which trains specialized mix-of-policy RL agents using data synthesized by an LLM-based driving agent and online adaptation. RAPID features three key designs: 1) utilization of offline data collected from an LLM agent to distil expert knowledge into RL policies for faster real-time inference; 2) introduction of robust distillation in RL to inherit both performance and robustness from LLM-based teacher; and 3) employment of a mix-of-policy approach for joint decision decoding with a policy adapter. Through fine-tuning via online environment interaction, RAPID reduces the forgetting of LLM knowledge while maintaining adaptability to different tasks. Extensive experiments demonstrate RAPID's capability to effectively integrate LLM knowledge into scaled-down RL policies in an efficient, adaptable, and robust way. Code and checkpoints will be made publicly available upon acceptance.
STRUX: An LLM for Decision-Making with Structured Explanations
Venue: arXiv cs.CL 2024
Year: 2024

Countless decisions shape our daily lives, and it is paramount to understand the how and why behind these choices. In this paper, we introduce a new LLM decision-making framework called STRUX, which enhances LLM decision-making by providing structured explanations. These include favorable and adverse facts related to the decision, along with their respective strengths. STRUX begins by distilling lengthy information into a concise table of key facts. It then employs a series of self-reflection steps to determine which of these facts are pivotal, categorizing them as either favorable or adverse in relation to a specific decision. Lastly, we fine-tune an LLM to identify and prioritize these key facts to optimize decision-making. STRUX has been evaluated on the challenging task of forecasting stock investment decisions based on earnings call transcripts and demonstrated superior performance against strong baselines. It enhances decision transparency by allowing users to understand the impact of different factors, representing a meaningful step towards practical decision-making with LLMs.
CMAL: A Novel Cross-Modal Associative Learning Framework for Vision-Language Pre-Training
Venue: arXiv cs.CV 2024
Year: 2024

With the flourishing of social media platforms, vision-language pre-training (VLP) recently has received great attention and many remarkable progresses have been achieved. The success of VLP largely benefits from the information complementation and enhancement between different modalities. However, most of recent studies focus on cross-modal contrastive learning (CMCL) to promote image-text alignment by pulling embeddings of positive sample pairs together while pushing those of negative pairs apart, which ignores the natural asymmetry property between different modalities and requires large-scale image-text corpus to achieve arduous progress. To mitigate this predicament, we propose CMAL, a Cross-Modal Associative Learning framework with anchor points detection and cross-modal associative learning for VLP. Specifically, we first respectively embed visual objects and textual tokens into separate hypersphere spaces to learn intra-modal hidden features, and then design a cross-modal associative prompt layer to perform anchor point masking and swap feature filling for constructing a hybrid cross-modal associative prompt. Afterwards, we exploit a unified semantic encoder to learn their cross-modal interactive features for context adaptation. Finally, we design an associative mapping classification layer to learn potential associative mappings between modalities at anchor points, within which we develop a fresh self-supervised associative mapping classification task to boost CMAL's performance. Experimental results verify the effectiveness of CMAL, showing that it achieves competitive performance against previous CMCL-based methods on four common downstream vision-and-language tasks, with significantly fewer corpus. Especially, CMAL obtains new state-of-the-art results on SNLI-VE and REC (testA).
Personalized Prediction Models for Changes in Knee Pain among Patients with Osteoarthritis Participating in Supervised Exercise and Education
Venue: arXiv cs.LG 2024
Year: 2024

Knee osteoarthritis (OA) is a widespread chronic condition that impairs mobility and diminishes quality of life. Despite the proven benefits of exercise therapy and patient education in managing the OA symptoms pain and functional limitations, these strategies are often underutilized. Personalized outcome prediction models can help motivate and engage patients, but the accuracy of existing models in predicting changes in knee pain remains insufficiently examined. To validate existing models and introduce a concise personalized model predicting changes in knee pain before to after participating in a supervised education and exercise therapy program (GLA:D) for knee OA patients. Our models use self-reported patient information and functional measures. To refine the number of variables, we evaluated the variable importance and applied clinical reasoning. We trained random forest regression models and compared the rate of true predictions of our models with those utilizing average values. We evaluated the performance of a full, continuous, and concise model including all 34, all 11 continuous, and the six most predictive variables respectively. All three models performed similarly and were comparable to the existing model, with R-squares of 0.31-0.32 and RMSEs of 18.65-18.85 - despite our increased sample size. Allowing a deviation of 15 VAS points from the true change in pain, our concise model and utilizing the average values estimated the change in pain at 58% and 51% correctly, respectively. Our supplementary analysis led to similar outcomes. Our concise personalized prediction model more accurately predicts changes in knee pain following the GLA:D program compared to average pain improvement values. Neither the increase in sample size nor the inclusion of additional variables improved previous models. To improve predictions, new variables beyond those in the GLA:D are required.
The Bayesian Confidence (BACON) Estimator for Deep Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

This paper introduces the Bayesian Confidence Estimator (BACON) for deep neural networks. Current practice of interpreting Softmax values in the output layer as probabilities of outcomes is prone to extreme predictions of class probability. In this work we extend Waagen's method of representing the terminal layers with a geometric model, where the probability associated with an output vector is estimated with Bayes' Rule using validation data to provide likelihood and normalization values. This estimator provides superior ECE and ACE calibration error compared to Softmax for ResNet-18 at 85% network accuracy, and EfficientNet-B0 at 95% network accuracy, on the CIFAR-10 dataset with an imbalanced test set, except for very high accuracy edge cases. In addition, when using the ACE metric, BACON demonstrated improved calibration error when estimating probabilities for the imbalanced test set when using actual class distribution fractions.
Advancing Web Browser Forensics: Critical Evaluation of Emerging Tools and Techniques
Venue: arXiv cs.CR 2024
Year: 2024

As the use of web browsers continues to grow, the potential for cybercrime and web-related criminal activities also increases. Digital forensic investigators must understand how different browsers function and the critical areas to consider during web forensic analysis. Web forensics, a subfield of digital forensics, involves collecting and analyzing browser artifacts, such as browser history, search keywords, and downloads, which serve as potential evidence. While existing research has provided valuable insights, many studies focus on individual browsing modes or limited forensic scenarios, leaving gaps in understanding the full scope of data retention and recovery across different modes and browsers. This paper addresses these gaps by defining four browsing scenarios and critically analyzing browser artifacts across normal, private, and portable modes using various forensic tools. We define four browsing scenarios to perform a comprehensive evaluation of popular browsers -- Google Chrome, Mozilla Firefox, Brave, Tor, and Microsoft Edge -- by monitoring changes in key data storage areas such as cache files, cookies, browsing history, and local storage across different browsing modes. Overall, this paper contributes to a deeper understanding of browser forensic analysis and identifies key areas for enhancing privacy protection and forensic methodologies.
From Measurement Instruments to Data: Leveraging Theory-Driven Synthetic Training Data for Classifying Social Constructs
Venue: arXiv cs.CL 2024
Year: 2024

Computational text classification is a challenging task, especially for multi-dimensional social constructs. Recently, there has been increasing discussion that synthetic training data could enhance classification by offering examples of how these constructs are represented in texts. In this paper, we systematically examine the potential of theory-driven synthetic training data for improving the measurement of social constructs. In particular, we explore how researchers can transfer established knowledge from measurement instruments in the social sciences, such as survey scales or annotation codebooks, into theory-driven generation of synthetic data. Using two studies on measuring sexism and political topics, we assess the added value of synthetic training data for fine-tuning text classification models. Although the results of the sexism study were less promising, our findings demonstrate that synthetic data can be highly effective in reducing the need for labeled data in political topic classification. With only a minimal drop in performance, synthetic data allows for substituting large amounts of labeled data. Furthermore, theory-driven synthetic data performed markedly better than data generated without conceptual information in mind.
An Exact Finite-dimensional Explicit Feature Map for Kernel Functions
Venue: arXiv cs.LG 2024
Year: 2024

Kernel methods in machine learning use a kernel function that takes two data points as input and returns their inner product after mapping them to a Hilbert space, implicitly and without actually computing the mapping. For many kernel functions, such as Gaussian and Laplacian kernels, the feature space is known to be infinite-dimensional, making operations in this space possible only implicitly. This implicit nature necessitates algorithms to be expressed using dual representations and the kernel trick. In this paper, given an arbitrary kernel function, we introduce an explicit, finite-dimensional feature map for any arbitrary kernel function that ensures the inner product of data points in the feature space equals the kernel function value, during both training and testing. The existence of this explicit mapping allows for kernelized algorithms to be formulated in their primal form, without the need for the kernel trick or the dual representation. As a first application, we demonstrate how to derive kernelized machine learning algorithms directly, without resorting to the dual representation, and apply this method specifically to PCA. As another application, without any changes to the t-SNE algorithm and its implementation, we use it for visualizing the feature space of kernel functions.
Non-Conservative Obstacle Avoidance for Multi-Body Systems Leveraging Convex Hulls and Predicted Closest Points
Venue: arXiv cs.RO 2024
Year: 2024

This paper introduces a novel approach that integrates future closest point predictions into the distance constraints of a collision avoidance controller, leveraging convex hulls with closest point distance calculations. By addressing abrupt shifts in closest points, this method effectively reduces collision risks and enhances controller performance. Applied to an Image Guided Therapy robot and validated through simulations and user experiments, the framework demonstrates improved distance prediction accuracy, smoother trajectories, and safer navigation near obstacles.
Do They Understand What They Are Using? -- Assessing Perception and Usage of Biometrics
Venue: arXiv cs.HC 2024
Year: 2024

In this paper we assess how well users know biometric authentication methods, how they perceive them, and if they have misconceptions about them. We present the results of an online survey that we conducted in two rounds (2019, N=57; and 2023, N=47) to understand the impact of the increasing availability of biometrics on their use and perception. The survey covered participants' general understanding of physiological and behavioral biometrics and their perceived usability and security. While most participants were able to name examples and stated that they use biometrics in their daily lives, they still had difficulties explaining the concepts behind them. We shed light on participants' misconceptions, their coping strategies with authentication failures and potential attacks, as well as their perception of the usability and security of biometrics in general. As such, our results can support the design of both further studies to gain deeper insights and future biometric interfaces to foster the informed use of biometrics.
Identity Emergence in the Context of Vaccine Criticism in France
Venue: arXiv cs.SI 2024
Year: 2024

This study investigates the emergence of collective identity among individuals critical of vaccination policies in France during the COVID-19 pandemic. As concerns grew over mandated health measures, a loose collective formed on Twitter to assert autonomy over vaccination decisions. Using analyses of pronoun usage, outgroup labeling, and tweet similarity, we examine how this identity emerged. A turning point occurred following President Macron's announcement of mandatory vaccination for health workers and the health pass, sparking substantial changes in linguistic patterns. We observed a shift from first-person singular (I) to first-person plural (we) pronouns, alongside an increased focus on vaccinated individuals as a central outgroup, in addition to authority figures. This shift in language patterns was further reflected in the behavior of new users. An analysis of incoming users revealed that a core group of frequent posters played a crucial role in fostering cohesion and shaping norms. New users who joined during the week of Macron's announcement and continued posting afterward showed an increased similarity with the language of the core group, contributing to the crystallization of the emerging collective identity.
Optimizing Multi-Task Learning for Accurate Spacecraft Pose Estimation
Venue: arXiv cs.LG 2024
Year: 2024

Accurate satellite pose estimation is crucial for autonomous guidance, navigation, and control (GNC) systems in in-orbit servicing (IOS) missions. This paper explores the impact of different tasks within a multi-task learning (MTL) framework for satellite pose estimation using monocular images. By integrating tasks such as direct pose estimation, keypoint prediction, object localization, and segmentation into a single network, the study aims to evaluate the reciprocal influence between tasks by testing different multi-task configurations thanks to the modularity of the convolutional neural network (CNN) used in this work. The trends of mutual bias between the analyzed tasks are found by employing different weighting strategies to further test the robustness of the findings. A synthetic dataset was developed to train and test the MTL network. Results indicate that direct pose estimation and heatmap-based pose estimation positively influence each other in general, while both the bounding box and segmentation tasks do not provide significant contributions and tend to degrade the overall estimation accuracy.
Machine Learning-Augmented Ontology-Based Data Access for Renewable Energy Data
Venue: arXiv cs.DB 2024
Year: 2024

Managing the growing data from renewable energy production plants for effective decision-making often involves leveraging Ontology-based Data Access (OBDA), a well-established approach that facilitates querying diverse data through a shared vocabulary, presented in the form of an ontology. Our work addresses one of the common problems in this context, deriving from feeding complex class hierarchies defined by such ontologies from fragmented and imbalanced (w.r.t. class labels) data sources. We introduce an innovative framework that enhances existing OBDA systems. This framework incorporates a dynamic class management approach to address hierarchical classification, leveraging machine learning. The primary objectives are to enhance system performance, extract richer insights from underrepresented data, and automate data classification beyond the typical capabilities of basic deductive reasoning at the ontological level. We experimentally validate our methodology via real-world, industrial case studies from the renewable energy sector, demonstrating the practical applicability and effectiveness of the proposed solution.
Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions
Venue: arXiv cs.RO 2024
Year: 2024

Humanoid robots, with their human-like embodiment, have the potential to integrate seamlessly into human environments. Critical to their coexistence and cooperation with humans is the ability to understand natural language communications and exhibit human-like behaviors. This work focuses on generating diverse whole-body motions for humanoid robots from language descriptions. We leverage human motion priors from extensive human motion datasets to initialize humanoid motions and employ the commonsense reasoning capabilities of Vision Language Models (VLMs) to edit and refine these motions. Our approach demonstrates the capability to produce natural, expressive, and text-aligned humanoid motions, validated through both simulated and real-world experiments. More videos can be found at https://ut-austin-rpl.github.io/Harmon/.