OpenRaiser/Intern-Atlas huggingface dataset:
Affirmative safety: An approach to risk management for high-risk AI
Venue: arXiv cs.CY 2024
Year: 2024

Prominent AI experts have suggested that companies developing high-risk AI systems should be required to show that such systems are safe before they can be developed or deployed. The goal of this paper is to expand on this idea and explore its implications for risk management. We argue that entities developing or deploying high-risk AI systems should be required to present evidence of affirmative safety: a proactive case that their activities keep risks below acceptable thresholds. We begin the paper by highlighting global security risks from AI that have been acknowledged by AI experts and world governments. Next, we briefly describe principles of risk management from other high-risk fields (e.g., nuclear safety). Then, we propose a risk management approach for advanced AI in which model developers must provide evidence that their activities keep certain risks below regulator-set thresholds. As a first step toward understanding what affirmative safety cases should include, we illustrate how certain kinds of technical evidence and operational evidence can support an affirmative safety case. In the technical section, we discuss behavioral evidence (evidence about model outputs), cognitive evidence (evidence about model internals), and developmental evidence (evidence about the training process). In the operational section, we offer examples of organizational practices that could contribute to affirmative safety cases: information security practices, safety culture, and emergency response capacity. Finally, we briefly compare our approach to the NIST AI Risk Management Framework. Overall, we hope our work contributes to ongoing discussions about national and global security risks posed by AI and regulatory approaches to address these risks.
Crossing the principle-practice gap in AI ethics with ethical problem-solving
Venue: arXiv cs.CY 2024
Year: 2024

The past years have presented a surge in (AI) development, fueled by breakthroughs in deep learning, increased computational power, and substantial investments in the field. Given the generative capabilities of more recent AI systems, the era of large-scale AI models has transformed various domains that intersect our daily lives. However, this progress raises concerns about the balance between technological advancement, ethical considerations, safety measures, and financial interests. Moreover, using such systems in sensitive areas amplifies our general ethical awareness, prompting a reemergence of debates on governance, regulation, and human values. However, amidst this landscape, how to bridge the principle-practice gap separating ethical discourse from the technical side of AI development remains an open problem. In response to this challenge, the present work proposes a framework to help shorten this gap: ethical problem-solving (EPS). EPS is a methodology promoting responsible, human-centric, and value-oriented AI development. The framework's core resides in translating principles into practical implementations using impact assessment surveys and a differential recommendation methodology. We utilize EPS as a blueprint to propose the implementation of Ethics as a Service Platform, which is currently available as a simple demonstration. We released all framework components openly and with a permissive license, hoping the community would adopt and extend our efforts into other contexts. Available in https://github.com/Nkluge\-correa/ethical\-problem\-solving
Automatic parking planning control method based on improved A* algorithm
Venue: arXiv cs.AI 2024
Year: 2024

As the trend of moving away from high-precision maps gradually emerges in the autonomous driving industry,traditional planning algorithms are gradually exposing some problems. To address the high real-time, high precision, and high trajectory quality requirements posed by the automatic parking task under real-time perceived local maps,this paper proposes an improved automatic parking planning algorithm based on the A* algorithm, and uses Model Predictive Control (MPC) as the control module for automatic parking.The algorithm enhances the planning real-time performance by optimizing heuristic functions, binary heap optimization, and bidirectional search; it calculates the passability of narrow areas by dynamically loading obstacles and introduces the vehicle's own volume during planning; it improves trajectory quality by using neighborhood expansion and Bezier curve optimization methods to meet the high trajectory quality requirements of the parking task. After obtaining the output results of the planning algorithm, a loss function is designed according to the characteristics of the automatic parking task under local maps, and the MPC algorithm is used to output control commands to drive the car along the planned trajectory. This paper uses the perception results of real driving environments converted into maps as planning inputs to conduct simulation tests and ablation experiments on the algorithm. Experimental results show that the improved algorithm proposed in this paper can effectively meet the special requirements of automatic parking under local maps and complete the automatic parking planning and control tasks.
Mental Disorder Classification via Temporal Representation of Text
Venue: arXiv cs.CL 2024
Year: 2024

Mental disorders pose a global challenge, aggravated by the shortage of qualified mental health professionals. Mental disorder prediction from social media posts by current LLMs is challenging due to the complexities of sequential text data and the limited context length of language models. Current language model-based approaches split a single data instance into multiple chunks to compensate for limited context size. The predictive model is then applied to each chunk individually, and the most voted output is selected as the final prediction. This results in the loss of inter-post dependencies and important time variant information, leading to poor performance. We propose a novel framework which first compresses the large sequence of chronologically ordered social media posts into a series of numbers. We then use this time variant representation for mental disorder classification. We demonstrate the generalization capabilities of our framework by outperforming the current SOTA in three different mental conditions: depression, self-harm, and anorexia, with an absolute improvement of 5% in the F1 score. We investigate the situation where current data instances fall within the context length of language models and present empirical results highlighting the importance of temporal properties of textual data. Furthermore, we utilize the proposed framework for a cross-domain study, exploring commonalities across disorders and the possibility of inter-domain data usage.
Improving Large Models with Small models: Lower Costs and Better Performance
Venue: arXiv cs.CL 2024
Year: 2024

Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable performance across diverse tasks. However, the significant computational requirements of PLMs have discouraged most product teams from running or fine-tuning them. In such cases, to harness the exceptional performance of PLMs, one must rely on expensive APIs, thereby exacerbating the economic burden. Despite the overall inferior performance of small models, in specific distributions, they can achieve comparable or even superior results. Consequently, some input can be processed exclusively by small models. On the other hand, certain tasks can be broken down into multiple subtasks, some of which can be completed without powerful capabilities. Under these circumstances, small models can handle the simple subtasks, allowing large models to focus on challenging subtasks, thus improving the performance. We propose Data Shunt$^+$ (DS$^+$), a general paradigm for collaboration of small and large models. DS$^+$ not only substantially reduces the cost associated with querying large models but also effectively improves large models' performance. For instance, ChatGPT achieves an accuracy of $94.43\%$ on Amazon Product sentiment analysis, and DS$^+$ achieves an accuracy of $95.64\%$, while the cost has been reduced to only $31.18\%$. Besides, experiments also prove that the proposed collaborative-based paradigm can better inject specific task knowledge into PLMs compared to fine-tuning.
WundtGPT: Shaping Large Language Models To Be An Empathetic, Proactive Psychologist
Venue: arXiv cs.AI 2024
Year: 2024

Large language models (LLMs) are raging over the medical domain, and their momentum has carried over into the mental health domain, leading to the emergence of few mental health LLMs. Although such mental health LLMs could provide reasonable suggestions for psychological counseling, how to develop an authentic and effective doctor-patient relationship (DPR) through LLMs is still an important problem. To fill this gap, we dissect DPR into two key attributes, i.e., the psychologist's empathy and proactive guidance. We thus present WundtGPT, an empathetic and proactive mental health large language model that is acquired by fine-tuning it with instruction and real conversation between psychologists and patients. It is designed to assist psychologists in diagnosis and help patients who are reluctant to communicate face-to-face understand their psychological conditions. Its uniqueness lies in that it could not only pose purposeful questions to guide patients in detailing their symptoms but also offer warm emotional reassurance. In particular, WundtGPT incorporates Collection of Questions, Chain of Psychodiagnosis, and Empathy Constraints into a comprehensive prompt for eliciting LLMs' questions and diagnoses. Additionally, WundtGPT proposes a reward model to promote alignment with empathetic mental health professionals, which encompasses two key factors: cognitive empathy and emotional empathy. We offer a comprehensive evaluation of our proposed model. Based on these outcomes, we further conduct the manual evaluation based on proactivity, effectiveness, professionalism and coherence. We notice that WundtGPT can offer professional and effective consultation. The model is available at huggingface.
Multi-View Empowered Structural Graph Wordification for Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Significant efforts have been dedicated to integrating the powerful Large Language Models (LLMs) with diverse modalities, particularly focusing on the fusion of language, vision and audio data. However, the graph-structured data, which is inherently rich in structural and domain-specific knowledge, has not yet been gracefully adapted to LLMs. Existing methods either describe the graph with raw text, suffering the loss of graph structural information, or feed Graph Neural Network (GNN) embeddings into LLMs at the cost of losing explainable prompt semantics. To bridge this gap, we introduce an end-to-end modality-aligning framework for LLM-graph alignment: Dual-Residual Vector Quantized-Variational AutoEncoder, namely Dr.E. Our approach is purposefully designed to facilitate token-level alignment with LLMs, enabling an effective translation of the intrinsic `language' of graphs into comprehensible natural language. We also manage to enhance LLMs' more robust structural understanding of graphs by incorporating multiple views of the central nodes based on their surrounding nodes at various distances. Our experimental evaluations on standard graph tasks demonstrate competitive performance against other state-of-the-art (SOTA) approaches. Additionally, our framework ensures certain visual interpretability, efficiency, and robustness, marking the promising successful endeavor to achieve token-level alignment between LLMs and GNNs. Our code is available at: https://github.com/Timothy914/Dr.E.
Data Efficient Evaluation of Large Language Models and Text-to-Image Models via Adaptive Sampling
Venue: arXiv cs.LG 2024
Year: 2024

Evaluating LLMs and text-to-image models is a computationally intensive task often overlooked. Efficient evaluation is crucial for understanding the diverse capabilities of these models and enabling comparisons across a growing number of new models and benchmarks. To address this, we introduce SubLIME, a data-efficient evaluation framework that employs adaptive sampling techniques, such as clustering and quality-based methods, to create representative subsets of benchmarks. Our approach ensures statistically aligned model rankings compared to full datasets, evidenced by high Pearson correlation coefficients. Empirical analysis across six NLP benchmarks reveals that: (1) quality-based sampling consistently achieves strong correlations (0.85 to 0.95) with full datasets at a 10\% sampling rate such as Quality SE and Quality CPD (2) clustering methods excel in specific benchmarks such as MMLU (3) no single method universally outperforms others across all metrics. Extending this framework, we leverage the HEIM leaderboard to cover 25 text-to-image models on 17 different benchmarks. SubLIME dynamically selects the optimal technique for each benchmark, significantly reducing evaluation costs while preserving ranking integrity and score distribution. Notably, a minimal sampling rate of 1% proves effective for benchmarks like MMLU. Additionally, we demonstrate that employing difficulty-based sampling to target more challenging benchmark segments enhances model differentiation with broader score distributions. We also combine semantic search, tool use, and GPT-4 review to identify redundancy across benchmarks within specific LLM categories, such as coding benchmarks. This allows us to further reduce the number of samples needed to maintain targeted rank preservation. Overall, SubLIME offers a versatile and cost-effective solution for the robust evaluation of LLMs and text-to-image models.
Supersonic OT: Fast Unconditionally Secure Oblivious Transfer
Venue: arXiv cs.CR 2024
Year: 2024

Oblivious Transfer (OT) is a fundamental cryptographic protocol with applications in secure Multi-Party Computation, Federated Learning, and Private Set Intersection. With the advent of quantum computing, it is crucial to develop unconditionally secure core primitives like OT to ensure their continued security in the post-quantum era. Despite over four decades since OT's introduction, the literature has predominantly relied on computational assumptions, except in cases using unconventional methods like noisy channels or a fully trusted party. Introducing "Supersonic OT", a highly efficient and unconditionally secure OT scheme that avoids public-key-based primitives, we offer an alternative to traditional approaches. Supersonic OT enables a receiver to obtain a response of size O(1). Its simple (yet non-trivial) design facilitates easy security analysis and implementation. The protocol employs a basic secret-sharing scheme, controlled swaps, the one-time pad, and a third-party helper who may be corrupted by a semi-honest adversary. Our implementation and runtime analysis indicate that a single instance of Supersonic OT completes in 0.35 milliseconds, making it up to 2000 times faster than the state-of-the-art base OT.
Model-based generation of representative rear-end crash scenarios across the full severity range using pre-crash data
Venue: arXiv cs.RO 2024
Year: 2024

Generating representative rear-end crash scenarios is crucial for safety assessments of Advanced Driver Assistance Systems (ADAS) and Automated Driving systems (ADS). However, existing methods for scenario generation face challenges such as limited and biased in-depth crash data and difficulties in validation. This study sought to overcome these challenges by combining naturalistic driving data and pre-crash kinematics data from rear-end crashes. The combined dataset was weighted to create a representative dataset of rear-end crash characteristics across the full severity range in the United States. Multivariate distribution models were built for the combined dataset, and a driver behavior model for the following vehicle was created by combining two existing models. Simulations were conducted to generate a set of synthetic rear-end crash scenarios, which were then weighted to create a representative synthetic rear-end crash dataset. Finally, the synthetic dataset was validated by comparing the distributions of parameters and the outcomes (Delta-v, the total change in vehicle velocity over the duration of the crash event) of the generated crashes with those in the original combined dataset. The synthetic crash dataset can be used for the safety assessments of ADAS and ADS and as a benchmark when evaluating the representativeness of scenarios generated through other methods.
Exponential Time Approximation for Coloring 3-Colorable Graphs
Venue: arXiv cs.DS 2024
Year: 2024

The problem of efficiently coloring $3$-colorable graphs with few colors has received much attention on both the algorithmic and inapproximability fronts. We consider exponential time approximations, in which given a parameter $r$, we aim to develop an $r$-approximation algorithm with the best possible runtime, providing a tradeoff between runtime and approximation ratio. In this vein, an algorithm to $O(n^\varepsilon)$-color a 3-colorable graphs in time $2^{Θ(n^{1-2\varepsilon}\log(n))}$ is given in (Atserias and Dalmau, SODA 2022.) We build on tools developed in (Bansal et al., Algorithmic, 2019) to obtain an algorithm to color $3$-colorable graphs with $O(r)$ colors in $\exp\left(\tilde{O}\left(\frac {n\log^{11/2}r} {r^3}\right)\right)$ time, asymptotically improving upon the bound given by Atserias and Dalmau.
Neural Moving Horizon Estimation: A Systematic Literature Review
Venue: arXiv cs.RO 2024
Year: 2024

The neural moving horizon estimator (NMHE) is a relatively new and powerful state estimator that combines the strengths of neural networks (NNs) and model-based state estimation techniques. Various approaches exist for constructing NMHEs, each with its unique advantages and limitations. However, a comprehensive literature review that consolidates existing knowledge, outlines design guidelines and highlights future research directions is currently lacking. This systematic literature review synthesizes the existing knowledge on NMHE, addressing the above knowledge gap. The paper (1) explains the fundamental principles of NMHE, (2) explores different NMHE architectures, discussing the pros and cons of each, (3) investigates the NN architectures used in NMHE, providing insights for future designs, (4) examines the real-time implementability of current approaches, offering recommendations for practical applications, and (5) discusses the current limitations of NMHE approaches and outlines directions for future research. These insights can significantly improve the design and application of NMHE, which is critical for enhancing state estimation in complex systems.
Ten Years of ZMap
Venue: arXiv cs.CR 2024
Year: 2024

Since ZMap's debut in 2013, networking and security researchers have used the open-source scanner to write hundreds of research papers that study Internet behavior. In addition, ZMap has been adopted by the security industry to build new classes of enterprise security and compliance products. Over the past decade, much of ZMap's behavior -- ranging from its pseudorandom IP generation to its packet construction -- has evolved as we have learned more about how to scan the Internet. In this work, we quantify ZMap's adoption over the ten years since its release, describe its modern behavior (and the measurements that motivated changes), and offer lessons from releasing and maintaining ZMap for future tools.
VR-NRP: A Virtual Reality Simulation for Training in the Neonatal Resuscitation Program
Venue: arXiv cs.HC 2024
Year: 2024

The use of Virtual Reality (VR) technologies has been extensively researched in surgical and anatomical education. VR provides a lifelike and interactive environment where healthcare providers can practice and refresh their skills in a safe environment. VR has been shown to be as effective as traditional medical education teaching methods, with the potential to provide more cost-effective and convenient means of curriculum delivery, especially in rural and remote areas or in environments with limited access to hands-on training. In this sense, VR offers the potential to be used to support resuscitation training for healthcare providers such as the Neonatal Resuscitation Program (NRP). The NRP program is an evidence-based and standardized approach for training healthcare providers on the resuscitation of the newborn. In this article, we describe a VR simulation environment that was designed and developed to refresh the skills of NRP providers. To validate this platform, we compared the VR-NRP simulation with exposure to 360-degree immersive video. We found that both VR technologies were positively viewed by healthcare professionals and performed very similarly to each other. However, the VR simulation provided a significantly increased feeling of presence. Furthermore, participants found the VR simulation more useful, leading to improved experiential learning outcomes. Also, participants using VR simulation reported higher confidence in certain NRP skills, such as proper mask placement and newborn response evaluation. This research represents a step forward in understanding how VR and related extended reality (XR) technologies can be applied for effective, immersive medical education, with potential benefits for remote and rural healthcare providers.
Multi-Model Predictive Attitude Control of Quadrotors
Venue: arXiv cs.RO 2024
Year: 2024

This paper introduces a new multi-model predictive control (MMPC) method for quadrotor attitude control with performance nearly on par with nonlinear model predictive control (NMPC) and computational efficiency similar to linear model predictive control (LMPC). Conventional NMPC, while effective, is computationally intensive, especially for attitude control that needs a high refresh rate. Conversely, LMPC offers computational advantages but suffers from poor performance and local stability. Our approach relies on multiple linear models of attitude dynamics, each accompanied by a linear model predictive controller, dynamically switching between them given flight conditions. We leverage gap metric analysis to minimize the number of models required to accurately predict the vehicle behavior in various conditions and incorporate a soft switching mechanism to ensure system stability during controller transitions. Our results show that with just 15 models, the vehicle attitude can be accurately controlled across various set points. Comparative evaluations with existing controllers such as incremental nonlinear dynamic inversion, sliding mode control, LMPC, and NMPC reveal that our approach closely matches the effectiveness of NMPC, outperforming other methods, with a running time comparable to LMPC.