OpenRaiser/Intern-Atlas huggingface dataset:
Style Transfer with Multi-iteration Preference Optimization
Venue: arXiv cs.CL 2024
Year: 2024

Numerous recent techniques for text style transfer characterize their approaches as variants of reinforcement learning and preference optimization. In this work, we consider the relationship between these approaches and a class of optimization approaches developed primarily for (non-neural) statistical machine translation, formerly known as `tuning'. Inspired by these techniques from the past, we improve upon established preference optimization approaches, incorporating multiple iterations of exploration and optimization, and choosing contrastive examples by following a `hope' vs `fear' sampling strategy. Cognizant of the difference between machine translation and style transfer, however, we further tailor our framework with a new pseudo-parallel generation method and a dynamic weighted reward aggregation method to tackle the lack of parallel data and the need for a multi-objective reward. We evaluate our model on two commonly used text style transfer datasets. Through automatic and human evaluation results we show the effectiveness and the superiority of our model compared to state-of-the-art baselines.
The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance
Venue: arXiv cs.CL 2024
Year: 2024

Cloze testing is a common method for measuring the behavior of large language models on a number of benchmark tasks. Using the MMLU dataset, we show that the base-rate probability (BRP) differences across answer tokens are significant and affect task performance ie. guess A if uncertain. We find that counterfactual prompting does sufficiently mitigate the BRP effect. The BRP effect is found to have a similar effect to test taking strategies employed by humans leading to the conflation of task performance and test-taking ability. We propose the Nvr-X-MMLU task, a variation of MMLU, which helps to disambiguate test-taking ability from task performance and reports the latter.
CustAny: Customizing Anything from A Single Example
Venue: arXiv cs.CV 2024
Year: 2024

Recent advances in diffusion-based text-to-image models have simplified creating high-fidelity images, but preserving the identity (ID) of specific elements, like a personal dog, is still challenging. Object customization, using reference images and textual descriptions, is key to addressing this issue. Current object customization methods are either object-specific, requiring extensive fine-tuning, or object-agnostic, offering zero-shot customization but limited to specialized domains. The primary issue of promoting zero-shot object customization from specific domains to the general domain is to establish a large-scale general ID dataset for model pre-training, which is time-consuming and labor-intensive. In this paper, we propose a novel pipeline to construct a large dataset of general objects and build the Multi-Category ID-Consistent (MC-IDC) dataset, featuring 315k text-image samples across 10k categories. With the help of MC-IDC, we introduce Customizing Anything (CustAny), a zero-shot framework that maintains ID fidelity and supports flexible text editing for general objects. CustAny features three key components: a general ID extraction module, a dual-level ID injection module, and an ID-aware decoupling module, allowing it to customize any object from a single reference image and text prompt. Experiments demonstrate that CustAny outperforms existing methods in both general object customization and specialized domains like human customization and virtual try-on. Our contributions include a large-scale dataset, the CustAny framework and novel ID processing to advance this field. Code and dataset will be released soon in https://github.com/LingjieKong-fdu/CustAny.
Nemotron-4 340B Technical Report
Venue: arXiv cs.CL 2024
Year: 2024

We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open Model License Agreement, a permissive model license that allows distribution, modification, and use of the models and its outputs. These models perform competitively to open access models on a wide range of evaluation benchmarks, and were sized to fit on a single DGX H100 with 8 GPUs when deployed in FP8 precision. We believe that the community can benefit from these models in various research studies and commercial applications, especially for generating synthetic data to train smaller language models. Notably, over 98% of data used in our model alignment process is synthetically generated, showcasing the effectiveness of these models in generating synthetic data. To further support open research and facilitate model development, we are also open-sourcing the synthetic data generation pipeline used in our model alignment process.
A First Physical-World Trajectory Prediction Attack via LiDAR-induced Deceptions in Autonomous Driving
Venue: arXiv cs.CR 2024
Year: 2024

Trajectory prediction forecasts nearby agents' moves based on their historical trajectories. Accurate trajectory prediction is crucial for autonomous vehicles. Existing attacks compromise the prediction model of a victim AV by directly manipulating the historical trajectory of an attacker AV, which has limited real-world applicability. This paper, for the first time, explores an indirect attack approach that induces prediction errors via attacks against the perception module of a victim AV. Although it has been shown that physically realizable attacks against LiDAR-based perception are possible by placing a few objects at strategic locations, it is still an open challenge to find an object location from the vast search space in order to launch effective attacks against prediction under varying victim AV velocities. Through analysis, we observe that a prediction model is prone to an attack focusing on a single point in the scene. Consequently, we propose a novel two-stage attack framework to realize the single-point attack. The first stage of prediction-side attack efficiently identifies, guided by the distribution of detection results under object-based attacks against perception, the state perturbations for the prediction model that are effective and velocity-insensitive. In the second stage of location matching, we match the feasible object locations with the found state perturbations. Our evaluation using a public autonomous driving dataset shows that our attack causes a collision rate of up to 63% and various hazardous responses of the victim AV. The effectiveness of our attack is also demonstrated on a real testbed car. To the best of our knowledge, this study is the first security analysis spanning from LiDAR-based perception to prediction in autonomous driving, leading to a realistic attack on prediction. To counteract the proposed attack, potential defenses are discussed.
Threat analysis and adversarial model for Smart Grids
Venue: arXiv cs.CR 2024
Year: 2024

The power grid is a critical infrastructure that allows for the efficient and robust generation, transmission, delivery and consumption of electricity. In the recent years, the physical components have been equipped with computing and network devices, which optimizes the operation and maintenance of the grid. The cyber domain of this smart power grid opens a new plethora of threats, which adds to classical threats on the physical domain. Accordingly, different stakeholders including regulation bodies, industry and academy, are making increasing efforts to provide security mechanisms to mitigate and reduce cyber-risks. Despite these efforts, there have been various cyberattacks that have affected the smart grid, leading in some cases to catastrophic consequences, showcasing that the industry might not be prepared for attacks from high profile adversaries. At the same time, recent work shows a lack of agreement among grid practitioners and academic experts on the feasibility and consequences of academic-proposed threats. This is in part due to inadequate simulation models which do not evaluate threats based on attackers full capabilities and goals. To address this gap, in this work we first analyze the main attack surfaces of the smart grid, and then conduct a threat analysis from the adversarial model perspective, including different levels of knowledge, goals, motivations and capabilities. To validate the model, we provide real-world examples of the potential capabilities by studying known vulnerabilities in critical components, and then analyzing existing cyber-attacks that have affected the smart grid, either directly or indirectly.
Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policies
Venue: arXiv cs.RO 2024
Year: 2024

Humans can imagine goal states during planning and perform actions to match those goals. In this work, we propose Imagination Policy, a novel multi-task key-frame policy network for solving high-precision pick and place tasks. Instead of learning actions directly, Imagination Policy generates point clouds to imagine desired states which are then translated to actions using rigid action estimation. This transforms action inference into a local generative task. We leverage pick and place symmetries underlying the tasks in the generation process and achieve extremely high sample efficiency and generalizability to unseen configurations. Finally, we demonstrate state-of-the-art performance across various tasks on the RLbench benchmark compared with several strong baselines and validate our approach on a real robot.
Improving Multi-Agent Debate with Sparse Communication Topology
Venue: arXiv cs.CL 2024
Year: 2024

Multi-agent debate has proven effective in improving large language models quality for reasoning and factuality tasks. While various role-playing strategies in multi-agent debates have been explored, in terms of the communication among agents, existing approaches adopt a brute force algorithm -- each agent can communicate with all other agents. In this paper, we systematically investigate the effect of communication connectivity in multi-agent systems. Our experiments on GPT and Mistral models reveal that multi-agent debates leveraging sparse communication topology can achieve comparable or superior performance while significantly reducing computational costs. Furthermore, we extend the multi-agent debate framework to multimodal reasoning and alignment labeling tasks, showcasing its broad applicability and effectiveness. Our findings underscore the importance of communication connectivity on enhancing the efficiency and effectiveness of the "society of minds" approach.
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs
Venue: arXiv cs.LG 2024
Year: 2024

Large language models (LLMs) have shown to pose social and ethical risks such as generating toxic language or facilitating malicious use of hazardous knowledge. Machine unlearning is a promising approach to improve LLM safety by directly removing harmful behaviors and knowledge. In this paper, we propose "SPlit, UNlearn, MerGE" (SPUNGE), a framework that can be used with any unlearning method to amplify its effectiveness. SPUNGE leverages data attributes during unlearning by splitting unlearning data into subsets based on specific attribute values, unlearning each subset separately, and merging the unlearned models. We empirically demonstrate that SPUNGE significantly improves the performance of two recent unlearning methods on state-of-the-art LLMs while maintaining their general capabilities on standard academic benchmarks.
WPO: Enhancing RLHF with Weighted Preference Optimization
Venue: arXiv cs.CL 2024
Year: 2024

Reinforcement learning from human feedback (RLHF) is a promising solution to align large language models (LLMs) more closely with human values. Off-policy preference optimization, where the preference data is obtained from other models, is widely adopted due to its cost efficiency and scalability. However, off-policy preference optimization often suffers from a distributional gap between the policy used for data collection and the target policy, leading to suboptimal optimization. In this paper, we propose a novel strategy to mitigate this problem by simulating on-policy learning with off-policy preference data. Our Weighted Preference Optimization (WPO) method adapts off-policy data to resemble on-policy data more closely by reweighting preference pairs according to their probability under the current policy. This method not only addresses the distributional gap problem but also enhances the optimization process without incurring additional costs. We validate our method on instruction following benchmarks including Alpaca Eval 2 and MT-bench. WPO not only outperforms Direct Preference Optimization (DPO) by up to 5.6% on Alpaca Eval 2 but also establishes a remarkable length-controlled winning rate against GPT-4-turbo of 76.7% based on Gemma-2-9b-it. We release the code and models at https://github.com/wzhouad/WPO.
Lifecycle of a sub-metered tertiary multi-use (GreEn-ER) building's open energy data: from resource mobilisation to data re-usability
Venue: arXiv cs.CY 2024
Year: 2024

The proliferation of sensors in buildings has given us access to more data than before. To shepherd this rise in data, many open data lifecycles have been proposed over the past decade. However, many of the proposed lifecycles do not reflect the necessary complexity in the built environment. In this paper, we present a new open data lifecycle model: Open Energy Data Lifecycle (OPENDAL). OPENDAL builds on the key themes in more popular lifecycles and looks to extend them by better accounting for the information flows between cycles and the interactions between stakeholders around the data. These elements are included in the lifecycle in a bid to increase the reuse of published datasets. In addition, we apply the lifecycle model to the datasets from the GreEn-ER building, a mixed-use education building in France. Different use cases of these datasets are highlighted and discussed as a way to incentivise the use of data by other individuals.
Integrating behavior analysis with machine learning to predict online learning performance: A scientometric review and empirical study
Venue: arXiv cs.CY 2024
Year: 2024

The interest in predicting online learning performance using ML algorithms has been steadily increasing. We first conducted a scientometric analysis to provide a systematic review of research in this area. The findings show that most existing studies apply the ML methods without considering learning behavior patterns, which may compromise the prediction accuracy and precision of the ML methods. This study proposes an integration framework that blends learning behavior analysis with ML algorithms to enhance the prediction accuracy of students' online learning performance. Specifically, the framework identifies distinct learning patterns among students by employing clustering analysis and implements various ML algorithms to predict performance within each pattern. For demonstration, the integration framework is applied to a real dataset from edX and distinguishes two learning patterns, as in, low autonomy students and motivated students. The results show that the framework yields nearly perfect prediction performance for autonomous students and satisfactory performance for motivated students. Additionally, this study compares the prediction performance of the integration framework to that of directly applying ML methods without learning behavior analysis using comprehensive evaluation metrics. The results consistently demonstrate the superiority of the integration framework over the direct approach, particularly when integrated with the best-performing XGBoosting method. Moreover, the framework significantly improves prediction accuracy for the motivated students and for the worst-performing random forest method. This study also evaluates the importance of various learning behaviors within each pattern using LightGBM with SHAP values. The implications of the integration framework and the results for online education practice and future research are discussed.
Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation
Venue: arXiv cs.CY 2024
Year: 2024

Amidst the growing interest in developing task-autonomous AI for automated mental health care, this paper addresses the ethical and practical challenges associated with the issue and proposes a structured framework that delineates levels of autonomy, outlines ethical requirements, and defines beneficial default behaviors for AI agents in the context of mental health support. We also evaluate fourteen state-of-the-art language models (ten off-the-shelf, four fine-tuned) using 16 mental health-related questionnaires designed to reflect various mental health conditions, such as psychosis, mania, depression, suicidal thoughts, and homicidal tendencies. The questionnaire design and response evaluations were conducted by mental health clinicians (M.D.s). We find that existing language models are insufficient to match the standard provided by human professionals who can navigate nuances and appreciate context. This is due to a range of issues, including overly cautious or sycophantic responses and the absence of necessary safeguards. Alarmingly, we find that most of the tested models could cause harm if accessed in mental health emergencies, failing to protect users and potentially exacerbating existing symptoms. We explore solutions to enhance the safety of current models. Before the release of increasingly task-autonomous AI systems in mental health, it is crucial to ensure that these models can reliably detect and manage symptoms of common psychiatric disorders to prevent harm to users. This involves aligning with the ethical framework and default behaviors outlined in our study. We contend that model developers are responsible for refining their systems per these guidelines to safeguard against the risks posed by current AI technologies to user mental health and safety. Trigger warning: Contains and discusses examples of sensitive mental health topics, including suicide and self-harm.
AI Royalties -- an IP Framework to Compensate Artists & IP Holders for AI-Generated Content
Venue: arXiv cs.CY 2024
Year: 2024

This article investigates how AI-generated content can disrupt central revenue streams of the creative industries, in particular the collection of dividends from intellectual property (IP) rights. It reviews the IP and copyright questions related to the input and output of generative AI systems. A systematic method is proposed to assess whether AI-generated outputs, especially images, infringe previous copyrights, using a similarity metric (CLIP) between images against historical copyright rulings. An examination (economic and technical feasibility) of previously proposed compensation frameworks reveals their financial implications for creatives and IP holders. Lastly, we propose a novel IP framework for compensation of artists and IP holders based on their published "licensed AIs" as a new medium and asset from which to collect AI royalties.
An ethical study of generative AI from the Actor-Network Theory perspective
Venue: arXiv cs.CY 2024
Year: 2024

The widespread use of Generative Artificial Intelligence in the innovation and generation of communication content is mainly due to its exceptional creative ability, operational efficiency, and compatibility with diverse industries. Nevertheless, this has also sparked ethical problems, such as unauthorized access to data, biased decision-making by algorithms, and criminal use of generated content. In order to tackle the security vulnerabilities linked to Generative Artificial Intelligence, we analyze ChatGPT as a case study within the framework of Actor-Network Theory. We have discovered a total of nine actors, including both human and non-human creatures. We examine the actors and processes of translation involved in the ethical issues related to ChatGPT and analyze the key players involved in the emergence of moral issues. The objective is to explore the origins of the ethical issues that arise with Generative Artificial Intelligence and provide a particular perspective on the governance of Generative Artificial Intelligence.
Gender Differences in Class Participation in Online versus In-Person Core CS Courses
Venue: arXiv cs.CY 2024
Year: 2024

The COVID-19 pandemic significantly altered how post-secondary students receive their education. Namely, the transition from an in-person to an online class format changed how students interact with their instructors and their classmates. In this paper, we use student participation scores from two core computer science classes across ten in-person and three online quarters at a public research university to analyze whether the shift to primarily asynchronous online learning has impacted the gender gap in student participation scores and students' attitudes towards themselves and their peers. We observe a shift on the online class forum: in in-person classes, males score higher on average and dominate the top scores while in online classes, male and female students participate at approximately the same rate classwide. To understand what might be driving changes in participation behavior, we analyze survey responses from over a quarter of the students enrolled in the online classes. While we find that students of both genders tend to compare themselves to their peers less when classes are online, we also find that this trend is much more accentuated for females than males. This data suggests that observed female participation habits in typical in-person classes are not inherent gender differences, but rather, a product of the environment. Therefore, it is critical the community investigates the root causes of these behavioral differences, and experiments with ways to mitigate them, before we soon return to an in-person format.
Not as Simple as It Looked: Are We Concluding for Biased Arrest Practices?
Venue: arXiv cs.CY 2024
Year: 2024

This study examines racial disparities in violent arrest outcomes, challenging conventional methods through a nuanced analysis of Cincinnati Police Department data. Acknowledging the intricate nature of racial disparity, the study categorizes explanations into types of place, types of person, and a combination of both, emphasizing the impact of neighborhood characteristics on crime distribution and police deployment. By introducing alternative scenarios, such as spuriousness, directed policing, and the geo-concentration of racial groups, the study underscores the complexity of racial disparity calculations. Employing a case study approach, the analysis of violent arrest outcomes reveals approximately 40 percent of the observed variation attributed to neighborhood-level characteristics, with concentrated disadvantage neutralizing the influence of race on arrest rates. Contrary to expectations, the study challenges the notion of unintentional racism, suggesting that neighborhood factors play a more significant role than the racial composition in explaining arrests. Policymakers are urged to focus on comprehensive community development initiatives addressing socioeconomic inequalities and support the development of robust racial disparity indices. The study calls for nuanced explorations of unintentional racism and future research addressing potential limitations, aiming to enhance understanding of the complexities surrounding racial disparities in arrests.