OpenRaiser/Intern-Atlas huggingface dataset:
Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges
Venue: arXiv cs.CL 2024
Year: 2024

In the rapidly evolving field of large language models (LLMs), data augmentation (DA) has emerged as a pivotal technique for enhancing model performance by diversifying training examples without the need for additional data collection. This survey explores the transformative impact of LLMs on DA, particularly addressing the unique challenges and opportunities they present in the context of natural language processing (NLP) and beyond. From both data and learning perspectives, we examine various strategies that utilize LLMs for data augmentation, including a novel exploration of learning paradigms where LLM-generated data is used for diverse forms of further training. Additionally, this paper highlights the primary open challenges faced in this domain, ranging from controllable data augmentation to multi-modal data augmentation. This survey highlights a paradigm shift introduced by LLMs in DA, and aims to serve as a comprehensive guide for researchers and practitioners.
SplAgger: Split Aggregation for Meta-Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

A core ambition of reinforcement learning (RL) is the creation of agents capable of rapid learning in novel tasks. Meta-RL aims to achieve this by directly learning such agents. Black box methods do so by training off-the-shelf sequence models end-to-end. By contrast, task inference methods explicitly infer a posterior distribution over the unknown task, typically using distinct objectives and sequence models designed to enable task inference. Recent work has shown that task inference methods are not necessary for strong performance. However, it remains unclear whether task inference sequence models are beneficial even when task inference objectives are not. In this paper, we present evidence that task inference sequence models are indeed still beneficial. In particular, we investigate sequence models with permutation invariant aggregation, which exploit the fact that, due to the Markov property, the task posterior does not depend on the order of data. We empirically confirm the advantage of permutation invariant sequence models without the use of task inference objectives. However, we also find, surprisingly, that there are multiple conditions under which permutation variance remains useful. Therefore, we propose SplAgger, which uses both permutation variant and invariant components to achieve the best of both worlds, outperforming all baselines evaluated on continuous control and memory environments. Code is provided at https://github.com/jacooba/hyper.
Mars 2.0: A Toolchain for Modeling, Analysis, Verification and Code Generation of Cyber-Physical Systems
Venue: arXiv cs.PL 2024
Year: 2024

We introduce Mars 2.0 for modeling, analysis, verification and code generation of Cyber-Physical Systems. Mars 2.0 integrates Mars 1.0 with several important extensions and improvements, allowing the design of cyber-physical systems using the combination of AADL and Simulink/Stateflow, which provide a unified graphical framework for modeling the functionality, physicality and architecture of the system to be developed. For a safety-critical system, formal analysis and verification of its combined AADL and Simulink/Stateflow model can be conducted via the following steps. First, the toolchain automatically translates AADL and Simulink/Stateflow models into Hybrid CSP (HCSP), an extension of CSP for formally modeling hybrid systems. Second, the HCSP processes can be simulated using the HCSP simulator, and to complement incomplete simulation, they can be verified using the Hybrid Hoare Logic prover in Isabelle/HOL, as well as the more automated HHLPy prover. Finally, implementations in SystemC or C can be automatically generated from the verified HCSP processes. The transformation from AADL and Simulink/Stateflow to HCSP, and the one from HCSP to SystemC or C, are both guaranteed to be correct with formal proofs. This approach allows model-driven design of safety-critical cyber-physical systems based on graphical and formal models and proven-correct translation procedures. We demonstrate the use of the toolchain on several benchmarks of varying complexity, including several industrial-sized examples.
Efficient Interaction-Based Offline Runtime Verification of Distributed Systems with Lifeline Removal
Venue: arXiv cs.FL 2024
Year: 2024

Runtime Verification (RV) refers to a family of techniques in which system executions are observed and confronted to formal specifications, with the aim of identifying faults. In Offline RV, observation is done in a first step and verification in a second, on a static artifact collected during observation. In this paper, we define an approach to offline RV of Distributed Systems (DS) against interactions. Interactions are formal models describing communications within a DS. DS are composed of subsystems deployed on different machines and interacting via message passing. Therefore, observing executions of a DS entails logging a collection of local execution traces, one for each subsystem, that we call a multi-trace. A major challenge in analyzing multi-traces is that there are no practical means to synchronize the ends of observations of all local traces. We address this via an operation, called lifeline removal, which we apply on-the-fly on the specification during verification once a local trace has been entirely analyzed. This operation removes from the interaction the specification of actions occurring on the subsystem that is no-longer observed. This may allow further execution of the specification via removing deadlocks due to the partial orders of actions. We prove the correctness of the resulting RV algorithm and introduce two optimization techniques which we also prove correct. We implement a Partial Order Reduction (POR) technique via the selection of a one-unambiguous action (as a unique first step to a linearization) which existence is determined via another use of the lifeline removal operator. Additionally, Local Analyses (LOC) i.e., the verification of local traces, can be leveraged during the global multi-trace analysis to prove failure more quickly. Experiments illustrate the application of our RV approach and the benefits of our optimizations.
When Industry meets Trustworthy AI: A Systematic Review of AI for Industry 5.0
Venue: arXiv cs.CY 2024
Year: 2024

Industry is at the forefront of adopting new technologies, and the process followed by the adoption has a significant impact on the economy and society. In this work, we focus on analysing the current paradigm in which industry evolves, making it more sustainable and Trustworthy. In Industry 5.0, Artificial Intelligence (AI), among other technology enablers, is used to build services from a sustainable, human-centric and resilient perspective. It is crucial to understand those aspects that can bring AI to industry, respecting Trustworthy principles by collecting information to define how it is incorporated in the early stages, its impact, and the trends observed in the field. In addition, to understand the challenges and gaps in the transition from Industry 4.0 to Industry 5.0, a general perspective on the industry's readiness for new technologies is described. This provides practitioners with novel opportunities to be explored in pursuit of the adoption of Trustworthy AI in the sector.
Tooling Offline Runtime Verification against Interaction Models : recognizing sliced behaviors using parameterized simulation
Venue: arXiv cs.SE 2024
Year: 2024

Offline runtime verification involves the static analysis of executions of a system against a specification. For distributed systems, it is generally not possible to characterize executions in the form of global traces, given the absence of a global clock. To account for this, we model executions as collections of local traces called multi-traces, with one local trace per group of co-localized actors that share a common clock. Due to the difficulty of synchronizing the start and end of the recordings of local traces, events may be missing at their beginning or end. Considering such partially observed multi-traces is challenging for runtime verification. To that end, we propose an algorithm that verifies the conformity of such traces against formal specifications called Interactions (akin to Message Sequence Charts). It relies on parameterized simulation to reconstitute unobserved behaviors.
Tappy: Predicting Tap Accuracy of User-Interface Elements by Reverse-Engineering Webpage Structures
Venue: arXiv cs.HC 2024
Year: 2024

Selecting a UI element is a fundamental operation on webpages, and the ease of tapping a target object has a significant impact on usability. It is thus important to analyze existing UIs in order to design better ones. However, tools proposed in previous studies cannot identify whether an element is tappable on modern webpages. In this study, we developed Tappy that can identify tappable UI elements on webpages and estimate the tap-success rate based on the element size. Our interviews of professional designers and engineers showed that Tappy helped discussions of UI design on the basis of its quantitative metric. Furthermore, we have launched this tool to be freely available to external users, so readers can access Tappy by visiting the website (https://tappy.yahoo.co.jp).
PalmProbNet: A Probabilistic Approach to Understanding Palm Distributions in Ecuadorian Tropical Forest via Transfer Learning
Venue: arXiv cs.CV 2024
Year: 2024

Palms play an outsized role in tropical forests and are important resources for humans and wildlife. A central question in tropical ecosystems is understanding palm distribution and abundance. However, accurately identifying and localizing palms in geospatial imagery presents significant challenges due to dense vegetation, overlapping canopies, and variable lighting conditions in mixed-forest landscapes. Addressing this, we introduce PalmProbNet, a probabilistic approach utilizing transfer learning to analyze high-resolution UAV-derived orthomosaic imagery, enabling the detection of palm trees within the dense canopy of the Ecuadorian Rainforest. This approach represents a substantial advancement in automated palm detection, effectively pinpointing palm presence and locality in mixed tropical rainforests. Our process begins by generating an orthomosaic image from UAV images, from which we extract and label palm and non-palm image patches in two distinct sizes. These patches are then used to train models with an identical architecture, consisting of an unaltered pre-trained ResNet-18 and a Multilayer Perceptron (MLP) with specifically trained parameters. Subsequently, PalmProbNet employs a sliding window technique on the landscape orthomosaic, using both small and large window sizes to generate a probability heatmap. This heatmap effectively visualizes the distribution of palms, showcasing the scalability and adaptability of our approach in various forest densities. Despite the challenging terrain, our method demonstrated remarkable performance, achieving an accuracy of 97.32% and a Cohen's kappa of 94.59% in testing.
How Well Can Transformers Emulate In-context Newton's Method?
Venue: arXiv cs.LG 2024
Year: 2024

Transformer-based models have demonstrated remarkable in-context learning capabilities, prompting extensive research into its underlying mechanisms. Recent studies have suggested that Transformers can implement first-order optimization algorithms for in-context learning and even second order ones for the case of linear regression. In this work, we study whether Transformers can perform higher order optimization methods, beyond the case of linear regression. We establish that linear attention Transformers with ReLU layers can approximate second order optimization algorithms for the task of logistic regression and achieve $ε$ error with only a logarithmic to the error more layers. As a by-product we demonstrate the ability of even linear attention-only Transformers in implementing a single step of Newton's iteration for matrix inversion with merely two layers. These results suggest the ability of the Transformer architecture to implement complex algorithms, beyond gradient descent.
Best of Both Worlds: A Pliable and Generalizable Neuro-Symbolic Approach for Relation Classification
Venue: arXiv cs.CL 2024
Year: 2024

This paper introduces a novel neuro-symbolic architecture for relation classification (RC) that combines rule-based methods with contemporary deep learning techniques. This approach capitalizes on the strengths of both paradigms: the adaptability of rule-based systems and the generalization power of neural networks. Our architecture consists of two components: a declarative rule-based model for transparent classification and a neural component to enhance rule generalizability through semantic text matching. Notably, our semantic matcher is trained in an unsupervised domain-agnostic way, solely with synthetic data. Further, these components are loosely coupled, allowing for rule modifications without retraining the semantic matcher. In our evaluation, we focused on two few-shot relation classification datasets: Few-Shot TACRED and a Few-Shot version of NYT29. We show that our proposed method outperforms previous state-of-the-art models in three out of four settings, despite not seeing any human-annotated training data. Further, we show that our approach remains modular and pliable, i.e., the corresponding rules can be locally modified to improve the overall model. Human interventions to the rules for the TACRED relation \texttt{org:parents} boost the performance on that relation by as much as 26\% relative improvement, without negatively impacting the other relations, and without retraining the semantic matching component.
Beyond the Dashboard: Investigating Distracted Driver Communication Preferences for ADAS
Venue: arXiv cs.HC 2024
Year: 2024

Distracted driving is a major cause of road fatalities. With improvements in driver (in)attention detection, these distracted situations can be caught early to alert drivers and improve road safety and comfort. However, drivers may have differing preferences for the modes of such communication based on the driving scenario and their current distraction state. To this end, we present an (N=147) where videos of simulated driving scenarios were utilized to learn drivers preferences for modes of communication and their evolution with the drivers changing attention. The survey queried participants preferred modes of communication for scenarios such as collisions or stagnation at a green light. that inform the future of communication between drivers and their vehicles. We showcase the different driver preferences based on the nature of the driving scenario and also show that they evolve as the drivers distraction state changes
Guardrail Baselines for Unlearning in LLMs
Venue: arXiv cs.CL 2024
Year: 2024

Recent work has demonstrated that finetuning is a promising approach to 'unlearn' concepts from large language models. However, finetuning can be expensive, as it requires both generating a set of examples and running iterations of finetuning to update the model. In this work, we show that simple guardrail-based approaches such as prompting and filtering can achieve unlearning results comparable to finetuning. We recommend that researchers investigate these lightweight baselines when evaluating the performance of more computationally intensive finetuning methods. While we do not claim that methods such as prompting or filtering are universal solutions to the problem of unlearning, our work suggests the need for evaluation metrics that can better separate the power of guardrails vs. finetuning, and highlights scenarios where guardrails expose possible unintended behavior in existing metrics and benchmarks.
Fine-Grained Privacy Guarantees for Coverage Problems
Venue: arXiv cs.DS 2024
Year: 2024

We introduce a new notion of neighboring databases for coverage problems such as Max Cover and Set Cover under differential privacy. In contrast to the standard privacy notion for these problems, which is analogous to node-privacy in graphs, our new definition gives a more fine-grained privacy guarantee, which is analogous to edge-privacy. We illustrate several scenarios of Set Cover and Max Cover where our privacy notion is desired one for the application. Our main result is an $ε$-edge differentially private algorithm for Max Cover which obtains an $(1-1/e-η,\tilde{O}(k/ε))$-approximation with high probability. Furthermore, we show that this result is nearly tight: we give a lower bound show that an additive error of $Ω(k/ε)$ is necessary under edge-differential privacy. Via group privacy properties, this implies a new algorithm for $ε$-node differentially private Max Cover which obtains an $(1-1/e-η,\tilde{O}(fk/ε))$-approximation, where $f$ is the maximum degree of an element in the set system. When $f\ll k$, this improves over the best known algorithm for Max Cover under pure (node) differential privacy, which obtains an $(1-1/e,\tilde{O}(k^2/ε))$-approximation.
Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations
Venue: arXiv cs.CY 2024
Year: 2024

To some, the advent of artificial intelligence (AI) promises better decision-making and increased military effectiveness while reducing the influence of human error and emotions. However, there is still debate about how AI systems, especially large language models (LLMs) that can be applied to many tasks, behave compared to humans in high-stakes military decision-making scenarios with the potential for increased risks towards escalation. To test this potential and scrutinize the use of LLMs for such purposes, we use a new wargame experiment with 214 national security experts designed to examine crisis escalation in a fictional U.S.-China scenario and compare the behavior of human player teams to LLM-simulated team responses in separate simulations. Here, we find that the LLM-simulated responses can be more aggressive and significantly affected by changes in the scenario. We show a considerable high-level agreement in the LLM and human responses and significant quantitative and qualitative differences in individual actions and strategic tendencies. These differences depend on intrinsic biases in LLMs regarding the appropriate level of violence following strategic instructions, the choice of LLM, and whether the LLMs are tasked to decide for a team of players directly or first to simulate dialog between a team of players. When simulating the dialog, the discussions lack quality and maintain a farcical harmony. The LLM simulations cannot account for human player characteristics, showing no significant difference even for extreme traits, such as "pacifist" or "aggressive sociopath." When probing behavioral consistency across individual moves of the simulation, the tested LLMs deviated from each other but generally showed somewhat consistent behavior. Our results motivate policymakers to be cautious before granting autonomy or following AI-based strategy recommendations.
CrossNet: Leveraging Global, Cross-Band, Narrow-Band, and Positional Encoding for Single- and Multi-Channel Speaker Separation
Venue: arXiv cs.SD 2024
Year: 2024

We introduce CrossNet, a complex spectral mapping approach to speaker separation and enhancement in reverberant and noisy conditions. The proposed architecture comprises an encoder layer, a global multi-head self-attention module, a cross-band module, a narrow-band module, and an output layer. CrossNet captures global, cross-band, and narrow-band correlations in the time-frequency domain. To address performance degradation in long utterances, we introduce a random chunk positional encoding. Experimental results on multiple datasets demonstrate the effectiveness and robustness of CrossNet, achieving state-of-the-art performance in tasks including reverberant and noisy-reverberant speaker separation. Furthermore, CrossNet exhibits faster and more stable training in comparison to recent baselines. Additionally, CrossNet's high performance extends to multi-microphone conditions, demonstrating its versatility in various acoustic scenarios.
An AI-enabled Agent-Based Model and Its Application in Measles Outbreak Simulation for New Zealand
Venue: arXiv cs.MA 2024
Year: 2024

Agent Based Models (ABMs) have emerged as a powerful tool for investigating complex social interactions, particularly in the context of public health and infectious disease investigation. In an effort to enhance the conventional ABM, enabling automated model calibration and reducing the computational resources needed for scaling up the model, we have developed a tensorized and differentiable agent-based model by coupling Graph Neural Network (GNN) and Long Short-Term Memory (LSTM) network. The model was employed to investigate the 2019 measles outbreak occurred in New Zealand, demonstrating a promising ability to accurately simulate the outbreak dynamics, particularly during the peak period of repeated cases. This paper shows that by leveraging the latest Artificial Intelligence (AI) technology and the capabilities of traditional ABMs, we gain deeper insights into the dynamics of infectious disease outbreaks. This, in turn, helps us make more informed decision when developing effective strategies that strike a balance between managing outbreaks and minimizing disruptions to everyday life.
Kernel Correlation-Dissimilarity for Multiple Kernel k-Means Clustering
Venue: arXiv cs.LG 2024
Year: 2024

The main objective of the Multiple Kernel k-Means (MKKM) algorithm is to extract non-linear information and achieve optimal clustering by optimizing base kernel matrices. Current methods enhance information diversity and reduce redundancy by exploiting interdependencies among multiple kernels based on correlations or dissimilarities. Nevertheless, relying solely on a single metric, such as correlation or dissimilarity, to define kernel relationships introduces bias and incomplete characterization. Consequently, this limitation hinders efficient information extraction, ultimately compromising clustering performance. To tackle this challenge, we introduce a novel method that systematically integrates both kernel correlation and dissimilarity. Our approach comprehensively captures kernel relationships, facilitating more efficient classification information extraction and improving clustering performance. By emphasizing the coherence between kernel correlation and dissimilarity, our method offers a more objective and transparent strategy for extracting non-linear information and significantly improving clustering precision, supported by theoretical rationale. We assess the performance of our algorithm on 13 challenging benchmark datasets, demonstrating its superiority over contemporary state-of-the-art MKKM techniques.