OpenRaiser/Intern-Atlas huggingface dataset:
A toolbox for calculating objective image properties in aesthetics research
Venue: arXiv cs.CV 2024
Year: 2024

Over the past two decades, researchers in the field of visual aesthetics have studied numerous quantitative (objective) image properties and how they relate to visual aesthetic appreciation. However, results are difficult to compare between research groups. One reason is that researchers use different sets of image properties in their studies. But even if the same properties are used, the image pre-processing techniques may differ and often researchers use their own customized scripts to calculate the image properties. To provide greater accessibility and comparability of research results in visual experimental aesthetics, we developed an open-access and easy-to-use toolbox (called the 'Aesthetics Toolbox'). The Toolbox allows users to calculate a well-defined set of quantitative image properties popular in contemporary research. The properties include lightness and color statistics, Fourier spectral properties, fractality, self-similarity, symmetry, as well as different entropy measures and CNN-based variances. Compatible with most devices, the Toolbox provides an intuitive click-and-drop web interface. In the Toolbox, we integrated the original scripts of four different research groups and translated them into Python 3. To ensure that results were consistent across analyses, we took care that results from the Python versions of the scripts were the same as those from the original scripts. The toolbox, detailed documentation, and a link to the cloud version are available via Github: https://github.com/RBartho/Aesthetics-Toolbox. In summary, we developed a toolbox that helps to standardize and simplify the calculation of quantitative image properties for visual aesthetics research.
Safety Metric Aware Trajectory Repairing for Automated Driving
Venue: arXiv cs.RO 2024
Year: 2024

Recent analyses highlight challenges in autonomous vehicle technologies, particularly failures in decision-making under dynamic or emergency conditions. Traditional automated driving systems recalculate the entire trajectory in a changing environment. Instead, a novel approach retains valid trajectory segments, minimizing the need for complete replanning and reducing changes to the original plan. This work introduces a trajectory repairing framework that calculates a feasible evasive trajectory while computing the Feasible Time-to-React (F-TTR), balancing the maintenance of the original plan with safety assurance. The framework employs a binary search algorithm to iteratively create repaired trajectories, guaranteeing both the safety and feasibility of the trajectory repairing result. In contrast to earlier approaches that separated the calculation of safety metrics from trajectory repairing, which resulted in unsuccessful plans for evasive maneuvers, our work has the anytime capability to provide both a Feasible Time-to-React and an evasive trajectory for further execution.
TextMastero: Mastering High-Quality Scene Text Editing in Diverse Languages and Styles
Venue: arXiv cs.CV 2024
Year: 2024

Scene text editing aims to modify texts on images while maintaining the style of newly generated text similar to the original. Given an image, a target area, and target text, the task produces an output image with the target text in the selected area, replacing the original. This task has been studied extensively, with initial success using Generative Adversarial Networks (GANs) to balance text fidelity and style similarity. However, GAN-based methods struggled with complex backgrounds or text styles. Recent works leverage diffusion models, showing improved results, yet still face challenges, especially with non-Latin languages like CJK characters (Chinese, Japanese, Korean) that have complex glyphs, often producing inaccurate or unrecognizable characters. To address these issues, we present \emph{TextMastero} - a carefully designed multilingual scene text editing architecture based on latent diffusion models (LDMs). TextMastero introduces two key modules: a glyph conditioning module for fine-grained content control in generating accurate texts, and a latent guidance module for providing comprehensive style information to ensure similarity before and after editing. Both qualitative and quantitative experiments demonstrate that our method surpasses all known existing works in text fidelity and style similarity.
Privacy-preserving Universal Adversarial Defense for Black-box Models
Venue: arXiv cs.LG 2024
Year: 2024

Deep neural networks (DNNs) are increasingly used in critical applications such as identity authentication and autonomous driving, where robustness against adversarial attacks is crucial. These attacks can exploit minor perturbations to cause significant prediction errors, making it essential to enhance the resilience of DNNs. Traditional defense methods often rely on access to detailed model information, which raises privacy concerns, as model owners may be reluctant to share such data. In contrast, existing black-box defense methods fail to offer a universal defense against various types of adversarial attacks. To address these challenges, we introduce DUCD, a universal black-box defense method that does not require access to the target model's parameters or architecture. Our approach involves distilling the target model by querying it with data, creating a white-box surrogate while preserving data privacy. We further enhance this surrogate model using a certified defense based on randomized smoothing and optimized noise selection, enabling robust defense against a broad range of adversarial attacks. Comparative evaluations between the certified defenses of the surrogate and target models demonstrate the effectiveness of our approach. Experiments on multiple image classification datasets show that DUCD not only outperforms existing black-box defenses but also matches the accuracy of white-box defenses, all while enhancing data privacy and reducing the success rate of membership inference attacks.
Incorporating a 'ladder of trust' into dynamic Allocation of Function in Human-Autonomous Agent Collectives
Venue: arXiv cs.HC 2024
Year: 2024

A major, ongoing social transition is the inclusion of autonomous agents into human organizations. For example, in defence and security applications, robots may be used alongside human operatives to reduce risk or add capability. But a key barrier to the transition to successful human-autonomous agent collectives is the need for sufficient trust between team members. A critical enabling factor for this trust will be a suitably designed dynamic allocation of function (AoF). We consider AoF in terms of a 'ladder of trust' (from low to high) with individual team members adjusting trust in their teammates based on variation in 'score' over time. The score is derived by the ability of team member to perceive and understand its situation based on the gathered information and act to acheive team or self goals. Combining these trust scores gives a system-level perspective on how AoF might be adjusted during a mission. That is, the most suitable teammate for a function might have a low trust rating from its fellow teammates, so it might be preferable to choose the next most suitable teammate for the function at that point in time. Of course, this is only in the situation where the next most suitable teammate is also likely to perform within the set framework of moral, ethical, and legal constraints. The trade-offs between trust in the individual agent's capability and predictability need to be considered within the broader context of the agent's integrity and accountability. From this perspective, the Allocation Space is defined by more than ability of each agent to perform a function.
Iterative Window Mean Filter: Thwarting Diffusion-based Adversarial Purification
Venue: arXiv cs.CR 2024
Year: 2024

Face authentication systems have brought significant convenience and advanced developments, yet they have become unreliable due to their sensitivity to inconspicuous perturbations, such as adversarial attacks. Existing defenses often exhibit weaknesses when facing various attack algorithms and adaptive attacks or compromise accuracy for enhanced security. To address these challenges, we have developed a novel and highly efficient non-deep-learning-based image filter called the Iterative Window Mean Filter (IWMF) and proposed a new framework for adversarial purification, named IWMF-Diff, which integrates IWMF and denoising diffusion models. These methods can function as pre-processing modules to eliminate adversarial perturbations without necessitating further modifications or retraining of the target system. We demonstrate that our proposed methodologies fulfill four critical requirements: preserved accuracy, improved security, generalizability to various threats in different settings, and better resistance to adaptive attacks. This performance surpasses that of the state-of-the-art adversarial purification method, DiffPure.
HMoE: Heterogeneous Mixture of Experts for Language Modeling
Venue: arXiv cs.CL 2024
Year: 2024

Mixture of Experts (MoE) offers remarkable performance and computational efficiency by selectively activating subsets of model parameters. Traditionally, MoE models use homogeneous experts, each with identical capacity. However, varying complexity in input data necessitates experts with diverse capabilities, while homogeneous MoE hinders effective expert specialization and efficient parameter utilization. In this study, we propose a novel Heterogeneous Mixture of Experts (HMoE), where experts differ in size and thus possess diverse capacities. This heterogeneity allows for more specialized experts to handle varying token complexities more effectively. To address the imbalance in expert activation, we propose a novel training objective that encourages the frequent activation of smaller experts, enhancing computational efficiency and parameter utilization. Extensive experiments demonstrate that HMoE achieves lower loss with fewer activated parameters and outperforms conventional homogeneous MoE models on various pre-training evaluation benchmarks. Codes will be released upon acceptance.
Proving Cutoff Bounds for Safety Properties in First-Order Logic
Venue: arXiv cs.LO 2024
Year: 2024

First-order logic has been established as an important tool for modeling and verifying intricate systems such as distributed protocols and concurrent systems. These systems are parametric in the number of nodes in the network or the number of threads, which is finite in any system instance, but unbounded. One disadvantage of first-order logic is that it cannot distinguish between finite and infinite structures, leading to spurious counterexamples. To mitigate this, we offer a verification approach that captures only finite system instances. Our approach is an adaptation of the cutoff method to systems modeled in first-order logic. The idea is to show that any safety violation in a system instance of size larger than some bound can be simulated by a safety violation in a system of a smaller size. The simulation provides an inductive argument for correctness in finite instances, reducing the problem to showing safety of instances with bounded size. To this end, we develop a framework to (i) encode such simulation relations in first-order logic and to (ii) validate the simulation relation by a set of verification conditions given to an SMT solver. We apply our approach to verify safety of a set of examples, some of which cannot be proven by a first-order inductive invariant.
Genesis: Towards the Automation of Systems Biology Research
Venue: arXiv cs.AI 2024
Year: 2024

The cutting edge of applying AI to science is the closed-loop automation of scientific research: robot scientists. We have previously developed two robot scientists: `Adam' (for yeast functional biology), and `Eve' (for early-stage drug design)). We are now developing a next generation robot scientist Genesis. With Genesis we aim to demonstrate that an area of science can be investigated using robot scientists unambiguously faster, and at lower cost, than with human scientists. Here we report progress on the Genesis project. Genesis is designed to automatically improve system biology models with thousands of interacting causal components. When complete Genesis will be able to initiate and execute in parallel one thousand hypothesis-led closed-loop cycles of experiment per-day. Here we describe the core Genesis hardware: the one thousand computer-controlled $μ$-bioreactors. For the integrated Mass Spectrometry platform we have developed AutonoMS, a system to automatically run, process, and analyse high-throughput experiments. We have also developed Genesis-DB, a database system designed to enable software agents access to large quantities of structured domain information. We have developed RIMBO (Revisions for Improvements of Models in Biology Ontology) to describe the planned hundreds of thousands of changes to the models. We have demonstrated the utility of this infrastructure by developed two relational learning bioinformatic projects. Finally, we describe LGEM+ a relational learning system for the automated abductive improvement of genome-scale metabolic models.
Large Language Models for Multimodal Deformable Image Registration
Venue: arXiv cs.CV 2024
Year: 2024

The challenge of Multimodal Deformable Image Registration (MDIR) lies in the conversion and alignment of features between images of different modalities. Generative models (GMs) cannot retain the necessary information enough from the source modality to the target one, while non-GMs struggle to align features across these two modalities. In this paper, we propose a novel coarse-to-fine MDIR framework,LLM-Morph, which is applicable to various pre-trained Large Language Models (LLMs) to solve these concerns by aligning the deep features from different modal medical images. Specifically, we first utilize a CNN encoder to extract deep visual features from cross-modal image pairs, then we use the first adapter to adjust these tokens, and use LoRA in pre-trained LLMs to fine-tune their weights, both aimed at eliminating the domain gap between the pre-trained LLMs and the MDIR task. Third, for the alignment of tokens, we utilize other four adapters to transform the LLM-encoded tokens into multi-scale visual features, generating multi-scale deformation fields and facilitating the coarse-to-fine MDIR task. Extensive experiments in MR-CT Abdomen and SR-Reg Brain datasets demonstrate the effectiveness of our framework and the potential of pre-trained LLMs for MDIR task. Our code is availabel at: https://github.com/ninjannn/LLM-Morph.
Distribution of Reconfiguration Languages maintaining Tree-like Communication Topology
Venue: arXiv cs.FL 2024
Year: 2024

We study how to distribute trace languages in a setting where processes communicate via reconfigurable communication channels. That is, the different processes can connect and disconnect from channels at run time. We restrict attention to communication via tree-like communication architectures. These allow channels to connect more than two processes in a way that maintains an underlying spanning tree and keeps communication continuous on the tree. We make the reconfiguration explicit in the language allowing both a centralized automaton as well as the distributed processes to share relevant information about the current communication configuration. We show that Zielonka's seminal result regarding distribution of regular languages for asynchronous automata can be generalized in this setting, incorporating both reconfiguration and more than binary tree architectures.
Vector Symbolic Open Source Information Discovery
Venue: arXiv cs.IR 2024
Year: 2024

Combined, joint, intra-governmental, inter-agency and multinational (CJIIM) operations require rapid data sharing without the bottlenecks of metadata curation and alignment. Curation and alignment is particularly infeasible for external open source information (OSINF), e.g., social media, which has become increasingly valuable in understanding unfolding situations. Large language models (transformers) facilitate semantic data and metadata alignment but are inefficient in CJIIM settings characterised as denied, degraded, intermittent and low bandwidth (DDIL). Vector symbolic architectures (VSA) support semantic information processing using highly compact binary vectors, typically 1-10k bits, suitable in a DDIL setting. We demonstrate a novel integration of transformer models with VSA, combining the power of the former for semantic matching with the compactness and representational structure of the latter. The approach is illustrated via a proof-of-concept OSINF data discovery portal that allows partners in a CJIIM operation to share data sources with minimal metadata curation and low communications bandwidth. This work was carried out as a bridge between previous low technology readiness level (TRL) research and future higher-TRL technology demonstration and deployment.
Ghost Echoes Revealed: Benchmarking Maintainability Metrics and Machine Learning Predictions Against Human Assessments
Venue: arXiv cs.SE 2024
Year: 2024

As generative AI is expected to increase global code volumes, the importance of maintainability from a human perspective will become even greater. Various methods have been developed to identify the most important maintainability issues, including aggregated metrics and advanced Machine Learning (ML) models. This study benchmarks several maintainability prediction approaches, including State-of-the-Art (SotA) ML, SonarQube's Maintainability Rating, CodeScene's Code Health, and Microsoft's Maintainability Index. Our results indicate that CodeScene matches the accuracy of SotA ML and outperforms the average human expert. Importantly, unlike SotA ML, CodeScene also provides end users with actionable code smell details to remedy identified issues. Finally, caution is advised with SonarQube due to its tendency to generate many false positives. Unfortunately, our findings call into question the validity of previous studies that solely relied on SonarQube output for establishing ground truth labels. To improve reliability in future maintainability and technical debt studies, we recommend employing more accurate metrics. Moreover, reevaluating previous findings with Code Health would mitigate this revealed validity threat.
Data Ethics and Practices of Human-Nonhuman Sound Technologies and Ecologies
Venue: arXiv cs.HC 2024
Year: 2024

Human-nonhuman sound interaction and technologies aim to bridge the gap of inter-species communication. While they emerge from attempts to understand and communicate with nonhumans, they also raise questions on the ethics of nonhuman data use, for example regarding the unintended consequences such data extraction can have to nonhumans. In this paper, we discuss power relations and aspects of representation in nonhuman data practices, and their potential critical implications to nonhumans. Drawing from prior research on data ethics and posthumanities, we conceptualize two challenges of nonhuman data ethics for the design of Human-Nonhuman Interaction (HNI) and technologies in sound ecologies. We provide takeaways for how sensitivities toward nonhuman stakeholders can be considered in the design of HNI in the context of sound ecologies.
Does Co-Development with AI Assistants Lead to More Maintainable Code? A Registered Report
Venue: arXiv cs.SE 2024
Year: 2024

[Background/Context] AI assistants like GitHub Copilot are transforming software engineering; several studies have highlighted productivity improvements. However, their impact on code quality, particularly in terms of maintainability, requires further investigation. [Objective/Aim] This study aims to examine the influence of AI assistants on software maintainability, specifically assessing how these tools affect the ability of developers to evolve code. [Method] We will conduct a two-phased controlled experiment involving professional developers. In Phase 1, developers will add a new feature to a Java project, with or without the aid of an AI assistant. Phase 2, a randomized controlled trial, will involve a different set of developers evolving random Phase 1 projects - working without AI assistants. We will employ Bayesian analysis to evaluate differences in completion time, perceived productivity, code quality, and test coverage.
Detection of Intracranial Hemorrhage for Trauma Patients
Venue: arXiv cs.CV 2024
Year: 2024

Whole-body CT is used for multi-trauma patients in the search of any and all injuries. Since an initial assessment needs to be rapid and the search for lesions is done for the whole body, very little time can be allocated for the inspection of a specific anatomy. In particular, intracranial hemorrhages are still missed, especially by clinical students. In this work, we present a Deep Learning approach for highlighting such lesions to improve the diagnostic accuracy. While most works on intracranial hemorrhages perform segmentation, detection only requires bounding boxes for the localization of the bleeding. In this paper, we propose a novel Voxel-Complete IoU (VC-IoU) loss that encourages the network to learn the 3D aspect ratios of bounding boxes and leads to more precise detections. We extensively experiment on brain bleeding detection using a publicly available dataset, and validate it on a private cohort, where we achieve 0.877 AR30, 0.728 AP30, and 0.653 AR30, 0.514 AP30 respectively. These results constitute a relative +5% improvement in Average Recall for both datasets compared to other loss functions. Finally, as there is little data currently publicly available for 3D object detection and as annotation resources are limited in the clinical setting, we evaluate the cost of different annotation methods, as well as the impact of imprecise bounding boxes in the training data on the detection performance.
Multi-Agent Based Simulation for Investigating Centralized Charging Strategies and their Impact on Electric Vehicle Home Charging Ecosystem
Venue: arXiv cs.MA 2024
Year: 2024

This paper addresses the critical integration of electric vehicles (EVs) into the electricity grid, which is essential for achieving carbon neutrality by 2050. The rapid increase in EV adoption poses significant challenges to the existing grid infrastructure, particularly in managing the increasing electricity demand and mitigating the risk of grid overloads. Centralized EV charging strategies are investigated due to their potential to optimize grid stability and efficiency, compared to decentralized approaches that may exacerbate grid stress. Utilizing a multi-agent based simulation model, the study provides a realistic representation of the electric vehicle home charging ecosystem in a case study of Strib, Denmark. The findings show that the Earliest-deadline-first and Round Robin perform best with 100% EV adoption in terms of EV user satisfaction. The simulation considers a realistic adoption curve, EV charging strategies, EV models, and driving patterns to capture the full ecosystem dynamics over a long-term period with high resolution (hourly). Additionally, the study offers detailed load profiles for future distribution grids, demonstrating how centralized charging strategies can efficiently manage grid loads and prevent overloads.