OpenRaiser/Intern-Atlas huggingface dataset:
KpopMT: Translation Dataset with Terminology for Kpop Fandom
Venue: arXiv cs.CL 2024
Year: 2024

While machines learn from existing corpora, humans have the unique capability to establish and accept new language systems. This makes human form unique language systems within social groups. Aligning with this, we focus on a gap remaining in addressing translation challenges within social groups, where in-group members utilize unique terminologies. We propose KpopMT dataset, which aims to fill this gap by enabling precise terminology translation, choosing Kpop fandom as an initiative for social groups given its global popularity. Expert translators provide 1k English translations for Korean posts and comments, each annotated with specific terminology within social groups' language systems. We evaluate existing translation systems including GPT models on KpopMT to identify their failure cases. Results show overall low scores, underscoring the challenges of reflecting group-specific terminologies and styles in translation. We make KpopMT publicly available.
Hybridizing Target- and SHAP-encoded Features for Algorithm Selection in Mixed-variable Black-box Optimization
Venue: arXiv cs.NE 2024
Year: 2024

Exploratory landscape analysis (ELA) is a well-established tool to characterize optimization problems via numerical features. ELA is used for problem comprehension, algorithm design, and applications such as automated algorithm selection and configuration. Until recently, however, ELA was limited to search spaces with either continuous or discrete variables, neglecting problems with mixed variable types. This gap was addressed in a recent study that uses an approach based on target-encoding to compute exploratory landscape features for mixedvariable problems. In this work, we investigate an alternative encoding scheme based on SHAP values. While these features do not lead to better results in the algorithm selection setting considered in previous work, the two different encoding mechanisms exhibit complementary performance. Combining both feature sets into a hybrid approach outperforms each encoding mechanism individually. Finally, we experiment with two different ways of meta-selecting between the two feature sets. Both approaches are capable of taking advantage of the performance complementarity of the models trained on target-encoded and SHAP-encoded feature sets, respectively.
FUNAvg: Federated Uncertainty Weighted Averaging for Datasets with Diverse Labels
Venue: arXiv cs.CV 2024
Year: 2024

Federated learning is one popular paradigm to train a joint model in a distributed, privacy-preserving environment. But partial annotations pose an obstacle meaning that categories of labels are heterogeneous over clients. We propose to learn a joint backbone in a federated manner, while each site receives its own multi-label segmentation head. By using Bayesian techniques we observe that the different segmentation heads although only trained on the individual client's labels also learn information about the other labels not present at the respective site. This information is encoded in their predictive uncertainty. To obtain a final prediction we leverage this uncertainty and perform a weighted averaging of the ensemble of distributed segmentation heads, which allows us to segment "locally unknown" structures. With our method, which we refer to as FUNAvg, we are even on-par with the models trained and tested on the same dataset on average. The code is publicly available at https://github.com/Cardio-AI/FUNAvg.
A New Approach for Approximating Directed Rooted Networks
Venue: arXiv cs.DS 2024
Year: 2024

We consider the k-outconnected directed Steiner tree problem (k-DST). Given a directed edge-weighted graph $G=(V,E,w)$, where $V=\{r\}\cup S \cup T$, and an integer $k$, the goal is to find a minimum cost subgraph of $G$ in which there are $k$ edge-disjoint $rt$-paths for every terminal $t\in T$. The problem is know to be NP-hard. Furthermore, the question on whether a polynomial time, subpolynomial approximation algorithm exists for $k$-DST was answered negatively by Grandoni et al. (2018), by proving an approximation hardness of $Ω(|T|/\log |T|)$ under $NP\neq ZPP$. Inspired by modern day applications, we focus on developing efficient algorithms for $k$-DST in graphs where terminals have out-degree $0$, and furthermore constitute the vast majority in the graph. We provide the first approximation algorithm for $k$-DST on such graphs, in which the approximation ratio depends (primarily) on the size of $S$. We present a randomized algorithm that finds a solution of weight at most $\mathcal O(k|S|\log |T|)$ times the optimal weight, and with high probability runs in polynomial time.
Instrumentation and Analysis of Native ML Pipelines via Logical Query Plans
Venue: arXiv cs.DB 2024
Year: 2024

Machine Learning (ML) is increasingly used to automate impactful decisions, which leads to concerns regarding their correctness, reliability, and fairness. We envision highly-automated software platforms to assist data scientists with developing, validating, monitoring, and analysing their ML pipelines. In contrast to existing work, our key idea is to extract "logical query plans" from ML pipeline code relying on popular libraries. Based on these plans, we automatically infer pipeline semantics and instrument and rewrite the ML pipelines to enable diverse use cases without requiring data scientists to manually annotate or rewrite their code. First, we developed such an abstract ML pipeline representation together with machinery to extract it from Python code. Next, we used this representation to efficiently instrument static ML pipelines and apply provenance tracking, which enables lightweight screening for common data preparation issues. Finally, we built machinery to automatically rewrite ML pipelines to perform more advanced what-if analyses and proposed using multi-query optimisation for the resulting workloads. In future work, we aim to interactively assist data scientists as they work on their ML pipelines.
A 'MAP' to find high-performing soft robot designs: Traversing complex design spaces using MAP-elites and Topology Optimization
Venue: arXiv cs.RO 2024
Year: 2024

Soft robotics has emerged as the standard solution for grasping deformable objects, and has proven invaluable for mobile robotic exploration in extreme environments. However, despite this growth, there are no widely adopted computational design tools that produce quality, manufacturable designs. To advance beyond the diminishing returns of heuristic bio-inspiration, the field needs efficient tools to explore the complex, non-linear design spaces present in soft robotics, and find novel high-performing designs. In this work, we investigate a hierarchical design optimization methodology which combines the strengths of topology optimization and quality diversity optimization to generate diverse and high-performance soft robots by evolving the design domain. The method embeds variably sized void regions within the design domain and evolves their size and position, to facilitating a richer exploration of the design space and find a diverse set of high-performing soft robots. We demonstrate its efficacy on both benchmark topology optimization problems and soft robotic design problems, and show the method enhances grasp performance when applied to soft grippers. Our method provides a new framework to design parts in complex design domains, both soft and rigid.
Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices
Venue: arXiv cs.CV 2024
Year: 2024

The aging population poses numerous challenges to healthcare, including the increase in chronic wounds in the elderly. The current approach to wound assessment by therapists based on photographic documentation is subjective, highlighting the need for computer-aided wound recognition from smartphone photos. This offers objective and convenient therapy monitoring, while being accessible to patients from their home at any time. However, despite research in mobile image segmentation, there is a lack of focus on mobile wound segmentation. To address this gap, we conduct initial research on three lightweight architectures to investigate their suitability for smartphone-based wound segmentation. Using public datasets and UNet as a baseline, our results are promising, with both ENet and TopFormer, as well as the larger UNeXt variant, showing comparable performance to UNet. Furthermore, we deploy the models into a smartphone app for visual assessment of live segmentation, where results demonstrate the effectiveness of TopFormer in distinguishing wounds from wound-coloured objects. While our study highlights the potential of transformer models for mobile wound segmentation, future work should aim to further improve the mask contours.
MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis
Venue: arXiv cs.CV 2024
Year: 2024

Auto-regressive models have made significant progress in the realm of language generation, yet they do not perform on par with diffusion models in the domain of image synthesis. In this work, we introduce MARS, a novel framework for T2I generation that incorporates a specially designed Semantic Vision-Language Integration Expert (SemVIE). This innovative component integrates pre-trained LLMs by independently processing linguistic and visual information, freezing the textual component while fine-tuning the visual component. This methodology preserves the NLP capabilities of LLMs while imbuing them with exceptional visual understanding. Building upon the powerful base of the pre-trained Qwen-7B, MARS stands out with its bilingual generative capabilities corresponding to both English and Chinese language prompts and the capacity for joint image and text generation. The flexibility of this framework lends itself to migration towards any-to-any task adaptability. Furthermore, MARS employs a multi-stage training strategy that first establishes robust image-text alignment through complementary bidirectional tasks and subsequently concentrates on refining the T2I generation process, significantly augmenting text-image synchrony and the granularity of image details. Notably, MARS requires only 9% of the GPU days needed by SD1.5, yet it achieves remarkable results across a variety of benchmarks, illustrating the training efficiency and the potential for swift deployment in various applications.
Satellite Image Time Series Semantic Change Detection: Novel Architecture and Analysis of Domain Shift
Venue: arXiv cs.CV 2024
Year: 2024

Satellite imagery plays a crucial role in monitoring changes happening on Earth's surface and aiding in climate analysis, ecosystem assessment, and disaster response. In this paper, we tackle semantic change detection with satellite image time series (SITS-SCD) which encompasses both change detection and semantic segmentation tasks. We propose a new architecture that improves over the state of the art, scales better with the number of parameters, and leverages long-term temporal information. However, for practical use cases, models need to adapt to spatial and temporal shifts, which remains a challenge. We investigate the impact of temporal and spatial shifts separately on global, multi-year SITS datasets using DynamicEarthNet and MUDS. We show that the spatial domain shift represents the most complex setting and that the impact of temporal shift on performance is more pronounced on change detection than on semantic segmentation, highlighting that it is a specific issue deserving further attention.
Cosserat Rods for Modeling Tendon-Driven Robotic Catheter Systems
Venue: arXiv cs.RO 2024
Year: 2024

Tendon-driven robotic catheters are capable of precise execution of minimally invasive cardiac procedures including ablations and imaging. These procedures require accurate mathematical models of not only the catheter and tendons but also their interactions with surrounding tissue and vasculature in order to control the robot path and interaction. This paper presents a mechanical model of a tendon-driven robotic catheter system based on Cosserat rods and integrated with a stable, implicit Euler scheme. We implement the Cosserat rod as a model for a simple catheter centerline and validate its physical accuracy against a large deformation analytical model and experimental data. The catheter model is then supplemented by adding a second Cosserat rod to model a single tendon, using penalty forces to define the constraints of the tendon-catheter system. All the model parameters are defined by the catheter properties established by the design. The combined model is validated against experimental data to confirm its physical accuracy. This model represents a new contribution to the field of robotic catheter modeling in which both the tendons and catheter are modeled by mechanical Cosserat rods and fully-validated against experimental data in the case of the single rod system.
Synthetic to Authentic: Transferring Realism to 3D Face Renderings for Boosting Face Recognition
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we investigate the potential of image-to-image translation (I2I) techniques for transferring realism to 3D-rendered facial images in the context of Face Recognition (FR) systems. The primary motivation for using 3D-rendered facial images lies in their ability to circumvent the challenges associated with collecting large real face datasets for training FR systems. These images are generated entirely by 3D rendering engines, facilitating the generation of synthetic identities. However, it has been observed that FR systems trained on such synthetic datasets underperform when compared to those trained on real datasets, on various FR benchmarks. In this work, we demonstrate that by transferring the realism to 3D-rendered images (i.e., making the 3D-rendered images look more real), we can boost the performance of FR systems trained on these more photorealistic images. This improvement is evident when these systems are evaluated against FR benchmarks utilizing real-world data, thereby paving new pathways for employing synthetic data in real-world applications.
AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition
Venue: arXiv cs.HC 2024
Year: 2024

Explainable Multimodal Emotion Recognition (EMER) is an emerging task that aims to achieve reliable and accurate emotion recognition. However, due to the high annotation cost, the existing dataset (denoted as EMER-Fine) is small, making it difficult to perform supervised training. To reduce the annotation cost and expand the dataset size, this paper reviews the previous dataset construction process. Then, we simplify the annotation pipeline, avoid manual checks, and replace the closed-source models with open-source models. Finally, we build \textbf{EMER-Coarse}, a coarsely-labeled dataset containing large-scale samples. Besides the dataset, we propose a two-stage training framework \textbf{AffectGPT}. The first stage exploits EMER-Coarse to learn a coarse mapping between multimodal inputs and emotion-related descriptions; the second stage uses EMER-Fine to better align with manually-checked results. Experimental results demonstrate the effectiveness of our proposed method on the challenging EMER task. To facilitate further research, we will make the code and dataset available at: https://github.com/zeroQiaoba/AffectGPT.
Co-designing heterogeneous models: a distributed systems approach
Venue: arXiv cs.SE 2024
Year: 2024

The nature of information security has been, and probably will continue to be, marked by the asymmetric competition of attackers and defenders over the control of an uncertain environment. The reduction of this degree of uncertainty via an increase in understanding of that environment is a primary objective for both sides. Models are useful tools in this context because they provide a way to understand and experiment with their targets without the usual operational constraints. However, given the technological and social advancements of today, the object of modelling has increased in complexity. Such objects are no longer singular entities, but heterogeneous socio-technical systems interlinked to form large-scale ecosystems. Furthermore, the underlying components of a system might be based on very different epistemic assumptions and methodologies for construction and use. Naturally, consistent, rigorous reasoning about such systems is hard, but necessary for achieving both security and resilience. The goal of this paper is to present a modelling approach tailored for heterogeneous systems based on three elements: an inferentialist interpretation of what a model is, a distributed systems metaphor to structure that interpretation and a co-design cycle to describe the practical design and construction of the model. The underlying idea is that an open world interpretation, supported by a formal, yet generic abstraction facilitating knowledge translation and providing properties for structured reasoning and, used in practice according to the co-design cycle could lead to models that are more likely to achieve their pre-stated goals. We explore the suitability of this method in the context of three different security-oriented models: a physical data loss model, an organisational recovery under ransomware model and an surge capacity trauma unit model.
VEnhancer: Generative Space-Time Enhancement for Video Generation
Venue: arXiv cs.CV 2024
Year: 2024

We present VEnhancer, a generative space-time enhancement framework that improves the existing text-to-video results by adding more details in spatial domain and synthetic detailed motion in temporal domain. Given a generated low-quality video, our approach can increase its spatial and temporal resolution simultaneously with arbitrary up-sampling space and time scales through a unified video diffusion model. Furthermore, VEnhancer effectively removes generated spatial artifacts and temporal flickering of generated videos. To achieve this, basing on a pretrained video diffusion model, we train a video ControlNet and inject it to the diffusion model as a condition on low frame-rate and low-resolution videos. To effectively train this video ControlNet, we design space-time data augmentation as well as video-aware conditioning. Benefiting from the above designs, VEnhancer yields to be stable during training and shares an elegant end-to-end training manner. Extensive experiments show that VEnhancer surpasses existing state-of-the-art video super-resolution and space-time super-resolution methods in enhancing AI-generated videos. Moreover, with VEnhancer, exisiting open-source state-of-the-art text-to-video method, VideoCrafter-2, reaches the top one in video generation benchmark -- VBench.
African Democracy in the Era of Generative Disinformation: Challenges and Countermeasures against AI-Generated Propaganda
Venue: arXiv cs.CY 2024
Year: 2024

In light of prominent discourse around the negative implications of generative AI, an emerging area of research is investigating the current and estimated impacts of AI-generated propaganda on African citizens participating in elections. Throughout Africa, there have already been suspected cases of AI-generated propaganda influencing electoral outcomes or precipitating coups in countries like Nigeria, Burkina Faso, and Gabon, underscoring the need for comprehensive research in this domain. This paper aims to highlight the risks associated with the spread of generative AI-driven disinformation within Africa while concurrently examining the roles of government, civil society, academia, and the general public in the responsible development, practical use, and robust governance of AI. To understand how African governments might effectively counteract the impact of AI-generated propaganda, this paper presents case studies illustrating the current usage of generative AI for election-related propaganda in Africa. Subsequently, this paper discusses efforts by fact-checking organisations to mitigate the negative impacts of disinformation, explores the potential for new initiatives to actively engage citizens in literacy efforts to combat disinformation spread, and advocates for increased governmental regulatory measures. Overall, this research seeks to increase comprehension of the potential ramifications of AI-generated propaganda on democratic processes within Africa and propose actionable strategies for stakeholders to address these multifaceted challenges.
Transmission Design for XL-RIS-Aided Massive MIMO System with Visibility Regions
Venue: arXiv cs.IT 2024
Year: 2024

This paper proposes a two-timescale transmission scheme for extremely large-scale (XL)-reconfigurable intelligent surfaces (RIS)-aided massive multi-input multi-output (MIMO) systems considering visibility regions (VRs). The beamforming of base stations (BS) is designed based on rapidly changing instantaneous channel state information (CSI), while the phase shifts of RIS are configured based on slowly changing statistical CSI. Specifically, we first formulate a system model with spatially correlated Rician fading channels and introduce the concept of VRs. Then, we derive a closed-form approximate expression for the achievable rate applicable to any number of BS antennas and RIS elements, and analyze the impact of VRs on system performance and complexity. Next, we solve the problem of maximizing the minimum user rate by optimizing the phase shifts of RIS through an algorithm based on accelerated gradient ascent. Finally, we present numerical results to demonstrate the performance of the gradient algorithm from different aspects and reveal the low system complexity of deploying XL-RIS in massive MIMO systems with the help of VRs.
Group Projected Subspace Pursuit for Block Sparse Signal Reconstruction: Convergence Analysis and Applications
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we present a convergence analysis of the Group Projected Subspace Pursuit (GPSP) algorithm proposed by He et al. [HKL+23] (Group Projected subspace pursuit for IDENTification of variable coefficient differential equations (GP-IDENT), Journal of Computational Physics, 494, 112526) and extend its application to general tasks of block sparse signal recovery. We prove that when the sampling matrix satisfies the Block Restricted Isometry Property (BRIP) with a sufficiently small Block Restricted Isometry Constant (BRIC), GPSP exactly recovers the true block sparse signals. When the observations are noisy, this convergence property of GPSP remains valid if the magnitude of true signal is sufficiently large. GPSP selects the features by subspace projection criterion (SPC) for candidate inclusion and response magnitude criterion (RMC) for candidate exclusion. We compare these criteria with counterparts of other state-of-the-art greedy algorithms. Our theoretical analysis and numerical ablation studies reveal that SPC is critical to the superior performances of GPSP, and that RMC can enhance the robustness of feature identification when observations contain noises. We test and compare GPSP with other methods in diverse settings, including heterogeneous random block matrices, inexact observations, face recognition, and PDE identification. We find that GPSP outperforms the other algorithms in most cases for various levels of block sparsity and block sizes, justifying its effectiveness for general applications.