OpenRaiser/Intern-Atlas huggingface dataset:
Fluid Antenna Index Modulation for MIMO Systems: Robust Transmission and Low-Complexity Detection
Venue: arXiv cs.IT 2024
Year: 2024

The fluid antenna (FA) index modulation (IM)-enabled multiple-input multiple-output (MIMO) system, referred to as FA-IM, significantly enhances spectral efficiency (SE) compared to the conventional FA-assisted MIMO system. To improve robustness against the high spatial correlation among multiple activated ports of the fluid antenna, this paper proposes an innovative FA grouping-based IM (FAG-IM) system. A block grouping scheme is employed based on the spatial correlation model and the distribution structure of the ports. Then, a closed-form expression for the average bit error probability (ABEP) upper bound of the FAG-IM system is derived. To reduce the complexity of the receiver, the message passing architecture is incorporated into the FAG-IM system. Building on this, an efficient approximate message passing (AMP) detector, named structured AMP (S-AMP) detector, is proposed by exploiting the structural characteristics of the transmitted signals. Simulation results confirm that the proposed FAG-IM system significantly outperforms the existing FA-IM system in the presence of spatial correlation, achieving more robust transmission. Furthermore, it is demonstrated that the proposed low-complexity S-AMP detector not only reduces time complexity to a linear scale but also substantially improves bit error rate (BER) performance compared to the minimum mean square error (MMSE) detector, thereby enhancing the practical feasibility of the FAG-IM system.
A free lunch: manifolds of positive reach can be smoothed without decreasing the reach
Venue: arXiv cs.CG 2024
Year: 2024

Assumptions on the reach are crucial for ensuring the correctness of many geometric and topological algorithms, including triangulation, manifold reconstruction and learning, homotopy reconstruction, and methods for estimating curvature or reach. However, these assumptions are often coupled with the requirement that the manifold be smooth, typically at least C^2 .In this paper, we prove that any manifold with positive reach can be approximated arbitrarily well by a C^$\infty$ manifold without significantly reducing the reach, by employing techniques from differential topology -partitions of unity and smoothing using convolution kernels. This result implies that nearly all theorems established for C^2 manifolds with a certain reach naturally extend to manifolds with the same reach, even if they are not C^2 , for free!
EACO: Enhancing Alignment in Multimodal LLMs via Critical Observation
Venue: arXiv cs.CV 2024
Year: 2024

Multimodal large language models (MLLMs) have achieved remarkable progress on various visual question answering and reasoning tasks leveraging instruction fine-tuning specific datasets. They can also learn from preference data annotated by human to enhance their reasoning ability and mitigate hallucinations. Most of preference data is generated from the model itself. However, existing methods require high-quality critical labels, which are costly and rely on human or proprietary models like GPT-4V. In this work, we propose Enhancing Alignment in MLLMs via Critical Observation (EACO), which aligns MLLMs by self-generated preference data using only 5k images economically. Our approach begins with collecting and refining a Scoring Evaluation Instruction-tuning dataset to train a critical evaluation model, termed the Critic. This Critic observes model responses across multiple dimensions, selecting preferred and non-preferred outputs for refined Direct Preference Optimization (DPO) tuning. To further enhance model performance, we employ an additional supervised fine-tuning stage after preference tuning. EACO reduces the overall hallucinations by 65.6% on HallusionBench and improves the reasoning ability by 21.8% on MME-Cognition. EACO achieves an 8.5% improvement over LLaVA-v1.6-Mistral-7B across multiple benchmarks. Remarkably, EACO also shows the potential critical ability in open-source MLLMs, demonstrating that EACO is a viable path to boost the competence of MLLMs.
Continuous Speech Tokens Makes LLMs Robust Multi-Modality Learners
Venue: arXiv cs.SD 2024
Year: 2024

Recent advances in GPT-4o like multi-modality models have demonstrated remarkable progress for direct speech-to-speech conversation, with real-time speech interaction experience and strong speech understanding ability. However, current research focuses on discrete speech tokens to align with discrete text tokens for language modelling, which depends on an audio codec with residual connections or independent group tokens, such a codec usually leverages large scale and diverse datasets training to ensure that the discrete speech codes have good representation for varied domain, noise, style data reconstruction as well as a well-designed codec quantizer and encoder-decoder architecture for discrete token language modelling. This paper introduces Flow-Omni, a continuous speech token based GPT-4o like model, capable of real-time speech interaction and low streaming latency. Specifically, first, instead of cross-entropy loss only, we combine flow matching loss with a pretrained autoregressive LLM and a small MLP network to predict the probability distribution of the continuous-valued speech tokens from speech prompt. second, we incorporated the continuous speech tokens to Flow-Omni multi-modality training, thereby achieving robust speech-to-speech performance with discrete text tokens and continuous speech tokens together. Experiments demonstrate that, compared to discrete text and speech multi-modality training and its variants, the continuous speech tokens mitigate robustness issues by avoiding the inherent flaws of discrete speech code's representation loss for LLM.
Hard Math -- Easy UVM: Pragmatic solutions for verifying hardware algorithms using UVM
Venue: arXiv cs.AI 2024
Year: 2024

This paper presents pragmatic solutions for verifying complex mathematical algorithms implemented in hardware in an efficient and effective manner. Maximizing leverage of a known-answer-test strategy, based on predefined data scenarios combined with design-for-verification modes, we demonstrate how to find and isolate concept and design bugs early in the flow. The solutions presented are based on real project experience with single chip radar sensors for a variety of applications. The verification environments supporting the presented strategies are based on SystemVerilog and the Universal Verification Methodology.
Putting the Iterative Training of Decision Trees to the Test on a Real-World Robotic Task
Venue: arXiv cs.LG 2024
Year: 2024

In previous research, we developed methods to train decision trees (DT) as agents for reinforcement learning tasks, based on deep reinforcement learning (DRL) networks. The samples from which the DTs are built, use the environment's state as features and the corresponding action as label. To solve the nontrivial task of selecting samples, which on one hand reflect the DRL agent's capabilities of choosing the right action but on the other hand also cover enough state space to generalize well, we developed an algorithm to iteratively train DTs. In this short paper, we apply this algorithm to a real-world implementation of a robotic task for the first time. Real-world tasks pose additional challenges compared to simulations, such as noise and delays. The task consists of a physical pendulum attached to a cart, which moves on a linear track. By movements to the left and to the right, the pendulum is to be swung in the upright position and balanced in the unstable equilibrium. Our results demonstrate the applicability of the algorithm to real-world tasks by generating a DT whose performance matches the performance of the DRL agent, while consisting of fewer parameters. This research could be a starting point for distilling DTs from DRL agents to obtain transparent, lightweight models for real-world reinforcement learning tasks.
PETapter: Leveraging PET-style classification heads for modular few-shot parameter-efficient fine-tuning
Venue: arXiv cs.CL 2024
Year: 2024

Few-shot learning and parameter-efficient fine-tuning (PEFT) are crucial to overcome the challenges of data scarcity and ever growing language model sizes. This applies in particular to specialized scientific domains, where researchers might lack expertise and resources to fine-tune high-performing language models to nuanced tasks. We propose PETapter, a novel method that effectively combines PEFT methods with PET-style classification heads to boost few-shot learning capabilities without the significant computational overhead typically associated with full model training. We validate our approach on three established NLP benchmark datasets and one real-world dataset from communication research. We show that PETapter not only achieves comparable performance to full few-shot fine-tuning using pattern-exploiting training (PET), but also provides greater reliability and higher parameter efficiency while enabling higher modularity and easy sharing of the trained modules, which enables more researchers to utilize high-performing NLP-methods in their research.
A Key Encapsulation Mechanism from Low Density Lattice Codes
Venue: arXiv cs.CR 2024
Year: 2024

Key Encapsulation Mechanisms (KEMs) are a set of cryptographic techniques that are designed to provide symmetric encryption key using asymmetric mechanism (public key). In the current study, we concentrate on design and analysis of key encapsulation mechanism from low density lattice codes (KEM-LDLC) to go down the key size by keeping an acceptable level of security. The security of the proposed KEM-LDLC relies on the difficulty of solving the closest vector problem (CVP) and the shortest basis problem (SBP) of the lattices. Furthermore, this paper discusses other performance analyses results such as key size, error performance, and computational complexity, as well as conventional security analysis against applied attacks. Reducing the key size is performed by two approaches: (i) saving the generation sequence of the latin square LDLCs parity-check matrix of as a part of the secret key set; (ii) using the hermite normal form (HNF) of the latin square LDLCs generator matrix as part of the public key set. These enhancements enable us to attain greater efficiency and security compared to earlier code-based KEMs.
Power Plant Detection for Energy Estimation using GIS with Remote Sensing, CNN & Vision Transformers
Venue: arXiv cs.LG 2024
Year: 2024

In this research, we propose a hybrid model for power plant detection to assist energy estimation applications, by pipelining GIS (Geographical Information Systems) having Remote Sensing capabilities with CNN (Convolutional Neural Networks) and ViT (Vision Transformers). Our proposed approach enables real-time analysis with multiple data types on a common map via the GIS, entails feature-extraction abilities due to the CNN, and captures long-range dependencies through the ViT. This hybrid approach is found to enhance classification, thus helping in the monitoring and operational management of power plants; hence assisting energy estimation and sustainable energy planning in the future. It exemplifies adequate deployment of machine learning methods in conjunction with domain-specific approaches to enhance performance.
Analog-Only Beamforming for Near-Field Multiuser MIMO Communications
Venue: arXiv cs.IT 2024
Year: 2024

For the existing near-field multiuser communications based on hybrid beamforming (HBF) architectures, high-quality effective channel estimation is required to obtain the channel state information (CSI) for the design of the digital beamformer. To simplify the system reconfiguration and eliminate the pilot overhead required by the effective channel estimation, we considered an analog-only beamforming (AoBF) architecture in this study. The AoBF is designed with the aim of maximizing the sum rate, which is then transformed into a problem, maximizing the power transmitted to the target user equipment (UE) and meanwhile minimizing the power leaked to the other UEs. To solve this problem, we used beam focusing and beam nulling and proposed two AoBF schemes based on the majorization-minimization (MM) algorithm. First, the AoBF scheme based on perfect CSI is proposed, with the focus on the beamforming performance and regardless of the CSI acquisition. Then, the AoBF scheme based on imperfect CSI is proposed, where the low-dimensional imperfect CSI is obtained by beam sweeping based on a near-field codebook. Simulation results demonstrate that the two AoBF schemes can approach the sum rate of the HBF schemes but outperform HBF schemes in terms of energy efficiency (EE).
Get It Right: Improving Comprehensibility with Adaptable Speech Expression of a Humanoid Service Robot
Venue: arXiv cs.RO 2024
Year: 2024

As humanoid service robots are becoming more and more perceptible in public service settings for instance as a guide to welcome visitors or to explain a procedure to follow, it is desirable to improve the comprehensibility of complex issues for human customers and to adapt the level of difficulty of the information provided as well as the language used to individual requirements. This work examines a case study using a humanoid social robot Pepper performing support for customers in a public service environment offering advice and information. An application architecture is proposed that improves the intelligibility of the information received by providing the possibility to translate this information into easy language and/or into another spoken language.
Talking Like One of Us: Effects of Using Regional Language in a Humanoid Social Robot
Venue: arXiv cs.RO 2024
Year: 2024

Social robots are becoming more and more perceptible in public service settings. For engaging people in a natural environment a smooth social interaction as well as acceptance by the users are important issues for future successful Human-Robot Interaction (HRI). The type of verbal communication has a special significance here. In this paper we investigate the effects of spoken language varieties of a non-standard/regional language compared to standard language. More precisely we compare a human dialog with a humanoid social robot Pepper where the robot on the one hand is answering in High German and on the other hand in Low German, a regional language that is understood and partly still spoken in the northern parts of Germany. The content of what the robot says remains the same in both variants. We are interested in the effects that these two different ways of robot talk have on human interlocutors who are more or less familiar with Low German in terms of perceived warmth, competence and possible discomfort in conversation against a background of cultural identity. To measure these factors we use the Robotic Social Attributes Scale (RoSAS) on 17 participants with an age ranging from 19 to 61. Our results show that significantly higher warmth is perceived in the Low German version of the conversation.
Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models
Venue: arXiv cs.CY 2024
Year: 2024

Large language models have increasingly been proposed as a powerful replacement for classical agent-based models (ABMs) to simulate social dynamics. By using LLMs as a proxy for human behavior, the hope of this new approach is to be able to simulate significantly more complex dynamics than with classical ABMs and gain new insights in fields such as social science, political science, and economics. However, due to the black box nature of LLMs, it is unclear whether LLM agents actually execute the intended semantics that are encoded in their natural language instructions and, if the resulting dynamics of interactions are meaningful. To study this question, we propose a new evaluation framework that grounds LLM simulations within the dynamics of established reference models of social science. By treating LLMs as a black-box function, we evaluate their input-output behavior relative to this reference model, which allows us to evaluate detailed aspects of their behavior. Our results show that, while it is possible to engineer prompts that approximate the intended dynamics, the quality of these simulations is highly sensitive to the particular choice of prompts. Importantly, simulations are even sensitive to arbitrary variations such as minor wording changes and whitespace. This puts into question the usefulness of current versions of LLMs for meaningful simulations, as without a reference model, it is impossible to determine a priori what impact seemingly meaningless changes in prompt will have on the simulation.
The Prompt Canvas: A Literature-Based Practitioner Guide for Creating Effective Prompts in Large Language Models
Venue: arXiv cs.AI 2024
Year: 2024

The rise of large language models (LLMs) has highlighted the importance of prompt engineering as a crucial technique for optimizing model outputs. While experimentation with various prompting methods, such as Few-shot, Chain-of-Thought, and role-based techniques, has yielded promising results, these advancements remain fragmented across academic papers, blog posts and anecdotal experimentation. The lack of a single, unified resource to consolidate the field's knowledge impedes the progress of both research and practical application. This paper argues for the creation of an overarching framework that synthesizes existing methodologies into a cohesive overview for practitioners. Using a design-based research approach, we present the Prompt Canvas, a structured framework resulting from an extensive literature review on prompt engineering that captures current knowledge and expertise. By combining the conceptual foundations and practical strategies identified in prompt engineering, the Prompt Canvas provides a practical approach for leveraging the potential of Large Language Models. It is primarily designed as a learning resource for pupils, students and employees, offering a structured introduction to prompt engineering. This work aims to contribute to the growing discourse on prompt engineering by establishing a unified methodology for researchers and providing guidance for practitioners.
Explingo: Explaining AI Predictions using Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Explanations of machine learning (ML) model predictions generated by Explainable AI (XAI) techniques such as SHAP are essential for people using ML outputs for decision-making. We explore the potential of Large Language Models (LLMs) to transform these explanations into human-readable, narrative formats that align with natural communication. We address two key research questions: (1) Can LLMs reliably transform traditional explanations into high-quality narratives? and (2) How can we effectively evaluate the quality of narrative explanations? To answer these questions, we introduce Explingo, which consists of two LLM-based subsystems, a Narrator and Grader. The Narrator takes in ML explanations and transforms them into natural-language descriptions. The Grader scores these narratives on a set of metrics including accuracy, completeness, fluency, and conciseness. Our experiments demonstrate that LLMs can generate high-quality narratives that achieve high scores across all metrics, particularly when guided by a small number of human-labeled and bootstrapped examples. We also identified areas that remain challenging, in particular for effectively scoring narratives in complex domains. The findings from this work have been integrated into an open-source tool that makes narrative explanations available for further applications.
Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategies
Venue: arXiv cs.CL 2024
Year: 2024

This study evaluates the effectiveness of Vision Language Models (VLMs) in representing and utilizing multimodal content for fact-checking. To be more specific, we investigate whether incorporating multimodal content improves performance compared to text-only models and how well VLMs utilize text and image information to enhance misinformation detection. Furthermore we propose a probing classifier based solution using VLMs. Our approach extracts embeddings from the last hidden layer of selected VLMs and inputs them into a neural probing classifier for multi-class veracity classification. Through a series of experiments on two fact-checking datasets, we demonstrate that while multimodality can enhance performance, fusing separate embeddings from text and image encoders yielded superior results compared to using VLM embeddings. Furthermore, the proposed neural classifier significantly outperformed KNN and SVM baselines in leveraging extracted embeddings, highlighting its effectiveness for multimodal fact-checking.
Enhancing Cross-Language Code Translation via Task-Specific Embedding Alignment in Retrieval-Augmented Generation
Venue: arXiv cs.AI 2024
Year: 2024

We introduce a novel method to enhance cross-language code translation from Fortran to C++ by integrating task-specific embedding alignment into a Retrieval-Augmented Generation (RAG) framework. Unlike conventional retrieval approaches that utilize generic embeddings agnostic to the downstream task, our strategy aligns the retrieval model directly with the objective of maximizing translation quality, as quantified by the CodeBLEU metric. This alignment ensures that the embeddings are semantically and syntactically meaningful for the specific code translation task. Our methodology involves constructing a dataset of 25,000 Fortran code snippets sourced from Stack-V2 dataset and generating their corresponding C++ translations using the LLaMA 3.1-8B language model. We compute pairwise CodeBLEU scores between the generated translations and ground truth examples to capture fine-grained similarities. These scores serve as supervision signals in a contrastive learning framework, where we optimize the embedding model to retrieve Fortran-C++ pairs that are most beneficial for improving the language model's translation performance. By integrating these CodeBLEU-optimized embeddings into the RAG framework, our approach significantly enhances both retrieval accuracy and code generation quality over methods employing generic embeddings. On the HPC Fortran2C++ dataset, our method elevates the average CodeBLEU score from 0.64 to 0.73, achieving a 14% relative improvement. On the Numerical Recipes dataset, we observe an increase from 0.52 to 0.60, marking a 15% relative improvement. Importantly, these gains are realized without any fine-tuning of the language model, underscoring the efficiency and practicality of our approach.
GRFsaw: A lightweight stochastic microstructure generator
Venue: arXiv cs.CE 2024
Year: 2024

This article presents GRFsaw, an open-source software for generating two-phase (binary) microstructures with user-defined structural properties. Unlike most standard software for microstructure generation, GRFsaw is based on the concept of thresholding Gaussian random fields (GRF). It is designed to be used by researchers or engineers in need of a lightweight tool to generate microstructures of various geometries, for example as input to simulations or to other models where such geometries are needed. This could be simulations of fluid flow through porous media, in predictive models of electromagnetic scattering by materials, or in mechanical loading simulations in order to assess, e.g., the material's elasticity or strength.