OpenRaiser/Intern-Atlas huggingface dataset:
Frame Representation Hypothesis: Multi-Token LLM Interpretability and Concept-Guided Text Generation
Venue: arXiv cs.CL 2024
Year: 2024

Interpretability is a key challenge in fostering trust for Large Language Models (LLMs), which stems from the complexity of extracting reasoning from model's parameters. We present the Frame Representation Hypothesis, a theoretically robust framework grounded in the Linear Representation Hypothesis (LRH) to interpret and control LLMs by modeling multi-token words. Prior research explored LRH to connect LLM representations with linguistic concepts, but was limited to single token analysis. As most words are composed of several tokens, we extend LRH to multi-token words, thereby enabling usage on any textual data with thousands of concepts. To this end, we propose words can be interpreted as frames, ordered sequences of vectors that better capture token-word relationships. Then, concepts can be represented as the average of word frames sharing a common concept. We showcase these tools through Top-k Concept-Guided Decoding, which can intuitively steer text generation using concepts of choice. We verify said ideas on Llama 3.1, Gemma 2, and Phi 3 families, demonstrating gender and language biases, exposing harmful content, but also potential to remediate them, leading to safer and more transparent LLMs. Code is available at https://github.com/phvv-me/frame-representation-hypothesis.git
Interfacing with history: Curating with audio augmented objects
Venue: arXiv cs.HC 2024
Year: 2024

This article presents and discusses the results from visitors' interactions with two audio augmented reality experiences containing audio augmented objects; physical, real-world objects to which virtual audio sources have been attached. It then proceeds to discusses the commonly identified themes arising from the observation of visitors' behaviour within these experiences and the analysis of their verbal and written feedback. The curatorial potential of audio augmented objects is discussed and, by way of conclusion, their functionality as interfaces to digital audio archival content is proposed, along with their ability to reframe, re-contextualise and create renewed experiences with existing collections of silenced museum exhibits.
Quantitative Comparison of the Total Focusing Method, Reverse Time Migration, and Full Waveform Inversion for Ultrasonic Imaging
Venue: arXiv cs.CE 2024
Year: 2024

Phased array ultrasound is a widely used technique in non-destructive testing. Using piezoelectric elements as both sources and receivers provides a significant gain in information and enables more accurate defect detection. When all source-receiver combinations are used, the process is called full matrix capture. The total focusing method~(TFM), which exploits such datasets, relies on a delay and sum algorithm to sum up the signals on a pixel grid. However, TFM only uses the first arriving p-waves, making it challenging to size complex-shaped defects. By contrast, more advanced methods such as reverse time migration~(RTM) and full waveform inversion~(FWI) use full waveforms to reconstruct defects. Both methods compare measured signals with ultrasound simulations. While RTM identifies defects by convolving forward and backward wavefields once, FWI iteratively updates material models to reconstruct the actual distribution of material properties. This study compares TFM, RTM, and FWI for six specimens featuring circular defects or Y-shaped notches. The reconstructed results are first evaluated qualitatively using different thresholds and then quantitatively using metrics such as AUPRC, AUROC, and F1-score. The results show that FWI performs best in most cases, both qualitatively and quantitatively.
Non-Progressive Influence Maximization in Dynamic Social Networks
Venue: arXiv cs.SI 2024
Year: 2024

The influence maximization (IM) problem involves identifying a set of key individuals in a social network who can maximize the spread of influence through their network connections. With the advent of geometric deep learning on graphs, great progress has been made towards better solutions for the IM problem. In this paper, we focus on the dynamic non-progressive IM problem, which considers the dynamic nature of real-world social networks and the special case where the influence diffusion is non-progressive, i.e., nodes can be activated multiple times. We first extend an existing diffusion model to capture the non-progressive influence propagation in dynamic social networks. We then propose the method, DNIMRL, which employs deep reinforcement learning and dynamic graph embedding to solve the dynamic non-progressive IM problem. In particular, we propose a novel algorithm that effectively leverages graph embedding to capture the temporal changes of dynamic networks and seamlessly integrates with deep reinforcement learning. The experiments, on different types of real-world social network datasets, demonstrate that our method outperforms state-of-the-art baselines.
Generating Knowledge Graphs from Large Language Models: A Comparative Study of GPT-4, LLaMA 2, and BERT
Venue: arXiv cs.CL 2024
Year: 2024

Knowledge Graphs (KGs) are essential for the functionality of GraphRAGs, a form of Retrieval-Augmented Generative Systems (RAGs) that excel in tasks requiring structured reasoning and semantic understanding. However, creating KGs for GraphRAGs remains a significant challenge due to accuracy and scalability limitations of traditional methods. This paper introduces a novel approach leveraging large language models (LLMs) like GPT-4, LLaMA 2 (13B), and BERT to generate KGs directly from unstructured data, bypassing traditional pipelines. Using metrics such as Precision, Recall, F1-Score, Graph Edit Distance, and Semantic Similarity, we evaluate the models' ability to generate high-quality KGs. Results demonstrate that GPT-4 achieves superior semantic fidelity and structural accuracy, LLaMA 2 excels in lightweight, domain-specific graphs, and BERT provides insights into challenges in entity-relationship modeling. This study underscores the potential of LLMs to streamline KG creation and enhance GraphRAG accessibility for real-world applications, while setting a foundation for future advancements.
Beyond Search Engines: Can Large Language Models Improve Curriculum Development?
Venue: arXiv cs.CY 2024
Year: 2024

While Online Learning is growing and becoming widespread, the associated curricula often suffer from a lack of coverage and outdated content. In this regard, a key question is how to dynamically define the topics that must be covered to thoroughly learn a subject (e.g., a course). Large Language Models (LLMs) are considered candidates that can be used to address curriculum development challenges. Therefore, we developed a framework and a novel dataset, built on YouTube, to evaluate LLMs' performance when it comes to generating learning topics for specific courses. The experiment was conducted across over 100 courses and nearly 7,000 YouTube playlists in various subject areas. Our results indicate that GPT-4 can produce more accurate topics for the given courses than extracted topics from YouTube video playlists in terms of BERTScore
Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation
Venue: arXiv cs.CL 2024
Year: 2024

Automatic evaluation by large language models (LLMs) is a prominent topic today; however, judgment and evaluation tasks are often subjective and influenced by various factors, making adaptation challenging. While many studies demonstrate the capabilities of state-of-the-art proprietary LLMs in comparison to human evaluators, they often struggle to adapt to reference evaluators over time, a requirement for achieving personalized judgment. Additionally, numerous works have attempted to apply open LLMs as judges or evaluators, but these efforts frequently overlook the limitations of working with scarce data. Personalized judgment is inherently associated with limited data scenarios, which are common in many real-world problems. Our work aims to present a data augmentation technique to select a more effective sample from limited data in order to align an open LLM with human preference. Our work achieves approximately 7% improvements in Pearson correlation with a reference judge over the baseline,and 30% improvement over the base model (Llama3.1-8B-Instruct) in the mathematical reasoning evaluation task. demonstrating that augmenting selecting more effective preference data enables our approach to surpass baseline methods.
Performance Evaluation of ROS2-DDS middleware implementations facilitating Cooperative Driving in Autonomous Vehicle
Venue: arXiv cs.RO 2024
Year: 2024

In the autonomous vehicle and self-driving paradigm, cooperative perception or exchanging sensor information among vehicles over wireless communication has added a new dimension. Generally, an autonomous vehicle is a special type of robot that requires real-time, highly reliable sensor inputs due to functional safety. Autonomous vehicles are equipped with a considerable number of sensors to provide different required sensor data to make the driving decision and share with other surrounding vehicles. The inclusion of Data Distribution Service(DDS) as a communication middleware in ROS2 has proved its potential capability to be a reliable real-time distributed system. DDS comes with a scoping mechanism known as domain. Whenever a ROS2 process is initiated, it creates a DDS participant. It is important to note that there is a limit to the number of participants allowed in a single domain. The efficient handling of numerous in-vehicle sensors and their messages demands the use of multiple ROS2 nodes in a single vehicle. Additionally, in the cooperative perception paradigm, a significant number of ROS2 nodes can be required when a vehicle functions as a single ROS2 node. These ROS2 nodes cannot be part of a single domain due to DDS participant limitation; thus, different domain communication is unavoidable. Moreover, there are different vendor-specific implementations of DDS, and each vendor has their configurations, which is an inevitable communication catalyst between the ROS2 nodes. The communication between vehicles or robots or ROS2 nodes depends directly on the vendor-specific configuration, data type, data size, and the DDS implementation used as middleware; in our study, we evaluate and investigate the limitations, capabilities, and prospects of the different domain communication for various vendor-specific DDS implementations for diverse sensor data type.
ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

Recent advances in Meta-learning for Black-Box Optimization (MetaBBO) have shown the potential of using neural networks to dynamically configure evolutionary algorithms (EAs), enhancing their performance and adaptability across various BBO instances. However, they are often tailored to a specific EA, which limits their generalizability and necessitates retraining or redesigns for different EAs and optimization problems. To address this limitation, we introduce ConfigX, a new paradigm of the MetaBBO framework that is capable of learning a universal configuration agent (model) for boosting diverse EAs. To achieve so, our ConfigX first leverages a novel modularization system that enables the flexible combination of various optimization sub-modules to generate diverse EAs during training. Additionally, we propose a Transformer-based neural network to meta-learn a universal configuration policy through multitask reinforcement learning across a designed joint optimization task space. Extensive experiments verify that, our ConfigX, after large-scale pre-training, achieves robust zero-shot generalization to unseen tasks and outperforms state-of-the-art baselines. Moreover, ConfigX exhibits strong lifelong learning capabilities, allowing efficient adaptation to new tasks through fine-tuning. Our proposed ConfigX represents a significant step toward an automatic, all-purpose configuration agent for EAs.
Distributed Uplink Rate Splitting Multiple Access (DU-RSMA): Principles and Performance Analysis
Venue: arXiv cs.IT 2024
Year: 2024

One of the main goals of the upcoming sixth-generation (6G) wireless networks is the ability to support higher network density, while ensuring a high quality of service for each user. In this paper, we introduce distributed uplink rate-splitting multiple access (DU-RSMA), define its basic principles, and provide insights into its advantages. Specifically, a system with two remote radio heads (RRHs) and two users is investigated. To improve the performance of the system, we consider that the RRHs can communicate through a feedback link, and thus they are able to decode the received messages either independently or with the assistance of the other RRH, since the decoded information can be shared through the feedback link. It should be noted that this scheme increases the achievable capacity region compared to the known multiple access schemes, which is also evaluated by a novel metric termed ``fill factor''. Both the case of adaptive transmission rates and the case of fixed transmission rates are investigated. To this end, the ergodic rate is investigated to cover the former case, while the outage probability is studied for the latter. Closed-form expressions are derived for both metrics. Finally, the analytical expressions are validated by simulation results, which explicitly show the impact of each parameter on the performance of the system, and prove that the proposed scheme outperforms the corresponding benchmarks.
Enhancing 3D Object Detection in Autonomous Vehicles Based on Synthetic Virtual Environment Analysis
Venue: arXiv cs.CV 2024
Year: 2024

Autonomous Vehicles (AVs) use natural images and videos as input to understand the real world by overlaying and inferring digital elements, facilitating proactive detection in an effort to assure safety. A crucial aspect of this process is real-time, accurate object recognition through automatic scene analysis. While traditional methods primarily concentrate on 2D object detection, exploring 3D object detection, which involves projecting 3D bounding boxes into the three-dimensional environment, holds significance and can be notably enhanced using the AR ecosystem. This study examines an AI model's ability to deduce 3D bounding boxes in the context of real-time scene analysis while producing and evaluating the model's performance and processing time, in the virtual domain, which is then applied to AVs. This work also employs a synthetic dataset that includes artificially generated images mimicking various environmental, lighting, and spatiotemporal states. This evaluation is oriented in handling images featuring objects in diverse weather conditions, captured with varying camera settings. These variations pose more challenging detection and recognition scenarios, which the outcomes of this work can help achieve competitive results under most of the tested conditions.
CoPrUS: Consistency Preserving Utterance Synthesis towards more realistic benchmark dialogues
Venue: arXiv cs.CL 2024
Year: 2024

Large-scale Wizard-Of-Oz dialogue datasets have enabled the training of deep learning-based dialogue systems. While they are successful as benchmark datasets, they lack certain types of utterances, which would make them more realistic. In this work, we investigate the creation of synthetic communication errors in an automatic pipeline. Based on linguistic theory, we propose and follow a simple error taxonomy. We focus on three types of miscommunications that could happen in real-world dialogues but are underrepresented in the benchmark dataset: misunderstandings, non-understandings and vaguely related questions. Our two-step approach uses a state-of-the-art Large Language Model (LLM) to first create the error and secondly the repairing utterance. We perform Language Model-based evaluation to ensure the quality of the generated utterances. We apply the method to the MultiWOZ dataset and evaluate it both qualitatively and empirically as well as with human judges. Our results indicate that current LLMs can aid in adding post-hoc miscommunications to benchmark datasets as a form of data augmentation. We publish the resulting dataset, in which nearly 1900 dialogues have been modified, as CoPrUS-MultiWOZ to facilitate future work on dialogue systems.
Hallucination Elimination and Semantic Enhancement Framework for Vision-Language Models in Traffic Scenarios
Venue: arXiv cs.CV 2024
Year: 2024

Large vision-language models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding and generation tasks. However, these models occasionally generate hallucinatory texts, resulting in descriptions that seem reasonable but do not correspond to the image. This phenomenon can lead to wrong driving decisions of the autonomous driving system. To address this challenge, this paper proposes HCOENet, a plug-and-play chain-of-thought correction method designed to eliminate object hallucinations and generate enhanced descriptions for critical objects overlooked in the initial response. Specifically, HCOENet employs a cross-checking mechanism to filter entities and directly extracts critical objects from the given image, enriching the descriptive text. Experimental results on the POPE benchmark demonstrate that HCOENet improves the F1-score of the Mini-InternVL-4B and mPLUG-Owl3 models by 12.58% and 4.28%, respectively. Additionally, qualitative results using images collected in open campus scene further highlight the practical applicability of the proposed method. Compared with the GPT-4o model, HCOENet achieves comparable descriptive performance while significantly reducing costs. Finally, two novel semantic understanding datasets, CODA_desc and nuScenes_desc, are created for traffic scenarios to support future research. The codes and datasets are publicly available at https://github.com/fjq-tongji/HCOENet.
Knowledge-based model validation using a custom metric
Venue: arXiv cs.CE 2024
Year: 2024

Vehicle models have a long history of research and as of today are able to model the involved physics in a reasonable manner. However, each new vehicle has its new characteristics or parameters. The identification of these is the main task of an engineer. To validate whether the correct parameter set has been chosen is a tedious task and often can only be performed by experts. Metrics known commonly used in literature are able to compare different results under certain aspects. However, they fail to answer the question: Are the models accurate enough? In this article, we propose the usage of a custom metric trained on the knowledge of experts to tackle this problem. Our approach involves three main steps: first, the formalized collection of subject matter experts' opinion on the question: Having seen the measurement and simulation time series in comparison, is the model quality sufficient? From this step, we obtain a data set that is able to quantify the sufficiency of a simulation result based on a comparison to corresponding experimental data. In a second step, we compute common model metrics on the measurement and simulation time series and use these model metrics as features to a regression model. Third, we fit a regression model to the experts' opinions. This regression model, i.e., our custom metric, can than predict the sufficiency of a new simulation result and gives a confidence on this prediction.
Anomaly detection using Diffusion-based methods
Venue: arXiv cs.LG 2024
Year: 2024

This paper explores the utility of diffusion-based models for anomaly detection, focusing on their efficacy in identifying deviations in both compact and high-resolution datasets. Diffusion-based architectures, including Denoising Diffusion Probabilistic Models (DDPMs) and Diffusion Transformers (DiTs), are evaluated for their performance using reconstruction objectives. By leveraging the strengths of these models, this study benchmarks their performance against traditional anomaly detection methods such as Isolation Forests, One-Class SVMs, and COPOD. The results demonstrate the superior adaptability, scalability, and robustness of diffusion-based methods in handling complex real-world anomaly detection tasks. Key findings highlight the role of reconstruction error in enhancing detection accuracy and underscore the scalability of these models to high-dimensional datasets. Future directions include optimizing encoder-decoder architectures and exploring multi-modal datasets to further advance diffusion-based anomaly detection.
RIS-aided Wireless-Powered Backscatter Communications for Sustainable Internet of Underground Things
Venue: arXiv cs.NI 2024
Year: 2024

Wireless-powered underground sensor networks (WPUSNs), which enable wireless energy transfer to sensors located underground, is a promising approach for establishing sustainable internet of underground things (IoUT). To support urgent information transmission and improve resource utilization within WPUSNs, backscatter communication (BC) is introduced, resulting in what is known as wireless-powered backscatter underground sensor networks (WPBUSNs). Nevertheless, the performance of WPBUSNs is significantly limited by severe channel impairments caused by the underground soil and the blockage of direct links. To overcome this challenge, in this work, we propose integrating reconfigurable intelligent surface (RIS) with WPBUSNs, leading to the development of RIS-aided WPBUSNs. We begin by reviewing the recent advancements in BC-WPUSNs and RIS. Then, we propose a general architecture of RIS-aided WPBUSNs across various IoUT scenarios, and discuss its advantages and implementation challenges. To illustrate the effectiveness of RIS-aided WPBUSNs, we focus on a realistic farming case study, demonstrating that our proposed framework outperforms the three benchmarks in terms of the sum throughput. Finally, we discuss the open challenges and future research directions for translating this study into practical IoUT applications.
Is ChatGPT 3 safe for students?
Venue: arXiv cs.SE 2024
Year: 2024

ChatGPT3 is a chat engine that fulfils the promises of an AI-based chat engine: users can ask a question (prompt) and it answers in a reasonable manner. The coding-related skills of ChatGPT are especially impressive: informal testing shows that it is difficult to find simple questions that ChatGPT3 does not know how to answer properly. Some students are certainly already using it to answer programming assignments. This article studies whether it is safe for students to use ChatGPT3 to answer coding assignments (safe means that they will not be caught for plagiarism if they use it). The main result is that it is generally not safe for students to use ChatGPT3. We evaluated the safety of code generated with ChatGPT3, by performing a search with a Codequiry, a plagiarism detection tool, and searching plagiarized code in Google (only considering the first page of results). In 38% of the cases, Codequiry finds a piece of code that is partially copied by the answer of ChatGPT3. In 96% of the cases, the Google search finds a piece of code very similar to the generated code. Overall, it is not safe for students to use ChatGPT3 in 96% of the cases.