pwc-archive/papers-with-abstracts huggingface dataset:
SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving
Authors: Andreas Kosmas Kakolyris, Dimosthenis Masouros, Petros Vavaroutsos, Sotirios Xydis, Dimitrios Soudris
Date: 2024-08-05 00:00:00
Tasks: GPU

As Large Language Models (LLMs) gain traction, their reliance on power-hungry GPUs places ever-increasing energy demands, raising environmental and monetary concerns. Inference dominates LLM workloads, presenting a critical challenge for providers: minimizing energy costs under Service-Level Objectives (SLOs) that ensure optimal user experience. In this paper, we present \textit{throttLL'eM}, a framework that reduces energy consumption while meeting SLOs through the use of instance and GPU frequency scaling. \textit{throttLL'eM} features mechanisms that project future KV cache usage and batch size. Leveraging a Machine-Learning (ML) model that receives these projections as inputs, \textit{throttLL'eM} manages performance at the iteration level to satisfy SLOs with reduced frequencies and instance sizes. We show that the proposed ML model achieves $R^2$ scores greater than 0.97 and miss-predicts performance by less than 1 iteration per second on average. Experimental results on LLM inference traces show that \textit{throttLL'eM} achieves up to 43.8\% lower energy consumption and an energy efficiency improvement of at least $1.71\times$ under SLOs, when compared to NVIDIA's Triton server.
Implicit Neural Representation For Accurate CFD Flow Field Prediction
Authors: Laurent de Vito, Nils Pinnau, Simone Dey
Date: 2024-08-12 00:00:00
Tasks: Prediction

Despite the plethora of deep learning frameworks for flow field prediction, most of them deal with flow fields on regular domains, and although the best ones can cope with irregular domains, they mostly rely on graph networks, so that real industrial applications remain currently elusive. We present a deep learning framework for 3D flow field prediction applied to blades of aircraft engine turbines and compressors. Crucially, we view any 3D field as a function from coordinates that is modeled by a neural network we call the backbone-net. It inherits the property of coordinate-based MLPs, namely the discretization-agnostic representation of flow fields in domains of arbitrary topology at infinite resolution. First, we demonstrate the performance of the backbone-net solo in regressing 3D steady simulations of single blade rows in various flow regimes: it can accurately render important flow characteristics such as boundary layers, wakes and shock waves. Second, we introduce a hyper-net that maps the surface mesh of a blade to the parameters of the backbone-net. By doing so, the flow solution can be directly predicted from the blade geometry, irrespective of its parameterization. Together, backbone-net and hyper-net form a highly-accurate memory-efficient data-driven proxy to CFD solvers with good generalization on unseen geometries.
Trustworthiness for an Ultra-Wideband Localization Service
Authors: Philipp Peterseil, Bernhard Etzlinger, Jan Horáček, Roya Khanzadeh, Andreas Springer
Date: 2024-08-10 00:00:00

Trustworthiness assessment is an essential step to assure that interdependent systems perform critical functions as anticipated, even under adverse conditions. In this paper, a holistic trustworthiness assessment framework for ultra-wideband self-localization is proposed, including attributes of reliability, security, privacy, and resilience. Our goal is to provide guidance for evaluating a system's trustworthiness based on objective evidence, so-called trustworthiness indicators. These indicators are carefully selected through the threat analysis of the particular system. Our approach guarantees that the resulting trustworthiness indicators correspond to chosen real-world threats. Moreover, experimental evaluations are conducted to demonstrate the effectiveness of the proposed method. While the framework is tailored for this specific use case, the process itself serves as a versatile template, which can be used in other applications in the domains of the Internet of Things or cyber-physical systems.
Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Models
Authors: Nunzio Lore, Sepehr Ilami, Babak Heydari
Date: 2024-08-05 00:00:00
Tasks: Decision Making, model, Navigate

As the performance of larger, newer Large Language Models continues to improve for strategic Theory of Mind (ToM) tasks, the demand for these state-of-the-art models increases commensurately. However, their deployment is costly both in terms of processing power and time. In this paper, we investigate the feasibility of creating smaller, highly-performing specialized algorithms by way of fine-tuning. To do this, we first present a large pre-trained model with 20 unique scenarios that combine different social contexts with games of varying social dilemmas, record its answers, and use them for Q&A fine-tuning on a smaller model of the same family. Our focus is on in-context game-theoretic decision-making, the same domain within which human interaction occurs and that requires both a theory of mind (or a semblance thereof) and an understanding of social dynamics. The smaller model is therefore trained not just on the answers provided, but also on the motivations provided by the larger model, which should contain advice and guidelines to navigate both strategic dilemmas and social cues. We find that the fine-tuned smaller language model consistently bridged the gap in performance between the smaller pre-trained version of the model and its larger relative and that its improvements extended in areas and contexts beyond the ones provided in the training examples, including on out-of-sample scenarios that include completely different game structures. On average for all games, through fine-tuning, the smaller model showed a 46% improvement measured as alignment towards the behavior of the larger model, with 100% representing indistinguishable behavior. When presented with out-of-sample social contexts and games, the fine-tuned model still displays remarkable levels of alignment, reaching an improvement of 18% and 28% respectively.
The Literature Review Network: An Explainable Artificial Intelligence for Systematic Literature Reviews, Meta-analyses, and Method Development
Authors: Joshua Morriss, Tod Brindle, Jessica Bah Rösman, Daniel Reibsamen, Andreas Enz
Date: 2024-08-05 00:00:00
Tasks: Explainable artificial intelligence

Systematic literature reviews are the highest quality of evidence in research. However, the review process is hindered by significant resource and data constraints. The Literature Review Network (LRN) is the first of its kind explainable AI platform adhering to PRISMA 2020 standards, designed to automate the entire literature review process. LRN was evaluated in the domain of surgical glove practices using 3 search strings developed by experts to query PubMed. A non-expert trained all LRN models. Performance was benchmarked against an expert manual review. Explainability and performance metrics assessed LRN's ability to replicate the experts' review. Concordance was measured with the Jaccard index and confusion matrices. Researchers were blinded to the other's results until study completion. Overlapping studies were integrated into an LRN-generated systematic review. LRN models demonstrated superior classification accuracy without expert training, achieving 84.78% and 85.71% accuracy. The highest performance model achieved high interrater reliability (k = 0.4953) and explainability metrics, linking 'reduce', 'accident', and 'sharp' with 'double-gloving'. Another LRN model covered 91.51% of the relevant literature despite diverging from the non-expert's judgments (k = 0.2174), with the terms 'latex', 'double' (gloves), and 'indication'. LRN outperformed the manual review (19,920 minutes over 11 months), reducing the entire process to 288.6 minutes over 5 days. This study demonstrates that explainable AI does not require expert training to successfully conduct PRISMA-compliant systematic literature reviews like an expert. LRN summarized the results of surgical glove studies and identified themes that were nearly identical to the clinical researchers' findings. Explainable AI can accurately expedite our understanding of clinical practices, potentially revolutionizing healthcare research.
Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course
Authors: Sebastian Kahl, Felix Löffler, Martin Maciol, Fabian Ridder, Marius Schmitz, Jennifer Spanagel, Jens Wienkamp, Christopher Burgahn, Malte Schilling
Date: 2024-08-02 00:00:00
Tasks: Prompt Engineering, RAG, Retrieval-augmented Generation

This study evaluates the performance of Large Language Models (LLMs) as an Artificial Intelligence-based tutor for a university course. In particular, different advanced techniques are utilized, such as prompt engineering, Retrieval-Augmented-Generation (RAG), and fine-tuning. We assessed the different models and applied techniques using common similarity metrics like BLEU-4, ROUGE, and BERTScore, complemented by a small human evaluation of helpfulness and trustworthiness. Our findings indicate that RAG combined with prompt engineering significantly enhances model responses and produces better factual answers. In the context of education, RAG appears as an ideal technique as it is based on enriching the input of the model with additional information and material which usually is already present for a university course. Fine-tuning, on the other hand, can produce quite small, still strong expert models, but poses the danger of overfitting. Our study further asks how we measure performance of LLMs and how well current measurements represent correctness or relevance? We find high correlation on similarity metrics and a bias of most of these metrics towards shorter responses. Overall, our research points to both the potential and challenges of integrating LLMs in educational settings, suggesting a need for balanced training approaches and advanced evaluation frameworks.
Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective
Authors: Yiqun Zhang, Xiaocui Yang, Xingle Xu, Zeran Gao, YiJie Huang, Shiyi Mu, Shi Feng, Daling Wang, Yifei Zhang, Kaisong Song, Ge Yu
Date: 2024-07-30 00:00:00
Tasks: Common Sense Reasoning, In-Context Learning, Prompt Engineering

Affective Computing (AC), integrating computer science, psychology, and cognitive science knowledge, aims to enable machines to recognize, interpret, and simulate human emotions.To create more value, AC can be applied to diverse scenarios, including social media, finance, healthcare, education, etc. Affective Computing (AC) includes two mainstream tasks, i.e., Affective Understanding (AU) and Affective Generation (AG). Fine-tuning Pre-trained Language Models (PLMs) for AU tasks has succeeded considerably. However, these models lack generalization ability, requiring specialized models for specific tasks. Additionally, traditional PLMs face challenges in AG, particularly in generating diverse and emotionally rich responses. The emergence of Large Language Models (LLMs), such as the ChatGPT series and LLaMA models, brings new opportunities and challenges, catalyzing a paradigm shift in AC. LLMs possess capabilities of in-context learning, common sense reasoning, and advanced sequence generation, which present unprecedented opportunities for AU. To provide a comprehensive overview of AC in the LLMs era from an NLP perspective, we summarize the development of LLMs research in this field, aiming to offer new insights. Specifically, we first summarize the traditional tasks related to AC and introduce the preliminary study based on LLMs. Subsequently, we outline the relevant techniques of popular LLMs to improve AC tasks, including Instruction Tuning and Prompt Engineering. For Instruction Tuning, we discuss full parameter fine-tuning and parameter-efficient methods such as LoRA, P-Tuning, and Prompt Tuning. In Prompt Engineering, we examine Zero-shot, Few-shot, Chain of Thought (CoT), and Agent-based methods for AU and AG. To clearly understand the performance of LLMs on different Affective Computing tasks, we further summarize the existing benchmarks and evaluation methods.
A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?
Authors: Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuxin Zhang, Ju Fan, Guoliang Li, Nan Tang, Yuyu Luo
Date: 2024-08-09 00:00:00
Tasks: Natural Language Queries, Text to SQL, Text-To-SQL

Translating users' natural language queries (NL) into SQL queries (i.e., Text-to-SQL, a.k.a. NL2SQL) can significantly reduce barriers to accessing relational databases and support various commercial applications. The performance of Text-to-SQL has been greatly enhanced with the emergence of Large Language Models (LLMs). In this survey, we provide a comprehensive review of Text-to-SQL techniques powered by LLMs, covering its entire lifecycle from the following four aspects: (1) Model: Text-to-SQL translation techniques that tackle not only NL ambiguity and under-specification, but also properly map NL with database schema and instances; (2) Data: From the collection of training data, data synthesis due to training data scarcity, to Text-to-SQL benchmarks; (3) Evaluation: Evaluating Text-to-SQL methods from multiple angles using different metrics and granularities; and (4) Error Analysis: analyzing Text-to-SQL errors to find the root cause and guiding Text-to-SQL models to evolve. Moreover, we offer a rule of thumb for developing Text-to-SQL solutions. Finally, we discuss the research challenges and open problems of Text-to-SQL in the LLMs era.
Prioritizing Modalities: Flexible Importance Scheduling in Federated Multimodal Learning
Authors: Jieming Bian, Lei Wang, Jie Xu
Date: 2024-08-13 00:00:00
Tasks: Computational Efficiency, Deep Reinforcement Learning, Federated Learning, Scheduling

Federated Learning (FL) is a distributed machine learning approach that enables devices to collaboratively train models without sharing their local data, ensuring user privacy and scalability. However, applying FL to real-world data presents challenges, particularly as most existing FL research focuses on unimodal data. Multimodal Federated Learning (MFL) has emerged to address these challenges, leveraging modality-specific encoder models to process diverse datasets. Current MFL methods often uniformly allocate computational frequencies across all modalities, which is inefficient for IoT devices with limited resources. In this paper, we propose FlexMod, a novel approach to enhance computational efficiency in MFL by adaptively allocating training resources for each modality encoder based on their importance and training requirements. We employ prototype learning to assess the quality of modality encoders, use Shapley values to quantify the importance of each modality, and adopt the Deep Deterministic Policy Gradient (DDPG) method from deep reinforcement learning to optimize the allocation of training resources. Our method prioritizes critical modalities, optimizing model performance and resource utilization. Experimental results on three real-world datasets demonstrate that our proposed method significantly improves the performance of MFL models.
Economic Struggles and Inflation: How Does that affect voting decision?
Authors: Muhammad Hassan Bin Afzal
Date: 2024-07-27 00:00:00

Economic hardships significantly affect public perception and voting intentions in general elections. The primary focus of my study is to capture the degree of influence that individual economic hardships have on their voting. I utilize the ANES 2024 Pilot Study1 Survey dataset and introduce a novel composite Inflation Behavior Index (IBR) that captures individuals' cumulative economic and cost of living experience. To that effect, the primary objectives of the current study are threefold: first, to develop a composite economic behavior index from available data and variables to capture the overall economic experience of U.S. individuals due to ongoing inflation; second, to examine how this economic behavior impacts political engagement and voting behavior utilizing appropriate and fitting mathematical models; and finally which specific personal experiences and perceptions about economy and cost of living likely to revoke party loyalty in upcoming U.S. presidential election. My study finds that increased personal economic struggles (pocketbook voting) due to inflation make it more likely for individuals to express an intention to vote against the Incumbent even if the Incumbent is from their self-identified political party. Conversely, having a negative perception of the national economy (sociotropic voting) is less likely to revoke party loyalty in the upcoming General election. In simpler terms, voters are more likely to vote along party lines even if they perceive their party (the Incumbent) is not handling the economy and cost of living well.
Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools
Authors: Jung In Park, Mahyar Abbasian, Iman Azimi, Dawn T. Bounds, Angela Jun, Jaesu Han, Robert M. McCarron, Jessica Borelli, Parmida Safavi, Sanaz Mirbaha, Jia Li, Mona Mahmoudi, Carmen Wiedenhoeft, Amir M. Rahmani
Date: 2024-08-03 00:00:00
Tasks: Chatbot, Large Language Model

Objective: This study aims to develop and validate an evaluation framework to ensure the safety and reliability of mental health chatbots, which are increasingly popular due to their accessibility, human-like interactions, and context-aware support. Materials and Methods: We created an evaluation framework with 100 benchmark questions and ideal responses, and five guideline questions for chatbot responses. This framework, validated by mental health experts, was tested on a GPT-3.5-turbo-based chatbot. Automated evaluation methods explored included large language model (LLM)-based scoring, an agentic approach using real-time data, and embedding models to compare chatbot responses against ground truth standards. Results: The results highlight the importance of guidelines and ground truth for improving LLM evaluation accuracy. The agentic method, dynamically accessing reliable information, demonstrated the best alignment with human assessments. Adherence to a standardized, expert-validated framework significantly enhanced chatbot response safety and reliability. Discussion: Our findings emphasize the need for comprehensive, expert-tailored safety evaluation metrics for mental health chatbots. While LLMs have significant potential, careful implementation is necessary to mitigate risks. The superior performance of the agentic approach underscores the importance of real-time data access in enhancing chatbot reliability. Conclusion: The study validated an evaluation framework for mental health chatbots, proving its effectiveness in improving safety and reliability. Future work should extend evaluations to accuracy, bias, empathy, and privacy to ensure holistic assessment and responsible integration into healthcare. Standardized evaluations will build trust among users and professionals, facilitating broader adoption and improved mental health support through technology.
Stochastic Geometry Analysis of RIS-Assisted Cellular Networks with Reflective Intelligent Surfaces on Roads
Authors: Chang-Sik Choi, Junhyeong Kim, Junil Choi
Date: 2024-08-09 00:00:00
Tasks: Point Processes

Reconfigurable intelligent surfaces (RISs) provide alternative routes for reflected signals to network users, offering numerous applications. This paper explores an innovative approach of strategically deploying RISs along road areas to leverage various propagation and blockage conditions present in cellular networks with roads. To address the local network geometries shown by such networks, we use a stochastic geometry framework, specifically the Cox point processes, to model the locations of RISs and vehicle users. Then, we define the coverage probability as the chance that either a base station or an RIS is in line of sight (LOS) of the typical user and that the LOS signal has a signal-to-noise ratio (SNR) greater than a threshold. We derive the coverage probability as a function of key parameters such as RIS density and path loss exponent. We observe that the network geometry highly affects the coverage and that the proposed RIS deployment effectively leverages the underlying difference of attenuation and blockage, significantly increasing the coverage of vehicle users in the network. With experimental results addressing the impact of key variables to network performance, this work serves as a versatile tool for designing, analyzing, and optimizing RIS-assisted cellular networks with many vehicles.
Source Separation of Multi-source Raw Music using a Residual Quantized Variational Autoencoder
Authors: Leonardo Berti
Date: 2024-08-12 00:00:00

I developed a neural audio codec model based on the residual quantized variational autoencoder architecture. I train the model on the Slakh2100 dataset, a standard dataset for musical source separation, composed of multi-track audio. The model can separate audio sources, achieving almost SoTA results with much less computing power. The code is publicly available at github.com/LeonardoBerti00/Source-Separation-of-Multi-source-Music-using-Residual-Quantizad-Variational-Autoencoder
Reinsurance with neural networks
Authors: Aleksandar Arandjelović, Julia Eisenberg
Date: 2024-08-12 00:00:00

We consider an insurance company which faces financial risk in the form of insurance claims and market-dependent surplus fluctuations. The company aims to simultaneously control its terminal wealth (e.g. at the end of an accounting period) and the ruin probability in a finite time interval by purchasing reinsurance. The target functional is given by the expected utility of terminal wealth perturbed by a modified Gerber-Shiu penalty function. We solve the problem of finding the optimal reinsurance strategy and the corresponding maximal target functional via neural networks. The procedure is illustrated by a numerical example, where the surplus process is given by a Cram\'er-Lundberg model perturbed by a mean-reverting Ornstein-Uhlenbeck process.
Telecom Foundation Models: Applications, Challenges, and Future Trends
Authors: Tahar Zanouda, Meysam Masoudi, Fitsum Gaim Gebre, Mischa Dohler
Date: 2024-08-02 00:00:00

Telecom networks are becoming increasingly complex, with diversified deployment scenarios, multi-standards, and multi-vendor support. The intricate nature of the telecom network ecosystem presents challenges to effectively manage, operate, and optimize networks. To address these hurdles, Artificial Intelligence (AI) has been widely adopted to solve different tasks in telecom networks. However, these conventional AI models are often designed for specific tasks, rely on extensive and costly-to-collect labeled data that require specialized telecom expertise for development and maintenance. The AI models usually fail to generalize and support diverse deployment scenarios and applications. In contrast, Foundation Models (FMs) show effective generalization capabilities in various domains in language, vision, and decision-making tasks. FMs can be trained on multiple data modalities generated from the telecom ecosystem and leverage specialized domain knowledge. Moreover, FMs can be fine-tuned to solve numerous specialized tasks with minimal task-specific labeled data and, in some instances, are able to leverage context to solve previously unseen problems. At the dawn of 6G, this paper investigates the potential opportunities of using FMs to shape the future of telecom technologies and standards. In particular, the paper outlines a conceptual process for developing Telecom FMs (TFMs) and discusses emerging opportunities for orchestrating specialized TFMs for network configuration, operation, and maintenance. Finally, the paper discusses the limitations and challenges of developing and deploying TFMs.