pwc-archive/papers-with-abstracts huggingface dataset:
StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction
Authors: Shengkun Wang, Taoran Ji, Linhan Wang, Yanshen Sun, Shang-Ching Liu, Amit Kumar, Chang-Tien Lu
Date: 2024-08-25 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Stock Price Prediction, Time Series

The stock price prediction task holds a significant role in the financial domain and has been studied for a long time. Recently, large language models (LLMs) have brought new ways to improve these predictions. While recent financial large language models (FinLLMs) have shown considerable progress in financial NLP tasks compared to smaller pre-trained language models (PLMs), challenges persist in stock price forecasting. Firstly, effectively integrating the modalities of time series data and natural language to fully leverage these capabilities remains complex. Secondly, FinLLMs focus more on analysis and interpretability, which can overlook the essential features of time series data. Moreover, due to the abundance of false and redundant information in financial markets, models often produce less accurate predictions when faced with such input data. In this paper, we introduce StockTime, a novel LLM-based architecture designed specifically for stock price data. Unlike recent FinLLMs, StockTime is specifically designed for stock price time series data. It leverages the natural ability of LLMs to predict the next token by treating stock prices as consecutive tokens, extracting textual information such as stock correlations, statistical trends and timestamps directly from these stock prices. StockTime then integrates both textual and time series data into the embedding space. By fusing this multimodal data, StockTime effectively predicts stock prices across arbitrary look-back periods. Our experiments demonstrate that StockTime outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs.
AI Meets the Classroom: When Do Large Language Models Harm Learning?
Authors: Matthias Lehmann, Philipp B. Cornelius, Fabian J. Sting
Date: 2024-08-29 00:00:00

The effect of large language models (LLMs) in education is debated: Previous research shows that LLMs can help as well as hurt learning. In two pre-registered and incentivized laboratory experiments, we find no effect of LLMs on overall learning outcomes. In exploratory analyses and a field study, we provide evidence that the effect of LLMs on learning outcomes depends on usage behavior. Students who substitute some of their learning activities with LLMs (e.g., by generating solutions to exercises) increase the volume of topics they can learn about but decrease their understanding of each topic. Students who complement their learning activities with LLMs (e.g., by asking for explanations) do not increase topic volume but do increase their understanding. We also observe that LLMs widen the gap between students with low and high prior knowledge. While LLMs show great potential to improve learning, their use must be tailored to the educational context and students' needs.
United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections
Authors: Leah von der Heyde, Anna-Carolina Haensch, Alexander Wenz, Bolei Ma
Date: 2024-08-29 00:00:00
Tasks: Diversity

"Synthetic samples" based on large language models (LLMs) have been argued to serve as efficient alternatives to surveys of humans, assuming that their training data includes information on human attitudes and behavior. However, LLM-synthetic samples might exhibit bias, for example due to training data and fine-tuning processes being unrepresentative of diverse contexts. Such biases risk reinforcing existing biases in research, policymaking, and society. Therefore, researchers need to investigate if and under which conditions LLM-generated synthetic samples can be used for public opinion prediction. In this study, we examine to what extent LLM-based predictions of individual public opinion exhibit context-dependent biases by predicting the results of the 2024 European Parliament elections. Prompting three LLMs with individual-level background information of 26,000 eligible European voters, we ask the LLMs to predict each person's voting behavior. By comparing them to the actual results, we show that LLM-based predictions of future voting behavior largely fail, their accuracy is unequally distributed across national and linguistic contexts, and they require detailed attitudinal information in the prompt. The findings emphasize the limited applicability of LLM-synthetic samples to public opinion prediction. In investigating their contextual biases, this study contributes to the understanding and mitigation of inequalities in the development of LLMs and their applications in computational social science.
Acceptable Use Policies for Foundation Models
Authors: Kevin Klyman
Date: 2024-08-29 00:00:00

As foundation models have accumulated hundreds of millions of users, developers have begun to take steps to prevent harmful types of uses. One salient intervention that foundation model developers adopt is acceptable use policies: legally binding policies that prohibit users from using a model for specific purposes. This paper identifies acceptable use policies from 30 foundation model developers, analyzes the use restrictions they contain, and argues that acceptable use policies are an important lens for understanding the regulation of foundation models. Taken together, developers' acceptable use policies include 127 distinct use restrictions; the wide variety in the number and type of use restrictions may create fragmentation across the AI supply chain. Developers also employ acceptable use policies to prevent competitors or specific industries from making use of their models. Developers alone decide what constitutes acceptable use, and rarely provide transparency about how they enforce their policies. In practice, acceptable use policies are difficult to enforce, and scrupulous enforcement can act as a barrier to researcher access and limit beneficial uses of foundation models. Nevertheless, acceptable use policies for foundation models are an early example of self-regulation that have a significant impact on the market for foundation models and the overall AI ecosystem.
Inf-MLLM: Efficient Streaming Inference of Multimodal Large Language Models on a Single GPU
Authors: Zhenyu Ning, Jieru Zhao, Qihao Jin, Wenchao Ding, Minyi Guo
Date: 2024-09-11 00:00:00
Tasks: Autonomous Driving, GPU

Multimodal Large Language Models (MLLMs) are distinguished by their multimodal comprehensive ability and widely used in many real-world applications including GPT-4o, autonomous driving and robotics. Despite their impressive performance, the multimodal inputs always incur long context. The inference under long context requires caching massive Key and Value states (KV cache) of previous tokens, which introduces high latency and excessive memory consumption. Due to this reason, it is challenging to deploy streaming inference of MLLMs on edge devices, which largely constrains the power and usage of MLLMs in real-world applications. In this paper, we introduce Inf-MLLM, an efficient inference framework for MLLMs, which enable streaming inference of MLLM on a single GPU with infinite context. Inf-MLLM is based on our key observation of the attention pattern in both LLMs and MLLMs called "attention saddles". Thanks to the newly discovered attention pattern, Inf-MLLM maintains a size-constrained KV cache by dynamically caching recent tokens and relevant tokens. Furthermore, Inf-MLLM proposes attention bias, a novel approach to enable MLLMs to capture long-term dependency. We show that Inf-MLLM enables multiple LLMs and MLLMs to achieve stable performance over 4M-token long texts and multi-round conversations with 1-hour-long videos on a single GPU. In addition, Inf-MLLM exhibits superior streaming reasoning quality than existing methods such as StreamingLLM and 2x speedup than H2O.
Distributed Convolutional Neural Network Training on Mobile and Edge Clusters
Authors: Pranav Rama, Madison Threadgill, Andreas Gerstlauer
Date: 2024-09-11 00:00:00
Tasks: object-detection, Object Detection, Quantization, Transfer Learning

The training of deep and/or convolutional neural networks (DNNs/CNNs) is traditionally done on servers with powerful CPUs and GPUs. Recent efforts have emerged to localize machine learning tasks fully on the edge. This brings advantages in reduced latency and increased privacy, but necessitates working with resource-constrained devices. Approaches for inference and training in mobile and edge devices based on pruning, quantization or incremental and transfer learning require trading off accuracy. Several works have explored distributing inference operations on mobile and edge clusters instead. However, there is limited literature on distributed training on the edge. Existing approaches all require a central, potentially powerful edge or cloud server for coordination or offloading. In this paper, we describe an approach for distributed CNN training exclusively on mobile and edge devices. Our approach is beneficial for the initial CNN layers that are feature map dominated. It is based on partitioning forward inference and back-propagation operations among devices through tiling and fusing to maximize locality and expose communication and memory-aware parallelism. We also introduce the concept of layer grouping to further fine-tune performance based on computation and communication trade-off. Results show that for a cluster of 2-6 quad-core Raspberry Pi3 devices, training of an object-detection CNN provides a 2x-15x speedup with respect to a single core and up to 8x reduction in memory usage per device, all without sacrificing accuracy. Grouping offers up to 1.5x speedup depending on the reference profile and batch size.
D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks
Authors: Rustam Guliyev, Aparajita Haldar, Hakan Ferhatosmanoglu
Date: 2024-09-10 00:00:00
Tasks: Graph Neural Network

Graph Neural Network (GNN) models on streaming graphs entail algorithmic challenges to continuously capture its dynamic state, as well as systems challenges to optimize latency, memory, and throughput during both inference and training. We present D3-GNN, the first distributed, hybrid-parallel, streaming GNN system designed to handle real-time graph updates under online query setting. Our system addresses data management, algorithmic, and systems challenges, enabling continuous capturing of the dynamic state of the graph and updating node representations with fault-tolerance and optimal latency, load-balance, and throughput. D3-GNN utilizes streaming GNN aggregators and an unrolled, distributed computation graph architecture to handle cascading graph updates. To counteract data skew and neighborhood explosion issues, we introduce inter-layer and intra-layer windowed forward pass solutions. Experiments on large-scale graph streams demonstrate that D3-GNN achieves high efficiency and scalability. Compared to DGL, D3-GNN achieves a significant throughput improvement of about 76x for streaming workloads. The windowed enhancement further reduces running times by around 10x and message volumes by up to 15x at higher parallelism.
Joint Model Assignment and Resource Allocation for Cost-Effective Mobile Generative Services
Authors: Shuangwei Gao, Peng Yang, Yuxin Kong, Feng Lyu, Ning Zhang
Date: 2024-09-09 00:00:00

Artificial Intelligence Generated Content (AIGC) services can efficiently satisfy user-specified content creation demands, but the high computational requirements pose various challenges to supporting mobile users at scale. In this paper, we present our design of an edge-enabled AIGC service provisioning system to properly assign computing tasks of generative models to edge servers, thereby improving overall user experience and reducing content generation latency. Specifically, once the edge server receives user requested task prompts, it dynamically assigns appropriate models and allocates computing resources based on features of each category of prompts. The generated contents are then delivered to users. The key to this system is a proposed probabilistic model assignment approach, which estimates the quality score of generated contents for each prompt based on category labels. Next, we introduce a heuristic algorithm that enables adaptive configuration of both generation steps and resource allocation, according to the various task requests received by each generative model on the edge.Simulation results demonstrate that the designed system can effectively enhance the quality of generated content by up to 4.7% while reducing response delay by up to 39.1% compared to benchmarks.
ELMS: Elasticized Large Language Models On Mobile Devices
Authors: Wangsong Yin, Rongjie Yi, Daliang Xu, Gang Huang, Mengwei Xu, Xuanzhe Liu
Date: 2024-09-08 00:00:00
Tasks: GPU, Language Modelling

On-device Large Language Models (LLMs) are revolutionizing mobile AI, enabling applications such as UI automation while addressing privacy concerns. Currently, the standard approach involves deploying a single, robust LLM as a universal solution for various applications, often referred to as LLM-as-a-Service (LLMaaS). However, this approach faces a significant system challenge: existing LLMs lack the flexibility to accommodate the diverse Service-Level Objectives (SLOs) regarding inference latency across different applications. To address this issue, we introduce ELMS, an on-device LLM service designed to provide elasticity in both the model and prompt dimensions of an LLMaaS. This system includes: A one-time neuron reordering technique, which utilizes the inherent permutation consistency within transformer models to create high-quality, elastic sub-models with minimal runtime switching costs. A dual-head compact language model, which efficiently refines prompts and coordinates the elastic adaptation between the model and the prompt. We have implemented this elastic on-device LLM service on several off-the-shelf (COTS) smartphones and evaluate ELMS using both standalone NLP/mobile-agent datasets and synthesized end-to-end traces. Across a range of SLOs, ELMS surpasses four strong baselines by up to 16.83% and 11.04% in absolute accuracy on average, with less than 1% Time-To-First-Token (TTFT) switching overhead, comparable memory usage, and fewer than 100 offline GPU hours.
TS-EoH: An Edge Server Task Scheduling Algorithm Based on Evolution of Heuristic
Authors: Wang Yatong, Pei Yuchen, Zhao Yuqi
Date: 2024-09-04 00:00:00
Tasks: Edge-computing, Scheduling

With the widespread adoption of 5G and Internet of Things (IoT) technologies, the low latency provided by edge computing has great importance for real-time processing. However, managing numerous simultaneous service requests poses a significant challenge to maintaining low latency. Current edge server task scheduling methods often fail to balance multiple optimization goals effectively. This paper introduces a novel task-scheduling approach based on Evolutionary Computing (EC) theory and heuristic algorithms. We model service requests as task sequences and evaluate various scheduling schemes during each evolutionary process using Large Language Models (LLMs) services. Experimental results show that our task-scheduling algorithm outperforms existing heuristic and traditional reinforcement learning methods. Additionally, we investigate the effects of different heuristic strategies and compare the evolutionary outcomes across various LLM services.
Redefining Data-Centric Design: A New Approach with a Domain Model and Core Data Ontology for Computational Systems
Authors: William Johnson, James Davis, Tara Kelly
Date: 2024-09-01 00:00:00

This paper presents an innovative data-centric paradigm for designing computational systems by introducing a new informatics domain model. The proposed model moves away from the conventional node-centric framework and focuses on data-centric categorization, using a multimodal approach that incorporates objects, events, concepts, and actions. By drawing on interdisciplinary research and establishing a foundational ontology based on these core elements, the model promotes semantic consistency and secure data handling across distributed ecosystems. We also explore the implementation of this model as an OWL 2 ontology, discuss its potential applications, and outline its scalability and future directions for research. This work aims to serve as a foundational guide for system designers and data architects in developing more secure, interoperable, and scalable data systems.
Fast DCT+: A Family of Fast Transforms Based on Rank-One Updates of the Path Graph
Authors: Samuel Fernández-Menduiña, Eduardo Pavez, Antonio Ortega
Date: 2024-09-13 00:00:00

This paper develops fast graph Fourier transform (GFT) algorithms with O(n log n) runtime complexity for rank-one updates of the path graph. We first show that several commonly-used audio and video coding transforms belong to this class of GFTs, which we denote by DCT+. Next, starting from an arbitrary generalized graph Laplacian and using rank-one perturbation theory, we provide a factorization for the GFT after perturbation. This factorization is our central result and reveals a progressive structure: we first apply the unperturbed Laplacian's GFT and then multiply the result by a Cauchy matrix. By specializing this decomposition to path graphs and exploiting the properties of Cauchy matrices, we show that Fast DCT+ algorithms exist. We also demonstrate that progressivity can speed up computations in applications involving multiple transforms related by rank-one perturbations (e.g., video coding) when combined with pruning strategies. Our results can be extended to other graphs and rank-k perturbations. Runtime analyses show that Fast DCT+ provides computational gains over the naive method for graph sizes larger than 64, with runtime approximately equal to that of 8 DCTs.
An integrated design of robust decentralized observer and controller for load frequency control
Authors: Xianxian Zhao, Jianglin Lan
Date: 2024-09-16 00:00:00

This paper focuses on designing completely decentralized load frequency control (LFC) for multi-area power systems to achieve global optimized performance. To this end, a new concept of integrated design is introduced for designing the decentralized LFC observers and controllers simultaneously off-line, by taking into account of the interactions between areas and the bidirectional effects between the local observer and controller in each area. The integrated design in this paper is realized via $H_\infty$ optimization with a single-step linear matrix inequality (LMI) formulation. The LMI regional eigenvalue assignment technique is further incorporated with $H_\infty$ optimization to improve the closed-loop system transient performance. A three-area power system is simulated to validate the superiority of the proposed integrated design over the conventional decentralized designs.
Risk-Aware Autonomous Driving with Linear Temporal Logic Specifications
Authors: Shuhao Qi, Zengjie Zhang, Zhiyong Sun, Sofie Haesaert
Date: 2024-09-15 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Decision Making

Humans naturally balance the risks of different concerns while driving, including traffic rule violations, minor accidents, and fatalities. However, achieving the same behavior in autonomous systems remains an open problem. This paper extends a risk metric that has been verified in human-like driving studies to encompass more complex driving scenarios specified by linear temporal logic (LTL) that go beyond just collision risks. This extension incorporates the timing and severity of events into LTL specifications, thereby reflecting a human-like risk awareness. Without sacrificing expressivity for traffic rules, we adopt LTL specifications composed of safety and co-safety formulas, allowing the control synthesis problem to be reformulated as a reachability problem. By leveraging occupation measures, we formulate a linear programming (LP) problem for this LTL-based risk metric. Consequently, the synthesized policy balances different types of risks, including not only collision risks but also traffic rule violations. The effectiveness of the proposed approach is validated by three typical traffic scenarios in the Carla simulator.
Wavenumber-Domain Near-Field Channel Estimation: Beyond the Fresnel Bound
Authors: Xufeng Guo, Yuanbin Chen, Ying Wang, Zhaocheng Wang, Chau Yuen
Date: 2024-09-16 00:00:00

In the near-field context, the Fresnel approximation is typically employed to mathematically represent solvable functions of spherical waves. However, these efforts may fail to take into account the significant increase in the lower limit of the Fresnel approximation, known as the Fresnel distance. The lower bound of the Fresnel approximation imposes a constraint that becomes more pronounced as the array size grows. Beyond this constraint, the validity of the Fresnel approximation is broken. As a potential solution, the wavenumber-domain paradigm characterizes the spherical wave using a spectrum composed of a series of linear orthogonal bases. However, this approach falls short of covering the effects of the array geometry, especially when using Gaussian-mixed-model (GMM)-based von Mises-Fisher distributions to approximate all spectra. To fill this gap, this paper introduces a novel wavenumber-domain ellipse fitting (WDEF) method to tackle these challenges. Particularly, the channel is accurately estimated in the near-field region, by maximizing the closed-form likelihood function of the wavenumber-domain spectrum conditioned on the scatterers' geometric parameters. Simulation results are provided to demonstrate the robustness of the proposed scheme against both the distance and angles of arrival.
Winning Strategy Templates for Stochastic Parity Games towards Permissive and Resilient Control
Authors: Kittiphon Phalakarn, Sasinee Pruekprasert, Ichiro Hasuo
Date: 2024-09-13 00:00:00

Stochastic games play an important role for many purposes such as the control of cyber-physical systems (CPS), where the controller and the environment are modeled as players. Conventional algorithms typically solve the game for a single winning strategy in order to develop a controller. However, in applications such as CPS control, permissive controllers are crucial as they allow the controlled system to adapt if additional constraints need to be imposed and also remain resilient to system changes at runtime. In this work, we generalize the concept of permissive winning strategy templates, introduced by Anand et al. at TACAS and CAV 2023 for deterministic games, to encompass stochastic games. These templates represent an infinite number of winning strategies and can adapt strategies to system changes efficiently. We focus on five key winning objectives -- safety, reachability, B\"uchi, co-B\"uchi, and parity -- and present algorithms to construct templates for each objective. In addition, we propose a novel method to extract a winning strategy from a template and provide discussions on template comparison.
Temporal Many-valued Conditional Logics: a Preliminary Report
Authors: Mario Alviano, Laura Giordano, Daniele Theseider Dupré
Date: 2024-09-06 00:00:00

In this paper we propose a many-valued temporal conditional logic. We start from a many-valued logic with typicality, and extend it with the temporal operators of the Linear Time Temporal Logic (LTL), thus providing a formalism which is able to capture the dynamics of a system, trough strict and defeasible temporal properties. We also consider an instantiation of the formalism for gradual argumentation.