# A Novel Framework Leveraging Large Language Models to Enhance Cold-Start Advertising Systems
## 1 Introduction  
The CTR prediction problem is central to personalized advertising systems, enabling real-time ad ranking and selection so as to maximize the click probability from a user. Even small improvements in prediction accuracy can yield measurable gains in conversions and revenue [1, 2, 3]. Deep learning models have transformed CTR prediction in recent years, using embeddings and deep Neural Nets (NNs) to model complex interactions and outperform traditional approaches like logistic regression [4, 5]. A persistent limitation of deep CTR models is their reliance on historical data. In cold-start scenarios, common issues involve new users or ads lacking interaction history,or conventional deep learning methods that fail to personalize effectively [6]. This issue becomes even more critical in a special cold-start case when there is no historical data available, called frozen-start. Frozen-start refers to the challenge of providing accurate recommendations for new users/items with no historical interaction data [3]. Common industry workarounds include default profiles, content-based heuristics, and exploration methods to bootstrap data [7]. Yet, these models still falter with new IDs, as embeddings for unseen users or ads are often random or default, limiting model effectiveness in cold-start cases. To address the persistent cold-start problem in click-through rate (CTR) prediction, recent studies have increasingly focused on transformer-based architectures, which have demonstrated strong capabilities in sequence modeling and contextual representation learning [8; 9]. Drawing inspiration from natural language-processing models such as BERT, these approaches enable the dynamic embedding of user behavior and advertisement content, thereby facilitating the modeling of temporal shifts in user interests and improving the interpretation of raw sequential data [10]. Notably, Meta's implementation of a sequence-based ad recommender system yielded a 2-4% improvement in conversion rates, while large-scale language models that encode ad textual content have reported CTR gains of up to 2% [11]. The attention mechanisms inherent in transformers further enhance the modeling of complex feature interactions, positioning them as particularly effective tools for sparse-data contexts such as cold-start scenarios [12]. Moreover, large language models (LLMs), including GPT-4 and Gemini, have shown considerable potential in cold-start recommendation settings by leveraging their extensive world knowledge and zero-shot reasoning capabilities. These models can infer user preferences directly from textual descriptions, making them well-suited for content-based personalization when behavioral data is unavailable [13]. These developments collectively suggest that the integration of LLMs into advertising systems represents a promising research frontier. However, such integration presents non-trivial challenges: LLMs are computationally demanding and, in their default state, lack the domain-specific alignment necessary for effective personalization without additional adaptation. In response to these challenges, this work proposes a hybrid framework that integrates a deep CTR prediction model with an LLM-based semantic encoder to enhance performance in frozen-start advertising scenarios. This work presents a comprehensive three-stage implementation architecture that addresses frozen-start challenges in advertising platforms through the strategic integration of transformer-based CTR prediction with LLM capabilities. The first stage employs transformer-enhanced feature representation with self-attention mechanisms to capture semantic relationships among sparse user attributes and advertisement content. The second stage implements an ensemble methodology combining four transformer-enhanced models through learnable weighted aggregation to generate robust predictions and select top-performing candidates.