# A Novel Framework Leveraging Large Language Models to Enhance Cold-Start Advertising Systems
## 4 Validation Methodology
### Dataset Selection for Validation of Stages 1 and 2  
To validate the computational components of our framework (transformer-enhanced feature representation and ensemble model integration), we identified and selected two established datasets from the computational advertising domain that are widely accepted by the scientific community for CTR prediction research. The first dataset selected for validation is the Avazu dataset ([[https://www.kaggle.com/c/avazu-ctr-prediction/overview](https://www.kaggle.com/c/avazu-ctr-prediction/overview)]([https://www.kaggle.com/c/avazu-ctr-prediction/overview](https://www.kaggle.com/c/avazu-ctr-prediction/overview)) \begin{table}
\begin{tabular}{l l} \hline \hline You are an expert copywriter and evaluator for targeted advertising. Given a user profile [attributes: location, time, website type etc.] and a list of five advertisements [categories, descriptions], perform the following: \\
1. Select the best-matching ad by evaluating: - Contextual Relevance: Prioritize ads whose metadata (description, category) closely aligns with user attributes, using semantic understanding for sparse data. - Recommendation Diversification: Favor ads from distinct categories to avoid redundancy. - Sentiment Alignment: Prioritize ads with positive sentiment matching user preferences. - Bias Mitigation: Ensure balanced category representation, penalizing overrepresented categories. - Real-Time Adaptation: If feedback signals [e.g., clicks, dwell time] are provided, adjust selection to favor ads with positive feedback. \\
2. Craft a compelling one-sentence ad message (max 50 words) for the best-matched ad, tailored to the user’s profile while preserving the ad’s core proposition. \\
3. Provide a one-sentence justification (20-30 words) explaining why the best-matched ad was selected, referencing relevant criteria (e.g., relevance, sentiment). \\
4. For each of the four remaining ads, generate an alternative one-sentence ad message (max 50 words) and a concise reason (10-20 words) explaining why it was not selected. \\  
Output format:  
1: \"Best-fit ad message\". (Reason: Why this ad was selected based on relevance, sentiment, diversity, fairness, feedback)
2: \"Alternative ad message\". (Reason: Why this ad was not selected)
3: \"Alternative ad message\". (Reason: Why this ad was not selected)
4: \"Alternative ad message\". (Reason: Why this ad was not selected)
5: \"Alternative ad message\". (Reason: Why this ad was not selected)  
Constraints:  
- Keep each ad message to one sentence (max 50 words).
- Keep reasons objective, relevant, and concise (20-30 words for best-match justification, 10-20 words for rejections).
- Use a consistent, professional tone across all entries.  
\end{table} 
Table 1: LLM prompt. (accessed on 13 January 2025)). This dataset provides a robust foundation for testing our approach due to its comprehensive representation of mobile advertising scenarios. The dataset contains nearly 1.1 million records with 23 discrete attributes, offering sufficient data volume for transformer training while maintaining manageable computational requirements. A total of 150K random rows were used in the experiments to reduce training overhead. Based on relevance criteria for frozen-start scenarios, to simulate a case like this, the following attributes were selected as model inputs:  
* Temporal attributes: Time period for capturing temporal patterns;
* Contextual information: Application and site category and domain, application name and ID for content understanding;
* Placement characteristics: Advertisement position indicating display location;
* Device specifications: Device type and model for user profiling;
* Network conditions: Connection type for contextual awareness;  
All selected variables are categorical in nature, with the binary 'click' variable (0 or 1) serving as the prediction target.