# Advertising Popularity Feature Collaborative Recommendation Algorithm Based on Attention-LSTM Model

Authors: Yang Su; Xiangwei Kong; Guobao Liu
Journal: Security and Communication Networks
 (2021)

---

## Abstract

To accurately predict the click-through rate (CTR) and use it for ad recommendation, we propose a deep attention AD popularity prediction model (DAFCT) based on label recommendation technology and collaborative filtering method, which integrates content features and temporal information. First, we construct an Attention-LSTM model to capture the popularity trends and exploit the temporal information based on users’ feedback; finally, we use the concatenate method to fuse temporal information and content features and design a Deep Attention Popularity Prediction (DAVPP) algorithm to solve DAFCT. We experimentally adjust the weighted composite similarity metric parameters of Query pages and verify the scalability of the algorithm. Experimental results on the KDDCUP2012 dataset show that this model collaborative filtering and recommendation algorithm has better scalability and better recommendation quality. Compared with the Attention-LSTM model and the NFM model, the F1 score of DAFCT is improved by 9.80 and 3.07 percentage points, respectively.


---

## Body

## 1. Introduction

At present, global Internet advertising has a scale of hundreds of billions of dollars, and search advertising has become an important form of online advertising [1]. The advertising industry has gradually developed from targeted delivery, where the value of ad delivery can be precisely measured, with a user-friendly and advertiser-beneficial advertising market [2]. Search engine revenue (Revenue Per Search, RPS) is one of the important evaluation indicators of the success of the search advertising recommendation system, which can be reflected by the pricing method of search advertising (such as CPC, Cost Per Click) and the ability of ads to attract users to click (Click-ThroughRate, CTR); that is, RPS = CTR × CPC [3]. Therefore, it is important to predict CTR accurately and use it reasonably for ad recommendation.Predicting the popularity of ads in advance is an important part of many applications, such as recommendations, advertising, and information retrieval [4, 5]. By observing a large number of user feedback behaviors on YouTube and sites, we found that the popularity of some ads tends to increase after user feedback for a period of time after they are posted. To capture the dynamic change process of ad popularity, this paper first uses Recurrent Neural Network (RNN) to model and calculate the trend index of ads, and discretion the amount of likes and stomps of ads to transform the popularity prediction task into a classification problem; the popularity of ads is classified into “popular,” the popularity of advertisements is classified into “popular” and “unpopular” categories, and the content features of advertisements are modeled using a neural network model; finally, the popularity of advertisements and content features are fused to predict the popularity of advertisements [6].The macroscopic accumulation process based on the amount of user feedback to predict popularity has great practical value, and Long Short-Term Memory (LSTM) networks can effectively capture the change process of events [7], which is widely used in stock trend prediction [8], temperature change trend prediction [9], and depression trend prediction in medical research [10], and so on. LSTM networks can effectively capture the popularity trends of advertisements, and researchers have used LSTM networks to model and predict the popularity dynamics of advertisements and achieved good results [11]. Inspired by these research works, this paper adopts LSTM networks to model the popularity trends of advertisements and capture the trends of the popularity of advertisements.Social tagging is a representation of collective intelligence in the Web 2.0 era, which is a bridge between users and resources. Tagging system has been widely used not only in the fields of music, movies, and books [12] but also in advertising recommendation. At present, the tag ad recommendation method mainly obtains its recommendation rules by analyzing the relationship between tags (AD keywords), users, and resources (ads) [13]. For example, [14] proposes an improved FolkRank ad recommendation method, which iteratively computes to the user, resource, and tag tried to find the recommendation tags, where resources correspond to ads in the ad recommendation system and tags correspond to ad keywords. Agarwal et al. [15] analyze the similarity of users’ behavioral trajectories in different time gaps and uses the similarity between time gaps as the weight value to collaboratively recommend the ads viewed by users in different time gaps. Therefore, how to reasonably find the combination of tag recommendation technology and collaborative filtering technology and fully explore the relationship between ad keywords, Query pages, and ads is one of the keys to improve the quality of ad recommendation.A comprehensive analysis of the existing research results shows that the dynamic change process of popularity is difficult to capture, but the content features of advertisements have a great influence on the performance of popularity prediction models. At present, there are few research works that combine the change process of popularity and content features modeling. LSTM networks are efficient to be applied to model the dynamic change process of popularity [16] and can effectively capture the change trend of popularity. The deep learning-based model of NFM, for example, effectively combines linear second-order feature interactions and nonlinear higher-order feature interactions and has excellent model representation and generalization capabilities but cannot capture the trend of popularity changes of advertisements. In this paper, we propose DAFCT, which combines content features and temporal information of advertisements, has excellent feature representation and generalization ability and can capture the popularity trends of advertisements.The contributions of this paper are as follows:CTR and make rational use of it for advertising recommendation. Based on label recommendation technology and collaborative filtering method, DAFCT integrating content features and timing information is proposed.We use concatenate method to integrate temporal information and content features, and design a deep attention popularity prediction algorithm to solve DAFCT.The experimental results on KDDCUP2012 dataset demonstrate that the proposed model has better scalability and better recommendation quality.


## 2. Related Work

Attention-based LSTM networks reduce the reliance on external information [17] and have been widely used in text classification, sentiment analysis, and click-through rate prediction. He et al. [18] proposed a Deep Temporal Context Network to predict the popularity of posts by modeling the dynamic change process of the popularity of face book tweets, and the prediction results showed that the model has significant ability to predict the long-term popularity dynamics. Chen et al. [19] modeled the popularity of citations by introducing attention into the LSTM network to predict the popularity of citations with an accuracy of 85% on the computer science citation dataset. Lynch et al. [20] constructed time series information by analyzing feedback events such as Fork and Star of projects on Github, and used LSTM to model the time series information on Github and predict the popularity trend. Gao et al. [21] analyzed the number of macro events such as clicks, views, retreats, and likes of online articles and introduced the attention mechanism into the LSTM network to model the change process of WeChat articles over time to predict the popularity trend and further integrated the content features, so as to predict the popularity of WeChat articles.Deep learning-based popularity prediction models have been a hot research topic in academia and industry since 2016, and the Deep Neural Network (DNN) model proposed by [22], which is modeled based on content features such as image pixels and image descriptions, was initially applied to advertising click-through rate prediction and later widely used in popularity prediction studies. To improve the prediction performance, [23] improved the deep learning model and proposed a Neural Collaborative Filtering (NCF) model based on the neural network and experimentally verified that the prediction accuracy of NCF model reached 87.30%.Recently, deep learning-based models have been applied to user preference prediction [24], app popularity prediction [8], and movie popularity prediction [9]. Some of the deep learning models mainly analyze the effect of content features on prediction performance [25], while another part of more novel research work focuses on the performance of classification models [11], where one of the more famous deep learning-based classification models is the Neural Factorization Machine (NFM) [26], which has a better performance than traditional the more famous deep learning-based classification model is NFM, which has better feature representation than the traditional deep neural network models. Ma et al. [27] proposed a collaborative filtering ad recommendation algorithm without location bias, which takes into account the influence of ad position on CTR and uses the relevance of page and ad instead of user’s rating of the product.The aforementioned research results only modeled based on the content characteristics of individual items, ignoring the influence of popularity trends on the performance of the popularity prediction model.


## 3. Proposed Model

### 3.1. Attention-LSTM Model

In this paper, we use LSTM network to capture the trend of advertisement popularity and introduce the attention mechanism into LSTM network to reduce the interference of external factors, and construct the Attention-LSTM model based on the attention mechanism. To more graphically characterize the growth trend of popularity, this paper uses OA⁃L to represent the trend index of ad popularity, which is calculated by the Attention-LSTM model, and the Attention-LSTM model is shown in Figure1. By analyzing the feedback information of users’ likes, views, comments, etc., given the time interval t, the amount of user feedback over time is constructed as a time series di, and then the feedback series d1,d2,…,di is obtained, and the calculation formula of LSTM network is rewritten as(1)Gti=σWixt−1+Uict−1+Diht−1+biGtf=σWfxt−1+Ufct−1+Dfht−1+bfct=Gtfhht−1+Gti∗tanhWcxt−1+Dcht−1+bcGt°=σW0xt−1+U0ct−1+D°ht−1+b°ht=Gt0∗tanhct,where: Gti, Gtf, and Gt° are the input gate, forgetting gate, and output gate at a given time interval t, respectively; ht−1 is the hidden output at t − 1; xt−1 is the input at t − 1; ct−1 is the cell state at t − 1; and s is the sigmoid activation function.Figure 1
Attention-LSTM model.The computation process of the Attention-LSTM model is as follows：Given a certain time intervalt, the n hidden layer outputs are denoted as Hi(2)Hi=h1,h2,…,hn.These hidden layer outputshi after the softmax layer to obtain the attention weights:(3)ai=softmaxhi∗Gt°∗tanhct.Record the attention weight asAi:(4)Ai=a1,a2,⋯,an.Then the Attention-LSTM model outputs the popular trend at this time:(5)OA−L=aiGt0∗tanhct.From the above calculation, the value of OA⁃L is a number less than 1, which characterizes the trend index of a certain advertisement, and then the popularity of the advertisement is calculated by combining it with the prediction results of the NFM model. From the experimental results, the Attention-LSTM model can effectively capture the popularity trend of advertisements, and it is very helpful to improve the performance of the popularity prediction model [28].


### 3.2. NFM Model

The content of ads, including ad types and numerical information, usually provides useful information for popularity prediction and is one of the key factors affecting the popularity of ads. For different types of advertisements, different users have different preferences, and their feedback performance is very different [19, 29]. In this paper, we adopt NFM model to model the content features of advertisements; first, we use one-hot coding technique to convert the type features into one-hot vectors, then we input the one-hot vectors of ad types into the embedding layer of NFM model, and then the ad type features and numerical features are combined by a second-order feature interaction pool layer, which is input into the hidden layer to obtain the content features of advertisements. The NFM model combines linear second-order feature interactions and nonlinear higher-order feature interactions to learn features from sparse data, which effectively improve the feature representation capability. Figure 2 shows the NFM model for content feature learning, which includes three layers: input layer, embedding layer, and memory layer. The specific description is as follows. We describe each of its components in the form of data.Figure 2
NFM model.For example, given the set of ad types asM=m1,m2,…,mk, for the i (i = 1, 2,...,k) ad mi types, the one-hot feature vector x of the ad types is dimensioned down using the embedding technique to obtain the embedding vector representation of the video types:(6)νx=x1λ1,x2λ2,…,xnλn,where λi is the embedding vector for the i-th ad type, xi is the one-hot vector for the i-th ad type.Input the embedding vectorνx to the second-order interaction layer:(7)zBIνx=∑i=1n∑j=i+1nxiλiΘxjλj,where Θ means λiΘλjk=λikλjk, the second-order interaction layer is a pooling operation that converts the embedding matrix of text features into a vector. The output zBIνx of the second-order interaction layer is input to the hidden layer and is computed as(8)y1=σW1zBIνx+b1,y2=σW2y1+b2,⋮yL=σWLyL−1+bL,where σ, WL, and bL are the sigmoid function, the weight matrix, and bias vector of the hidden layer, respectively, and L is the number of layers of the hidden layer [30].The output of the NFM model is obtained by feeding the output of the hidden layeryL to the fully connected layer:(9)ONFM=w0+∑i=1nwixi+qTsoftmaxWLyL−1+bL,where w0+∑i=1nwixi is the linear part of the NFM model, w0 is the initialized weight, wi represents the weight of the i-th feature, and qTsoftmaxWLyL−1+bL is the nonlinear part, i.e., DNN, q is the weight matrix of the output layer.


### 3.3. DAFCT

The temporal information in the process of popularity change is difficult to capture, while the content features of ads largely determine the popularity of ads, which are an essential condition for the popularity prediction task [31]. In this paper, DAFCT first adopts RNN to mine the temporal information to capture the popularity trend of advertisements and introduces the attention mechanism to eliminate the interference of external factors; then adopts deep neural network to process the content features, and adopts embedding technique to reduce the computational complexity of the model for sparse and high-dimensional features; finally, the concatenate method is used to combine the temporal information and content features.Givenn ads, denote the popularity of the ad by pi and denote all n popularity pi as P:(10)P=p1,p2,…,pn,pi=11+e−z.Equation (11) is the linking probability of the ad, which is the popularity of the ad. Combining the popularity trend of the Attention-LSTM model and the output of the NFM model, a fully connected layer calculates(11)z=WOA−L,ONFM+b,where OA⁃L is the popularity trend index of the advertisement, and ONFM is the prediction result of the NFM model. After substitution into equation (12), we get the prevalence:(12)pi=11+e−WOΛ−1⋅oNMF+b.


### 3.4. Collaborative Filtering Ad Recommendation Algorithm

Based on the model design, the collaborative filtering and recommendation algorithms in this paper is described in Algorithm1.Algorithm 1: Collaborative filtering ad recommendation algorithm with labels.
Input: Target Query pageqi (i = 1, 2, ..., m), Query page set Q(|Q | = r), ad keyword set K (|K| = n), ad set A, CTR set C, number of neighbors N.Output the best recommended ad set for the target Query pageqiA∗Step 1
For each Query pageqi in the set, 1 ≤ j ≤ Q, j ≠ 1, the loop performs the following operations.Step 2
Calculate the co-hit similarity between Query pagessimQAqi,qj.Step 3
Calculate the similarity of co-labeling between Query pagessimQkqi,qj.Step 4
Calculate the similarity of co-contained relationships between Query pagessimQkAqi,qj.Step 5
Calculate the combined similarity between Query pagessimqi,qj.Step 6
Sort the remaining objects in the set, except the target Query pageqi, from largest to smallest, according to simqi,qj.Step 7
select the topn query pages in the set as the nearest neighborhood Nqi of the target query page qi.Step 8
Select the top N ads with the highest predicted click-through rate in the set A’ as the TOP-N best recommended ad set A∗. The key time overhead of the ADR-CF_T algorithm is the similarity calculation between Query pages, and the time overhead for calculating the co-hit similarity SimQA between Query pages is the same as the traditional CF algorithm [32].


## 3.1. Attention-LSTM Model

In this paper, we use LSTM network to capture the trend of advertisement popularity and introduce the attention mechanism into LSTM network to reduce the interference of external factors, and construct the Attention-LSTM model based on the attention mechanism. To more graphically characterize the growth trend of popularity, this paper uses OA⁃L to represent the trend index of ad popularity, which is calculated by the Attention-LSTM model, and the Attention-LSTM model is shown in Figure1. By analyzing the feedback information of users’ likes, views, comments, etc., given the time interval t, the amount of user feedback over time is constructed as a time series di, and then the feedback series d1,d2,…,di is obtained, and the calculation formula of LSTM network is rewritten as(1)Gti=σWixt−1+Uict−1+Diht−1+biGtf=σWfxt−1+Ufct−1+Dfht−1+bfct=Gtfhht−1+Gti∗tanhWcxt−1+Dcht−1+bcGt°=σW0xt−1+U0ct−1+D°ht−1+b°ht=Gt0∗tanhct,where: Gti, Gtf, and Gt° are the input gate, forgetting gate, and output gate at a given time interval t, respectively; ht−1 is the hidden output at t − 1; xt−1 is the input at t − 1; ct−1 is the cell state at t − 1; and s is the sigmoid activation function.Figure 1
Attention-LSTM model.The computation process of the Attention-LSTM model is as follows：Given a certain time intervalt, the n hidden layer outputs are denoted as Hi(2)Hi=h1,h2,…,hn.These hidden layer outputshi after the softmax layer to obtain the attention weights:(3)ai=softmaxhi∗Gt°∗tanhct.Record the attention weight asAi:(4)Ai=a1,a2,⋯,an.Then the Attention-LSTM model outputs the popular trend at this time:(5)OA−L=aiGt0∗tanhct.From the above calculation, the value of OA⁃L is a number less than 1, which characterizes the trend index of a certain advertisement, and then the popularity of the advertisement is calculated by combining it with the prediction results of the NFM model. From the experimental results, the Attention-LSTM model can effectively capture the popularity trend of advertisements, and it is very helpful to improve the performance of the popularity prediction model [28].


## 3.2. NFM Model

The content of ads, including ad types and numerical information, usually provides useful information for popularity prediction and is one of the key factors affecting the popularity of ads. For different types of advertisements, different users have different preferences, and their feedback performance is very different [19, 29]. In this paper, we adopt NFM model to model the content features of advertisements; first, we use one-hot coding technique to convert the type features into one-hot vectors, then we input the one-hot vectors of ad types into the embedding layer of NFM model, and then the ad type features and numerical features are combined by a second-order feature interaction pool layer, which is input into the hidden layer to obtain the content features of advertisements. The NFM model combines linear second-order feature interactions and nonlinear higher-order feature interactions to learn features from sparse data, which effectively improve the feature representation capability. Figure 2 shows the NFM model for content feature learning, which includes three layers: input layer, embedding layer, and memory layer. The specific description is as follows. We describe each of its components in the form of data.Figure 2
NFM model.For example, given the set of ad types asM=m1,m2,…,mk, for the i (i = 1, 2,...,k) ad mi types, the one-hot feature vector x of the ad types is dimensioned down using the embedding technique to obtain the embedding vector representation of the video types:(6)νx=x1λ1,x2λ2,…,xnλn,where λi is the embedding vector for the i-th ad type, xi is the one-hot vector for the i-th ad type.Input the embedding vectorνx to the second-order interaction layer:(7)zBIνx=∑i=1n∑j=i+1nxiλiΘxjλj,where Θ means λiΘλjk=λikλjk, the second-order interaction layer is a pooling operation that converts the embedding matrix of text features into a vector. The output zBIνx of the second-order interaction layer is input to the hidden layer and is computed as(8)y1=σW1zBIνx+b1,y2=σW2y1+b2,⋮yL=σWLyL−1+bL,where σ, WL, and bL are the sigmoid function, the weight matrix, and bias vector of the hidden layer, respectively, and L is the number of layers of the hidden layer [30].The output of the NFM model is obtained by feeding the output of the hidden layeryL to the fully connected layer:(9)ONFM=w0+∑i=1nwixi+qTsoftmaxWLyL−1+bL,where w0+∑i=1nwixi is the linear part of the NFM model, w0 is the initialized weight, wi represents the weight of the i-th feature, and qTsoftmaxWLyL−1+bL is the nonlinear part, i.e., DNN, q is the weight matrix of the output layer.


## 3.3. DAFCT

The temporal information in the process of popularity change is difficult to capture, while the content features of ads largely determine the popularity of ads, which are an essential condition for the popularity prediction task [31]. In this paper, DAFCT first adopts RNN to mine the temporal information to capture the popularity trend of advertisements and introduces the attention mechanism to eliminate the interference of external factors; then adopts deep neural network to process the content features, and adopts embedding technique to reduce the computational complexity of the model for sparse and high-dimensional features; finally, the concatenate method is used to combine the temporal information and content features.Givenn ads, denote the popularity of the ad by pi and denote all n popularity pi as P:(10)P=p1,p2,…,pn,pi=11+e−z.Equation (11) is the linking probability of the ad, which is the popularity of the ad. Combining the popularity trend of the Attention-LSTM model and the output of the NFM model, a fully connected layer calculates(11)z=WOA−L,ONFM+b,where OA⁃L is the popularity trend index of the advertisement, and ONFM is the prediction result of the NFM model. After substitution into equation (12), we get the prevalence:(12)pi=11+e−WOΛ−1⋅oNMF+b.


## 3.4. Collaborative Filtering Ad Recommendation Algorithm

Based on the model design, the collaborative filtering and recommendation algorithms in this paper is described in Algorithm1.Algorithm 1: Collaborative filtering ad recommendation algorithm with labels.
Input: Target Query pageqi (i = 1, 2, ..., m), Query page set Q(|Q | = r), ad keyword set K (|K| = n), ad set A, CTR set C, number of neighbors N.Output the best recommended ad set for the target Query pageqiA∗Step 1
For each Query pageqi in the set, 1 ≤ j ≤ Q, j ≠ 1, the loop performs the following operations.Step 2
Calculate the co-hit similarity between Query pagessimQAqi,qj.Step 3
Calculate the similarity of co-labeling between Query pagessimQkqi,qj.Step 4
Calculate the similarity of co-contained relationships between Query pagessimQkAqi,qj.Step 5
Calculate the combined similarity between Query pagessimqi,qj.Step 6
Sort the remaining objects in the set, except the target Query pageqi, from largest to smallest, according to simqi,qj.Step 7
select the topn query pages in the set as the nearest neighborhood Nqi of the target query page qi.Step 8
Select the top N ads with the highest predicted click-through rate in the set A’ as the TOP-N best recommended ad set A∗. The key time overhead of the ADR-CF_T algorithm is the similarity calculation between Query pages, and the time overhead for calculating the co-hit similarity SimQA between Query pages is the same as the traditional CF algorithm [32].


## 4. Experimental Results and Analysis

In this paper, the training dataset of track in KDDCUP2012 [33] is selected as the experimental data. This data provides the search advertisement click data of Tencent Soso, with a total size of 10.6 GB and 149,639,105 data. In this paper, the five attributes of data attributes, namely, Click, Impression, AdID, QueryID, and QueryID, are selected as the experiments of the search ad recommendation system. Impression, AdID, QueryID, KeywordID.


### 4.1. Data Preprocessing

In this paper, we first randomly sampled the original data and selected 1,000,000 pieces of data; according to the data requirements of this experiment, we deleted 7 other attribute columns and got 641,566 pieces of data after removing duplicate items. To avoid the serious data sparsity problem, we select the Query pages and advertisements with at least 30 clock records, leaving 19,436 data, including 10,936 Query pages, 8,789 advertisements, and 10,439 advertising keywords. In each Query page, 80% of the ads are randomly selected as the training set, and the remaining data are used as the test set.


### 4.2. Evaluation Metrics

Since the Top-N recommendation approach is adopted, Precision, Recall, and F-measure calculated for different number of neighbors are used to evaluate the quality of the search ad recommendation system in this paper. They define as following [11, 13, 14], respectively.


### 4.3. Analysis of Results

#### 4.3.1. Parameter Adjustment

In the collaborative filtering ad recommendation algorithm with labels, the key similarity calculation method is to weight the co-hit similarity, co-matched label similarity, and co-contained relationship similarity among Query pages to make the similarity calculation more accurate. In this paper, we select 10%, 20%, and 30% data sets for experiments, respectively, and observe the changes of MAE (α, β) by taking values of α and ß iteratively, and weigh the weight of each similarity measure. Considering α + β + γ = 1, only α and ß are taken as dependent variables, and the experimental results are shown in Figures 3–5.Figure 3
Effect of 10% dataset weight adjustment factor on MAE.Figure 4
Effect of weight adjustment factor on MAE for 20% data set.Figure 5
Effect of weight adjustment factor on MAE for 30% data set.Figures3–5, we can see that the variation of α and ß can affect the prediction accuracy of the ad recommendation algorithm, and the performance of the proposed collaborative filtering ad recommendation algorithm with labels is optimal when 0.2 ＜ α ＜ 0.4 and 0.4 ＜ ß ＜ 0.6. In this paper, the optimal values of α, β, and γ are selected as 0.2, 0.4, and 0.4, respectively.


### 4.4. Scalability Verification

To test the scalability performance of ADR-CF_T algorithm, this paper compares the execution time of 20%, 40%, 60%, and 80% of the data set size with the overall data set by randomly selecting the data set size, and the experimental results are shown in Figure6.Figure 6
Relationship between data set size and execution time.From Figure6, it can be seen that as the data size increases, the execution time of the algorithm changes from slowly increasing to sharply increasing and gradually and smoothly increasing. It can be seen that the growth of the execution time of the collaborative filtering ad recommendation algorithm with labels is within an acceptable range when the data size increases, so the algorithm has good scalability.


### 4.5. Recommended Quality Comparison Experiment

In this paper, the data set is divided into two parts: a training set and test set, where the training set accounts for 80% and the test set accounts for 20%. The recommendation list is output by Top-N, and the accuracy, recall, and F-measure is used to evaluate the recommendation quality of the experiment. To more obviously show the effectiveness of the collaborative filtering ad recommendation algorithm with labels proposed in this paper, the weight reconciliation factorsα, β, and γ are adjusted to 1 respectively, that is, the user-based collaborative ad recommendation algorithm [3], the label-based ad recommendation algorithm [15], and the label-item relationship-based ad recommendation algorithm [16] are obtained. To compare the recommendation quality of the above three algorithms and the proposed collaborative filtering ad recommendation algorithm with labels, three sets of experiments are designed in this paper: recommendation quality comparison of each algorithm for TOP5 recommendation, recommendation quality comparison of each algorithm with different N values, and recommendation quality optimization degree comparison.


#### 4.5.1. TOP5 Comparison of Recommendation Quality of Each Algorithm

In this paper, the proposed collaborative filtering ad recommendation algorithm with tags is compared with the user-based collaborative ad recommendation algorithm, the tag-based ad recommendation algorithm, and the ad recommendation algorithm based on the relationship between tags and items in terms of accuracy, recall, and F-measure value. The experimental results are shown in Table1 and Figure 7.Table 1
Results of experimental evaluation metrics for TOP5 recommendations of four algorithms.
AlgorithmAccuracyRecallF-MeasureUser-basedCF0.0250.06120.035TAG-based0.0310.07530.043TAG-IT–M-based0.0330.07650.046ADR-CF-T0.0380.07680.051Figure 7
Comparison of Top five recommendation quality of different recommendation algorithms.Through comparison, it is found that the collaborative filtering ad recommendation algorithm with labels proposed in this paper improves 52% in accuracy, 25% in recall, 46% in F-measure value, and nearly 41% in overall effectiveness over the traditional collaborative filtering algorithm. Since this paper considers the influence of three factors, namely CTR, ad keywords, and the relationship between ad keywords and ads, when calculating the similarity between Query pages, the integrated similarity calculation method can effectively reflect the preference information of Query pages for ads, the relevance of ad keywords for Query pages, and ads as well as the characteristics of the ads themselves for a more complete description. Meanwhile, the weight reconciliation factorsα, ß, and γ of the similarity metric proposed in this paper are found to have a greater impact on the prediction accuracy of the recommendation algorithm by analyzing their values.


#### 4.5.2. Comparison of the Recommendation Quality of Each Algorithm with DifferentN Values

The selection of the number of nearest neighbors also affects the recommendation quality of the recommendation algorithm; therefore, this paper compares the accuracy, recall, and F-measure of the user-based collaborative ad recommendation algorithm, the tag-based ad recommendation algorithm, the tag-item relationship-based ad recommendation algorithm, and the proposed collaborative ad recommendation algorithm with tags for the cases of 5, 10, 15, 20, 25, and 30, respectively. The accuracy, recall, and F-measure of the user-based collaborative ad recommendation algorithm, the label-based ad recommendation algorithm, the label-item relationship-based ad recommendation algorithm, and the collaborative filtering ad recommendation algorithm with labels proposed in this paper are compared, and the comparison results are shown in Figures8–10.Figure 8
Accuracy comparison with differentN values.Figure 9
Comparison of the recall rate with differentN values.Figure 10
F-metric values for differentN values.By comparison, the proposed collaborative filtering ad recommendation algorithm with labels improves the accuracy by at least 17%, the recall by at least 0.9%, and the F-measure by at least 21% compared with the other three algorithms when 25 ads are recommended for each page. As the number of nearest neighbors increases, the recommendation effect appears to decrease rather than increase. This is because the number of truly similar Query pages in the ad recommendation system is limited. When more dissimilar neighbors are selected, these Query pages show ads with higher click-through rates from the dissimilar Query pages, resulting in a decrease in recommendation quality.Therefore, only by correctly selecting similar Query pages as nearest neighbors in the ad recommendation system can we obtain the desired collaborative recommendation effect。


#### 4.5.3. Recommended Quality Optimization Degree Comparison

In this paper, the accuracy and recall of four recommendation algorithms are fitted with Gaussians to verify the degree of optimization of the recommendation results, corresponding to a Gaussian fitting function offx=a×exp−x−b/c2 and a thrust coefficient of 95%. The accuracy and recall Gaussian fitting curves of the recommendation results of each algorithm are shown in Figure 11.Figure 11
Gaussian fitting curves for accuracy and recall of recommendation results.The Gaussian fitted curves show that the user-based collaborative ad recommendation algorithm and the tag and item relationship-based ad recommendation algorithm have intersection points as the recall rate increases, and the coefficients provided in Table2 give the intersection point coordinates of (0.0472, 0.0168). In the interval [0.0472, 0.08], the accuracy of the user-based collaborative ad recommendation algorithm is higher than that of the ad recommendation algorithm based on the relationship between tags and items, and the degree of difference gradually decreases. With the increase of recall, the accuracy of the collaborative filtering ad recommendation algorithm with labels proposed in this paper is significantly higher than the other three algorithms, compared with the lowest accuracy of the label-based ad recommendation algorithm, which is consistent with the results of the previous experiment.Table 2
Coefficient selection and goodness-of-fit indices of different algorithms.
AlgorithmCoefficient 1Coefficient 2Coefficient 3Goodness of fit indexSSER-squareAdjustedRMSECF-based0.087320.15970.08774.38e −070.99790.99580.0004867TAG-based0.112400.19330.10342.468e −060.99100.98210.0011110TAG-item-based0.216500.26020.13332.488e −060.99100.98200.0011150ADR-cf_t0.087660.17050.10235.313e −070.99860.99700.0005154


## 4.1. Data Preprocessing

In this paper, we first randomly sampled the original data and selected 1,000,000 pieces of data; according to the data requirements of this experiment, we deleted 7 other attribute columns and got 641,566 pieces of data after removing duplicate items. To avoid the serious data sparsity problem, we select the Query pages and advertisements with at least 30 clock records, leaving 19,436 data, including 10,936 Query pages, 8,789 advertisements, and 10,439 advertising keywords. In each Query page, 80% of the ads are randomly selected as the training set, and the remaining data are used as the test set.


## 4.2. Evaluation Metrics

Since the Top-N recommendation approach is adopted, Precision, Recall, and F-measure calculated for different number of neighbors are used to evaluate the quality of the search ad recommendation system in this paper. They define as following [11, 13, 14], respectively.


## 4.3. Analysis of Results

### 4.3.1. Parameter Adjustment

In the collaborative filtering ad recommendation algorithm with labels, the key similarity calculation method is to weight the co-hit similarity, co-matched label similarity, and co-contained relationship similarity among Query pages to make the similarity calculation more accurate. In this paper, we select 10%, 20%, and 30% data sets for experiments, respectively, and observe the changes of MAE (α, β) by taking values of α and ß iteratively, and weigh the weight of each similarity measure. Considering α + β + γ = 1, only α and ß are taken as dependent variables, and the experimental results are shown in Figures 3–5.Figure 3
Effect of 10% dataset weight adjustment factor on MAE.Figure 4
Effect of weight adjustment factor on MAE for 20% data set.Figure 5
Effect of weight adjustment factor on MAE for 30% data set.Figures3–5, we can see that the variation of α and ß can affect the prediction accuracy of the ad recommendation algorithm, and the performance of the proposed collaborative filtering ad recommendation algorithm with labels is optimal when 0.2 ＜ α ＜ 0.4 and 0.4 ＜ ß ＜ 0.6. In this paper, the optimal values of α, β, and γ are selected as 0.2, 0.4, and 0.4, respectively.


## 4.3.1. Parameter Adjustment

In the collaborative filtering ad recommendation algorithm with labels, the key similarity calculation method is to weight the co-hit similarity, co-matched label similarity, and co-contained relationship similarity among Query pages to make the similarity calculation more accurate. In this paper, we select 10%, 20%, and 30% data sets for experiments, respectively, and observe the changes of MAE (α, β) by taking values of α and ß iteratively, and weigh the weight of each similarity measure. Considering α + β + γ = 1, only α and ß are taken as dependent variables, and the experimental results are shown in Figures 3–5.Figure 3
Effect of 10% dataset weight adjustment factor on MAE.Figure 4
Effect of weight adjustment factor on MAE for 20% data set.Figure 5
Effect of weight adjustment factor on MAE for 30% data set.Figures3–5, we can see that the variation of α and ß can affect the prediction accuracy of the ad recommendation algorithm, and the performance of the proposed collaborative filtering ad recommendation algorithm with labels is optimal when 0.2 ＜ α ＜ 0.4 and 0.4 ＜ ß ＜ 0.6. In this paper, the optimal values of α, β, and γ are selected as 0.2, 0.4, and 0.4, respectively.


## 4.4. Scalability Verification

To test the scalability performance of ADR-CF_T algorithm, this paper compares the execution time of 20%, 40%, 60%, and 80% of the data set size with the overall data set by randomly selecting the data set size, and the experimental results are shown in Figure6.Figure 6
Relationship between data set size and execution time.From Figure6, it can be seen that as the data size increases, the execution time of the algorithm changes from slowly increasing to sharply increasing and gradually and smoothly increasing. It can be seen that the growth of the execution time of the collaborative filtering ad recommendation algorithm with labels is within an acceptable range when the data size increases, so the algorithm has good scalability.


## 4.5. Recommended Quality Comparison Experiment

In this paper, the data set is divided into two parts: a training set and test set, where the training set accounts for 80% and the test set accounts for 20%. The recommendation list is output by Top-N, and the accuracy, recall, and F-measure is used to evaluate the recommendation quality of the experiment. To more obviously show the effectiveness of the collaborative filtering ad recommendation algorithm with labels proposed in this paper, the weight reconciliation factorsα, β, and γ are adjusted to 1 respectively, that is, the user-based collaborative ad recommendation algorithm [3], the label-based ad recommendation algorithm [15], and the label-item relationship-based ad recommendation algorithm [16] are obtained. To compare the recommendation quality of the above three algorithms and the proposed collaborative filtering ad recommendation algorithm with labels, three sets of experiments are designed in this paper: recommendation quality comparison of each algorithm for TOP5 recommendation, recommendation quality comparison of each algorithm with different N values, and recommendation quality optimization degree comparison.


### 4.5.1. TOP5 Comparison of Recommendation Quality of Each Algorithm

In this paper, the proposed collaborative filtering ad recommendation algorithm with tags is compared with the user-based collaborative ad recommendation algorithm, the tag-based ad recommendation algorithm, and the ad recommendation algorithm based on the relationship between tags and items in terms of accuracy, recall, and F-measure value. The experimental results are shown in Table1 and Figure 7.Table 1
Results of experimental evaluation metrics for TOP5 recommendations of four algorithms.
AlgorithmAccuracyRecallF-MeasureUser-basedCF0.0250.06120.035TAG-based0.0310.07530.043TAG-IT–M-based0.0330.07650.046ADR-CF-T0.0380.07680.051Figure 7
Comparison of Top five recommendation quality of different recommendation algorithms.Through comparison, it is found that the collaborative filtering ad recommendation algorithm with labels proposed in this paper improves 52% in accuracy, 25% in recall, 46% in F-measure value, and nearly 41% in overall effectiveness over the traditional collaborative filtering algorithm. Since this paper considers the influence of three factors, namely CTR, ad keywords, and the relationship between ad keywords and ads, when calculating the similarity between Query pages, the integrated similarity calculation method can effectively reflect the preference information of Query pages for ads, the relevance of ad keywords for Query pages, and ads as well as the characteristics of the ads themselves for a more complete description. Meanwhile, the weight reconciliation factorsα, ß, and γ of the similarity metric proposed in this paper are found to have a greater impact on the prediction accuracy of the recommendation algorithm by analyzing their values.


### 4.5.2. Comparison of the Recommendation Quality of Each Algorithm with DifferentN Values

The selection of the number of nearest neighbors also affects the recommendation quality of the recommendation algorithm; therefore, this paper compares the accuracy, recall, and F-measure of the user-based collaborative ad recommendation algorithm, the tag-based ad recommendation algorithm, the tag-item relationship-based ad recommendation algorithm, and the proposed collaborative ad recommendation algorithm with tags for the cases of 5, 10, 15, 20, 25, and 30, respectively. The accuracy, recall, and F-measure of the user-based collaborative ad recommendation algorithm, the label-based ad recommendation algorithm, the label-item relationship-based ad recommendation algorithm, and the collaborative filtering ad recommendation algorithm with labels proposed in this paper are compared, and the comparison results are shown in Figures8–10.Figure 8
Accuracy comparison with differentN values.Figure 9
Comparison of the recall rate with differentN values.Figure 10
F-metric values for differentN values.By comparison, the proposed collaborative filtering ad recommendation algorithm with labels improves the accuracy by at least 17%, the recall by at least 0.9%, and the F-measure by at least 21% compared with the other three algorithms when 25 ads are recommended for each page. As the number of nearest neighbors increases, the recommendation effect appears to decrease rather than increase. This is because the number of truly similar Query pages in the ad recommendation system is limited. When more dissimilar neighbors are selected, these Query pages show ads with higher click-through rates from the dissimilar Query pages, resulting in a decrease in recommendation quality.Therefore, only by correctly selecting similar Query pages as nearest neighbors in the ad recommendation system can we obtain the desired collaborative recommendation effect。


### 4.5.3. Recommended Quality Optimization Degree Comparison

In this paper, the accuracy and recall of four recommendation algorithms are fitted with Gaussians to verify the degree of optimization of the recommendation results, corresponding to a Gaussian fitting function offx=a×exp−x−b/c2 and a thrust coefficient of 95%. The accuracy and recall Gaussian fitting curves of the recommendation results of each algorithm are shown in Figure 11.Figure 11
Gaussian fitting curves for accuracy and recall of recommendation results.The Gaussian fitted curves show that the user-based collaborative ad recommendation algorithm and the tag and item relationship-based ad recommendation algorithm have intersection points as the recall rate increases, and the coefficients provided in Table2 give the intersection point coordinates of (0.0472, 0.0168). In the interval [0.0472, 0.08], the accuracy of the user-based collaborative ad recommendation algorithm is higher than that of the ad recommendation algorithm based on the relationship between tags and items, and the degree of difference gradually decreases. With the increase of recall, the accuracy of the collaborative filtering ad recommendation algorithm with labels proposed in this paper is significantly higher than the other three algorithms, compared with the lowest accuracy of the label-based ad recommendation algorithm, which is consistent with the results of the previous experiment.Table 2
Coefficient selection and goodness-of-fit indices of different algorithms.
AlgorithmCoefficient 1Coefficient 2Coefficient 3Goodness of fit indexSSER-squareAdjustedRMSECF-based0.087320.15970.08774.38e −070.99790.99580.0004867TAG-based0.112400.19330.10342.468e −060.99100.98210.0011110TAG-item-based0.216500.26020.13332.488e −060.99100.98200.0011150ADR-cf_t0.087660.17050.10235.313e −070.99860.99700.0005154


## 4.5.1. TOP5 Comparison of Recommendation Quality of Each Algorithm

In this paper, the proposed collaborative filtering ad recommendation algorithm with tags is compared with the user-based collaborative ad recommendation algorithm, the tag-based ad recommendation algorithm, and the ad recommendation algorithm based on the relationship between tags and items in terms of accuracy, recall, and F-measure value. The experimental results are shown in Table1 and Figure 7.Table 1
Results of experimental evaluation metrics for TOP5 recommendations of four algorithms.
AlgorithmAccuracyRecallF-MeasureUser-basedCF0.0250.06120.035TAG-based0.0310.07530.043TAG-IT–M-based0.0330.07650.046ADR-CF-T0.0380.07680.051Figure 7
Comparison of Top five recommendation quality of different recommendation algorithms.Through comparison, it is found that the collaborative filtering ad recommendation algorithm with labels proposed in this paper improves 52% in accuracy, 25% in recall, 46% in F-measure value, and nearly 41% in overall effectiveness over the traditional collaborative filtering algorithm. Since this paper considers the influence of three factors, namely CTR, ad keywords, and the relationship between ad keywords and ads, when calculating the similarity between Query pages, the integrated similarity calculation method can effectively reflect the preference information of Query pages for ads, the relevance of ad keywords for Query pages, and ads as well as the characteristics of the ads themselves for a more complete description. Meanwhile, the weight reconciliation factorsα, ß, and γ of the similarity metric proposed in this paper are found to have a greater impact on the prediction accuracy of the recommendation algorithm by analyzing their values.


## 4.5.2. Comparison of the Recommendation Quality of Each Algorithm with DifferentN Values

The selection of the number of nearest neighbors also affects the recommendation quality of the recommendation algorithm; therefore, this paper compares the accuracy, recall, and F-measure of the user-based collaborative ad recommendation algorithm, the tag-based ad recommendation algorithm, the tag-item relationship-based ad recommendation algorithm, and the proposed collaborative ad recommendation algorithm with tags for the cases of 5, 10, 15, 20, 25, and 30, respectively. The accuracy, recall, and F-measure of the user-based collaborative ad recommendation algorithm, the label-based ad recommendation algorithm, the label-item relationship-based ad recommendation algorithm, and the collaborative filtering ad recommendation algorithm with labels proposed in this paper are compared, and the comparison results are shown in Figures8–10.Figure 8
Accuracy comparison with differentN values.Figure 9
Comparison of the recall rate with differentN values.Figure 10
F-metric values for differentN values.By comparison, the proposed collaborative filtering ad recommendation algorithm with labels improves the accuracy by at least 17%, the recall by at least 0.9%, and the F-measure by at least 21% compared with the other three algorithms when 25 ads are recommended for each page. As the number of nearest neighbors increases, the recommendation effect appears to decrease rather than increase. This is because the number of truly similar Query pages in the ad recommendation system is limited. When more dissimilar neighbors are selected, these Query pages show ads with higher click-through rates from the dissimilar Query pages, resulting in a decrease in recommendation quality.Therefore, only by correctly selecting similar Query pages as nearest neighbors in the ad recommendation system can we obtain the desired collaborative recommendation effect。


## 4.5.3. Recommended Quality Optimization Degree Comparison

In this paper, the accuracy and recall of four recommendation algorithms are fitted with Gaussians to verify the degree of optimization of the recommendation results, corresponding to a Gaussian fitting function offx=a×exp−x−b/c2 and a thrust coefficient of 95%. The accuracy and recall Gaussian fitting curves of the recommendation results of each algorithm are shown in Figure 11.Figure 11
Gaussian fitting curves for accuracy and recall of recommendation results.The Gaussian fitted curves show that the user-based collaborative ad recommendation algorithm and the tag and item relationship-based ad recommendation algorithm have intersection points as the recall rate increases, and the coefficients provided in Table2 give the intersection point coordinates of (0.0472, 0.0168). In the interval [0.0472, 0.08], the accuracy of the user-based collaborative ad recommendation algorithm is higher than that of the ad recommendation algorithm based on the relationship between tags and items, and the degree of difference gradually decreases. With the increase of recall, the accuracy of the collaborative filtering ad recommendation algorithm with labels proposed in this paper is significantly higher than the other three algorithms, compared with the lowest accuracy of the label-based ad recommendation algorithm, which is consistent with the results of the previous experiment.Table 2
Coefficient selection and goodness-of-fit indices of different algorithms.
AlgorithmCoefficient 1Coefficient 2Coefficient 3Goodness of fit indexSSER-squareAdjustedRMSECF-based0.087320.15970.08774.38e −070.99790.99580.0004867TAG-based0.112400.19330.10342.468e −060.99100.98210.0011110TAG-item-based0.216500.26020.13332.488e −060.99100.98200.0011150ADR-cf_t0.087660.17050.10235.313e −070.99860.99700.0005154


## 5. Conclusions

In this paper, we propose a deep attention popularity prediction model DAFCT that combines content features and temporal information, which can effectively express the popularity trend of ads and improve the performance of popularity prediction. The experimental results show that the collaborative filtering ad recommendation algorithm with labels have better scalability and better recommendation quality than the traditional collaborative filtering algorithm, label-based recommendation algorithm, and recommendation algorithm based on the relationship between labels and items. However, the algorithm does not consider other factors affecting the click-through rate of ads, such as location and bidding price. Therefore, the next step will be to consider combining with machine learning algorithms, mining the attributes of ads themselves, extracting feature information, analyzing the factors affecting the click-through rate of ads in practical applications, and improving the recommendation accuracy.



---

*Source: 9940232-2021-12-18.xml*