# Multi-Feature-Enhanced Academic Paper Recommendation Model with Knowledge Graph
## 3 Approach of This Paper
### Text Vector Representation  
In this paper, the ALBERT [24] model is employed to process the titles and abstracts of academic papers to generate vector representations of words. ALBERT (A Lite BERT) is a lightweight variant of the BERT [25] model that leverages the advantages of the Transformer [26] model. It optimizes the model structure through parameter sharing and the compression of embedding layers, thereby enhancing computational efficiency. Compared to traditional word embedding methods such as Word2 Vec [27] and GloVe [28], ALBERT is better at utilizing contextual information to provide richer and more accurate semantic representations. This capability gives ALBERT a significant advantage in handling phenomena of polysemy and capturing the deep semantics of text. Subsequently, the word vectors obtained from the ALBERT model are input into a bidirectional long short-term memory (Bi-LSTM) [29] network for processing. The Bi-LSTM network is capable of capturing broader contextual information between words, consisting of forward and backward LSTM layers that capture contextual information in both directions. The forward hidden state \(h_{f,t}\) is calculated as follows:  
\[h_{f,t}=LSTM(x_{f,t},h_{f,t-1}) \tag{6}\]  
The backward hidden state \(h_{b,t}\) is calculated as follows:  
\[h_{b,t}=LSTM(x_{b,t},h_{b,t-1}) \tag{7}\]  
Ultimately, the forward hidden state and the backward hidden state are concatenated to obtain the final hidden state \(h_{t}\). \[h_{t}=[h_{f,t},h_{b,t}] \tag{8}\]  
In the formula above, \"LSTM\" represents the function of the bidirectional long short-term memory (Bi-LSTM) unit, and its specific form is as follows:  
\[LSTM(x,h)=i\odot\tanh(W_{i}\cdot[h,x]+b_{i})+f\odot c_{t-1}+g\odot c_{t} \tag{9}\]  
\[c_{t}=f\odot c_{t-1}+i\odot\tanh(W_{c}\cdot[h,x]+b_{c}) \tag{10}\]  
The \(i\), \(f\), and \(g\) in the aforementioned formulas represent the outputs of the gate control units, where the symbol \(\odot\) denotes element-wise multiplication. \(W_{i}\), \(W_{c}\), \(b_{i}\), and \(b_{c}\) are the weight matrices and bias terms, respectively. After processing through the bidirectional LSTM module, this model obtains title and abstract vectors that encapsulate the hidden semantic relationships between words. Since titles play a crucial role in a paper, this paper further enhances the processing of the title portion. To ensure a more representative paper vector, a self-attention mechanism is introduced to better capture the relevance of various words in the title sequence. Each word in the title is assigned a weight, and the title vector representation is adjusted according to these weights to obtain a weighted word vector \(W_{h}\) and a new title vector \(H^{\prime}_{h}\). The calculation is as follows:  
\[W_{h}=softmax(\tanh(W_{q}H_{h}+b_{q})) \tag{11}\]  
\[H^{\prime}_{h}=\sum H_{h}\times W_{h} \tag{12}\]  
Inspired by the vision transformer (ViT) model [30], this paper segments the abstracts of papers into independent sentences and converts each sentence into a vector representation. By breaking down the abstract into sentence-level representations, each sentence vector encapsulates distinct semantic information. This approach better captures the semantic content of each sentence in the abstract, enhancing the model's ability to understand the semantics of the abstract. It also allows for more effective modeling within the encoder, resulting in a more representative and semantically rich paper text vector. Compared totreating the entire abstract as a single entity for model input, this method, which combines attention mechanisms at the sentence level, enables the model to more subtly comprehend the structure and content of the abstract. After processing the abstract with ALBERT and Bi-LSTM, the grammatical and semantic information within the abstract is accurately captured. The approach involves averaging the vectors of each word within every sentence of the abstract to obtain a vector representation for that sentence.