# Enhancing Sequence Movie Recommendation System Using Deep Learning and KMeans
## 3 Methodology of Movie Recommendation Systems
### Data Processing and Sequence Creation  
Firstly, we explain the preprocessing steps required in our implementation. In our work, we utilize users' ratings from the MovieLens dataset to construct a recommendation system. Initially, we merge all the rating information for an individual user into the required input format for our transformer model [44]. We then construct a vocabulary for movie IDs and user IDs and create sequences of user interactions in chronological order. All user interactions are first sorted by their interaction timestamp and then divided into subsequences for the training model. To facilitate subsequent calculations, we convert the list of movie IDs and the movie ratings to a fixed length and retain a position set of subsequence information. These sequences are further divided into subsequences to create a structured input format that encapsulates a user's interaction history up to a fixed length, thereby preparing the data for the transformer model. After processing, the input is generated for each individual user in a sequential manner, including user ID, movie ID, sequence rating, the target (label) that the model attempts to predict--which would be the target movie ID--and the rating of the last item in the sequences, as shown in Table 1.  
To enhance the input data, we integrate additional attributes, such as movie genres in the sequences and the demographic information (age, gender, occupation) of the users, into our final dataset. The final dataset was then subjected to a series of cleaning procedures, including removing duplicates, handling missing values, and addressing outliers, before being finalized for analysis. Furthermore, some data had to be converted from categorical to numerical formats to be compatible with the transformer model and k-means clustering in the embedding layer. Variables like gender and occupation were encoded from categorical to numerical formats through a process of factorization. The age feature underwent normalization using the MinMaxScaler, which scales the data to fall within a specified range. This enhances model performance by ensuring numerical features are on a similar scale. Movie genres were expanded into binary features, each indicating the presence of a specific genre, thus enriching the dataset with explicit genre information, as shown in Figure 1 on step 1 and in Figure 2.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**User ID** & \multicolumn{3}{c}{**Movie ID in Sequence**} & \multicolumn{3}{c}{**Sequence Rating**} & \multicolumn{3}{c}{**Target Movie ID**} & \multicolumn{1}{c}{**Target Rating**} \\ \hline User\_1 & 2926 & 2915 & 2344 & 2968 & 2.0 & 3.0 & 3.0 & 4.0 & 2968 & 4.0 \\ User\_2 & 1307 & 2791 & 73 & 3269 & 3.0 & 5.0 & 1.0 & 5.0 & 3269 & 5.0 \\ User\_3 & 420 & 3688 & 838 & 2174 & 2.0 & 1.0 & 4.0 & 3.0 & 2174 & 3.0 \\ User\_4 & 1704 & 541 & 1219 & 3521 & 4.0 & 2.0 & 3.0 & 2.0 & 3521 & 2.0 \\ User\_5 & 44 & 1608 & 2064 & 3835 & 5.0 & 1.0 & 1.0 & 4.0 & 2064 & 4.0 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Sequence interaction of users and movies.