## 3 Proposed System
### Detail Implementation Flow
#### 3.3.2 Phase Two
### Algorithms for the Proposed System

In this section, we show the entire algorithm for our recommendation system. Following the implementation section, we keep the same structure, and the process of overall system starts as shown in Algorithm 3. ```
Input: ratings_df, movies_df, users_df Output: evaluation_metrics, specific user_recommendations, top-N recommendation Procedure:
1:Load and preprocess data: ratings_df, movies_df, users_df = load_movie_data() R_df = create_user_item_matrix(ratings_df)
2:Split data into training and testing sets: train_users, test_users = split_users_based_on_ratings(R_df)
3:Prepare data for deep reinforcement learning: train_dataloader, test_dataloader = create_data_loaders(train_users, test_users)
4:Initialize reinforcement learning models: actor_model = initialize_actor_model() critic_model = initialize_critic_model() target_actor_model = actor_model target_critic_model = critic_model replay_buffer = initialize_replay_buffer()
5:Define state representation: define_state_representation_functions()
6:Train the models--for the detail refer to Step 2: for episode = 1 to num_episodes do for batch in train_dataloader do state = compute_state_representation(batch) action = actor_model(state) reward = calculate_reward(batch) next_state = compute_state_representation(batch) replay_buffer.push(state, action, reward, next_state) if replay_buffer.size() > batch_size then update_actor_critic_models(replay_buffer) end if end for
7:Test the models: for batch in test_dataloader do state = compute_state_representation(batch) recommendations = generate_recommendations(actor_model, state) evaluate_recommendations(recommendations, batch) end for
8:Compute evaluation metrics: evaluation_metrics = calculate_evaluation_metrics()
9:Generate user-specific recommendations: selected_user_id = choose_user_id(test_users) user_recommendations = generate_user_specific_recommendations(selected_user_id)
10:Analyze recommendations using cosine similarity: cosine_similarity_matrix = compute_cosine_similarity(user_recommendations)
11:Return Output
```

**Algorithm 3** The Proposed System for Movie Recommendation system

## 4 Experiments and Results

In this section, we conduct extensive experiments with a dataset from GroupLens, which includes approximately 1 million movie ratings. Moreover, we introduce essential tasks in our system, including the experiment settings, evaluation metrics, benchmark methods, and the discussion results. ### Experiment Setting

The current study utilizes several tools to support the implementation process, including Python version 3.9, Torch, Pandas, Numpy, and Matlibplot for visualization. Additionally, we have developed the entire model on a Windows OS platform with a RAMcapacity of 64 GB. For evaluation, we use the MovieLens 1M dataset, which comprises 1,000,209 anonymous ratings for a total of 3900 movies. These ratings are collected from a user base of 6,040 individuals who joined MovieLens in the year 2000. We select users who have rated movies with a rating greater than 3 and have provided more than 10 ratings. Additionally, we choose an embedding size of 100 for the MovieLens datasets. For the initial parameters, the dropout is set to 0.6, with a hidden size of 128 and 256, both for the Actor and Critic networks. There are several optimizers that are used for model tuning. Adam, Adadelta, Adamax, RMSprop, and SGD are involved for the purpose of finding the optimal recommendation result. The learning rate of each optimizer is set to 0.0001. We use a gamma value (discount factor) equal to 0.98 to determine the importance of future rewards. Some optimizers support with momentum; for these, we set the value to 0.9, since this is the most recommended value. ### Evaluation Metrics

Three common metrics are selected for the purpose of evaluation in recommendation systems. Precision, Recall, and the F1 Score are used for evaluating and fine-tuning recommendation systems and other retrieval algorithms, ensuring that they provide relevant and comprehensive results.