## 3 Proposed System
### Detail Implementation Flow
#### 3.3.2 Phase Two
```
Define the Actor class Inherits nn.Module:  Initialize neural network layers:  Linear(input_dim,hidden_dim)>LeakyReLU>Dropout(optimal_prob)  Linear(hidden_dim,hidden_dim)>LeakyReLU>Dropout(optimal_prob)  Linear(hidden_dim, output_dim) #Second Linear layer from hidden_dim to hidden_dim  LeakyReLU activation layer  Define forward(state):  state-> Linear1>LeakyReLU ->Dropout ->Linear2>LeakyReLU>Dropout->Linear3  return output End Actor Class  Define the Critic class Inherit from nn.Module  Initialize:  Linear(input_dim+output_dim, hidden_dim)>LeakyReLU->Dropout(optimal_prob)  Linear(hidden_dim, hidden_dim)>LeakyReLU->Dropout(optimal_prob)  Linear(hidden_dim, 1)  Initialize weights of last layer uniformly within specified range  Define forward(state, action):  Concatenate(state,action)->Linear1->LeakyReLU>Dropout->Linear2>LeakyReLU>Dropout->Linear3  return value_estimate End Critic Class  Model Initialization:  Set input_dim, output_dim, hidden_dim as per dataset characteristics  actor = Actor(input_dim, hidden_dim, output_dim, dropout_probability)  critic = Critic(input_dim, output_dim, hidden_dim, dropout_probability)  actor_target = actor_model  critic_target = critic_model  Define Optimizers:  actor_optimizer = Optimizer(actor.parameters(), learning_rate)  critic_optimizer = Optimizer(critic.parameters(), learning_rate)
```  
**Algorithm 1** Actor-Critic Model Initialization  
The Deep Deterministic Policy Gradient (DDPG) is an actor-critic model-free algorithm designed for solving problems with continuous action spaces, making it particularly suitable for reinforcement-learning tasks. Specifically, the algorithm extends concepts from Deep Q learning to handle continuous action spaces by combining elements. The following Algorithm 2 is for training the DDPG and is processed as follows:```
Initialize Networks and Parameters: actor_model = initialize_actor_network(input_dim, hidden_dim, output_dim) critic_model = initialize_critic_network(input_dim, output_dim, hidden_dim) actor_target = actor_model critic_target = critic_model replay_buffer = create_replay_buffer() Set exploration_noise, discount_factor, tau (soft update parameter) Training Loop: For episode = 1 to max_episodes do: initialize exploration_noise state = observe_initial_state() for t = 1 to max_timesteps do action = actor(state) + exploration_noise next_state, reward, done = execute_action(action) replay_buffer.store(state, action, reward, next_state, done) batch = replay_buffer.sample() target_Q = compute_target_Q(batch, critic_target, actor_target, discount_factor) update_critic(critic, batch, target_Q) update_actor(actor_model, critic_model batch) soft_update(actor_target, actor_model, tau) soft_update(critic_target, critic_model, tau) update exploration_noise if done then break end if end for
```

**Algorithm 2** DDPG Training

#### 3.3.3 Phase Three

This phase of the development system involves rigorously testing the recommendation system with a diverse set of users not seen during training and focusing on generating accurate recommendations. * Testing and Recommendation Generation

In the final phase of the recommendation-system development, we undertake the crucial task of system evaluation and recommendation generation for users. To ensure the robustness of our assessment, we employ a randomization of userIDs to ensure a fair test; i.e., userIDs are chosen randomly for testing. This method helps to avoid bias because the system has not seen these users during their training phase. Addressing cold-start scenarios, a common challenge in recommendation systems is dealing with new users who have very little interaction history (known as \"cold start\" users). To tackle this, users who have given fewer than 10 high ratings (rated 4 or 5) are intentionally excluded from the test set. Another point is the separation from the training data; by ensuring that the users selected for testing are not from the training dataset, the system's evaluation becomes more reliable and freer from biases that could arise if it were tested on familiar data.