## 2 Methodology
### Reinforcement Learning  
This algorithm was selected for the case evaluation due to the possibility of training numerous parallel agents in the environment, which would facilitate the possible expansion of new content reception sites. That is, according to the current workflow, it is a single site that receives and evaluates the content, but due to this algorithm, it would be easy in future work to expand the capacities with an indefinite number of parallel sites, limited only by the capacity of resources. The streaming use case in which we train and apply the algorithm focuses on optimization of the streaming bitrate in order to obtain the best MOS received by the probe in the position of an end-user. The environment is defined by the parameters (states) that model the status of the use case workflow. These states are dependent on the received MOS, the emitted and received bitrate, the encoding quality, and the RAM usage of the transcoder. This set of states is chosen because they can appropriately reflect both the quality received and the current situation of the components. To perform the optimization, and generally for all RL algorithms, the analysis of the environment provides a policy of actions. The set of actions should reflect the amount of options that the algorithm can produce. The set developed for this case consists of a selection of network profiles in which we configure the target bitrate to which the transmission must adapt. Actions are applied to the site transcoder via Rest API, changing the internal throughput of the streaming. The profile set consists of: 10 Mbps, 7.5 Mbps, 5 Mbps, 4 Mbps, 2.5 Mbps, 1 Mbps. Both states and actions are summarized in Table 2.  
Finally, the algorithm returns a reward based on the actions taken and the resulting states. This reward is calculated from a combination of specific reward functions, depending on the received streaming MOS; the emitted and received bitrate; the selected profile; and \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Input** & **Learning Rate** & **Output Activation** & **Output** \\ \hline \multirow{2}{*}{Actor} & \(5\times 8\) & \multirow{2}{*}{0.01\%} & \multirow{2}{*}{SoftMax} & Action policies \\  & (states \(\times\) states batch) & & & (Probabilities) \\ \multirow{2}{*}{Critic} & \(5\times 8\) & \multirow{2}{*}{0.1\%} & \multirow{2}{*}{Linear} & V(s) \\  & (states \(\times\) states batch) & & & (Value estimation) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Neural Networks hyperparameters. Figure 3: Neural Networks architecture implementation. the smoothness (avoid very aggressive bitrate adjustments) of profile selection. First actions are randomly performed, and lower rewards will be returned. The optimizer will analyze possible actions and learn the best suitable option based on the information as it explores possible actions over time. We present a detailed pseudocode of A2C in Algorithm 1 in which, the process behind how the algorithm models the optimal policy of actions of the NN actor is detailed step-by-step. That is, for a set of episodes, the algorithm performs a fixed number of steps for training. Each episode takes an action which returns a reward, and through the computation of the Advantage Function, the weights of the NN are updated (Critic network using Stochastic Gradient Descent).