## 3 Proposed Intelligent Weighting Factors Design Approach
### Proposed Weighting Factor Tuning Scheme Based on DRL
#### 3.2.1 Generalities  
Reinforcement learning is a distinct paradigm within the field of machine learning [36]. It focuses on teaching an agent how to make sequences of decisions by interacting with an environment in order to solve a so-called sequential decision problem. The sequential decision problem can be represented using a Markov decision process (MDP). An MDP is defined by a list of components, \((\mathcal{S},\mathcal{A},\mathbb{P},\rho,\gamma)\) which are described as follows:  
* The state space \(\mathcal{S}\) corresponding to the set of all possible values of the states describing the environment with which the agent interacts;
* The action space \(\mathcal{A}\) corresponding to the set of all possible choices the agent can make to act on the environment;
* The transition dynamics \(\mathbb{P}\): \(\mathcal{S}\times\mathcal{A}\times\mathcal{S}\rightarrow[0,1]\) characterizing the dynamics of the environment by giving the probability \(\mathbb{P}(s(k+1)=S^{+}\mid a(k)=A,s(k)=S)\) of obtaining a state \(S^{+}\in\mathcal{S}\) at time \(k+1\) knowing a state-action pair \((S,A)\in\mathcal{S}\times\mathcal{A}\) at time \(k\);
* The reward function \(\rho\): \(\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R}\) quantifying the desirability for the agent to perform action \(A\in\mathcal{A}\) when the environment is in some state \(S\in\mathcal{S}\);
* The discount factor \(\gamma\in(0,1)\) determining the relative importance of future rewards compared to immediate ones, therefore influencing the agent's consideration of long-term consequences when selecting an action to perform. When the transition dynamics are stochastic as described above, the state \(s(k+1)\in\mathcal{S}\) at time \(k+1\) is a random variable following the probability distribution \(\mathbb{P}(\cdot\mid a(k),s(k))\) depending on the values at the time \(k\) of the state \(s(k)\in\mathcal{S}\) and the action \(a(k)\in\mathcal{A}\). It is often denoted \(s(k+1)\sim\mathbb{P}(\cdot\mid a(k),s(k))\). When these transition dynamics are deterministic, there is only one possible value for the random variable \(s(k+1)\) with probability \(\mathbb{P}(s(k+1)\mid a(k),s(k))=1\). To simplify the notations, such a case is often denoted \(s(k+1)=\mathbb{P}(a(k),s(k))\). The RL aims to solve the sequential decision problem described above by constructing a policy function \(\pi\colon\mathcal{A}\times\mathcal{S}\rightarrow[0,1]\) that can be either stochastic or deterministic. This policy function corresponds to the probability for the agent to perform a given action \(a(k)\in\mathcal{A}\) at time \(k\) while being in a given state \(s(k)\in\mathcal{S}\) at the same time instant. In other words, it is defined such that for any \(A\in\mathcal{A}\) and \(S\in\mathcal{S}\), we have, at time \(k\), that \(\pi(A|S)=\mathbb{P}(a(k)=A|s(k)=S)\). This policy defines the agent's behavior so that, if the environment is in state \(s(k)\) at time \(k\), the action \(a(k)\in\mathcal{A}\) is a random variable following the probability distribution \(\pi(\cdot\mid s(k))\), denoted by \(a(k)\sim\pi(\cdot\mid s(k))\). The goal of RL is then to find the optimal policy \(\pi^{*}\) achieving the maximum expected reward from all states,  
\[\pi^{*}=\arg\max_{\pi}\sum_{k=0}^{+\infty}\gamma^{k}\mathbb{E}_{(s(k),a(k)) \sim_{\mathbb{T}_{\pi}}}[\rho(s(k),a(k))] \tag{12}\]where \(\mathbb{E}_{(s(k),a(k))\sim\tau_{\pi}}[\gamma^{k}\rho(s(k),a(k))]\) denotes the expected value of \(\gamma^{k}\rho(s(k),a(k))\) when the pair \((s(k),a(k))\) is sampled from the distribution \(\tau_{\pi}\) of trajectories induced by a given policy \(\pi\). **Remark 5**. : _As is the case for the transition dynamics as presented in Remark 4, the policy can also be deterministic. In this case, there is only one possible value for the random variable \(a(k)\) with probability \(\pi(a(k)\mid s(k))=1\).