# TRUST-ME: Trust-Based Resource Allocation and Server Selection in Multi-Access Edge Computing
## 3 Trust-Based Reinforcement-Learning-Enabled MEC Server Selection
### Reinforcement-Learning-Based MEC Server Selection  
The objective for the users is to independently choose an MEC server to efficiently handle their computational tasks. In this scenario, the users function as reinforcement learning (RL) agents that evaluate the potential reward \(\rho^{(ite)}_{n,a_{s}}\) associated with each action \(a_{s}\) at each iteration \(ite\) of the RL algorithm. This process continues until they reach a stable and optimal MEC server selection. The possible actions available to the users are represented by the set \(\mathcal{A}=\{a_{1},\ldots,a_{s},\ldots,a_{s}\}\). The reward \(\rho^{(ite)}_{n,a_{s}}\) for a user depends on several factors, i.e., the computational capacity \(f_{s,n}\) provided by the chosen MEC server \(s\), the user's confidence in the quality of service offered by the MEC server, the proximity of the user to the MEC server (which affects the user's communication latency), and the MEC server's overall computing capacity. By combining all these factors, the reward experienced by user \(n\) when offloading its computing task to MEC server \(s\) is given as follows:\[\rho_{n_{a_{s}}}^{(ite)}=w_{1}\frac{f_{s,n}}{F_{s}}+w_{2}tr_{s,n}+w_{3}\frac{x}{d_ {s,n}}+w_{4}\frac{F_{s}}{\sum\limits_{\forall s\in\mathcal{S}}F_{s}} \tag{15}\]  
where the trust level \(tr_{s,n}\in[0,1]\) quantifies how much user \(n\) trusts MEC server \(s\); \(d_{s,n}\) [m] denotes the distance between MEC server \(s\) and user \(n\); \(x\) [m] represents the closest distance of a user to the selected MEC server; and the weights satisfy the following conditions: \(w_{1}+w_{2}+w_{3}+w_{4}=1\), where \(w_{1},w_{2},w_{3},w_{4}\in\mathbb{R}^{+}\). A higher value of \(w_{1}\) increases the importance of the computational capacity ratio and gives priority to servers with higher available capacity relative to their total capacity. In this case, the users tend to prefer the servers with better computational resources more in order to optimize the task processing speed. A higher value of \(w_{2}\) emphasizes the trust level, and thus, MEC servers with higher trust scores are preferred. In this case, the reliability and security of the task offloading process are improved, as the users are more likely to choose trusted servers. A higher value of \(w_{3}\) increases the importance of proximity, and thus, the users tend to prefer closer servers to offload their tasks. In this case, the communication latency is decreased and the users' experience and system responsiveness are improved. Finally, a higher value of \(w_{4}\) emphasizes the overall computing capacity and promotes the selection of servers with a larger total capacity. In this case, a balance of the load across the network is achieved, and the servers' overloading is prevented. The optimistic Q-Learning with upper bound confidence (OQ-UCB) action selection RL algorithm is introduced for the users to autonomously choose an MEC server. According to the experienced reward \(\rho_{n_{a_{s}}}^{(ite)}\) at the iteration \(ite\) of the OQ-UCB algorithm, each user updates its expected reward \(Q_{a_{s}}^{(ite)}\) to select an MEC server in the next iteration, which is defined as follows:  
\[Q_{a_{s}}^{(ite)}=Q_{a_{s}}^{(ite-1)}+\gamma(\rho_{n_{a_{s}}}^{(ite)}-Q_{a_{s}} ^{(ite-1)}) \tag{16}\]  
where \(\gamma\in(0,1)\) represents the learning rate of the OQ-UCB algorithm. Every user chooses an MEC server based on their anticipated reward and uncertainty regarding the outcome, which can be described as follows:  
\[a_{s}^{(ite)}=\operatorname*{arg\,max}_{a_{s}^{(ite)}}[Q_{a_{s}}^{(ite)}+c\cdot \sqrt{\frac{\ln(ite)}{N_{a_{s}}^{(ite)}}}] \tag{17}\]  
where \(c\in\mathbb{R}^{+}\) represents the confidence level and \(N_{a_{s}}^{(ite)}\) denotes the number of times action \(a_{s}\) has been selected up to iteration \(ite\). Equation (17) incorporates two main components.