## 2 System Model  
The suggested system architecture for resource allocation and compute offloading in uplink E-APs is shown in Figure 1. The network comprises a collection of E-APs \(\mathcal{L}=\{1,2,\ldots,L\}\) and the number pieces of UE \(\mathcal{K}=\{1,2,\ldots,K\}\). We assume that each E-AP has the storage capacity \(S_{l}\), which can store the maximum amount of data, including libraries and databases [14]. All the notations used in the system model are listed in Table 1.  
We consider there to be three shared environments, each encompassing a maximum of three E-APs. The E-AP is allocated with the best channel capacity and maximum resources and performs as an agent of its environment; the other two E-APs work as assistive E-APs. In the proposed model, three scenarios are considered based on the user's demand. Shared environment 1 illustrates that if the assistive E-APs are capable of satisfying the users' demand within the tolerable delay \(\delta^{max}\), UE will be served by the assistive E-APs where the agent E-AP is responsible for allocating the optimal resources and selecting the suitable E-AP based on user demand. According to shared environment 2, all the E-APs participate in executing the computation tasks where the agent distributes the UE among them. In shared environment 3, the agent E-AP is preferred to accomplish the computation since it has superior computing resources and channel capacity than the assistive E-AP. The proposed model considers two types of IoT devices. UE that has processing and cache capacities is considered to be smart user equipment (SUE), i.e., smartphones or laptops. On the other hand, UE without processing and cache capacities is regarded as general user equipment (GUE), which includes sensors and industrial monitoring devices. In the context of computation offloading, UE sends the offloading request to the closest E-AP. \begin{table}
\begin{tabular}{l l} \hline \hline
**Symbol** & **Definition** \\ \hline A3C & Advanced Asynchronous Advantage Actor-Critic \\ AI & Artificial Intelligence \\ DRL & Deep Reinforcement Learning \\ MECNs & Mobile Edge Computing Networks \\ CP & Cloud Processor \\ CRRM & Collaborative Radio Resource Management \\ CRSP & Collaborative Radio Signal Processing \\ E-AP & Edge Access Point \\ SUE & Smart User Equipment \\ CSI & Channel State Information \\ GUE & General User Equipment \\ s\({}_{k}\) & transmitted signal vector \\ \(p_{k,l}\) & the transmitting power from user \(k\) to E-AP \(l\) \\ \(h_{k,l}^{H}\) & the channel gain between user \(k\) to E-AP \(l\) \\ \(s_{k}^{nl}\) & available mode for user \(k\) \\ \(T_{min}\) & the lower bound of QoS for user \(k\) \\ \(R_{k}\) & maximum achievable transmission rate \\ \(t_{k}^{max}\) & tolerable maximum latency for the task completion \\ \(M_{k}\) & task of user \(k\) \\ \(R_{k,l}\) & transmission rate between user \(k\) to E-AP \(l\) \\ \(R_{k,j}\) & transmission rate between user \(k\) to RRH \(j\) \\ \(f_{l,k}\) & the allocated computing resources of E-AP \(l\) to user \(k\) \\ \(\omega_{l}\) & energy consumption for per CPU cycle at E-AP \(l\) \\ \(s\) & state in the environment \\ \(a\) & the agent action \\ \(\xi\) & the discount factor \\ \(r\) & reward obtained from appropriate action \\ \((s,a)_{t}\) & action-state pair for every time \(t\) cycle \\ \(Q^{*}(s_{t},a_{t})\) & optimal Q-function at time \(t\) \\ \(r(s,a)\) & the reward obtained from the current action state pair \\ \(\alpha\) & the learning rate of Q-learning \\ \(L_{i}(\omega_{i})\) & The loss function in which \(\omega\) represents the parameter of the neural network \\ \(E\) & Expected outcome \\ \hline \hline \end{tabular}
\end{table} 
Table 1: List of Notations. Figure 1: MECNs with deep reinforcement learning (DRL) for computation offloading. After receiving a service request from the SUE, the agent rigorously analyzes the task to take the best possible action. If the task is suitable for execution by the SUE with its embedded local resources, it will be responsible for executing it.