## 4 The Proposed Framework Approach
### Agent Model for Telepresence Robot  
The implementation in this article is based upon double deep q networks (DDQN), which further presents a fully connected (FC) layer and a one-to-many gated recurrent unit (GRU) network layer. The GRU pulls the state's relevant information to the cell units. The network is a double net frame containing the primary and target networks, as shown in Figure 11. The current state data go to the primary network, and the next state data go to the destination network. Lastly, we use the values acquired from multiple networks, which behave as input to the loss function, to attain the error value. The revision of each network's parameter is calculated in the main network, and its synchronization is based on the error gradient. DDQN, with the integration of GRU, supplies the efficient results of training to provide the opportunity of lesser value of loss function and having the idea of better actions. After selecting the final state decision, those related parameters will collect in memory D. We have collected all the state parameters  
Figure 11: Proposed framework based on integration of Gated Recurrent Unit (GRU) with Double Deep Q-learning Network (DDQN). and transmitted the reward value to the loss function. The proposed algorithm is briefly explained in Algorithm 2. ```
1:Initialize: input data and Update associated status information \((s,a,r,s^{\prime})\) from memory D,
2:for each episode,do
3: input will be sent to the main and target networks
4: input data will send to GRU unit, which has eight behavior sets (columns) \[A= \begin{bmatrix}\text{front, back, left, right, leftfront,}\\ \text{rightfront, leftrear, rightrear}\end{bmatrix} .\]
5:for each step in the GRU unit,do
6: GRU layers are responsible for processing the \(n=8\) data  middle data will be fed to two layers of the FC layer
7: The FC layer parameter is \(8\times 64\) and \(64\times 8\) matrix the activation function uses the rectified linear in the neural unit in the FC  a dropout structure is set in the GRU and FC layers to prevent overfitting
8:endfor
9:for each update step do
10: find the action of \(\text{argmax}_{s^{\prime}}Q(s^{\prime},a^{\prime};\theta_{1})\)
11: then
12: find the action in the target network
13: The Q\({}_{\text{target}}\) value is calculated by using th Q value
14: The Q value is not certainly the largest in the target network,  so, we can avoid selecting the overestimated suboptimal action
15: Memory pool provides the training data
16: Calculate the loss function: \[L=E\bigg{\{}(r+\gamma Q_{\text{target}}(s^{\prime},\underset{a}{\text{argmax }}(Q_{\text{main}}(s^{\prime},a)))-Q_{\text{main}}(s,a))^{2}\bigg{\}}\]
17:endfor
```  
**Algorithm 2**Gated Recurrent Unit (GRU) with Double Deep Q-Networks Learning (DDQN)  
Our proposed approach algorithm first retrieves the input data and associated status information from memory module D, which is \(s,s^{\prime}\) in the fourth state \((s,a,r,s^{\prime})\). It expresses the shorthand as (state, action, reward, and next state). Then input will be sent to the main and target networks. The main and target networks are in real-time synchronization, and the network characteristics are identical. The eight-column GRU unit will then receive the input status data. These eight states represent the behavior set in Equation (8). \[A= \begin{bmatrix}\text{front, back, left, right,}\\ \text{leftfront, rightfront, leftrear}\end{bmatrix}  \tag{8}\]  
Three GRU layers are responsible for processing the \(n=8\) data, after which middle data are fed to two layers of the FC layer. The FC layer parameter structure is \(8\times 64\) and \(64\times 8\) matrix, respectively. 1. The activation function uses the rectified linear in the neural unit in the FC. 2. A dropout structure is set in the GRU and FC layers to prevent overfitting. 3. Initially, we find the action of \(\text{argmax}_{a^{\prime}}Q(s^{\prime},a^{\prime};\theta_{1})\) with the main network and then find the action in the target network. 1. The Q\({}_{\text{target}}\) value is calculated by using the Q-value.