## 5 Performance Evaluation  
We have built our simulation environment using MATLAB 2020a and consider nine content routers illustrated in Figure 2. Our proposed ISO strategy utilizes the mesh network of routers as a core network for 5G and next-generation cellular networks. We have compared our proposed strategy with random routing algorithm and history aware routing protocol (HAPR). In random algorithm the agent randomly takes the action for the routing path and there is no consistency in the routing path. Furthermore, HAPR follows the first path taken. Therefore, as the name suggests it tends to follow the same path for content retrieval and interest forwarding once the successful routing path is completed in the previous iteration. In our model AR/VR users are connected to BS1 and BS2, and the router \(H_{9}\) has the desired content in its cached memory, according to consumer AR1. Our simulation results illustrate that the RL-based network will help the consumer retrieve the content from that path with less stretch that will help get a better data rate and lesser latency as compared to random and HAPR. Table 2 illustrates the default parameter values of simulation environment. This section will evaluate the proposed ISO strategy in ICN systems for optimal routing path that lowers the E2E for haptic-driven AR/VR applications in TI. We computed the stretch as the performance metric for the routing path of the agent. The stretch reward is maximum for the least number of hops. Meanwhile, the proposed ISO algorithm opts for a better routing path for the least stretch. We kept different parameters of the learning rate (\(\alpha\)) and discounted factor (\(\gamma\)) while keeping the exploration/exploitation factor (\(\varepsilon\)) constant. The learning speed of an agent is controlled and supervised by using \(\alpha\). Similarly, the learning rate \(\gamma\) controls the effect of past values on the current value or transition state. While \(\varepsilon\) factor controls the exploration or exploitation of the agent. On the basis of the learning rate, discounted factor, and epsilon, we have illustrated our model in terms of accumulated reward, instant episode, the average stretch of agent, and total hopes covered in each episode. We have kept the value of \(\varepsilon\) at 0.5 in all of our simulation results. Therefore, it means that the model explores and exploits 50%. We have computed accumulated reward as the agent covers the routing path to content router where content is placed. in Figure 6a, we have varied the learning rate from 0.1-0.9 and illustrated our agent's convergence at three different values of learning rate. The agent converges at the 121 st iteration when the alpha value is 0.1, but still, the convergence is not complete, as evident from successive episodes. In comparison to the value of \(\alpha\) at 0.5, the results are better than \(\alpha\) value at 0.1 as it convergence at 117 th episode, and hence a little better reward is attained. However, at the value of \(\alpha\) at 0.9, convergence has been observed at the 128 th episode. This shows that \(\alpha\) value at 0.5, the proposed algorithm achieves better and optimal reward with the value of \(\gamma\) at 0.1. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Values** \\ \hline Simulation environment & Matlab 2020a \\ Network topology & Mesh network of routers \\ Number of AR/VR user requesting & 1 at a time \\ RCR nodes in core network & \(n=9\) \\ Link delay & 10 ms \\ Routing decision & MDP (\(A_{i}\)) \\ Learning rate (\(\alpha\)) & 0.1, 0.5, 0.9 \\ Discounted factor (\(\gamma\)) & 0.1, 0.5, 0.9 \\ Exploitation/exploration factor (\(\varepsilon\)) & 0.5 \\ Action space & [1,5] \\ State space & [1,9] \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Simulation parameters. Furthermore, the average stretch has been evaluated and illustrated in Figure 6b. We have illustrated the random routing strategy and HARP and compared them with our proposed strategy.