## 4 Experiment Results and Discussion
### Evaluation and Discussion
After an offline training process of the tracker, the network was updated and connected to the virtual realistic environment in order to be tested in real time. A drone agent gave several output parameter results (shown in Table 1), which were configured before the testing process, illustrating the virtual drone agent's behavior during the testing process as well as providing summarizing episode (Table 2) results. Figure 8: The average achieved epsilon score (Epsilon, **left**) and used replay memory (Memory-GB, **right**) value during the training process (red—target and blue—DQN value loss in epsilon graph; orange—used replay memory unit in GB). Epsilon graph shows the balances exploration and exploitation by choosing randomly in y axes that refers to the probability of choosing to explore, while x axes depict the training epochs. The memory graph represents the used memory space during training in the y axes as well as training epochs in the x axes. Figure 7: The average predicted action value of the MeanQ (**left**) and MaxQ (**right**) values with two colored lines (red—target and blue—DQN value loss). The MeanQ figure (left) represents the variation values of the average Q on the y axis along with the number of training epochs, and MaxQ on the right figure shows the maximum achieved Q value variation while training on the y axis jointly with the number of epochs on the x axis. Shadow variables are copies of the main train variables that are used to maintain the moving average. The idea is that they follow the main variable like a shadow. Table 1 depicts the parameters of the drone agent while testing it in the VCE model. Information given in Table 1 provides the drone agent's coordinates while learning the environment in random episodes as well as rewards of the DQN model drone. In every randomly chosen episode of the training, the location parameters show the drone agent's tracking path in a virtual environment. Reward XY, Z, and +T parameters emphasize the drone agent's adaptation to environmental conditions while analyzing it in sequential actions. It can be seen that while training the realistic VCE, the drone agent obtains negative values by flying in different directions because of environmental space and conditions; behavior of the VCE model affects the result of the rewards. Additionally, Table 2 shows a summarized action Q-value result during the testing process. The result of the tracker in the realistic VCE model is presented in Table 2. The summary results of randomly selected episodes, such as time steps, duration, total reward, average max Q-value, and average loss results of the testing procedure, are displayed in this table. The drone traveled in a random area and identified the item sites to track during the testing phase, which means that the drone flew in a random location and identified the object locations to track. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Parameters**} & \multicolumn{8}{c}{**Episodes (Randomly Chosen)**} \\ \cline{2-11}  & **276** & **279** & **287** & **315** & **333** & **382** & **768** & **815** & **921** & **999** \\ \hline
**Delta X** & 17.08 & 67.51 & 23.14 & 43.24 & 51.45 & 31.46 & 45.12 & 18.53 & 14.36 & 24.47 \\ \hline
**Delta Y** & \(-56.3\) & \(-43.16\) & \(-53.64\) & \(-66.56\) & \(-61.03\) & \(-68.00\) & \(-64.96\) & \(-59.21\) & \(-51.65\) & \(-74.82\) \\ \hline
**Delta Z** & \(-20.89\) & \(-11.40\) & \(-20.66\) & \(-10.07\) & 15.00 & 18.31 & \(-0.36\) & \(-20.66\) & \(-20.46\) & \(-8.57\) \\ \hline
**UAV Vel** & \( \begin{array}{c}0.77,\\ -0.83,\\ 1.03\end{array} \) & \( \begin{array}{c}1.29,-0.1,\\ 0.12,\\ 0.12,\\ -0.75\end{array} \) & \( \begin{array}{c}1.36,\\ -0.79,\\ 0.61,\\ 0.61\end{array} \) & \( \begin{array}{c}0.63,\\ -0.12,\\ 0.12,\\ 1.59\end{array} \) & \( \begin{array}{c}1.19,\\ -0.07,\\ 1.03,\\ 1.13\end{array} \) & \( \begin{array}{c}0.12,\\ -0.92,\\ -1.01,\\ -1.01\end{array} \) & \( \begin{array}{c}1.80,\\ -0.92,\\ -1.55\end{array} \) & \( \begin{array}{c}0.64,\\ -0.89,\\ -0.89,\\ 1.42\end{array}