## 4 Experiment Results and Discussion
### Evaluation and Discussion  
The proposed reinforcement learning-based object tracking algorithm has been explored and evaluated using AirSim [22], a well-known simulation platform that is highly useful and advantageous for exploring freely in a realistic environment simulation. The implementation of the proposed algorithm was connected to the VCE simulation platform with AirSim Python Client that was accomplished by Microsoft developers. Firstly, we trained the network to learn the object classes with environmental feature information, so that it was ready for training with a reinforcement learning approach. After completing the object classification learning process, the output was used in an integrated tracking framework to track objects with a drone agent. Figure 6 shows the evolution of the learning rate and loss function of the trained tracker. It can be seen that learning rate image on the left side illustrates small variation of the learning outcome; due to the environmental conditions, that drone flew in an extensive area and provided a small learning rate. Alternatively, on right side, loss functions results increase during training epochs relatively by learning a large-scale area of the virtual environment. This training was conducted in offline mode to understand the characteristics and details of the object class. For the training method, numerous sequential image sets taken from a realistic virtual world were employed as inputs. Only VCE realistic virtual model pictures were used in the trials to train the DQN tracker, which is based on a recurrent neural network. The figures shown below represent the average predicted action-value MeanQ and MaxQ computed over the held-out set of states. Figure 7 shows the training epochs outcome for computing average action values for MeanQ/MaxQ that examines learning and predicting procedures of the recurrent neural network-based DQN tracker. Figure 7 (left) portrays the evolution of the training procedure by using the virtual realistic images as training data. Figure 8 represents the evolution of both the epsilon score and the used replay memory during the tracker training process. The epsilon greedy exploration graph represents a chosen random action with a probability of epsilon that exploits the best-known action value and can probably reach nearly one epsilon value. Figure 8 (right) displays replay memory, also known as the replay buffer or experience buffer, and contains a collection of experience tuples (_s_, \(a\), \(r\), _s_) with training information. They were added gradually to the buffer as we interacted with the virtual  
Figure 6: Output of the Learning Rate (**left**) and Loss (**right**) function. The blue and red lines represent actual and targeted learned values, respectively. The left side of the learning rate figure y axis represents given parameter learning rate variation value and the x axis represents training epoch numbers. The right side of the image y axis represents loss variation values and the x axis is training epoch value. The shadow copies of trained variables and ops that maintain a moving average of the trained variables in their shadow copies are shown. realistic environment. This act of sampling a small batch of tuples from the replay buffer in order to learn the environment is also known as experience replay. It allowed us to learn the environment more deeply with individual tuples multiple times and make better use of learning experiences during training. In summary, the basic idea of using replay buffer memory, or experience replay memory, is to take advantage of a strong experience and use a random subset of the experience to update the Q-network. Rather than using the last single experience outcome during the tracking process, this action was originally used for the learning tuples of observation state, action, reward, done flag, and next state parameters to keep the obtained transitions from the virtual environment.