# Boosting Deep Reinforcement Learning Agents with Generative Data Augmentation
## 6 Discussion  
The key observations of the aforementioned process are reviewed in this section. At first, the experimental results confirm the instability of traditional image augmentation methods in the RL domain. Specifically, random rotation, translation, cutout, and vertical and horizontal flips have been implemented to augment states in the tested environment, similarly to [7; 12; 21]. Random convolutions, color jitter, brightness contrast, and cutout-color have also been examined in the literature but are not applicable to the current environment, since the input images are grayscale (Section 5.1). As outlined in [25; 26], different augmentation types may have a different effect, depending on the environment, and thus, lack generalization. In the examined Atari games, it is evident that their effect on the agent's learning depends highly on the characteristics of the environment. Rotation and cutout are in general the most beneficial ones; however, none of them outperforms the rest in all cases. For example, in the Pong game (Figure 12), the random rotation and \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **Max Reward** & **Timesteps** & **Predictor’s Accuracy** & **Game** \\ \hline
**No augmentation** & 46.02 & 1.9 M & - & \\
**Rotation** & 44.38 & 1.55 M & - & \\
**Translation** & 43.46 & 1.55 M & - & \\
**Cutout** & 41.92 & 1.95 M & - & \\
**Flip** & 17.94 & 1.75 M & - & \\
**Decceiver (0.1)** & **60.42** & 2 M & 0.438 & \\
**Decceiver (0.2)** & 56.38 & 1.8 M & 0.452 & \\ \hline
**No augmentation** & 20.78 & 1.15 M & - & \\
**Rotation** & \(-\)20.60 & 1.5 M & - & \\
**Translation** & \(-\)20.60 & 1.2 M & - & \\
**Cutout** & 20.80 & 1.5 M & - & \\
**Flip** & 3.04 & 1.9 M & - & \\
**Decceiver (0.1)** & **20.83** & 1.1 M & 0.644 & \\
**Decceiver (0.2)** & 20.72 & 1.7 M & 0.644 & \\ \hline
**No augmentation** & 18.88 & 1.95 M & - & \\
**Rotation** & **19.60** & 1.85 M & - & \\
**Translation** & 9.82 & 1.25 M & - & \\
**Cutout** & 16.80 & 1.7 M & - & \\
**Flip** & 11.90 & 1.5 M & - & \\
**Decceiver (0.1)** & 16.28 & 1.65 M & 0.257 & \\
**Decceiver (0.2)** & 19.14 & 1.95 M & 0.368 & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Different augmentation-based agents’ performance (the highest reward per game is indicated in bold). Figure 13: Augmentation methods’ average reward (Riverraid). translation methods have a negative impact on the performance of the agent, as it is unable to achieve positive scores, which is quite reasonable since the paddles (and occasionally the ball), which are the most informative parts of the state, are on the edges of the image, meaning that after applying translations or rotations they could be cropped out of the image. On the contrary, in the Riverraid game (Figure 13), where the most informative part lies in the center of the image, rotation seems to provide more diversity and improve the agent's performance. Concerning the proposed augmentation methodology, it is among the first to consider generative augmentation directly on the original state space, to the best of our knowledge. Recent works have emphasized projecting the states to a latent space and then processing and augmenting the latter representations [11; 28; 43]. The presented algorithm is completely independent of embedding transformation models. Instead, a generative model is trained on raw observations collected by the agent and produces states which can be used directly for training. In this way, it overcomes the aforementioned generalization issues of traditional augmentation techniques without the need to encode the initial information. The boost in the learning process is not the same across all tested environments in our case as well; however, the agent demonstrates a quite consistent performance. In the worst case, it is comparable to the no-augmentation algorithm (Table 1), indicating that it is not highly affected by the attributes of the images.