# ODEM-GAN: An Object Deformation Enhancement Model Based on Generative Adversarial Networks
## 3 Approach
### Network Architecture of the ODEM-GAN Module  
#### 3.2.1 ODEM-GAN Module  
Figure 7 shows the GAN branch in the red-dashed box includes an online deformable sample creation module, a generator \(G\), and a discriminator \(D\). Among them, the online deformable sample creation module is used to choose and create the deformation samples that are difficult to predict by the classification and regression module (which refers to the hard sample feature of the deformation) and uses the deformation samples as the input of \(D\). \(G\) is used to generate diverse deformation feature patches and add them to the merged feature patches with context information, which is used as the input of the classification and regression module so that the detector can learn more features of the deformed samples and improve the purpose of the detector to stably recognize the deformed samples. \(D\) is  
Figure 5: The process of generating an adversary network. Figure 6: Overview of network architecture. used to compete with \(G\) so that \(G\) can generate a variety of deformed samples. Note that the GAN branch only participates in training. Note that GAN can only generate one certain category of object. **Online deformable sample creation module. ** To obtain more hard samples, the most direct method is the horizontal and vertical flipping of existing hard samples. Therefore, this module proposes an online creation module for deformed samples, as shown in Figure 8. In Section 3.1, we obtain the current feature patches and contextual feature patches and the flip operation on the current feature patches and their contextual feature patches to obtain the new deformation samples. Specifically, if the sliding window contains the feature patches of hard samples, the online deformable sample creation module flips contextual feature patches horizontally, vertically, on the left diagonal and right diagonal in the sliding window (see the red-dashed box on the feature map in Figure 8). However, the current feature patches in the sliding window do not contain the feature patches of hard samples, the contextual feature patches are filled with 0, and the flip operation is not performed. The flip operation obtains four new hard sample deformation features (we call the current feature patch and its contextual feature patches features) which are used as the input of the contextual merger layer to obtain new merged feature patches. Then, the new merged feature patches are used as the input of the classification and regression modules, and the sum of their classification loss and regression loss is calculated. What the detector needs is to make it difficult to classify and regress samples, and the purpose of improving robustness  
Figure 8: Online deformable sample creation module. Each sample passed through the CBAF module to compute the focal loss and smooth \(L_{1}\) loss. Then, the level with the largest sum of the two losses is selected as the supervision signal for hard samples. Figure 7: ODEM-GAN module includes a CBAF module (in the green-dashed box), a GAN branch (in the red-dashed box). and detection accuracy by learning these hard samples. The greater the loss of the sample, the more difficult it is for the detector to predict it. Therefore, the merged feature patches corresponding to the maximum loss are selected as the hard samples. Through the above steps, we can complete the online creation of deformation samples. **The design of generator \(G\). ** ODEM-GAN is built on each level of the pyramid feature, and the parameters of each GAN branch are not shared. For ease of presentation, only the ODEM-GAN module of \(PF_{3}\) is introduced, and the others are the same. As shown in Figure 9, the input of \(G\) is the noise vector \(z\) and the current feature patch, where \(z\) can increase the diversity of the generated samples, and the current feature patch is used as the condition of the generator, which can ensure that the expected sample is generated or can reduce the difficulty of training.