# ODEM-GAN: An Object Deformation Enhancement Model Based on Generative Adversarial Networks
## 3 Approach
### Joint Inference and Training  
Our ODEM-GAN module can work with the original RetinaNet, and all hyperparameters in training and inference are the same as those in RetinaNet. Note that the GAN branch of the ODEM-GAN module only participates in training and does not participate in inference. **Model pretraining:** Experiments show that it is necessary to pretrain ODEM-GAN before using ODEM-GAN to improve the performance of the detector. The specific pretraining process is as follows:  
* The training dataset: a dataset that only contains specific hard deformation objects that need to be detected;
* Use the trained CBAF object detection model to extract the features of specific hard deformation objects and use the online deformable sample creation module to create various hard deformation object samples. Note: During the pretraining process of ODEM-GAN, all parameters of the CBAF object detection model are frozen and not updated;
* Minimize Formulas (6) and (8) to alternately train the discriminator and the generator (note: update the discriminator parameters first, then update the generator parameters) until the deformable samples generated by the generator are enough to be real. Throughout the pretraining process, ODEM-GAN is trained for 30 epochs using a dataset containing only specific hard deformation objects. The purpose of pretraining is to allow the ODEM-GAN module to have a preliminary understanding of specific hard deformation object samples and reduce the difficulty of joint training of the ODEM-GAN module and CBAF object detection model. **Joint pretraining:**  
* The training set of joint training includes two parts: a dataset that only contains specific hard deformation object samples and a dataset that does not contain specific hard deformation object samples;
* Unfreeze all parameters of the CBAF object detection model and update the parameters;
* In each iteration, two images were taken from the datasets containing only specific hard deformation object samples and those without as the input of the network, so batch size = 4;
* After using the detector to extract the feature patches of the image containing specific hard deformation object samples and fuse the context information, use the training method of the pretraining process to update the parameters of the ODEM-GAN;
* Use the generator of ODEM-GAN to generate deformable samples of specific objects;
* Use the detector to extract feature patches that do not contain specific hard deformation object images and fuse context information;
* Place the fused feature patches of the specific hard deformation object images extracted by the detector, the fused contextual feature patches that do not contain the specific hard deformation object images, and the specific hard deformation object samples generated by ODEM-GAN together as a batch. Then, they are taken together as input to the classification and regression modules, and the optimizer is directly used to update the network parameters of the CBAF object detection model. Note that if the other specific category is changed, the GAN branch must be retrained. **Inference:** The GAN branch of the ODEM-GAN module only participates in training and does not participate in prediction. Its prediction process is the same as that of the CBAF object detection model. Therefore, there is no additional calculation during the prediction, and the running speed of the detection algorithm will not be affected. We only decoded box predictions whose confidence score threshold is higher than 0.05 and only selected up to at most 1k top-scoring in each pyramid level. The box predictionsfrom each pyramid level based on the anchor boxes branch and the anchor-free box branch are merged together; then, non-maximum suppression is performed with a threshold of 0.5 to produce the final detection result. **Initialization:** We use ImageNet1k [33] to pre-train the backbone network. For convolutional layers in our CBAF module, the classification and regression layers are initialized as in [13].