# Breast Density Transformations Using CycleGANs for Revealing Undetected Findings in Mammograms
## 2 The Proposed System Overview
### The Deep Learning Module
#### 2.2.3 CycleGANs  
CycleGANs are used to train an image-to-image translation model, which does not depend on paired datasets to learn the mapping between the input and the output images [40]. The key to CycleGAN's success is the idea of an adversarial loss that forces the generated images to be, in principle, indistinguishable from real images. In our work, we adopt the architecture of CycleGAN as proposed by [PERSON] et al. [41], which has shown impressive results for neural-style transfer and super-resolution. Formally, given a source domain X and a target domain Y, CycleGAN aims to learn the mapping of G: X \(\rightarrow\) Y between input and output images such that the G(X) is the translation of the image from domain X to domain Y. Additionally, it also aims to learn a reverse mapping of F: Y \(\rightarrow\) X such that F(Y) is the translation of the image from domain Y to domain X. In our work, the CycleGAN model is used to transform patches from classes (domain) ACR-B, -C, and -D to class ACR-A. The main purpose of this model is to subtract the effect of tissue masking from the breast patches and transform their tissue into class ACR-A. The training of the CycleGAN was performed after creating data belonging to two categories: one containing image patches from class ACR-A and the other constructed by considering all the remaining patches from classes ACR-B, ACR-C, and ACR-D (Figure 6). In Figures 7 and 8, patches from the unpaired categories ACR-A, ACR-B, -C, and -D are shown. The ROI patches in the first row of Figure 7 (belonging to classes ACR-B, ACR-C, and ACR-D) are transformed to the corresponding class ACR-A patches in the second row using the CycleGAN model. In Figure 8, the opposite transformation is depicted. Due to the cycle consistency characteristic of CycleGAN, we can also transform ACR-A patches back to the ACR-A class, thereby examining the convergence of the model (Figure 9). Figure 5: Examples of artificially generated images of ACR-A, ACR-B, ACR-C, and ACR-D classes using the GAN-ACR network. (Note that checkerboard effects and artifacts are present. )Figure 6: The CycleGAN Overview: Domain-A has patches with ACR-A density, and Domain-B has patches with ACR-B, -C and -D density. Figure 7: Transformation of ACR-B, -C, and -D to the ACR-A category. Since data from this categorization are highly unbalanced, the GAN-ACR model described in the previous section was used to produce synthetic data up to a total of 10 million patch images. The CycleGAN was left for several epochs to run (in each epoch the network uses a pair of 10 million image patches, which are randomly shuffled at the end of each epoch). In Figure 10, we present some examples of ROI patch transformations to ACR-A density class with the CycleGAN topology along with the changes in the heatmap between the density transformations. For these cases, even after their change in density, the classification of the patch remains unaltered. We have also noted that in a small number of cases, some artifacts appeared in the lower right corner of the resulted patches, but they do not affect the system's performance in any significant way. Figure 8: Transformation of ACR-A to ACR-D and ACR-C category. Figure 9: Consecutive ACR density transformations via CycleGAN (two times transformations of an ACR-A ROI). An advantage of the utilized translation model is that consecutive applications of density transformations of the input patches do not alter the ACR classification when the input patch falls in the ACR-A category. In Figure 10, we depict the results of two successive density transformations of a malignant ACR-A ROI to the ACR-A class via the CycleGAN. It can be seen that the ROI's density is not altered visually, while the network classifies it again as malignant. All experiments in this work were carried out on a Linux workstation equipped with an NVIDIA RTX 3090 24 GB, GDDR6X. The deep learning models were all implemented in Python 3.8, in Ubuntu 20.04, with TensorFlow 2.8 and Keras 2.8 API.