You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


---
abstract: 'Early detection of breast cancer has a major contribution to curability, and using mammographic images, this can be achieved non-invasively. Supervised deep learning, the dominant CADe tool currently, has played a great role in object detection in computer vision, but it suffers from a limiting property: the need of a large amount of labelled data. This becomes stricter when it comes to medical datasets which require high-cost and time-consuming annotations. Furthermore, medical datasets are usually imbalanced, a condition that often hinders classifiers performance. The aim of this paper is to learn the distribution of the minority class to synthesise new samples in order to improve lesion detection in mammography. Deep Convolutional Generative Adversarial Networks (DCGANs) can efficiently generate breast masses. They are trained on increasing-size subsets of one mammographic dataset and used to generate diverse and realistic breast masses. The effect of including the generated images and/or applying horizontal and vertical flipping is tested in an environment where a 1:10 imbalanced dataset of masses and normal tissue patches is classified by a fully-convolutional network. A maximum of $\sim0.09$ improvement of F1 score is reported by using DCGANs along with flipping augmentation over using the original images. We show that DCGANs can be used for synthesising photo-realistic breast mass patches with a considerable diversity. It is demonstrated that appending synthetic images in this environment, along with flipping, outperforms the traditional augmentation method of flipping solely, offering faster improvements as a function of the training set size.'
author:
- |
    Basel Alyafi , Oliver Diaz, Robert Marti Computer Vision and Robotics Institute, University of Girona, Spain\
    Center for Digital Medical Imaging. Parc Tauli Hospital Universitari. Institut d’Investigacio i Innovacio Parc Tauli (I3PT). Sabadell, Spain
bibliography:
- 'spie.bib'
title: 'DCGANs for Realistic Breast Mass Augmentation in X-ray Mammography'
---

Introduction {#sec:intro}
============

Breast cancer is the second deadliest cancer in women globally after lung cancer. This disease was the most frequently diagnosed cancer in 154 countries and the first cause of cancer death in women in 100 countries in 2018 [@cancer_stats]. Computer-aided detection (CADe) systems have been a good alternative for double reading strategies in breast cancer screening benefiting from recent advances in supervised deep learning to reduce false negative and false positive cases. Supervised deep learning tools, however, require large amounts of annotated data. Unfortunately, publicly-available medical datasets are usually small and imbalanced (cancer vs non-cancer) due to privacy issues and the high cost of expert annotations. Generative Adversarial Networks (GAN) [@GAN] have shown promising results in synthesising medical images [@liver_aug]. GAN consist of a network (called generator or G) that learns the distribution of the input data implicitly by the aid of another network (called discriminator or D) which, in turn, tries to learn to distinguish real among synthetic images. Deep Convolutional GAN (DCGAN) [@radford_DCGAN], chosen due to training stability, are used to neutralise a wide range of non-pertinent sources of variance given that the dataset has enough examples. Thereafter, the synthetic breast mass images are used to augment an imbalanced dataset for improving the classification performance. All materials are available online for scientific use ([link](https://github.com/Basel1991/Projects/tree/master/master_thesis/code)).

Description of Purpose {#sec:purpose}
======================

To use DCGAN in order to synthesise realistic and diverse breast masses to augment unbalanced datasets in a classification problem. The ultimate goal is to improve the performance of CADe systems in breast mass detection tasks.

Materials {#sec:materials}
=========

The dataset used in this work is OPTIMAM Mammography Image Database (OMI-DB) [@Optimam]. This database includes over 145,000 cases (over 2.4 million images) and comprises unprocessed and processed digital mammograms from the NHS Breast Screening Programme of the United Kingdom. A subset of this database was obtained comprising over 80,000 cases. In this dataset, there are images from four vendors, however, only images belonging to Hologic Selenia Dimensions (Hologic, Inc; Bedford, Massachusetts, USA) were used in this work. This database has expert annotations identifying the image and any clinical observation. A total of 2,215 mass lesion and 22,000 normal tissue patches were extracted with size $128 \times 128$ pixels after applying histogram normalisation. Negative patches (normal tissue) were extracted randomly given that there was no overlap with the background or masses.

Method {#se:methods}
======

DCGAN [@radford_DCGAN] was used in this work with an additional layer for both of G and D to allow the generation of $128 \times 128$ pixel patches. The aim of the generator is to learn the mapping between the latent space (the normal distribution in this case) and the space of breast mass in a sense that it can transform a 200-value vector from the latent space to a breast mass image that can fool the discriminator. The loss functions used to train the DCGAN, originally recommended in Ref. , were cross entropy for D (Equation (\[Ld\])) and the non-saturated loss for G (Equation (\[Lg\])). $$\begin{aligned}
{L}_{D}&=  -E_{x\ \in\ P_x,\ z\ \in P_z}[\log(x) + \log(1-D(G(z)))], \label{Ld}\\
{L}_{G}&= -E_{z\ \in\ P_z}[\ log(D(G(z)))\ ] \label{Lg},\end{aligned}$$ where $P_x$ is the distribution of real breast masses, and $P_z$ is the normal distribution with zero mean and unitary standard deviation. As mentioned in Ref. , this $L_G$ is preferred to $log(1-D(G(z)))$ because it has larger gradients at the beginning of the training process which makes G learn faster.

[0.47]{} ![(a) Training DCGAN. Dotted arrows refer to fake patches. Steps from one to seven are: generate a noise batch z, forward z through G, forward the real and fake batches through D, calculate $L_D$, update D, calculate $L_G$, and update G. (b) The proposed framework for evaluating the DCGAN when used in data augmentation for supporting the minority class in an unbalanced dataset. Four strategies are investigated: ORG for using real images only, GAN for using real and synthetic images, Aug ORG for applying horizontal and vertical flipping on real images only, and Aug GAN for applying horizontal and vertical flipping on real and synthetic images.](GAN.pdf "fig:")

[0.5]{}

Fig. \[fig:GAN\_training\] shows a schematic framework of the DCGAN. For each training iteration, 64 random latent vectors are sampled $z\in P_z;\ P_z=\mathcal{N}(0,1)$ (step 1 in Fig. \[fig:GAN\_training\]). This pure-noise batch is normalized to the range $[-1,1]$ then forwarded through G to generate a batch of fake images (G(z)) (step two). These fake images are normalised to the range $[0,1]$ then forwarded through D to get realism probabilities, see step three with dashed arrows. An equal-size batch of real images is normalised and forwarded through D to learn the boundary between real and fake breast mass spaces, see step three with the dense arrow. In step four, Equation (\[Ld\]) is used to calculate $L_D$, then D parameters are updated (step five). Thereafter, the fake batch is forwarded through D and Equation (\[Lg\]) is used to calculate $L_G$ in step six. Backpropagation is done eventually to update G parameters (step seven). To complete one epoch, these steps are repeated until all the real breast mass patches are covered. As recommended in Ref. , one-sided label smoothing was used to reduce over-confidence problems. In addition, conventional data augmentation, horizontal and vertical flipping, was used for increasing the diversity of the generated images. One critical issue was faced during training was the checkerboard effect in which a grid pattern appears in the synthesised images. The solution was inspired by a talk of Goodfellow [@nips2016] , where the use of different kernel sizes between G and D was suggested. In order to evaluate the trained generator, an augmentation environment was used where a 1:10 imbalanced dataset of masses (positive minority class) and normal tissue (negative majority class) was classified by a fully-convolutional network. In this setting, the classifier has a similar architecture to the DCGAN discriminator. Fig. \[fig:methodology\] shows the pipeline used to evaluate the effect of data augmentation using four different approaches. In addition, these augmentation effects on classification were investigated for different sizes of the training dataset. For training the classifier, $60\%, 6.6\%$ and $33.4\%$ were used for training, validation and testing, respectively. With respect to the positive class (2,215 breast masses), the training part was divided as {$P_k; k \in \{100, 250, 500, 750, 1000, 1300\}$}, where the subscripts refer to the size of the training subset. These subsets were sampled so that each subset is contained in the next larger one. Regarding the negative class (22,000 normal tissue patches), training subsets were designed to have an imbalance ratio of 10 {$N_{1000}, N_{2500}, N_{5000}, N_{7500}, N_{10000}, N_{13000}$}. The four augmentation approaches investigated (see Fig. \[fig:methodology\]) are:

-   *ORG*: using original images, the input for the classifier is $P_k$ as positive images plus $N_k$ as negative.

-   *Aug ORG*: original images were augmented using random horizontal and vertical flipping.

-   *GAN*: the training set of the classifier is $k$ real masses and $1.5 \times k$ synthetic masses as the positive class, and $10 \times k$ normal tissue patches as the negative class.

-   *Aug GAN*: the $1.5\times k$ generated images as well as the real ones were augmented on the fly by random horizontal and vertical flipping.

Because the dataset is imbalanced, F1 score was used as an evaluation metric. This provides equal importance to precision and recall. As observed in Fig. \[fig:methodology\], the test and validation sets were fixed for all k’s. 3-fold cross validation was used to assure reliable results.

Results and Discussion {#sec:results}
======================

Fig. \[fig:synthetic\_samples\] shows two synthetic masses (left column) and two real masses (right column) depicting that DCGAN could generate visually-similar masses to the ones it was trained on. Moreover, Fig. \[fig:F1Score\_augmentation\] shows the F1 score for different training sizes, where each line represents one augmentation approach. The blue line (*ORG*) shows the classification results using the original unbalanced training dataset. As more training images are available, the classifier increases its performance until k=750 where the performance saturates. When comparing the blue and the green lines (*GAN*), the latter shows faster improvements which shows that the generator has learned to unlock unseen images in the real distribution helping the classifier to distinguish masses among normal tissue. If, on the other hand, the original data is augmented using horizontal and vertical flipping (the orange line of *Aug ORG*), the classifier performs similarly to GAN (green) at medium sizes. Finally, the red line (*Aug GAN*) shows the F1 score when random online flipping was applied on the combined real and synthetically-generated images. As can be depicted in the figure, *Aug GAN* outperformed all other modes at any k, except a negligible drop at 1,300, with the maximum improvement at 250 with about 0.09 over ORG approach.

[0.45]{} ![(a) Synthetic (left column) and real (right column) breast mass lesions. (b) F1 score as a function of the size of the positive training set investigating four approaches: ORG, GAN, Aug ORG, and Aug GAN.](synthetic_samples.pdf "fig:")

[.45]{} ![(a) Synthetic (left column) and real (right column) breast mass lesions. (b) F1 score as a function of the size of the positive training set investigating four approaches: ORG, GAN, Aug ORG, and Aug GAN.](F1Score_allFolds.pdf "fig:")

Breakthrough work {#sec:breakthrough_work}
=================

The main contribution of this work is the development of a DCGAN-based model to generate synthetic breast masses. Additionally, the performance of a fully-convolutional network classifier in an imbalanced mammography image dataset was investigated when the training dataset was enriched by including synthetic images, by flipping augmentation, or by a combination of them. This analysis on this scale is the first to the best of our knowledge. This work neither is being nor has been submitted to elsewhere.

Conclusions {#sec:conclusion}
===========

In this study, we used a modified version of DCGAN to generate realistic breast mass patches with dimensions of $128 \times 128$ pixels. These synthetically-generated images were used to increase the size of the training dataset in a breast mass classifier. This was compared with conventional augmentation, i.e. flipping. Results, based on F1 score, suggest that classifiers with a training dataset smaller than 750 cases can greatly benefit from the synthetic images. On the contrary, conventional data augmentation strategies are sufficient for larger datasets.