Style Randomization Improves the Robustness of Breast Density Estimation in MR Images

Authors: goksenin  yuksel
koen  eppenhof
marcel  worring
Venue: MIDL_io_2024_Conference
Published: 2024-01-29 13:33:06

Abstract: Breast density, a crucial risk factor for future breast cancer development, is defined by
the ratio of fat to fibro-glandular tissue (FGT) in the breast. Accurate breast and FGT
segmentation is essential for robust density estimation. Previous research on FGT segmen-
tation in MRI has highlighted the significance of training on both images with and without
fat suppression to enhance network’s robustness. In this study, we propose a novel data
augmentation technique to further exploit the multi-modal training setup motivated by the
research in style randomization. We demonstrate that the network trained with the pro-
posed augmentation is resilient to variations in fat content, showcasing improved robustness
compared to solely training with multi-modal data. Our method effectively improves FGT
segmentation, thereby enhancing the overall reliability of breast density estimation

Main paper: 
 introduction: 
 Breast density is defined as the amount of fibro-glandular tissue (FGT) in the breast, and it is a well-established risk factor for the future development of breast cancer (Boyd et al., 2007) . While automated software solutions for breast density estimation are readily available for mammography, limited efforts have been directed towards creating an automated robust breast density calculation tool tailored for breast MRI (Chalfant and Hoyt, 2022 ). Yet, MRI allows for more accurate quantification of breast density compared to digital mammography (Gubern-Mérida et al., 2014) .There are multiple ways of performing breast density estimation in MRI (van der Velden et al., 2020) . Recent literature in this area mostly focused on breast density estimation via segmentation (Doran et al., 2017; van der Velden et al., 2020) . Volumetric breast density estimation is then calculated by comparing the number of voxels classified as FGT to the overall volume of the breast.Automatic breast and FGT segmentation in breast MRI has been addressed by several papers (Dalmış et al., 2017; Huo et al., 2021; Zhang et al., 2019) . Recent literature suggests that deep learning methods have consistently shown great results in automating various medical imaging tasks, consistently outperforming traditional methods including breast-FGT segmentation. Dalmış et al. (2017) were the first to propose a deep learning-based approach to breast density estimation, exceeding the performance of previously developed traditional machine learning-based approaches. Later, Zhang et al. (2019) extended the previous research on 2D U-Net for FGT segmentation using a multi-vendor dataset. Later, Huo et al. (2021) used a 3D U-Net, exceeding the performance of all known 2D U-Net architectures on FGT segmentation.The papers mentioned above did not carry out testing on both fat-suppressed (FS) and without fat-suppressed (WOFS) acquisitions. To address this issue, Samperna et al. (2022) trained a 3D U-Net architecture on both FS and WOFS acquisitions. They utilized Dixon images in the training set to include both acquisitions of the same patient. The study demonstrates that incorporating both WOFS and FS acquisitions in the training set enables the achievement of similar results, while training exclusively on one modality leads to sub-optimal performance when tested on the other.Sub-optimal results may be attributed to the style difference in the WOFS and FS images since Convolutional Neural Networks (CNNs) exhibit bias towards styles (Baker et al., 2018) . This style bias does not arise from the inductive properties of the CNNs, but rather from the contents of the training data (Geirhos et al., 2018) . The observed performance improvement in Samperna et al. (2022) may be attributed to the elimination of style bias by including different styles of the same patient.The style bias can be further reduced by appearance-modifying data augmentation (Hermann et al., 2020) . Xu et al. (2021) proposes to modify the amplitude information of images where mostly low-level signals are present, and keep the phase information intact where high-level semantics are present. This augmentation leads to a model that can better extract the semantic concepts, and is more robust to domain shift. Moreover Jackson et al. (2019) introduces style randomization as data augmentation. This technique incorporates various styles of the same image during training. This approach aids in developing a more robust model and shows enhanced performance across multiple domains without requiring specific data from each.In this paper, we apply style randomization with Dixon images to enhance robustness of breast and FGT segmentation, and thus breast density estimation. We argue that this randomization technique introduces more style variability while keeping the high level semantics present. This method forces the network to be more robust by eliminating the style bias even further, and teaches the network to focus more on the shape rather than texture.We find that style randomization helps the network to generalize better, and alleviates the style bias even further. Another important finding is that style randomization improves FGT dice score on both training and test sets. These findings imply that utilizing style randomization during the training not only boosts the robustness of the network but also helps adapt the network to various potential target domains.The Dixon method provides fat-water separation in MR images. The key idea behind Dixon imaging is to acquire multiple images at different echo times, exploiting the chemical shift difference between fat and water protons. Fat and water have slightly different resonant frequencies due to their different chemical compositions, and by acquiring images at specific echo times, it's possible to create separate images highlighting fat and water components (Dixon, 1984) . 
style randomization: 
 By style randomization, we extend on the methodology from Samperna et al. (2022) . Given that Dixon images contain water (W) and fat (F), we randomize the style of the image by introducing mix-up parameter α. We call this Mixed Modality.I =    W FS W + αF 0 < α < 1 W + F WOFSTo randomize the style of an image we follow two strategies. First, we use alpha levels from [0, 0.2, 0.5, 0.8, 1]. Second, we use alpha levels from 0 to 1, with uniform steps of 0.1. From now on, for the sake of readability, we will refer to these as Augmented and More Augmented strategies. We use both strategies to quantify the effect of different style randomization levels. Style randomization is applied intra-patient to achieve linear mixture of WOFS and FS modalities. When the image has 0 or 1 fat weight it results FS and WOFS images.Additionally, we employ a test time augmentation for evaluation purposes. We randomly chose two fat fractions and augment one image. We refer to this strategy as Randomly Weighted Fat.We employ the nn-UNet framework (Isensee et al., 2021) . We used patch-based fullresolution 3D U-Net for all our experiments. We treat each image as a separate sample while using the same ground truth segmentation for the same patients. The network's input was a single patch of size 192 × 192 × 64, with a batch size of 2. We trained all networks for 1000 epochs with 250 iterations per epoch. The default augmentation strategy was applied with instance-based z-score normalization. No post-processing was applied.To implement the proposed method, we employed an in-house training set with Dixon images. They are acquired between the periods between 2015-2017. Breast MRI acquisitions were obtained using a 3T breast MRI scanner. All the acquisitions included in this study were pre-contrast T1-weighted acquisitions. We divided the dataset into training (n = 51 patients) and testing (n = 9 patients, 18 acquisitions (9 WOFS, 9 FS)). Additionally, we employed 5-fold cross-validation. Ground truth labels were artificially generated using a pre-trained network.For independent test sets, we used an evaluation set published by Müller-Franzes et al. (2023) . This is a subset of DUKE, which was collected between 2000 and 2014 at the Duke Hospital, USA, which is publicly available Saha et al. (2021) . This dataset contains acquisitions from 40 patients. The acquisitions were obtained by a 1.5 Tesla, or 3.0 Tesla scanner from General Electric or Siemens. The MRI protocol consisted of a T1weighted fat-suppressed sequence(one pre-contrast, and four post-contrast scans) and a nonfat-suppressed T1-weighted sequence. We used a pre-contrast T1-weighted fat-suppressed image to evaluate our algorithm. We refer to this test set as Müller-Franzes et al. (2023) . Additionally, we utilized published breast segmentation and FGT segmentations by DUKE. This set contains 100 patients and has the same characteristics as the set described above. After processing the masks, we decided to exclude the ones that we could not parse in SITK. We also found that some images has more than 3 dimensions. Due to lack of metadata to guide us on how to use them, we decided to drop these as well. This left us with 88 patients. Later, we only used segmentation labels for breast and FGT. We refer to this test set as Saha et al. (2021) .For the BI-RADS density estimation task, we employed radiologists estimations from DUKE. This data contains BIRADS density label estimations from three radiologists for 50 instances. In all cases two out of three radiologists agreed on the density grade, with the third radiologist estimate differing at most one grade from the other. Therefore we utilized majority voting to consolidate the estimations into a single label. 
results: 
 Table 1 demonstrates that mixed modality training does not show significant performance drift when tested on WOFS or FS modality. We visualize the lowest performance case from the test set. The visualization of the test sample is included in Table 2 . This patient has a breast implant. In the WOFS image, we classify the implant as part of the FGT tissue, therefore breast density prediction is higher than the ground truth. Moreover, Table 1 shows that style randomization improves on mixed-modality training in every dataset. Domain shift does not affect the performance of the networks trained with augmentations. We see that the test performance of mixed modality in Randomly Weighted Fat degraded heavily on FGT segmentation, while augmented networks are not affected by it. Moreover, in independent test sets, more augmented network significantly outperforms mixed modality training. More augmented network demonstrates a significant performance improvement over mixed modality training. Notably, the more augmented network not only suppresses the results of mixed modality and augmentation but also eliminates outliers, such as the case with a breast implant in the test set.Figure 2 illustrates that more augmentation enhances the dice scores of individual cases by up to 0.4 compared to the baseline. Additionally, it reveals that augmentation may result in a deterioration in dice scores for certain cases. Nevertheless, the overall enhancement outweighs the instances of decline. Table 2 shows that failing segmentations from mixedmodality training are over segmented. Whereas a network trained on style randomized images correctly segments the FGT tissue, yielding a better FGT segmentation performance.Pearson's r from Table 1 shows that predicted density estimations from style randomized networks correlate significantly more with the ground truth compared to mixed modality training.Figure 3 visualizes the Bland-Altman plots for numerical agreement between predicted density, and ground truth density. We observe that all models predict the breast density well. The mean disagreement falls between 0.02% to 0.04%. Furthermore, density predictions from networks trained with the proposed augmentation are more precise. The confidence intervals indicate that proposed augmentation has less variance in predicted density error, and is closer to 0. The outliers in the Bland-Altman plots are alleviated with the style randomization. Figure 3 depicts a positive correlation between the density of breast and error in predictions for (Saha et al., 2021) .We quantify the agreement between radiologists and the density predictions from the network using the BI-RADS labeled data. Figure 4 depicts a box plot of BIRADS density labels to the numerical density estimation. All networks show a clear trend of increased density when from left to right agreeing with the radiologists. Whisker in Figure 4 depicts that the network trained on mixed modality over-estimates the density. This phenomenon is especially visible for BIRADS label B. On the other hand, networks trained on style randomization yields more reliable density estimations for all the density labels. This suggests that predicting BIRADS density labels from numerical output is less prone to false positives. 
discussion: 
 In this study, we explored the effect of style randomization for breast density estimation and proposed the implementation using Dixon images. Experimental results indicate that the network trained with style randomization reduces style bias and forces robustness. Firstly, style randomization helps with the out-of-domain distribution performance. This finding is inline with Jackson et al. (2019), and Xu et al. (2021) . Compared to mixed modality training, we do not observe performance degradation in the FGT Dice score, resolving the style bias observed with mixed modality training. This finding suggests that style randomization may improve the generalization capabilities of the network. Furthermore, correctly segmenting the patient with breast implant may suggest that it forces networks to recognize breast and FGT shape better. This finding is reinforced by significantly higher dice scores with augmentation in independent test sets, and better breast density estimations.Style randomization improves the breast density estimates both in BIRADS, and numerical estimation tasks. We find that style randomization significantly improves the correlation between predicted and ground truth density. In Figure 3 and 4 we observe that augmentation alleviates the positive outliers. Additionally, the strong agreement between radiologists' assessments and the network predictions reinforces the reliability of the proposed technique. We observe that all networks overshoot the breast density estimation in denser breasts regardless of style randomization. However, this trend is less noticeable with style randomization.Moreover, we do not observe a statistically significant difference between augmentation strategies in independent test sets. However, we see that using more augmentation further improves the segmentation, and density estimation results in the held-out test set. This finding may emphasize that having more augmentation further reduces the style bias, and enforces more robustness.There are several limitations of the study. Firstly, we only trained two different augmentation strategies with arbitrarily chosen α values. Comparing more augmentation strategies would better quantify the effect of different style randomization levels. Secondly, training dataset relies on labeled information generated by a pre-trained network. This network over-segments the FGT areas, resulting in lower resolution of FGT segmentation. Networks trained on biased data not only fail to alleviate but can also exacerbate these biases. Consequently, irrespective of the chosen training schema, our model tends to exhibit an over-segmentation tendency in FGT, and may not generate high-resolution FGT segmentations. This may explain the trend we observed with breast density estimations in denser breasts. Denser breasts have more FGT tissue, and thus over segmenting it yields erroneous density estimates. 
conclusion: 
 Our paper introduces novel style randomization using Dixon's images, demonstrating its efficacy in enhancing network robustness. The proposed technique further reduces style bias, offering a valuable approach for leveraging Dixon images in breast density estimation. Further research may explore style randomization as random data augmentation and training diverse network families to quantify its broader impact. 
 
 Appendix:  


Decision: Two of the three reviewers support accepting this work as a poster and believe it will generate meaningful discussions. I believe the rebuttal has addressed most of the important comments. The authors are requested to revise the camera-ready version in accordance with the feedback provided in the rebuttal.

Reviews: review: paper_summary: paper_summary: This paper proposes a data augmentation strategy/style randomization technique mixing different MRI modalities which results in more accurate breast and fibro-glandular tissue (FGT) segmentation on Breast MRI. This enables more precise breast density estimation. The authors performed training on an in-house dataset and evaluated the performances of the semantic segmentation model on diverse datasets (in-house and independent test sets).
main_review: main_review: I thank the authors for their answers as it has clarified some points of the paper. I still think that the paper lacks experiments using different mixup settings (inter patient - Intra modality, Inter patient - Inter modality). The proposed method is almost a direct application of mixup on opposing modalities for a very specific segmentation problem, I think it is important that the paper explores the different possible mixup settings as It would prove that the proposed mixing strategy is optimal for this problem. Since, no experiments have been carried out to compare the different possible mixing strategies, I will keep the grade as it was.
strength_weakness: strength_weakness: * The originality/novelty of the work is somewhat limited. Even though the mixing strategy is performed differently than usual (intra-patient and inter-modality). The paper seems to be a very slight adaptation of the mixup data augmentation for a breast MRI segmentation task. Using mixup augmentation in the context of MRI segmentation tasks is already common like in [1].  
* The paper lacks ablation studies and other comparisons with other augmentation strategies:
     * For instance, what happens if the mixup is performed inter-modality but also inter-patient or intra-modality and inter-patient? 
     * How does this method compare to classic and powerful augmentation/style randomization strategies such as Fourier-based augmentation [2]?

References:
* [1] Panfilov, E., Tiulpin, A., Klein, S., Nieminen, M. T., & Saarakkala, S. (2019). Improving robustness of deep learning based knee mri segmentation: Mixup and adversarial domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (pp. 0-0).
* [2] Xu, Q., Zhang, R., Zhang, Y., Wang, Y., & Tian, Q. (2021). A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 14383-14392).
questions: questions: Please address concerns in the weaknesses section.
paper_summary: paper_summary: To alleviate the style bias inherent to segmentation models trained on multi-modality MRI, the authors propose to use style randomization as a data augmentation strategy. They conduct experiments against a baseline and a second augmentation method to show that their proposed approach outperforms both in terms of generalisation/robustness and consequently, accuracy.
main_review: main_review: The basic idea isn't excessively novel, the datasets are not really large, and the improvements are large only in a few cases (well, of course a consequence of the small datasets). 
It's a simple to implement strategy for a quite particular purpose and modality, so nothing wrong about presenting it, just perhaps not quite in an oral.
strength_weakness: strength_weakness: * Style randomization is a catchy word for "linear interpolation between Fat and Water contrast... 
* Ground truth for the parenchyma masks was another algorithm, introducing a bias.
* Ground truth for BI-RADS labels were likely physician-generated, and might be highly variable.
* Claiming that augmentation leads to robustness, I would have expected a validation that shows this: It should show the improvement in relation to the baseline, ie. it should improve the worst baseline cases the most. I think this is an opportunity missed.
* In your comment to figure 4, what do you mean when you say the network "overestimates" the density? Isn't it only important to be able to map the numerical estimate to the proper score (define thresholds)? In fact, looking at the whiskers of group D in the augmented scenarios, they reach into the box area, which they don't for the "mixed modality" scenario -- setting a threshold causes less group confusion! The same holds for the situation between groups B and C, which is the dividing line between "not so dense" and "fairly dense" in a binary decision scenario. This would actually mean that the mapping from the numerical density to the score suffers from the augmentation.
questions: questions: All above ;-)
paper_summary: paper_summary: The work is concerned with breast and fibroglandular tissue (FGT) segmentation in breast MRI for breast density estimation purposes. The novel idea suggested is to interpolate between fat suppressed (FS) and regular T1-weighted images (WOFS) as an augmentation method. Both contrasts can be reconstructed from the same Dixon MR protocol, so they are automatically spatially aligned. The evaluation is done on FS images from an external, public dataset (/ two related ones).
main_review: main_review: I see that I have been by far the most positive reviewer, but I still like the paper, because it gives a very clear message to the community: If you work with Dixon images, this specific simple data augmentation method is definitely worth implementing. The contribution is small in the sense that it is not very surprising and only affects a niche, but if you happen to be in that niche, it appears to be a solid and reliable (small) contribution, and I like such works.

I am not sure how much I agree with the other reviewer about the mixup comparison – on the one hand, the question "what happens if the mixup is performed inter-modality but also inter-patient or intra-modality and inter-patient?" seems to neglect that fact that the "inter-modality" mixup is the only one that interpolates between two contrasts that by definition *come from the same position* (without any registration). I am also not a particular fan of the concept of mixup augmentation, given that the inputs usually do *not* share the same voxel grid and produce very unreasonably looking inputs. On the other hand, I must agree that the work is close enough to mixup, and the technique is relatively common (well-known and used), so it could have been a good candidate for an ablation study.

In the light of the critical discussion, I slightly change the rating to "weak accept", but my overall impression has not changed much and I still think this is a nice contribution worth being discussed at MIDL.

Finally, I thank the authors for the well-formatted replies to our reviews, which were convenient to read.
strength_weakness: strength_weakness: The idea is relatively simple and I can imagine that other reviewers would claim lack of novelty. However, given the otherwise solid paper, I think it's worth being published.

The "ground truth" (I personally dislike that term in medical imaging and prefer others such as "reference") is from "a pre-trained network" the provenance of which is unclear. I think this is the strongest weakness of the paper, since the quality of the "ground truth" used for training is

The training set is not very large (n = 51 patients / Dixon acquisitions), so a larger training set could alleviate the need for this kind of augmentation.  However, it is very hard to hand-annotate FGT.

The augmentation is not done on the fly with random mixing factor α (which would have been easily possible also with the nnU-Net / batch generators framework), but only a few fixed uniform steps between 0 and 1.  This also leads to the strange separation between "augmented" (three interpolated steps) and "more augmented" (eight steps), which would otherwise not have been necessary.
questions: questions: What is the provenance of the model used for FGT "ground truth"?

What do you mean with "[DUKE masks] we could not read" and "were not in the 3D space", respectively?

Why do you use majority voting for the density classes and not some kind of numerical averaging?  Yes, the labels are more ordinal than numerical, but you're already measuring correlation against numerical model outputs.

What do you mean with "we randomly chose two fat fractions and augment *one* image", so why only *one*?

Comments: title: Paper is open for dsicussions
We thank the reviewers for assesing our paper
Rebuttal
Rebuttal
Rebuttal
comment: I thank the authors for their answers as it has clarified some points of the paper. I still think that the paper lacks experiments using different mixup settings (inter patient - Intra modality, Inter patient - Inter modality). Since the proposed method is almost a direct application of mixup on opposing modalities for a very specific segmentation problem, I think it is important that the paper explores the different possible mixup settings. It would prove that the proposed mixup strategy is optimal for this problem.
Dear Reviewers The authors have submitted their rebuttal addressing the raised questions. The paper remains open for further discussion and engagement.
In the rebuttal we address their major concerns point by point.
Furthermore, we address the detailed comments regarding the language, citations, and figures in the revised version of our paper.
We thank the reviewer for their critical assessment of our work. In the following we address their major concerns point by point.


__Reviewer Point 1__ — What is the provenance of the model used for FGT ”ground truth”?

The model used for the ground truth is a pretrained model on a small subset of our data with manual
segmentations of the breast and FGT. This model was used to generate our training set. We manually
inspected the network-generated segmentations and checked for inaccuracies, excluding segmentations
with obvious mistakes.
***
__Reviewer Point 2__ — What do you mean with ”[DUKE masks] we could not read” and
”were not in the 3D space”, respectively?

There were segmentation images that had more than three dimensions. Because there was
no metadata available for us to parse these we decided to discard them. Moreover, we utilized SITK
to read the segmentations, and some could not be read. We decided to discard them due to lack of
metadata as well.
To address this weakness, we will make sure to make this explicit in the paper.
***
__Reviewer Point 3__ — Why do you use majority voting for the density classes and not some
kind of numerical averaging? Yes, the labels are more ordinal than numerical, but you’re already
measuring correlation against numerical model outputs.

We indeed used majority voting because the density classes are ordinal. It turns out that there
would be no difference between averaging or majority voting as in all cases two out of three radiologists
agreed on the density grade, with the third radiologist estimate differing at most one grade from the
other.
We are planning to add further explanation in the methodology section regarding the average pooling,
and used BIRADS labels.
***
__Reviewer Point 4__ — What do you mean with ”we randomly chose two fat fractions and
augment one image”, so why only one?
Mathematically.
    \begin{align*}
        &\alpha_{1} \sim Uniform(0,1)\\
        &\alpha_{2} \sim Uniform(0,1)\\
        &I_{1} = W_{1} + \alpha_{1} * F_{1}\\
        &I_{2} = W_{1} + \alpha_{2} * F_{1}
    \end{align*}
  
We use these to create random augmentation of the test data. Our Test Data contains 18 images [9WOFS, 9FS]. To reach to the same number of cases in our style randomized test data, we augment one image using two fat fractions. It is possible to choose more alpha values, however we decided to go with two for the aforementioned reason.
We thank the reviewer for their critical assessment of our work. In the following we address their major concerns point by point.

__Reviewer Point 1__ — Style randomization is a catchy word for ”linear interpolation between
Fat and Water contrast...

Our paper introduces a new way to augment training data using the linear interpolation
between fat and water images. We extend on the idea of using Dixon images for multi modality
training.  We argue that this falls into the category of style randomization as the ”style” of the image change
while keeping the global structure of the image. Figure 1 shows this phenomenon. Moreover referring
to our method as ”style randomization” places the paper in a more broad scientific context.
However, we do agree that in this case style randomization boils down to relatively simple linear
interpolation.
***
__Reviewer Point  2__ — Ground truth for the parenchyma masks was another algorithm,
introducing a bias.

We are aware of this limitation, and discuss this in the last paragraph of discussion section on
page 8.

> Secondly, training dataset relies on labeled information generated by a pre-trained network.This network over-segments the FGT areas, resulting in lower resolution of FGT segmentation. Networks trained on biased data not only fail to alleviate but can also exacerbate these biases ...
***
__Reviewer Point 3__ — Ground truth for BI-RADS labels were likely physician-generated,
and might be highly variable.

Yes, there are variances to the BIRADS density grades, however in this case two out of three
radiologists always agreed on a grade. And, the odd one out was always one label level off. We also
used average pooling to reduce the variance as much as possible. We will add further explanation in
the methodology section to address this weakness.
***
__Reviewer Point 4__ — Claiming that augmentation leads to robustness, I would have expected
a validation that shows this: It should show the improvement in relation to the baseline, ie. it should
improve the worst baseline cases the most. I think this is an opportunity missed.

Tables 1 and 2 indicate this phenomenon where there is improvement over baseline. Furthermore, Table 2 indicates that the cases that are predicted poorly by mixed modality training are improved by using style randomization. One of the most interesting case comes from MR Image with Breast Implant. Without using style randomization, we see a bad segmentation of breast and FGT tissue, however
with proposed augmentation this phenomenon is fixed and the segmentations are more reliable.
Moreover, Figure 2 supports our claims as well. In the sub figure (a) it is shown that compared to
mixed modality training, augmentation improves a case by 0.40 dice score . Moreover, subfigure (b)
shows that it improves a case by 0.25 dice score.

We also discuss why augmentation could lead to robustness in the discussion section.
Furthermore, correctly segmenting the patient with breast implant may suggest that it forces
networks to recognize breast and FGT shape better. This finding is reinforced by significantly
higher dice scores with augmentation in independent test sets, and better breast density
estimations

To address this weakness, we plan to comment on Table 2 further with respect to ”it should improve
the worst baseline cases the most”.
***
__Reviewer Point 5__ — In your comment to figure 4, what do you mean when you say the
network ”overestimates” the density? Isn’t it only important to be able to map the numerical
estimate to the proper score (define thresholds)? In fact, looking at the whiskers of group D in
the augmented scenarios, they reach into the box area, which they don’t for the ”mixed modality”
scenario – setting a threshold causes less group confusion! The same holds for the situation between
groups B and C, which is the dividing line between ”not so dense” and ”fairly dense” in a binary
decision scenario. This would actually mean that the mapping from the numerical density to the
score suffers from the augmentation.

When network predicts density estimation that is an outlier for the BIRADS density distribution, we call this ”over-estimation”. We show that over-estimation is less prominent when we use
style randomization. Our method pushes edge cases towards the median of the BIRADS label density
distribution. Furthermore, figure 4 shows that augmentation leads to more reliable density predictions.

Augmentation actually causes less group confusion, because the predictions that are over-estimated
were placed in correct vicinity, however this also makes the tails longer. By looking at the outliers that
are alleviated, the reason why whiskers reach to group ”D” is due to ”lowering” the predictions of the
outlier cases to fit into the tails.
To take the stated weakness into account, we plan to further explain figure 4 in the results section.
We thank the reviewer for their critical assessment of our work.
In the following we address their major concerns point by point.

__Reviewer Point 1__ — The originality/novelty of the work is somewhat limited. Even though
the mixing strategy is performed differently than usual (intra-patient and inter-modality). The
paper seems to be a very slight adaptation of the mixup data augmentation for a breast MRI
segmentation task. Using mixup augmentation in the context of MRI segmentation tasks is already
common like in [1].

Thank you for providing further resources for the paper. We’d like to acknowledge that our
method is a mix-up between Fat and Water MR Images. However, the proposed mix-up has a special
property of linearly mixing two modalities when used in intra-patient and inter-modality setting.
The FS and WOFS images can be viewed as two opposing sides of fat content distribution. FS has
no fat, and WOFS has all the fat. Using the proposed augmentation, we then can generate images that
look more like WOFS, or FS. Or, images that do not look like either. In page 3 figure 1, we demonstrate
this effect.
By using mix-up this way, we force the network to focus more on the FGT, and breast rather than
the fat content in the image. Moreover, the methodology in paper [1] was not applied on Dixon Images,
therefore it does not have the special property of generating from the distribution of two opposing
modalities.
Our contribution relies on realising that the use mix-up with Dixon Images could generate possible
MR acquisitions from two opposing modalities, and further. This way we explore the realm of MR
Images with any fat content, and we can go beyond WOFS, and FS domain. Training the segmentation
network with the generated data results in robustness properties. We think this is also refereed as
”Domain Generalisation” in the literature.

***
__Reviewer Point 2__— For instance, what happens if the mixup is performed inter-modality
but also inter-patient or intra-modality and inter-patient?
Reply: As explained above, we achieve linearly mixing two domains when we perform Dixon augmentation on inter-modality and intra-patient setting. Therefore, we think that it is not possible to use this
method for other possible settings, and have performance gains.

As explained above, we achieve linearly mixture of two modalities when we perform Dixon augmen-
tation on inter-modality and intra-patient setting. Therefore, we think that it is not possible to use this
method for other possible settings, and have performance gains

***
__Reviewer Point 3__ — How does this method compare to classic and powerful augmentation/style randomization strategies such as Fourier-based augmentation [2]?

We thank the reviewer for providing an interesting paper on the augmentation literature. We
believe that our proposed method can be viewed as a special case of Fourier augmentation method. In
the paper [2], authors propose to mix amplitude signals of two images linearly, while keeping the phase
signal unchanged. They argue that the phase signal corresponds to the global object structure, thus using such augmentation teaches network to focus more on the global shape rather then local texture.
In our method, we augment water images with the fat content. Due to the reconstruction of water
and fat images, they will have the same global structure, but different local textures. Therefore, in our
method, we are also keeping the global structure intact while mixing the local textures. However, we
deviate from the Fourier transformation by augmenting the local texture where it is necessary. The fat
and water signals are found in the breast’s and the chest. And, that is the most interesting part when
we perform the segmentation task. Moreover, our method is capable of augmenting only local textures
in this area without any need of prior segmentations to mask the region of interest. We will include
this paper in the related research, and discuss the findings with respect to aforementioned paper’s view
point as well as comparing our methodologies.
***
__Reviewer Point 4__ — The paper lacks ablation studies and other comparisons with other
augmentation strategies

Our paper’s main focus is on developing a method that can be used to force the network
to be more robust on the shape rather then the texture. We carried out an ablation study for the
level of augmentation to understand the change in performance with respect to the augmentation level.
Moreover, we compared it with solely training on mixed modality setting as well.