Generative AI for misalignment-resistant virtual staining to accelerate histopathology workflows

Domain: neuroscience
Published: 2026-03-27
Editorial outcome: minor_revision

Abstract
Accurate histopathological diagnosis typically relies on multiple chemical stains, a process that is labor-intensive, tissue-consuming, and environmentally taxing. While virtual staining offers a faster, tissue-conserving alternative, its clinical adoption is hindered by the requirement for perfectly aligned paired data, which is difficult to obtain due to tissue distortion during chemical processing. We present a robust virtual staining framework that mitigates spatial mismatches through a cascaded registration mechanism. By decoupling image generation from spatial alignment, our method enables high-fidelity staining even from imperfectly paired or misaligned datasets without altering existing model architectures. Our approach significantly outperforms state-of-the-art models across five datasets, showing a remarkable 23.8% improvement in image quality for highly misaligned samples. In blinded evaluations, experienced pathologists achieved 52% accuracy in distinguishing virtual from chemical stains, indicating that the two were indistinguishable. This framework simplifies data acquisition and provides a scalable pathway for integrating virtual staining into routine clinical workflows.

Introduction
Histopathological analysis is a cornerstone of clinical diagnostics, enabling the identification and characterization of disease states based on the microscopic examination of tissue samples [1, 2]. Traditionally, this process relies on multiple chemically stained tissue sections, where each stain highlights specific tissue components or structures. For example, hematoxylin and eosin (H&E) staining, which differentiates cell nuclei from the extracellular tissue matrix, is the most widely used staining technique in histopathology [3]. In contrast, the Periodic Acid-Schiff stain, which specifically targets glycoproteins, is commonly utilized in renal pathology [4]. However, the conventional staining pipeline is resource-intensive, requiring significant time, labor, and the use of chemical reagents that contribute to environmental burdens [5]. These limitations have motivated the development of alternative approaches to streamline histopathological workflows, among which virtual staining has emerged as a promising solution [6, 7, 8]. By digitally transforming label-free tissue images or one stain into target stain, virtual staining techniques offer the potential to save time, conserve tissue, and reduce dependency on chemical reagents, thereby addressing key challenges in modern diagnostic pathology (Fig. 1a).

Despite these advantages, virtual staining methods are not yet widely adopted in clinical settings due to significant technical challenges. Most existing methods rely on well-aligned paired datasets for training, which are difficult to obtain in practice [9, 10]. Chemical staining processes inherently alter tissue structures, making it nearly impossible to achieve precise alignment between unstained and stained tissue sections [11, 12]. Furthermore, a single tissue section cannot undergo multiple staining procedures without causing damage or information loss, leaving most available datasets either unpaired or only roughly paired [6]. This misalignment hinders the performance of current virtual staining methods, particularly for applications requiring pixel-level accuracy [6, 7, 8, 9, 13, 14].

To address the problems caused by image misalignment, the CycleGAN framework [15] has been widely adopted. This approach uses an additional generator and adopts cycle-consistency to train both generators, which usually perform better on unpaired data compared with Pix2pix models [14]. However, the cycle-consistency constraint can cause the model to focus more on image style than content, which may lead to content being lost in the generation process [16, 17]. Another strategy focuses on minimizing the influence of misalignment. For instance, the pyramid Pix2pix model [18] incorporates a multi-scale loss function to enhance robustness against structural misalignment in the data. Furthermore, RegGAN [19] incorporates a registration model to align the generated image with the target image. By using the aligned generated image to compute the loss, RegGAN further reduces the impact of misalignment. Ideally, the generated image should align well with the input image, while the registered generated image should closely match the target image. However, in RegGAN, only the latter condition is explicitly enforced because the generator and registration model are tightly coupled. This coupling lacks additional constraints to ensure accurate alignment between the input image and the generated image.

To address these limitations, we propose a robust virtual staining framework, termed DGR (Decoupled Generation and Registration), which decouples the image generation and registration processes. Specifically, we introduce two constraints: one to maintain alignment between the input image and the generated image, and another to ensure alignment between the registered generated image and the noisy ground truth (GT). By decoupling the generator and registration networks, DGR enables effective virtual staining of roughly paired images. We evaluated DGR against state-of-the-art virtual staining methods across five datasets. Experimental results demonstrate that DGR outperforms existing approaches, particularly in cases of severe misalignment. Notably, DGR achieves a 3.4 dB (23.8%) improvement over baseline methods. Furthermore, DGR significantly reduces the reliance on accurately aligned paired images, which is critical for real-world applications of virtual staining in pathology.

Results
We evaluated our proposed DGR framework across four tasks using five datasets (Fig. 1b). DGR was compared against representative methods spanning four categories: (1) paired image-to-image translation models (e.g., Pix2pix (Isola et al.) [14]); (2) unpaired approaches (e.g., CycleGAN (Zhu et al.) [15], UNSB (Kim et al.) [20], StegoGAN (Wu et al.) [21]); (3) pathology-specific virtual staining models (e.g., BCI (Liu et al.) [18], Her2 (Bai et al.) [13], ASP (Li et al.) [22], ST (De et al.) [23]); and (4) misalignment-robust frameworks (e.g., RegGAN (Kong et al.) [19]). DGR demonstrated consistently superior average performance on the five datasets, achieving a Peak Signal-to-Noise Ratio (PSNR) of 22.914 dB (+4.4%, Fig. 1c), an Structural Similarity Index Measure (SSIM) of 0.676 (+4.0%, Fig. 1d), and an Learned Perceptual Image Patch Similarity (LPIPS) of 0.180 (+7.7%, Fig. 1e). Evaluated on simulated data, DGR consistently outperformed the baseline across multiple noise levels. Notably, under significant misalignment conditions (±5° rotation, ±10% translation, and ±10% scaling), it achieved a substantial PSNR gain of 3.4 dB (23.8%) (Fig. 1f), highlighting its robustness and suitability for real-world clinical applications where precise image alignment is often unattainable.

2.1 Virtual H&E Staining from Label-free Autofluorescence Images
Autofluorescence (AF) imaging offers label-free, intrinsic tissue contrast but lacks the histological detail and diagnostic interpretability of H&E staining. To bridge this gap, we evaluated the capacity of current methods to synthesize diagnostically plausible H&E images from AF inputs. Unsupervised methods like CycleGAN and StegoGAN exhibited poor fidelity due to the absence of pixel-wise supervision, resulting in low PSNR, SSIM, and high LPIPS (Fig. 2a-c). Supervised models such as Pix2pix and BCI substantially improved performance, underscoring the value of paired training. However, their reliance on assumed pixel-level alignment led to artifacts when applied to real-world, roughly aligned data.

To mitigate misalignment, Her2 introduced a multi-scale reconstruction loss, while RegGAN embedded a registration module within the generator. Both strategies enhanced alignment and boosted quantitative scores. DGR, by contrast, explicitly decouples virtual staining (generation) from spatial alignment (registration), enabling the generator to focus exclusively on stain transformation. DGR achieved the highest PSNR (27.80 dB; +5.3% over the next-best method) and SSIM (0.766; +4.8%) on the test set (Supplementary Table 4). It also obtained the best scores on perceptual metrics, LPIPS (0.159) and FID (20.264), and the second-best KID (0.011), only 0.001 higher than the ASP model (Fig. 2d-e and Supplementary Table 5). Given that lower LPIPS and FID values indicate superior visual fidelity, DGR represents a 15.9% and 12.0% relative reduction in perceptual error compared to the next-best performer.

Qualitatively, DGR’s outputs closely recapitulated ground-truth H&E images in both color accuracy and fine morphological detail (Fig. 2g). Mean absolute error heatmaps revealed substantially lower prediction residuals, and intensity profiles across representative tissue axes showed the strongest concordance with GT, achieving Pearson correlation coefficients of 0.908 and 0.745 (Fig. 2f, top and bottom, respectively). These results demonstrate that explicitly separating alignment from staining not only improves quantitative metrics but also enhances diagnostic plausibility.

2.2 Translating H&E to PAS-AB
We next evaluated virtual staining of Periodic Acid-Schiff-Alcian Blue (PAS-AB), a specialized histological stain that highlights mucins and basement membranes. Our proposed method, DGR, consistently outperformed others across pixel-level metrics PSNR and SSIM (Fig. 3a-b, Supplementary Table 6) on both internal and external test sets. Specifically, on the internal test set, DGR achieved a PSNR of 18.677 dB (+1.2%) and an SSIM of 0.547 (+1.0%). On the external test set, DGR demonstrated more pronounced gains, improving PSNR and SSIM by 10.1% and 8.0% over the second-best method, respectively.

In terms of perceptual similarity, DGR produced predictions that were more aligned with the GT in semantic feature space, achieving the lowest LPIPS scores of 0.168 (-3.4% relative error) and 0.220 (-3.1%) on the internal and external test sets, respectively (Fig. 3c). In contrast, while CycleGAN yielded favorable FID and KID scores (Fig. 3d-e, Supplementary Table 7), its generated images exhibited substantial deviations from the GT in both structure and appearance (Fig. 3g). This discrepancy arises because FID and KID rely on features pretrained on natural images, which differ substantially in distribution from histopathology images [24, 25]. To further assess structural fidelity, we compared intensity profiles across representative image regions (Fig. 3f). DGR’s intensity profiles showed the closest alignment with GT, achieving the highest Pearson correlation coefficients of 0.908 and 0.745. Collectively, our quantitative and qualitative analyses demonstrate that DGR is highly effective at translating H&E images into high-fidelity virtual PAS-AB stains.

2.3 Converting H&E to mIHC
Multiplex immunohistochemistry (mIHC) is a cutting-edge technique widely used in biomedical research and clinical diagnostics to simultaneously visualize and quantify multiple protein biomarkers within a single tissue section. Translating routinely acquired H&E-stained images into high-quality virtual mIHC images represents a compelling yet highly challenging task, with significant implications for reducing reliance on costly and time-consuming re-staining procedures. In this study, we leveraged a publicly available paired H&E-mIHC dataset to evaluate the performance of DGR in this translation task [26].

Quantitative evaluation demonstrates that DGR achieves the highest fidelity across pixel- and perceptual-level metrics: PSNR of 27.306 (+5.1%), SSIM of 0.830 (+2.5%), and LPIPS of 0.248 (-3.9% relative error), outperforming all competing methods (Fig. 4a-c, Supplementary Table 8). In terms of distribution-based metrics, Pix2pix attained the lowest FID (33.184), while RegGAN achieved the best KID (0.013) (Fig. 4d-e, Supplementary Table 9). However, despite its favorable FID score, Pix2pix’s generated images exhibit substantial structural and textural deviations from the GT. In contrast, DGR produces outputs that closely resemble chemically stained mIHC images in both appearance and biomarker localization (Fig. 4g). Intensity profile analysis further confirms that DGR’s predictions align most closely with GT across spatial regions (Fig. 4f).

2.4 Normalizing Staining Style of H&E Image
To further assess the versatility of DGR in stain-related tasks, we evaluated its performance on H&E stain normalization using a publicly available benchmark dataset [27].

Quantitative results demonstrate that DGR outperforms all competing methods in pixel-level fidelity, achieving a PSNR of 23.823 (+1.7% over RegGAN) and an SSIM of 0.734 (+2.1%) (Fig. 5a-b, Supplementary Table 10). In terms of perceptual similarity, DGR performs comparably to RegGAN, with only a marginal difference of 0.005 in LPIPS (Fig. 5c). Notably, DGR also achieves the best performance on distribution-level metrics, attaining the lowest FID (10.253) and KID (0.007) scores (Fig. 5d-e, Supplementary Table 11). Visual inspection also shows that DGR produces stain-normalized images with more consistent color appearance and structural preservation compared to other approaches (Fig. 5g). Further analysis of intensity profiles demonstrates that DGR’s outputs align most closely with the GT, achieving the highest PCC of 0.982 (Fig. 5f).

2.5 Blinded Human Evaluation
Across four distinct stain transformation tasks, DGR consistently achieves state-of-the-art performance, generating images that are both quantitatively accurate and visually realistic. However, high fidelity in automated metrics does not necessarily imply perceptual indistinguishability to expert human observers, particularly to pathologists, for whom visual plausibility is critical.

To assess whether DGR’s outputs are perceptually indistinguishable from chemically stained images, we conducted a blinded evaluation with an experienced pathologist. We randomly selected 250 virtually stained H&E images alongside their 250 chemically stained counterparts, and similarly compiled 250 virtually stained and 250 chemically stained PAS-AB image pairs. For each staining type, the 500 images were shuffled and presented in random order. The pathologist was asked to classify each image as “real” (chemically stained) or “virtual” (computationally generated).

For H&E images, the resulting confusion matrix (Fig. 1g) reveals near-chance performance, with an overall accuracy of only 52%. A chi-squared test confirmed no statistically significant difference in the pathologist’s responses between real and virtual images. Similarly, in the PAS-AB task, the pathologist was unable to reliably differentiate between chemically and virtually stained images (Fig. 1h). Additional examples of generated H&E and PAS-AB images are provided in Supplementary Figure 4 and Supplementary Figure 5, respectively.

Together, these results demonstrate that DGR not only outperforms existing methods across quantitative benchmarks but also produces virtual stains that are perceptually indistinguishable from real, chemically stained images, even to an expert pathologist. This strongly supports the clinical potential of DGR for real-world diagnostic applications where visual authenticity is essential.

2.6 Downstream Task Evaluation
To validate the clinical utility of the virtually stained images generated by DGR, we conducted comprehensive downstream task evaluations on two publicly available histopathology datasets: UniToPatho for colorectal polyp classification and dysplasia grading [28], and GCHTID for gastric cancer tumor microenvironment decomposition [29]. The central hypothesis of this evaluation is that high-fidelity virtual staining should not only produce visually plausible results but also enhance or at least preserve the diagnostic information required by downstream machine learning models. To test this, we adopted a two-stage experimental protocol: (1) train a baseline classifier using only H&E images; and (2) train a multimodal classifier that fuses H&E images with corresponding virtually stained mIHC images generated by each method under comparison.

On UniToPatho dataset, the H&E-only model achieved 0.699 accuracy. When augmented with DGR-generated mIHC images, performance improved to 0.723, a relative gain of 2.4% (Supplementary Table 14). Among all compared methods, DGR was the only one that consistently improved both accuracy and F1-score (Fig. 6a). On GCHTID dataset, which comprises eight histologically distinct tissue types, the H&E-only baseline attained 0.660 accuracy. Integration of DGR-synthesized mIHC images increased accuracy to 0.672 (+1.2%, Supplementary Table 13), again outperforming all competing approaches. While StegoGAN and the ST method also exceeded the baseline, several other methods (e.g., CycleGAN), led to performance degradation, likely due to staining artifacts or loss of diagnostically critical structures (Fig. 6b). Confusion matrices further illustrate that the H&E+DGR’s virtual mIHC reduces misclassification rates across most tissue classes (Fig. 6c-f).

To further assess structural fidelity at the cellular level, we conducted a nuclei segmentation analysis using Cellpose [30, 31]. We used the chemically stained HE2PAS-AB dataset as GT and applied the same segmentation pipeline to virtual stains generated by each method. Segmentation accuracy was quantified using the Dice coefficient (DICE), with higher values indicating better preservation of cellular morphology. DGR achieved the highest overall DICE score of 0.422 (Supplementary Table 12), indicating superior structural consistency with real stains. Qualitatively, DGR’s output exhibits well-defined nuclear boundaries and morphological clarity that closely match GT (Fig. 6k), further confirming its ability to preserve diagnostically relevant tissue architecture.

Collectively, these findings demonstrate that DGR not only produces virtual stains that are superior in visual and quantitative fidelity but also generates representations that are functionally valuable for real-world diagnostic tasks. When combined with blinded human evaluation showing pathologist-level indistinguishability, the consistent gains in downstream classification performance provide compelling evidence for the clinical applicability of DGR, particularly in settings where mIHC is desired but limited by cost, tissue availability, or workflow constraints.

2.7 Misalignment Resilience
Pixel-level misalignment is a pervasive challenge in real-world virtual staining due to physical sectioning artifacts, tissue deformation, and imperfect slide registration. To evaluate the robustness of DGR under such conditions, we curated a high-quality, spatially aligned dataset of 11,918 H&E-PAS-AB patch pairs (256×256 pixels). We then synthetically introduced five increasing levels of misalignment by applying random combinations of rotation, translation, and scaling to the input images (Fig. 6j).

We first evaluated the baseline model [13] and observed a pronounced and consistent degradation in PSNR, SSIM, and LPIPS as misalignment severity increased (Fig. 6g-i, Supplementary Table 15), highlighting its sensitivity to spatial discrepancies. RegGAN, which incorporates a registration module, exhibited improved resilience, showing a smaller performance drop under severe misalignment, confirming that explicit alignment helps mitigate this issue. In contrast, DGR consistently outperformed all competing methods across all five misalignment levels. Notably, at the most severe misalignment level (Level 5), DGR achieved relative improvements of 3.7% in PSNR, 8.4% in SSIM, and 10.7% in LPIPS over the second-best method. This robustness stems from DGR’s decoupled architecture, which explicitly separates image generation from spatial alignment, enabling each component to specialize without interference. The effectiveness of this disentanglement is further corroborated by the deformation fields produced by the registration networks, which accurately capture complex tissue distortions (Supplementary Figure 3).

In summary, DGR’s ability to jointly optimize generation and registration, while keeping the two processes distinct, yields a virtual staining framework that is not only quantitatively superior but also highly resilient to real-world misalignment. This represents a critical advancement toward deploying virtual staining in clinical settings, where perfect spatial correspondence between serial sections is rarely achievable.

Methods
4.1 Ethics declarations
This project has been reviewed and approved by the Human and Artefacts Research Ethics Committee (HAREC) of Hong Kong University of Science and Technology. The protocol number is HREP-2024-0212.

4.2 Problem Setting
To aid in the illustration of our method, we introduce the relevant notation in this section. A misaligned image pair is denoted as (x_n, y_n), where x_n and y_n belong to two different modalities and are spatially misaligned. We assume the existence of a well-aligned counterpart y^_n for each x_n; however, y^_n is not available in real-world scenarios. Our objective is to train a model using the misaligned dataset [(x_1, y_1), ..., (x_N, y_N)], which consists of N pairs, and achieve performance comparable to that of the well-aligned dataset [(x_1, y^_1), ..., (x_N, y^_N)].

4.3 Baseline
The virtual staining methods based on the Pix2pix framework [14] rely on pixel-level L1 loss for optimization, as shown in Equation 1. However, this framework does not account for misalignment that may be present in the paired images. Therefore, the performance of the framework is heavily dependent on the degree of alignment between the paired images.

min_G L(x,y) = 1/N * sum_{n=1}^N ||G(x_n) - y_n||_1 (1)

To alleviate the influence of misalignment, Kong et al. [19, 37] proposed RegGAN based on the idea of “loss correction" [38] for the magnetic resonance image translation task. This work introduced a registration network (R) to estimate the deformation field \phi between the generated image G(x) and the noisy ground truth y, and then use \phi to re-sample G(x) for loss computation. This process can be represented by the following formula:

\phi_n = R(G(x_n), y_n) (2)

min_{G,R} L(x,y) = 1/N * sum_{n=1}^N ||\phi_n \circ G(x_n) - y_n||_1, (3)

where \circ represents a re-sampling operator. In the RegGAN, the authors assumed the noisy ground truth can be expressed as a displacement error, i.e., y = \phi \circ y^. However, it is worth noting that Equation 3 demonstrates that the RegGAN method only guarantees the re-sampled generated image \phi \circ G(x) is well-aligned with the noisy ground truth y, and does not explicitly ensure that G(x) is well-aligned with the corresponding ideal ground truth y^. Since y^ is well aligned with the input image x, the misalignment between G(x) and y^ also implies that the generated image may be not well aligned with the input image. Supplementary Figure 1a illustrates the limitations of solely relying on the registration module (RegGAN [19]) and presents the ideal solution. To address this limitation, we propose an approach that incorporates additional constraints to ensure the alignment between G(x) and ideal ground truth y^.

4.4 Decoupling Generation and Registration
Based on the analysis of RegGAN, our findings suggest that the generation and registration processes are inherently interdependent. This occurs because the correction loss function is applied to the re-sampled image \phi \circ G(x), while the generated image G(x) itself is not subject to strong constraints such as a correction loss. As a result, the generation and registration stages must be carefully balanced and coordinated to achieve optimal results. Drawing inspiration from GAN models [39], we introduce an adversarial registration process to decouple generation and registration ensuring that the generated image G(x) is accurately aligned with the ideal output y^. The framework of our method is shown in Supplementary Figure 1b including registration for noise reduction module and position-consistency generation module for forcing generator to stain input image only without changing the position of pixels.

Registration for Noise Reduction. To reduce the influence of misalignment between G(x) and y, we first adopt a registration network (R_1) to align the generated image G(x) with the “noisy" ground truth y. With the assistance of registration for noise reduction, the reconstruction loss can be represented as following equation:

L_{L1}(x,y) = 1/N * sum_{n=1}^N ||R_1(G(x_n), y_n) \circ G(x_n) - y_n||_1, (4)

where R_1(G(x_n), y_n) is the deformation field between generated image G(x) and “noisy" ground truth y. Additionally, the structural similarity (SSIM) metric is also adopted to improve the structural similarity between generated images and target images. To use this metric, we utilize following equation:

L_{ssim}(x,y) = -1/N * sum_{n=1}^N log(1 + ssim(R_1(G(x_n), y_n) \circ G(x_n), y_n)) (5)

where ssim is function that can calculate the structural similarity between two images.

To prevent overfitting of the deformation field predicted by the registration network R_1, we utilize a smoothness regularization as shown in the following equation:

L_{smooth}(x,y) = 1/N * sum_{n=1}^N ||\nabla R_1(G(x_n), y_n)||_2, (6)

where \nabla indicates differential operator. To make the model generate realistic images, we also adopt generative adversarial training, the adversarial loss for the generator G and the discriminator D is shown in Equations 7 and 8.

L_{adv}(x) = -1/N * sum_{n=1}^N log(D(G(x_n))) (7)

L_{dis}(x,y) = -1/N * sum_{n=1}^N [log(1 - D(G(x_n)) + log(D(y_n))] (8)

The total loss for the Registration for Noise Reduction (RNR) is expressed as the following equation.

L_{RNR}(x, y) = \alpha L_{L1}(x, y) + \beta L_{ssim}(x, y) + \gamma L_{smooth}(x, y) + \delta L_{adv}(x) (9)

Position-consistency Generation. With the constraint imposed by the registration for noise reduction module, the re-sampled generated image \phi \circ G(x) and the “noisy" ground truth y are encouraged to be aligned, which improves the credibility of the reconstruction loss (e.g., L_{L1}). However, a critical limitation persists: there is no explicit constraint ensuring that the generated image itself, G(x), is aligned with the input image, x. This absence of direct constraint can lead to a coupled optimization where the generator G learns to hide structural misalignment in its output, expecting the subsequent registration network R_1 to correct it. This coupling contradicts the desired objective of a virtual staining model, which should perform a pure color transformation without altering the spatial position of cellular structures. The intuition behind our approach is that an ideal virtual staining process should be spatially consistent. Since the input image x (e.g., autofluorescence) and the target stain y (e.g., H&E) are acquired from the exact same physical tissue section, they are inherently paired and share an identical underlying biological structure. The primary difference between them is their visual appearance (modality), not their geometry. Therefore, a correctly generated image G(x) must constitute a pixel-wise translation of the input x into the target stain’s style. Any spatial misalignment between G(x) and x is not a property of the data but an artifact of an imperfectly trained generator. Our goal is to enforce this spatial consistency, ensuring the generator learns to change only the stain appearance, analogous to applying a color filter that does not distort the image.

To achieve this goal, we design a constraint as defined in Equation 10 to optimize R_2.

L_{T1}(x) = 1/N * sum_{n=1}^N ||R_2(x_n, G(x_n)) \circ G(x_n) - G(x_n)||_1 (10)

However, relying solely on this constraint will prevent the registration network R_2 from learning informative deformation field. R_2 could simply produce an identity deformation field for any given paired images, failing to extract useful information. To prevent R_2 from circumventing the learning process, we need an additional constraint that compels R_2 to accurately predict the true deformation field.

From Equation 10, we can see that generated image G(x) and the input image x will be well aligned. This alignment allows us to conclude that the deformation field between input image x and target image y is identical to the deformation field between generated image G(x) and target image y. This property can be leveraged to facilitate adversarial training of the registration network R_2. To this end, we utilize the loss function defined in Equation 11:

L_{T2}(x,y) = 1/N * sum_{n=1}^N ||R_2(x_n, y_n) \circ G(x_n) - R_2(G(x_n), y_n) \circ G(x_n)|| (11)

Here, we aim to minimize the L1 distance between the deformation field of x_n and y_n, and the deformation field of G(x_n) and y_n. This loss function encourages registration network R_2 to effectively learn how to align cross-modal images (i.e., the input image and its corresponding target image). Additionally, similar to Equation 6, we also adopt smooth regularization for R_2 to avoid overfitting.

L_{T1R}(x) = 1/N * sum_{n=1}^N ||\nabla R_2(x_n, G(x_n))||_2 (12)

L_{T2R}(x,y) = 1/N * sum_{n=1}^N ||\nabla R_2(x_n, y_n)||_2 (13)

Training Strategy. The above registration for noise reduction and position-consistency generation modules include generator G, discriminator D, registration networks R_1 and R_2. We regard training as two adversarial processes. Firstly, we update the parameters of generator G and registration network R_1, the loss can be represented as:

min_{G,R1} L(x,y) = L_{RNR}(x,y) + \epsilon L_{T1}(x). (14)

Based on this loss, we can infer that the re-sampled image R_1(G(x_n), y_n) \circ G(x_n) will be aligned well with the target image y, however, it can not ensure that the input image x can be well aligned with the translated image G(x) due to that R_2 could simply output identity mapping for all images to minimize L_{T1}(x). To avoid laziness of R_2, we update the registration network R_2 using constraints described in the position-consistency generation module as shown in following equation.

min_{R2} L(x,y) = L_{T1}(x) + \gamma L_{T1R}(x) + L_{T2}(x,y) + \gamma L_{T2R}(x,y) (15)

Finally, we update the discriminator using the loss function defined in Equation 8.

This training strategy enables us to jointly train the generator and both registration networks, while also ensuring that the generated image is well-aligned with the input image. By incorporating adversarial components into our training process, our model can effectively learn informative features and achieve superior performance.

4.5 Visualization of decoupling strategy
To gain a clear understanding of proposed DGR model, we intentionally introduced misalignment into the paired data. Then we visualized the generated images and the related registered images (Supplementary Figure 1c). It is evident that the input image x and the generated image G(x) are properly aligned, meanwhile the registered image G(x)_{M1} also shows alignment with the target image y. The decrease of mean absolute error (MAE) of error map |G(x)_{M1} - y| compared to |G(x) - y| further supports the efficacy of registration network R_1. To evaluate the efficacy of R_2, the error maps |G(x)_{M2} - G(x)| and |G(x)_{M2} - G(x)_{M1}| demonstrates that the deformation field M_2 produced by R_2 with input x and G(x) closely resembles an identity mapping, aligning with our intended objective. However, challenges persist for R_2 in achieving perfect alignment between input and target images, indicating potential for improvement in its performance.

4.6 Datasets
To validate the effectiveness of DGR, we adopted five datasets for various virtual staining tasks. The size and number of images are reported in Supplementary Table 1.

AF2HE Dataset. The AF2HE dataset comprises 15 whole slide images (WSIs) of breast and lung cancer tissue samples [40]. The specimens were initially imaged in their native autofluorescence (AF) state using a widefield microscopy system equipped with a 265 nm ultraviolet excitation source and a 10× objective lens, enabling high-contrast nuclear visualization without staining [41]. Following AF imaging, the same slides were subjected to standard hematoxylin and eosin (H&E) staining and rescanned under a 20× objective lens using a whole-slide scanner. The image pairs were rigorously aligned using the VALIS framework to ensure spatial correspondence [42]. Subsequently, the registered WSIs were partitioned into 128 × 128 patches, resulting in a total of 50,447 paired patches for training and 4,422 for testing.

HE2PAS-AB Dataset. This dataset was collected from clinical specimens at the Prince of Wales Hospital in Hong Kong. It consists of WSIs from which a total of 10,727 aligned patch pairs (128 × 128 pixels) were extracted for training and 1,191 for testing. Each tissue section underwent sequential staining: initially with H&E, followed by de-staining via acid alcohol treatment, and finally restaining with PAS-AB. All WSIs were aligned using the VALIS framework to ensure precise spatial registration between staining modalities. To rigorously assess model generalizability across different patient populations within the same institution, an external validation set containing 2,841 patches was additionally constructed. These patches were sourced from separate tissue sections and distinct patient cohorts that did not overlap temporally or spatially with the training or test samples, thereby ensuring independent evaluation.

HEMIT Dataset. The dataset employed in this work is derived from the collection introduced by Bian et al. [26], consisting of histologically paired H&E and multiplex immunohistochemistry (mIHC) whole-slide images originating from the ImmunoAlzer [43]. A key advantage of this dataset is that both staining protocols were applied sequentially on the same tissue section, thereby minimizing structural discrepancies and ensuring high pixel-level alignment—a notable improvement over conventional consecutive-section approaches. The mIHC staining incorporates three biologically relevant markers: DAPI for nuclear visualization, panCK for tumor region delineation, and CD3 for T-cell identification, each corresponding to a distinct channel in the image (blue, red, and green, respectively). We adopt the official partitioning of the data into training, validation, and test sets with 3717, 630, and 945 image pairs, respectively, ensuring that each partition contains tissue samples from different patients to prevent data leakage. To reduce computational demands during model training, we extract a central region of 512×512 pixels from each original 1024×1024 patch. Additionally, all H&E images undergo stain normalization to enhance consistency and generalization across samples.

A2H Dataset. The A2H dataset was re-organized by Kang et al. [27], and the images were obtained from the MITOS_ATYPIA ICPR-14 challenge [44]. Original WSIs were stained with H&E and digitized using two distinct slide scanners (Aperio Scanscope XT and Hamamatsu Nanozoomer 2.0-HT). The processed images have a size of 256 × 256. This dataset consists of 19,200 pairs for training and 7,936 pairs for testing. It is used to evaluate the performance of the proposed method in the stain normalization task.

Well Aligned HE2PAS-AB Dataset To investigate the resilience of DGR on datasets with varying degrees of misalignment, we extracted paired patches of size 512×512 from H&E and PAS-AB WSIs. These patches were then registered, resulting in a training set of 10,727 pairs and a test set of 1,191 pairs, each sized 256×256 pixels. Through this registration process, the images were aligned to the greatest extent possible. Subsequently, we introduced five levels of misalignment involving rotations, translations, and scalings to the input images. The input and target images were then center-cropped to 128×128 pixels for model training. Detailed configurations for these misalignments are provided in Supplementary Table 15.

UniToPatho Dataset. The UniToPatho dataset is a labeled histopathological dataset designed for colorectal polyp classification and adenoma dysplasia grading, comprising 8,699 image patches [28]. It includes the following classes: 545 Hyperplastic Polyps (HP), 950 samples of Normal tissue (NORM), 454 Tubular Adenomas with High-Grade dysplasia (TA.HG), 3,618 Tubular Adenomas with Low-Grade dysplasia (TA.LG), 916 Tubulo-Villous Adenomas with High-Grade dysplasia (TVA.HG), and 2,186 Tubulo-Villous Adenomas with Low-Grade dysplasia (TVA.LG). For downstream task evaluation, we randomly partitioned the dataset into training, validation, and test sets in a 7:1:2 ratio, respectively.

GCHTID dataset. The Gastric Cancer Histopathology Tissue Image Dataset (GCHTID) is a labeled histopathological dataset for gastric cancer tumor microenvironment (TME) decomposition, containing 31,096 non-overlapping 224×224 patches extracted from 300 whole slide images of H&E-stained gastric cancer tissues [29]. It includes eight tissue classes: Adipose (ADI), Debris (DEB), Lymphocytes (LYM), Mucus (MUC), Smooth Muscle (MUS), Normal mucosa (NORM), Stroma (STR), and Tumor (TUM), with exactly 3,887 patches per class. For downstream evaluation, we randomly split the dataset into training, validation, and test sets in a 7:1:2 ratio.

4.7 Experiments Setting
For the models utilized in this study, we adhered to the hyperparameters recommended in the original papers for training. Our method employs the Adam optimizer [45] for all models. Specifically, for the AF2HE, HE2PAS-AB, and A2H datasets, the learning rate for G is set to 5e-5, while for the remaining HEMIT dataset, the learning rate is decayed to 1e-5. A comprehensive list of the hyperparameters is provided in Supplementary Table 2. All experiments were conducted using PyTorch [46] on a workstation equipped with eight NVIDIA GeForce RTX3090 GPUs.

Computational Efficiency To evaluate the computational efficiency of the proposed method, we compare the training speed, memory consumption, and inference time of various models (Supplementary Table 3). As shown in the results, our approach achieves highly efficient inference since the registration modules are not used during this stage, making it particularly suitable for practical deployment. Even during training, our method remains significantly faster than traditional unsupervised approaches such as CycleGAN, demonstrating the computational advantages of the DGR architecture.

Metrics and Statistical Analysis To evaluate the generated images, we adopt Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS) [47], Fréchet Inception Distance (FID) [24], and Kernel Inception Distance (KID) [25]. PSNR and SSIM quantify pixel-level fidelity between generated and reference images, while LPIPS measures perceptual dissimilarity based on deep feature representations. In contrast, FID and KID assess the similarity between the distributions of generated and real images in feature space, with KID offering an unbiased estimator of distributional divergence. Additionally, we utilized a two-tailed independent samples t-test for statistical analysis unless other methods were explicitly stated.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Ma et al. present a new virtual staining method to overcome the issues generated by the image/slide misalignment. The\nmethod shows some promising results and outperform other methods in the tests presented. Nevertheless, there are two\ntasks which are taken into account (registration and virtual staining) and they should probably be treated as such to allow a\nfair comparison among methods. The fact that Kong et al. method, the only one from the ones tested which includes a\nprocess for registration, is performing very similarly to the proposed method strengthens this concern.\nThe registration module should be compared to other recently presented methods such as http://nature.com/articles/s41592-\n024-02199-5 and https://www.nature.com/articles/s41467-021-25221-2. The best performing registration method should be\ncoupled with the virtual staining methods, especially when exploring the effect of misalignment. This exercise would strongly\nstrengthen the confidence in the presented results and showcase the improvement over the state-of-the-art.\nExploring the computational efficiency (e.g. RAM, processing time) of the proposed method compared to the others would\nallow for a more complete comparison. Additionally, the paper would benefit from a detailed analysis of computational\nefficiency, including memory usage, training time, and inference speed, to assess the practicality of deploying DGR in\nclinical settings. While the authors have made their code and some datasets publicly available, further exploration of\ngeneralization to external datasets and visualization of deformation fields would enhance interpretability and trust in the\nmodel. Overall, DGR represents a significant advancement in virtual staining, but its full potential and comparative\nadvantage would be more convincingly demonstrated through broader benchmarking and efficiency analysis.", "reviewer_2": "Strengths of the Works:\n1. The study leverages a comprehensive set of five datasets, encompassing H&E to IHC, mIHC to H&E, and H&E to mIHC\ntransformations. This diverse coverage enhances the robustness of the results and makes the findings both compelling and\nbroadly applicable.\n2. The ablation study across varying levels of alignment is a valuable and noteworthy contribution, offering meaningful\ninsights that will undoubtedly benefit the field.\nI have identified several weaknesses in the current draft, which need to be rectified, addressed, or explained well:\n1. Methodology and Premise Problems with DGR:\n(a) Key Premise Difference Between RegGAN and the Proposed DTR Method: (I) RegGAN was originally designed to\ngenerate and register CT and MRI scans of the same patient, either acquired simultaneously or at different time points. While\nthere may be minor anatomical variations between the scans, the underlying assumption—that both images represent the\nsame brain—is largely reasonable. (II) In contrast, this assumption does not hold for certain datasets under consideration in\nyour work. Specifically, some of these datasets(all except HEMIT and AF2HE, I think) consist of tissue samples obtained\nfrom adjacent but distinct sections. While the overall tissue structure may appear similar, the actual cellular content differs\nbetween sections. As a result, the assumption that the two images represent the \"same\" tissue breaks down. Consequently,\nRegGAN’s foundational premise is not applicable in these cases, and your proposed DGR method is designed to address\nthis more challenging scenario. Therefore I do not believe that the model your proposed is logically sound.\n(b) Claim in line 610 column 2, that in RegGAN “No explicit constraints are imposed on the generated image G(x)” is wrong.\nBecause adversarial loss is explicitly on the generated images. So i do not think that statement is completely correct.\n(c) Because X_n and Y_n are inherently unpaired(with ideal ground truth not available), why do we want the generated\nY=G(X_n) to be registered to X_n? Given that X_n and Y_n​ are inherently different, why is pixel-level registration\nnecessary? It feels analogous to registering apples and oranges. While structural alignment using edges—as done in prior\nwork [13, 14]—is reasonable, pixel-wise registration as proposed here lacks intuitive justification. I believe this is one of the\nkey contributions of the paper, but the explanation and motivation are currently not well articulated. The work appears to offer\nlimited methodology novelty(with RnR being essentially same as RegGAN), and the reasoning behind the PcG module is\ninsufficiently explained, which I find to be a significant weakness.\nBased on the three points above, the underlying intuition behind the proposed methods—particularly in the context of current\npublic virtual staining datasets—appears either irrelevant, poorly articulated, or fundamentally flawed. That said, the method\ndoes yield promising results, which may justify some leniency. However, there are also notable issues within the results\nsection that warrant attention(explained in the second point below).\n2. Problems with Results:\n(A) I understand DGR being better and used for other unpaired datasets, but what’s the use of using DGR for paired datasets\nHEMIT and AF2HE? As both have aligned edges and cells, I feel as though they are not needed as the images are already\nwell registered.\n(B) Comparison Done is pretty old(Limited Comparisons): All the comparisons done are older models\nPyramidPix2Pix(2022), RegGAN(2021), Pix2Pix and CycleGAN(2016-17)None of the latest models are compared such as\n(I) Li, Fangda, Zhiqiang Hu, Wen Chen, and Avinash Kak. \"Adaptive supervised patchnce loss for learning h&e-to-ihc stain\ntranslation with inconsistent groundtruth image pairs.\" In International Conference on Medical Image Computing and\nComputer-Assisted Intervention, pp. 632-641. Cham: Springer Nature Switzerland, 2023.(MICCAI 2023)\n(II)Kim, Beomsu, Gihyun Kwon, Kwanyoung Kim, and Jong Chul Ye. \"Unpaired image-to-image translation via neural schr\\\"odinger bridge.\" arXiv preprint arXiv:2305.15086 (2023).(ICLR 2024)\n(III) Wu, Sidi, Yizi Chen, Samuel Mermet, Lorenz Hurni, Konrad Schindler, Nicolas Gonthier, and Loic Landrieu. \"Stegogan: Leveraging steganography for non-bijective image-to-image translation.\" In Proceedings of the IEEE/CVF Conference on\nComputer Vision and Pattern Recognition, pp. 7922-7931. 2024.(CVPR 2024)\n(IV) And Many others which has been shown by multiple works to perform well such as “Kataria, Tushar, Beatrice Knudsen,\nand Shireen Y. Elhabian. \"StainDiffuser: multitask dual diffusion model for virtual staining.\" arXiv preprint arXiv:2403.11340\n(2024).” to work well for Virtual staining tasks.\n(C) Inconsistent Results on HEMIT dataset: In the HEMIT paper, the reported PSNR for their baseline ranges between 27–\n28 dB. In contrast, the values reported in your paper are significantly lower across all models, with even the best-performing\nDGR model yielding inferior PSNR and SSIM compared to HEMIT’s method. This discrepancy is concerning and warrants\nfurther explanation.\n(D)Blinded Human Evaluation study.\n(I) I have concerns about the study's design. It focuses solely on assessing whether humans can distinguish between real\nand virtual IHC images—essentially replicating what metrics like FID and KID already quantify. However, this approach\ndoes not provide any insight into whether the virtual IHC images accurately capture the underlying biological staining\npatterns. How can we evaluate or communicate the accuracy of the staining with respect to the actual cellular structures?\n(II) Paired datasets, such as those used in [13] and HEMIT, do not face this issue since they are pixel-aligned on the same\ntissue. However, for unpaired datasets, I believe the human evaluation approach described above is not appropriate.\n(E) No distribution-based metrics are reported, such as FID, KID, etc.\n3. Other Writing and Dataset Details Missing\n(A) Issues in naming conventions in figures and Tables, I think shorthand for methodology like “PyramidPix2Pix”, “Pix2Pix”,\n“RegGAN” would be easily understood than “Liu et al, Isola et al” etc. This is more confusing and less readable.\n(B) I found Figure 1(f) unclear—it's not evident what the reader is expected to observe, and the figure appears incomplete.\n(C) The dataset section lacks sufficient detail, making it unclear which datasets are paired (i.e., two stains applied to the\nsame tissue section, such as AF2HE and HEMIT) and which are unpaired(stains on adjacent tissue sections, I am guessing\nthe other 3 remaining datasets). This ambiguity makes it difficult to interpret the results accurately, as the experimental setup\nand evaluation depend heavily on whether the data are paired or unpaired.\n(D) No References Added for multiple claims made in the Introduction: Line 090 Column 1, starting with “However, the\nconventional ..”, Line 080 Column 2, starting with “Most existing methods..”, Line 082 Column 2, starting with “Chemical\nstaining inherent alters”, Line 085 Column 2, starting with “Furthermore, a single tissue ..”\n(E) How can multi-scale generation by PyramidPix2Pix be claimed to mitigate effects of misalignment (line 156, column 1)? I\ndo not think even the authors claimed that."}, "author_response": "Dear Reviewers,\nWe sincerely appreciate your valuable feedback and constructive comments, which\nhave greatly contributed to improving the quality of our work. We have carefully ad-\ndressed each of the points raised and provided detailed responses to all comments. For\nyour convenience, we have organized our responses by each reviewer:\n• Reviewer #1\n• Reviewer #2\nPlease feel free to navigate to the respective sections using the links above. Should you\nrequire any additional information or clarification, we would be happy to provide it.\nOnce again, we thank you for your time and effort in reviewing our manuscript.\nBest regards,\nHao Chen\nReviewer #1:\nComment 1: Ma et al. present a new virtual staining method to overcome the issues\ngenerated by the image/slide misalignment. The method shows some promising results\nand outperform other methods in the tests presented. Nevertheless, there are two\ntasks which are taken into account (registration and virtual staining) and they should probably\nbe treated as such to allow a fair comparison among methods. The fact that Kong et al.\nmethod, the only one from the ones tested which includes a process for registration, is\nperforming very similarly to the proposed method strengthens this concern. The regis-\ntration module should be compared to other recently presented methods such\nas http://nature.com/articles/s41592-024-02199-5 and https://www.nature.com/arti-\ncles/s41467-021-25221-2. \nResponse:\nThank you for this insightful comment and for suggesting these highly relevant refer-\nences. We agree that a rigorous comparison of the registration component is crucial for\nvalidating our method's contribution. In our study, the primary focus was to develop an\nend-to-end virtual staining framework that inherently corrects for misalignments, rather\nthan to propose a novel registration module. Such methods could reduce the time of\nprocessing data for training models and speed up research and applications.\nBased on your suggestions, we added another four latest state-of-the-art models (ASP,\nUNSB, StegoGAN, ST) [1-4] for comparison. We have updated the manuscript accord-\ningly.\nRevised Figures:\nRevised Tables:\nIn the draft, all tables have been updated.\nAF2HE:\nHE2PAS-AB:\nHEMIT:\nA2H:\nReferences:\n[1] https://github.com/lifangda01/AdaptiveSupervisedPatchNCE\n[2] https://github.com/cyclomon/UNSB\n[3] https://github.com/sian-wusidi/StegoGAN\n[4] de Haan, K., Zhang, Y., Zuckerman, J.E. et al. Deep learning-based transformation\nof H&E stained tissues into special stains. Nat Commun 12, 4884 (2021).\nhttps://doi.org/10.1038/s41467-021-25221-2\nComment 2: The best performing registration method should be coupled with the vir-\ntual staining methods, especially when exploring the effect of misalignment. This exer-\ncise would strongly strengthen the confidence in the presented results and showcase the\nimprovement over the state-of-the-art.\nResponse:\nWe agree that integrating a high-performance registration method is essential when\nstudying the impact of misalignment, and that such a combination would more clearly\ndemonstrate the advantages of our approach. In our study, we employed VALIS [1], a\nstate-of-the-art registration tool, to pre-align all training and testing data. We have ex-\npanded the Dataset section to provide greater clarity and avoid potential misunderstand-\nings. The updated text now includes the following details:\n“AF2HE Dataset. The AF2HE dataset comprises 15 whole slide images (WSIs) of\nbreast and lung cancer tissue samples [39]. The specimens were initially imaged in\ntheir native autofluorescence (AF) state using a widefield microscopy system equipped\nwith a 265 nm ultraviolet excitation source and a 10× objective lens, enabling high-\ncontrast nuclear visualization without staining [40]. Following AF imaging, the same\nslides were subjected to standard hematoxylin and eosin (H&E) staining and rescanned\nunder a 20× objective lens using a whole-slide scanner. The image pairs were rigor-\nously aligned using the VALIS framework to ensure spatial correspondence [41]. Sub-\nsequently, the registered WSIs were partitioned into 128 × 128 patches, resulting in a\ntotal of 50,447 paired patches for training and 4,422 for testing.”\n“HE2PAS-AB Dataset. This dataset was collected from clinical specimens at the\nPrince of Wales Hospital in Hong Kong. It consists of WSIs from which a total of 10,727\naligned patch pairs (128 × 128 pixels) were extracted for training and 1,191 for test-\ning. Each tissue section underwent sequential staining: initially with H&E, followed by\nde-staining via acid alcohol treatment, and finally restaining with PAS-AB. All WSIs\nwere aligned using the VALIS framework to ensure precise spatial registration between\nstaining modalities. To rigorously assess model generalizability across different patient\npopulations within the same institution, an external validation set containing 2,841\npatches was additionally constructed. These patches were sourced from separate tissue\nsections and distinct patient cohorts that did not overlap temporally or spatially with\nthe training or test samples, thereby ensuring independent evaluation.”\nFurthermore, to specifically evaluate the robustness of our method to misalignment, we\nintentionally introduced artificial distortions to the pre-aligned data. As demonstrated\nin the figure below, while our method already outperforms baselines on well-aligned\ndata, the improvement becomes even more substantial under severe misalignment con-\nditions (e.g., Level 5). These results confirm that our approach is particularly effective\nin handling misaligned inputs, underscoring the core contribution of our work.\nReferences:\n[1] Gatenbee C D, Baker A M, Prabhakaran S, et al. Virtual alignment of pathology\nimage series for multi-gigapixel whole slide images[J]. Nature communications, 2023,\n14(1): 4502.\nComment 3: Exploring the computational efficiency (e.g. RAM, processing time) of\nthe proposed method compared to the others would allow for a more complete compar-\nison. Additionally, the paper would benefit from a detailed analysis of computational\nefficiency, including memory usage, training time, and inference speed, to assess the\npracticality of deploying DGR in clinical settings.\nReferences:\nThank you for your valuable suggestions. We have now included an analysis of com-\nputational efficiency, covering memory usage, inference time, and training speed, to\nfacilitate a more comprehensive comparison and better assess the practical viability of\ndeploying our method in clinical environments.\nThe memory usage and inference time of each model have been added to our evaluation.\nSince training time is highly dependent on dataset size, we report the average pro-\ncessing speed across all models. All experiments were conducted using an NVIDIA L20\nGPU.\nThe tested results are as follows:\nFurthermore, we disccused this problem in the Disscution Section:\n“However, the generalization capability of the evaluated method still has room for im-\nprovement when applied to external datasets. Additionally, the current DGR framework\nrequires relatively higher computational resources compared to some baseline methods,\nwhich may limit its deployment in resource-constrained scenarios (Extended Data Ta-\nble A3). To address these limitations, future research should focus on curating large-\nscale, diverse datasets to enhance robustness and adaptability, along with optimizing\nthe model architecture to improve efficiency. Developing advanced generative AI mod-\nels, such as diffusion models, for virtual staining also presents a promising opportunity\nto standardize the virtual staining process across various stain types. By expanding\ndataset size, leveraging cutting-edge generative AI techniques, and improving compu-\ntational efficiency, we can produce virtually stained images that rival the quality of\nchemically stained ones, while avoiding the inherent drawbacks of chemical staining.”\nComment 4: While the authors have made their code and some datasets publicly avail-\nable, further exploration of generalization to external datasets and visualization of de-\nformation fields would enhance interpretability and trust in the model. Overall, DGR\nrepresents a significant advancement in virtual staining, but its full potential and com-\nparative advantage would be more convincingly demonstrated through broader bench-\nmarking and efficiency analysis.\nReferences:\nThank you for your positive recognition of our work and for these constructive sugges-\ntions. We have further addressed the points you raised regarding generalization and\ninterpretability.\nAs recommended, we actually have included an external validation experiment to eval-\nuate the generalization capability of different models. The results are summarized in\nthe table below:\nIn addition, we have visualized the deformation fields learned by the registration mod-\nules of our DGR framework and provided an analysis in the Supplementary Information\n(Section 2.6 Misalignment Resilience). The added content is as follows:\n“The effectiveness of the disentangled process is further visualized through the defor-\nmation fields generated by R1 and R2 (Extended Data Fig. A2). In conclusion, the DGR\nframework effectively disentangles generation and registration processes, yielding a\nmore robust virtual staining model capable of handling misaligned image pairs.”\nThese additions aim to provide deeper insight into the model's behavior and strengthen\nthe evidence for its general applicability and comparative advantage.\nReviewer #2:\nStrengths of the Works:\nComment 1: The study leverages a comprehensive set of five datasets, encompassing\nH&E to IHC, mIHC to H&E, and H&E to mIHC transformations. This diverse coverage\nenhances the robustness of the results and makes the findings both compelling and\nbroadly applicable. The ablation study across varying levels of alignment is a valuable\nand noteworthy contribution, offering meaningful insights that will undoubtedly benefit\nthe field.\nResponse:\nThank you for your positive feedback and for recognizing the value of our multi-dataset\nvalidation and the ablation study on alignment. We are pleased that you found the\ndataset diversity and the misalignment analysis to be compelling and broadly applicable\ncontributions to the field. We have carefully considered your subsequent comments and\nprovide our point-by-point responses below.\nComment 2: I have identified several weaknesses in the current draft, which need to be\nrectified, addressed, or explained well. Methodology and Premise Problems with DGR:\n(a) Key Premise Difference Between RegGAN and the Proposed DTR Method: (I) Reg-\nGAN was originally designed to generate and register CT and MRI scans of the same\npatient, either acquired simultaneously or at different time points. While there may be\nminor anatomical variations between the scans, the underlying assumption—that both\nimages represent the same brain—is largely reasonable. (II) In contrast, this assumption\ndoes not hold for certain datasets under consideration in your work. Specifically, some\nof these datasets(all except HEMIT and AF2HE, I think) consist of tissue samples ob-\ntained from adjacent but distinct sections. While the overall tissue structure may appear\nsimilar, the actual cellular content differs between sections. As a result, the assumption\nthat the two images represent the \"same\" tissue breaks down. Consequently, RegGAN’s\nfoundational premise is not applicable in these cases, and your proposed DGR method\nis designed to address this more challenging scenario. Therefore I do not believe that\nthe model your proposed is logically sound.\nResponse:\nThank you for your insightful comments and for raising these important methodological\nconsiderations. We appreciate your deep understanding of the challenges in image reg-\nistration and virtual staining.\nYou are correct in pointing out that a core premise of methods like RegGAN is the\nassumption that the input and target images represent the same underlying biological\nstructure. In our previous draft, the dataset section lacks details leading to potential\nmisleading.\nTo address this, we have now added detailed descriptions of the data collection and\nprocessing procedures for each dataset to the manuscript. The key point is that all five\ndatasets used in our study do, in fact, consist of image pairs derived from the very\nsame physical tissue section, thereby fulfilling the essential premise for registration-\nbased generative models.\nBelow we provide a point-by-point explanation for the datasets in question:\n1. A2H Dataset: This dataset is sourced from the MITOS_ATYPIA ICPR-14\nchallenge [1] and the work of Kang et al. [2]. The original Whole Slide Image\n(WSI) was stained with H&E and then scanned by two different slide scanners\n(Aperio Scanscope XT and Hamamatsu Nanozoomer 2.0-HT). The image from\nthe Aperio scanner is used as the input, and the image from the Hamamatsu\nscanner is the target. Therefore, both images originate from the same physical\ntissue section. The inherent challenges in perfect alignment arise from differ-\nences in scanner resolution and mechanical errors, which is precisely why this\ndataset is suitable for evaluating misalignment resilience.\n2. HE2PAS-AB Dataset: The H&E and PAS-AB stained images are also obtained\nfrom the same tissue section. Our collaborators at the hospital prepared the\nslides using the following protocol: the tissue was first stained with H&E and\nscanned; then, acid alcohol [3] was used to destain the H&E dye; finally, the\nsame exact tissue section was restained with PAS-AB and scanned again. The\nexternal validation data for HE2PAS-AB was created using an identical process.\nThe corresponding additions to the manuscript are as follows:\n“AF2HE Dataset. The AF2HE dataset comprises 15 whole slide images (WSIs) of\nbreast and lung cancer tissue samples [39]. The specimens were initially imaged in\ntheir native autofluorescence (AF) state using a widefield microscopy system equipped\nwith a 265 nm ultraviolet excitation source and a 10× objective lens, enabling high-\ncontrast nuclear visualization without staining [40]. Following AF imaging, the same\nslides were subjected to standard hematoxylin and eosin (H&E) staining and rescanned\nunder a 20× objective lens using a whole-slide scanner. The image pairs were rigor-\nously aligned using the VALIS framework to ensure spatial correspondence [41]. Sub-\nsequently, the registered WSIs were partitioned into 128 × 128 patches, resulting in a\ntotal of 50,447 paired patches for training and 4,422 for testing.”\n“HE2PAS-AB Dataset. This dataset was collected from clinical specimens at the\nPrince of Wales Hospital in Hong Kong. It consists of WSIs from which a total of 10,727\naligned patch pairs (128 × 128 pixels) were extracted for training and 1,191 for test-\ning. Each tissue section underwent sequential staining: initially with H&E, followed by\nde-staining via acid alcohol treatment, and finally restaining with PAS-AB. All WSIs\nwere aligned using the VALIS framework to ensure precise spatial registration between\nstaining modalities. To rigorously assess model generalizability across different patient\npopulations within the same institution, an external validation set containing 2,841\npatches was additionally constructed. These patches were sourced from separate tissue\nsections and distinct patient cohorts that did not overlap temporally or spatially with\nthe training or test samples, thereby ensuring independent evaluation.”\nReferences:\n[1] Roux, L.: Detection of mitosis and evaluation of nuclear atypia score in breast can-\ncer histological images. In: 22nd International Conference on Pattern Recognition\n(2014)\n[2] Kang, H., Luo, D., Feng, W., Zeng, S., Quan, T., Hu, J., Liu, X.: Stainnet: a fast and\nrobust stain normalization network. Frontiers in Medicine 8, 746307 (2021)\n[3]https://www.biognost.com/wp-content/uploads/2019/12/Acid-alcohol-IFU-V9-\nEN4.pdf\nComment 3: (b) Claim in line 610 column 2, that in RegGAN “No explicit constraints\nare imposed on the generated image G(x)” is wrong. Because adversarial loss is explic-\nitly on the generated images. So i do not think that statement is completely correct.\nResponse:\nThank you for this correction. You are absolutely right to point out the inaccuracy in\nour original statement. The adversarial loss does indeed act as an explicit constraint on\nthe generated image. Thank you for pointing this out and helping improve the precision\nof our description.\nOur intended meaning was to clarify that a correction loss (e.g., a pixel-wise between\nthe generated and target images) is not applied directly to G(x), but rather to the regis-\ntered output ϕ ◦ G(x). To eliminate any misunderstanding and to accurately reflect the\nframework's design, we have revised the sentence as follows:\nPrevious:\n“This is due to the fact that the loss function is based on the re-sampled image ϕ ◦ G(x),\nwith no explicit constraints imposed on the generated image G(x).”\nAfter:\n“This occurs because the correction loss function is applied to the re-sampled image\nϕ◦G(x), while the generated image G(x) itself is not subject to strong constraints such\nas a correction loss.”\nComment 4: (c) Because X_n and Y_n are inherently unpaired(with ideal ground truth\nnot available), why do we want the generated Y=G(X_n) to be registered to X_n? Given\nthat X_n and Y_n are inherently different, why is pixel-level registration necessary? It\nfeels analogous to registering apples and oranges. While structural alignment using\nedges—as done in prior work [13, 14]—is reasonable, pixel-wise registration as pro-\nposed here lacks intuitive justification. I believe this is one of the key contributions of\nthe paper, but the explanation and motivation are currently not well articulated. The\nwork appears to offer limited methodology novelty(with RnR being essentially same as\nRegGAN), and the reasoning behind the PcG module is insufficiently explained, which\nI find to be a significant weakness.\nResponse:\nThank you for your comments. Regarding your first point concerning the “inherently\nunpaired” nature of the data, we apologize for any confusion caused by the insufficient\ndescription of our dataset. In fact, all image pairs in our dataset are inherently paired,\nas both the input and target images are acquired from the same tissue region. This en-\nsures that they are spatially corresponding, making pixel-level alignment both mean-\ningful and necessary. We have provided further details about the dataset in our response\nto Comment 2.\nRegarding the second point on methodological novelty: We appreciate your insight, and\nwe agree that the Registration for Noise Reduction (RNR) module shares a similar\nstructure to RegGAN. However, the key contribution of our work lies not in RNR alone,\nbut in the proposed decoupled framework. We identified that RegGAN’s coupled gen-\neration and registration process may lead to suboptimal alignment of the generated im-\nage with the input. To address this, we introduced the Position-consistency Generation\n(PcG) module, which explicitly constrains the generator to produce outputs that are\npixel-wise aligned with the input—thereby “decoupling” the translation and registra-\ntion tasks. This allows the generator to focus solely on stain transformation without\nintroducing structural shifts, while the registration network handles misalignment cor-\nrection.\nTo demonstrate the effectiveness of this decoupling strategy, we conducted experiments\nunder varying degrees of misalignment. The results show that while both RegGAN\n(equivalent to “Baseline + RNR”) and our full model (“Baseline + RNR + PcG”) out-\nperform the baseline in the presence of misalignment, our model achieves superior per-\nformance due to the additional constraints from PcG. This confirms that the proposed\ndecoupling mechanism is beneficial and novel.\nWe acknowledge that the original manuscript did not sufficiently explain the motivation\nand mechanism of the PcG module. Therefore, we have revised the corresponding sec-\ntion to provide a clearer justification of the pixel-level registration objective and its role\nin ensuring position-consistent generation. We have also improved the figure captions\nto better illustrate the concept and results. Thank you again for your valuable feedback,\nwhich has helped us improve the clarity and contribution of our work.\nFigure with Revised Caption:\nRevised PcG Section:\n“Position-consistency Generation. With the constraint imposed by the registration for\nnoise reduction module, the re-sampled generated image ϕ ◦ G(x) and the “noisy”\nground truth y are encouraged to be aligned, which improves the credibility of the re-\nconstruction loss (e.g., LL1). However, a critical limitation persists: there is no explicit\nconstraint ensuring that the generated image itself, G(x), is aligned with the input im-\nage, x. This absence of direct constraint can lead to a coupled optimization where the\ngenerator G learns to “hide” structural misalignment in its output, expecting the sub-\nsequent registration network R1 to correct it. This coupling contradicts the desired ob-\njective of a virtual staining model, which should perform a pure color transformation\nwithout altering the spatial position of cellular structures. The intuition behind our ap-\nproach is that an ideal virtual staining process should be spatially consistent. Since the\ninput image x (e.g., autofluorescence) and the target stain y (e.g., H&E) are acquired\nfrom the exact same physical tissue section, they are inherently paired and share an\nidentical underlying biological structure. The primary difference between them is their\nvisual appearance (modality), not their geometry. Therefore, a correctly generated im-\nage G(x) must constitute a pixel-wise translation of the input x into the target stain’s\nstyle. Any spatial misalignment between G(x) and x is not a property of the data but an\nartifact of an imperfectly trained generator. Our goal is to enforce this spatial con-\nsistency, ensuring the generator learns to change only the stain appearance, analogous\nto applying a color filter that does not distort the image.”\nComment 5: Based on the three points above, the underlying intuition behind the\nproposed methods—particularly in the context of current public virtual staining da-\ntasets—appears either irrelevant, poorly articulated, or fundamentally flawed. That said,\nthe method does yield promising results, which may justify some leniency. However,\nthere are also notable issues within the results section that warrant attention(explained\nin the second point below).\nResponse:\nThank you for your constructive feedback. We appreciate your comments and have ad-\ndressed each of the points you raised in our previous responses. We acknowledge that\nthe initial description of the dataset and methodology lacked sufficient detail, which\nmay have led to misunderstandings. We have revised the relevant sections to clarify\nthese points and to better articulate the motivation and applicability of our method\nwithin the context of existing public virtual staining datasets. We are grateful for your\ninsightful suggestions, which have greatly helped us improve the manuscript.\n2. Problems with Results:\nComment 6: (A) I understand DGR being better and used for other unpaired datasets,\nbut what’s the use of using DGR for paired datasets HEMIT and AF2HE? As both have\naligned edges and cells, I feel as though they are not needed as the images are already\nwell registered.\nResponse:\nThank you for this insightful comment. You are correct that the HEMIT and AF2HE\ndatasets are provided as pre-registered paired datasets. However, even in such carefully\naligned datasets, subtle non-linear distortions and local misalignments are often intro-\nduced during the physical processes of tissue staining, fixation, and slide scanning.\nThese imperfections are practically unavoidable. The HEMIT paper also reported this\nproblem “Despite both H&E and mIHC staining being performed on the same tissue\nslide, the re-staining and scanning processes mean that the captured images do not align\nperfectly.”[1].\nOur DGR module is specifically designed to be robust against such misalignments. It\nacts as a form of regularization, preventing the model from learning spurious, pixel-to-\npixel correlations that can arise from even minor registration artifacts. This improves\nthe model's generalization and ensures it learns the actual biological and structural map-\nping between stains, rather than relying on perfect spatial correspondence.\nAs our ablation studies show, the DGR provides a significant boost in performance and\nrobustness, particularly when misalignment is present. By handling these real-world\nimperfections intrinsically, the DGR reduces the dependency on computationally heavy\nand often imperfect pre-processing, thereby facilitating more reliable and scalable vir-\ntual staining workflows in digital pathology.\nReferences:\n[1] Bian C, Philips B, Cootes T, et al. HEMIT: H&E to Multiplex-immunohistochem-\nistry Image Translation with Dual-Branch Pix2pix Generator[J]. arXiv preprint\narXiv:2403.18501, 2024.\nComment 7: (B) Comparison Done is pretty old(Limited Comparisons): All the com-\nparisons done are older models PyramidPix2Pix(2022), RegGAN(2021), Pix2Pix and\nCycleGAN(2016-17)None of the latest models are compared such as:\n(I) Li, Fangda, Zhiqiang Hu, Wen Chen, and Avinash Kak. \"Adaptive supervised\npatchnce loss for learning h&e-to-ihc stain translation with inconsistent groundtruth\nimage pairs.\" In International Conference on Medical Image Computing and Computer-\nAssisted Intervention, pp. 632-641. Cham: Springer Nature Switzerland, 2023.(MIC-\nCAI 2023)\n(II)Kim, Beomsu, Gihyun Kwon, Kwanyoung Kim, and Jong Chul Ye. \"Unpaired im-\nage-to-image translation via neural schr\\\"odinger bridge.\" arXiv preprint\narXiv:2305.15086 (2023).(ICLR 2024)\n(III) Wu, Sidi, Yizi Chen, Samuel Mermet, Lorenz Hurni, Konrad Schindler, Nicolas\nGonthier, and Loic Landrieu. \"Stegogan: Leveraging steganography for non-bijective\nimage-to-image translation.\" In Proceedings of the IEEE/CVF Conference on\nComputer Vision and Pattern Recognition, pp. 7922-7931. 2024.(CVPR 2024)\n(IV) And Many others which has been shown by multiple works to perform well such\nas “Kataria, Tushar, Beatrice Knudsen, and Shireen Y. Elhabian. \"StainDiffuser: multi-\ntask dual diffusion model for virtual staining.\" arXiv preprint arXiv:2403.11340 (2024).”\nto work well for Virtual staining tasks.\nResponse:\nThank you for your suggestions. We use the public availability code released by paper\n(I) [1], (II) [2], and (III) [3]. Since the paper (IV) did not release the code, we ignore\nthis one. Additionally, as suggested by the reviewer 1. We also include another work\n[4]. We strictly follow the implementation of their released code and replace with our\ndatasets. The results show that the DGR and RegGAN are still the best performing\nmodels on the misaligned paired datasets. The experimental results are as follows:\nUpdated Tables:\nIn the draft, all tables has been updated.\nVisualized figures:\nReferences:\n[1] https://github.com/lifangda01/AdaptiveSupervisedPatchNCE\n[2] https://github.com/cyclomon/UNSB\n[3] https://github.com/sian-wusidi/StegoGAN\n[4] de Haan, K., Zhang, Y., Zuckerman, J.E. et al. Deep learning-based transformation\nof H&E stained tissues into special stains. Nat Commun 12, 4884 (2021).\nhttps://doi.org/10.1038/s41467-021-25221-2\nComment 8: (C) Inconsistent Results on HEMIT dataset: In the HEMIT paper, the re-\nported PSNR for their baseline ranges between 27–28 dB. In contrast, the values re-\nported in your paper are significantly lower across all models, with even the best-per-\nforming DGR model yielding inferior PSNR and SSIM compared to HEMIT’s method.\nThis discrepancy is concerning and warrants further explanation.\nResponse:\nThank you for raising this important point regarding the performance discrepancy on\nthe HEMIT dataset. The difference in reported PSNR values stems primarily from dif-\nferences in data preprocessing. In our work, all models were trained and evaluated on\n512×512 center-cropped regions extracted from the original 1024×1024 patches, rather\nthan using the full-size images as in the original HEMIT paper. This cropping strategy\nwas adopted both to reduce computational cost and to maintain consistency with the\nprocessing pipelines used for other datasets in our study (AF2HE and HE2PAS-AB).\nConsequently, the test set used in our experiments is not identical to that used in the\nHEMIT baseline, which accounts for the observed differences in quantitative results.\nWe have updated the Dataset section to provide a clearer and more detailed description\nof our preprocessing steps to avoid potential misunderstandings.\n“HEMIT Dataset. The dataset employed in this work is derived from the collection\nintroduced by Bian et al. [42], consisting of histologically paired H&E and multiplex\nimmunohistochemistry (mIHC) whole-slide images originating from the ImmunoAlzer\n[43]. A key advantage of this dataset is that both staining protocols were applied se-\nquentially on the sametissue section, thereby minimizing structural discrepancies and\nensuring high pixel-level alignment—a notable improvement over conventional con-\nsecutive-section approaches. The mIHC staining incorporates three biologically rele-\nvant markers: DAPI for nuclear visualization, panCK for tumor region delineation, and\nCD3 for T-cell identification, each corresponding to a distinct channel in the image\n(blue, red, and green, respectively). We adopt the official partitioning of the data into\ntraining, validation, and test sets with 3717, 630, and 945 image pairs, respectively,\nensuring that each partition contains tissue samples from different patients to prevent\ndata leakage. To reduce computational demands during model training, we extract a\ncentral region of 512×512 pixels from each original 1024×1024 patch. Additionally,\nall H&E images undergo stain normalization to enhance consistency and generaliza-\ntion across samples.”\nComment 9: (D)Blinded Human Evaluation study.\n(I) I have concerns about the study's design. It focuses solely on assessing whether hu-\nmans can distinguish between real and virtual IHC images—essentially replicating\nwhat metrics like FID and KID already quantify. However, this approach does not\nprovide any insight into whether the virtual IHC images accurately capture the underlying\nbiological staining patterns. How can we evaluate or communicate the accuracy of the\nstaining with respect to the actual cellular structures?\nResponse:\nThank you for your insightful comments. We agree that evaluating cellular-level accu-\nracy is a critical challenge in virtual staining research. While the blinded human evalu-\nation study assesses perceptual similarity—a common approach in the field, as seen in\nprior work [1]—we acknowledge that it does not directly capture biological accuracy\nat the cellular structure level.\nTo address this concern, we have incorporated an additional quantitative evaluation us-\ning Cellpose [2] to segment cells in both chemically stained and virtually stained im-\nages. This allows us to objectively compare the morphological agreement between the\nreal and virtual stained images. The results show that our method (DGR) achieves the\nhighest Dice score of 0.422, indicating better structural preservation compared to other\nmethods. We have also included a visual comparison of a sample with high cell density,\nwhich further supports the superior performance of DGR and RegGAN in preserving\ncellular details.\nThese additions have been included in the revised manuscript to provide a more com-\nprehensive assessment of the biological accuracy of the virtual staining results.\nReference:\n[1] Rivenson Y, Wang H, Wei Z, et al. Virtual histological staining of unlabelled tissue-\nautofluorescence images via deep learning[J]. Nature biomedical engineering, 2019,\n3(6): 466-477.\n[2] Pachitariu M, Rariden M, Stringer C. Cellpose-SAM: superhuman generalization\nfor cellular segmentation[J]. bioRxiv, 2025: 2025.04. 28.651001.\nComment 10: (II) Paired datasets, such as those used in [13] and HEMIT, do not face\nthis issue since they are pixel-aligned on the same tissue. However, for unpaired da-\ntasets, I believe the human evaluation approach described above is not appropriate.\nResponse:\nWe appreciate the reviewer's comment regarding dataset alignment. To clarify, all da-\ntasets used in this study are paired but contain inherent pixel-level misalignment due to\nthe nature of the tissue processing and image acquisition protocols. We have expanded\nthe dataset section to provide a more comprehensive description of the dataset to avoid\npotential misleading.\nComment 11: (E) No distribution-based metrics are reported, such as FID, KID, etc.\nResponse:\nThank you for this important suggestion. We have now incorporated the recommended\ndistribution-based metrics, FID and KID, into our evaluation. These results have also\nbeen added to the revised manuscript. The corresponding results are presented below:\nComment 12: 3. Other Writing and Dataset Details Missing\n(A) Issues in naming conventions in figures and Tables, I think shorthand for method-\nology like “PyramidPix2Pix”, “Pix2Pix”, “RegGAN” would be easily understood than\n“Liu et al, Isola et al” etc. This is more confusing and less readable.\nResponse:\nThank you for this valuable suggestion. We agree that using shorthand names for the\nmethodologies improves clarity and readability. Accordingly, we have revised the fig-\nures and tables to use the abbreviated names instead of the author-name references. The\nupdated examples are provided in the revised manuscript.\nRevised contents:\nSome Tables (All tables have been updated to the draft):\nComment 13: (B) I found Figure 1(f) unclear—it's not evident what the reader is ex-\npected to observe, and the figure appears incomplete.\nResponse:\nThank you for this helpful feedback. We agree that the original version of Figure 1(f)\nwas unclear and have revised it to better convey our intended message.\nThe goal of this panel is to demonstrate the robustness of the proposed model (DGR)\nagainst misaligned data, in comparison to several ablated variants: the Baseline,\nBaseline+RNR, and Baseline+RNR+PcG. The updated figure now more clearly illustrates\nthat the DGR model maintains higher performance under misalignment, highlighting\nthe contribution of our full architecture.\nThe revised figure is presented below:\nComment 14: (C) The dataset section lacks sufficient detail, making it unclear which\ndatasets are paired (i.e., two stains applied to the same tissue section, such as AF2HE\nand HEMIT) and which are unpaired(stains on adjacent tissue sections, I am guessing\nthe other 3 remaining datasets). This ambiguity makes it difficult to interpret the results\naccurately, as the experimental setup and evaluation depend heavily on whether the data\nare paired or unpaired.\nResponse:\nThank you for this important comment. We appreciate your feedback regarding the need\nfor greater clarity in our dataset descriptions. We very agree with your observation\nabout the significance of paired versus unpaired data for interpreting the results.\nWe have revised the Dataset section to explicitly outline the pairing methodology for\neach dataset and to include more comprehensive details on their preparation. All da-\ntasets used in this work are paired, meaning that different stains were applied sequen-\ntially to the same tissue section, thereby ensuring high structural alignment.\nThe updated descriptions are as follows:\n“AF2HE Dataset. The AF2HE dataset comprises 15 whole slide images (WSIs) of\nbreast and lung cancer tissue samples [39]. The specimens were initially imaged in\ntheir native autofluorescence (AF) state using a widefield microscopy system equipped\nwith a 265 nm ultraviolet excitation source and a 10× objective lens, enabling high-\ncontrast nuclear visualization without staining [40]. Following AF imaging, the same\nslides were subjected to standard hematoxylin and eosin (H&E) staining and rescanned\nunder a 20× objective lens using a whole-slide scanner. The image pairs were rigor-\nously aligned using the VALIS framework to ensure spatial correspondence [41]. Sub-\nsequently, the registered WSIs were partitioned into 128 × 128 patches, resulting in a\ntotal of 50,447 paired patches for training and 4,422 for testing.”\n“HE2PAS-AB Dataset. This dataset was collected from clinical specimens at the\nPrince of Wales Hospital in Hong Kong. It consists of WSIs from which a total of 10,727\naligned patch pairs (128 × 128 pixels) were extracted for training and 1,191 for test-\ning. Each tissue section underwent sequential staining: initially with H&E, followed by\nde-staining via acid alcohol treatment, and finally restaining with PAS-AB. All WSIs\nwere aligned using the VALIS framework to ensure precise spatial registration between\nstaining modalities. To rigorously assess model generalizability across different patient\npopulations within the same institution, an external validation set containing 2,841\npatches was additionally constructed. These patches were sourced from separate tissue\nsections and distinct patient cohorts that did not overlap temporally or spatially with\nthe training or test samples, thereby ensuring independent evaluation.”\n“HEMIT Dataset. The dataset employed in this work is derived from the collection\nintroduced by Bian et al. [42], consisting of histologically paired H&E and multiplex\nimmunohistochemistry (mIHC) whole-slide images originating from the ImmunoAlzer\n[43]. A key advantage of this dataset is that both staining protocols were applied se-\nquentially on the sametissue section, thereby minimizing structural discrepancies and\nensuring high pixel-level alignment—a notable improvement over conventional con-\nsecutive-section approaches. The mIHC staining incorporates three biologically rele-\nvant markers: DAPI for nuclear visualization, panCK for tumor region delineation, and\nCD3 for T-cell identification, each corresponding to a distinct channel in the image\n(blue, red, and green, respectively). We adopt the official partitioning of the data into\ntraining, validation, and test sets with 3717, 630, and 945 image pairs, respectively,\nensuring that each partition contains tissue samples from different patients to prevent\ndata leakage. To reduce computational demands during model training, we extract a\ncentral region of 512×512 pixels from each original 1024×1024 patch. Additionally,\nall H&E images undergo stain normalization to enhance consistency and generaliza-\ntion across samples.”\n“A2H Dataset. The A2H dataset was re-organized by Kang et al. [20], and the images\nwere obtained from the MITOS ATYPIA ICPR-14 challenge [44]. Original WSIs were\nstained with H&E and digitized using two distinct slide scanners (Aperio Scanscope XT\nand Hamamatsu Nanozoomer 2.0-HT). The processed images have a size of 256 × 256.\nThis dataset consists of 19,200 pairs for training and 7,936 pairs for testing. It is used\nto evaluate the performance of the proposed method in the stain normalization task.”\nComment 15: (D) No References Added for multiple claims made in the Introduction:\nLine 090 Column 1, starting with “However, the conventional ..”, Line 080 Column 2,\nstarting with “Most existing methods..”, Line 082 Column 2, starting with “Chemical\nstaining inherent alters”, Line 085 Column 2, starting with “Furthermore, a single tis-\nsue ..”\nResponse:\nThank you for this valuable comment. We sincerely appreciate your careful reading and\nconstructive suggestions. In response to your feedback, we have added relevant refer-\nences to better support the claims mentioned in the Introduction.\nThe corresponding revisions are as follows:\n“However, the conventional staining pipeline is resource-intensive, requiring signifi-\ncant time, labor, and the use of chemical reagents that contribute to environmental bur-\ndens [5]”\n“Most existing methods rely on well-aligned paired datasets for training, which are\ndifficult to obtain in practice. [9,10]”\n“Chemical staining processes inherently alter tissue structures, making it nearly im-\npossible to achieve precise alignment between unstained and stained tissue sections [11,\n12].”\n“Furthermore, a single tissue section cannot undergo multiple staining procedures\nwithout causing damage or information loss, leaving most available datasets either un-\npaired or only roughly paired [6]”\nReferences:\n[5] Latonen, L., Koivukoski, S., Khan, U., Ruusuvuori, P.: Virtual staining for histology\nby deep learning. Trends in Biotechnology (2024)\n[6] Rivenson, Y., Wang, H., Wei, Z., Haan, K., Zhang, Y., Wu, Y., G¨unaydın, H., Zuck-\nerman, J.E., Chong, T., Sisk, A.E., et al.: Virtual histological staining of unlabelled tis-\nsue-autofluorescence images via deep learning. Nature biomedical engineering 3(6),\n466–477(2019)\n[9] Ma, J., Chen, H.: Efficient supervised pretraining of swin-transformer for virtual\nstaining of microscopy images. IEEE Transactions on Medical Imaging (2023)\n[10] Lin, W., Hu, Y., Zhu, R., Wang, B., Wang, L.: Virtual staining for pathology: Chal-\nlenges, limitations and perspectives. Intelligent Oncology (2025)\n[11] Li, Y., Pillar, N., Li, J., Liu, T., Wu, D., Sun, S., Ma, G., Haan, K., Huang, L., Zhang,\nY., et al.: Virtual histological staining of unlabeled autopsy tissue. Nature Communica-\ntions 15(1), 1684 (2024)\n[12] Xu, Z., Guo, L., Tsutsui, S., Zhang, S., Kot, A.C., Wen, B.: Digital staining with\nknowledge distillation: A unified framework for unpaired and paired-but-misaligned\ndata. IEEE Transactions on Medical Imaging (2025)\nComment 16: (E) How can multi-scale generation by PyramidPix2Pix be claimed to\nmitigate effects of misalignment (line 156, column 1)? I do not think even the authors\nclaimed that.\nResponse:\nWe sincerely appreciate your insightful comment. You are correct that the original au-\nthors of Pyramid Pix2Pix did not explicitly claim that their method mitigates misalign-\nment effects. Our statement was an inference drawn from the challenges and motiva-\ntions described in their work. Specifically, they note in their paper:\n“Our BCI dataset presents a new challenge for image translation. In our dataset, the\nimages of the two domains are paired and registered at the structural level. However,\ndue to the existence of image differences between the two domains, some positions can-\nnot achieve pixel-level alignment, which makes the existing pix2pix series of algo-\nrithms difficult to work; at the same time, we need to perform targeted output for each\nHE stained image, which is not the strength of the unsupervised algorithms. Therefore,\nwe propose a pyramid pix2pix model suitable for structural aligned data. “\nWe agree that our original wording may have been too strong. We have therefore revised\nthe sentence to more accurately reflect the intended meaning and avoid overstatement.\nThe modified sentence now reads:\n“For instance, the pyramid Pix2Pix model [15] incorporates a multi-scale loss function\nto enhance robustness against structural misalignment in the data.”\nThe previous sentence:\n“For instance, the pyramid Pix2Pix model [15] enforces constraints on the predicted\nimages across multiple scales, contributing to mitigate the effects of misalignment”\nThe revision was also updated to the draft.\nReviewer #2 (Remarks on code availability):\nWhile I did not examine the GitHub repository in detail, its structure appears to be well-\norganized and user-friendly for those looking to build upon the work.\nResponse:\nWe thank the reviewer for their positive feedback on the code organization and acces-\nsibility. We are glad to hear that the GitHub repository appears well-structured and user-\nfriendly. Ensuring that our work is reproducible and easy to build upon is a key goal of\nour project, and we appreciate the reviewer’s acknowledgment.\nShould the reviewer or any researchers have further questions or suggestions regarding\nthe code implementation, we welcome them to reach out or open an issue in the repos-\nitory."}, {"round": 2, "reviewer_comments": {"reviewer_1": "I have no additional comments; the authors have addressed my concerns.", "reviewer_2": "Thank you for addressing some of my comments from the previous review.\nIn the updated manuscript, it appears that the authors have made some edits, but the main paper’s Results and Discussion\nsections remain minimally revised, which I do not consider an ideal approach to updating a manuscript. If new results, such\nas FID, KID values, or segmentation metrics, are included, they should be discussed thoroughly in the main text rather than\nbeing mentioned briefly as a formality or just added in supplementary. I had initially considered evaluating the updated\nsubmission as a new submission, but the limited scope of the edits in the main paper suggests that this is not feasible.\nProblems in Text:\n1. The text in Sections 2.1, 2.2, and 2.3 should be updated to reflect the changes made in Figures 2, 3, and 4. Currently, the\nfigures use model names such as BCI and Pix2Pix, while the text refers to them as Isola et al., which may be confusing to\nreaders. I suggest adopting a consistent format, such as Isola et al. (Pix2Pix) or Pix2Pix (Isola et al.), to clearly link the\nfigures and their corresponding descriptions in the text. Additionally, the Results section requires substantial revision, as it\nremains difficult to follow and lacks clarity in its current form.\n2. The Discussion section currently reads more like a combination of Related Work, Limitations, and Future Work, which is\nnot appropriate. A proper Discussion section should focus on interpreting the results, explaining why the findings are\nimportant, and situating them in the context of existing literature. It should address questions such as: What do these results\nreveal about the problem being studied? How do they compare with prior work? What new insights or understanding do they\nprovide? Simply listing limitations or summarizing related studies does not fulfill the purpose of a Discussion section. I\nrecommend rewriting this section to clearly emphasize the interpretation, significance, and implications of the results in\nrelation to the broader field. Needs a full rewrite.\nProblems in Figures:\n1. Statistical significance indicators are missing in Figures 1C, 1D, 1E, and Figure 6. In particular, the results in Figure 6\nrequire clear indication of statistical significance, as this figure presents a direct comparison of the proposed methodology\nwith the most closely related methods. Without these indicators, it is difficult to assess the reliability and relevance of the\nreported differences.\n2. What is the purpose of the Rotation, Translation, and Rescaling labels in Figure 1F? Since no results are presented for\ndifferent rotations, translations, or rescaling values, the meaning of this legend is unclear. I suggest clarifying or removing it\nto avoid confusion.\n3. The Intensity Correlation graphs and their interpretation may be flawed (Figures 2b, 3b, 4b). From the current plots, it\nappears that the intensity profiles of generated images may be more similar to each other than to the real images. I\nrecommend calculating these correlations explicitly, which I would expect to be reasonably high for some models, as the\nplots—particularly in Figures 2b and 3b—appear very similar. If the correlation among generated images is higher than that\nbetween real and virtual images, this does not indicate that one method is superior. Instead, it may reflect a bias in the\nmodel’s predictions toward certain intensity values, and the interpretation should be revised accordingly.\nThere are major issues in the current version of the Results and Discussion sections. I strongly recommend a substantial\nrewrite to make these sections more coherent, readable, and well-structured, with clearer interpretation and discussion of the\nfindings. I believe in its current state, the manuscript is not ready for publication.\nAdditional things missing in the paper that I missed in my previous review:\n1. The manuscript lacks a clear downstream clinical application. One of the main potential uses of a virtual staining model\nwould be to substitute a virtual IHC for a real IHC in a clinical or pseudo-clinical task, where the analysis depends on IHC\nimage availability. Without such an application, it is difficult to assess whether one model is truly better than another in a\nclinically relevant context, and it remains unclear how differences in model performance would translate to practical utility.\nAs demonstrated in [1] and [2], for different methods of MRI translation, evaluating models in a relevant downstream task is\nessential to meaningfully compare their effectiveness.\n[1] Huijben, Evi MC, Maarten L. Terpstra, Suraj Pai, Adrian Thummerer, Peter Koopmans, Manya Afonso, Maureen Van\nEijnatten et al. \"Generating synthetic computed tomography for radiotherapy: SynthRAD2023 challenge report.\" Medical\nimage analysis 97 (2024): 103276.\n[2]Zhang, Yunfei, Ruofan Sheng, Xianling Qian, Heqing Wang, Fei Wu, Haoran Dai, Mingyue Song et al. \"Deep learning\nempowered gadolinium-free contrast-enhanced abbreviated MRI for diagnosing hepatocellular carcinoma.\" JHEP Reports\n7, no. 5 (2025): 101392."}, "author_response": "Response Letter\nDear Reviewers,\nWe sincerely appreciate your thoughtful and constructive feedback, which has greatly\nhelped us improve the scientific rigor, clarity, and clinical relevance of our manuscript.\nWe have carefully addressed each of your comments and implemented substantial\nrevisions throughout the paper, particularly in the Results and Discussion sections, to\nensure a cohesive, well-structured, and interpretable presentation of our work.\nShould you require any further clarification or additional information, we would be\ndelighted to provide it. Thank you once again for your time, expertise, and valuable\nsuggestions.\nBest regards,\nHao Chen\nReviewer #1:\nComment 1: I have no additional comments; the authors have addressed my concerns.\nResponse:\nThank you for your confirmation. We are pleased to hear that our responses and revisions have\naddressed your previous concerns.\nReviewer #2:\nThank you for addressing some of my comments from the previous review.\nIn the updated manuscript, it appears that the authors have made some edits, but the\nmain paper’s Results and Discussion sections remain minimally revised, which I do not\nconsider an ideal approach to updating a manuscript. If new results, such as FID,\nKID values, or segmentation metrics, are included, they should be discussed thoroughly in\nthe main text rather than being mentioned briefly as a formality or just added in\nsupplementary. I had initially considered evaluating the updated submission as a new\nsubmission, but the limited scope of the edits in the main paper suggests that this is not\nfeasible.\nResponse:\nWe sincerely thank you for your thoughtful feedback and for highlighting the\nimportance of integrating new results meaningfully into the main narrative of the paper.\nWe acknowledge that our previous revision did not sufficiently expand the Results and\nDiscussion sections to reflect the added quantitative analyses.\nIn response to your comment, we have now thoroughly revised both sections to include\na detailed discussion of the newly reported metrics, specifically FID, KID, and\nsegmentation performance, alongside their implications for image fidelity, structural\naccuracy, and potential clinical utility. These updates are no longer confined to the\nsupplementary material but have been fully integrated into the core manuscript to\nensure a cohesive and rigorous presentation of our findings.\nWe appreciate your guidance in helping us strengthen the manuscript, and we hope the\nrevised version now meets your expectations for depth and clarity.\nThe revision record can be found at “marked_copy.pdf” file.\nHere are some revised main figures (left: previous, right: revised):\nProblem in Text:\n1. The text in Sections 2.1, 2.2, and 2.3 should be updated to reflect the changes made\nin Figures 2, 3, and 4. Currently, the figures use model names such as BCI and Pix2Pix,\nwhile the text refers to them as Isola et al., which may be confusing to readers. I\nsuggest adopting a consistent format, such as Isola et al. (Pix2Pix) or Pix2Pix (Isola\net al.), to clearly link the figures and their corresponding descriptions in the text.\nAdditionally, the Results section requires substantial revision, as it remains difficult to\nfollow and lacks clarity in its current form.\nResponse:\nThank you for your valuable feedback. We have carefully addressed both issues raised\nin your comments:\n1. Model naming consistency: We have updated the text in Sections 2.1-2.7 and\nSection 3 to be fully consistent with the model labels in all Figures. To clearly\nlink the figures to their original descriptions, we now introduce each model\nusing a combined format (e.g., Pix2Pix (Isola et al.)) at the first mention.\nSubsequently, we refer to the models by their common names (e.g., Pix2Pix) for\nclarity and brevity.\n2. Restructuring of the Results section: As suggested, we have substantially\nrevised the Results section to improve its clarity and logical flow. The section\nhas been reorganized into a more intuitive structure:\n First, an overview of performance trends across all evaluated methods.\n Next, detailed results for each of the four virtual staining tasks, presented\nwith consistent quantitative metrics (Sections 2.1 to 2.4).\n This is followed by the outcomes of a blind human evaluation conducted\nby pathologists (Section 2.5).\n We then present downstream task validation, including performance on\nclassification and segmentation benchmarks (Section 2.6).\n Finally, an ablation study that systematically examines the contribution of\nkey architectural and training components (Section 2.7).\nThis reorganization ensures that the Results section is more readable, better organized,\nand directly supports the claims made in the Discussion.\nWe believe these revisions significantly enhance the manuscript’s clarity and coherence.\nThe revision record can be found at “marked_copy.pdf” file.\nExample for Revised Section 2.2 is:\nPrevious:\n“In this task, we further evaluated the performance of various methods in converting\nH&E images into Periodic Acid-Schiff-Alcian Blue (PAS-AB) images. Similar to the\ntask of staining autofluorescence images into H&E, Zhu et al.’s [15] method\ndemonstrated the poorest performance compared to other supervised approaches.\nAmong pixel level supervised methods, our proposed method, DGR, consistently\noutperformed others across all three evaluation metrics (Fig. 3c-e, Extended Data\nTable A8) in both internal and external test sets. Specifically, DGR achieved a PSNR of\n18.677 dB on the internal test set, surpassing Kong et al. [19] by approximately 0.2 dB\n(+1.2%). On the external test set, DGR achieved a significant improvement of\napproximately 1.6 dB (+10.1%) compared to the second-best-performing model,\nKong et al. [19] (Extended Data Table A8). To further examine the performance of different\nmethods, we visualized the virtually stained images. It is evident that Zhu et al.’s [15]\nmethod struggles to accurately reproduce cellular components in the generated images,\nleading to significant discrepancies compared to real PAS-AB images (Fig. 3a).\nAdditionally, intensity of two profiles are provided (Fig. 3b). The results show that\nDGR’s intensity values align most closely with the GT, achieving the highest PCC of\n0.908 and 0.745, respectively. Based on both quantitative and qualitative analyses,\nour findings demonstrate that the proposed DGR method is highly effective for virtually\nstaining H&E images into PAS-AB images.”\nRevised:\n“We next evaluated virtual staining of Periodic Acid-Schiff-Alcian Blue (PAS-AB), a\nspecialized histological stain that highlights mucins and basement membranes. Our\nproposed method, DGR, consistently outperformed others across pixel-level metrics\nPSNR and SSIM (Fig. 3a-b, Extended Data Table A6) on both internal and external test\nsets. Specifically, on the internal test set, DGR achieved a PSNR of 18.677 dB (+1.2%)\nand an SSIM of 0.547 (+1.0%). On the external test set, DGR demonstrated more\npronounced gains, improving PSNR and SSIM by 10.1% and 8.0% over the second-best\nmethod, respectively.\nIn terms of perceptual similarity, DGR produced predictions that were more aligned\nwith the GT in semantic feature space, achieving the lowest LPIPS scores of 0.168 (-\n3.4% relative error) and 0.220 (-3.1%) on the internal and external test sets,\nrespectively (Fig. 3c). In contrast, while CycleGAN yielded favorable FID and KID\nscores (Fig. 3d-e, Extended Data Table A7), its generated images exhibited substantial\ndeviations from the GT in both structure and appearance (Fig. 3g). This discrepancy\narises because FID and KID rely on features pretrained on natural images, which differ\nsubstantially in distribution from histopathology images [24, 25]. To further assess\nstructural fidelity, we compared intensity profiles across representative image regions\n(Fig. 3f). DGR’s intensity profiles showed the closest alignment with GT, achieving the\nhighest Pearson correlation coefficients of 0.908 and 0.745. Collectively, our\nquantitative and qualitative analyses demonstrate that DGR is highly effective at\ntranslating H&E images into high-fidelity virtual PAS-AB stains.”\n2. The Discussion section currently reads more like a combination of Related Work,\nLimitations, and Future Work, which is not appropriate. A proper Discussion section\nshould focus on interpreting the results, explaining why the findings are important,\nand situating them in the context of existing literature. It should address questions\nsuch as: What do these results reveal about the problem being studied? How do they\ncompare with prior work? What new insights or understanding do they provide?\nSimply listing limitations or summarizing related studies does not fulfill the purpose\nof a Discussion section. I recommend rewriting this section to clearly emphasize\nthe interpretation, significance, and implications of the results in relation to the\nbroader field. Needs a full rewrite.\nResponse:\nThank you for your insightful comment. We fully agree that the original Discussion\nsection did not adequately emphasize the interpretation and significance of our findings.\nIn response, we have completely rewritten the Discussion to focus on interpretation of\nkey results, comparative context, and novel insights.\nThe revised Discussion avoids merely listing limitations or summarizing related\nwork. Instead, it centers on explaining why our results matter, what they advance\nbeyond the state of the art, and how they contribute to the growing body of research on\ncomputational staining in digital pathology.\nBelow is the revised Discussion section:\n“Our results demonstrate that explicitly decoupling image generation from spatial\nregistration resolves a fundamental barrier to clinical adoption of virtual staining:\nrobustness to tissue misalignment. While prior methods either require perfectly aligned\ntraining pairs [6, 13, 18, 31-33] or sacrifice structural fidelity for style transfer [15,\n34,\n35], DGR’s architecture uniquely preserves diagnostic content under real-world\ndistortion. This is evidenced not only by consistent PSNR/SSIM/LPIPS gains across five\ndatasets (Fig. 1c-e) but critically, by pathologist-level indistinguishability in blinded\nevaluation (Fig. 1g-h). These findings reveal that pixel-level alignment constraints are\nnon-negotiable for diagnostic-grade virtual staining, a principle overlooked in style-\nfocused frameworks like CycleGAN that achieved favorable FID/KID scores yet failed\nperceptually.\nIn the case of severe misalignment (Fig. 1f), the PSNR increased by 3.4 dB\ncompared to the baseline model, which quantifies a clinically critical improvement of\nthe DGR on real tissue, as consecutive tissue sections undergo inherent deformation\nduring processing. Unlike RegGAN [19], which couples registration to generation and\noptimizes only alignment between the registered output and target stain, DGR enforces\ndual constraints: (1) alignment between input and generated images to preserve tissue\nmorphology, and (2) alignment between registered output and target stain to ensure\ncolor fidelity. This architectural separation explains DGR’s advantages, validated by\nnuclei segmentation (Dice=0.422, Fig. 6k) and intensity profile correlations that\nclosely mirror chemical stains. This high-fidelity directly improved diagnostic\nperformance: only images enhanced by our DGR method increased accuracy across all\npathology tasks (+2.4% on UniToPatho, +1.2% on GCHTID). In contrast, methods like\nCycleGAN degraded performance by introducing hallucinated structures. This\nestablishes that virtual staining must prioritize anatomical accuracy over perceptual\nmetrics to retain diagnostic information.\nOur work recontextualizes misalignment from a nuisance to an intrinsic property\nof histopathology workflows. Prior approaches treated it as noise to be minimized via\nmulti-scale losses [18] or embedded registration modules [19], but DGR proves it\nrequires architectural rethinking. By decoupling generation and registration, DGR\nachieves consistent performance across alignment conditions (Fig. 1f) without\nsacrificing cellular detail. The blinded pathologist evaluation further validates that\nsuch design choices directly impact clinical trust: near-chance differentiation accuracy\n(52%) between virtual and chemical stains suggests DGR outputs contain\ndiagnostically equivalent information, a milestone unachieved by prior unpaired\nmethods [20, 21].\nWhile generalization to highly heterogeneous external cohorts warrants further\nvalidation, DGR establishes that robust virtual staining is attainable without perfectly\npaired data. This shifts the field’s focus from impractical data curation toward\nalgorithmic resilience, a necessary step toward replacing resource-intensive chemical\nworkflows. Future work will optimize computational efficiency while extending this\ndecoupling principle to diverse stain transformations.”\nProblem in Figures:\n1. Statistical significance indicators are missing in Figures 1C, 1D, 1E, and Figure 6.\nIn particular, the results in Figure 6 require clear indication of statistical significance,\nas this figure presents a direct comparison of the proposed methodology with the\nmost closely related methods. Without these indicators, it is difficult to assess the\nreliability and relevance of the reported differences.\nResponse:\nWe sincerely thank the reviewer for this important observation. We agree that clear\nstatistical significance indicators are essential for evaluating the reliability of\nperformance comparisons, especially in Figure 6. In response to this comment, we have\ncarefully revised all relevant figures to ensure that statistical significance is explicitly\nand consistently reported.\nRegarding Figures 1C, 1D, and 1E:\nAs requested, we have now included P-values on all three subfigures (1C, 1D, 1E),\ncomputed using the non-parametric Wilcoxon signed-rank test (two-tailed), as stated in\nthe figure caption. These values reflect the statistical comparison between our proposed\nDGR model and the second-best performing method across five independent pathology\ndatasets under the virtual staining task.\n• Figure 1C (PSNR) and Figure 1D (SSIM) both show statistically significant\nimprovements (P < 0.05) of DGR over the second-best method, confirming the\nrobustness of our method in terms of image fidelity and structural similarity.\n• Figure 1E (LPIPS) reports a P-value of 0.06, indicating no statistically\nsignificant difference from the second-best method. This outcome is not\nsurprising given the known limitations of LPIPS when applied to medical\nimaging tasks. LPIPS was originally trained on large-scale natural image\ndatasets (e.g., ImageNet) and is optimized to capture perceptual differences\nrelevant to everyday scenes, such as textures, object boundaries, and lighting,\nrather than the fine-grained histological structures critical in pathology. As a\nresult, it often lacks sensitivity to subtle but diagnostically meaningful\nvariations in tissue morphology, nuclear chromatin patterns, or staining intensity,\nprecisely the features our virtual staining task aims to preserve.\nWe have retained the P-value in Figure 1E to transparently report the statistical outcome,\neven when non-significant, to avoid selective reporting and to provide a complete\npicture of our method’s performance across different evaluation dimensions.\nRegarding Figure 6:\nWe fully agree with the reviewer that Figure 6 requires explicit statistical significance\nindicators. In the revised version, all bar plots in Figure 6 (g-i) now include P-values\nabove each group of bars. The revised figure is attached in following lines.\nWe believe these revisions substantially strengthen the manuscript by providing full\ntransparency and statistical rigor in performance evaluation. All figures have been\nupdated accordingly in the revised submission.\n2. What is the purpose of the Rotation, Translation, and Rescaling labels in Figure 1F?\nSince no results are presented for different rotations, translations, or rescaling\nvalues, the meaning of this legend is unclear. I suggest clarifying or removing it\nto avoid confusion.\nResponse:\nThank you for this thoughtful comment. We agree that the original labeling in Figure\n1f was ambiguous and could lead to misinterpretation.\nThe “Rotation, Translation, Rescaling” labels shown in the legend of Figure 1f were\nintended to indicate the types of spatial perturbations applied during the robustness\nevaluation, specifically, ±5% rotation, ±10% translation, and ±10% scaling, as\ndescribed in the figure caption. However, we recognize that these labels were\nmisleadingly placed as if they represented separate experimental conditions or data\nseries plotted on the ternary diagram, which is not the case. In fact, Figure 1f does not\nshow results for individual perturbation levels (e.g., 5% vs. 10%), but rather\npresents a composite robustness score for each model variant (Baseline,\nBaseline+RNR, Baseline+RNR+PcG) under the combined effect of all three types of\nperturbations simultaneously. The position of each point on the ternary plot reflects its overall\nperformance across PSNR, SSIM, and LPIPS metrics under this composite test.\nTo avoid confusion, we have revised the figure and its caption as follows:\n• Removed the misleading “Rotation, Translation, Rescaling” legend from\nFigure 1f.\n• Revised the caption to explicitly clarify that the analysis evaluates model\nrobustness under combined artificial misalignment (±5% rotation, ±10%\ntranslation, ±10% scaling), and that the ternary plot shows the aggregate\nperformance of each model variant under this condition.\nWe believe these changes significantly improve clarity and eliminate potential\nmisinterpretation. Thank you again for helping us strengthen the presentation of our\nresults.\nThe right figure is the revised version.\n3. The Intensity Correlation graphs and their interpretation may be flawed (Figures 2b,\n3b, 4b). From the current plots, it appears that the intensity profiles of generated\nimages may be more similar to each other than to the real images. I recommend\ncalculating these correlations explicitly, which I would expect to be reasonably high\nfor some models, as the plots—particularly in Figures 2b and 3b—appear very\nsimilar. If the correlation among generated images is higher than that between real\nand virtual images, this does not indicate that one method is superior. Instead, it\nmay reflect a bias in the model’s predictions toward certain intensity values, and the\ninterpretation should be revised accordingly.\nResponse:\nWe sincerely thank you for this critical insight, which has significantly strengthened\nour analysis. We agree that visual similarity among generated images could indicate a\nmodel bias rather than superior performance. As recommended, we have explicitly\ncalculated the Pearson correlation coefficients (PCC) for all pairwise combinations of\nintensity profiles (including real images) for the datasets in Figures 2b, 3b, and 4b.\nOur quantitative analysis confirms that, in some cases, generated images exhibit\nhigher mutual correlations with each other than with the corresponding ground truth\n(GT), a pattern consistent with the reviewer’s observation. This reflect in some hard\nregions, generative models may produce similar intensity distributions. However,\ncritically, our proposed model DGR achieves the highest PCC with GT across 5 out\nof 6 intensity profiles, demonstrating its superior fidelity to the true data distribution\ncompared to all other methods. In the remaining profile, the PCC between RegGAN\nand DGR is 0.96, while the PCC between RegGAN and GT is 0.91, and that between\nDGR and GT is 0.93. Although RegGAN shows slightly higher internal consistency in\nthis instance, its correlation with GT is notably lower than DGR’s. \nWe acknowledge that the reviewer’s concern about potential model bias may\napply in isolated cases; however, it accounts for only a very small portion of the overall\nresults. Across the vast majority of profiles, our method consistently aligns more closely\nwith ground truth and shows no systematic deviation, indicating that such bias is not a\ndominant factor in our model. We have added these detailed correlation results to the\nsupplementary material for full transparency. We believe this clarification strengthens\nour analysis and provides a more nuanced interpretation of intensity correlation trends.\nWe are grateful to the reviewer for prompting this important refinement\n4. There are major issues in the current version of the Results and Discussion sections.\nI strongly recommend a substantial rewrite to make these sections more coherent,\nreadable, and well-structured, with clearer interpretation and discussion of the\nfindings. I believe in its current state, the manuscript is not ready for publication.\nResponse:\nThank you for your highly constructive comments regarding the major issues in the\ncurrent Results and Discussion sections. We sincerely appreciate your direct and clear\nfeedback, as it has been instrumental in guiding a fundamental improvement to the\nmanuscript. In direct response to your recommendation for a substantial rewrite, we\nhave undertaken a comprehensive restructuring and revision of both the Results and\nDiscussion sections.\n1. Comprehensive Restructuring of the Results Section:\nWe have completely reorganized the Results section to provide a logical and reader-\nfriendly flow:\n• It now begins with a high-level overview of the key performance trends\nobserved across all evaluated methods and datasets.\n• This is followed by detailed, task-by-task results for each of the four virtual\nstaining applications (Sections 2.1-2.4). Each subsection now consistently\npresents quantitative metrics (PSNR, SSIM, LPIPS, FID, KID), key visual\ncomparisons, and a concise narrative interpreting the findings.\n• We then present the outcomes of the blind human evaluation by\npathologists (Section 2.5), linking expert assessment to the quantitative metrics.\n• Next, we report the downstream task validation on classification and\nsegmentation (Section 2.6), demonstrating the practical diagnostic utility of the\ngenerated images.\n• Finally, we include a dedicated ablation study (Section 2.7) that systematically\nevaluates the contribution of each major component of our proposed framework,\nstrengthening the causal claims about our design choices.\nThis new structure ensures the findings are presented clearly, methodically, and in a\nsequence that builds a compelling argument for the robustness and effectiveness of our\nproposed method.\n2. Complete Rewrite of the Discussion Section:\nWe have entirely rewritten the Discussion section to focus on the interpretation,\nsignificance, and implications of our results, as you rightly suggested. The revised\nsection:\n• Interprets the key results: It explains why decoupling generation from\nregistration is a critical advancement for handling real-world tissue\nmisalignment, moving beyond simply reporting that it improves scores.\n• Compares findings with prior work: It contextualizes our results against the\nstate-of-the-art, clarifying how our approach differs from and advances beyond\nmethods reliant on perfect alignment (e.g., Pix2pix) or those prioritizing\nperceptual metrics over structural fidelity (e.g., CycleGAN).\n• Articulates new insights: It proposes that our work reframes tissue\nmisalignment from a nuisance to be minimized into an intrinsic workflow\nvariable that must be addressed architecturally, a shift in perspective for the field.\n• Discusses clinical relevance: It explicitly connects the technical improvements\n(e.g., superior segmentation Dice scores, high pathologist indistinguishability)\nto potential impacts on diagnostic trust and workflow efficiency.\n• Addresses limitations and future work appropriately: These points are now\nframed as a natural conclusion stemming from the interpretation of the results,\nrather than as a separate list.\nWe are confident that these extensive revisions have transformed the manuscript. The\nResults section is now coherent and clearly structured, while the Discussion section\nrobustly interprets the findings, explains their importance, and situates them within the\nbroader research landscape. We are deeply grateful for your insistence on rigor and\nclarity, which has significantly strengthened the paper.\nThe revision record can be found at “marked_copy.pdf” file.\nAdditional Comments:\n1. The manuscript lacks a clear downstream clinical application. One of the main\npotential uses of a virtual staining model would be to substitute a virtual IHC for a real\nIHC in a clinical or pseudo-clinical task, where the analysis depends on IHC image\navailability. Without such an application, it is difficult to assess whether one model is\ntruly better than another in a clinically relevant context, and it remains unclear how\ndifferences in model performance would translate to practical utility. As demonstrated\nin [1] and [2], for different methods of MRI translation, evaluating models in a relevant\ndownstream task is essential to meaningfully compare their effectiveness.\n[1] Huijben, Evi MC, Maarten L. Terpstra, Suraj Pai, Adrian Thummerer, Peter\nKoopmans, Manya Afonso, Maureen Van Eijnatten et al. \"Generating synthetic\ncomputed tomography for radiotherapy: SynthRAD2023 challenge report.\" Medical\nimage analysis 97 (2024): 103276.\n[2]Zhang, Yunfei, Ruofan Sheng, Xianling Qian, Heqing Wang, Fei Wu, Haoran Dai,\nMingyue Song et al. \"Deep learning empowered gadolinium-free contrast-enhanced\nabbreviated MRI for diagnosing hepatocellular carcinoma.\" JHEP Reports 7, no. 5\n(2025): 101392.\nResponse:\nWe sincerely thank the reviewer for this insightful comment. We fully agree that\ndemonstrating the utility of virtual staining in a clinically relevant downstream task is\nessential for evaluating its practical value.\nWe acknowledge that the ideal clinical validation would involve substituting real IHC\nwith virtual IHC in a diagnostic workflow. However, truly pixel-level aligned H&E-\nIHC image pairs from the same tissue section are virtually unattainable in routine\npathology practice. This is because H&E and IHC are typically performed on serial\nsections (adjacent but not identical tissue planes), and clinical protocols do not involve\nstaining a slide with H&E, imaging it, then chemically decolorizing it for subsequent\nIHC, a process that is both technically challenging and rarely adopted outside\nspecialized research settings. Consequently, large-scale, perfectly aligned H&E-IHC\ndatasets are very limited, making end-to-end validation with ground-truth IHC on the\nexact same tissue location infeasible.\nGiven this inherent limitation, we designed a clinically grounded proxy task that\nreflects actual diagnostic reasoning: assessing whether virtual IHC provides\nincremental diagnostic value beyond H&E alone. In clinical practice, pathologists often\nturn to IHC precisely when H&E morphology is insufficient for a confident diagnosis.\nOur downstream evaluation mirrors this workflow by comparing diagnostic\nperformance using:\n H&E along\n H&E + virtual IHC (generated by each method).\nWe conducted this evaluation on two public datasets, GCHTID [1] and UniToPatho [2].\nWe adopted two dataset GCHTID [1] and UniToPatho[2]. As shown in the following\nfigure, our method (DGR) consistently outperforms competing virtual staining\napproaches. Most importantly, DGR + H&E achieves higher diagnostic accuracy and\nF1-score than H&E alone, for example, on UniToPatho, accuracy improves from 0.699\nto 0.723 (p < 0.05), a statistically and potentially clinically meaningful gain. This\ndemonstrates that DGR synthesizes diagnostically relevant, biomarker-like features that\ncomplement H&E morphology and are not otherwise discernible.\nWe believe this evaluation directly addresses the reviewer’s concern by linking\nmodel performance to a task that approximates real clinical decision-making under\nrealistic constraints. While not a full substitute for prospective validation with ground-\ntruth IHC on the same tissue plane (which is currently unattainable), it provides\ncompelling evidence of practical utility, particularly in settings where IHC is\nunavailable due to tissue exhaustion, cost, or logistical limitations. We have revised\nSection 2.6 to include this analysis and its clinical interpretation.\n[1] Lou, Shenghan, et al. \"A large histological images dataset of gastric cancer with\ntumour microenvironment annotation for AI.\" Scientific Data 12.1 (2025): 138.\n[2] Barbano, Carlo Alberto, et al. \"Unitopatho, a labeled histopathological dataset for\ncolorectal polyps classification and adenoma dysplasia grading.\" 2021 IEEE\nInternational Conference on Image Processing (ICIP). IEEE, 2021."}]

Decision letter