Reducing bulky medical images via shape-texture decoupled deep neural networks

Domain: neuroscience
Published: 2026-01-15
Editorial outcome: major_revision

Abstract
The explosive growth of medical data poses significant challenges for storage and sharing. Current compression techniques utilizing Implicit Neural Representations (INRs) effectively strike a balance between encoding accuracy and compression ratio, yet they suffer from slow encoding speeds. By contrast, data-driven compressors encode fast but heavily rely on the training data and cannot generalize well. To develop a practical compression tool overcoming all these limitations, we introduce Shape-Texture Decoupled Compression (DeepSTD), which focuses on the dataset of the same modality and body parts and proposes decoupling the variations into shape and texture components for separate encoding. Disentangling two components facilitates designing proper encoding strategies suitable for their respective characteristics—swift shape encoding based on INRs and effective data-driven texture encoding. The proposed approach combines the advantages of INR-based and data-driven models, to achieve high fidelity, fast encoding speed, as well as good generalizability. Comprehensive evaluations on large-scale Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) datasets demonstrate superior performance across encoding quality, compression ratio, and speed. Besides, with features like parallel acceleration with multiple Graphics Processing Units (multi-GPU), flexible control of compression ratio, and broad applicability, DeepSTD offers a robust and efficient solution for the pressing demands of modern medical data compression.

Introduction
As the resolution of medical images increases with advances in imaging technologies1–7 and the healthcare expands8–11, the data size in medical institutions grows significantly12–17. For example, the CTSpine1K dataset18 consisting of just 1,005 Computed Tomography (CT) scans requires nearly 200 GB of storage, while estimating based on CT scanners at USA19 at least 30,000 TB CT data are generated annually. Many other widely used modalities such as Magnetic Resonance Imaging (MRI), ultrasound, and X-rays, will produce much more data, creating grand storage and sharing challenges20–23. In addition, various information technologies have been applied for medical image processing, data mining, and large-scale data management, but the limited storage space and bandwidth often require downsampling or filtering of datasets24. Although high-capacity servers25,26 offer some relief, user-side storage and bandwidth constraints continue to slow down the research progress and raise costs. Therefore, efficient data encoding is crucial to reduce the volume of massive medical image data, which will advance the diverse fields related to medical images.

The explosive growth of medical data necessitates high-performance data compression techniques to accommodate the limited storage and bandwidth. Existing algorithms can be broadly categorized into three classes: (i) Conventional image and video compression techniques, such as H.26427, HEVC28, and VVC29, which are primarily intended for natural scenes, and thus struggle with medical image data, leading to either low compression ratios or noticeable artifacts due to the distinct characteristics of medical images. (ii) Data-driven compression techniques design deep neural networks30–32 and leverage supervised training to derive optimal compression specifications from extensive datasets, and have demonstrated enhanced performance compared to conventional methods, yet they cannot generalize well across various datasets and scenarios. (iii) Implicit neural representation (INR)-based techniques utilize the powerful representational capacity of deep neural networks to represent the original data as a continuous function, which pursues an optimal network to map each spatial coordinate towards its corresponding image intensity33. Among these options, the INR methods strike the best balance between data fidelity and compression ratio, but their reliance on time-consuming optimization processes limit their effectiveness in settings like online clinical diagnostics and real-time monitoring. While substantial advancements have been made, each of these methods comes with its own set of benefits and drawbacks. Few existing compressors can effectively combine fast compression with high data fidelity at sizable compression ratios.

Texture analysis has long been a fundamental tool of understanding medical images for a long time. Haralick et al. proposed gray-level co-occurrence matrix (GLCM)-based texture features, which have since been extended and widely used in medical image analysis tasks, including tumor characterization, tissue classification, and disease diagnosis34,35. In parallel, the Digital Imaging and Communications in Medicine (DICOM) standard has provided a unified format for storage and transmission of medical images, along with metadata that describes acquisition parameters and patient-specific context, which are crucial for texture interpretation in clinical contexts. These developments indicate a strong connection between texture features and DICOM image properties: textures capture local intensity variations, while DICOM preserves the overall quality of the acquisition. Building on these foundations, our shape-texture decoupled compression revisits the challenge from a representational perspective, where the “shape” component reflects global structural variations, while the “texture” component corresponds to fine-scale intensity patterns, effectively bridging classical texture analysis with contemporary neural representations.

The INR-based encoding runs slowly because the medical data encompasses large individual variations and rich details, and the encoder needs a long time to fit the complex function mapping spatial coordinates toward the intensity33,36–38. To accelerate the encoding process, we propose to focus on the dataset of the same region and modality that exists widely in clinical applications and intend to exploit the within-group similarity for fast compression. Here we draw inspiration from research on medical templates39,40, where global anatomical variability and local intensity fluctuations are modeled as separable components. This idea echoes classical image processing principles of shape-texture separation (e.g., Haralick’s texture descriptors and DICOM metadata34). In our framework, shape is represented as a smooth deformation field well-suited for implicit neural representations, while texture—after template alignment—becomes a more homogeneous residual pattern suitable for data-driven encoding. This conceptually grounded decoupling ensures that each component is efficiently compressed by a dedicated algorithm, thereby avoiding the entanglement of shape and texture. In this paradigm, the data compression process can be revisited from a template-based perspective, i.e., the data encoding and decoding are performed concerning a reference template of its located data group.

Benefiting from the shape-texture decoupling, supported by a template-based anatomical prior, one can adopt proper encoding strategies separately for two components. The template provides a spatial reference ensuring that shape encoding captures smooth anatomical deformations while texture encoding operates on anatomically aligned data, thus enhancing fidelity and compression efficiency.

Building on this analysis, we propose a compositional encoding scheme—Shape-Texture Decoupled Compression (DeepSTD). On the one hand, the shape difference resembles an elastic deformation and is fitted by an INR efficiently. On the other hand, after being geometrically aligned, the diversity among the data group is largely reduced and can be modeled well with a pre-trained network learned in a data-driven manner. The registration and decreased texture variation also mitigate the generalization issue of data-driven compressors. As illustrated in Fig. 1a, we divide the data encoding task into two distinct phases: shape encoding that models the shape differences between the data and the template with two INRs; texture encoding that encodes the texture after shape aligned with the template. The shape and texture information are concatenated to form the final compressed data. In implementation, we specially designed a Cycle INR algorithm tailored for shape encoding and a 3DTCMPE (3D Transformer-CNN Mixed Encoder with Positional Encoding) algorithm optimized for texture encoding. This decoupled approach leverages the strengths of INRs for fast low-frequency encoding38 while enabling efficient and effective data-driven compression of textures, and thus simultaneously achieves accelerated encoding, high compression efficiency, and high compression accuracy. The decoupled strategy also retains the advantage of INRs in precisely controlling the compression ratio while overcoming the limitation of data-driven methods that heavily rely on extensive training datasets. To handle the complex and diverse shapes inherent in medical image data, we further introduced a progressive encoding strategy. Moreover, for cases with limited availability of medical training data, we also developed data augmentation and knowledge distillation strategies, enhancing the robustness and effectiveness of our approach in these scenarios.

DeepSTD has been extensively evaluated on large-scale CT and MRI datasets, demonstrating superior performance across encoding quality, compression ratio, and speed. As the algorithm with the highest encoding quality, DeepSTD delivers a two-order-of-magnitude speed improvement over the best INR-based algorithm—BRIEF33, across all tested datasets and compression ratios. DeepSTD also applies to a wide range of medical images. These results establish DeepSTD as the leading solution for modern medical data compression. Beyond the exceptional performance, DeepSTD also offers practical features that enhance its utility. The proposed compressor also supports acceleration with multiple graphics processing units (multi-GPU), significantly speeding up encoding and decoding processes, and can be trained effectively even with limited training data by leveraging data augmentation and knowledge distillation strategies. Furthermore, it provides stable and flexible control over compression ratios, ensuring precise fit to the available storage or bandwidth.

The key contributions of this work can be summarized as follows:
• Proposal of shape-texture decoupled compression scheme (DeepSTD). We introduce a template for the data of the same organ(s) and modality, and decouple the given data into shape and texture concerning the template for separate encoding. The proposed scheme uses INR-based and data-driven compression algorithms respectively to compress shape and texture components and aggregate the advantages of both regimes for leading compression performance.
• Design of medical image specific algorithm. We propose a compressor that encompasses a cycle INR optimized for shape encoding and a 3DTCMPE algorithm designed for texture encoding, which is implemented progressively to handle complex and diverse variations in medical data.
• Leading performance across diverse data. DeepSTD demonstrates superior performance on various medical data in terms of a joint assessment of data fidelity, compression ratio, and speed, achieving up to a two-order-of-magnitude speed improvement over the existing best-performance compressor while maintaining the best encoding quality.
• DeepSTD’s advantageous features supporting its practical applications. DeepSTD allows multi-GPU acceleration, efficient training from limited data, stable and flexible compression ratio control, and wide applicability across diverse medical images, establishing it as a versatile and efficient solution for modern medical data compression.

Results
The principle of the proposed DeepSTD
Shape-Texture Decoupled Compression (DeepSTD) decouples the shape and texture variations in a specific dataset (the same modality and body part) and conducts compact encoding separately using methodologies suitable for their characteristics. Encoding and decoding are conducted with respect to an introduced template that describes the “average” or “typical” shape of this set, and the complete workflow of DeepSTD is shown in Fig. 1a. During encoding, for a given data, DeepSTD first extracts and compactly encodes its geometrical deviations from the template as the shape information. Then the given data is aligned with the template to remove the shape individuality and undergoes the texture encoding to retrieve the texture information. Finally, the shape and texture information are packaged into the compressed data. The decoding process is the reverse of encoding: the texture and shape information are unpacked from the encoded file, followed by texture decoding and shape decoding (morphing back to the original shape) sequentially.

The rationale behind decoupling the encoding process into shape and texture components is twofold. First, shape information predominantly can be easily represented as a continuous function, which is inherently suitable for efficient encoding using INRs and supports fast and high-performance encoding. Second, after being registered to the same template, the textures of different samples are more semantically aligned and facilitate data-driven methods to efficiently removing redundancies, achieving both high fidelity and high compression ratio. Eliminating the shape variations also help to resolve the generalization issue faced by data-driven algorithms. Together, these two principles underpin the proposed DeepSTD framework, enabling accelerated encoding, enhanced compression quality, and high robustness across dataset. In the following, we provide a detailed explanation of the key designs for shape encoding and texture encoding algorithms that specially considers the unique characteristics of medical data.

The shape encoding module targets to encode the shape differences between the given data and a predefined template. To facilitate encoding and decoding, we define a pair of reciprocal mappings—a forward deformation field that registers the given data to the predefined template and a backward field that deforms conversely. Then we designed an algorithm to pursue these two reciprocal deformation fields and restore the backward deformation field as the shape codes of the given data. We employ INRs38 to represent the deformation fields, i.e., encoding the fields with INR parameters. To ensure perfect reciprocity of the forward and backward deformations, we draw inspiration from the cyclic registration algorithm proposed in41 to simultaneously optimize both fields by introducing a cyclic loss function to encourage the deformation fields approximate inverse transformations of each other. The joint optimization of two fields outperforms the sequential in computational complexity, as shown in Supplementary Fig. 9 (a). Benefiting from its capability and suitability to describe the 3D continuous morphing, INRs provide superior registration accuracy, less parameters, and higher encoding speed to other methods, as shown in Supplementary Fig. 8. Considering that even for the same body part, the set of medical image data exhibits significant shape variations, such as varying body types of different individuals, pre- and post-surgical changes, the appearance of lesions, or slight variations in imaging regions. We introduced a gradual degradation strategy to enhance the robustness of shape encoding, by downsampling the data before shape encoding and redesigning the INR hyperparameters to focus on the geometrical deformations and disregard texture details. The contribution of the strategy is illustrated in Supplementary Fig. 9 (b), demonstrating consistent performance improvements.

The texture encoding module aims to represent the geometrically aligned medical images compactly. After being registered to the template, the data share high similarity and one can learn a dedicated encoder in a data-driven manner. Referring to the common textures of the training set as an internal “template”, this data-driven texture encoder-decoder effectively encodes the texture differences between the shape-aligned data and the template. Specifically, we propose a Transformer-CNN mixed texture encoder, which is built on the state-of-the-art (SOTA) image encoding algorithm TCM42, featuring two major improvements: (i) Extension from 2D to 3D, to fully leverage the 3D spatial structure of the medical data and thus raise the encoding quality; (2) Introduction of positional encoding, which divides the given data volume into blocks to circumvent the demanding memory and computational consumption when compressing large-scale data, and associates the position of each data block with respect to the reference shape template. These two improvements deliver moderate gains in encoding quality and significant boosts in encoding speed, as illustrated in Supplementary Fig. 11 (a). The 3DTCMPE algorithm also outperforms other SOTA methods in both encoding quality and speed, as shown in Supplementary Fig. 10.

It is worth emphasizing that shape registration plays a crucial role in the success of the 3DTCMPE algorithm, since before registration the intensity variations among the set of individual data volumes are highly complex and beyond the capabilities of both CNN- and Transformer-based deep neural networks. As experimentally shown in Supplementary Fig. 6 (b), the performance of 3DTCMPE improves significantly with shape encoding.

DeepSTD Achieves High Performance and Fast Compression
DeepSTD achieves high encoding efficiency by decoupling the encoding task into two problems that can be processed independently and efficiently. While maintaining high compression quality, DeepSTD achieves a speed improvement of two-order-of-magnitude over the benchmark compressor with the best quality, BRIEF33, and one order of magnitude over the next follow-up competitor, VVC29, across various compression ratios. These attributes make DeepSTD well-suited for modern medical data applications, which require high precision, high compression, as well as high-speed encoding. The exceptional performance of DeepSTD has been comprehensively validated in large-scale data sets in both CT and magnetic resonance modalities, demonstrating its robustness and adaptability in diverse medical imaging scenarios.

First, we evaluated the performance of our approach on highly representative CT data from the CTSpine1K dataset18, a torso CT dataset covering the upper part of the human body. The average data size is 512 × 512 × 504 pixels, and we selected the first 787 data samples in this experiment, in which 630 samples for training and 157 for testing. The dataset occupies approximately 194 GB. Considering that CT pixel values have physical significance, we evaluated the pixel-level compression loss in term of Mean Absolute Error (MAE) and Structural Similarity Index (SSIM) between the compressed data with respect to the original data. It is important to note that we do not use the Peak Signal-to-Noise Ratio (PSNR) because the large black background regions in the images can artificially inflate PSNR values. To further assess the loss from the medical perspective, we use the Dice Similarity Coefficient (DSC) and the 95% Hausdorff Distance (HD95) between the organ segmentation results of the data before and after compression.

DeepSTD achieves 256× compression at high data fidelity, which significantly reduces the difficulty of data storage and sharing. For example, downloading the entire CTSpine1K dataset (787 data samples) from a dataset website with a 500 KB/s bandwidth would typically take 5 days, and network instability may cause download failures. After applying our method, the same data can be downloaded in just 26 minutes under the same network conditions, as shown in Fig. 2a. Even at a 256 × compression ratio, our algorithm achieves almost lossless encoding performance. In terms of pixel-level data fidelity, the average MAE is 6.8 HU and SSIM is 0.993, as shown in Fig. 1b and Fig. 2b–d. From the medical perspective, we compared the organ segmentation accuracy and achieved an average DSC of 90.55 and HD95 of 4.52, as shown in Fig. 2b and e. Here the organ segmentation was performed using the UNETR++43 trained on the Synapse dataset44 for Liver, Stomach, and Spleen, and on the CTSpine1K dataset18 for the Spine. As for the encoding efficiency, DeepSTD achieves an average encoding speed of 3.1 MB/s, processing about 1,604,963 voxels per second (assuming 16-bit precision) on a workstation equipped with an AMD Ryzen Threadripper 3970X CPU and NVIDIA RTX 3090 GPUs. On average, encoding a 512 × 512 × 504-pixel volume takes 82 seconds, as shown in Fig. 2f. Encoding the 787 CT volumes totaling 194 GB takes approximately 18 hours.

We conducted a comprehensive comparison with SOTA algorithms across three dimensions: encoding quality, compression ratio, and speed. The compared methods include i) Traditional image compression algorithms— JPEG45 and JPEG200046, ii) video compression algorithms— HEVC28 and VVC29, iii) algorithms based on implicit representation— BRIEF33, iv) explicit representation-based methods— 3DGS47,48, and v) data-driven deep learning methods— DVC49 and TCM42. The results are shown in Fig. 2g–i. Note that the JPEG algorithm cannot achieve a 256× compression and the result is thus not plotted. From the comparison, one can see that DeepSTD outperforms all SOTA algorithms in encoding quality across compression ratios ranging from 32× to 256×. Moreover, as the algorithm with the highest encoding quality, DeepSTD achieves a significantly higher efficiency than the two competitors with the best quality, 37 to 139 times faster than BRIEF across compression ratios from 32× to 256×, and 18 to 46 times faster than VVC. These advancements make DeepSTD exceptionally well-suited for modern medical data applications.

Secondly, we tested our algorithm on the MRI modality data from the Amos dataset50, an MRI modality dataset that targets the abdominal as the imaging region and includes multiple organs. We started with the 1,200 MRI scans provided as supplementary data and filtered low-resolution scans with spatial resolution below 4 mm or longitudinal resolution below 20 mm. The remaining 1,144 scans were rescaled to a uniform resolution of 2 mm × 2 mm × 10 mm, and the average size of the rescaled scans is 387 × 362 × 44 pixels. We split these data into a training set containing 915 samples and a test set containing 229 samples. The total file size of the dataset is approximately 13 GB. We used the same evaluation metrics as in the above CT data, except that MAE is not used to evaluate the pixel-level fidelity because the MRI pixel values do not have physical significance.

DeepSTD achieves a compression ratio of 128 × , which significantly reduces the difficulty of data sharing. With a 500 KB/s bandwidth, downloading the entire Amos dataset (preprocessed) typically takes 8 hours and even fails if the network is unstable. After compression, the same data can be downloaded within just 4 minutes, as shown in Fig. 3(a). Even at such a high compression ratio, DeepSTD achieves an average SSIM of 0.993, as shown in Figs. 1c and 3b–d, and an average DSC of 87.8 and HD95 of 9.1, as shown in Fig. 3b and e. The organ segmentation was performed using the UNETR++43 trained on the labeled Amos dataset50 for the Liver, Stomach, and Spleen. Regarding efficiency, DeepSTD achieves an average encoding speed of 0.3 MB/s, encoding an average of 152,445 voxels per second (assuming 16-bit precision). Encoding a 387 × 362 × 44-pixel volume takes 40 seconds, as shown in Fig. 3f, and compressing the entire Amos dataset (the preprocessed totaling 13 GB) takes approximately 13 hours.

We conducted a comprehensive comparison with the same SOTA algorithms as in the previous CT compression task, the results are shown in Fig. 3g–i. One can arrive at the same conclusion that DeepSTD outperforms all SOTA algorithms in compression quality across compression ratios from 32× to 256×. At the highest encoding quality, DeepSTD is also of similar advantages in running efficiency as in CT, with 44 to 165 times faster than BRIEF at compression ratios ranging from 32× to 256×.

Besides, DeepSTD demonstrates broad applicability across datasets from other imaging regions and modalities. (i) CheXpert-small dataset51, an X-ray dataset targets chest imaging and consists of 2D data, differing from the 3D datasets used previously. To accommodate 2D data, we adapted DeepSTD’s texture encoder-decoder to a 2D version. We randomly selected 10,000 frontal-view samples from the study 1 subset of the dataset for training, with a compression ratio set to 128×, and the results are shown in Supplementary Fig. 5. (ii) Brain MRI dataset, a subset of the NYU fastMRI dataset52,53, containing 700 randomly selected T2-weighted reconstructed samples. The compression ratio was set to 128×, and the results are presented in Supplementary Fig. 6. (ii) Knee MRI dataset, another subset of the NYU fastMRI dataset52,53, containing 460 randomly selected samples, and the result at 128× compression are shown in Supplementary Fig. 7.

DeepSTD owns beneficial features supporting its practicality
One can encode and decode the texture information using multiple GPUs for acceleration. The texture encoder-decoder operates in two modes: full-data mode directly processes the entire data volume, after being geometrically aligned with the template; the block-based mode assigns positional encoding to each data block according to its spatial location and feeds the position code into TCM’s Swin Transformer module to extract the corresponding texture information. Using the block-based encoding mode, different data blocks can be encoded in parallel on multiple GPUs, accelerating the encoding process. As shown in Fig. 4a, we evaluated the encoding time on the CTSpine1K and Amos datasets using different numbers of GPUs (1, 2, 4, and 8). The results demonstrate a linear decrease in encoding time with the number of GPUs, while the shape encoding time remains nearly constant. This indicates that scalability is favorable but bounded by communication overhead and the non-parallelizable shape optimization, reflecting the key computational trade-offs of multi-GPU acceleration. It is worth noting that in this experiment, we only applied parallel processing on the texture encoding and the shape encoding time remains constant. Similarly, texture decoding can be accelerated by data division and multi-GPU processing as well. Shape decoding also benefits from multi-GPU parallelization by distributing the inference of the deformation fields in different regions on multiple GPUs. This significantly accelerates the decoding of the shape information, as shown in Fig. 4b, in which we evaluated the decoding time on the CTSpine1K and Amos datasets using varying numbers of GPUs (1, 2, 4, and 8), and observed a linear reduction in decoding time with the GPU number.

The training of DeepSTD is highly efficient, with training a DeepSTD for the CTSpine1K data taking only 25 hours using 4 RTX 3090 GPUs and that for the Amos data taking around 22 hours. Note that we can employ more GPUs to shorten the training time further. We quantitatively analyzed the time consumption of each stage during training, as shown in Fig. 4(c). The most time-intensive stage is the training of the texture encoder-decoder ("Training Encoder Decoder” in the figure), which accounts for approximately 96% of the total training time. The second most time-consuming stage is aligning the training data with the reference shape ("Aligning Shape” in the figure), contributing about 4% of the total time. Finally, the reference shape selection stage ("Selecting Reference Shape”) takes only 0.01% of the training time. Additionally, we recorded the performance changes of our algorithm when varying the training durations, as shown in Fig. 4d, to visualize the convergence.

For certain modalities, collecting training data may be more challenging than widely-used torso CT or MRI. Insufficient training data can significantly degrade the performance of data-driven methods. To simulate this scenario, we randomly selected 50 samples from the Amos training set to form a reduced training set and re-trained our algorithm on this smaller dataset. The result is shown in Fig. 4e, showing that the performance of our algorithm declined. To mitigate the impact of limited training data, we propose two strategies to enhance DeepSTD’s performance: (i) Data augmentation, in which we applied randomly generated deformation fields to the training data and created a more diverse set of training samples; (ii) Knowledge distillation, to learn a teacher model from a dataset comprised sufficient data of a similar imaging modality, and distill its knowledge into a student model designed for the target modality with scarce data. Each strategy independently improves performance, and combining them yields further gains. However, it is important to note that these improvements come at the cost of increased training time, as shown in Fig. 4e.

Efficient management of large-scale medical data calls for a compressor that makes extensive use of the available storage or transmission budget, so a flexible approach that allows users to set desired compression ratios is largely demanded. DeepSTD features a flexible control of the final compression ratio. To validate this point, we set a target compression ratio of 256× on 50 random data from Amos test set and configured all comparison algorithms through pre-experiments to achieve this target ratio. The deviations from the target compression ratios of all compression algorithms are shown in the left panel of Fig. 4f. The plot shows that DeepSTD achieves a stable compression ratio, unaffected by variations in data size or content.

This stability is jointly achieved in two stages. Firstly, the compression ratio of texture encoding is content-independent, with deviations typically less than 0.3% across CT and MRI datasets when zero-padding is needed to match block dimensions. Secondly, shape encoding compensates for deviations in texture encoding by adjusting the number of neurons in the INR network structure. Since shape encoding is implemented by solving an optimization problem, DeepSTD can flexibly adapt to given bandwidth or storage constraints by adjusting the compression ratio and controlling encoding quality, as shown in Fig. 4f right.

Methods
Encoding shape and texture
Given the data D and the reference template R, we initially pad D to match the dimensions of R and downsample them by a factor of 2. We then apply a cyclic Implicit Neural Representation (INR) registration algorithm to obtain the forward deformation field ΦF and the backward field ΦB using the downsampled D and R. The forward deformation field ΦF transforms the shape of D toward that of R, while the backward deformation field ΦB performs the inverse transformation. Finally, the forward-transformed data Dwarped is computed as

Dwarped[x] = D[ΦF(x)] (1)

with [⋅] denoting a coordinate-to-intensity indexing, and then passed to subsequent texture encoding.

After shape encoding, we use a trained encoder to extract a latent variable y as the texture information:

y = g_enc(Dwarped), (2)

where g_enc represents the encoder. The latent variable y is stored in the compressed data as texture information. At this point, the encoding process is complete, with the compressed data concatenating shape and texture information.

Decoding texture and shape
After obtaining the compressed data, the texture information y is extracted, and then a trained decoder is used to reconstruct Dwarped from the texture information.

Dwarped = g_dec(y), (3)

After texture decoding, we use the backward deformation field ΦB to restore the original shape of the data as:

D[x] = Dwarped[ΦB(x)], (4)

where D is the final decompressed data.

Selecting template for shape registration
The training sample with the widest imaging range is selected as the template to maximize detail coverage and facilitate registration. To avoid artifacts from non-principal imaging regions, we exclude samples with imaging range smaller than the template. We apply a gradual degradation strategy in the Cycle-INR encoder to focus on global geometry while down-weighting local irregularities. This makes the method robust to common MRI artifacts and outlier shapes. The selected templates for CTSpine1K and Amos datasets are visualized in Fig. 1b and c, respectively. The imaging range of each sample is evaluated based on its dimensions and imaging resolution. For the sample set with an identical imaging range, one can use a randomly selected sample as the template. In a sensitivity study on AMOS MRI (Supplementary Table 2), at a fixed compression ratio (128 × ), the widest-range exemplar yields the best overall fidelity, a random exemplar is moderately worse, and a simple average template-constructed by rigidly aligning five volumes and voxel-wise averaging-performs noticeably worse in SSIM/DSC and HD95, likely due to template blurring from unresolved inter-subject anatomical variability and the resulting increase in residual complexity56.

Cycle-INR based shape registration
We build on the cycle INR registration algorithm proposed in ref. 41 to compute both forward and backward deformation fields between the target image and the template. The forward deformation field is denoted as ΦF(x) = x + uF(x), which maps coordinates x in R to x + uF(x) in D. Similarly, we define the backward deformation field ΦB(x) = x + uB(x) to map coordinates from D back to R.

Here, the functions uF and uB provide the deformation vectors corresponding to specific coordinates, and both are parameterized by a SIREN network with an identical structure. The SIREN network is a fully connected network utilizing sinusoidal activation functions, functioning as an implicit neural representation. We simultaneously optimize ΦF(x) and ΦB(x) as shown in Supplementary Fig. 12 (a). The objective function is as follows:

L_total = L_data_F + αL_reg_F + βL_cycle_F!B + L_data_B + αL_reg_B + βL_cycle_B!F (5)

where α and β are weight coefficients. L_data_F and L_data_B respectively measure the registration accuracy of the forward and backward transformations by calculating

L_data_F = 1/bs * Σ_{i=1}^{bs} NCC(D[x_i], R[ΦF(x_i)]) (6)

L_data_B = 1/bs * Σ_{i=1}^{bs} NCC(R[x_i], D[ΦB(x_i)])

where NCC is the normalized cross-correlation, x are sampled coordinates and bs is the batch size. We adopt the Jacobian determinant regularization as the regularization term for the deformation fields

L_reg_F = 1/bs * Σ_{i=1}^{bs} |det∇ΦF(x_i)| (7)

L_reg_B = 1/bs * Σ_{i=1}^{bs} |det∇ΦB(x_i)|

The cycle loss terms L_cycle_F!B and L_cycle_B!F are designed to encourage the two deformation fields to approximate each other’s inverse transformation, and defined as

L_cycle_F!B = 1/bs * Σ_{i=1}^{bs} ||ΦB(ΦF(x_i)) - x_i||^2 (8)

L_cycle_B!F = 1/bs * Σ_{i=1}^{bs} ||ΦF(ΦB(x_i)) - x_i||^2

We adopt the squared L2 norm here because it strongly penalizes large deviations, stabilizing joint optimization and helping to maintain anatomically plausible deformations in medical image registration. Additionally, we employ SIREN37 as the architecture for the INRs.

Gradual degradation strategy for robust registration
Medical data may exhibit significant shape variations across different samples, such as differences in body types, pre- and post-surgical changes, or variations in the imaging regions. To address the shape variations and improve the robustness of shape encoding, we adopted the gradual degradation strategy. Specifically, we initially set the hyperparameter Ω of the SIREN network to its default value, e.g., Ω = 10 for the CTSpine1K dataset and Ω = 12 for the Amos dataset. If, after 100 optimization steps, the registration loss L_total falls below a predefined threshold, we reduce the value of Ω by 2 and re-encode the data. The registration process is repeated until either the loss exceeds the threshold or Ω drops below 5. The hyperparameter Ω of the SIREN network determines the spectral range of the implicit neural representation36. Lowering the value of Ω enables the registration process to focus on larger-scale alignment while ignoring smaller shape details57, thereby preventing the registration process from being overly influenced by unique shapes, which could lead to erroneous results. The threshold is determined based on the training process. Specifically, we use the 75th percentile of the registration loss values on the training data as the threshold for testing. Unless otherwise specified, the total optimization time for shape encoding is set to be the same as the encoding time for texture encoding in all experiments.

Architecture of texture encoder
The encoder architecture, as shown in Supplementary Fig. 12 (b), adopts the classical autoencoder structure, consisting of alternating feature extraction modules and down-sampling modules connected in sequence. For the feature extraction module, we follow the design of the Transformer-CNN Mixture (TCM) block from58, which utilizes parallel CNN and Transformer paths for feature extraction. At the module’s input, a 1 × 1 × 1 convolution layer is applied to increase the data dimensionality (i.e., the number of channels). The data is then split along the channel dimension into two parts: one processed by a CNN and the other by a Transformer. The CNN path consists of two sequential groups of 3 × 3 × 3 convolution layers and Leaky ReLU activation layers, with the entire structure employing residual connections. The Transformer path adopts the Swin Transformer Block structure proposed in ref. 59. The feature extraction module as a whole also utilizes residual connections. For the down-sampling module, we used the Residual Block with stride (RBS) proposed in ref. 42, which includes a strided convolution with a kernel size of 3 × 3 × 3 and a stride of 2 for down-sampling, followed by a LeakyReLU activation layer, and another 3 × 3 × 3 convolution layer. The entire module employs residual connections to expand the receptive field.

The texture encoder-decoder has two modes: full-data encoding/decoding mode and block-based encoding/decoding mode. In the full-data encoding/decoding mode, a single texture encoder-decoder is used to encode and decode the entire data, which is the default mode we adopt. However, the drawback is that it cannot utilize multi-GPU acceleration. In the block-based encoding/decoding mode, multiple texture encoders are used, each responsible for encoding and decoding a fixed data subregion. The advantage of this mode is that different encoders can parallelize the encoding/decoding process across multiple GPUs, speeding up the overall operation. This approach is also employed when the memory capacity of a single GPU is insufficient. Specifically, in the block-based encoding/decoding mode, each block is assigned a learnable positional encoding according to its relative position. This positional encoding is then input into the Swin Transformer Block of the TCM Block, enabling the encoder to recognize each block’s relative position and extract texture information accordingly. The above working modes both benefit from the intensified data similarity introduced by the shape registration.

Architecture of the texture decoder
The structure of the decoder is shown in Supplementary Fig. 12 (b), which adopts a design similar to the encoder, with the primary difference being the replacement of down-sampling modules with up-sampling modules. For the up-sampling module, we used the Residual Block with Upsampling (RBU) proposed in ref. 42, employing subpixel convolution instead of transposed convolution as upsampling units to retain more details. This module also employs residual connections to expand the receptive field.

Standard training procedure
Before training, all samples are first aligned to the template using Cycle INR optimization to ensure consistent geometry across the dataset. The encoder and decoder are then trained simultaneously, and the loss function is defined as the pixel-wise difference between the decoded data and the original version:

L = 1/N * Σ_{i=1}^{N} ||D_i - g_dec(g_enc(D_i))||^2 (9)

where N is the total number of training samples, g_enc is the encoder, and g_dec is the decoder. The Adam60 optimizer is used for training with a batch size of 2, and the learning rate is set to be 1 × 10−4.

Data augmentation and knowledge distillation for limited training data
For modalities with limited data due to image collection difficulties, we adopted two strategies to train the encoder and decoder. On the one hand, we applied data augmentation by generating random deformation fields to dynamically alter the shape and texture of the training data during training. Specifically, we adopted the method proposed in ref. 61. Random Gaussian parameters are generated to create a deformation field that is then applied to the next 10 training samples to introduce shape and texture variations. This strategy ensures diversity in the training set, helping to improve the robustness and generalization ability of the encoder-decoder.

On the other hand, we employed knowledge distillation for effective training on a small data set. First, we identified a similar modality with abundant training data. Using the method described in “Standard Training Procedure”, we trained an encoder-decoder, called the teacher model. Next, we trained an encoder-decoder for the target modality, called the student model. In addition to using the loss function defined in Equation (9), we introduced a loss function that leverages the intermediate layer features of the teacher model as supervision to guide the training of the student model:

L_total = L + αL_feature (10)

L_feature = 1/(N*M) * Σ_{j=1}^{M} Σ_{i=1}^{N} ||f_TCM_j(D_i) - f_TCM_j(D_i)||^2 (11)

where α is a weighting factor, M is the total number of TCM blocks in the encoder-decoder, f_TCM_j(D_i) represents the features extracted by the j-th TCM block of the teacher model from data D, and f_TCM_j(D_i) represents the features extracted by the j-th TCM block of the student model from the same data. For the knowledge distillation experiments (Fig. 4e), the teacher model was trained on the full Amos training set, while the student model was trained on a reduced subset consisting of 50 randomly selected samples. In Equation (10), the weighting factor for the distillation loss was set to α = 0.5. Note that when training the teacher model, if data block partitioning is required, we set the positional encoding to random noise to prevent the teacher model from learning incorrect shape information from a different modality.

Comparison methods
The baseline algorithms adopted for performance comparison and their detailed implementations are detailed here.
As for medical image registration algorithms, we chose six algorithms.
– Affine and B-spline: We used Elastix software62 for their implementation.
– SyN: We adopted the implementation provided by ANTs63.
– CorrField, VoxelMorph, CycleMorph, and INR: We used the official implementations of CorrField64, VoxelMorph65, CycleMorph66, and INR38.

To validate our compression performance, we compared both conventional compression tools and recent deep-learning-based ones.
– JPEG: We used the OpenCV implementation of JPEG45. To ensure compatibility, we normalize the data to the range [0, 255] before encoding.
– JPEG2000: We utilized the Glymur implementation of JPEG200046. Similar to JPEG, we normalize the data to the range [0, 255] before encoding.
– HEVC: We employed the FFmpeg implementation of HEVC28. we identified that using the gray16 encoding format produced better results. Therefore, we normalize the data to the range [0, 65535] before encoding.
– VVC: We used the VTM implementation of VVC29. Since the 16-bit format results in excessive computational overhead, making it impractical for experiments, we encode the data using the 8-bit format instead.
– BRIEF, 3DGS, DVC, and TCM: We adopted the official implementations of BRIEF33, 3DGS47, DVC49, and TCM42.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "In the article author presents a methodology to reduce the bulky medical images (DICOM) through integration of shape and texture using deep neural network models. I have following observations and suggestions to upgrade the article further.\nTechnical\n1. In the introduction of the article, I suggest providing some background details about the texture features, DICOM images properties, and their possible linkups. Texture was exhaustively analyzed by Haralick, and later texture feature were analyzed for several application including medical data analysis. So, provide a linkup between these methodologies.\n2. In the introduction again I wish to know how does DeepSTD reconcile the trade-off between encoding speed and generalizability, this is basically a limitation in both INR-based and data-driven methods, how you plan to overcome this challenge?\n3. What theoretical or empirical background supports the decoupling of shape and texture in medical imaging data for compression purposes, these are several lossy and lossless compression algorithms so which compression algorithms provide the optimum results?\n4. In what ways does the use of Implicit Neural Representations (INRs) for shape encoding improves the speed and fidelity in DeepSTD?\n5. How the overfitting issues commonly seen in data-driven compressors are overcome by the DeepSTD's?\n6. While disentangling shape and texture components, a lot of challenges arise how does DeepSTD address them?\n7. What computational trade-offs which are involved in implementing multi-GPU acceleration in DeepSTD, and how do they influence scalability?\n8. How does DeepSTD maintain a balance between compression ratio and image quality, because both are inversely proportional to each other. Usually, we go for the optimum output but if this is possible this is certainly useful for the medical applications.\n9. What specific role do spatial or anatomical priors play in the success of the shape-texture decoupling mechanism in DeepSTD?\n10. Further I wish to understand in the case of non-medical and daily images how do the DeepSTD framework adapt and perform, do it is possible to observe any sought of performance issue if yes what architectural changes would be necessary?\nMinor\nIf you can increase the font size (x- & y- labels) of images like Fig 3 (g,h,i), Fig 4 (f), that will be really good for readers of all ages. Try to keep text size and Figure labels of same size.", "reviewer_2": "Summary\nThis paper introduces DeepSTD, a deep learning-based method for compressing medical images by decoupling them into shape and texture components relative to a predefined template. Shape information, representing geometric deviations from the template, is encoded using a Cycle Implicit Neural Representation (INR) approach. Texture information, captured after aligning the image to the template, is compressed using a data-driven 3D Transformer-CNN Mixed encoder with Positional Encoding (3DTCMPE). The method aims to combine the high fidelity of INRs with the speed of data-driven approaches. Experiments on large-scale CT and MRI datasets, along with X-ray and other MRI datasets, demonstrate superior performance compared to state-of-the-art methods in terms of compression quality and encoding speed. The paper also highlights practical features like multi-GPU acceleration, handling limited training data via augmentation and distillation, and flexible compression ratio control.\nMajor Concerns\n1.Template Selection and Visualization: The template is central to the method, yet its selection process and characteristics are insufficiently detailed and justified.\n- The \"widest imaging range\" criterion needs more rationale. For example, MRI sometimes high noise and suffer from the uniformity problem. From your description, the image which has a severe artifact could be selected as the template. Its robustness to images with significant artifacts (common in MRI) or anatomical outlier shapes is not discussed. How are potential negative impacts on registration accuracy for the rest of the dataset mitigated?\n- Visualizations of the actual templates selected for the main datasets (CTSpine1K, Amos) should be provided in the main text or supplementary materials.\n2.Clarity and Definitions:\n- The acronym \"3DTCMPE\" should be explicitly defined upon first use (presumably 3D Transformer-CNN Mixed encoder with Positional Encoding).\n- The training process needs slight clarification. Confirm explicitly that shape alignment (Cycle INR optimization) is performed for training samples before/during the 3DTCMPE training phase.\n3.Handling Limited Data and Distillation Details:\n- The knowledge distillation setup requires more detail for reproducibility. Explicitly state the teacher/student datasets used for the experiment in Fig 4e (presumably full Amos vs. limited Amos).\n- The value of the loss weighting factor α (Eq. 10) used in the distillation experiments should be provided.\n4. Scope and Limitations (OOD/Functional Data):\n- The paper should explicitly discuss the limitations regarding Out-of-Distribution (OOD) data. While applicability across different retrained datasets is shown, the performance on unseen variations (pathologies, scanners, protocols) within a dataset is not explored.\n- A critical limitation regarding applicability to functional imaging (fMRI, PET, etc.) should be discussed. The reliance on anatomical registration for shape-texture decoupling makes the current approach unsuitable for modalities where intensity lacks consistent anatomical correspondence.\nThe paper presents a promising and high-performing approach for medical image compression. The core idea of shape-texture decoupling leveraging both INR and data-driven methods is sound and yields impressive results. However, revisions are needed to improve clarity, particularly regarding the crucial template selection process, provide necessary details for reproducibility, and explicitly discuss the method's scope and limitations, especially concerning OOD and functional data. Addressing these points will significantly strengthen the manuscript for publication.", "reviewer_3": "The authors present DeepSTD, a novel approach to medical image compression that decouples shape and texture components for separate encoding. This method aims to overcome limitations of existing compression techniques by combining the advantages of Implicit Neural Representations (INRs) and data-driven methods. The manuscript demonstrates impressive performance across various metrics (compression ratio, encoding quality, and speed) on CT and MRI datasets, achieving up to two orders of magnitude faster encoding than competing methods while maintaining superior quality. While the concept is innovative and the results are compelling, several areas require clarification and improvement before publication.\n## Major comments:\n1)The authors mention that this approach was inspired by research on medical templates (lines 44-56), but they don't go into much detail about the theoretical basics. How exactly does the suggested decoupling connect to the well-established principles of shape-texture separation in image processing? And what ensures that this method actually achieves the best possible separation between shape and texture components?\n2) The authors mention (lines 490-495) that the widest imaging range sample is selected as the template, but do not sufficiently investigate how template selection impacts performance. How sensitive is the compression performance to the choice of template? Would an \"average\" template constructed from multiple samples yield better results than a single exemplar?\n3) The authors present DeepSTD as a hybrid approach that integrates both autoencoder-based compression and INR for shape encoding. While lines 56-63 briefly suggest this combination aims to leverage the strengths of both paradigms, the manuscript lacks an explanation of why this particular fusion is optimal. A more thorough analysis of how the shape-texture decoupling specifically enables this synergy would strengthen the contribution.\n4) The manuscript shows significantly better compression performance on CT data (a compression ratio of 256× with an MAE of 6.8 HU and SSIM of 0.993) compared to MRI data (128× compression ratio with an SSIM of 0.993). However, the authors don't offer a deep dive into why this big difference exists. This gap raises some important questions about the method's basic assumptions and limitations. I suggest the authors look into whether differences in how images are acquired, signal-to-noise ratios, or natural tissue contrasts impact how well the shape and texture decoupling works. Also, the manuscript should spell out more clearly the specific conditions and limitations that define where DeepSTD works best.\n5) While the authors present shape and texture encoding as separate components, there is limited discussion of potential interdependencies between these aspects. How do errors or limitations in shape encoding propagate to affect texture encoding performance? Conversely, how sensitive is the overall system to variations in texture encoding quality when shape registration remains constant?\n## Minor comments:\nPlease provide some missing implementation details\n1)Line 249: When mentioning \"encoding an average of 1,604,963 voxels per second,\" the authors should clarify whether this is measured on a specific hardware configuration to enable fair comparisons.\n2)Line 383: \"deviating only when padding is necessary\" lacks clarity – please specify the typical magnitude of this deviation when padding occurs.\n3)Could the authors elaborate on how their shape-texture decoupling approach specifically preserves high-frequency information?\n4)The registration approach seems promising, but perhaps the authors could discuss how they ensure the shape encoding represents meaningful anatomical variations rather than simply finding deformations that minimize the registration loss function?\n5)Could the authors clarify why the L2 norm was chosen for the cycle consistency constraint in equation 8? Different distance metrics might affect the registration performance, particularly for preserving fine anatomical details.\n6)It would be valuable if the authors could discuss the error bounds when encountering anatomical variations or pathological conditions not well-represented in the training data.\nThere are several typos, e.g., Line 202: \"neep neural net-works\"->\"deep neural networks\"， \"cycle INR algorithm\"->\"Cycle INR algorithm\""}, "author_response": "We sincerely appreciate the insightful comments and suggestions from the editor and reviewers on our manuscript (NCOMMS-25-17687A). Their feedback has significantly enhanced the quality of our work. We have carefully revised the manuscript in accordance with their recommendations, and our detailed responses are provided below. For clarity, in this letter the reviewers' comments are marked in blue, the point-to-point responses in black, and the revised paragraphs of the manuscript reproduced here in gray-shaded text. Reviewer #1’s Remarks and Responses... [Full response text omitted for brevity, but included in the parsing logic]"}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have presented good work in the article. It is rewarding to see how the article has emerged after revision. All my comments have been implemented well. Good Luck", "reviewer_2": "Dear authors,\nThank you for the careful and thorough revision and for the detailed response to the review. The revised manuscript substantially improves clarity and addresses core requests. We appreciate the thoughtful updates.\nTo finalize the paper, please address the following minor points:\n- Remove duplicated sentence (line 453, p. 8).\nThe following sentence appears twice—please delete the duplicate occurrence:\n“For shape encoding, we utilize a Cycle INR to address the challenges associated with encoding significant shape differences relative to a predefined template, by solving forward and backward registration problems simultaneously.”\n- Code availability statement.\nWe believe in your commitment to releasing the full, cleaned code upon acceptance. Please ensure that the final version includes an active repository link containing the complete pipeline and scripts necessary to reproduce all benchmarks.\nWith these minor edits, I will be pleased to recommend acceptance. Thank you again for your careful work and responsiveness.", "reviewer_3": "The authors have adequately addressed all my concerns, and the revised manuscript has been significantly improved. I recommend accepting the manuscript."}, "author_response": "We sincerely appreciate the insightful comments and suggestions from the editor and reviewers on our manuscript (NCOMMS-25-17687A). Their feedback has significantly enhanced the quality of our work. We have carefully revised the manuscript in accordance with their recommendations, and our detailed responses are provided below. For clarity, in this letter the reviewers' comments are marked in blue, the point-to-point responses in black. Reviewer #1’s Remarks and Responses... [Full response text omitted for brevity, but included in the parsing logic]"}]

Decision letter