Computer Methods and Programs in Biomedicine 245 (2024) 108009

Contents lists available at ScienceDirect

Computer Methods and Programs in Biomedicine
journal homepage: www.elsevier.com/locate/cmpb

A convolutional neural network-based method for the generation ofsuper-resolution 3D models from clinical CT images
Yijun Zhou a, Eva Klintström b, c, Benjamin Klintström d, Stephen J. Ferguson e,
Benedikt Helgason e, Cecilia Persson a, *aDivision of Biomedical Engineering, Department of Materials Science and Engineering, Ångströmlaboratoriet, Uppsala University, Lägerhyddsvägen 1, Uppsala 75237,
SwedenCenter for Medical Image Science and Visualization (CMIV), Linköping University, Swedenc
Department of Radiology and Department of Health, Medicine and Caring Sciences, Linköping University, SwedendDepartment of Biomedical Engineering and Health Systems, KTH Royal Institute of Technology, Huddinge, SwedeneInstitute for Biomechanics, ETH Zürich, Zürich, Switzerlandb

A R T I C L E I N F O

A B S T R A C T

Keywords:
Neural network based super-resolution
Clinical CT datasets
Morphological and mechanical validation

Background and objective: The accurate evaluation of bone mechanical properties is essential for predicting
fracture risk based on clinical computed tomography (CT) images. However, blurring and noise in clinical CT
images can compromise the accuracy of these predictions, leading to incorrect diagnoses. Although previous
studies have explored enhancing trabecular bone CT images to super-resolution (SR), none of these studies haveexamined the possibility of using clinical CT images from different instruments, typically of lower resolution, as abasis for analysis. Additionally, previous studies rely on 2D SR images, which may not be sufficient for accuratemechanical property evaluation, due to the complex nature of the 3D trabecular bone structures. The objective of
this study was to address these limitations.
Methods: A workflow was developed that utilizes convolutional neural networks to generate SR 3D models acrossdifferent clinical CT instruments. The morphological and finite-element-derived mechanical properties of these
SR models were compared with ground truth models obtained from micro-CT scans.
Results: A significant improvement in analysis accuracy was demonstrated, where the new SR models increased
the accuracy by up to 700 % compared with the low-resolution data, i.e. clinical CT images. Additionally, we
found that the mixture of different CT image datasets may improve the SR model performance.
Conclusions: SR images, generated by convolutional neural networks, outperformed clinical CT images in thedetermination of morphological and mechanical properties. The developed workflow could be implemented for
fracture risk prediction, potentially leading to improved diagnoses and subsequent clinical decision making.

1. Introduction
The increased rate of bone fragility fractures in the elderly popula­tion has become an increasing concern with our aging societies [1,2].
This demographic group has a much higher incidence of fractures than
younger individuals, largely due to age-dependent decreases in bonemass and deterioration of bone structure. Current clinical guidelines rely
on areal bone mineral density (aBMD) estimations derived from Dual
Energy X-ray scans (DXA) as a surrogate measure for fracture risk.
However, the sensitivity of aBMD in terms of classifying fracture risk
ranges from low to moderate [1,3,4]. Two of the main drawbacks ofusing DXA scanners for assessing fracture risk are the low resolution of
the image data and the lack of a 3D representation of the skeleton. The
low resolution does not allow visualization of the bone architecture,
which has been shown to explain a large portion of the measured vari­ance in ultimate strength of bone specimens tested ex vivo [2].
Micro-CT instruments have been utilized to scan small bone speci­
mens at a resolution down to 1 μm [5]. While these images provide an
accurate representation of the trabecular structure, the use of micro-CT
scanning on living human beings is not feasible due to the high radiation
dose. Advancements in CT imaging in recent years has enabled the use ofhigher resolution scans (HR-CT) for limited regions of peripheral bones* Corresponding author.
E-mail address: Cecilia.Persson@angstrom.uu.se (C. Persson).
https://doi.org/10.1016/j.cmpb.2024.108009
Received 14 September 2023; Received in revised form 1 November 2023; Accepted 5 January 2024
Available online 6 January 2024
0169-2607/© 2024 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009[6]. These techniques generate CT images with a voxel size of 60–125μm [6–9] albeit at a modest signal-to-noise ratio. This is close to thethickness of an individual trabecula radius, being about (or slightly lessthan) 100 μm [10,11]. However, the HR-CT models may still result inmorphological bias compared to ground truth (GT) trabecular bone
structures [9,12,13]. These findings suggest that the current resolution
is not sufficient to accurately distinguish the trabecular bone structurefrom the background, and the presence of noise induces further bias.
Additionally, a long scanning time increases the likelihood of motion
artifacts occurring.
To fully utilize the HR-CT data, there is a need to develop compu­
tational postprocessing methods to optimize the image quality, which
would allow for accurate quantification of bone architecture. Neuralnetworks (NN) have developed rapidly in the last few decades, espe­
cially in the biomedical engineering field [14,15]. The use of NN-based
SR algorithms for enhancing image quality was reported in recent years[16–23]. These algorithms have been utilized to refine optical micro­
scopy [19] and CT-images [16,17,21–23], generating higher-resolution(HR) images from relatively lower-resolution (LR) data. Recent SR
NN-based studies have used the technique on CT images of trabecularbone [16,22,23]. However, most of them have targeted quality
enhancement in 2D, making it difficult to transform the results into 3D
morphological parameters or derived mechanical properties. A recently
published study worked on 3D SR enhancement, however the LR datautilized were based on down-sampled micro-CT images [21], which does
not necessarily reflect the level of quality present in image data acquired
directly by clinical CT scanners.
This study presents a workflow that utilizes the enhanced deepsuper-resolution network (EDSR) [20], to improve the resolution of LR(HR-CT) images of trabecular bone structures, imaged with different
scanners. The GT image data that was utilized for model training andvalidation was acquired using a micro-CT scanner. Mechanical proper­
ties were predicted by finite element analysis. Morphological and me­
chanical properties of 3D SR models were evaluated and compared to
the results of corresponding GT models. We hypothesized that NN-basedalgorithms can effectively generate three-dimensional SR trabecularbone structures from clinical CT data, exhibiting a high degree ofagreement with the GT models for morphological and derived me­
chanical properties.

Healthineers, Erlangen, Germany), and Verity (Planmed, Helsinki,
Finland). The resolution of the different HR-CT scanners can be found in
Table 1. The clinical HR-CT images represent the LR datasets in thisstudy and the micro-CT images were utilized as the GT datasets. The
registration process of the clinical CT data to the micro-CT references
was conducted manually using a two-step approach in MeVisLab (MeVis
Medical Solutions AG, Bremen, Germany) [12].
2.2. Neural network
The EDSR model was adapted from a previous study on convolu­
tional neural network (CNN) for SR [20], where the work was focused on
SR tasks on RGB images with a large diversity of content, e.g. animals,buildings, etc., with no application to medical imaging SR tasks. The
code used was adapted from the torchsr (v1.0.3) Python library (3.7.4)[24], which was built on PyTorch [25]. The code was modified to suitsingle layer CT images, where only the grayscale (luminance) of theimages was used.
The performance of the EDSR model was evaluated by identifying
the optimum parameters and hyperparameters of the EDSR model for
the Accuitomo dataset, i.e. the upscaling factor (×2, ×4), number ofresidual block layers (r), number of feature channels (f), patch size,
initial learning rate, learning rate decay steps, loss function, and the
optimizer [25]. The number of residual block layers and feature chan­nels influence the model size, i.e. the number of parameters included in
the model. A larger patch size may provide more information for the
training, however, at the cost of training time. Initial learning rate,
learning rate decay steps, and optimizer influenced the optimization
path. The loss function was used to evaluate the SR images’ differences
to the GT images. For the validation, the parameters’ peak signal-tonoise ratio (PSNR) and structural similarity (SSIM) were used to eval­uate image quality [26,27]. The r16f128×4 model showed the highest
performance with a low training time and was selected for further
training with all datasets. The hyperparameters utilized in this casewere: initial learning rate of 1E-4, learning rate decay step of 1000, l1
loss, and the use of the Adam optimizer [28].
The performance of the EDSR models in this study was compared tostate-of-the-art SR algorithms, that demonstrated good performance inthe recent NTIRE 2023 challenge on efficient super resolution [29], thespatially-adaptive feature modulation (SAFM) [30] algorithm and themulti-level dispersion residual network (MDRN) [31] algorithm. The
evaluation involved comparing the SR images generated by the EDSR
models and these algorithms, using metrics such as the PSNR and SSIM.

2. Methods
2.1. Data acquisition and image registration
Clinical HR-CT images of cubic bone samples (12–15 mm side
length) were obtained in previous studies from the trabecularcompartment of 14 human donor radii, using different clinical HR-CT
instruments (Table 1) [9,12,13]. Scanners used were: SkyScan 1176
(Bruker micro-CT, Kontich, Belgium), NewTom 5 G (QR Verona, Verona,
Italy), 3D Accuitomo 80 (J. Morita MFg., Kyoto, Japan), Multitom Rax(Siemens Healthineers GmbH, Forchheim, Germany), XtremeCT (Scanco
Medical AG, Brüttisellen, Switzerland), Somatom Force (Siemens

2.3. Neural network training
Following the image acquisition and registration, the GT images
were paired to the corresponding LR images. Since the GT images were
not always divisible by the upscale factor, they were resized to ensuredivisibility using bicubic interpolation to obtain a LR representation of
the GT images. Given that the GT images had a voxel size of 8.67 μm,
where neighbouring layer images were often very similar, we selected
every 4th (×2) or every 2nd (×4) layer for the LR and GT image pairs to
avoid overfitting and to reduce training time. This approach ensured
that the number of training images was the same for both the ×2 and ×4
models. Furthermore, the CT images from different datasets were mixed
with every 10th image selected, and the new mixed dataset was used fortraining as an additional model. A schematic representation of theworkflow is illustrated in Fig. 1.
LR and GT image pairs were available for 14 bone samples. For themixed dataset, the number of LR GT image pairs was 3006, and thenumber of validation pairs was 156. All training images were augmented
by random rotation or flipping to mitigate overfitting caused by image
similarity. The ESDR SR models were trained on 32 threads on a com­puter cluster (Euler, ETH-Zurich), with each model training taking 2–5
days to complete. The SAFM and MDRN models were trained on an AMD

Table 1
Scanning parameters for the micro-CT scanner (GT) and the 6 different clinicalscanners used in this study.
Instrument

Voxel size
(μm)

Voltage(kV)

Current(mA)

Imaging time(s)

Skyscan(GT)
NewTom
Accuitomo
Multitom
XtremeCT
Somatom
Verity

8.67

65

0.385

>3600

75
80
80
82
100
125

110
85
80
60
120
90

4.2–4.6
5
128
0.9
210
12

36
17
12
336
1
6

2

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

Fig. 1. 3D super-resolution workflow and the architecture of the EDSR model [20]. LR CT images were used as input to the EDSR model which outputs SR images.
Initially, the LR images underwent processing via a convolutional layer, subsequently followed by several residual blocks. These blocks were assembled utilizing
convolutional layers paired with rectified linear unit (ReLU) activation layers [20]. Ultimately, the up-sampling module was trained at the terminal segment of the
network, employing varying upscale factors. The generated SR images were compared to ground truth images, and the EDSR internal parameters (e.g. weights andbiases) were adjusted to minimize differences. Finally, the EDSR model was used to generate SR images from LR images of the test datasets, and the SR images
subsequently stacked in 3D for morphological evaluation and FE model simulations.

RX 6900 XT GPU, with each model training taking approximately one
day to complete.

Table 2
Material properties for bone tissue used in the FE simulations [38].

2.4. 3D morphological analysis
To construct 3D models and evaluate their morphological and me­
chanical properties, the 2D images were stacked in the third dimension,i.e. missing slices for the ×4 model were generated with bicubic inter­
polation, with three images interpolated between two SR images.
To compare the microstructure of LR, GT, and SR models, the 3Dmodels from the different groups were segmented. As the segmentation
methods may influence the morphology and mechanical properties [32],different algorithms were implemented and compared. The LR models
were segmented with the Otsu and max entropy method [33,34], while
the GT model was segmented with the Otsu method only. For the SR
models, both Otsu and adaptive methods were used for segmentation forcomparison [33,35,36]. After segmentation, unconnected small volumes
were identified with component labelling and deleted, as they could
cause numerical instability in the downstream FE simulations. The
segmentation was implemented using ImageJ, MATLAB (R2022b,
MathWorks, Natick, USA), and CTAn (1.17.7.2, Bruker, Kontich,
Belgium). Bone volume fraction (BVF), bone to surface volume ratio(BS/BV) and trabecular thickness (Tb.Th) were quantified with CTAn.

Property

Compression

TensionModulus of elasticity (GPa)Poisson’s ratio (-)
Yield stress (MPa)Ultimate stress (MPa)
Deletion stress (MPa)Ultimate strain (-)
Deletion strain (-)

16.45
0.3
159.10175.018.75
0.03040.230416.45
0.3
111.37122.506.13
0.03040.23042.5. Finite element analysis
The segmented 3D models were meshed using a direct voxel-to-meshconversion for a commercial FE solver (ABAQUS 2021, Dassault
Systèmes, Vélizy-Villacoublay, France) [37]. A built-in concrete
damaged plasticity (CDP) material model was used for bone tissue.
Modulus of elasticity was set to 16.45 GPa, which represents dense bone
at 1.8 g/cm3 [38], and a Poisson’s ratio of 0.3. Other important CDP
material properties are summarized in Table 2.
Uniaxial compression was simulated in three orthogonal directions(i.e. x-, y-, and z-direction as shown in Fig. 2). A displacement of 0.5 mm
was applied on one surface of the bone samples, with displacement
constrained in all directions on the opposite surface. The forcedisplacement curves were transformed to stress-strain curves by

Fig. 2. A schematic illustration of the coordinate system and boundary con­ditions applied in the FE simulations (loading in the z-direction case).

3

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

bone sample 4 showed the highest (16.76 dB). The SR image of bone
sample 3 trained on multi-datasets exhibited the best performance withthe highest PSNR value (18.02 dB), while the bone sample 1 showed theworst (PSNR 15.11 dB).

considering the cross-section area and height of the samples. The
apparent modulus of elasticity was calculated using linear regression of
the most linear part, i.e. between 30 % and 80 % of the ultimate stress.
Ultimate stress was calculated as the maximum reaction force in the
direction of the applied displacement divided by the nominal crosssectional area of the specimen. The FE comparison was only imple­
mented for GT, LR Otsu, and SR mix adaptive models, since the best
morphological performances were found in these groups.

3.2. 3D morphological analysis
Several segmented 3D structures are illustrated in Fig. 6. The LR max
entropy method could generate 3D structures with low BVF error (e.g.
bone sample 1). However, it also resulted in structures with low con­nectivity and missing trabeculae (e.g. bone sample 2–4). In contrast, theLR Otsu models tend to preserve the whole trabecular structure but
generate denser structures compared to the GT. SR adaptive models
produced lower (improved) BVF values than the LR Otsu models, while
preserving the structural geometry. The SR Otsu model had a similar
performance to the adaptive model, but lost connectivity when the BVF
was low.
The performance of the SR models was also compared between
different CT datasets. As shown in Fig. 7, when the quality of the original
LR CT images was low (e.g. group Verity, Fig. 4), the generated 3Dmodels (e.g. group Verity, Fig. 7) performed worse than 3D models
based on higher quality LR scans. Mixing datasets from different scan­ners was found to improve the SR model results when applied to low
quality CT images compared to when learning only from its own CT
dataset, e.g. group NewTom and Verity, Fig. 7.
As shown in Fig. 8a, the BVF closest to the GT was the SR mixed
dataset with adaptive segmentation (17±23 %). In contrast, the worst
one was the LR Otsu segmentation (200±86 %). For the BS/BV the SR
mixed dataset Otsu model (− 9±17 %) showed the best outcome, while
the LR models performed the worst (− 75±9 % and − 83±6 %). The Tb.
Th showed similar results as BS/BV, i.e. SR mixed dataset Otsu model
performed the best (21±38 %), while the LR models performed theworst (732±1718 % and 630±358 %). Overall, the SR mix adaptivemodel provided the lowest errors.

3. Results
3.1. Neural network training
The parametric studies revealed that models trained with an upscale
factor 2 (×2) did not outperform those with an upscale factor 4 (×4), as
shown in Fig. 3. Furthermore, we found that larger models (i.e. more NN
model parameters) did not consistently yield better results compared tothe simpler models. Similarly, the performance of SAFM×4 (with a
PSNR of 15.95 and SSIM of 0.6471) and MDRN×4 (with a PSNR 15.91
and SSIM of 0.6628) models did not exhibit a notable enhancement in
performance when compared to the EDSR model (r16f128×4 with a
PSNR of 15.99 and SSIM of 0.6509). As a result, we selected the EDSR
model with 16 block layers (r), 128 feature channels (f), and an upscale
factor of 4 (×4) for subsequent training, taking into account both per­formance and training time.
Fig. 4 illustrates the performance of LR, SR, and an SR model trainedwith mixed data (from different CT datasets). Among the LR groups, theSomatom gave the lowest PSNR value (12.57 dB), and the Accuitomo
showed the highest (14.56 dB). The SR image trained on the Accuitomo
dataset exhibited the best performance with the highest SSIM and PSNR
values (PSNR 15.98 dB, SSIM 0.6577), while the Verity group showed
the worst performance with the lowest PSNR and SSIM values (PSNR
12.82 dB, SSIM 0.6111). Fig. 5 illustrates the performance of LR, SR, andSR models of different trabecular bone samples. Among the LR groups,
the bone sample 1 provided a lowest PSNR value (14.56 dB), and theFig. 3. Super-resolution models (EDSR, SAFM, MDRN) results for different parameters, as applied to dataset Accuitomo. The SR images generated by SAFM×4 andMDRN×4 models were highlighted by the orange box. The EDSR r16f128×4 model was utilized for further training (highlighted by the green dashed rectangle).
4

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

Fig. 4. Super-resolution model (EDSR r16f128×4) results using different training sets. The SR models were trained individuals from different clinical CT instrument
data (LR). In contrast, the SR mix model was trained based on a mixture of all the images from the different clinical CT instruments.

3.3. Finite element model prediction of mechanical properties

clinical CT images to acquire an accurate representation of trabecular
structure using a CNN. The SR models were validated against groundtruth models of morphology and FE model derived mechanical proper­
ties, demonstrating that CNNs can improve prediction accuracy. The
results further demonstrate that augmenting training data with CT im­
ages from different instruments may enhance the SR performance,
particularly for the low-quality images.
A previous study suggested that the EDSR×2 model [20] would
outperform the ×4 model, i.e. a model with an upscale factor 4, but we
did not find this to be the case (Fig. 3). One possible reason is that the LR
dataset used in the previous study was downscaled from GT images,
whereas our LR and GT datasets were obtained separately using clinicaland micro-resolution scanners. Therefore, we opted to use the ×4 EDSR
model in our study, as it can significantly improve both training andtesting time while reducing the memory required for the testing and 3Dmodelling process.
We did not observe significant improvements by enlarging the model
size (Fig. 3). This may be due to the limited number of datasets available,

The SR model based on the Accuitomo 80 results, was found toaccurately predict the force-displacement response of the specimens
when loaded in compression in the x-, y-, and z-directions, respectively(Fig. 9a–c), whereas the FE models built based on the LR data substan­tially overestimated stiffness and strength (Fig. 9d-f). On average the LR
models exhibited a 771 % higher modulus and 539 % higher strengththan the GT models. In contrast, the SR models showed errors lower than
10 % on average (stiffness: − 9.65 %; strength: 9.34 %). The modulus in
the z-direction was 16.3 % higher than the GT models, whereas in the xand y-directions the SR models underestimated the modulus by 14.5 %and 31 %, respectively (Fig. 9e). Similar results were found for thecomparison of strength (Fig. 9f).
4. Discussion
This study proposes a workflow that generates SR CT models from5

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

Fig. 5. Super-resolution model (EDSR r16f128×4) result for different trabecular bone samples from the Accuitomo dataset, i.e. bone sample 1, 2, 3, and 4 from thevalidation group. The GT, lLR, SR, and super resolution image from mixed dataset (SRMix) structures are illustrated with the corresponding SSIM and PSNR values.

as we only had around 1500 LR images for each LR dataset, but
increasing the number of parameters might cause overfitting instead ofenhancing the CNN performance. Furthermore, since neighbouring im­ages can be similar, repeated images could be used for training multiple
times by the CNN, exacerbating the issue. Nevertheless, we observedthat even a small model (r16f64×4) can also provide a high prediction
accuracy (PSNR 15.81 dB, SSIM 0.6443), which could be attributed to

the relatively simpler SR model, as we only need to fit a grayscale, i.e. asingle layer, whereas a typical RGB image has three layers. Furthermore,
the primary object, trabecular bone, is always a two-phase structure(background and bone), which is much simpler than other complex
features that can be present in other types of pictures, such as those ofhumans or buildings.
The variation in EDSR parameters led to different SSIM and PSNR
6

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

Fig. 6. 3D GT, LR, SR, which were segmented with different algorithms. BVF values are indicated (%). LRME: low-resolution max entropy segmentation; LROt: lowresolution Otsu segmentation; SRAd: super-resolution adaptive segmentation; SROt: super-resolution Otsu segmentation.

7

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

Fig. 7. 3D SR results for a sample specimen based on input from different CT scanners with corresponding BVF values indicated (%).

Fig. 8. Morphological parameter comparison between GT, LR and SR models: error in (a) BVF, (b) BS/BV, and (c) Tb.Th. Values next to boxplots indicate averages ±standard deviations.

values (Fig. 3). Nonetheless, differences in image quality between the SR
images were difficult to discern. Additionally, the SSIM and PSNR values
showed different SR qualities for the same SR images. It should be notedthat both SSIM and PSNR are sensitive to the grayscale values, which areoften altered during the segmentation and binarization process. Thissuggests that none of these metrics is optimal for trabecular bone SR
evaluation. Therefore, an ideal evaluation method for trabecular bone
SR should consider the contrast between the bone structure and thebackground, rather than the absolute grey value. This limitation isfurther evidenced by Figs. 5 and 6, which demonstrate that a low SSIM
value in 2D images does not necessarily result in large errors in 3D BVF.
Despite the limitation of the parameters used to quantify the quality of
SR images, we chose to use them for the 2D SR images since it is a wellaccepted measurement in the SR field [26,27].
Furthermore, as illustrated in Fig. 3, we did not observe a superior
performance in our trabecular bone SR outcomes compared with therecently developed NN models, the SAFM and MDRN. There could be
two plausible explanations for this. First, the enhancement in SR per­formance has been marginal in recent years, even in the broader SR
field, as reported in [29,30]. Second, the impact of registration errors,
noise originating from the clinical CT images, and other artifacts
induced by the original training datasets may outweigh the differencesin SR NNs. This trend is particularly pronounced in our SR tasks, giventhat the LR images employed were not downscaled from GT images
using algorithms, for instance, through interpolation. Consequently,
only one EDSR model was used for the subsequent tasks.
The differences in performance across different CT datasets can beobserved in Fig. 4, for the SR results. Our findings indicate that the

Accuitomo 80 dataset produced the best results in 2D, while the Verity
dataset produced the worst. This can potentially be attributed to the
pixel size of the CT images, with Accuitomo images having a pixel size of
80 μm, providing more detailed information than the Verity images witha pixel size of 125 μm. However, the noise is also a crucial factor
influencing the SR performance. The NewTom dataset has a pixel size of
75 μm but showed relatively poor performance due to high levels ofnoise.
Segmentation is a critical process before morphological analysis, as it
can significantly impact the regions considered as background or
trabecular bone [32], which would influence the BVF as well as pre­dicted mechanical properties. In this study, we employed different
segmentation methods for LR images, namely the Otsu method which iscommonly used [33], and the max entropy method, which has been usedin micro-FE analysis in previous work [34]. Our findings revealed thatthe max entropy method resulted in a 10–20 % reduction in BVF
compared to the Otsu method (Fig. 6), i.e. towards an improved pre­diction. However, the connectivity between the trabeculae was also
affected by the segmentation algorithm, and LR max entropy segmen­
tation may lead to losses in connectivity, which may affect the predictedmechanical response. In contrast to the segmentation of LR CT images,
when segmenting images for SR modelling it is preferred to broaden thethreshold boundaries rather than using a smaller interval. This isbecause the LR CT images may leave small features of the trabecularbone structure undetected, giving an underestimation of the SR struc­tural volume and connectivity. A slightly higher BVF could balance thiseffect. Therefore, we also evaluated an adaptive segmentation method in
addition to the Otsu method. Indeed, the BVF increased by around 8 %8Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

Fig. 9. Force-displacement curves from FE simulations of GT, LR, and SR models in the (a) x-, (b) y-, and (c) z-directions. Different colours represent different bone
samples in the validation dataset. The modulus and strength error comparison between LR and SR is illustrated in (d) as average values, and the differences indifferent directions between GT and SR are shown in (e) and (f).

using the adaptive method, which resulted in an error as low as 0.04 %compared to the GT (Fig. 7). Comparing the SR models based on theirown CT datasets, the mixture of CT images from different instruments
may improve the results as well, especially for the low-quality CT im­ages, e.g. group Verity (Fig. 7). Similar conclusions can be drawn for themorphological analysis (Fig. 8). Hence, the developed SR Mix model
could give good results for clinical data across different CT instruments.
Several important morphological parameters were evaluated,namely the BVF, BS/BV, and Tb.Th, for the GT, LR data, and SR models.
The BVF, in particular, is an important determinant of bone fracture risk
[1]. The BVF of LR Otsu models was much higher than the GT models(200±86 %, Fig. 8a). In contrast, the SR model could reduce the BVF
error to below 20 % (17±23 %). This can be attributed to the high Tb.Th
in the LR images (Fig. 8c), which was significantly reduced in the SR
models compared to the LR models. The trabecular separation (Tb.Sp)
was not measured here as we assume the position of trabeculae detectedby clinical CT instruments to be similar to that of the GT, and hence theTb.Th and Tb.Sp are correlated. Moreover, the LR and SR differedsignificantly in terms of BS/BV (Fig. 8b). In conclusion, the SR withadaptive segmentation gave the best overall performance for themorphological parameters (mean error≤50 %), while the LR performedmuch worse (mean error can be over 100 %). Compared to the previous
studies with the same database (e.g. [13,39,40]), a significant
improvement can be found in morphological parameters evaluation. The
results from the SR CT model are comparable to those from the micro-CT
GT models, even without applying a linear regression correction.
Furthermore, this approach may also address the issue of biases inclinical CT models that linear regression cannot rectify, as discussed in[39]. Furthermore, the prediction accuracy for morphological parame­ters is also comparable with SR studies that used micro-CT for their LR
datasets [21]. In contrast, this study employed clinical CT images, which
could introduce additional noise, motion artifacts, and other challenges.
The comparison of stress-strain response derived from FE model re­sults (Fig. 9a–c) revealed that the LR Otsu method tends to substantiallyoverestimate modulus and strength, likely due to its tendency to over­estimate BVF and Tb.Th (Fig. 8). In contrast, the mechanical evaluationof SR adaptive models showed good agreement with the GT models,even in the post-ultimate stress regime. Both stiffness and strength of the
SR models demonstrated low average errors (modulus − 10±37 %;strength 9±56 %) compared to the GT models (Fig. 9d). This prediction
accuracy is significantly higher than that in a previous study with thesame bone models [13]. However, the prediction of modulus andstrength varied in different directions. Specifically, the SR models
underestimated the mechanical properties in x- and y-directions, how­ever, overestimated them in the z-direction (Fig. 9e, f). This could be
attributed to the limitation of the stack algorithm in 3D.
One limitation of this study is the low number of samples available
for training and testing. Although thousands of clinical CT images wereacquired for each group, most of them provided redundant information,
particularly the neighbouring layers. The low number of samples also
led to the restriction of the EDSR model to 2D rather than 3D, i.e. thenumber of specimens was insufficient for direct 3D deep learning modeltraining. Additionally, the restriction of a 2D model may also enlarge theinfluence of model registration error, compared to a direct 3D model.
However, this is the first time SR trabecular bone models have beendeveloped from different clinical CT datasets, demonstrating a superior
extraction of morphological and mechanical properties compared to thenative clinical CT images.
Another limitation pertains to the quality and accessibility of theclinical CT employed in this study. As shown in Table 1, the acquisition
time was long, potentially increasing the risk of motion artifacts.
Furthermore, it has been reported that the HR-pQCT is confined to small
volumes of interest and specific body sites. These limitations inherent toCT instruments may pose challenges for clinical application at the cur­rent stage.
Potential future investigations could encompass the creation of NN
architecture tailored explicitly for SR tasks related to trabecular bone (e.
g. [21]), employing advanced CT techniques to shorten scanning9Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009

duration and mitigate motion artifacts (e.g. PCCT [41,42]), and utilizing
a greater number of trabecular bone samples to directly construct 3D SR
NN models. These endeavours would contribute to enhancing the ac­
curacy of SR models and reducing the computing time associated with
NN. Additionally, another prospective study could involve employing
NN to directly predict morphological parameters and mechanicalproperties of trabecular bone, which might further accelerate the anal­
ysis process and provide the possibility of large bone sample analysis
considering the trabecular bone micro-structure.

[8] N. Mikolajewicz, N. Bishop, A.J. Burghardt, L. Folkestad, A. Hall, K.M. Kozloff, F.
H. Glorieux, HR-pQCT measures of bone microarchitecture predict fracture:systematic review and meta-analysis, J. Bone Miner. Res. 35 (3) (2020) 446–459.
[9] E. Klintström, B. Klintström, D. Pahr, T.B. Brismar, Ö. Smedby, R. Moreno, Direct
estimation of human trabecular bone stiffness using cone beam computed
tomography, Oral Surg. Oral Med. Oral Pathol. Oral Radiol. 126 (1) (2018) 72–82.
[10] G.J. Kazakia, A.J. Burghardt, T.M. Link, S. Majumdar, Variations in morphologicaland biomechanical indices at the distal radius in subjects with identical BMD,
J. Biomech. 44 (2) (2011) 257–266.
[11] W. Tjong, G.J. Kazakia, A.J. Burghardt, S. Majumdar, The effect of voxel size on
high-resolution peripheral computed tomography measurements of trabecular andcortical bone microstructure, Med. Phys. 39 (4) (2012) 1893–1903.
[12] E. Klintström, B. Klintström, A. Spångeus, M. Sandborg, M. Woisetschläger,
Trabecular bone microstructure analysis on data from a novel twin robotic X-ray
device, Acta Radiol. 64 (4) (2023) 1566–1572.
[13] E. Klintström, B. Klintström, R. Moreno, T.B. Brismar, D.H. Pahr, Ö. Smedby,
Predicting trabecular bone stiffness from clinical cone-beam CT and HR-pQCT
data; an in vitro study using finite element analysis, PLoS ONE 11 (8) (2016)
e0161101.
[14] S. Lu, Z. Zhu, J.M. Gorriz, S.H. Wang, Y.D. Zhang, NAGNN: classification of COVID19 based on neighboring aware representation from deep graph neural network,
Int. J. Intell. Syst. 37 (2) (2022) 1572–1598.
[15] S. Lu, S.H. Wang, Y.D. Zhang, Detection of abnormal brain in MRI via improved
AlexNet and ELM optimized by chaotic bat algorithm, Neural Comput. Appl. 33(2021) 10799–10811.
[16] I. Guha, S.A. Nadeem, X. Zhang, S.M. Levy, J.C. Torner, P.K. Saha, Unsupervised
GAN-CIRCLE for high-resolution reconstruction of bone microstructure from lowresolution CT scans, in: Proceedings of the Medical Imaging 2021: Biomedical
Applications in Molecular, Structural, and Functional Imaging 11600, SPIE, 2021,
pp. 331–341.
[17] C. Jiang, Q. Zhang, R. Fan, Z. Hu, Super-resolution ct image reconstruction basedon dictionary learning and sparse representation, Sci. Rep. 8 (1) (2018) 8799.
[18] W. Yang, X. Zhang, Y. Tian, W. Wang, J.H. Xue, Q. Liao, Deep learning for single
image super-resolution: a brief review, IEEE Trans. Multimed. 21 (12) (2019)
3106–3121.
[19] C. Qiao, D. Li, Y. Guo, C. Liu, T. Jiang, Q. Dai, D. Li, Evaluation and development of
deep neural networks for image super-resolution in optical microscopy, Nat.
Methods 18 (2) (2021) 194–202.
[20] B. Lim, S. Son, H. Kim, S. Nah, K. Mu Lee, Enhanced deep residual networks for
single image super-resolution, in: Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition Workshops, 2017, pp. 136–144.
[21] Zhang, S., Liang, Y., Li, X., Li, S., Xiong, X., & Zhang, L. (2023). AESR3D: 3D
overcomplete autoencoder for trabecular computed tomography super resolution.
CAAI Transactions on Intelligence Technology.
[22] C. You, G. Li, Y. Zhang, X. Zhang, H. Shan, M. Li, G. Wang, CT super-resolution
GAN constrained by the identical, residual, and cycle learning ensemble (GANCIRCLE), IEEE Trans. Med. Imaging 39 (1) (2019) 188–203.
[23] H. Xiao, X. Teng, C. Liu, T. Li, G. Ren, R. Yang, J. Cai, A review of deep learningbased three-dimensional medical image registration methods, Quant. Imaging
Med. Surg. 11 (12) (2021) 4895.
[24] Gouvine, G., (2021). Super-resolution networks for pytorch. GitHub repository. htt
ps://github.com/Coloquinte/torchSR.
[25] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, S. Chintala,
Pytorch: an imperative style, high-performance deep learning library, Adv. Neural
Inf. Process. Syst. 32 (2019), https://doi.org/10.48550/arXiv.1912.01703.
[26] Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: fromerror visibility to structural similarity, IEEE Trans. Image Process. 13 (4) (2004)
600–612.
[27] A. Hore, D. Ziou, Image quality metrics: PSNR vs. SSIM, in: Proceedings of the 20th
International Conference on Pattern Recognition, IEEE, 2010, pp. 2366–2369.
[28] Kingma, D.P., & Ba, J. (2014). Adam: a method for stochastic optimization. arXiv
preprint arXiv:1412.6980.
[29] Y. Li, Y. Zhang, R. Timofte, L. Van Gool, L. Yu, Y. Li, X. Wang, NTIRE 2023
challenge on efficient super-resolution: methods and results, in: Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023,
pp. 1921–1959.
[30] Sun, L., Dong, J., Tang, J., & Pan, J. (2023). Spatially-adaptive feature modulationfor efficient image super-resolution. arXiv preprint arXiv:2302.13800.
[31] Y. Mao, N. Zhang, Q. Wang, B. Bai, W. Bai, H. Fang, S. Yan, Multi-level dispersion
residual network for efficient image super-resolution, in: Proceedings of the IEEE/
CVF Conference on Computer Vision and Pattern Recognition, 2023,
pp. 1660–1669.
[32] P.K. Sahoo, S.A.K.C. Soltani, A.K. Wong, A survey of thresholding techniques,Comput. Vis. Graph. Image Process. 41 (2) (1988) 233–260.
[33] N. Otsu, A threshold selection method from gray-level histograms, IEEE Trans.
Syst. Man Cybern. 9 (1) (1979) 62–66.
[34] M. Ovesy, M. Aeschlimann, P.K. Zysset, Explicit finite element analysis can predictthe mechanical response of conical implant press-fit in homogenized trabecularbone, J. Biomech. 107 (2020) 109844.
[35] J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch,
A. Cardona, Fiji: an open-source platform for biological-image analysis, Nat.
Methods 9 (7) (2012) 676–682, https://doi.org/10.1038/nmeth.2019.
[36] D. Bradley, G. Roth, Adaptive thresholding using the integral image, J. Graph.
Tools 12 (2) (2007) 13–21.
[37] Zhou, Y. (2024). Bone-Screw-Constructs-eFEM. GitHub. https://github.com/zhou4
36/Bone-Screw-Constructs-eFEM.

5. Conclusion
In conclusion, a modified enhanced deep SR network based on CNNs
was successfully implemented to generate SR images using data fromdifferent clinical HR-CT scans. Our results demonstrate that the SR im­
ages outperform the LR (HR-CT) images for the determination ofmorphological parameters, as confirmed by a 3D comparison to GT
micro-CT scans. The performance differences between different SR NN
models were marginal. However, the SR quality was highly influenced
by the original HR-CT image quality. The mixture of different CT image
datasets may improve the SR model performance, especially for the lowquality CT images. The adaptive segmentation method can help the SR
models to better match the connectivity and predicted mechanicalresponse to that derived from ground truth image data. Finally, we
believe that this study demonstrates that SR techniques could signifi­
cantly enhance image quality of clinical image data and have a sub­stantial impact on assessment of fracture risk in future clinical practice.
Ethical approval
The specimens were donated for medical research. The donations
were in accordance with the ethical guideline regulations at the Uni­
versity of California, San Francisco. All specimens have been used inprevious studies [13,39,40].
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Funding
The authors are grateful for support from the European Union’s
Horizon 2020 research and innovation program under the Marie Skło­
dowska-Curie grant agreement No 812765.
References[1] K.L. Stone, D.G. Seeley, L.Y. Lui, J.A. Cauley, K. Ensrud, W.S. Browner, S.
R. Cummings, BMD at multiple sites and risk of fracture of multiple types: longterm results from the study of osteoporotic fractures, J. Bone Miner. Res. 18 (11)(2003) 1947–1954.
[2] I. Fleps, P. Guy, S.J. Ferguson, P.A. Cripton, B. Helgason, Explicit finite element
models accurately predict subject-specific and velocity-dependent kinetics ofsideways fall impact, J. Bone Miner. Res. 34 (10) (2019) 1837–1850.
[3] S.C.E. Schuit, M. Van der Klift, A.E.A.M. Weel, C.E.D.H. De Laet, H. Burger,
E. Seeman, H.A.P. Pols, Fracture incidence and association with bone mineraldensity in elderly men and women: the Rotterdam Study, Bone 34 (1) (2004)
195–202.
[4] S.A. Wainwright, L.M. Marshall, K.E. Ensrud, J.A. Cauley, D.M. Black, T.A. Hillier,Study of Osteoporotic Fractures Research Group, Hip fracture in women without
osteoporosis, J. Clin. Endocrinol. Metab. 90 (5) (2005) 2787–2793.
[5] J. Rueckel, M. Stockmar, F. Pfeiffer, J. Herzen, Spatial resolution characterizationof a X-ray microCT system, Appl. Radiat. Isot. 94 (2014) 230–234.
[6] K.K. Nishiyama, E. Shane, Clinical imaging of bone microarchitecture with HRpQCT, Curr. Osteoporos. Rep. 11 (2013) 147–155.
[7] S. Boutroy, M.L. Bouxsein, F. Munoz, P.D. Delmas, In vivo assessment of trabecularbone microarchitecture by high-resolution peripheral quantitative computed
tomography, J. Clin. Endocrinol. Metab. 90 (12) (2005) 6508–6515.

10

Y. Zhou et al.

Computer Methods and Programs in Biomedicine 245 (2024) 108009[38] W.S. Enns-Bray, H. Bahaloo, I. Fleps, O. Ariza, S. Gilchrist, R. Widmer, B. Helgason,
Material mapping strategy to improve the predicted response of the proximal
femur to a sideways fall impact, J. Mech. Behav. Biomed. Mater. 78 (2018)
196–205.
[39] B. Klintström, L. Henriksson, R. Moreno, A. Malusek, Ö. Smedby,
M. Woisetschläger, E. Klintström, Photon-counting detector CT and energyintegrating detector CT for trabecular bone microstructure analysis of cubic
specimens from human radius, Eur. Radiol. Exp. 6 (1) (2022) 31.
[40] E. Klintström, Ö. Smedby, R. Moreno, T.B. Brismar, Trabecular bone structure
parameters from 3D image processing of clinical multi-slice and cone-beamcomputed tomography data, Skelet. Radiol. 43 (2014) 197–204.

[41] F. Azari, W. Coudyzer, C.E. Wyers, J.P. van den Berg, G van Lenthe, Photoncounting CT appears promising in quantifying bone microstructure in the knee, in:Proceedings of the 50th Congress of the European Calcified Tissue Society,
Location, Liverpool, UK, 2023. Date: 2023/04/14-2023/04/18.
[42] F. Azari, W. Coudyzer, C.E. Wyers, J.P. van den Bergh, G. van Lenthe, Quantitative
assessment of bone microarchitecture in the human knee using photon-counting CT
is feasible, in: Proceedings of the 28th Congress of the European Society ofBiomechanics, Maastricht, the Netherlands, 2023. Date: 2023/07/09-2023/07/
12Location.

11