# International validation of a pre-transplant risk assessment tool for graft survival in pediatric kidney transplant recipients

Loes Oomen, Liesbeth L de Wall, Burkhard Tonshoff, Kai Krupka, Jerome Harambat, Julien Hogan, Cecile Couchoud, Emilie Savoye, Huib de Jong, Elisabeth a M Cornelissen, et al.

► To cite this version:

Loes Oomen, Liesbeth L de Wall, Burkhard Tonshoff, Kai Krupka, Jerome Harambat, et al. International validation of a pre-transplant risk assessment tool for graft survival in pediatric kidney transplant recipients. Clinical Kidney Journal, 2025, 18 (3), pp.saf031. 10.1093/ckj/saf031. hal-05043817

HAL Id: hal-05043817

https://hal.science/hal-05043817v1

Submitted on 23 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ORIGINAL ARTICLE

# International validation of a pre-transplant risk assessment tool for graft survival in pediatric kidney transplant recipients

Loes Oomen ID¹, Liesbeth L. de Wall¹, Burkhard Tönshoff², Kai Krupka², Jerome Harambat³, Julien Hogan⁴, Cécile Couchoud⁵, Emilie Savoye ID⁵, Huib de Jong⁶, Elisabeth A.M. Cornelissen⁷, Antonia H.M. Bouts ID⁸, Mandy G. Keijzer-Veen⁹, Wout F.J. Feitz¹ and Charlotte M.H.H.T. Bootsma-Robroeks⁷,¹⁰

¹Department of Urology, Division of Paediatric Urology, Radboudumc Amalia Children's Hospital, Nijmegen, The Netherlands, ²CERTAIN Registry, Department of Paediatrics I, University Children's Hospital Heidelberg, Heidelberg, Germany, ³Department of Pediatrics, Pediatric Nephrology Unit, Bordeaux University Hospital, Bordeaux, France, ⁴Department of Pediatric Nephrology and Transplantation, Robert-Debré Hospital, APHP, Paris, France, ⁵CRISTAL Registry, Agence de La Biomédicine, Paris, France, ⁶Department of Paediatric Nephrology, Erasmus MC-Sophia Children's Hospital, Rotterdam, The Netherlands, ⁷Department of Paediatric Nephrology, Radboudumc Amalia Children's Hospital, Nijmegen, The Netherlands, ⁸Department of Paediatric Nephrology, Amsterdam University Medical Centre, Emma Children's Hospital, Amsterdam, The Netherlands, ⁹Department of Paediatric Nephrology, Wilhelmina Children's Hospital, University Medical Centre Utrecht, Utrecht, The Netherlands and ¹⁰University of Groningen, University Medical Centre Groningen, Department of Paediatrics, Paediatric Nephrology, Beatrix Children's Hospital, Groningen, The Netherlands

Correspondence to: Loes Oomen; E-mail: loes.oomen@radboudumc.nl

## ABSTRACT

**Background.** A pre-transplant prediction model using commonly available factors is valuable for optimizing donor selection, communication, and counseling for pediatric kidney transplant (PKT) recipients. This study aims to externally validate a Dutch PKT prediction model and assess its international applicability.

**Materials and methods.** Data from the Dutch-, CERTAIN-, and CRISTAL registries, covering PKT from 2005 to 2021, were used. The Dutch prediction model was externally validated in a German and French cohort and then adapted to these specific countries. An international prediction model was also developed using all available data. Models were based on 80% derivation cohorts and internally validated using areas under the receiver operating characteristic curve (ROC-AUC) and calibration plots.
**Results.** Of 3266 transplantations, 2475 (273 Dutch, 356 German, 1622 French, and 224 other) were used for analysis. Cohorts differed significantly in baseline characteristics and outcomes. Internal validation of the Dutch model showed ROC-AUC of 0.77 and 0.75 at 10 and 15 years. External validation in German and French cohorts yielded 10-year ROC-AUC of 0.63 and 0.60, respectively.

Internal validation of the international prediction model showed AUC of 0.61 and 0.60 at 10 and 15 years with poor calibration, indicating inferior performance. The adapted national models showed better internal validation performance, with 10-year ROC-AUC of 0.77, 0.76, and 0.73 in Dutch, French, and German cohorts, respectively.

**Conclusions.** The Dutch PKT prediction tool requires country-specific adaptations for use in other countries, given the diversity of clinical practice across Europe. A country-specific model is preferable to an international model in the current landscape.

## GRAPHICAL ABSTRACT

<figure><img src="image_2.png" /></figure>

Clinical
Kidney
Journal

# International validation of a pre-transplant risk assessment tool for graft survival in pediatric kidney transplant recipients

External validation Dutch prediction model for graft failure in a French and German cohort.
Development and internal validation of national models and an international model based on the whole cohort.

### Methods

<figure><img src="image_3.png" /></figure>

2475 transplantations

<figure><img src="image_4.png" /></figure>

2005–2021

<figure><img src="image_5.png" /></figure>

### Results

<figure><img src="image_6.png" /></figure>

External validation Dutch model: poor performance

<figure><img src="image_7.png" /></figure>

Internal validation international model: poor performance

<figure><img src="image_8.png" /></figure>

Internal validation country-specific models: good performance

<figure><img src="image_9.png" /></figure>

**Conclusion:** The Dutch prediction tool requires country-specific adaptations for use in other countries, given the diversity of clinical practice across Europe. A country-specific model is preferable to an international model in the current landscape.

Oomen, L.
Clinical Kidney Journal (2025)
@CKJsocial

**Keywords:** graft survival, pediatric kidney transplantation, prediction model, validation study

## INTRODUCTION

Pediatric kidney transplantation (PKT) is the preferred treatment for children with kidney failure (stage 5 chronic kidney disease), and continuous advances have significantly improved both patient and graft survival outcomes over time [1, 2]. However, there is still room for improvement in long-term outcomes, and individualized prediction of long-term allograft failure would facilitate clinical decision-making and patient counseling.

Transplantation success involves a complex interplay of several factors [3–7]. Although several pre-transplant parameters are known to independently influence graft survival, their relative weights and combined impact remain relatively unexplored [4, 8, 9]. Besides, the relative hazard of these parameters is not constant over time, as described by Coens et al. [10]. To facilitate

optimal donor selection for PKT recipients, a pre-transplant prediction model incorporating only commonly available pre-transplant factors could be of great value.

Over time, several models have been developed to predict graft survival. Yet, these models either rely on post-transplantation data or lack specificity for the pediatric population [11–13]. Therefore, we developed a pediatric pre-transplant risk assessment tool based on data from all PKTs in the Netherlands between 1990 and 2021. Through internal validation, the tool has demonstrated good prediction performance for graft survival in Dutch PKT patients [14].

The care of PKT recipients in the Netherlands may differ from that in other (European) countries. To ensure wider applicability of the model, it is important to assess and confirm its reproducibility and generalizability across Europe.
# KEY LEARNING POINTS

## What was known:

* The outcome of kidney transplantation is influenced by an interplay of multiple factors.

* Predicting graft survival prior to transplantation can help in determining the optimal transplantation strategy.

* A Dutch pre-transplant prediction model for graft survival after pediatric kidney transplantation has demonstrated good predictive performance in the Netherlands.

## This study adds:

* These predictive models only work in their own countries.

* External validation of the Dutch model showed poor predictive performance, requiring adaptations for international use.

* Similarly, a generalized international model based on 2475 European transplants performed poorly, likely due to country heterogeneity.

* However, country-specific models performed well in their own country.

## Potential impact:

* Country-specific prediction models show promise in supporting clinical decision-making and personalized medicine by comparing different scenarios for individual patients.

* Differences among countries hinder international models, presenting an opportunity for future research to learn from each other and potentially develop unified guidelines.

In addition, given the potential requirements for adapting this model to other populations, it is worth exploring whether the predictive accuracy of such a model benefits from international data or whether a country-specific approach is preferable.

Therefore, the primary objective of this study was to perform an external validation of the Dutch risk assessment tool.

The secondary objective was to develop and evaluate an international prediction model and to determine whether an international or a country-specific approach is more accurate.

β-regression coefficients from this analysis were used to develop a prediction model. These β-coefficients served as slope coefficients in the prediction formula that calculated the hazard of graft failure for each patient at a given time point after transplantation. These hazard estimates were then used to compute predicted survival, which facilitated the creation of individual survival curves based on the available parameters.

This predictive model enables clinicians to estimate monthly graft survival in various clinical scenarios using pre-transplantation data. By allowing comparisons between different donors for a specific patient, it supports personalized clinical decision-making.

# MATERIALS AND METHODS

## Original model and dataset

Our original risk assessment tool was based on data from the Dutch organ transplantation registry NOTR (Nederlandse Organ Transplantatie Registratie; Dutch Organ Transplantation Registry) as described previously [14].

In brief, this model was based on 554 PKTs from 1990 to 2021. To develop this model, a multivariable binary logistic regression analysis was performed, including the following parameters: recipient age, donor age, living donor (LD) status, pre-emptive transplantation, re-transplant, number of human leukocyte antigen (HLA) mismatches, number of HLA-DR mismatches, and the underlying primary disease responsible for kidney failure. These were classified as congenital anomalies of the kidney and urinary tract (CAKUT), ciliopathy, glomerulopathy, metabolic nephropathy, rare cause of hypertension, tubulopathy, or other. The model was adjusted to account for the decade (era) in which the transplantations took place (Appendix 2).

Variable selection was knowledge driven: incorporating findings from literature reviews and expert opinions to identify potential predictors of graft function. All selected predictors were readily available clinical parameters known prior to transplantation. Variables such as sex mismatches, duration of dialysis, HLA-DQ mismatches, and donor health were excluded due to missing data, while ABO incompatibility was excluded because of its low incidence.

A person-period file was constructed, with each month of follow-up treated as an individual observation point. Multivariate binary logistic regression was performed, and the

## Data collection for external validation

An independent geographical validation approach was used to test the Dutch prediction model in a set of international patient data [15, 16]. The data for this external validation were derived from the Cooperative European Paediatric Renal Transplant Initiative (CERTAIN) registry [17] and the French CRISTAL [18] registry.

The web-based CERTAIN registry, launched in 2011, serves as a research platform for PKT and collects comprehensive longitudinal data on recipients under the age of 21 years at the time of transplantation from 95 actively participating centers in 26 countries [17, 19]. Enrollment in the CERTAIN registry requires the approval of the ethics committee of each participating center. Informed consent from parents or legal guardians is required before patients can be enrolled, and patients are required to give consent if appropriate for their age [17, 20]. Data are collected at several time points: baseline (pre-transplant), 1, 3-, 6-, 9-, and 12-months post-transplant, and every 6 months thereafter. Data in this database are collected retrospectively starting from 1991, with prospective data collection implemented since 2011.

CRISTAL is a mandatory national database, initiated in 1996, driven by the Agence de la biomédecine [18, 21]. Data on all organ transplantation candidates, recipients, and donors in France are collected prospectively by each transplantation center. Demographics, clinical, and laboratory data are collected at several key points, such as listing, transplantation surgery, and at subsequent annual intervals until the patient's death.
<figure><img src="image_10.png" /><figcaption>**Figure 1:** Flowchart of study data. Creation of cohorts used for development and analysis of predictive models. The modified CERTAIN cohort consisted of CERTAIN data excluding German, Dutch, and French data and was not used to develop a prediction model. CERTAIN: Cooperative European Pediatric Renal Transplant, NOTR: Nederlandse Orgaan Transplantatie Registratie.</figcaption></figure>

Follow-up data were used until either graft loss or death of a patient with a functioning graft. Graft loss was defined as the initiation of dialysis or the need for re-transplant. When a patient died with a functioning graft, this was considered as graft survival as the death was not due to renal problems. Transplantations in recipients under 19 years of age performed between 2005 and 2021 were included, using a complete case analysis. Patients with missing data were therefore excluded from the multivariate analysis. Other exclusion criteria comprised multiple organ transplantations, and transplantations performed outside Europe.

Five cohorts were developed based on the available data (Fig. 1), including three national cohorts: Dutch, French, and German cohorts. A modified CERTAIN cohort was formed, incorporating the data from countries beyond the aforementioned ones (excluding non-European countries and transplantations before 2011). Furthermore, an international cohort combining all these datasets was used to develop an international prediction model.

The current external validation study used data from 2005 to 2021 onwards to ensure an adequately powered model that takes into account the substantial changes in health care over time. This decision was motivated by the scarcity of data in the external validation cohort. Therefore, an adapted 'derivation cohort' was developed based on 80% of the Dutch data on transplantations performed between 2005 and 2021.

## Statistical analysis

To address the primary objective, an external validation of the adapted Dutch model (2005–2021) was performed in the German and French cohorts. For the second objective, the international cohorts were used to re-estimate the regression coefficients of this Dutch model. As stated by Moons et al., it is recommended to adjust or update existing prediction models using the available external validation dataset by re-estimating the regression coefficients [16, 22].

Therefore, an adjusted prediction model was developed for the international cohort, as well as country-specific models for

Germany and France, using the same methodology and identical predictors as the original model [14].

## Model development

In summary, a person-period file was generated, treating each month of follow-up as a single observation point to account for the time-varying patterns of the determinants [23]. Each cohort was randomly divided into a derivation cohort (80%) and a validation cohort (20%).

The multivariable logistic regression model was fitted using the derivation cohorts and included the parameters mentioned before, with the time elapsed after transplantation modeled using restricted cubic splines [24, 25].

Natural splines with four nodes positioned at quantiles 0.05, 0.35, 0.65, and 0.95 were utilized, resulting in five time-related variables. The β-coefficients of this model were employed to construct a prediction score, predicting the probability of graft survival for each month after transplantation (Appendix 2, Tables 9–12).

To account for the different contribution rates of the individual countries within the international model, countries with fewer transplantations were weighted more heavily to equalize their influence. In addition, a dummy variable was created for each cohort.

## Model performances

The quality of these models was evaluated by comparing the predicted survival probability of each model with the actual recorded outcome. Discriminative performance was assessed using the area under the curve (AUC) of the time-dependent receiver operating characteristics (ROC) curve for each year after transplantation in the validation cohorts [26]. Calibration in the validation cohorts was assessed using calibration plots and the Hosmer–Lemeshow test [22].

The prediction model was reported following the guidelines of the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement (Appendix 3) [27].
<figure><img src="image_11.png" /><figcaption>**Figure 2:** Annual transplantation rates. (a) Annual number of pediatric kidney transplantations per cohort. (b) The annual percentage of graft loss per cohort. The modified CERTAIN cohort consisted of CERTAIN data excluding German, Dutch, and French data.</figcaption></figure>

Continuous baseline characteristics were summarized as means and standard deviations, or medians and interquartile ranges if the data deviated from normal distribution. Graft survival was evaluated using Kaplan-Meier curves and log-rank tests. Group comparisons were performed using chi-squared tests and one way ANOVA-tests, while the Mann-Whitney U-test was used for non-normally distributed data. Data analysis was performed using IBM SPSS Statistics v.29.0 and R v.4.3.1 [28, 29]. Statistical significance was set at $P < .05$. Graphics were created using BioRender.

## RESULTS

In total, the data from 3266 transplantations performed between 1991 and 2023 were collected from the different registries: two population-based national registries and one hospital-based European registry (Fig. 1). Among these, 2475 transplantations were included in the analysis, forming the international cohort. This international cohort was divided by country for the country-

specific analysis, while the modified CERTAIN cohort exclusively comprised transplantations performed in other countries than Netherlands, France, or Germany.

The annual number of transplantations in the international cohort during the period 2005–2021 ranged from 106 to 183, most of which were performed in France (Fig. 2a). In addition, the percentage of annual graft loss varied between time periods and cohorts (Fig. 2b). These differences were likewise reflected in the survival curves of each cohort, with significant differences observed in transplantations performed in the first and second eras (Fig. 3). While graft survival significantly improved over time in the Dutch and French cohorts, this was not significant in the German cohort (Appendix 1, Table 1).

## Cohorts for prediction model development and validation

By excluding transplantations performed before 2005 and after 2021, a total of 2475 transplantations from four cohorts were
<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>Figure 3: Graft survival across cohorts. Kaplan-Meier curves illustrating graft survival within each national cohort by era of transplantations. Detailed statistical comparisons between cohorts are provided in Appendix 1, Table 1. (a) Graft survival in the Dutch cohort per period of transplantation: P = .02. (b) Graft survival in the French cohort per period of transplantation: P = .02. (c) Graft survival in the German cohort per period of transplantation: P = .09.</figcaption></figure>

<table><caption>Table 1: Composition of the cohorts.</caption><thead><tr><th></th><th>Dutch cohort<br>(n = 273)</th><th>French cohort<br>(n = 1622)</th><th>German cohort<br>(n = 356)</th><th>Modified CERTAIN<br>cohort (n = 224)</th></tr></thead><tbody><tr><td>Registry</td><td>NOTR</td><td></td><td>CERTAIN</td><td>CERTAIN</td></tr><tr><td>Transplantations per<br>country % (n)</td><td></td><td></td><td></td><td></td></tr><tr><td>Germany</td><td></td><td></td><td></td><td></td></tr><tr><td>Greece</td><td></td><td></td><td></td><td>2 (5)</td></tr><tr><td>France</td><td></td><td></td><td></td><td></td></tr><tr><td>Hungary</td><td></td><td></td><td></td><td>3 (7)</td></tr><tr><td>Ireland</td><td></td><td></td><td></td><td>18 (41)</td></tr><tr><td>Italy</td><td></td><td></td><td></td><td>24 (54)</td></tr><tr><td>The Netherlands</td><td></td><td>100 (1622)</td><td>100 (356)</td><td></td></tr><tr><td>Poland</td><td></td><td></td><td></td><td>15 (34)</td></tr><tr><td>Russia</td><td></td><td></td><td></td><td>2 (4)</td></tr><tr><td>Spain</td><td>100 (273)</td><td></td><td></td><td>3 (7)</td></tr><tr><td>Switzerland</td><td></td><td></td><td></td><td>5 (12)</td></tr><tr><td>Turkey</td><td></td><td></td><td></td><td>10 (23)</td></tr><tr><td>United Kingdom</td><td></td><td></td><td></td><td>17 (37)</td></tr></tbody></table>

Data sources that were used to develop the different cohorts. The modified CERTAIN cohort consisted of CERTAIN data excluding the German, Dutch, and French transplantations.

eligible for analysis (Fig. 1, Table 1). There were significant differences in characteristics and outcome between the cohorts (Table 2). The Dutch cohort had a significantly higher number of living donations than both the French and German cohorts, and the donors in the Dutch cohort were significantly older. Detailed comparisons between cohorts can be found in Appendix 1, Table 2.

## External validation of the Dutch prediction model

The Dutch prediction model developed using the 80% derivation cohort (Appendix 1, Tables 4, and 9) showed good performance within the Dutch validation cohort (Fig. 4, orange line, Appendix Fig. 1) achieving an AUC of 0.76, 0.77, and 0.75 after 1, 10, and 15 years, respectively. However, external validation in the German and French cohorts resulted in a 10-year AUC of 0.63 and 0.60 (Fig. 4). In summary, the Dutch model showed good performance in the Dutch population, but exhibited limited predictive accuracy in other countries.

## International model

An international model was developed based on the 80% derivation cohort of all transplantations (Appendix Tables 5 and 12). Internal validation showed an AUC of 0.68, 0.61, and 0.60 at 1, 10, and 15 years (Fig. 5a) and poor calibration beyond 1-year post-transplantation (Fig. 5b) with overestimation of the true graft survival.

## Predictive capacity national models

To assess whether nationally derived models (Appendix Tables 6, 7, 10 and 11) performed better than an international model, internal validation was performed on the three different country-specific models (Fig. 6). The 10-year AUC was found to be 0.77, 0.76, and 0.73 in the Dutch, French, and German cohorts, respectively. Calibration plots showed a good fit within the first years after transplantation, although predictions became less accurate on the longer term. In summary, the
<table><caption>Table 2: Cohort characteristics.</caption><thead><tr><th></th><th>International cohort (n = 2475)</th><th>Dutch cohort (n = 273)</th><th>French cohort (n = 1622)</th><th>German cohort (n = 356)</th><th>Modified CERTAIN cohort (n = 224)</th><th>P value</th><th>Test performed</th></tr></thead><tbody><tr><td>Period PKT</td><td>2005-2021</td><td>2005-2021</td><td>2005-2021</td><td>2005-2021</td><td>2011-2021</td><td></td><td></td></tr><tr><td>Follow-up (m) median [IQR]</td><td>76 [37-133]</td><td>94 [45-144]</td><td>85 [42-136]</td><td>68 [32-123]</td><td>39 [15-69]</td><td>&lt;.01</td><td>Kruskall-Wallis</td></tr><tr><td>Graft loss % (n)</td><td>25 (418)</td><td>22 (61)</td><td>16 (262)</td><td>12 (42)</td><td>24 (53)</td><td>&lt;.01</td><td>Chi<sup>2</sup> test</td></tr><tr><td>Recipient age median [IQR]</td><td>13 [7-16]</td><td>11 [6-15]</td><td>13 [8-16]</td><td>11 [5-14]</td><td>13 [9-15]</td><td>&lt;.01</td><td>Kruskall-Wallis</td></tr><tr><td>Donor age median [IQR]</td><td>19 [13-41]</td><td>43 [33-50]</td><td>16 [13-27]</td><td>35 [15-44]</td><td>34 [14-44]</td><td>&lt;.01</td><td>Kruskall-Wallis</td></tr><tr><td>HLA MM median [IQR]</td><td>3 [2-4]</td><td>3 [2-3]</td><td>3 [3-4]</td><td>3 [2-3]</td><td>3 [2-4]</td><td>&lt;.01</td><td>Kruskall-Wallis</td></tr><tr><td>LD % (n)</td><td>25 (619)</td><td>43 (116)</td><td>19 (308)</td><td>34 (122)</td><td>33 (73)</td><td>&lt;.01</td><td>Chi<sup>2</sup> test</td></tr><tr><td>Pre-emptive PKT % (n)</td><td>25 (608)</td><td>26 (71)</td><td>25 (401)</td><td>24 (84)</td><td>23 (52)</td><td>.87</td><td>Chi<sup>2</sup> test</td></tr><tr><td>Re-transplant % (n)</td><td>8 (203)</td><td>13 (34)</td><td>7 (119)</td><td>10 (34)</td><td>7 (16)</td><td>.03</td><td>Chi<sup>2</sup> test</td></tr><tr><td>Underlying disease % (n)</td><td></td><td></td><td></td><td></td><td></td><td></td><td>Chi<sup>2</sup> test</td></tr><tr><td>CAKUT</td><td>36 (900)</td><td>29 (79)</td><td>37 (594)</td><td>41 (146)</td><td>36 (81)</td><td>.02</td><td></td></tr><tr><td>Ciliopathy</td><td>9 (224)</td><td>8 (23)</td><td>8 (122)</td><td>14 (49)</td><td>13 (30)</td><td>&lt;.01</td><td></td></tr><tr><td>Glomerulopathy</td><td>16 (404)</td><td>11 (30)</td><td>16 (265)</td><td>19 (69)</td><td>18 (40)</td><td>.04</td><td></td></tr><tr><td>Tubulopathy</td><td>4 (86)</td><td>1 (2)</td><td>4 (72)</td><td>2 (8)</td><td>2 (4)</td><td>&lt;.01</td><td></td></tr><tr><td>Microvascular thrombopathy</td><td>4 (110)</td><td>3 (9)</td><td>4 (70)</td><td>6 (22)</td><td>4 (9)</td><td>.32</td><td></td></tr><tr><td>Hereditary nephropathy</td><td>7 (181)</td><td>9 (25)</td><td>9 (150)</td><td>1 (2)</td><td>2 (4)</td><td>&lt;.01</td><td></td></tr><tr><td>Metabolic nephropathy</td><td>2 (54)</td><td>2 (6)</td><td>2 (30)</td><td>3 (9)</td><td>4 (9)</td><td>.21</td><td></td></tr><tr><td>Other underlying disease cause</td><td>21 (516)</td><td>36 (99)</td><td>20 (319)</td><td>14 (51)</td><td>21 (47)</td><td>&lt;.01</td><td></td></tr></tbody></table>

IQR, interquartile range.

Overview of the cohort characteristics with overall statistics, significance is highlighted in bold. Detailed statistical comparisons between cohorts are presented in the Appendix Table 4. The modified CERTAIN cohort consisted of CERTAIN data without the German, Dutch, and French data.

<figure><img src="image_15.png" /><figcaption>**Figure 4:** External validation of the Dutch prediction model. Predictive performance of a model based on the Dutch cohort. The model was externally validated in the French (n = 1622) and German (n = 356) cohorts and internally validated in the Dutch validation cohort. The predictive capacity was assessed by calculating the AUC for each year post-transplant, accompanied by a 95% confidence interval. A predictive capacity >0.7 was considered to be good.</figcaption></figure>

national prediction models demonstrated strong predictive accuracy within their respective countries.

## DISCUSSION

This study reveals that, although the Dutch prediction model performs well within the Dutch population, its predictive accu-

racy diminishes significantly when applied to German or French cohorts. In addition, the model based on combined international data showed inferior predictive ability, despite being based on a larger database. This might be (partially) attributed to the heterogeneity of cohorts and predictor effects, both of which are influenced by differences in clinical care and protocols across countries.
<figure><img src="image_16.png" /><figcaption>**Figure 5:** Predictive capacity international model. Predictive performance of a model derived from an international dataset of 2475 transplantations, where 80% of the data served as the derivation cohort. Characteristics of the derivation and validation cohorts are shown in the **Appendix 1**, **Table 5**. (a) AUC for each year after transplantation within the validation cohort with 95% confidence interval. (b) Calibration plot. Observed graft survival in the validation cohort and the graft survival predicted by the international prediction model. Hosmer–Lemeshow test: 12.119, (P = .15).</figcaption></figure>

## Cohort heterogeneity

The 'modified CERTAIN cohort' illustrates the heterogeneity of the European landscape. Preferably, an international model is based on the data of all those countries, however, the population and clinical practices seem to differ largely [31]. When the international model demonstrated limited predictive value, a country-specific approach was adopted. Country-specific tailored models had a good predictive ability in their own country. This discrepancy may be partly explained by differences in clinical practice between countries, as shown by differences in cohort characteristics.

Notable differences were observed in factors such as the use of living donors, the age of recipients and donors, and the prevalence of retransplants. Previous studies have also shown large differences in kidney allocation policies and transplantation practices among European countries [30, 31].

Moreover, there were notable disparities in the underlying diseases leading to kidney failure, such as the prevalence of patients with CAKUT. Similar variations in the reported number of patients with CAKUT have also been observed in other studies [32, 33]. These differences may be due to different classification systems and definitions. In addition, the utilization of prenatal
<figure><img src="image_17.png" /><figcaption>**Figure 6:** Predictive capacity national models. Internal validation of country-specific models assessing discrimination and calibration in the respective validation cohorts. (a) AUC for each year after transplantation for the Dutch model within the Dutch validation cohort with 95% CI. (b) AUC for each year after transplantation for the French model within the French validation cohort with 95% CI. (c) AUC for each year after transplantation for the German model within the German validation cohort with 95% CI. (d) Observed graft survival in the Dutch validation cohort compared to the graft survival predicted by the Dutch prediction model. Hosmer-Lemeshow test: 4668 (P = .79). (e) Observed graft survival in the French validation cohort versus graft survival predicted by the French prediction model. Hosmer-Lemeshow test: 9764 (P = .28). (f) Observed graft survival in the German validation cohort compared to graft survival predicted by the German prediction model. Hosmer-Lemeshow test: 11732 (P = .16).</figcaption></figure>

screening could potentially contribute to the variations observed among countries [34, 35].

## Heterogeneity of predictor effects

External validation is an essential step in the development of prediction models, ensuring the reliability and applicability of a developed model beyond its initial dataset [16]. In general, the performance of prediction models is lower in an external cohort than in the derivation cohort. A well-established method for enhancing these models is to refine existing prediction models by incorporating information from the external validation datasets [16]. To improve both calibration and discriminative performance in the new cohort, it is recommended that all regression coefficients be re-estimated using data from the validation sample only [16, 36].

As expected, this geographical validation study revealed a decline in the predictive performance of the model when applied to external cohorts. Consequently, we adapted the model to these new datasets by developing country-specific models for each cohort, while also creating a generalized international model by re-estimating the regression coefficients. These updated coefficients differed significantly from the original model, highlighting the need for adaptation to ensure accurate predictions.

Heterogeneity in predictor effects is a well-known phenomenon and could be explained by several factors. Some of these discrepancies may be attributed to variations in cohort characteristics, as discussed previously [15]. For example, in the French cohort, more young deceased donors were used (n = 1064,

66%), particularly in contrast to the Dutch cohort (n = 33, 12%). This discrepancy may have influenced the divergent coefficients observed (negative in the Dutch model, positive in the French model) associated with a higher donor age. Besides, the higher rate of graft loss in the Dutch cohort can partly be attributed to the increased frequency of retransplants and a longer follow-up. However, other discrepancies, such as different estimates for time-related variables and intercepts, cannot be attributed solely to known cohort differences. This suggests that other differences in patient selection, protocols, or clinical practice may contribute to these differences.

Furthermore, several previously published prediction models, such as the iBOX, have also shown that various post-transplantation parameters significantly influence graft outcome [13, 37]. These factors are not included in the current model, and the diversity in post-transplant parameters may contribute to the heterogeneity of the regression coefficients. To enhance clinical applicability, this prediction model was intentionally designed to include only pre-transplantation variables. Including post-transplantation parameters might improve the international performance of the model. Furthermore, differences in transplantation outcome may also affect model performance, as our study showed a significant difference in graft survival between cohorts.

## Clinical applicability

As outlined in the original paper, this prediction model has the potential to support clinical decision-making regarding donor
selection and transplantation strategies. We developed a risk calculation tool based on the original model that could be used in daily practice.

The tool estimates monthly graft survival based on the char-
acteristics of both the patient and the potential donor, allowing
for the comparison of multiple scenarios tailored to individual
patients. For example, it can assist in evaluating the trade-off
between waiting for a well-matched deceased donor while re-
maining on dialysis or proceeding with a pre-emptive transplant
from a poorly matched LD [14].

While this tool is not intended to be the sole factor in
decision-making and may be less impactful for patients with
limited options, it contributes to personalized medicine by sup-
porting nuanced clinical decisions. Additionally, it enhances
communication with patients and their families, fostering
shared decision-making [14].

To date, the Dutch model has primarily been implemented
in Dutch clinics. To evaluate its influence on clinical decision-
making, conducting a model impact study would be a valuable
next step.

The utility of prediction models depends on their successful implementation and ongoing maintenance. The preference for country-specific models over a more generalized approach is a key finding of this study. It suggests that the significant diversity in clinical practice across Europe makes the uniform application of international prediction models a potential barrier.

Clinical decisions based on inaccurate prediction models
could have major consequences for a wide range of patient
outcomes, and the differences observed in the adapted models
highlight the critical role of external validation.

Moreover, it is acknowledged that the performance of predic-
tion models can deteriorate over time, a phenomenon known
as “calibration drift” [38]. Given the continuous evolution of
clinical practice and graft survival, it is essential to validate
these models every few years to ensure their continued valid-
ity. However, the frequency of model updates may change over
time.

In this study, graft survival showed significant changes over
a 10-year period (Fig. 3), although we may expect less change in
both clinical care and graft survival in the future. Innovative ap-
proaches, such as data-driven drift detection systems, may help
to warn of deteriorating model performance and the need for
updates [38, 39].

Strengths and limitations

One of the strengths of this unique study in the field of PKT is
its international collaboration and geographical diversity. The
inclusion of data from several European centers offers a com-
prehensive perspective on clinical practices. In addition, the
mandatory nature of the French and Dutch registries ensures
the representativeness of these cohorts. Another strength is the
comparison of country-specific models with an international
model and the use of both internal and external validation.

Furthermore, the recent study by Coens et al. [10] under-
scores the significance of time-varying parameters. This study
uniquely incorporates the influence of time on the relative haz-
ards of these parameters by modeling time and integrating time-
dependent variables.

Additionally, this study has direct clinical relevance, as these
prediction models can enhance communication with patients
and their families, as well as improve counseling on prognosis.
It also provides the basis for further research to standardize and

optimize clinical practice, with the possibility of developing a
more accurate generalized prediction model.

Study limitations

An important limitation of the study is the selection of European centers and the inclusion of patients with complete data only, as these may not be fully representative of the entire European population. Moreover, a potential selection bias could exist within the German cohort due to the partially retrospective nature of the CERTAIN registry. While the Dutch and French datasets were considered representative of their countries, the German data were sourced from the CERTAIN registry, where data collection was not compulsory. Consequently, data from German centers not participating in CERTAIN might be lacking. However, the primary aim of this study was not to provide a comprehensive validation across all of Europe, but rather to explore the potential applicability of the model in different countries.

Second, we categorized the data by country, assuming uni-
form implementation of protocols throughout the country. How-
ever, clinical care can vary considerably within a single country
due to regional differences and local practice. Unfortunately, in-
depth analysis of centers and other countries was not possible
due to limited sample size.

Third, the model could achieve better predictive performance
by including a larger number of parameters, such as more de-
tailed donor information or immunosuppressive regimen. How-
ever, the goal is to strike a balance between goodness of fit and
parsimony. Using a reduced number of variables improves both
the accuracy and the practicality of the model. The primary
objective was to develop an easy-to-use pre-transplant predic-
tion tool, rather than a prediction model that integrates post-
transplant parameters.

Fourth, the quality of a registry depends on data accuracy and completeness. A significant number of patients were excluded due to missing data, and the transition to adulthood might have resulted in loss to follow-up in the voluntary registry, potentially introducing bias.

Finally, as mentioned before, clinical practice and patient
populations may change over time, requiring regular updating of
the current model. Including the factor ‘era of transplantation’
could enhance the model’s accuracy, although it would add to
the number of parameters.

Future perspectives

The use of prediction models is increasing, driven by their poten-
tial to advance personalized medicine, individualized decision-
making, and risk stratification. The field of PKT has lacked a
prediction tool based solely on pre-transplant parameters. This
model could serve as a basic template for other countries to de-
velop their own nationally tailored models.

The results of this study show that there is considerable
variation in clinical practice between European centers with re-
gard to donor-recipient selection and transplant outcomes. At
present, the impact of this diversity on transplant outcomes re-
mains uncertain, and future research could assess whether cur-
rent care would benefit from a more standardized approach.
Should clinical practices in Europe become more standardized
and comparable in the future, there may be opportunities to de-
velop an international prediction model to benefit a wider range
of patients.

However, large data sets are needed to develop adequately
powered evidence-based guidelines. The establishment of
international registries such as CERTAIN, the European Society for Pediatric Nephrology (ESPN)/ERA registry, and ERN-specific registries are valuable steps in this direction [17, 19, 40, 41]. However, this study also highlights the importance of comprehensive and accurate data collection from a representative cohort, rather than fragmented data collection. Therefore, the impact of these registries could be further enhanced through increased harmonization and collaboration.

## CONCLUSION

This external validation study showed that the Dutch pre-transplant risk assessment tool for PKT cannot be readily applied in other countries without country-specific adaptations. Due to the current diversity of clinical practice in Europe, a country-specific model is preferable to a generalized international model.

## SUPPLEMENTARY DATA

Supplementary data are available at *Clinical Kidney Journal* online.

## ACKNOWLEDGEMENTS

We would like to thank Ben Pelzer for his valuable support and expertise in the field of statistics.

We acknowledge the Agence de la Biomédecine, the CERTAIN registry, and the NOTR for providing the data. We would also like to thank all the French kidney transplant teams who contributed to the collection of the data used in this paper. The funding of the CERTAIN registry by a grant from the Dietmar Hopp Foundation, the European Society for Pediatric Nephrology (ESPN), the German Society for Pediatric Nephrology (GPN), and by grants from the pharmaceutical companies Astellas and Novartis is gratefully acknowledged.

In addition, we would like to thank Giuseppina Sparta (Zurich), Jakub Zieg (Prague), Dominik Müller (Berlin), Gesa Schalk (Bonn), Lutz Weber (Cologne), Lars Pape (Essen), Matthias Hansen (Frankfurt), Martin Pohl (Freiburg), Jun Oh (Hamburg), Nele Kanzelmeyer (Hannover), Burkhard Tönshoff (Heidelberg), Christoph Hempel (Leipzig), Günter Klaus (Marburg), Henry Fehrenbach (Memmingen), Sabine König (Muenster), Staude Hagen (Rostock), Martin Bald (Stuttgart), Marcus Weitz (Tuebingen), Gema Ariceta (Barcelona), Anne-Laure Sellier-Leclerc (Lyon), Thomas Simon (Toulouse), Nikoleta Printza (Thessaloniki), Laszlo Berta (Budapest), Atif Awan (Dublin), Sara Testa (Milan), Elisa Benetti (Padua), Isabella Guzzo (Rome), Licia Peruzzi (Turin), Jacek Rubik (Warsaw), Michael Kaabak (Moscow), Alexandr Rumyantsev (Moscow Pirogov), Aytül Noyan (Adana), Fatos Yalcinkaya (Ankara), Ali Duzova (Hacettepe), Demet Alaygut (Izmir), Hulya Nalcacioglu (Samsun), David Milford (Birmingham), Stephen Marks (London), and Mohan Shenoy (Manchester) for their contribution to the CERTAIN registry. We would like to thank the ERN for Rare Urogenital Diseases and Complex Conditions (ERN eUROGEN) for their support.

## AUTHORS' CONTRIBUTIONS

LO: conceptualization, methodology, formal analysis, visualization, writing—original draft. L.W.: conceptualization, methodology, writing—review & editing. B.T., K.K., J.H., J.H., C.C., S.E., H.J., E.C., A.B., M.K.: resources, writing—review & editing. W.F.:

conceptualization, writing—review & editing, supervision. C.B.: conceptualization, methodology, writing—review & editing.

## CONFLICT OF INTEREST STATEMENT

None declared.

## DATA AVAILABILITY STATEMENT

The raw data supporting the findings of this study are not owned by the authors but are held by the respective providing registries. Interested parties may request access to the raw data directly from the relevant registries. Please contact the corresponding author for assistance in facilitating such requests. Additionally, all relevant statistics can be found in the supplementary files.

## REFERENCES

1. Van Arendonk KJ, Boyarsky BJ, Orandi BJ et al. National trends over 25 years in pediatric kidney transplant outcomes. *Pediatrics* 2014;133:594–601. https://doi.org/10.1542/peds.2013-2775

2. Verghese PS. Pediatric kidney transplantation: a historical review. *Pediatr Res* 2017;81:259–64. https://doi.org/10.1038/pr.2016.207

3. Sigurjonsdottir VK, Grimm PC. Living or deceased donor kidney transplantation in children. *Curr Opin Pediatr* 2019;31:232–36. https://doi.org/10.1097/mop.0000000000000740

4. Loubersac T, Roussey G, Dengu F et al. Comparison of the outcomes of the pediatric kidney transplantation between recipients below and above 15 kg: a single center retrospective study. *World J Urol* 2021;39:2789–94. https://doi.org/10.1007/s00345-020-03537-w

5. Anand A, Malik TH, Dunson J et al. Factors associated with long-term graft survival in pediatric kidney transplant recipients. *Pediatr Transplant* 2021;25:e13999. https://doi.org/10.1111/petr.13999

6. Chinnakotla S, Verghese P, Chavers B et al. Outcomes and risk factors for graft loss: lessons learned from 1,056 pediatric kidney transplants at the University of Minnesota. *J Am Coll Surg* 2017;224:473–86. https://doi.org/10.1016/j.jamcollsurg.2016.12.027

7. Oomen L, Bootsma-Robroeks C, Cornelissen E et al. Pearls and pitfalls in pediatric kidney transplantation after 5 decades. *Front Pediatr* 2022;10:856630. https://doi.org/10.3389/fped.2022.856630

8. Chandar J, Chen L, Defreitas M et al. Donor considerations in pediatric kidney transplantation. *Pediatr Nephrol* 2021;36:245–57. https://doi.org/10.1007/s00467-019-04362-z

9. Kim JJ, Fuggle SV, Marks SD. Does HLA matching matter in the modern era of renal transplantation? *Pediatr Nephrol* 2021;36:31–40. https://doi.org/10.1007/s00467-019-04393-6

10. Coens F, Knops N, Tieken I et al. Time-varying determinants of graft failure in pediatric kidney transplantation in Europe. *Clin J Am Soc Nephrol* 2024;19:345–54. https://doi.org/10.2215/cjn.0000000000000370

11. Riley S, Zhang Q, Tse WY et al. Using information available at the time of donor offer to predict kidney transplant survival outcomes: a systematic review of prediction models. *Transpl Int* 2022;35:10397. https://doi.org/10.3389/ti.2022.10397
12. Kaboré R, Ferrer L, Couchoud C et al. Dynamic prediction models for graft failure in paediatric kidney transplantation. Nephrol Dial Transplant 2021;36:927-35. https://doi.org/10.1093/ndt/gfaa180

13. Loupy A, Aubert O, Orandi BJ et al. Prediction system for risk of allograft loss in patients receiving kidney transplants: international derivation and validation study. BMJ 2019;366:l4923. https://doi.org/10.1136/bmj.l4923

14. Oomen L, de Jong H, Bouts AHM et al. A pre-transplantation risk assessment tool for graft survival in Dutch pediatric kidney recipients. Clin Kidney J 2023;16:1122-31. https://doi.org/10.1093/ckj/sfad057

15. Ramspek CL, Jager KJ, Dekker FW et al. External validation of prognostic models: what, why, how, when and where? Clin Kidney J 2021;14:49-58. https://doi.org/10.1093/ckj/sfaa188

16. Moons KG, Kengne AP, Grobbee DE et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012;98:691-8. https://doi.org/10.1136/heartjnl-2011-301247

17. Plotnicki L, Kohl CD, Höcker B et al. The CERTAIN Registry: a novel, web-based registry and research platform for pediatric renal transplantation in Europe. Transplant Proc 2013;45:1414-7. https://doi.org/10.1016/j.transproceed.2013.01.007

18. Strang WN, Tuppin P, Atinault A et al. The French organ transplant data system. Stud Health Technol Inform 2005; 116:77-82.

19. Oomen L, De Wall LL, Krupka K et al. The strengths and complexities of European registries concerning paediatric kidney transplantation health care. Front Pediatr 2023;11:1121282. https://doi.org/10.3389/fped.2023.1121282

20. CERTAIN. Cooperative European Paediatric Renal TransplantAnt INitiative registry 2023 [cited 2023 18-20-2023]. Available from: https://certain-registry.kikli.uni-heidelberg.de (1 April 2024, date last accessed).

21. Agence de la biomédecine. Organes—Greffe rénale pédiatrique. Available from: https://rams.agence-biomedecine.fr/greffe-renale-pediatrique-0 (3 February 2024, date last accessed).

22. Steyerberg EW, Vickers AJ, Cook NR et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38. https://doi.org/10.1097/EDE.0b013e3181c30fb2

23. Steele F. Multilevel discrete-time event history models with applications to the analysis of recurrent employment transitions. Aust NZ J Stat 2011;53:1-20. https://doi.org/10.1111/j.1467-842X.2011.00604.x

24. Lusa L, Ahlin Č. Restricted cubic splines for modelling periodic data. PLoS ONE 2020;15:e0241364. https://doi.org/10.1371/journal.pone.0241364

25. Harrel FE. Regression Modeling Strategies with Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Springers, editor: Heidelberg: Springer International Publishing Ag; 2015.

26. Su TL, Jaki T, Hickey GL et al. A review of statistical updating methods for clinical prediction models. Stat Methods Med Res 2018;27:185-97. https://doi.org/10.1177/09622802156 26466

27. Collins GS, Reitsma JB, Altman DG et al. Transparent reporting of a multivariable prediction model for individual

prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Medicine 2015;13:1. https://doi.org/10.1186/s12916-014-0241-z

28. IBM Corp. IBM SPSS Statistics for Windows, Version 27.0. Armonk, NY: IBM Corp; 2020.

29. Team RC. R: a Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2022.

30. Harambat J, van Stralen KJ, Schaefer F et al. Disparities in policies, practices and rates of pediatric kidney transplantation in Europe. Am J Transplant 2013;13:2066-74. https://doi.org/10.1111/ajt.12288

31. Oomen L, Bootsma-Robroeks CMHHT, Bouts AHM et al. Paediatric kidney transplantation in Europe, a clinical snapshot pilot. Front Pediatr 2024;12:1432027. https://doi.org/10.3389/fped.2024.1432027

32. Lemoine CP, Pozo ME, Superina RA. Overview of pediatric kidney transplantation. Semin Pediatr Surg 2022;31:151194. https://doi.org/10.1016/j.sempedsurg.2022.151194

33. Lentine KL, Pastan S, Mohan S et al. A roadmap for innovation to advance transplant access and outcomes: a position statement from the National Kidney Foundation. Am J Kidney Dis 2021;78:319-32. https://doi.org/10.1053/j.ajkd.2021.05.007

34. Murugapoopathy V, Gupta IR. A primer on congenital anomalies of the kidneys and urinary tracts (CAKUT). Clin J Am Soc Nephrol 2020;15:723-31. https://doi.org/10.2215/cjn.12581019

35. Lombel RM, Brakeman PR, Sack BS et al. Urologic considerations in pediatric chronic kidney disease. Adv Chronic Kidney Dis 2022;29:308-17. https://doi.org/10.1053/j.ackd.2022.02.006

36. Steyerberg EW, Borsboom GJ, van Houwelingen HC et al. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004;23:2567-86. https://doi.org/10.1002/sim.1844

37. Hogan J, Divard G, Aubert O et al. Validation of a prediction system for risk of kidney allograft failure in pediatric kidney transplant recipients: an international observational study. Am J Transplant 2023;23:1561-69. https://doi.org/10.1016/j.ajt.2023.07.004

38. Davis SE, Greevy RA, Lasko TA et al. Detection of calibration drift in clinical prediction models to inform model updating. J Biomed Inform 2020;112:103611. https://doi.org/10.1016/j.jbi.2020.103611

39. Davis SE, Greevy RA, Lasko TA et al. Comparison of prediction model performance updating protocols: using a data-driven testing procedure to guide updating. AMIA Annu Symp Proc 2019;2019:1002-10.

40. Bassanese G, Wlodkowski T, Servais A et al. The European Rare Kidney Disease Registry (ERKReg): objectives, design and initial results. Orphanet J Rare Dis 2021;16:251. https://doi.org/10.1186/s13023-021-01872-8

41. Tizard EJ, Verrina E, van Stralen KJ et al. Progress with the European Society for Paediatric Nephrology (ESPN)/ERA-EDTA Registry for children with established renal failure (ERF). Nephrol Dial Transplant 2009;24:2615-7. https://doi.org/10.1093/ndt/gfp275