# Development of Radiomic-Based Model to Predict Clinical Outcomes in Non-Small Cell Lung Cancer Patients Treated with Immunotherapy

Olena Tankyevych, Flora Trousset, Claire Latappy, Moran Berraho, Julien Dutilh, Jean Pierre Tasu, Corinne Lamour, Catherine Cheze Le Rest

<figure><img src="image_1.png" /></figure>

## To cite this version:

Olena Tankyevych, Flora Trousset, Claire Latappy, Moran Berraho, Julien Dutilh, et al., Development of Radiomic-Based Model to Predict Clinical Outcomes in Non-Small Cell Lung Cancer Patients Treated with Immunotherapy. Cancers, 2022, 14 (23), pp.5931. 10.3390/cancers14235931. hal-05248174

HAL Id: hal-05248174

https://hal.univ-brest.fr/hal-05248174v1

Submitted on 10 Sep 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# Development of Radiomic-Based Model to Predict Clinical Outcomes in Non-Small Cell Lung Cancer Patients Treated with Immunotherapy

Olena Tankyevych <fnref n="1" />,<fnref n="2" />,†, Flora Trousset <fnref n="1" />,†, Claire Latappy <fnref n="1" />, Moran Berraho <fnref n="1" />, Julien Dutilh <fnref n="3" />, Jean Pierre Tasu <fnref n="2" />,<fnref n="4" />,<fnref n="5" />, Corinne Lamour <fnref n="3" /> and Catherine Cheze Le Rest <fnref n="1" />,<fnref n="2" />,<fnref n="4" />,* [ID

<figure><img src="image_3.png" /></figure>

check for
updates

**Citation:** Tankyevych, O.; Trousset, F.; Latappy, C.; Berraho, M.; Dutilh, J.; Tasu, J.P.; Lamour, C.; Cheze Le Rest, C. Development of Radiomic-Based Model to Predict Clinical Outcomes in Non-Small Cell Lung Cancer Patients Treated with Immunotherapy. *Cancers* **2022**, *14*, 5931. https://doi.org/10.3390/cancers14235931

Academic Editors: Niels Reinmuth, Fortunato Ciardiello and Helmut H. Popper

Received: 4 August 2022

Accepted: 23 November 2022

Published: 30 November 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_4.png" /></figure>

**Copyright:** © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Simple Summary:** In locally advanced or metastatic non-small cell lung cancer (NSCLC), immunotherapy has become a standard as it can improve overall survival and progression-free survival. However, a durable clinical benefit (DCB) is only achieved in 20–50% of patients. Early identification of patients likely to benefit from this treatment is not only challenging but also crucial to avoid immune-related toxicities in patients unlikely to achieve DCB. The aim of our retrospective study was to assess the value of baseline and serial FDG-PET/CT radiomics for the prediction of response and survival in NSCLC patients undergoing immunotherapy. In a group of 83 patients, multimodality radiomics and delta-radiomics models provided added predictive value compared to conventional clinical parameters. Multimodality radiomics-based models developed using appropriate machine learning processes were able to predict progression, DCB, Overall Survival and Progression Free Survival with high confidence.

**Abstract:** Purpose: We aimed to assess the ability of radiomics features extracted from baseline (PET/CT0) and follow-up PET/CT scans, as well as their evolution (delta-radiomics), to predict clinical outcome (durable clinical benefit (DCB), progression, response to therapy, OS and PFS) in non-small cell lung cancer (NSCLC) patients treated with immunotherapy. Methods: 83 NSCLC patients treated with immunotherapy who underwent a baseline PET/CT were retrospectively included. Response was assessed at 6–8 weeks (PET/CT1) using PERCIST criteria and at 3 months with iPERCIST (PET/CT2) or RECIST 1.1 criteria using CT. The predictive performance of clinical parameters (CP), standard PET metrics (SUV, Metabolic Tumor volume, Total Lesion Glycolysis), delta-radiomics and PET and CT radiomics features extracted at baseline and during follow-up were studied. Seven multivariate models with different combinations of CP and radiomics were trained on a subset of patients (75%) using least absolute shrinkage, selection operator (LASSO) and random forest classification with 10-fold cross-validation to predict outcome. Model validation was performed on the remaining patients (25%). Overall and progression-free survival was also performed by Kaplan–Meier survival analysis. Results: Numerous radiomics and delta-radiomics parameters had a high individual predictive value of patient outcome with areas under receiver operating characteristics curves (AUCs) >0.80. Their performance was superior to that of CP and standard PET metrics. Several multivariate models were also promising, especially for the prediction of progression (AUCs of 1 and 0.96 for the training and testing subsets with the PET-CT model (PET/CT0)) or DCB (AUCs of 0.85 and 0.83 with the PET-CT-CP model (PET/CT0)). Conclusions: Delta-radiomics and radiomics features extracted from baseline and follow-up PET/CT images could predict outcome in NSCLC patients treated with immunotherapy and identify patients who would benefit from this new standard. These data reinforce the rationale for the use of advanced image analysis of PET/CT scans to further improve personalized treatment management in advanced NSCLC.

<footnote n="1">¹ Nuclear Medicine Department, Poitiers University Hospital, 86000 Poitiers, France</footnote>

<footnote n="2">² LaTIM, INSERM, UMR 1101, 29200 Brest, France</footnote>

<footnote n="3">³ Oncology Department, Poitiers University Hospital, 86000 Poitiers, France</footnote>

<footnote n="4">⁴ Medical School, University of Poitiers, 86000 Poitiers, France</footnote>

<footnote n="5">⁵ Radiology Department, Poitiers University Hospital, 86000 Poitiers, France</footnote>

<footnote>* Correspondence: catherine.cheze-le-rest@chu-poitiers.fr</footnote>

<footnote>† These authors contributed equally to this work.</footnote>
**Keywords:** PET/CT; radiomics; NSCLC; immunotherapy; response to therapy; durable clinical benefit; survival

# 1. Introduction

Lung cancer is the leading cause of cancer-related death worldwide [1]. Non-small cell lung cancer (NSCLC) accounts for 80-90% of primary lung cancers, and is mostly diagnosed at an advanced stage with prognosis remaining poor despite recent therapeutic advances [2]. The introduction of immunotherapy for the treatment of locally advanced or metastatic non-small cell lung cancer (NSCLC) has shown an improvement in terms of overall survival and progression-free survival. However, a durable clinical benefit (DCB) (>6 months) is only achieved in 20-50% of patients [3]. Early identification of patients likely to benefit from this treatment is therefore crucial. In addition, newly described response patterns (pseudo-progression, hyper-progression) make response assessment even more challenging [4]. Therefore, robust response-predictive biomarkers at baseline are crucial to avoid immune-related toxicities in patients unlikely to achieve DCB.

PD-L1 status has been an important element in treatment decision making until now. However, approximately 10% of patients whose tumors do not express PD-L1 respond to treatment, and response is also not certain in cases of high expression of this biomarker [3,5-8]. Alternative biomarker identification is an active research domain in which medical imaging plays an increasingly important role.

18F-FDG PET/CT is commonly used for characterization and staging of lung cancers. Medical images are easy to repeat over time, non-invasive and promising for personalized patient management by reflecting tumor heterogeneity, which seems to be a major cause of the disparity between response and prognosis. However, for such applications, quantitative analysis is required.

Radiomics is an approach that allows a finer characterization of tumor lesions by extracting numerous quantitative parameters from medical images (CT, PET/CT and MRI, for example) [9]. These features can be used to build complex mathematical models for lesion characterization (benign versus malignant lesions [7,8,10-12], histology [13] and for the prediction of patient outcome [14,15]. In the majority of these studies, hand-crafted features extracted from the segmented functional or fused image tumor volumes have been considered. More recently, some studies have also considered the use of deep learning models in order to directly classify lung cancer patients in terms of overall survival without explicitly extracting radiomics features [16]. All these studies have shown some promising results, but in both cases, there are no specific investigations addressing the use of PET/CT radiomics for patients undergoing immunotherapy treatment.

The changes in these features over time (delta-radiomics) have also been suggested to obtain a more accurate evaluation of tumor response [17]. While single time-point radiomics and delta-radiomics showed promising results in helping clinicians to choose the most appropriate treatment in various cancer models, there are still only limited data on the potential value of using these features in the evaluation of NSCLC patients undergoing immunotherapy using CT [18]. Mu et al. were recently the first to test a multiparametric radiomic model combining baseline PET and CT features to predict tumor response and survival [19]. Using advanced statistical analyses, they reported more heterogeneous tumors to have a greater probability of reaching DCB, but they did not investigate the potential additional value of serial examinations.

In this study we aim to assess the ability of radiomics features extracted from baseline and follow-up 18F-FDG PET/CT scans, as well as their changes during treatment, to predict response to immunotherapy, DCB, overall survival and progression-free survival.
## 2. Materials and Methods

### 2.1. *Patient Population*

In this retrospective study, data were retrieved from a total of 83 patients treated at Poitiers University Hospital (France) between September 2016 and December 2020. Inclusion criteria were as follows: Eastern Cooperative Oncology Group (ECOG) performance status (PS) ≤ 2, histologically proven NSCLC treated with anti-PD-L1 immunotherapy (nivolumab, pembrolizumab or atezolizumab, as monotherapy or associated with chemotherapy) and available baseline 18F-FDG PET/CT (PET/CT0) performed within 3 months before the initiation of immunotherapy.

Treatment was chosen depending on histology, PD-L1 expression, ECOG PS and risk factors and approved by the multidisciplinary oncological board. Immunotherapy was introduced as a first-line therapy in 23 patients (28%). Ten patients had PD-L1 expression higher than 50% and received pembrolizumab alone (n = 8) or combined with chemotherapy (n = 2). Thirteen patients were treated with pembrolizumab associated with platinum-based chemotherapy (cisplatin or carboplatin) and pemetrexed. Two patients received atezolizumab, either alone (n = 1) or associated with carboplatin and etoposide (PD-L1 expression not performed). Sixty patients (72%) received immunotherapy for recurrence or failure after at least one line of chemotherapy. Seventeen of them had PD-L1 expression ≥ 1% and received pembrolizumab, 41 received nivolumab and 3 received atezolizumab. Five patients were treated with a combination of pembrolizumab, carboplatin and pemetrexed. Immunotherapy administration followed the recommendations for each molecule at the time of treatment.

Baseline clinical, demographic and biological data were retrieved from medical records (age, sex, performance status, smoking history, histological subtype, stage (8th TNM classification of the International Association for the Study of Lung Cancer [20]), presence of brain metastasis or not, previous treatments and PD-L1 status (if available).

All patients gave their informed consent for the use of their personal and clinical data. No ethical committee approval was required, given the retrospective nature of this study of previously anonymized data.

### 2.2. *Image Acquisition, Segmentation, Pre-Processing and Feature Extraction*

Eighteen F-FDG PET/CT scans were performed following EANM guidelines. Acquisition parameters are described in Supplementary Materials. The largest lung lesion was semi-automatically segmented using the Fuzzy Locally Adaptive Bayesian (FLAB) algorithm [20], and the obtained volume of interest was adjusted manually by an experienced clinician, if needed. The performance of this algorithm has been extensively evaluated for functional tumor segmentation, demonstrating high reproducibility and robustness for different cancer models [21–23]. Tumor segmentation on CT images was performed by applying the PET VOI on CT images using 3D Slicer [24].

Prior to feature extraction, pre-processing operations were applied on PET and CT images, as described in Supplementary Materials, to take into account the variability of image acquisition (PET/CT scanners, acquisition protocols).

Metabolic and conventional volumetric parameters were extracted from CT and metabolic VOIs (including SUVmax, SUVmin, SUVmean, Metabolic Tumor Volume (MTV) and Total Lesion Glycolysis (TLG)). In addition, a total of 2430 radiomic features were extracted from both CT and PET VOIs using Pyradiomics [25], 107 were derived from original images and 2323 were derived from preprocessed images (using different combinations of interpolation, resampling and filtering), including: 2D and 3D shape parameters, first-order (histogram-based characteristics) and second-order (texture parameters) features. Detailed parameter definitions are available at the Pyradiomics website, and the list is presented in Table S1 [25].
## 2.3. *Patient Outcomes*

### 2.3.1. Follow-Up and Response Assessment

Patient follow-up was based on clinical examination every 12 weeks until disease progression or death occurred. Metabolic response was assessed using PERCIST criteria on follow-up PET/CT performed 6 to 8 weeks (PET/CT1) after treatment initiation [26]. Response was defined as: complete metabolic response (CMR), partial metabolic response (PMR), stable metabolic disease (SMD) or progressive metabolic disease (PMD), considering the most hypermetabolic lesion on each scan.

In cases of new lesions on PET/CT1 without clinical worsening, the response was qualified as unconfirmed progression (UPMD). Then, discrimination between true progressive disease and pseudo-progression was carried out using a second follow-up PET/CT (PET/CT2) performed after an additional month of treatment [27].

Response was also determined on follow-up CT performed at 8 to 12 weeks according to Response Evaluation Criteria in Solid Tumors version 1.1 (RECIST) [28]. When patients had no follow-up PET/CT, response was only assessed using RECIST. Patients were classified as responders if response was defined as either complete response (CR), partial response (PR) or stable disease (SD). Patients with progressive disease were considered as non-responders. Patients were treated until clinical worsening, confirmed progression or unacceptable toxicity.

### 2.3.2. Survival and Durable Clinical Benefit (DCB)

Overall survival (OS) was calculated as the time from initiation of immunotherapy to death, censored at the date of last follow-up for survivors. Progression-free survival (PFS) was determined as the time from treatment initiation to disease progression or death, censored at the date of last follow-up for survivors without progression. Durable clinical benefit (DCB) to immunotherapy was defined as alive and without disease progression at 6 months.

## 2.4. *Statistical Analysis*

We evaluated the discriminative power of clinical parameters (age, sex, smoking, ECOG PS, histology, PDL1, stage, previous treatment, immunotherapy line and molecule, tumor response) to predict the studied endpoints (OS, PFS, response, DCB) using chi-square and Mann-Whitney tests for quantitative and qualitative variables, respectively, with *p*-values < 0.05 considered to be statistically significant.

The Kaplan-Meier analyses with log-rank tests were performed for PFS and OS (Tables S5-S9).

To avoid overfitting, we discarded highly correlated features using Spearman's rank correlation, followed by a feature selection using Least Absolute Shrinkage and Selection Operator (LASSO) [29]. Area under the receiver operating characteristic curve (AUC) analysis, estimated using univariate logistic regression with Bootstrap, was then performed to evaluate the predictive value of each parameter (clinical parameters (CP), standard PET metrics and radiomics features from baseline and follow-up PET/CT) for the different endpoints.

In the multivariate analysis, 7 models (CP, PET, CT, PET-CP, CT-CP, PET-CT-CP) were built with the most promising selected features using the Random Forest classification method, with 10-fold cross validation repeated 10 times on the training set (75% of the initial data) for hyperparameters tuning with randomized patients split (100 iterations for sample bias correction). The number of variables selected was part of the model tuning process. Number of features varied between 3 and 8, with an increment of 1 for each. So, for each model, this number was unique, selected upon the best training results. Subsequently, the selected model (set of hyperparameters and selected features) was applied on the test sample (25% of the initial data). The performance of each model in predicting outcome was measured by AUC analysis.

The delta-radiomics features were defined as the relative net change between two images: relative Net Change = (FeatureT1 - FeatureT0)/FeatureT0. Here, FeatureT0
was the value of a feature before the treatment and FeatureT1 was the value of the feature at the 2-month evaluation of the treatment. The delta-radiomics predictive power was also tested using the same methodology. All the analyses were performed with standard Python modules, including Sklearn, pandas and Matplotlib.

## 3. Results

### 3.1. *Patients Characteristics*

Clinical and demographic characteristics of the 83 patients enrolled in this study are summarized in Table 1. No patient was lost at follow-up.

<table><caption>**Table 1.** Clinical and demographic characteristics of overall population (83 patients) and correlation of these parameters with iPERCIST response (evaluated in 71 patients) (chi-square and Mann-Whitney tests).</caption><thead><tr><th>Characteristic</th><th>Overall Population (n = 83)</th><th>Responders (n = 40)</th><th>Non-Responders (n = 31)</th><th>p Value</th></tr></thead><tbody><tr><td>Age (years), median (range)</td><td>63.5 (38–85)</td><td>61.5 (47–82)</td><td>65 (43–85)</td><td>0.28</td></tr><tr><td>Sex, n (%)</td><td></td><td></td><td></td><td>0.78</td></tr><tr><td>Men</td><td>59 (71)</td><td>31 (77)</td><td>22 (71)</td><td></td></tr><tr><td>Women</td><td>24 (29)</td><td>9 (23)</td><td>9 (29)</td><td></td></tr><tr><td>Smoking, n (%)</td><td></td><td></td><td></td><td>0.09</td></tr><tr><td>Smoker</td><td>79 (95)</td><td>40 (100)</td><td>29 (94)</td><td></td></tr><tr><td>Non-smoker</td><td>4 (5)</td><td>0</td><td>2 (6)</td><td></td></tr><tr><td>ECOG PS, n (%)</td><td></td><td></td><td></td><td>0.3</td></tr><tr><td>0</td><td>35 (42)</td><td>19 (47,5)</td><td>12 (39)</td><td></td></tr><tr><td>1</td><td>44 (53)</td><td>19 (47,5)</td><td>18 (58)</td><td></td></tr><tr><td>≥2</td><td>4 (5)</td><td>2 (5)</td><td>1 (3)</td><td></td></tr><tr><td>Histology, n (%)</td><td></td><td></td><td></td><td>0.13</td></tr><tr><td>Adenocarcinoma</td><td>48 (58)</td><td>26 (65)</td><td>15</td><td></td></tr><tr><td>Squamous cell carcinoma</td><td>22 (26)</td><td>8 (20)</td><td>11</td><td></td></tr><tr><td>Other type</td><td>13 (16)</td><td>6 (15)</td><td>5</td><td></td></tr><tr><td>PD-L1, n (%)</td><td></td><td></td><td></td><td>-</td></tr><tr><td>&lt;1%</td><td>20 (24)</td><td>12 (30)</td><td>9 (29)</td><td></td></tr><tr><td>1–49%</td><td>18 (22)</td><td>8 (20)</td><td>2 (6)</td><td></td></tr><tr><td>≥50%</td><td>15 (18)</td><td>11 (28)</td><td>3 (10)</td><td></td></tr><tr><td>Not performed</td><td>30 (36)</td><td>9 (22)</td><td>17 (55)</td><td></td></tr><tr><td>Stage before immunotherapy, n (%)</td><td></td><td></td><td></td><td>0.27</td></tr><tr><td>IIB</td><td>1 (1)</td><td>1 (2)</td><td>0</td><td></td></tr><tr><td>III</td><td>11 (13)</td><td>6 (15)</td><td>3 (10)</td><td></td></tr><tr><td>IV</td><td>71 (86)</td><td>33 (83)</td><td>28 (90)</td><td></td></tr><tr><td>Previous treatment, n (%)</td><td></td><td></td><td></td><td></td></tr><tr><td>Surgery</td><td>19 (23)</td><td>11 (28)</td><td>7 (23)</td><td></td></tr><tr><td>Radiotherapy</td><td>14 (17)</td><td>10 (25)</td><td>10 (32)</td><td></td></tr><tr><td>Chemotherapy</td><td>58 (70)</td><td>22 (55)</td><td>26 (84)</td><td></td></tr><tr><td>Immunotherapy, n (%)</td><td></td><td></td><td></td><td></td></tr><tr><td>First line</td><td>23 (28)</td><td>24 (60)</td><td>26 (84)</td><td></td></tr><tr><td>Second or &gt;line</td><td>60 (72)</td><td>16 (40)</td><td>5 (16)</td><td></td></tr><tr><td>Nivolumab</td><td>41 (49)</td><td>13 (33)</td><td>24 (78)</td><td></td></tr><tr><td>Pembrolizumab</td><td>37 (45)</td><td>26 (65)</td><td>6 (19)</td><td></td></tr><tr><td>Atezolizumab</td><td>5 (6)</td><td>1 (2)</td><td>1 (3)</td><td></td></tr><tr><td>PFS, median (days)</td><td>156</td><td>250</td><td>52</td><td>&lt;0.001</td></tr><tr><td>DCB, n (%)</td><td>35 (42)</td><td>25 (63)</td><td>3 (10)</td><td>&lt;0.001</td></tr><tr><td>OS, median (days)</td><td>256</td><td>351</td><td>194</td><td>&lt;0.001</td></tr></tbody></table>

ECOG PS: Eastern Cooperative Oncology Group performance status; OS: overall survival; PFS: progression-free survival.

Clinical parameters were not significantly different in responders and non-responders. Considering demographic data, ECOG performance status (PS) was associated with significant differences in OS and PFS. PFS and OS were significantly shorter in patients with PS ≥ 1 (p = 0.0005 and p = 0.003, respectively). PS and stage before immunotherapy were also significantly lower in patients who obtained DCB (p = 0.00009 and p = 0.03, respectively) (Table S3).

Survival tended to be longer in patients more than 63 years old (Figure S1A).

### 3.2. *Patient Outcomes*

The median follow-up was 865 days (931 ± 569, range 98–1759 days). At the last follow-up, 40 patients (48%) were still alive and 18 (22%) were still receiving immunotherapy. Nine patients (11%) experienced immunotherapy-induced adverse effects (five with thyroiditis,
one with rising liver enzymes, one with pneumonitis, one with Raynaud's syndrome and one with sensitive neuropathy).

Immunotherapy was stopped in 65 patients for various reasons: progression or clinical worsening in 49 patients (75%); unacceptable adverse event in four patients (6%); introduction of a high dose of corticosteroids (because of brain metastasis with edema) in four patients (6%); second cancer requiring the introduction of another chemotherapy in one patient (2%); 2 years of clinical benefit with immunotherapy in seven patients (11%).

Metabolic response was studied using PET/CT1 in 71 patients (85%), and 34 of them underwent a second PET/CT one month later (PET/CT2). According to iPERCIST criteria, metabolic response after 3 months of treatment was as follows: CMR in three patients (4%), PMR in 29 patients (41%), SMD in eight patients (11%) and PMD in 31 patients (44%). Responder rate was 56% (40 patients). Out of the 40 responders, four were first assessed as PMD based on PET/CT1 and reclassified as either PMR (n = 3) or SMD (n = 1) using PET/CT2. In these cases of pseudo-progression, the patients ultimately obtained a durable clinical benefit (DCB). Response was evaluated using only RECIST 1.1 in 12 patients (PET/CT1 was not performed) and was as follows: 5 PR, 4 SD and 3 PD. A total of 49 patients were considered as responders (59%) and 34 as non-responders (41%) after three months of therapy. DCB was then obtained in 35 patients (42%).

Outcome was more favorable for responders than non-responders, with a longer median PFS (250 days vs. 52, *p* <0.001) and OS (351 days vs. 194 *p* <0.001) (Table 1).

### 3.3. Radiomics and Prediction of Progression, DCB and Survival

#### 3.3.1. Univariate Analysis

Best results of radiomics features extracted from the largest lung lesion at baseline (PET/CT0) and 2 months (PET/CT1) after treatment initiation are presented in Tables S2–S4 and Figures S1 and S2. Predictive value of parameters derived from follow-up PET/CT scans performed at month 3 will not be described in this paper as the sample size (34 patients) was too small to obtain reliable results. The performance of radiomics parameters in predicting response at month 2 (PET/CT using PERCIST) and month 3 (using iPERCIST for PET/CT and RECIST for CT) were studied but will not be detailed further in the following sections, given that they were similar to those predicting other primary endpoints. Therefore, only progression, DCB and survival results are considered in the results below.

**At Baseline (PET/CT0)**

Standard PET-based features (SUVs, MTV, TLG) were not able to significantly discriminate between patients in terms of progression, DCB or survival (*p* = 0.44). Predictive performance of clinical parameters and standard PET/CT metrics were lower. The predictive power of ECOG PS and age were moderate with AUC of 0.70 for DCB and OS. AUC for volume and TLG did not exceed 0.67 to predict progression. SUV and all the other clinical features had low predictive power.

Advanced image analysis provided better results. For example, longer PFS and OS were observed in cases of high PET0_GLCM_Clustershade (*p* = 0.0007), low CT0_kurtosis (*p* = 0.001) and low CT0_GLSZM_GLNUN (*p* = 0.00003).

Moreover, PET- and CT-based radiomics features were able to predict outcomes with AUC up to 0.85 for progression and 0.80 for OS. Texture-based features had better performance than first-order and shape parameters. The highest predictive power was obtained with texture-based parameters representing regional or local heterogeneity characterization (such as CT0_GRLM_LRLGLE (AUC 0.85) and PET0_GLCM_lmc1 (AUC 0.85) for progression, CT0_GLDM_SDE (AUC 0.80) and CT0_GLSZM_ZonePercentage (AUC 0.80) for OS). Radiomics predictive power was moderate for PFS and DCB (AUC up to 0.74 for the best parameters).
### At Month 2 (PET/CT1)

OS and PFS were significantly longer in patients with more homogeneous tumors, as demonstrated by low GLCM-Imc1 (p = 0.00003, HR = 5.99, for TEP1), low kurtosis (p = 0.00007, HR = 3.79 for CT1), low busyness (p = 0.0008, HR = 2.94 for CT1), high GLCM-correlation (p = 0.0002, HR = 0.30 for CT1) and high NGTDM-coarseness (p = 0.004, HR = 0.37 for CT1). OS and PFS were also significantly longer in patients with more regular tumors (high Surface-Volume Ratio, p = 0.0004 for CT1; high sphericity, p = 0.000006, for PET1).

Performance of clinical parameters and standard PET/CT metrics to predict outcome remained low, similarly to those at baseline. A higher number of radiomics features demonstrated a high predictive value (as demonstrated by an AUC ≥ 0.80), especially for progression, with AUC up to 0.91 (Variance for CT1) and 0.88 (PET1 Maxim um), for example. After 2 months of treatment and unlike at baseline, first-order parameters (extracted from filtered images) managed to perform equally to shape-based and some more complex texture-based features. Radiomics overall predictive performances at month 2 were higher than at baseline, except for the prediction of DCB.

### Delta-Radiomics

Baseline and follow-up PET/CT studies were used to determine radiomics features variation during treatment (delta-radiomics). None of these features achieved higher predictive performance than parameters at month 2 (PET/CT1). Increasing size of the lesion (as reflected by the least axis length (p = 0.00006) and mesh volume (p = 0.0008)) was associated with non-response. We also observed that non-responder tumors tended to lose the roundness of their shape, with a decrease in sphericity (p = 0.1) and flatness (p = 0.16), but these changes were not statistically significant.

DeltaPET parameters demonstrated an equally high predictive power for PFS and OS (AUC = 0.81 for deltaPET_GLDM_LGLE and AUC = 0.83 for deltaPET GLRLM_RUNEntropy, respectively) compared with PET/CT1 features (AUC = 0.83 with PET1_GLCM_ClusterTendency, AUC = 0.84 with CT1_GLCM_Imc2 for PFS, and AUC = 0.82 for PET1_GRLRM_SRLGLE for OS). DeltaCT features had a slightly lower predictive power for OS compared with PET/CT1 features (AUC = 0.73 for deltaCT_GRLRM_LRLGLE and AUC = 0.81 for CT1_GLSZM_SZNU). The performance of delta-radiomics features were better than baseline parameters in predicting PFS (AUC of 0.84 for deltaCT, 0.81 for deltaPET and 0.74 and 0.73 for TEP0 and CT0, respectively). They did not perform better than single time-point features for DCB (AUC of 0.77 and 0.73 for deltaPET and deltaCT and AUC of 0.74 and 0.75 for PET0 and CT0, respectively) or for progression (AUC = 0.82 for deltaCT_flatness and 0.74 for deltaPET_GLCM_Correlation).

### 3.3.2. Multivariate Analysis

Seven multivariate prediction models were composed of a variable number of selected features (from 3 to 18): clinical parameters alone (CP), PET- and CT-derived parameters alone or combined with CP. The results are presented in Figure 1 (training) and Figure 2 (testing). The radiomics-based models consisted only of single-time point features (TEP/CT0 or TEP/CT1), given that delta-radiomics did not show higher performances. The metabolic response determined using PERCIST on PET/CT1 was integrated into the models established at month 2. The performances in predicting response to immunotherapy (assessed with PERCIST, iPERCIST and RECIST) of each model were also investigated and were similar to those of other outcomes (DCB, progression, survival) and therefore will not be further detailed.
<figure><img src="image_5.png" /><figcaption>**Figure 1.** Performances of seven multi-parametric models for the prediction of outcome using clinical data (CP) and radiomics features derived from PET and CT at baseline (A) and at month 2 (B) in the training data set, with their standard deviations integrated. OS: overall survival; PFS: progression-free survival, DCB: durable clinical benefit.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Figure 2.** Performances of seven multi-parametric models for the prediction of outcome using clinical data (CP) and radiomics features derived from PET and CT at baseline (A) and at month 2 (B) in the testing dataset.</figcaption></figure>
### At Baseline (PET/CT0)

Clinical parameters model showed low predictive value for progression, DCB, PFS and OS (AUC of 0.64, 0.40, 0.81, 0.5, and 0.67, 0.60, 0.71, 0.63 in the training and the testing subset, respectively) (Figures 1A and 2A). Among available clinical parameters, only ECOG PS was selected for DCB and PFS and only age for OS in the process of model building. Models including radiomics features had a high predictive value outcome with AUCs over 0.80, except for the CT-CP model (AUC of 0.75 for the prediction of DCB in the testing subset). The PET radiomics model outperformed the CT model in predicting DCB, progression and PFS. Combining PET and CT features further improved performance for the prediction of DCB, progression and OS, compared to PET or CT features-only models. The most contributing radiomic features to the models were first-order parameters extracted from filtered CT images and texture features extracted from PET and CT images (such as skewness, median, NGTDM_Complexity, GLCM_Autocorrelation and GLCM_imc1). During the building process, AUC varied greatly for some models, with standard deviation up to 15% for the PET-CT-CP model predicting PFS (Figure 1). Highest stability (SD = 3%) was obtained predicting progression using PET-CT (AUC of 0.96 and 1 for the training and validation subsets, respectively), DCB using PET (AUC of 0.82 and 1) and PET-CP (AUC of 0.78 and 1), PFS using PET-CT (AUC of 0.65 and 1) and PET-CT-CP (AUC of 0.74 and 1) and OS using PET-CP (AUC of 0.83 and 1).

### At Month 2 (PET/CT1)

We observed a similar trend compared with the results obtained at baseline (Figures 1B and 2B). The CP model was less efficient than models including radiomics features. There was an overall increase of prediction performances for most of the multivariate models for progression, PFS and OS, in addition to more consistent results between the training and testing subsets, especially for progression and PFS. The models that appeared to be the most promising for the different outcomes were: PET-PC (AUC of 0.91 and 1 for the training and testing subsets, respectively) and PET-CT (AUC of 0.84 and 1) for DCB; PET (AUC of 0.97 and 1), CT (AUC of 0.74 and 1), PET-PC (AUC of 1 and 1) and PET-CT (AUC of 0.96 and 1) for progression; PET for PFS (AUC of 0.96 and 0.96), and PET-CT for OS (AUC of 0.84 and 0.89).

The most contributing features to the models were textural features, first-order parameters extracted from filtered images (PET and CT), age, ECOG PS and response assessed with PERCIST on PET/CT.

We have performed an analysis of the radiomics quality, according to which our study scores19 points (out of 36, Table S10) [30].

## 4. Discussion

The action of anti-PD-1 and anti-PD-L1 antibodies differs from conventional chemotherapies and has induced new evolution patterns, making imaging-based response assessment more challenging. Response may be delayed, and early tumor infiltration by immune cells may induce an initial increase in lesion size. In order to take into account these atypical patterns, we chose to evaluate patients according to iPERCIST criteria, as proposed by Goldfarb et al. in 2019 [27]. We identified four patients (5%) with pseudo-progressions, which is consistent with previous studies in relation to a temporary increase in tumor burden to transient immune-cell infiltrate followed by tumor regression [31]. All of these patients subsequently obtained a lasting clinical benefit. This is again in agreement with previous studies reporting the occurrence of pseudo-progression being a good prognostic factor [32–34]. It is therefore important to be able to identify these response patterns to avoid stopping a potentially effective treatment too early.

We investigated the ability of clinical parameters and standard baseline PET/CT metrics (SUV, MTV, TLG) to discriminate between patients in terms of response (according to PERCIST at the first restaging, iPERCIST and RECIST1.1 after 3 months of treatment), DCB and survival. Low ECOG PS (PS = 0) and tumor stage (stage III vs. IV) were
associated with DCB. These results are consistent with those of Seban et al. [35] and Nardone et al. [36]. In addition, OS was longer in patients with smoking history, in agreement with immunotherapy having been found to have a greater benefit in NSCLC patients with a smoking history than in those who had never smoked [37]. None of the considered clinical parameters was able to discriminate responders from non-responders.

Some authors previously reported a relationship between large MTV, high TLG and progression during immunotherapy, suggesting a potential predictive value of these parameters to predict response and survival [38,39]. In our study, these easily calculated features were not statistically different according to response or outcome, and therefore they did not reach a high level of predictive power, probably because of a narrower distribution of their values in our selected population.

On the contrary, a large number of PET and CT radiomics features were correlated with therapeutic response, DCB and survival and had a high predictive value (AUC ≥ 0.80). The best-performing parameters at baseline were texture parameters. First-order and shape features also demonstrated a high predictive value, especially at the first restaging (PET1).

Overall, PET and CT parameters extracted from PET/CT1 were greater predictors than those at baseline (with a gain ranging from 5 to 10% for the AUC of the best parameters for progression and PFS). Moreover, features at 4 months (PET2) showed AUCs up to 1 for the prediction of DCB and OS. This gain in predictive power should be taken with caution as there were fewer patients in the group studied at month 2 (71 segmented tumors) and month 3 (n = 34) compared with baseline, which could result in a higher risk of overfitting.

OS and PFS were significantly longer in patients with more homogeneous tumors at baseline and at the first restaging. These results are in agreement with previous studies [36,38,40]. Nardone et al. [36] reported that high entropy related to tumor aggressiveness in the literature [41] and low correlation were both associated with poor prognosis. Ahn et al. also found that contrast and busyness were able to predict recurrence in surgically treated patients [42]. A low value of coarseness was an independent prognostic biomarker associated with a high risk of recurrence in our study. This finding is in contrast with Mu et al. [19] reporting patients with more heterogeneous tumors to have a higher probability of obtaining DCB.

In terms of shape, Mu et al. reported that patients with more convex tumors had a greater chance to achieve DCB [19]. Indeed, sphericity is thought to reflect PD-L1 expression and thus potential response to treatment, as PD-L1 cells could form rounder lesions according to Saeed-vafa et al. [43]. In our study, tumor shape also appeared to be significant for outcome, with longer PFS and OS in patients with more rounded but spiculated lesions on the first follow-up PET/CT, reflected by high sphericity and high Surface Volume Ratio, respectively. An increasing size of the lesion during the first 2 months was associated with non-response, as expected. Regarding tumor boundaries, Dercle et al. demonstrated that early regularization of tumor margins was strongly associated with response to nivolumab and gefitinib [44]. Complex tumor boundaries are indeed conventionally associated with tumor aggressiveness and worse outcome [45,46]. In our study, while non-responder tumors tended to lose the roundness of their shape, an increase in the smoothness of tumor contours (decrease in surface-to-volume ratio) was surprisingly associated with non-response. This atypical finding could result from the sensitivity of shape features to segmentation and respiratory motion. Deep learning used for feature extraction without the need for an intermediate segmentation step may be useful in the future to overcome such limitations [47].

Using PET-based multivariate modeling was more efficient in predicting DCB, progression and PFS, while CT remained a better predictor of OS. Those results underlined the complementarity of both imaging techniques. This was confirmed by an improvement of combined PET/CT models in predicting DCB, progression and OS. Unsurprisingly, the addition of low-performing clinical parameters did not further improvethe outcome prediction. It has been shown in prior studies that combined multi-modality models could perform better than single-modality approaches (clinical, radiomics or genomic data) for outcome prediction [48]. Overall, all multimodality models including radiomics features in
the testing subset achieved a high predictive power (AUC ≥ 0.80) for all outcomes. The most significant radiomics features included in the models were texture features extracted from PET and CT images and first-order parameters extracted from filtered CT images. The results of the models whose performance varied the most between the training and testing subsets should be taken with caution. The models that appeared the most promising were therefore PET-CT-CP for DCB, PET-CT for progression and PET for PFS. The lowest variability of performance estimates (AUCs) was obtained for the prediction of progression, which is consistent with the highest predictive performance during the univariate analysis. The performances of these models are similar to those obtained by Mu et al. [19] in developing a radiomic signature to predict DCB in NSCLC patients treated with immunotherapy. They combined radiomics features extracted from PET, CT images and the fused PET and CT images (KLD), ECOG PS and histology with an AUC of 0.89 for the training dataset and AUCs of 0.86 in both the retrospective and prospective tests cohorts.

Our study had some limitations, including its retrospective design and the small sample size. The interval of time between baseline PET/CT and the introduction of immunotherapy in patients treated in our institution was variable, and we chose to only include patients with a maximum delay of 3 months (from a few days to 3 months maximum) to take into account the natural history of lung cancer and avoid non-representative baseline PET/CT results in a rapidly evolving pathology. This meaningful choice reduced our inclusion potential. However, in this study we applied a highly appropriate methodology to overcome the risk of overfitting, considering the large number of features extracted and the small size of the population. Indeed, to address this issue, we selected features using two steps. First, over-correlated parameters using Spearman rank coefficients were discarded. Then, the LASSO algorithm was used to select the most promising parameters as it is known to be a robust statistical method for data reduction [49]. In addition, we performed a cross-validation to build our models, which has been shown to be an efficient method to provide an unbiased estimation on survival prediction [50]. This rigorous methodology allowed us to consider a very large number of parameters to start with, and to avoid missing potentially important ones. Radiomics features extraction and analysis is complex and involves many steps that can affect the reproducibility of studies based on this approach [51]. After acquisition, segmentation is the first critical step for which we used a valid and robust method (FLAB) on PET images. Using a low-dose CT performed for PET attenuation correction, tumor segmentation on CT images was facilitated by automated alignment of the datasets from the acquisition system (PET/CT). Various filtering and quantization were applied to original datasets since no pre-processing consensus has yet been achieved, despite harmonization efforts from the scientific community. We partly addressed the issue of reproducibility and validation following image biomarker standardization and consensus-based definitions and guidelines for generating radiomics [52].

In this study, the median predicted value was used as a cut-off point to dichotomize high- and low-risk patients. It is likely that different values would yield different results. However, testing multiple cutoffs to find the best one without an independent validation dataset in which to test it has been repeatedly shown to yield overly optimistic results [49,50]. By using the median, this source of bias is avoided, and the conclusions remain conservative.

The fact that patient recruitment was carried out over several years (2016 to 2020) explained the heterogeneity of our population reflecting the gradual appearance of the various immunotherapy molecules and their combination with chemotherapy. Rapidly evolving practice resulted in immunotherapy being introduced in our population either in the first or second line with different treatment regimens. This resulting population heterogeneity is representative of that observed in clinical routine. The predictive value of PD-L1 status could not be investigated since it was not available for a third of our population, given that it was not necessarily requested for immunotherapy prescription, especially before 2019.

Only 34 patients had a second follow-up PET/CT scan, given that this exam is not systematically recommended in routine for the follow-up of lung cancers. As demonstrated,
it was found useful to better characterize initial progression, but it may be interesting to also suggest its use in a larger study based on our promising results for the prediction of DCB and OS.

Validation is always a crucial point in radiomics studies. To validate our predictive models, we divided our population into training and validation datasets. This choice necessarily led to a small-size validation group in which individual variability could explain slightly lower performance.

**5. Conclusions**

The introduction of immunotherapy in the management of locally advanced and metastatic lung cancer has improved patient survival but only achieves a lasting clinical benefit in 20 to 50% of patients. The search for biomarkers that can successfully identify patients likely to benefit from this treatment is therefore a major challenge. Our study showed that radiomics parameters extracted from baseline and follow-up PET/CT scans, as well as their evolution during treatment, could play an important role in evaluating the risk of progression during treatment and in predicting response to therapy, DCB and survival of NSCLC patients treated with immunotherapy. As demonstrated in this real-clinical-conditions cohort, these parameters seemed to provide a real added value for outcome prediction compared with clinical or standard PET/CT metrics alone, which could be beneficial for personalized patient management. Although promising, these results provide the rationale for an external validation in a large prospective cohort to ensure their clinical significance.

**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers14235931/s1. Table S1: Acquisition, reconstruction et preprocessing parameters; Table S2: List of radiomics parameters; Table S3: Results of descriptive analysis for OS, PFS and DCB; Table S4: Results of univariate analysis with bootstrap for durable clinical benefit (DCB) (mean AUC with standard deviation and confidence intervals); Table S5: Kaplan-Meier and Cox analysis scores for clinical parameters: (A) Progression-Free Survival, (B) Overall survival; Table S6: Progression-Free Survival Kaplan-Meier and Cox analysis scores for PET radiomics; Table S7: Overall Survival Kaplan-Meier and Cox analysis scores for PET radiomics; Table S8: Progression-Free Survival Kaplan-Meier and Cox analysis scores for CT; Table S9: Overall Survival Kaplan-Meier and Cox analysis scores for CT radiomics; Table S10: Radiomics Quality Score; Figure S1: OS and PFS (Kaplan-Meier and log rank test) related to (A) age, (B) performance status, (C) histology, (D) smoking history, (E) baseline metabolic tumor volume, (F) baseline total lesion glycolysis; Figure S2: Prediction of outcome: Best clinical, PET and CT parameters performance at (A) baseline and (B,C) during treatment: best AUC value obtained for the considered parameters using univariate analysis with bootstrap.

**Author Contributions:** Conception and design, C.C.L.R., J.P.T., C.L. (Corinne Lamour) and O.T.; Methodology, C.C.L.R. and O.T.; Software and statistical analysis, O.T.; Validation O.T. and C.C.L.R.; Formal Analysis, F.T., O.T. and C.C.L.R.; Data Curation, F.T., O.T., C.L. (Claire Latappy), M.B., J.D., C.L. (Corinne Lamour) and J.P.T.; Original Draft Preparation, F.T., O.T. and C.C.L.R.; Review & Editing, O.T., C.L. (Claire Latappy), M.B., J.D., C.L. (Corinne Lamour), J.P.T., O.T. and C.C.L.R.; Supervision, O.T. and C.C.L.R. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** No ethical committee approval was required, given the retrospective nature of this study.

**Informed Consent Statement:** Ethical review and approval were waived for this study, due to the retrospective nature of this study of previously anonymized data.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.
## References

1. Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. *CA Cancer J. Clin.* **2021**, *71*, 209–249. [CrossRef] [PubMed]

2. Remon, J.; Ahn, M.J.; Girard, N.; Johnson, M.; Kim, D.W.; Lopes, G.; Pillai, R.N.; Solomon, B.; Villacampa, G.; Zhou, Q. Advanced-Stage Non-Small Cell Lung Cancer: Advances in Thoracic Oncology 2018. *J. Thorac. Oncol.* **2019**, *14*, 1134–1155. [CrossRef] [PubMed]

3. Garon, E.B.; Rizvi, N.A.; Hui, R.; Leighl, N.; Balmanoukian, A.S.; Eder, J.P.; Patnaik, A.; Aggarwal, C.; Gubens, M.; Horn, L.; et al. Pembrolizumab for the treatment of non-small-cell lung cancer. *N. Engl. J. Med.* **2015**, *372*, 2018–2028. [CrossRef]

4. Champiat, S.; Dercle, L.; Ammari, S.; Massard, C.; Hollebecque, A.; Postel-Vinay, S.; Chaput, N.; Eggermont, A.; Marabelle, A.; Soria, J.C.; et al. Hyperprogressive Disease Is a New Pattern of Progression in Cancer Patients Treated by Anti-PD-1/PD-L1. *Clin. Cancer Res.* **2017**, *23*, 1920–1928. [CrossRef]

5. Brahmer, J.; Reckamp, K.L.; Baas, P.; Crino, L.; Eberhardt, W.E.; Poddubskaya, E.; Antonia, S.; Pluzanski, A.; Vokes, E.E.; Holgado, E.; et al. Nivolumab versus Docetaxel in Advanced Squamous-Cell Non-Small-Cell Lung Cancer. *N. Engl. J. Med.* **2015**, *373*, 123–135. [CrossRef] [PubMed]

6. Borghaei, H.; Paz-Ares, L.; Horn, L.; Spigel, D.R.; Steins, M.; Ready, N.E.; Chow, L.Q.; Vokes, E.E.; Felip, E.; Holgado, E.; et al. Nivolumab versus Docetaxel in Advanced Nonsquamous Non-Small-Cell Lung Cancer. *N. Engl. J. Med.* **2015**, *373*, 1627–1639. [CrossRef]

7. Reck, M.; Rodriguez-Abreu, D.; Robinson, A.G.; Hui, R.; Csoszi, T.; Fulop, A.; Gottfried, M.; Peled, N.; Tafreshi, A.; Cuffe, S.; et al. Pembrolizumab versus Chemotherapy for PD-L1-Positive Non-Small-Cell Lung Cancer. *N. Engl. J. Med.* **2016**, *375*, 1823–1833. [CrossRef]

8. Reck, M.; Rodriguez-Abreu, D.; Robinson, A.G.; Hui, R.; Csoszi, T.; Fulop, A.; Gottfried, M.; Peled, N.; Tafreshi, A.; Cuffe, S.; et al. Updated Analysis of KEYNOTE-024: Pembrolizumab Versus Platinum-Based Chemotherapy for Advanced Non-Small-Cell Lung Cancer With PD-L1 Tumor Proportion Score of 50% or Greater. *J. Clin. Oncol.* **2019**, *37*, 537–546. [CrossRef]

9. Lambin, P.; Rios-Velazquez, E.; Leijenaar, R.; Carvalho, S.; van Stiphout, R.G.; Granton, P.; Zegers, C.M.; Gillies, R.; Boellard, R.; Dekker, A.; et al. Radiomics: Extracting more information from medical images using advanced feature analysis. *Eur. J. Cancer* **2012**, *48*, 441–446. [CrossRef]

10. Zhang, R.; Zhu, L.; Cai, Z.; Jiang, W.; Li, J.; Yang, C.; Yu, C.; Jiang, B.; Wang, W.; Xu, W.; et al. Potential feature exploration and model development based on 18F-FDG PET/CT images for differentiating benign and malignant lung lesions. *Eur. J. Radiol.* **2019**, *121*, 108735. [CrossRef]

11. Kim, S.K.; Allen-Auerbach, M.; Goldin, J.; Fueger, B.J.; Dahlbom, M.; Brown, M.; Czernin, J.; Schiepers, C. Accuracy of PET/CT in characterization of solitary pulmonary lesions. *J. Nucl. Med.* **2007**, *48*, 214–220. [PubMed]

12. Balagurunathan, Y.; Schabath, M.B.; Wang, H.; Liu, Y.; Gillies, R.J. Quantitative Imaging features Improve Discrimination of Malignancy in Pulmonary nodules. *Sci. Rep.* **2019**, *9*, 8528. [CrossRef] [PubMed]

13. Hyun, S.H.; Ahn, M.S.; Koh, Y.W.; Lee, S.J. A Machine-Learning Approach Using PET-Based Radiomics to Predict the Histological Subtypes of Lung Cancer. *Clin. Nucl. Med.* **2019**, *44*, 956–960. [CrossRef] [PubMed]

14. Kirienko, M.; Cozzi, L.; Antunovic, L.; Lozza, L.; Fogliata, A.; Voulaz, E.; Rossi, A.; Chiti, A.; Sollini, M. Prediction of disease-free survival by the PET/CT radiomic signature in non-small cell lung cancer patients undergoing surgery. *Eur. J. Nucl. Med. Mol. Imaging* **2018**, *45*, 207–217. [CrossRef]

15. Dissaux, G.; Visvikis, D.; Da-Ano, R.; Pradier, O.; Chajon, E.; Barillot, I.; Duverge, L.; Masson, I.; Abgral, R.; Santiago Ribeiro, M.J.; et al. Pretreatment (18)F-FDG PET/CT Radiomics Predict Local Recurrence in Patients Treated with Stereotactic Body Radiotherapy for Early-Stage Non-Small Cell Lung Cancer: A Multicentric Study. *J. Nucl. Med.* **2020**, *61*, 814–820. [CrossRef]

16. Baek, S.; He, Y.; Allen, B.G.; Buatti, J.; Smith, B.; Tong, L.; Sun, Z.; Wu, J.; Diehn, M.; Loo, B.; et al. Deep segmentation networks predict survival of non-small cell lung cancer. *Sci. Rep.* **2019**, *9*, 1286. [CrossRef]

17. Khorrami, M.; Prasanna, P.; Gupta, A.; Patil, P.; Velu, P.D.; Thawani, R.; Corredor, G.; Alilou, M.; Bera, K.; Fu, P.; et al. Changes in CT Radiomic Features Associated with Lymphocyte Distribution Predict Overall Survival and Response to Immunotherapy in Non-Small Cell Lung Cancer. *Cancer Immunol. Res.* **2020**, *8*, 108–119. [CrossRef]

18. Barabino, E.; Rossi, G.; Pamparino, S.; Fiannacca, M.; Caprioli, S.; Fedeli, A.; Zullo, L.; Vagge, S.; Cittadini, G.; Genova, C. Exploring Response to Immunotherapy in Non-Small Cell Lung Cancer Using Delta-Radiomics. *Cancers* **2022**, *14*, 350. [CrossRef]

19. Mu, W.; Tunali, I.; Gray, J.E.; Qi, J.; Schabath, M.B.; Gillies, R.J. Radiomics of (18)F-FDG PET/CT images predicts clinical benefit of advanced NSCLC patients to checkpoint blockade immunotherapy. *Eur. J. Nucl. Med. Mol. Imaging* **2020**, *47*, 1168–1182. [CrossRef]

20. Detterbeck, F.C.; Boffa, D.J.; Kim, A.W.; Tanoue, L.T. The Eighth Edition Lung Cancer Stage Classification. *Chest* **2017**, *151*, 193–203. [CrossRef]

21. Grootjans, W.; Usmanij, E.; Oyen, W.; van der Heijden, E.; Visser, E.; Visvikis, D.; Hatt, M.; Bussink, J.; de Geus-Oei, L. Performance of automatic image segmentation algorithms for calculating total lesion glycolysis for early response monitoring in non-small cell lung cancer patients during concomitant chemoradiotherapy. *Radiother. Oncol.* **2016**, *119*, 473–479. [CrossRef] [PubMed]

22. Hatt, M.; Cheze Le Rest, C.; Albarghach, N.; Pradier, O.; Visvikis, D. PET functional volume delineation: A robustness and repeatability study. *Eur. J. Nucl. Med. Mol. Imaging* **2011**, *38*, 663–672. [CrossRef] [PubMed]
23. Hatt, M.; Laurent, B.; Ouahabi, A.; Fayad, H.; Tan, S.; Li, L.; Lu, W.; Jaouen, V.; Tauber, C.; Czakon, J.; et al. The first MICCAI challenge on PET tumor segmentation. *Med. Image Anal.* **2018**, *44*, 177–195. [CrossRef] [PubMed]

24. Fedorov, A.; Beichel, R.; Kalpathy-Cramer, J.; Finet, J.; Fillion-Robin, J.C.; Pujol, S.; Bauer, C.; Jennings, D.; Fennessy, F.; Sonka, M.; et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. *Magn. Reson. Imaging* **2012**, *30*, 1323–1341. [CrossRef]

25. van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.C.; Pieper, S.; Aerts, H. Computational Radiomics System to Decode the Radiographic Phenotype. *Cancer Res.* **2017**, *77*, e104–e107. [CrossRef]

26. Wahl, R.L.; Jacene, H.; Kasamon, Y.; Lodge, M.A. From RECIST to PERCIST: Evolving Considerations for PET response criteria in solid tumors. *J. Nucl. Med.* **2009**, *50* (Suppl. 1), 122S–150S. [CrossRef] [PubMed]

27. Goldfarb, L.; Duchemann, B.; Chouahnia, K.; Zelek, L.; Soussan, M. Monitoring anti-PD-1-based immunotherapy in non-small cell lung cancer with FDG PET: Introduction of iPERCIST. *EJNMMI Res.* **2019**, *9*, 8. [CrossRef] [PubMed]

28. Eisenhauer, E.A.; Therasse, P.; Bogaerts, J.; Schwartz, L.H.; Sargent, D.; Ford, R.; Dancey, J.; Arbuck, S.; Gwyther, S.; Mooney, M.; et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). *Eur. J. Cancer* **2009**, *45*, 228–247. [CrossRef]

29. Tibshirani, R. Regression Shrinkage and Selection Via the Lasso. *J. R. Stat. Soc. Ser. B (Methodol.)* **1996**, *58*, 267–288. [CrossRef]

30. Lambin, P.; Leijenaar, R.T.H.; Deist, T.M.; Peerlings, J.; de Jong, E.E.C.; van Timmeren, J.; Sanduleanu, S.; Larue, R.; Even, A.J.G.; Jochems, A.; et al. Radiomics: The bridge between medical imaging and personalized medicine. *Nat. Rev. Clin. Oncol.* **2017**, *14*, 749–762. [CrossRef]

31. Humbert, O.; Cadour, N.; Paquet, M.; Schiappa, R.; Poudenx, M.; Chardin, D.; Borchiellini, D.; Benisvy, D.; Ouvrier, M.J.; Zwarthoed, C.; et al. (18)FDG PET/CT in the early assessment of non-small cell lung cancer response to immunotherapy: Frequency and clinical significance of atypical evolutive patterns. *Eur. J. Nucl. Med. Mol. Imaging* **2020**, *47*, 1158–1167. [CrossRef] [PubMed]

32. Tazdait, M.; Mezquita, L.; Lahmar, J.; Ferrara, R.; Bidault, F.; Ammari, S.; Balleyguier, C.; Planchard, D.; Gazzah, A.; Soria, J.C.; et al. Patterns of responses in metastatic NSCLC during PD-1 or PDL-1 inhibitor therapy: Comparison of RECIST 1.1, irRECIST and iRECIST criteria. *Eur. J. Cancer* **2018**, *88*, 38–47. [CrossRef] [PubMed]

33. Ricciuti, B.; Genova, C.; Bassanelli, M.; De Giglio, A.; Brambilla, M.; Metro, G.; Baglivo, S.; Dal Bello, M.G.; Ceribelli, A.; Grossi, F.; et al. Safety and Efficacy of Nivolumab in Patients With Advanced Non-small-cell Lung Cancer Treated Beyond Progression. *Clin. Lung Cancer* **2019**, *20*, 178–185 e172. [CrossRef] [PubMed]

34. Castello, A.; Toschi, L.; Rossi, S.; Mazziotti, E.; Lopci, E. The immune-metabolic-prognostic index and clinical outcomes in patients with non-small cell lung carcinoma under checkpoint inhibitors. *J. Cancer Res. Clin. Oncol.* **2020**, *146*, 1235–1243. [CrossRef]

35. Seban, R.D.; Mezquita, L.; Berenbaum, A.; Dercle, L.; Botticella, A.; Le Pechoux, C.; Caramella, C.; Deutsch, E.; Grimaldi, S.; Adam, J.; et al. Baseline metabolic tumor burden on FDG PET/CT scans predicts outcome in advanced NSCLC patients treated with immune checkpoint inhibitors. *Eur. J. Nucl. Med. Mol. Imaging* **2020**, *47*, 1147–1157. [CrossRef]

36. Nardone, V.; Tini, P.; Pastina, P.; Botta, C.; Reginelli, A.; Carbone, S.F.; Giannicola, R.; Calabrese, G.; Tebala, C.; Guida, C.; et al. Radiomics predicts survival of patients with advanced non-small cell lung cancer undergoing PD-1 blockade using Nivolumab. *Oncol. Lett.* **2020**, *19*, 1559–1566. [CrossRef]

37. Zhao, W.; Jiang, W.; Wang, H.; He, J.; Su, C.; Yu, Q. Impact of Smoking History on Response to Immunotherapy in Non-Small-Cell Lung Cancer: A Systematic Review and Meta-Analysis. *Front. Oncol.* **2021**, *11*, 703143. [CrossRef]

38. Polverari, G.; Ceci, F.; Bertaglia, V.; Reale, M.L.; Rampado, O.; Gallio, E.; Passera, R.; Liberini, V.; Scapoli, P.; Arena, V.; et al. (18)F-FDG Pet Parameters and Radiomics Features Analysis in Advanced Nsclc Treated with Immunotherapy as Predictors of Therapy Response and Survival. *Cancers* **2020**, *12*, 1163. [CrossRef]

39. Kaira, K.; Higuchi, T.; Naruse, I.; Arisaka, Y.; Tokue, A.; Altan, B.; Suda, S.; Mogi, A.; Shimizu, K.; Sunaga, N.; et al. Metabolic activity by (18)F-FDG-PET/CT is predictive of early response after nivolumab in previously treated NSCLC. *Eur. J. Nucl. Med. Mol. Imaging* **2018**, *45*, 56–66. [CrossRef]

40. Nishino, M.; Dahlberg, S.E.; Adeni, A.E.; Lydon, C.A.; Hatabu, H.; Janne, P.A.; Hodi, F.S.; Awad, M.M. Tumor Response Dynamics of Advanced Non-small Cell Lung Cancer Patients Treated with PD-1 Inhibitors: Imaging Markers for Treatment Outcome. *Clin. Cancer Res.* **2017**, *23*, 5737–5744. [CrossRef]

41. Valentini, V.; Lambin, P.; Myerson, R.J. Is it time for tailored treatment of rectal cancer? From prescribing by consensus to prescribing by numbers. *Radiother. Oncol.* **2012**, *102*, 1–3. [CrossRef]

42. Ahn, H.K.; Lee, H.; Kim, S.G.; Hyun, S.H. Pre-treatment (18)F-FDG PET-based radiomics predict survival in resected non-small cell lung cancer. *Clin. Radiol.* **2019**, *74*, 467–473. [CrossRef]

43. Saeed-Vafa, D.; Bravo, R.; Dean, J.A.; El-Kenawi, A.; Mon Père, N.; Strobl, M.; Daniels, C.; Stringfield, O.; Damaghi, M.; Tunali, I.; et al. Combining radiomics and mathematical modeling to elucidate mechanisms of resistance to immune checkpoint blockade in non-small cell lung cancer. *bioRxiv* **2017**. [CrossRef]

44. Dercle, L.; Fronheiser, M.; Lu, L.; Du, S.; Hayes, W.; Leung, D.K.; Roy, A.; Wilkerson, J.; Guo, P.; Fojo, A.T.; et al. Identification of Non-Small Cell Lung Cancer Sensitive to Systemic Cancer Therapies Using Radiomics. *Clin. Cancer Res.* **2020**, *26*, 2151–2162. [CrossRef] [PubMed]

45. Majdoub, M.; Hoeben, B.; Troost, E.G.C.; Oyen, W.; Kaanders, J.; Cheze Le Rest, C.; Visser, E.; Visvikis, D.; Hatt, M. Prognostic Value of Head and Neck Tumor Proliferative Sphericity From 3'-Deoxy-3'-[<sup>18</sup>Fluorothymidine Positron Emission Tomography. *IEEE Trans. Radiat. Plasma Med. Sci.* **2018**, *2*, 33–40. [CrossRef]
46. Grove, O.; Berglund, A.E.; Schabath, M.B.; Aerts, H.; Dekker, A.; Wang, H.; Velazquez, E.R.; Lambin, P.; Gu, Y.; Balagurunathan, Y.; et al. Correction: Quantitative Computed Tomographic Descriptors Associate Tumor Shape Complexity and Intratumor Heterogeneity with Prognosis in Lung Adenocarcinoma. *PLoS ONE* **2021**, *16*, e0248541. [CrossRef] [PubMed]

47. Avanzo, M.; Wei, L.; Stancanello, J.; Vallieres, M.; Rao, A.; Morin, O.; Mattonen, S.A.; El Naqa, I. Machine and deep learning methods for radiomics. *Med. Phys.* **2020**, *47*, e185–e202. [CrossRef]

48. Grossmann, P.; Stringfield, O.; El-Hachem, N.; Bui, M.M.; Rios Velazquez, E.; Parmar, C.; Leijenaar, R.T.; Haibe-Kains, B.; Lambin, P.; Gillies, R.J.; et al. Defining the biological basis of radiomic phenotypes in lung cancer. *Elife* **2017**, *6*, e23421. [CrossRef]

49. Xu, C.J.; van der Schaaaf, A.; Schilstra, C.; Langendijk, J.A.; van't Veld, A.A. Impact of statistical learning methods on the predictive power of multivariate normal tissue complication probability models. *Int. J. Radiat. Oncol. Biol. Phys.* **2012**, *82*, e677–e684. [CrossRef]

50. Hastie, T.; Tibshirani, R.; Friedlander, M. *The Elements of Statistical Learning*; Springer: New York, NY, USA, 2009.

51. Yip, S.S.; Aerts, H.J. Applications and limitations of radiomics. *Phys. Med. Biol.* **2016**, *61*, R150–R166. [CrossRef]

52. Zwanenburg, A.; Vallieres, M.; Abdalah, M.A.; Aerts, H.; Andrearczyk, V.; Apte, A.; Ashrafinia, S.; Bakas, S.; Beukinga, R.J.; Boellaard, R.; et al. The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. *Radiology* **2020**, *295*, 328–338. [CrossRef] [PubMed]