# Leveraging Deep Learning for Immune Cell Quantification and Prognostic Evaluation in Radiotherapy-Treated Oropharyngeal Squamous Cell Carcinomas.

Fanny Beltzung, Van-Linh Le, Ioana Molnar, Erwan Boutault, Claude

Darcha, François Le Loarer, Myriam Kossai, Olivier Saut, Julian Biau,

Frédérique Penault-Llorca, et al.

► To cite this version:

Fanny Beltzung, Van-Linh Le, Ioana Molnar, Erwan Boutault, Claude Darcha, et al. Leveraging Deep Learning for Immune Cell Quantification and Prognostic Evaluation in Radiotherapy-Treated Oropharyngeal Squamous Cell Carcinomas. Laboratory Investigation, 2025, pp.104094. 10.1016/j.labinv.2025.104094. hal-04927884

HAL Id: hal-04927884

https://inria.hal.science/hal-04927884v1

Submitted on 19 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Leveraging Deep Learning for Immune Cells Quantification and Prognostic Evaluation in
Radiotherapy-Treated Oropharyngeal Squamous Cell Carcinomas.

Fanny Beltzung<sup>1,2*</sup>, Van Linh Le<sup>3,4</sup>, Ioana Molnar<sup>1,5</sup>, Erwan Boutault<sup>1</sup>, Claude Darcha<sup>6</sup>, François Le Loarer<sup>7,8</sup>, Myriam Kossai<sup>1,8</sup>, Olivier Saut<sup>3</sup>, Julian Biau<sup>1,10</sup>, Frédérique Penault-Llorca<sup>1,8</sup>, Emmanuel Chautard<sup>1,8</sup>

<footnote n="1"><sup>1</sup> University Clermont Auvergne, INSERM U1240 [Molecular Imaging & Theragnostic Strategies (IMOST)], Clermont-Ferrand, France.</footnote>

<footnote n="2"><sup>2</sup> Department of Pathology, Hôpital Haut-Lévêque, CHU de Bordeaux, Pessac, France.</footnote>

<footnote n="3"><sup>3</sup> Bordeaux Mathematics Institute (IMB), UMR CNRS 5251, INRIA Project Team Monc, University of Bordeaux, Talence, France.</footnote>

<footnote n="4"><sup>4</sup> Department of Data and Digital Health, Bergonié Institute, Bordeaux, France.</footnote>

<footnote n="5"><sup>5</sup> Clinical Research Division, Clinical Research & Innovation Division, Centre Jean PERRIN, Clermont-Ferrand, France.</footnote>

<footnote n="6"><sup>6</sup> Department of Pathology, CHU Clermont-Ferrand, Clermont-Ferrand, France.</footnote>

<footnote n="7"><sup>7</sup> Department of Pathology, Bergonié Institute, Bordeaux, France.</footnote>

<footnote n="8"><sup>8</sup> Bordeaux Institute of Oncology, BRIC U1312, INSERM, Université de Bordeaux, Institut Bergonié, 33000 Bordeaux, France</footnote>

<footnote n="9"><sup>9</sup> Department of Pathology, Centre Jean PERRIN, Clermont-Ferrand, France.</footnote>

<footnote n="10"><sup>10</sup>Department of Radiation Therapy, Centre Jean PERRIN, Clermont-Ferrand, France.</footnote>
<footnote>*Corresponding author: Fanny Beltzung, Department of Pathology, Centre Hospitalier Universitaire de Bordeaux, Hôpital du Haut Lévêque, Avenue Magellan, 33600, Pessac, France. Fanny.beltzung@chubordeaux.fr. +33 5 24 54 91 00</footnote>
ABSTRACT

The tumor microenvironment (TME) is a critical factor in cancer progression and therapeutic responsiveness, with the immune component, known as the tumor immune microenvironment (TIME), playing a key role in modulating these processes. In head and neck squamous cell carcinomas (HNSCC), immune cell infiltration significantly impacts response to radiotherapy (RT) and may help in identifying patients most likely to benefit from immunotherapy. Currently, there is a lack of standardized, cost-effective methods for studying TIME in these cancers. This study aims to leverage deep learning (DL) to quantify immune cell densities using immunohistochemistry in untreated oropharyngeal squamous cell carcinoma (OPSCC) biopsies of patients scheduled for curative RT, assessing their prognostic value. We analyzed 84 pre-treatment formalin-fixed paraffin-embedded (FFPE) tumor biopsies from OPSCC patients. Immunohistochemistry (IHC) was performed for CD3, CD8, CD20, CD163, and FOXP3, and whole slide images (WSIs) were digitized for analysis using a U-Net-based DL model. Two quantification approaches were applied: a cell-counting method and an area-based method measuring stained regions. The DL model achieved high accuracy in detecting stained cells across all biomarkers. Strong correlations were found between our DL pipeline, the HALO® Image Analysis Platform, and the open-source QuPath software for estimating immune cell densities. Our DL pipeline provides an accurate and reproducible approach for quantifying immune cells in OPSCC. The area-based quantification demonstrated superior prognostic value for recurrence-free survival (RFS) compared to cell-counting. Elevated densities of CD3, CD8, CD20, and FOXP3 were associated with improved RFS, while CD163 showed no significant prognostic association. These results highlight the potential of DL in digital pathology for assessing TIME and predicting patient outcomes. Further validation in larger cohorts is recommended.
INTRODUCTION

The tumor immune microenvironment (TIME) is a complex and dynamic ecosystem of diverse immune
cells interacting with tumor cells and plays a crucial role in either promoting or inhibiting tumor growth
and progression (1,2). The tumor and immune system interactions evolve as tumors evade immune
detection and adapt to the different therapies received by the patient (3,4). Primary treatments for
upper aerodigestive tract cancers include surgery (if feasible), radiotherapy, and medical treatments,
such as chemotherapy and immunotherapy.

Understanding TIME in head and neck squamous cell carcinoma (HNSCC) is essential, as immune cell infiltration significantly impacts radiotherapy response (5). Additionally, while immunotherapy has been introduced for this cancer type, it has demonstrated efficacy in only a limited subset of patients, underscoring the need for improved methods to identify those most likely to benefit from such treatments (6). Research has consistently shown that TIME is a critical factor in predicting outcomes for patients with oropharyngeal squamous cell carcinoma (OPSCC) (7).

However, the approaches for studying TIME vary considerably across studies, from cost-effective
immunohistochemistry (IHC) to advanced molecular techniques like transcriptomics, each targeting
different biomarkers (8,9). As a result, a standardized methodology and a consensus on predictive
biomarkers for TIME in OPSCC remain unmet challenges. This study aims to apply deep learning (DL) to
assess immune cell densities in biopsies from untreated OPSCC patients scheduled for curative
radiotherapy, analyzing key immune markers using IHC to explore their prognostic significance. IHC
offers a practical and relatively low-cost approach to examine TIME. Common markers studied include
CD3 (general T-cells), CD8 (cytotoxic T-cells), FOXP3 (regulatory T-cells), CD20 (B-cells), and CD163 (M2
macrophages). While pathologists traditionally quantify immune cells through light microscopy and
various scoring methods (quantitative, semi-quantitative, and observational), these manual
approaches are limited by variability among observers (10). The transition to digital pathology has led
to the development of machine learning algorithms for analyzing whole slide images (WSI), offering
the promise of objective, efficient, and reproducible immune cell quantification through automated
detection (11).

In this study, our main objective was to evaluate the reliability of deep learning for automatic quantification of immune cell across a range of biomarkers, employing two distinct quantification approaches: the area method and the cell method, in a cohort of 84 OPSCC patients. A secondary objective was to assess the clinical relevance of our pipeline by examining its ability to predict patient prognosis. To achieve these aims, we first validated the algorithm's performance in segmenting DAB staining, using a model trained, validated, and tested on CD3 and then extending it to other biomarkers, including CD8, CD20, FOXP3, and CD163, as well as an external dataset (LYSTO) (12). We then compared the immune cell density estimates generated by our pipeline with manual counts by expert pathologists, results from a commercial platform (HALO® Image Analysis Platform, Indica Labs, Corrales, New Mexico, USA), and output from open-source software (QuPath). Finally, we assessed the clinical utility of these quantification results by testing their ability to predict recurrence-free survival (RFS) in our cohort of OPSCC patients treated with curative-intent radiotherapy (RT).
MATERIALS AND METHODS

Clinical samples and patients

Eighty-eight pre-treatment formalin-fixed paraffin-embedded (FFPE) tumor biopsies from OPSCC patients treated in the department of radiotherapy of the Comprehensive Cancer Center Jean Perrin between 2016 and 2019 were included in the present study and used for TIME exploration by immunohistochemistry and for DL analysis of staining patterns.

For clinical relevance assessment of our data, four patients were excluded from statistical analysis because of lack of complete clinical data. All patients (n=84) were treated with curative-intent RT. The mean age was 62.3 (SD : 9.89). There were 65 male (77.4%), and 19 female (22.7%). 47 patients were identified as smokers, characterized by either current smoking habits or cessation within the past year. Additionally, 35 patients exhibited excessive alcohol consumption, defined as either ongoing use or cessation within the past year. The initial location of the tumors was the tonsil in 48 patients (57,1%), the base of tongue in 22 patients (26,2%), the soft palate in 13 patients (15,5%) and the posterior pharyngeal wall in 1 patient (1,2%). Clinical stages at diagnosis (AJCC/UICC 8th edition) were as follows: 6 stage I (7.1%), 24 stage II (28.6%), 19 stage III (22.6%), and 35 stage IV (41.7%). The ECOG performance status was assessed as 0-1 for 75 patients (89.3%), 2 for 7 patients (8.3%), and 3 for 2 patients (2.4%).

The treatment modality of each patient was decided at a multidisciplinary team meeting. Fifty-eight patients (69%) had definitive RT (without surgery) and 26 had surgery followed by post-operative RT (31%). All patients received RT, alone for 16 patients (19%), after induction chemotherapy (CT) for 10 patients (11.9%), with concomitant CT (cisplatin or cetuximab) for 58 patients (69%). Patients and tumors characteristics are summarized in *Supplementary Table 1*.
The first post-treatment follow-up was 3 months after radiation completion. Then, every 3 months for the first and the second year, and at least twice a year for up to 5 years, and thereafter yearly. A follow-up imaging was performed at 3 months and then annually. Median follow-up was 28 months.

Study ethics approval was obtained on July 11, 2022 (IRB committee of CJP IRB 00013468).

## Immunohistochemistry and digitalization

Immunohistochemistry for CD3, CD8, CD20, CD163, and FOXP3 (Supplementary Table 2) was performed using standard procedures with a Ventana BenchMark ULTRA (Roche Diagnostics, Tucson, AZ, USA) device with 3,3'-Diaminobenzidine (DAB) revelation. IHC-stained whole tissue slides were scanned using Aperio GT 450 DX (Leica Biosystems, Buffalo Grove, IL, USA) into WSI. Stained slides were digitized at ×40 magnification and at a resolution of 0.26 µm/pixel.

## HPV testing

Tumor samples were initially assessed with p16 immunohistochemistry (IHC) (CINtec p16 Histology Kit, Roche mtm laboratories AG, Heidelberg, Germany). p16INK4 expression was deemed positive if there was strong and diffuse staining in the nuclei or cytoplasm in 70% or more of the tumor cells. All other staining patterns were scored as negative. In all samples, the presence of HPV-DNA was tested by Abbott RealTime HR HPV test (Abbott Molecular, Des Plaines, IL, USA), which is a qualitative real-time PCR test for the detection of DNA from 14 high-risk human papillomavirus (HPV) types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68) (13). The corresponding oropharynx SCCs were considered HPV-driven if both assays were positive. A total of 28 out of 84 tumors were HPV driven.
Immune cells quantification with deep learning (DL)

The proposed pipeline consisted of three steps, as illustrated in Fig. 1A. The first step involved pre-
processing the whole-slide images (WSIs). Regions of interest (ROIs), specifically tumor regions, were
identified within each image by an experienced pathologist (FB). Folds, bubbles, and staining artifacts
were manually excluded during this step. From the selected ROIs, tiles measuring 224 x 224 pixels were
extracted according to previously established methods (14,15) and saved in .png format to serve as
input for the segmentation model.

The second step corresponded to segmentation (i.e. DAB staining detection). We employed a deep-
learning model based on the U-Net model (16). The U-Net structure was modified from the traditional
U-Net model to fit our requirements as previoulsy described (17). Details of the CGPD-based model
architecture and the rationale for key design choices are provided in **Supplementary Method 1**. To
evaluate the performance of the U-Net in detecting DAB staining, the evaluation metrics used were
accuracy, Intersection over Union (IoU), F1-score (F1), and the area under the receiver operating
characteristic curve (AUC). A total of 88 CD3 WSIs were utilized in the study: 48 for training and
validation, and 40 for testing of the U-Net model. To generate segmentation masks for training, we
used ImageJ as previously described (18,19). The trained model was subsequently applied to segment
DAB staining across 84 WSIs covering all biomarkers. On the validation set (n=48) for CD3, the U-Net
achieved an accuracy of 0.9995, IoU of 0.9049, F1-score of 0.9632, and an AUC of 0.9981. To extend
the model's application to other biomarkers, we generated binary masks for 10 WSIs of CD8, CD20,
CD163, and FOXP3, using the pre-trained CD3 model for DAB staining detection in these tiles. To ensure
an objective evaluation, we also applied our pipeline to the LYSTO dataset (12), which consists of 20,000
tiles, each measuring 229 x 229 pixels. These tiles were extracted from 932 ROIs of 83 WSI of colon
(n=28), breast (n = 33), and prostate (n=22) cancer stained either CD3 or CD8 IHC collected and
produced in a multi-centric fashion, including tissue preparation or staining from hospital in Eindhoven,
Radboudumc, Rijnstate, Utrecht, Heerlen, AMC Amsterdam, as well as JBZ (Jeroen Bosch Ziekenhuis)
and LabPON (Laboratorium Pathologie Oost-Nederland). The same evaluation metrics were applied to assess performance. We observed differences in the detection of CD3 and FOXP3 (nuclear staining). To address this, the U-Net model was specifically re-trained for FOXP3 segmentation using a cross-validation approach, as detailed in **Supplementary Method 2**. Performance metrics were evaluated based on predictions for Set 3 to ensure comparability with segmentation results for other markers. Following re-training, the model achieved the following performance metrics for FOXP3 segmentation: accuracy = 0.9968, F1 score = 0.8199, IoU = 0.7091, and AUC = 0.9981. The trained model was applied to predict segmentations for the remaining 40 CD3 WSIs and other biomarkers (CD8, CD20, CD163, and FOXP3) in 84 patients. The resulting segmentations were used for cell density estimation.

The last step involved estimating the density of immune cells. Two methodologies were tested for immune cells density estimation: a cell method and an area method. The cell method aimed to identify and enumerate stained cells within the designated tumor ROI, with results expressed as "number of cells/µm²". To quantify cells within a tumor sample, stained cells across all tiles within the region were tallied. Two algorithms were employed for cell counting: the first algorithm predicted cell numbers based on the minimum distance between peak positions in the image, while the second algorithm utilized a disc-shaped footprint to identify contiguous regions corresponding to stained cell clusters. Details of the algorithms used for stained cell detection, including parameter settings and segmentation processes, are provided in **Supplementary Method 3**. For calibration, both algorithms were evaluated against a manual count of CD3+ cells in 1,000 tiles performed by observer 1 (FB). To assess inter-observer variability, a second observer (EC) independently counted CD3+ cells in the same tiles. The percentage difference between the two observers was calculated to identify tiles with the most divergent counts. Outliers were determined using the boxplot method, defined by thresholds of Q1 - 1.5×IQR and Q3 + 1.5×IQR. Among these, differences of a single count between observers 1 and 2 were considered negligible. Ultimately, 56 outlier tiles were identified and recounted by a third observer (FPL).
The area method provided the surface area of DAB staining (obtained during segmentation) within the ROI. The area of the ROI corresponded to the sum of the tissue area of all tiles belonging to this region. As this work was performed on a scaled image, the total number of tissue pixels was multiplied by the scaling ratio. Both staining surface and tumor surface were then converted to µm² by multiplying by 0.262. The results of the area method were expressed in "µm²/µm²".

Immune cells quantification on HALO® Image Analysis Platform

The module Area Quantification (version 3.3, HALO® Image Analysis Platform, Indicalabs, Corrales, NM, USA) was used to quantify the expression of the immunomarkers (CD3, CD8, CD20, CD163, FOXP3). This module is an area-based analysis module for quantifying the staining surface of immunohistochemistry slides. The digital slides were opened in HALO® platform in .svs format. The workflow contained three main steps: selection, classification, and analysis. These three steps were all done by an experienced pathologist (FB) and are represented in Fig. 1B. The objective of the selection step was to manually select the ROIs on the slides. The ROIs corresponded to the tumor tissue. Healthy tissue was excluded from selection. The classification step was done with the help of the additional module "Classifier" of the HALO® platform. This step aimed at defining three different areas in the ROIs: the tumor area, the artifact areas (areas with artifacts such as bubbles, folds, or staining artifacts, or ink), and the optically empty areas. Each WSI had its own Classifier. The objective of the analysis step was to configure parameters to produce the data, all parameter settings are provided in **Supplementary Table 3**. The output parameters generated by the algorithm and presented in the following results were the measurement of the tumor and staining areas, excluding artifacts and optically empty areas. Those measurements were used to calculate the area of staining per µm².

Immune cells quantification with the open-source software QuPath
Our image analysis pipeline in QuPath involved several key steps. First, ROIs corresponding to tumor tissue were manually annotated on each whole slide image. Next, the Positive Cell Detection tool was used to automatically count all positively stained cells within these ROIs. Detection parameters, including intensity thresholds and cell size criteria, were manually optimized for each biomarker, and all specific parameter settings are provided in **Supplementary Table 4**. The number of positive cells detected within each annotated area was then used to calculate the average number of positive cells per µm².

## Statistics analysis

The performance of the segmentation model was evaluated using ROC curve, confusion matrix (with True positive (TP), False Positive (FP), True negative (TN) and False Negative (FN) rates) and four metrics: accuracy (ACC), intersection over union (IoU), F1-score (F1), and the area under the receiver operating characteristic curve (AUC). Accuracy (ACC) measures the fraction of correct predictions:

$$ACC = \frac{TP + TN}{TP + TN + FP + FN}$$

The F1-score, the harmonic mean of precision and recall, is given by:

$$F1 = \frac{Precision * Recall}{Precision + Recall}$$

Where Precision = $\frac{TP}{TP+FP}$ and Recall = $\frac{TP}{TP+FN}$

The IoU assesses the overlap between predicted and true labels and is defined as:

$$IoU = \frac{Intersection}{Union} = \frac{TP}{TP + FP + FN}$$
Finally, AUC represents the area under the ROC curve, which plots the false positive rate (1-Specificity) against the true positive rate (Recall). All metrics were computed using Python (version 2.9) and scikit-learn (version 1.0.2).

The association between the number of manually counted cells and the algorithm's count, as well as between the counts of the two observers, was analyzed using Spearman's rank-order correlation and Pearson correlation coefficient. The Pearson correlation coefficient was also calculated to assess the correlation between the DL model's predictions of immune cells density and the results from HALO® and QuPath. The interpretation of Spearman and Pearson correlation coefficients was as follows (20) (considering absolute magnitude of the observed correlation coefficient): Very weak correlation (0.00-0.20); Weak correlation (0.20-0.39); Moderate correlation (0.40-0.59); Strong correlation (0.60-0.79) and Very strong correlation (0.80-1.00).

The Bland-Altman method was used to evaluate the agreement between the manual count of stained cells between two observers and between observer 1 and algorithms. The Bland-Altman method compares the differences between two counts to their mean value. We calculated the limits of agreement as ± 1.91 standard deviations of the differences.

The follow-up time for recurrence-free survival (RFS) was the time from initial diagnosis until last follow-up or the appearance of local, nodal, or distant recurrence, and time of death as an additional censoring time point. The follow-up time for overall survival (OS) of each patient was defined as the time from diagnosis to death or considered censored 5 years after diagnosis if alive at that point.

The Kaplan-Meier estimator was used to estimate OS and RFS, and differences in survival between groups were tested using the log-rank test. Univariate and multivariable Cox regression models were used to investigate the associations between RFS and the different biomarkers. Biomarker values were natural log-transformed. Hazard ratios were estimated with 95% Confidence Intervals (CI). The causal
effect of the continuous biomarker values on the survival outcome was represented by survival curves computed for specific values based on the multivariable Cox models using the contsurvplot R package.

All statistical analyses were performed using R statistical software (version 4.2.2). P-values were considered significant when <0.05.
# RESULTS

## Patients' outcomes

Mean follow-up time was 30.9 months. Thirty-one patients (36.9%) had a recurrence during follow-up: 11 (35.5%) tumor (T) local recurrence alone, 4 (12.9%) lymph node (N) recurrence alone, 4 (12.9%) T+N recurrence, 11 (35.5%) distant metastatic (M) recurrence alone, 1 (3.2%) T+N+M recurrence. Mean recurrence-free time was 9.8 months (range 0.9-33). At last follow-up, 46 were in remission, 32 (38%) patients had died. One- and two-year RFS were 74.9% and 62.3%, respectively. One- and two-year OS were 89.2% and 74.2%, respectively.

## Performance of the DL Model in DAB Staining Detection Across Various Biomarkers and Cell Types

First, we evaluated the performance of our U-Net model trained on CD3 IHC WSI to segmentate the DAB staining of CD8+, CD20+, CD163+, and FOXP3+-cells. Table 1 and Figure 2A and 2B present the performance of the U-Net model on CD8, CD20, CD163, and FOXP3 to detect DAB staining in the tiles of 10 WSIs. The U-Net model provided excellent detection for the markers CD8, CD20, and CD163 with an IoU superior to 0.9 and low rates of false positive (<0.0005) and false negative (<0.05) with less good results for FOXP3 (with an IoU metric inferior to 0.5). After re-training the model specifically for FOXP3, the performance metrics scores for the segmentation of FOXP3+-cells were improved : accuracy = 0.9968, F1 = 0.8199, IOU = 0.7091, AUC = 0.9981.

Examples of DAB staining detection by the U-Net model (input and output images) for each biomarker type are shown in Figure 2C. The input images are tiles extracted from the ROIs, and the output images (predictions) are the corresponding tiles computed by our proposed U-Net model. The overlap images enabled a visual comparison between input tiles and output tiles and resulted from a manual coloration of input tiles. Reviewing some examples enabled us to link false negative sometimes to faint DAB staining. On the other side false positive were mainly explained by the stronger blue
counterstaining. This was specifically the case for FOXP3 stained tiles. This difference in the counterstaining contrast for FOXP3 could explain an IOU score inferior to the other biomarkers (<0.8 for FOXP3, whereas the others are all >0.9).

We then evaluated the performance of our U-Net model to detect DAB staining in 20000 tiles from CD3 and CD8 IHC WSI of the external dataset LYSTO (Figure 2, Table 1). The pipeline's accuracy and AUC were satisfactory (0.7609 and 0.867, respectively), but F1-score and IoU (0.1 each) were low due to a high false-positive rate (0.24) and fewer true negatives (0.76).

Performance Assessment of Deep Learning Algorithms (cell method) for Immune Cell Density Estimation and Comparison with Manual Counting

The trained DL model was used to detect DAB staining in the testing set for CD3 (n=40) and across all WSIs for other markers in 84 patients. DAB staining detection was then used as input to estimate immune cells density through the cell method (using mathematical algorithms). Algorithm 1 and Algorithm 2 were applied to estimate the number of stained cells, expressed as the number of cells/µm². In the following results for CD3, the results on the 40 patients in the testing set are reported.

Across 1000 analyzed tiles, Observer 1 counted 10,430 CD3+ cells (mean = 10.43 cells/tile; SD = 7.70), while Observer 2 counted 11,426 CD3+ cells (mean = 11.43 cells/tile; SD = 8.39). Figure 3A show linear correlation and agreement between the two observers' CD3+ cell counts, with slight discrepancies in tiles exceeding 15 CD3+ cells.

When compared to the manual counts (Table 2), Algorithm 1 showed moderate correlation with manual counts for CD8, CD20, and FOXP3, but weak correlation for CD163. Algorithm 2 exhibited stronger correlations with manual counts for CD20 and FOXP3 (Pearson >0.60), but moderate correlations for CD8 (r = 0.58) and CD163 (r = 0.49).
Tiles stained with CD3 and CD20 frequently contained more than 15 cells, which defined cell aggregates within a tile (Figure 3B). Bland-Altman plots for CD3 and CD20 revealed a higher frequency of outliers beyond specific cutoffs: 20 cells for CD3 and 15 cells for CD20. Among the evaluated markers, the best agreement between the deep learning model and observer 1 was achieved by Algorithm 2 for FOXP3.

### Comparison of DL Methods with HALO® and QuPath Using Pearson Correlation Analysis for immune cell density estimation

Linear and Pearson correlations were used to compare results generated by DL methods with those obtained from HALO® (an alternative area method) and QuPath (an alternative cell method) (Table 3). Algorithm 2 (cell method) showed very strong (r>0.80) correlation with HALO® method and strong to very strong correlation with QuPath method regardless of the biomarkers studied (CD3, CD8, CD20, FOXP3, or CD163). Algorithm 1 (cell method) showed a more heterogeneous correlation with Halo® and QuPath, except of CD3 and CD8 for which consistently showed very strong correlation. Notably, weak correlation was found between algorithm 1 and HALO® method and moderate correlation with QuPath, when we considered FOXP3 with the model trained on CD3. Concerning area methods for immune cells quantification, strong to very strong linear correlation were found between HALO®, QuPath and our DL method.

### Univariate and multivariate analysis

In univariate analysis, for the 84 patients with complete follow-up data, RFS was significantly associated with HPV status (HR = 0.26, p=0.006) and UICC stage (HR = 3.6, p<0.001) but not with ECOG status (HR = 2.3, p=0.065) or with age (HR = 0.99, p=0.8).
We hypothesized that the estimation of immune cells density can predict the outcome of patients independently of the evaluation technique. For each biomarker, we conducted similar Cox regression analyses with the results generated by HALO®, by QuPath and by DL models. In univariate analysis, we found a statistically significant effect on RFS for CD3, CD8, CD20, and FOXP3 with all methods (DL cell methods, DL area method, HALO® and QuPath) but not for CD163 (Supplementary table 5).

Multivariate analyses identified CD8 status as an independent prognostic factor, regardless of the method used to estimate immune cell density—whether by cell-based methods with deep learning or QuPath, or area-based methods with deep learning or HALO®—after adjusting for HPV status, age, surgical treatment, ECOG performance status, and UICC stage (Table 4). FOXP3 and CD20 also demonstrated an impact on survival in three out of five quantification methods. In contrast, CD3 showed an effect on survival only when using the area-based method with HALO®. Among the methods, HALO®'s immune cell density estimation proved to be the most discriminative, showing an association of CD3, CD20, and FOXP3 statuses with recurrence-free survival (RFS). A higher density of immune cells in the tumor correlated with improved survival (Figure 4). Conversely, the cell-based method using Algorithm 1 was the least discriminative.
DISCUSSION

In the current study, we generated a pipeline combining the U-Net model and mathematical algorithms
to estimate lymphocyte and histiocyte density in a series of digitized IHC whole slides of OPSCC. We
compared these results with those of a commercial software (HALO®) and the open-source QuPath
software for quantifying immune cells. We also evaluate whether immune cells density could predict
disease prognosis in OPSCC patients treated with curative-intent radiation therapy.

Our primary objective was to develop a deep learning (DL) pipeline capable to accurately detect and
quantify immune cell density across different cell types and staining localizations in whole-slide IHC
images. For this purpose, we trained a DL model to segment immune cells stained with DAB and tested
its performance on multiple biomarkers (CD8, CD20, CD163, FOXP3), regardless of staining localization.
The pipeline achieved excellent performance scores for these markers, even though training was
conducted solely on CD3, underscoring the model's robustness for DAB-staining detection. The main
exception was FOXP3 (nuclear staining), where dark blue counterstaining led to some negative nuclei
being misclassified as positive. When tested on the external LYSTO dataset (CD3 and CD8), the pipeline
achieved satisfactory accuracy and AUC values. However, its F1-score and IoU were lower due to a
high false-positive rate and fewer true negatives. This analysis highlighted the model's strength in
identifying true DAB-positive signals, despite difficulties with negative nuclei in darker hematoxylin
counterstaining. Moreover, when we examined LYSTO's tiles more closely, we found that some
represented ink from the macroscopy step, which was not the case for our tiles, as a pathologist
specifically selected ROIs while avoiding such artifacts (Supplementary figure 1). These results
underscore the persistent challenges of domain generalization in pathology images, especially when
incorporating data from different medical centers and scanning equipment. Variability in IHC
techniques and the use of other automated platforms besides the Ventana BenchMark ULTRA from
Roche Diagnostics can influence model performance, emphasizing the limitations of our monocentric
study. Developing stain-invariant models across laboratories remains a significant challenge (21–24).
In conclusion, despite counterstaining variability, our DL model effectively detected DAB-stained areas
across diverse cell types and staining localizations. However, consistent background staining requires
further refinement to improve generalizability in multi-center applications.

In this study, we applied a semantic segmentation approach for DAB-stained images based on the U-
Net architecture, a convolutional neural network (CNN) renowned for segmentation tasks in medical
imaging (16). U-Net has been widely validated across various applications and cancer types, showing
high accuracy in segmenting immune cell markers (25,26). For instance, a recent study by Swiderska-
Chadaj et al. reported that U-Net outperformed other state-of-the-art (SOTA) deep learning models—
including fully convolutional networks (FCN)(27), You Only Look Once (YOLO) (28), and locality-
sensitive methods (LSM) (29)—in detecting CD3+ and CD8+ cells in breast, colon, and prostate cancer
samples (30). Unlike many deep learning models that rely on annotated datasets as ground truth for
training (31–34), our approach eliminated the need of manual annotations. Instead, we calibrated our
model using DAB segmentation results generated by ImageJ software, offering an effective,
annotation-free ground truth. Other researchers have reported promising results with similar
annotation-free training strategies using co-registered images for model training. For instance,
Ghahremani et al. and Bai et al. successfully trained their deep learning models with co-registered IHC
and multiplex immunofluorescence images (35,36). Similarly, Azam et al. achieved robust results using
co-registered H&E and IHC images as training pairs (37).

In our study, DAB-staining segmentation formed the basis for estimating immune cell density, utilizing
either mathematical algorithms (cell-based methods) or direct measurement of stained surface area
(area-based methods). Our approach showed a strong correlation with manual counts by pathologists,
except for CD163. We observed that CD163+ macrophages were particularly challenging to
individualize within a tile due to their dendritic morphology. In 2D sections, distinguishing dendrites
belonging to individual macrophages is challenging, raising concerns about the accuracy of the manual
ground truth. Representative examples of CD163-stained tiles are provided in **Supplementary Figure**
2. Our deep learning approach demonstrated consistency with the area-based method from the HALO® Platform and the cell-based method in the open-source software QuPath. Both HALO® and QuPath are well-established tools for immune cell density analysis(38–40). For example, Qianqian Xue et al. successfully utilized the HALO® Platform to quantify immune cell density in 44 whole-slide images (WSI) of stage I lung adenocarcinoma, identifying higher densities of CD11c+ myeloid DCs, stromal CD68+ macrophages, and stromal CD4+ T-cells as independent prognostic indicators for disease-free survival (41). Our area-based results aligned closely with those from HALO® and cell-based estimates, particularly with our second cell algorithm, underscoring the robustness of our method. While QuPath has shown efficacy in counting CD3+ and CD8+ cells on tissue microarray (TMA) samples, expressing results as the average cell count per mm² (42), our method has several advantages over both commercial and open-source platforms. A major advantage of our method is its elimination of individualized thresholding to distinguish DAB staining from background, improving reproducibility and enabling efficient analysis across multiple biomarkers. The area-based approach uniquely minimizes biases from cell clustering. In such clusters, lack of intercellular space complicates individual cell identification, which can skew results when using cell-based methods. Additionally, the area-based method is especially suited for quantifying irregularly shaped cells like dendritic cells (e.g., CD163+ macrophages) which often appear fragmented in 2D sections. This reduces the risk of misidentification, where segments of a single dendritic cell could be counted as multiple cells, as observed with the poorer performance of cell algorithm 1 in estimating CD163+ cell density. Supporting our findings, Bouzin et al. compared cell-based and area-based approaches on digitized immunostained slides, founding both methods correlated well with pathologist scores for CD8 and CD20 markers across multiple cancer types (43). Notably, they identified several advantages of the area-based method: it was faster, less influenced by tissue composition, and unaffected by minor artifacts such as folds (if < 5% of the slide area). This highlights the robustness of the area-based approach in accommodating preparation artifacts that can otherwise complicate analysis.
This work has the significant quality of studying immune cells density obtained with DL and comparing the results with patient prognosis. This practical approach is sometimes lacking in other studies that present DL models and focus on the technical aspects of DL. We have demonstrated an impact on RFS of the infiltration of T-cells, particularly cytotoxic T-cells (CD8+), and B-cells (CD20+). The size of the cohort used in this study is one of the limitations of the data obtained, and these promising results will need to be confirmed on larger cohorts. However, those results are in line with previous studies which found a positive effect of tumor-infiltrating lymphocytes (TILs) in HNSCC (9,44).

We have shown that high tumor infiltration by FOXP3+ cells is associated with better RFS in univariate and multivariate analyses, regardless of the evaluation method (area method or cell method). CD4+CD25+ forkhead box p3 (FOXP3)+ T-cells are an immunosuppressive T-cell subset capable of tolerizing the immune system and are therefore referred to as regulatory T-cells (Tregs). They are recognized as a subtype of immune cells able to promote tumor growth by dampening other T cells' activity (45). They can, for instance, inhibit the anti-tumor Tc1/TH1 response. Despite this "pro-tumor" role, FOXP3+ Tregs were associated with a favorable outcome in several types of solid tumors (46) as well as in HNSCC (47). Thus, the role of Tregs in tumors is not fully understood and is influenced by various factors such as tumor site, molecular subtype of Treg cells, and tumor stage (48).

CD163 was the only biomarker that showed no impact on prognosis. Tumor-associated macrophages (TAMs) include two phenotypes with different functions: M1 and M2. M1 macrophages suppress tumor development, while M2 macrophages promote tumor development. IHC-staining of CD163 features M2-like macrophages. Our results differ from the literature on M2-like macrophages in HNSCC. Troiano et al. conducted a systematic review and meta-analysis concerning the prognostic significance of CD163+ macrophages in HNSCCC and showed that the abundance of CD163+ TAM correlates with poor survival in HNSCC patients in terms of OS and PFS but not RFS (49). In another systematic review, Kumar et al. showed that a high number of M2-like macrophages is associated with more invasive and larger tumors, a higher rate of nodal metastasis, and a higher risk of vascular and
lymphatic invasion (50). In our cohort, where all patients were treated with Intensity-Modulated Radiation Therapy (IMRT), a plausible hypothesis is that the treatment's impact on macrophages remodeling is significant. Recent research indicates that macrophage polarization can shift between M1 and M2 phenotypes depending on the radiotherapy dose administered (51,52). Since patient samples were collected prior to any treatment, we hypothesize that, in some cases harboring numerous M2 macrophages, the plasticity of these cells may have changed towards M1 macrophages, subsequently altering their influence on tumor growth. This hypothesis needs to be further verified through comparative analysis with post-treatment samples.

This study presents a novel application of deep learning (DL) for the automated quantification of immune cell densities in OPSCC using IHC-stained markers. We demonstrated that our DL-based segmentation model delivers accurate and generalizable results across different immune cell types. By comparing cell-based and area-based methods, we identified the latter as more effective for prognostic evaluation. Crucially, our findings revealed that densities of CD3+, CD8+, CD20+, and FOXP3+ cells are significant prognostic markers for disease recurrence, unlike CD163+ cells. These results not only validate the prognostic relevance of immune cell densities in OPSCC but also highlight the potential of DL tools to advance research on the tumor immune microenvironment (TIME). Beyond their prognostic utility, these algorithms enable co-quantification of immune cell populations and their spatial relationships with tumor cells, paving the way for predictive models in OPSCC and other cancers. Future studies with larger cohorts will refine and validate these findings.
SUPPLEMENTARY INFORMATION

Supplementary information is available at Laboratory Investigation's website.
## ETHICS APPROVAL / CONSENT TO PARTICIPATE

Ethics approval for this study was obtained from the Institutional Review Board of the Centre Jean Perrin, Clermont-Ferrand, France. The approval reference number is IRB 00013468. Informed consent was obtained from all participants involved in the study. This study was conducted in accordance with the ethical standards outlined in the Declaration of Helsinki.
AUTHOR CONTRIBUTION STATEMENT

F.B., E.C., C.D. and J.B. identified patients and insured clinical data collection. F.B., M.K. and E.B. provided technical and material support. F.B., E.C., J.B. and F.P-L contributed to the acquisition of data. F.B., V.L.L. F.L.L and O.S. developed the methodology. F.B., L.V.L, E.C. and I.M. provided statistical analysis. E.C., J.B., F.P.L. and O.S. supervised the project. All authors read and approved the final paper.
FUNDING STATEMENT

This work was supported by the Clermont Auvergne University (Emergence program, CAP20-25) and
by Centre Jean PERRIN. F.L.L. and V.L.L. are supported by grants from BRIC Bordeaux and Fondation
Bergonié.
DATA AVAILABILITY STATEMENT

The authors confirm that the data supporting the findings of this study are available within the article
and the supplementary data. The images of the 1,000 CD3-stained tiles, along with the corresponding
cell counts provided by observers 1 and 2, as well as the code for our deep learning model can be found
on the Open Microscopy Environment (OME) platform at
https://demo.openmicroscopy.org/webclient/?show=dataset-2171 and on github
(https://github.com/linhlv-dl/Quantitative_biomarkercells).
FIGURE LEGENDS

FIG. 1.

Title: Pipelines for immune cells density estimation with deep learning model (1A) and HALO® platform (1B)

Caption: Fig. 1A. Pipeline of the deep learning method to estimate immune cells density. The examples shown in this image feature CD3 immunohistochemical slides at 0.5x magnification and in 224x224 pixel tiles.

The first step corresponded to the pre-processing of the WSI. To facilitate efficient processing of WSI, which can reach resolutions as high as 100,000 x 100,000 pixels at 40X magnification, initial pre-processing involved selecting region of interest (ROI, specifically, tumor regions) within each image. Subsequently, tiles were extracted from these selected regions and then classified into one of four categories: A = the tile has more than 80% tissue, B = the tile has more than 10% tissue and less than 80% tissue, C = the tile has more than 0% tissue and less than 10% tissue, and D = the tile has no tissue at all. The percentage of tissue in a tile was obtained by dividing the number of tissue pixels by the total pixels in the tile. Based on the percentage of tissue, we considered the tiles that contain more than 10% tissue for our analysis. The second step corresponded to the segmentation. The selected tiles served as input for the modified U-Net model. In the third step mathematical formula were applied to the output in order to estimate immune cell density.

Fig. 1B. Workflow of immune cells density estimation with HALO® platform. Selection, classification, and analysis steps using HALO® platform are represented (CD3, 2.0x magnification). The yellow oval is drawn manually and indicates the region of interest on the slide (the 'classified area'). For the classification and analysis steps, mark-up masks applied to the image are shown. Blue areas correspond to optically empty spaces, green areas denote artifact regions including folds or staining artifacts, yellow areas represent the tumor region, which is the area considered for the results data, and red areas correspond to DAB staining.
Fig.2. Performance Evaluation and Visualization of U-Net Model Predictions Across Immune Markers

ROC curves (A) and confusion matrix (B) of the U-Net model trained on CD3 and applied to 10 WSIs stained with CD8, CD20, FOXP3, and CD163, as well as 20,000 tiles from the LYSTO dataset. (C) Illustrates the U-Net model's DAB staining prediction results for CD3, CD8, CD20, CD163, and FOXP3. One example tiles (224 x 224 pixels) per immune marker is shown. Prediction images correspond to the model output, while overlap images represent manual annotations of the input tiles. TP = True Positive. FN = False Negative. FP = False Positive.

Fig. 3. Observer Concordance and Algorithmic Validation for Immune Cell Quantification Across Biomarkers

(A) Comparison of CD3-positive cell counts evaluated by two independent observers (FB and EC) across 1,000 tiles, shown through a linear correlation plot and a Bland-Altman plot. The Bland-Altman plot illustrates agreement between observers, with the x-axis representing the mean CD3+ cell count and the y-axis showing the difference between their counts. The solid black line represents the mean difference, while gray lines indicate the limits of agreement (mean difference ± 1.91 standard deviations).

(B) Examples of cell aggregate management by two algorithmic methods. Three representative tiles (224 x 224 pixels) stained for CD20 are shown, along with the corresponding cell count estimates from each algorithm.

(C) Bland-Altman analysis of concordance between manual cell counts and predictions by two algorithmic methods for CD3, CD8, CD20, FOXP3, and CD163. The gray lines indicate ±1.91 standard deviations from the mean, while the black line denotes the overall mean difference. For CD3 and CD20,
the red line highlights the mean cutoff point, beyond which data dispersion and outliers increase. Tile counts analyzed: CD3 (1,000 tiles), CD20 (638 tiles), CD8 (532 tiles), FOXP3 (531 tiles), and CD163 (546 tiles).

$$AL1 = \text{Algorithm 1.} \quad AL2 = \text{Algorithm 2}$$

Fig. 4.

Title: Comparison of the effect on the recurrence free survival (RFS) of 5 methods for density cell estimation.

Caption: Estimated survival curves for fixed specific values of the continuous biomarker covariate, obtained by g-computation for the multivariable Cox models (adjusted for HPV status, surgery status, Age, ECOG status, and UICC stage). Three specific values are shown, corresponding to the minimum (in red), median (in blue), and maximum (in green) biomarker values. The biomarkers are natural log-transformed and computed using three methods: HALO_AREA = area method with HALO® (dotted line); DL_area = area method with deep learning (dashed line); DL_AL2 = cell method with deep learning using algorithm 2 (solid line); QP =cell method with QuPath. The following biomarkers are consecutively represented in the subfigures: CD3, CD8, CD20, FOXP3, and CD163. (4A) represents all area method. (4B) represents all cell method.
REFERENCES

1. Joyce JA, Fearon DT. T cell exclusion, immune privilege, and the tumor microenvironment. Science. 2015 Apr 3;348(6230):74-80.

2. Fridman WH, Pagès F, Sautès-Fridman C, Galon J. The immune contexture in human tumours: impact on clinical outcome. Nat Rev Cancer. 2012 Mar 15;12(4):298-306.

3. Finn OJ. A Believer's Overview of Cancer Immunosurveillance and Immunotherapy. The Journal of Immunology. 2018 Jan 15;200(2):385-91.

4. Dunn GP, Bruce AT, Ikeda H, Old LJ, Schreiber RD. Cancer immunoediting: from immunosurveillance to tumor escape. Nature Immunology. 2002 Nov;3(11):991-8.

5. Miyauchi S, Kim SS, Pang J, Gold KA, Gutkind JS, Califano JA, et al. Immune Modulation of Head and Neck Squamous Cell Carcinoma and the Tumor Microenvironment by Conventional Therapeutics. Clin Cancer Res. 2019 Jul 15;25(14):4211-23.

6. Mellman I, Chen DS, Powles T, Turley SJ. The cancer-immunity cycle: Indication, genotype, and immunotype. Immunity. 2023 Oct 10;56(10):2188-205.

7. Ruffin AT, Li H, Vujanovic L, Zandberg DP, Ferris RL, Bruno TC. Improving head and neck cancer therapies by immunomodulation of the tumour microenvironment. Nature reviews Cancer. 2022 Dec 1;23(3):173.

8. Heller G, Fuereder T, Grandits AM, Wieser R. New perspectives on biology, disease progression, and therapy response of head and neck cancer gained from single cell RNA sequencing and spatial transcriptomics. Oncol Res. 2023;32(1):1-17.

9. de Ruiter EJ, Ooft ML, Devriese LA, Willems SM. The prognostic role of tumor infiltrating T-lymphocytes in squamous cell carcinoma of the head and neck: A systematic review and meta-analysis. Oncoimmunology. 2017;6(11):e1356148.

10. Klauschen F, Müller KR, Binder A, Bockmayr M, Hägele M, Seegerer P, et al. Scoring of tumor-infiltrating lymphocytes: From visual estimation to machine learning. Semin Cancer Biol. 2018 Oct;52(Pt 2):151-7.

11. Bruce C, Prassas I, Mokhtar M, Clarke B, Youssef E, Wang C, et al. Transforming diagnostics: The implementation of digital pathology in clinical laboratories. Histopathology. 2024 Mar 22;

12. Jiao Y, van der Laak J, Albarqouni S, Li Z, Tan T, Bhalerao A, et al. LYSTO: The Lymphocyte Assessment Hackathon and Benchmark Dataset. IEEE J Biomed Health Inform. 2024 Mar;28(3):1161-72.

13. Huang S, Tang N, Mak WB, Erickson B, Salituro J, Li Y, et al. Principles and analytical performance of Abbott RealTime High Risk HPV test. J Clin Virol. 2009 Jul;45 Suppl 1:S13-17.

14. Gustav M, Reitsam NG, Carrero ZI, Loeffler CML, van Treeck M, Yuan T, et al. Deep learning for dual detection of microsatellite instability and POLE mutations in colorectal cancer histopathology. NPJ Precis Oncol. 2024 May 23;8(1):115.
15. Kiehl L, Kuntz S, Höhn J, Jutzi T, Krieghoff-Henning E, Kather JN, et al. Deep learning can predict lymph node status directly from histology in colorectal cancer. Eur J Cancer. 2021 Nov;157:464-73.

16. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing; 2015. p. 234–41. (Lecture Notes in Computer Science).

17. Le VL, Saut O. RRc-UNet 3D for Lung Tumor Segmentation from CT Scans of Non-Small Cell Lung Cancer Patients. In: ICCV CVAMD 2023 - Workshop of International Conference on Computer Vision [Internet]. Paris, France; 2023. p. 2316–25. Available from: https://inria.hal.science/hal-04236510

18. Ash NF, Massengill MT, Harmer L, Jafri A, Lewin AS. Automated segmentation and analysis of retinal microglia within ImageJ. Exp Eye Res. 2021 Feb;203:108416.

19. Hollandi R, Diósdi Á, Hollandi G, Moshkov N, Horváth P. AnnotatorJ: an ImageJ plugin to ease hand annotation of cellular compartments. Mol Biol Cell. 2020 Sep 15;31(20):2179–86.

20. Evans JD. Straightforward statistics for the behavioral sciences. Belmont, CA, US: Thomson Brooks/Cole Publishing Co; 1996. xxii, 600 p. (Straightforward statistics for the behavioral sciences).

21. Khened M, Kori A, Rajkumar H, Krishnamurthi G, Srinivasan B. A generalized deep learning framework for whole-slide image segmentation and analysis. Sci Rep. 2021 Jun 2;11(1):11579.

22. Stacke K, Eilertsen G, Unger J, Lundstrom C. Measuring Domain Shift for Deep Learning in Histopathology. IEEE J Biomed Health Inform. 2021 Feb;25(2):325–36.

23. Lafarge MW, Pluim JPW, Eppenhof KAJ, Veta M. Learning Domain-Invariant Representations of Histological Images. Front Med (Lausanne). 2019;6:162.

24. Aubreville M, Bertram C, Veta M, Klopfleisch R, Stathonikos N, Breininger K, et al. Quantifying the Scanner-Induced Domain Gap in Mitosis Detection [Internet]. arXiv; 2021 [cited 2024 Oct 31]. Available from: http://arxiv.org/abs/2103.16515

25. Makhlouf Y, Singh VK, Craig S, McArdle A, French D, Loughrey MB, et al. True-T - Improving T-cell response quantification with holistic artificial intelligence based prediction in immunohistochemistry images. Comput Struct Biotechnol J. 2024 Dec;23:174–85.

26. Lu Z, Xu S, Shao W, Wu Y, Zhang J, Han Z, et al. Deep-Learning-Based Characterization of Tumor-Infiltrating Lymphocytes in Breast Cancers From Histopathology Images and Multiomics Data. JCO Clin Cancer Inform. 2020 May;4:480–90.

27. Shelhamer E, Long J, Darrell T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell. 2017 Apr;39(4):640–51.

28. Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection [Internet]. arXiv; 2016 [cited 2024 May 30]. Available from: http://arxiv.org/abs/1506.02640
29. Sirinukunwattana K, Ahmed Raza SE, Yee-Wah Tsang, null, Snead DRJ, Cree IA, Rajpoot NM. Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images. IEEE Trans Med Imaging. 2016 May;35(5):1196-206.

30. Swiderska-Chadaj Z, Pinckaers H, van Rijthoven M, Balkenhol M, Melnikova M, Geessink O, et al. Learning to detect lymphocytes in immunohistochemistry with deep learning. Med Image Anal. 2019 Dec;58:101547.

31. Singh VK, Sarker MMK, Makhlouf Y, Craig SG, Humphries MP, Loughrey MB, et al. ICOSeg: Real-Time ICOS Protein Expression Segmentation from Immunohistochemistry Slides Using a Lightweight Conv-Transformer Network. Cancers (Basel). 2022 Aug 13;14(16):3910.

32. Saltz J, Gupta R, Hou L, Kurc T, Singh P, Nguyen V, et al. Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. Cell Rep. 2018 Apr 3;23(1):181-193.e7.

33. Aprupe L, Litjens G, Brinker TJ, van der Laak J, Grabe N. Robust and accurate quantification of biomarkers of immune cells in lung cancer micro-environment using deep convolutional neural networks. PeerJ. 2019 Apr 10;7:e6335.

34. Linder N, Taylor JC, Colling R, Pell R, Alveyn E, Joseph J, et al. Deep learning for detecting tumour-infiltrating lymphocytes in testicular germ cell tumours. J Clin Pathol. 2019 Feb;72(2):157-64.

35. Ghahremani P, Li Y, Kaufman A, Vanguri R, Greenwald N, Angelo M, et al. Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification. Nat Mach Intell. 2022 Apr;4(4):401-12.

36. Bai B, Wang H, Li Y, de Haan K, Colonnese F, Wan Y, et al. Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning. BME Front. 2022;2022:9786242.

37. Azam AB, Wee F, Väyrynen JP, Yim WWY, Xue YZ, Chua BL, et al. Training immunophenotyping deep learning models with the same-section ground truth cell label derivation method improves virtual staining accuracy. Front Immunol. 2024;15:1404640.

38. Zwing N, Failmezger H, Ooi CH, Hibar DP, Cañamero M, Gomes B, et al. Analysis of Spatial Organization of Suppressive Myeloid Cells and Effector T Cells in Colorectal Cancer-A Potential Tool for Discovering Prognostic Biomarkers in Clinical Research. Front Immunol. 2020;11:550250.

39. Sobottka B, Nowak M, Frei AL, Haberecker M, Merki S, Tumor Profiler consortium, et al. Establishing standardized immune phenotyping of metastatic melanoma by digital pathology. Lab Invest. 2021 Dec;101(12):1561-70.

40. Stirling DR, Swain-Bowden MJ, Lucas AM, Carpenter AE, Cimini BA, Goodman A. CellProfiler 4: improvements in speed, utility and usability. BMC Bioinformatics. 2021 Sep 10;22(1):433.

41. Xue Q, Wang Y, Zheng Q, Chen L, Lin Y, Jin Y, et al. Prognostic value of tumor immune microenvironment factors in patients with stage I lung adenocarcinoma. Am J Cancer Res. 2023;13(3):950-63.

42. Lee K, Lockhart JH, Xie M, Chaudhary R, Slebos RJC, Flores ER, et al. Deep Learning of Histopathology Images at the Single Cell Level. Front Artif Intell. 2021;4:754641.
43. Bouzin C, Saini ML, Khaing KK, Ambroise J, Marbaix E, Grégoire V, et al. Digital pathology: elementary, rapid and reliable automated image analysis. Histopathology. 2016 May;68(6):888-96.

44. Hendry S, Salgado R, Gevaert T, Russell PA, John T, Thapa B, et al. Assessing Tumor-Infiltrating Lymphocytes in Solid Tumors: A Practical Review for Pathologists and Proposal for a Standardized Method from the International Immuno-Oncology Biomarkers Working Group: Part 2: TILs in Melanoma, Gastrointestinal Tract Carcinomas, Non-Small Cell Lung Carcinoma and Mesothelioma, Endometrial and Ovarian Carcinomas, Squamous Cell Carcinoma of the Head and Neck, Genitourinary Carcinomas, and Primary Brain Tumors. Adv Anat Pathol. 2017 Nov;24(6):311-35.

45. Saleh R, Elkord E. FoxP3+ T regulatory cells in cancer: Prognostic biomarkers and therapeutic targets. Cancer Lett. 2020 Oct 10;490:174-85.

46. Gentles AJ, Newman AM, Liu CL, Bratman SV, Feng W, Kim D, et al. The prognostic landscape of genes and infiltrating immune cells across human cancers. Nat Med. 2015 Aug;21(8):938-45.

47. Lei Y, Xie Y, Tan YS, Prince ME, Moyer JS, Nör J, et al. Telltale tumor infiltrating lymphocytes (TIL) in oral, head & neck cancer. Oral Oncol. 2016;61:159-65.

48. Shang B, Liu Y, Jiang S juan, Liu Y. Prognostic value of tumor-infiltrating FoxP3+ regulatory T cells in cancers: a systematic review and meta-analysis. Sci Rep. 2015 Oct 14;5:15179.

49. Troiano G, Caponio VCA, Adipietro I, Tepedino M, Santoro R, Laino L, et al. Prognostic significance of CD68+ and CD163+ tumor associated macrophages in head and neck squamous cell carcinoma: A systematic review and meta-analysis. Oral Oncol. 2019 Jun;93:66-75.

50. Kumar AT, Knops A, Swendseid B, Martinez-Outschoom U, Harshyne L, Philp N, et al. Prognostic Significance of Tumor-Associated Macrophage Content in Head and Neck Squamous Cell Carcinoma: A Meta-Analysis. Front Oncol. 2019 Jul 23;9:656.

51. Liu S, Wang W, Hu S, Jia B, Tuo B, Sun H, et al. Radiotherapy remodels the tumor microenvironment for enhancing immunotherapeutic sensitivity. Cell Death Dis. 2023 Oct 13;14(10):679.

52. Guo S, Yao Y, Tang Y, Xin Z, Wu D, Ni C, et al. Radiation-induced tumor immune microenvironments and potential targets for combination therapy. Signal Transduct Target Ther. 2023 May 19;8(1):205.
## ACKNOWLEDGEMENTS

We thank the Clinical Research Division, Clinical Research & Innovation Division of Centre Jen PERRIN, and particularly Catherine Abrial for her assistance. We also thank Anne Pham LEDARD and the INSERM TRIO2 team (BRIC, Bordeaux) for their help in the use of HALO® Image Analysis Platform.