<figure><img src="image_1.png" /></figure>

HAL
open science

# Prediction of Oncotype DX using DRFs

Romain Pic, Zeina Al Masry, Clément Dombry, Christine Devalland

► To cite this version:

Romain Pic, Zeina Al Masry, Clément Dombry, Christine Devalland. Prediction of Oncotype DX using DRFs. Journées Math Bio Santé 2022, Oct 2022, Besançon, France. . hal-03894286

HAL Id: hal-03894286

https://hal.science/hal-03894286

Submitted on 12 Dec 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
## Introduction

Oncotype DX (ODX) is a multi-gene expression signature for **breast cancer** that provides prognostic and predictive breast cancer recurrence information for estrogen receptor (ER)-positive and HER2 negative patients. This test can predict if a chemotherapy treatment would be beneficial. However, this test is **expensive** and several studies have shown its link with certain clinico-pathological data. The result of the test is a score between 0 and 100 and guides clinicians regarding **prescription of chemotherapy** (e.g. for score higher than 25).

**Aim:** Predict the ODX score based on histopathological variables.

**Proposed methodology:** Probabilistic forecast using Distributional Regression Forest (DRF; [1])

## Presentation of the cohort

The cohort is a retrospective study between 2012 and 2019 with 334 cases that underwent ODX assay from three hospitals: Besançon, Belfort and Dijon. All patients have ER-positive and HER2-negative early breast cancer. We used **9 variables** for the prediction of the ODX: age, tumor size, SBR grade, Nottingham grade, ER status, PR status, Ki67 index proliferation, P53 protein and lymph node status. In order to remove missing values of ODX, we had to remove one patient and we finally keep **333 patients**.

### Flowchart of Distributional Regression Forests

<figure><img src="image_3.png" /><figcaption>Figure 1. Flowchart of distributional random forest. Starting from the training data, a large number of subsamples are chosen and binary trees are constructed on each subsamples; the neighborhoods and weights at the point to predict (cross) are computed in each tree and then averaged so as to give the forest weights; the predictive distribution corresponds to the weighted sample of the original training data with these forest weights.</figcaption></figure>

## Outputs of DRF

DRF is a powerful tool and is able to cover numerous methods such as classification, mean regression and quantile regression. Moreover, DRFs assess the uncertainty of a prediction.

* **Histogram** of the predictive distribution;

* **Probability** of classes of interest;

* **Mean** or **median** prediction;

* **Uncertainty** associated to the prediction (standard deviation or confidence interval);

* **Similar/influential patients** for meaningful and informative comparison. (very important for practitioners)

<figure><img src="image_4.png" /><figcaption>Figure 2. Applications of DRF.</figcaption></figure>

## Evaluation of the predictive distribution

The Continuous Ranked Probability Score (CRPS; [5]) is a scoring rule used to compare a predictive distribution *F* and the outcome *y*. The CRPS is used in Figure 3 to evaluate the performances of DRF for each patient in an out-of-bag prediction. The CRPS evaluates the dispersion and the bias; thus, a **higher CRPS** correspond to **higher bias** and/or **higher dispersion** of the predictive distribution.

<figure><img src="image_5.png" /><figcaption>Figure 3. We highlight three cases : a "good" prediction, a "medium" prediction and a "bad" prediction associated with low, medium and high values of CRPS respectively. The red dashed line shows the true ODX, the regions in green and red corresponds to *ODX* ≤ 25 and *ODX* > 25 respectively and the numbers at the center of these regions are the predicted probabilities of each class.</figcaption></figure>

## Comparison with state-of-the-art techniques

<table><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="2">Predicted</th></tr><tr><th>ODX ≤ 25</th><th>ODX > 25</th></tr></thead><tbody><tr><th rowspan="2">True</th><th>ODX ≤ 25</th><td>231</td><td>20</td></tr><tr><th>ODX > 25</th><td>49</td><td>33</td></tr></tbody></table>

<table><caption>Table 1. Evaluation of DRF with classification diagnostics. Confusion matrix (left) and standard metrics (right).</caption><tr><td>Accuracy</td><td>79.3%</td></tr><tr><td>Sensitivity</td><td>92.0%</td></tr><tr><td>Specificity</td><td>40.3%</td></tr><tr><td>Positive Predictive Value</td><td>82.5%</td></tr><tr><td>Negative Predictive Value</td><td>62.3%</td></tr><tr><td>F1-score</td><td>0.870</td></tr><tr><td>Area Under Curve</td><td>0.759</td></tr></table>

<table><caption>Table 2. Comparison (on different datasets) of the DRF method with 3 other methods for the 2-class classification problem (ODX ≤ 25 and ODX > 25). The methods are taken from Kim et al. (2019), Orucevic et al. (2019) and Pawloski et al. (2021) [4, 6, 7]. The high class is taken as *positive*.</caption><thead><tr><th rowspan="2">Study</th><th rowspan="2">Data set<br>(n<sub>train</sub>, n<sub>test</sub>, p)</th><th>Accuracy</th><th>Sensitivity</th><th>Specificity</th><th>AUC</th></tr><tr><th>(%)</th><th>(%)</th><th>(%)</th><th></th></tr></thead><tbody><tr><td>Kim et al. (2019)</td><td>(208, 76, 20)</td><td>88.0</td><td>17.0</td><td>96.0</td><td>0.917</td></tr><tr><td>Orucevic et al. (2019)</td><td>(65, 754, 18, 585, 5)</td><td>87.2</td><td>18.3</td><td>99.2</td><td>0.81</td></tr><tr><td>Pawloski et al. (2021)</td><td>(2587, 1184, 6)</td><td>96.3</td><td>48.3</td><td>96.3</td><td>NA</td></tr><tr><td>DRF</td><td>(333, OOB, 9)</td><td>79.3</td><td>40.2</td><td>92.0</td><td>0.759</td></tr></tbody></table>

<table><caption>Table 3. Comparison (on different datasets) of the DRF method with 2 other methods (Hou et al. (2017) and Baltres et al. (2017) [3, 2]) for the 3-class classification problem: ODX ≤ 18 (low), 18 < ODX ≤ 30 (medium) and ODX > 30 (high). The sensitivity, specificity, PPV, NPV and AUC are given only for the lower class.</caption><thead><tr><th>Study</th><th>Acc.</th><th>Sensi.</th><th>Speci.</th><th>PPV</th><th>NPV</th><th>AUC</th></tr></thead><tbody><tr><td>Hou et al. (2017)</td><td>68.7</td><td>85.7</td><td>41.4</td><td>72.6</td><td>61.5</td><td>NA</td></tr><tr><td>Baltres et al. (2017)</td><td>46.3</td><td>55.0</td><td>78.0</td><td>58.3</td><td>62.4</td><td>0.63</td></tr><tr><td>DRF</td><td>57.7</td><td>58.5</td><td>83.6</td><td>87.0</td><td>51.9</td><td>0.748</td></tr></tbody></table>

## Conclusion

* Novel approach: **distribution prediction**;

* Various outputs providing information for practitioners (e.g. **similar profiles**);

* **Performance similar to state-of-the-art** techniques in terms of classification;

* **Explainable** and transparent method;

* Operational clinical **RShiny** app for practitioners.

* **Perspectives:** robustness to missing values and noise.

* **Limitation:** predictions are only as good as the training dataset they are based on.

[1] S. Athey et al. "Generalized random forests". In: *The Annals of Statistics* 47.2 (2019).

[2] A. Baltres et al. "Prediction of Oncotype DX recurrence score using deep multi-layer perceptrons in estrogen receptor-positive, HER2-negative breast cancer". In: *Breast Cancer* 27.5 (2020).

[3] Y. Hou et al. "Comparison of Oncotype DX With Modified Magee Equation Recurrence Scores in Low-Grade Invasive Carcinoma of Breast". In: *American Journal of Clinical Pathology* 148.2 (2017).

[4] I. Kim et al. "A predictive model for high/low risk group according to oncopype DX recurrence score using machine learning". In: *European Journal of Surgical Oncology* 45.2 (2019).

[5] J. E. Matheson and R. L. Winkler. "Scoring Rules for Continuous Probability Distributions". In: *Management Science Series A-theory* 22.10 (1976).

[6] A. Orucevic et al. "Nomogram update based on TAILORx clinical trial results - Oncotype DX breast cancer recurrence score can be predicted using clinicopathologic data". In: *The Breast* 46 (2019).

[7] K. R. Pawloski et al. "Supervised machine learning model to predict oncopype DX risk category in patients over age 50". In: *Breast Cancer Research and Treatment* 191.2 (2021).