OpenKBP-Opt: An international and reproducible evaluation of 76 knowledge-based planning pipelines
Creators: name: Babier, Aaron
name: Mahmood, Rafid
name: Zhang, Binghao
name: Alves, Victor G. L.
name: Barragán-Montero, Ana Maria
name: Beaudry, Joel
name: Cardenas, Carlos E.
name: Chang, Yankui
name: Chen, Zijie
name: Chun, Jaehee
name: Diaz, Kelly
name: Eraso, Harold David
name: Faustmann, Erik
name: Gaj, Sibaji
name: Gay, Skylar
name: Gronberg, Mary
name: Guo, Bingqi
name: He, Junjun
name: Heilemann, Gerd
name: Hira, Sanchit
name: Huang, Yuliang
name: Ji, Fuxin
name: Jiang, Dashan
name: Giraldo, Jean Carlo Jimenez
name: Lee, Hoyeon
name: Lian, Jun
name: Liu, Shuolin
name: Liu, Keng-Chi
name: Marrugo, José
name: Miki, Kentaro
name: Nakamura, Kunio
name: Netherton, Tucker
name: Nguyen, Dan
name: Nourzadeh, Hamidreza
name: Osman, Alexander F. I.
name: Peng, Zhao
name: Muñoz, José Darío Quinto
name: Ramsl, Christian
name: Rhee, Dong Joo
name: Rodriguez, Juan David
name: Shan, Hongming
name: Siebers, Jeffrey V.
name: Soomro, Mumtaz H.
name: Sun, Kay
name: Hoyos, Andrés Usuga
name: Valderrama, Carlos
name: Verbeek, Rob
name: Wang, Enpei
name: Willems, Siri
name: Wu, Qi
name: Xu, Xuanang
name: Yang, Sen
name: Yuan, Lulin
name: Zhu, Simeng
name: Zimmermann, Lukas
name: Moore, Kevin L.
name: Purdie, Thomas G.
name: McNiven, Andrea L.
name: Chan, Timothy C. Y.
Publisher: name: arXiv
Year: 2022
Subjects: subject: Medical Physics (physics.med-ph)
subject: Artificial Intelligence (cs.AI)
subject: Computer Vision and Pattern Recognition (cs.CV)
subject: FOS: Physical sciences
subject: FOS: Physical sciences
subject: FOS: Computer and information sciences
subject: FOS: Computer and information sciences

description: We establish an open framework for developing plan optimization models for knowledge-based planning (KBP) in radiotherapy. Our framework includes reference plans for 100 patients with head-and-neck cancer and high-quality dose predictions from 19 KBP models that were developed by different research groups during the OpenKBP Grand Challenge. The dose predictions were input to four optimization models to form 76 unique KBP pipelines that generated 7600 plans. The predictions and plans were compared to the reference plans via: dose score, which is the average mean absolute voxel-by-voxel difference in dose a model achieved; the deviation in dose-volume histogram (DVH) criterion; and the frequency of clinical planning criteria satisfaction. We also performed a theoretical investigation to justify our dose mimicking models. The range in rank order correlation of the dose score between predictions and their KBP pipelines was 0.50 to 0.62, which indicates that the quality of the predictions is generally positively correlated with the quality of the plans. Additionally, compared to the input predictions, the KBP-generated plans performed significantly better (P&lt;0.05; one-sided Wilcoxon test) on 18 of 23 DVH criteria. Similarly, each optimization model generated plans that satisfied a higher percentage of criteria than the reference plans. Lastly, our theoretical investigation demonstrated that the dose mimicking models generated plans that are also optimal for a conventional planning model. This was the largest international effort to date for evaluating the combination of KBP prediction and optimization models. In the interest of reproducibility, our data and code is freely available at https://github.com/ababier/open-kbp-opt.
description: 19 pages, 7 tables, 6 figures