The Liver Tumor Segmentation Benchmark (LiTS)
Creators: name: Bilic, Patrick
name: Christ, Patrick
name: Li, Hongwei Bran
name: Vorontsov, Eugene
name: Ben-Cohen, Avi
name: Kaissis, Georgios
name: Szeskin, Adi
name: Jacobs, Colin
name: Mamani, Gabriel Efrain Humpire
name: Chartrand, Gabriel
name: Lohöfer, Fabian
name: Holch, Julian Walter
name: Sommer, Wieland
name: Hofmann, Felix
name: Hostettler, Alexandre
name: Lev-Cohain, Naama
name: Drozdzal, Michal
name: Amitai, Michal Marianne
name: Vivantik, Refael
name: Sosna, Jacob
name: Ezhov, Ivan
name: Sekuboyina, Anjany
name: Navarro, Fernando
name: Kofler, Florian
name: Paetzold, Johannes C.
name: Shit, Suprosanna
name: Hu, Xiaobin
name: Lipková, Jana
name: Rempfler, Markus
name: Piraud, Marie
name: Kirschke, Jan
name: Wiestler, Benedikt
name: Zhang, Zhiheng
name: Hülsemeyer, Christian
name: Beetz, Marcel
name: Ettlinger, Florian
name: Antonelli, Michela
name: Bae, Woong
name: Bellver, Míriam
name: Bi, Lei
name: Chen, Hao
name: Chlebus, Grzegorz
name: Dam, Erik B.
name: Dou, Qi
name: Fu, Chi-Wing
name: Georgescu, Bogdan
name: Giró-i-Nieto, Xavier
name: Gruen, Felix
name: Han, Xu
name: Heng, Pheng-Ann
name: Hesser, Jürgen
name: Moltz, Jan Hendrik
name: Igel, Christian
name: Isensee, Fabian
name: Jäger, Paul
name: Jia, Fucang
name: Kaluva, Krishna Chaitanya
name: Khened, Mahendra
name: Kim, Ildoo
name: Kim, Jae-Hun
name: Kim, Sungwoong
name: Kohl, Simon
name: Konopczynski, Tomasz
name: Kori, Avinash
name: Krishnamurthi, Ganapathy
name: Li, Fan
name: Li, Hongchao
name: Li, Junbo
name: Li, Xiaomeng
name: Lowengrub, John
name: Ma, Jun
name: Maier-Hein, Klaus
name: Maninis, Kevis-Kokitsi
name: Meine, Hans
name: Merhof, Dorit
name: Pai, Akshay
name: Perslev, Mathias
name: Petersen, Jens
name: Pont-Tuset, Jordi
name: Qi, Jin
name: Qi, Xiaojuan
name: Rippel, Oliver
name: Roth, Karsten
name: Sarasua, Ignacio
name: Schenk, Andrea
name: Shen, Zengming
name: Torres, Jordi
name: Wachinger, Christian
name: Wang, Chunliang
name: Weninger, Leon
name: Wu, Jianrong
name: Xu, Daguang
name: Yang, Xiaoping
name: Yu, Simon Chun-Ho
name: Yuan, Yading
name: Yu, Miao
name: Zhang, Liping
name: Cardoso, Jorge
name: Bakas, Spyridon
name: Braren, Rickmer
name: Heinemann, Volker
name: Pal, Christopher
name: Tang, An
name: Kadoury, Samuel
name: Soler, Luc
name: van Ginneken, Bram
name: Greenspan, Hayit
name: Joskowicz, Leo
name: Menze, Bjoern
Publisher: name: arXiv
Year: 2019
Subjects: subject: Computer Vision and Pattern Recognition (cs.CV)
subject: FOS: Computer and information sciences
subject: FOS: Computer and information sciences

description: In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in \url{http://medicaldecathlon.com/}. In addition, both data and online evaluation are accessible via \url{www.lits-challenge.com}.
description: Patrick Bilic, Patrick Christ, Hongwei Bran Li, and Eugene Vorontsov made equal contributions to this work. Published in Medical Image Analysis