MICCAI Learn2Learn Challenge
Creators: name: Woerner, Stefano
name: Susu Sun
name: Baumgartner, Christian
name: Küstner, Thomas
name: Van Nguyen, Hien
Publisher: name: Zenodo
Year: 2023
Subjects: Meta-Learning
Few-shot Learning
Classification
Multiple Modalities
MICCAI Challenges

description: While deep learning approaches have led to breakthroughs in many medical image analysis tasks, the availability of large datasets notoriously hinders progress. The collection of such datasets is often complicated and costly and, in some cases, altogether prohibitive such as in the case of rare diseases. Few-shot learning, whereby a target task is learned from a small number of data points, has recently gained attention in the MICCAI community. Few-shot learning approaches leverage prior knowledge from pretraining on the task or other related datasets to learn a previously unseen task more efficiently. With this challenge, we aim to encourage the medical imaging community to develop algorithms with good few-shot learning capabilities that generalise to a wide range of tasks without prior knowledge about the target dataset. Various strategies exist to approach few-shot learning problems, including transfer learning, unsupervised or selfsupervised pre-training, and meta-learning. In this challenge, we will provide many fully labelled medical image classification training tasks and a smaller set of private testing tasks on which the challenge will be evaluated. These testing tasks will be kept secret during the challenge and only be published after the submission deadline. The tasks will be derived from publicly available medical imaging datasets and fall into the categories of binary, multiclass and multi-label classification, and ordinal regression. Challenge participants are free to leverage the training tasks in any way they wish to devise and/or train a method that can efficiently learn a medical image classification problem from only a few examples per class. The algorithm will be submitted in a Docker container to the challenge website. Algorithms will receive a few labelled support examples to adapt to the test task and should predict the labels of a few query images, after which we measure the classification performance. The procedure will be repeated multiple times with different examples to better estimate the average performance of the submitted algorithms. We will be exploring a range of 1 to 20 labelled examples. As part of this challenge, we provide a novel medical imaging meta-dataset containing several datasets and task definitions which we publish in conjunction with this challenge. Leveraging publicly available tasks to learn a new task with few labels is a realistic setting commonly encountered in practice. While transfer learning can be considered the most widely explored paradigm to solve this problem, multiple alternative approaches are possible. In particular, the strategy of meta-learning is currently underexplored for medical applications even though it may be particularly well suited for this problem setting. Meta-learning tries to mimic the human ability to learn new tasks quickly by leveraging life-long experience with learning diverse tasks. Specifically, meta-learning techniques first “learn to learn” from a set of training tasks before learning the desired test task. Indeed, it has been shown that meta-learning approaches can learn previously unseen tasks with very few examples. These methods can potentially bring the predictive power of deep networks to medical domains that were previously impractical to use due to limited data. However, most meta-learning methods assume idealised settings with homogeneous task definitions derived from a single dataset and a constant number of classes. Realistic medical tasks, on the other hand, can be highly heterogeneous. The appearance of medical images is very heterogeneous due to the different imaging modalities, and tasks vary widely in type (e.g. binary, multi-class, or multi-label classification) and number of classes. While some works have shown the feasibility of meta-learning methods in medical settings, the above limitations for practical medical classification tasks have thus far not been sufficiently addressed. This challenge aims to move from the artificial “toy problem” scenarios studied in computer vision to more realistic clinical scenarios. A secondary aim of this challenge is thus to explore the potential of meta-learning for medical few-shot learning and rigorously explore its performance with respect to more established strategies such as pre-training and transfer learning. The meta-dataset we will release as part of this challenge will be the first to allow robust assessment of the utility of meta-learning for few-shot learning in realistic medical settings. In particular, we are compiling a collection of publicly available datasets which were published with a licence that allows further processing and sharing. Our dataset is similar in spirit to MedMNIST [https://medmnist.com/]. However, we go significantly beyond MedMNIST in the number of tasks and task realism. Specifically, the images will be processed at high resolution, which may also be encountered in realistic medical image analysis scenarios. All data will be preprocessed such that they can be directly used for machine learning, and we will release code for working with the data as well as simple baselines. We have already assembled 24 tasks from 12 publicly available datasets and will further grow this number until the beginning of the challenge.