Automation of histological analysis of onchocerciasis nodules with artificial intelligence (AI)
Creators: name: Klarmann-Schulz, Ute
name: Kuehlwein, Daniel A.
name: Kuehlwein, Janina M.
name: Dubben, Bettina
name: Fischer, Kerstin
name: Bergmann, Marcel
name: Konieczkowicz, Wojciech
name: Poplawska, Julia
name: Rickmers, Marlene
name: Haase, David
name: Wauschkuhn, Caroline
name: Schmid, Matthias
name: Specht, Sabine
name: Debrah, Alexander Y.
name: Hoerauf, Achim
Publisher: name: Gates Open Research
Year: 2022

description: Onchocerciasis is a neglected tropical disease (NTD) caused by the filarial worm Onchocerca volvulus, residing in nodules under the skin of the human host. The NTD Road Map 2021–30, released by the World Health Organization, targets the elimination of onchocerciasis in 31% previously endemic countries by 2030 (current 4 countries, 12%). The achievability of this target depends on developing macrofilaricidal drugs, which do not only interrupt transmission by elimination of microfilariae (MF; baby worms) from the human host’s skin but also prevent the release of new MF from the adult worms. The efficacy of new treatments is assessed by histological analysis of the adult worms in surgically removed nodules. This time-consuming task is currently done manually by two independent human experts. By automating this task with the help of artificial intelligence (AI), we could reduce the time needed for evaluation of larger randomized clinical trials from over a year to several days. At the same time, the use of an AI would standardize the analysis and help to be less dependent from the availability of the few experts worldwide.<br> Having high-quality data in sufficient quantity is essential for building AI solutions. Microscopic slides from four previous clinical trials were used to create a diverse data set. In total, around 1,500 nodule slides were digitized and almost 75,000 worm sections were marked by two expert assessors. Due to the immense resolution of up to 150.000 pixels on one dimension, we went for a multi-phase approach. First, labelled sections were used to train a Faster R-CNN model to detect worm sections in the nodules. Next, separate EfficientNet models were then trained to classify the sections for vitality, sex, the presence of Wolbachia (endosymbiotic bacteria) and fertility of female worms. A slightly different approach using squares from all over the nodule instead of worm sections was taken to find MF that were not in the female uteri but in the nodule tissue surrounding the worms.<br> To measure the quality of the AI predictions we divided each data set into a training set (80%), which is used to train the model (i.e. creating slightly different versions) and a validation set (10%), which tests every version from the training on data that was not included before. Finally, the best-performing model version on the validation set is tested on a previously unseen test set (10%), to confirm the results. All models reached accuracies of over 90% on the test validation and test sets: detection of worm sections in a nodule reached 94.6% mAP; classification on single sections ranged from 90.2 to 96.9%, depending on the task. As expected, binary predictions (e.g., alive vs. dead or not judegable; accuracy 97.8%) were better compared to predictions with more attributes per class (e.g., alive vs. dead vs. not judegable; accuracy 94.7%). Next, all models were combined in a single tool which provides a complete analysis at nodule level, depending on the research question.<br> While each model achieved a high accuracy, a fully automated image analysis requires interdependent combination of the individual predictions. Hence, even small errors may have a bigger impact. The analysis of the combined tool is currently ongoing. First, validation is done on published data, followed by prospective validation in new clinical trials. For validation purposes we will focus on nodule (e.g. “Does the nodule contain live fertile female worms?”) and patient level (e.g. “Does the patient have any nodules with live fertile female worms?”) to reflect the usual analysis process. Additionally, validation on section level will help to find weaknesses of the AI.<br> To make the decisions of the AI more visible, we are also working on an easy-to-use user interface (UI). This tool will allow experts to see what decisions were made by the AI per nodule slide and for each section in those slides. It will allow to filter for all labelled attributes and to share interesting findings with colleagues worldwide.<br> The final goal is to register our AI tool for the routine analysis of onchocera nodules in clinical trials, leaving human experts to focus on difficult cases and quality checks.