Elaboration of a psychophysical model of the limits of lesion detection with PET as a benchmark for performance analysis (2020)

1402 Background: Lesion detection is a fundamental clinical task in nuclear medical imaging, especially oncologic PET, but there is no consensus on how to characterize limits-of-detection (LOD) with regards to lesion properties such as size and contrast. We seek to develop a parametric LOD model which can be used to derive objective figures-of-merit for benchmarking performance when evaluating alternative methods of image reconstruction, processing, display and interpretation.
 Methods: We have previously described a fully integrated workflow for synthesizing well-characterized artificial lesions in real PET data. [1] Lesion shape, size, location and contrast with background are defined on a real PET image. Lesion activity is then forward projected, added to the patient sinogram and then reconstructed using clinically available methods. We obtained FDG PET/CT data from 37 patients with low likelihood of hepatic disease and lesions of varying size (2-4 mm diameter) and activity contrast (1.5-6 times background) were manually defined within liver regions to produce 133 PET images, each with a single lesion. In a perception study, we submitted 2D transaxial images intersecting the center of each lesion in random order to 7 human observers, who indicated the perceived location of each lesion. A localization error <1.0 cm was considered accurate lesion detection, as opposed to a missed lesion. Based on these results we developed a psychophysical model of an observer’s probability, P, to detect a lesion of diameter d and contrast c using Weibull stimulus response and power-law stimulus functions [2], such that P ∝ AdDcC. The parameters A, D and C were fitted using a Bayesian fitting algorithm [3] to observers’ perception performance data. Lesion detection results were plotted against diameter and contrast (d and c) to visualize the LOD and to which the psychophysical model was fitted. For comparison, (n=10) random responders (50% probability of lesion detection at all sizes/contrast) were simulated and the psychophysical model parameters were fitted. Model parameters were compared between human and random observers using a Mann-Whitney test.
 Results: Lesion synthesis required ~50 hours processing time on a standard desktop computer. Observers completed the perception study comprising all lesions in 12±2 minutes. 4 mm lesions were detected at >5 contrast while smaller lesions were not detectable. 14 mm lesions were clearly detected at any contrast level (≥1.5). Psychophysical model parameters varied between observers (coefficient of variation 33%, 43% and 135% for D, C and A respectively) but differed significantly (p<0.0001) from those of random responders, which exceeded model fitting boundaries. The ratio D/C for human observers was more consistent (coefficient of variation 25%), potentially indicating a more precise figure-of-merit.
 Conclusions: Lesion detection performance can be characterized objectively by psychophysical model parameters obtained from lesion detection experiments employing well-characterized synthetic lesions. These model parameters demonstrate promise as figures-of-merit to compare competing technical approaches such as image reconstruction methods, image display parameters, and human versus artificial intelligence-based observers. Because the psychophysical model can inform which lesion parameter combinations are most informative for improving the precision of the LOD [3], it is a crucial ingredient for optimizing future studies.
References: 
[1] Gabrani-Juma H, Zuckier LS, Klein R, An integrated software tool for synthesis of lesions in patient PET and CT studies, Med Image Perception Soc Conf, 2019.
[2] Zhou Y, Scott A, Allahverdian J, et al., On the relationship of minimum detectable contrast to dose and lesion size in abdominal CT, Phys Med Biol, 2015:60(19);7671-94.
[3] Watson AB, QUEST+: A general multidimensional Bayesian adaptive psychometric method, J Vision, 2017:17(3);10.