  Universal lesion detection from computed tomography (CT) slices is important
for comprehensive disease screening. Since each lesion can locate in multiple
adjacent slices, 3D context modeling is of great significance for developing
automated lesion detection algorithms. In this work, we propose a Modified
Pseudo-3D Feature Pyramid Network (MP3D FPN) that leverages depthwise separable
convolutional filters and a group transform module (GTM) to efficiently extract
3D context enhanced 2D features for universal lesion detection in CT slices. To
facilitate faster convergence, a novel 3D network pre-training method is
derived using solely large-scale 2D object detection dataset in the natural
image domain. We demonstrate that with the novel pre-training method, the
proposed MP3D FPN achieves state-of-the-art detection performance on the
DeepLesion dataset (3.48% absolute improvement in the sensitivity of FPs@0.5),
significantly surpassing the baseline method by up to 6.06% (in MAP@0.5) which
adopts 2D convolution for 3D context modeling. Moreover, the proposed 3D
pre-trained weights can potentially be used to boost the performance of other
3D medical image analysis tasks.
