Patent Document ID: 9922272
Application ID: 14865565
Patent Flag: 1

Claim One:
1. A method for similarity metric learning for multimodal medical image data, the method comprising: receiving a first set of image data of a volume, wherein the first set of image data is captured with a first imaging modality; receiving a second set of image data of the volume, wherein the second set of image data is captured with a second imaging modality; aligning the first set of image data and the second set of image data; training a first set of parameters with a multimodal stacked denoising auto encoder to generate a shared feature representation of the first set of image data and the second set of image data, the multimodal stacked denoising auto encoder comprising a first layer with independent and parallel denoising auto encoders; training a second set of parameters with a denoising auto encoder to generate a transformation of the shared feature representation; initializing, using the first set of parameters and the second set of parameters, a neural network classifier; training, using training data from the aligned first set of image data and the second set of image data, the neural network classifier to generate a similarity metric for the first and second imaging modalities, the similarity metric identifying which voxels from the first set of image data that correspond to the same position in the volume as voxels from the second set of image data; and performing image fusion on the first set of image data and the second set of image data using the identified voxels.