Patent ID: 12186114

DETAILED DESCRIPTION

The following description is of various exemplary embodiments only, and is not intended to limit the scope, applicability or configuration of the present disclosure in any way. Rather, the following description is intended to provide a convenient illustration for implementing various embodiments including the best mode. As will become apparent, various changes may be made in the function and arrangement of the elements described in these embodiments without departing from principles of the present disclosure. For example, the steps recited in any of the method or process descriptions may be executed in any suitable order and are not necessarily limited to the order presented. Furthermore, any reference to singular includes plural embodiments, and any reference to more than one component or step may include a singular embodiment or step

For the sake of brevity, conventional techniques and components for mathematical processes, transforms, mapping, smoothing, and/or the like may not be described in detail herein. Furthermore, the connecting lines shown in various figures contained herein are intended to represent exemplary functional relationships and/or physical couplings between various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in exemplary methods and systems for imaging and/or components thereof.

As used herein, “data.” “information,” or the like may include encompassing information such as commands, queries, files, messages, data for storage, and/or the like in digital or any other form.

As used herein, “satisfy,” “meet.” “match,” “associated with,” or similar phrases may include an identical match, a partial match, meeting certain criteria, matching a subset of data, a correlation, satisfying certain criteria, a correspondence, an association, an algorithmic relationship, and/or the like. Similarly, as used herein, “authenticate” or similar terms may include an exact authentication, a partial authentication, authenticating a subset of data, a correspondence, satisfying certain criteria, an association, an algorithmic relationship, and/or the like.

Methods are provided. In the detailed description herein, references to “various embodiments,” “one embodiment,” “an embodiment,” “an example embodiment,” etc. indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. After reading the description, it will be apparent to one skilled in the relevant art(s) how to implement the disclosure in alternative embodiments.

Amyloid and tau are the defining pathologies of Alzheimer's disease (AD) and their abnormality initiates long before clinical symptoms onset. Positron emission tomography (PET) makes it possible to detect and image brain amyloid and tau pathology for an individual in vivo. However, due to the differences in chemical properties between different PET tracers, including target binding affinity, tracer kinetic behavior, non-specific binding, and tissue retention, the PET images have an inconsistent amyloid positivity threshold and pose challenges for multi-center studies.

In particular, clinical trials include amyloid and tau PET imaging for the assessment of treatment efficiency and target engagement as surrogate biomarkers. The current amyloid and tau PET imaging methods with different PET tracers have a shared variance ranging from approximately 70% to 90% depending on the quantification pipelines and cohorts. To address these shortcomings of prior approaches, exemplary embodiments provide methods using deep learning to produce harmonization of images between different PET tracers.

With reference now toFIGS.1,2,3, and4, in accordance with an exemplary embodiment, exemplary methods utilize concepts from PET image processing and provide techniques related to image reconstruction, attenuation, and scatter correction. However, all practical applications of principles of the present disclosure are contemplated herein.

In an exemplary embodiment, in an image conversion method, a method comprises five encoding blocks and four decoding blocks, and comprises various convolution operations within each encoding block, and comprises various deconvolution operations within each decoding block, as illustrated inFIG.1. This is an architecture similar to U-Net (Ronneburger et al.,U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention: Springer: 2015, p. 234-41; the entire contents of which are hereby incorporated by reference) and with the addition of a residual inception short-cut path that has been shown to improve training efficiency. Each block has a conventional convolution/deconvolution path with two 3 by 3 convolutional layers, and in parallel, a 1 by 1 convolution path. The output matrices from these two parallel paths are summed together and down/up sampled by a factor of 2 to serve as the input to the next block.

In an exemplary embodiment, in a convolution method, a method comprises learning the difference (F) between mapping and original input x, as illustrated inFIG.2. Usually, a deep network (like a U-Net) estimates an output by learning a mapping function H, from input x to an output H(x). Instead of learning the direct mapping H, residual networks (like a RIED-Net) attempt to learn the difference (F) between mapping and original input x. A projection or estimation of x (labeled G(x)) can be used instead of direct input features. G(x) is generated using a simple [1×1] convolution to match the dimensions of the input. It is easier to learn the residual of output and input than the actual input, so the residual inception block is better than the traditional convolution block. Further, the use of residual blocks can address gradient vanishing issues from deep models.

In an exemplary embodiment, as illustrated inFIG.3, in one or more encoding blocks, the input can be a 2D slice of a PET image, for example as represented as a 256 by 256 matrix. The matrix can be processed in two parallel paths: a traditional convolutional path with two layers of 3 by 3 kernels and a separate convolutional path with one layer of a 1 by 1 kernel (function G inFIG.2). The traditional convolution path comprises a first convolutional layer which uses a 32-channel filter bank and generates a 256 by 256 by 32 matrix (B) as an output. The traditional convolution path further comprises a second convolutional layer which receives matrix B an input and uses another 32-channel filter bank to generate another 256 by 256 by 32 matrix (C) as an output. The separate convolutional path uses a 32-channel 1 by 1 filter bank and generates another 256 by 256 by 32 matrix (A) as an output. Matrices A and C are summed together and then down-sampled by a factor of two using a rectified linear unit (ReLu) function (or other suitable function) to generate a 128 by 128 by 32 matrix (D). Matrix D serves as the input for the next encoding block. Matrix C serves as the partial input for the corresponding decoding block.

In an exemplary embodiment, as illustrated inFIG.4, in one or more decoding blocks, the input can comprise a 256 by 256 by 32 output matrix from the corresponding encoding block and a 256 by 256 by 32 output matrix from the previous decoding block. These two matrices are concatenated into a 256 by 256 by 64 matrix. This input matrix can be processed in two parallel paths: a traditional convolutional path with two layers of 3 by 3 kernels and a separate convolutional path with one layer of a 1 by 1 kernel. The traditional convolution path comprises a first convolutional layer which uses a 32-channel filter bank and generates a 256 by 256 by 32 matrix (B) as an output. The traditional convolution path further comprises a second convolutional layer which receives matrix B an input and uses another 32-channel filter bank to generate another 256 by 256 by 32 matrix (C) as an output. The separate convolutional path uses a 32-channel 1 by 1 filter bank and generates another 256 by 256 by 32 matrix (A) as an output. Matrices A and C can be summed together and processed by a 1 by 1 convolutional kernel to generate a target 2D output image slice.

Via application of principles of the present disclosure, an exemplary RIED-Net model is able to harmonize amyloid PET images acquired using different tracers. Further, an exemplary RIED-Net model is able to improve the agreement in amyloid measures from two different tracers for both global indices and voxel-wise similarities. An exemplary RIED-Net model can be generalized to external imaging data and achieve favorable performance without additional tuning of the model. An exemplary RIED-Net model is robust to variabilities in imaging acquisition protocols and scanner differences when standard scanner harmonization protocols are implemented.

An exemplary RIED-Net model has two major advantages. First, RIED-Net focuses on voxel mapping instead of using patch-based approaches. Compared to an exemplary RIED-Net model, patch-based approaches sacrifice synthesis performance at the voxel levels. Second, an exemplary RIED-Net model is computationally affordable. This gives an exemplary RIED-Net model the potential to perform PET harmonization in 3D. Because an exemplary RIED-Net model is trained on each of the three orthogonal views of the input 3D images, RIED-Net avoids the unwanted noises across 2D slices that occur with other 2D models.

Working Example

Participants. From Open Access Series of Imaging Studies (OASIS), 92 participants aged 43-88 years were identified who had [C11]-Pittsburgh compound (PIB) and florbetapir (FBP) PET scans within 3 months. This dataset was used for training and cross-validation of the RIED-Net as a PET harmonization model. An independent dataset of 46 participants aged 21-89 years with paired PIB and FBP PET scans were downloaded from the Centiloid Project website and served as the external validation (GAAIN) set. All studies were approved by their corresponding institutional review board and written informed consent was obtained for each participant.

Imaging. For the OASIS dataset, dynamic PIB PET scan was acquired on a Siemens Biograph 40 PET/CT or a Siemens/CTI EXACT HR+ scanner for 60 minutes after tracer administration and reconstructed using standard iterative methods with attenuation and scatter correction. Dynamic FBP PET was acquired on a Siemens Biograph mMR scanner for 70 minutes after FBP administration and reconstructed using an ordered subset expectation maximization (OSEM) algorithm and attenuation/scatter corrected using a separately acquired low dose CT scan. For each participant, a T1-weighted MRI scan was also acquired using a 3T MR scanner. All imaging were acquired at the Washington University in St. Louis, and individual scans for each participant were completed within 3 months. The imaging acquisition information for the GAAIN dataset has been previously described in Navitsky et al.,Standardization of amyloid quantitation with florbetapir standardized uptake value ratios to the Centiloid scale. Alzheimers Dement. 2018, the entire contents of which are hereby incorporated by reference. Briefly. PIB PET was acquired between 50-70 minutes post-injection, and FBP PET was acquired 50-60 minutes post-injection. The imaging pair was obtained an average of 18 days apart, and a 3T T1 MRI was obtained for each subject within 6 months of PET acquisition. One participant in the GAAIN cohort was excluded from further analysis due to poor quality of the T1 MRI.

The T1 weighted MRI data was analyzed using FreeSurfer (Martinos Center for Biomedical Imaging, Charlestown, Massachusetts, USA) to define anatomical regions. Amyloid PET imaging quantification was then performed using our standard protocols that included scanner harmonization, motion correction, target registration, and regional value extraction using a PET unified pipeline (PUP). The output included a standard uptake value ratio (SUVR) image using cerebellar cortex as the reference region and a mean cortical SUVR (MCSUVR) as the global index of brain amyloid burden. For the OASIS cohort, the PIB PET data were summed between 30-60 minutes and the FBP data were summed between 50-70 minutes post-injection to generate the SUVR images and the MCSUVR global indices. For the GAAIN cohort, the PIB and FPB PET data were summed between the 50-70 minutes and 50-60 minutes post-injection window for the quantification. The SUVR images were transformed in to the MNI152 template space via affine transformation established based on the T1 MRI image and served as the input to the RIED-Net model for training, internal validation, and external validation. All MCSUVR measurements were also converted to the Centiloid scale (CL) using pre-established equations and procedures to facilitate the cross-tracer comparison and interpretation.

Deep Learning Model for PET Harmonization. In an exemplary embodiment, we used 10-fold Cross-Validation (CV) in the training. Specifically, for the OASIS dataset, we shuffled the dataset randomly and created 10 different groups of the dataset, for an even split, we decided to use 90 out of 92 total samples (excluding the last two participants according to alphabetical order) and created 10 different folds of size 81:9 (total 90) where 81 was used for training and validation, and 9 was used for testing respectively. These folds are generated such that there is no overlap among the training and testing samples and the test dataset in each fold is always unique. We performed this 10-fold CV technique for all three views: coronal, sagittal and axial. For each view and fold, individual FBP 2D slices (256×256) from the 81 patients were used as the input, and the PIB 2D slices with respect to the same patients were taken as output to train and validate the RIED-Net model. Among the 2D slices obtained from 81 patients, we used a 90:10 split for training and internal validation. For each fold, we trained the model for a total of 40 epochs with batch-size of 16 (determined by the computing resource) and mean absolute error (MAE) as the loss function optimized by the model. We used Adam (Kingma et al.,A method for stochastic optimization, arXiv preprint arXiv:14126980. 2014: the entire contents of which are hereby incorporated by reference) as the optimizer with a learning rate of 0.002 and a decay rate of 0.0005. For the other parameters we used default settings of the Keras platform. The validated model for that fold was then used to generate synthetic PIB SUVR images from FBP image for the remaining 9 patients serving as testing. Using the 10-fold CV procedure, synthetic PIB SUVR image was generated for each view and an average synthetic 3D PIB SUVR image was then generated combining the three views which was used as the main target for performance evaluation.

To obtain a single model from the OASIS dataset and test its performance on the independent GAAIN dataset to further evaluate the generalizability of an exemplary approach, we retrained the models using 80 out of the 92 OASIS samples that had the largest field of view (FOV) coverage and applied the models to generate synthetic PIB SUVR images for samples within the independent GAAIN dataset. Similar to the experiment on OASIS dataset only, synthetic PIB SUVR images for GAAIN dataset were generated for each view and the average 3D image across all three views was used as the main target for performance evaluation.

Additional principles of the present disclosure may be set forth in this publication: Shah J. Gao F. Li B, et al. Deep residual inception encoder-decoder network for amyloid PET harmonization.Alzheimer's Dement.2022:1-10. Available at https://doi.org/10.1002/alz.12564 as of November 2022.) The disclosure of the foregoing is incorporated herein by reference in its entirety, including but not limited to those portions that specifically appear hereinafier, but except for any subject matter disclaimers or disavowals, and except to the extent that the incorporated material is inconsistent with the express disclosure herein, in which case the language in this disclosure shall control.

While the principles of this disclosure have been shown in various embodiments, many modifications of structure, arrangements, proportions, the elements, materials and components, used in practice, which are particularly adapted for a specific environment and operating requirements may be used without departing from the principles and scope of this disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure.

The present disclosure has been described with reference to various embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure. Accordingly, the specification is to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure. Likewise, benefits, other advantages, and solutions to problems have been described above with regard to various embodiments. However, benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or element.

As used herein, the terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Also, as used herein, the terms “coupled,” “coupling.” or any other variation thereof, are intended to cover a physical connection, an electrical connection, a magnetic connection, an optical connection, a communicative connection, a functional connection, and/or any other connection. When language similar to “at least one of A, B, or C” or “at least one of A, B, and C” is used in the specification or claims, the phrase is intended to mean any of the following: (1) at least one of A: (2) at least one of B; (3) at least one of C; (4) at least one of A and at least one of B; (5) at least one of B and at least one of C; (6) at least one of A and at least one of C; or (7) at least one of A, at least one of B. and at least one of C.

Furthermore, no element, component, or method step in the present disclosure is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. No claim element herein is to be construed under the provisions of 35 U.S.C. 112(f) unless the element is expressly recited using the phrase “means for.” As used herein, the terms “comprises.” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.