Patent Description:
Magnetic resonance imaging (MRI) has proven useful in diagnosis of many diseases. MRI provides detailed images of soft tissues, abnormal tissues such as tumors, and other structures, which cannot be readily imaged by other imaging modalities, such as computed tomography (CT). Further, MRI operates without exposing patients to ionizing radiation experienced in modalities such as CT and x-rays.

In MR imaging, a partial k-space is often sampled in order to increase the efficiency of the acquisition and/or to suppress artifacts. Reconstructing a partially-sampled k-space dataset results in an image that is contaminated by truncation artifacts in the form of both blurring and a characteristic ringing that severely degrades the diagnostic value of the MR image.

<NPL>" relates to a neural network-based method for Gibbs artifact and noise removal in diffusion-weighted imaging data. Two implementations were considered: one for magnitude images and one for complex images. Both models were based on the same encoder-decoder structure and were trained by simulating MRI acquisitions on synthetic non-MRI images.

<NPL>" describes a relaxed version of the iterative POCS algorithm into a network architecture which alternates between recurrent convolutional units, enforcing prior knowledge, and data consistency operations. Knowledge on the pixel-wise noise level of images is incorporated into data consistency operations within the reconstruction network.

<NPL>" describes a deep learning approach for partial Fourier reconstruction. A CNN model is designed and trained using large human knee datasets, and its performance is compared with the existing projection onto convex set (POCS) method.

In one aspect, a computer-implemented method of removing truncation artifacts in magnetic resonance (MR) images is provided in accordance with claim <NUM>. The method includes receiving a crude image that is based on partial k-space data from a partial k-space asymmetrically truncated in at least one k-space dimension at k-space locations corresponding to high spatial frequencies. The method also includes analyzing the crude image using a neural network model. The neural network model was trained with a pair of pristine images and corrupted images. The corrupted images are based on partial k-space data from partial k-spaces truncated in one or more partial sampling patterns at the k-space locations corresponding to the high spatial frequencies, the one or more partial sampling patterns including an asymmetrical truncation in at least one k-space dimension. The pristine images are based on full k-space data corresponding to the partial k-space data of the corrupted images, and target output images of the neural network model are the pristine images. The method further includes deriving an improved image of the crude image based on the analysis, wherein the derived improved image includes reduced truncation artifacts and increased high spatial frequency data, compared to the crude image, and outputting the improved image.

In another aspect, a computer-implemented method of removing truncation artifacts in magnetic resonance (MR) images is provided in accordance with claim <NUM>. The method includes receiving a pair of pristine images and corrupted images. The corrupted images are based on partial k-space data from partial k-spaces truncated in one or more partial sampling patterns at k-space locations corresponding to high spatial frequencies, the one or more partial sampling patterns including an asymmetrical truncation in at least one k-space dimension. The pristine images are based on full k-space data corresponding to the partial k-space data of the corrupted images. The method also includes training a neural network model using the pair of the pristine images and the corrupted images by inputting the corrupted images to the neural network model, setting the pristine images as target outputs of the neural network model, analyzing the corrupted images using the neural network model, comparing outputs of the neural network model with the target outputs, and adjusting the neural network model based on the comparison. The trained neural network model is configured to reduce truncation artifacts in the corrupted images and increase high spatial frequency data in the corrupted images.

In one more aspect, a truncation artifact reduction system is provided in accordance with claim <NUM>.

The system includes a truncation artifact reduction computing device, the truncation artifact reduction computing device including at least one processor in communication with at least one memory device. The at least one processor is programmed to receive a crude image that is based on partial k-space data from a partial k-space asymmetrically truncated in at least one k-space dimension at k-space locations corresponding to high spatial frequencies. The at least one processor is also programmed to analyze the crude image using a neural network model. The neural network model was trained with a pair of pristine images and corrupted images. The corrupted images are based on partial k-space data from partial k-spaces truncated in one or more partial sampling patterns at the k-space locations corresponding to the high spatial frequencies, the one or more partial sampling patterns including an asymmetrical truncation in at least one k-space dimension. The pristine images are based on full k-space data corresponding to the partial k-space data of the corrupted images, and target output images of the neural network model are the pristine images. The at least one processor is further programmed to derive an improved image of the crude image based on the analysis, wherein the derived improved image includes reduced truncation artifacts and increased high spatial frequency data, compared to the crude image, and output the improved image.

The disclosure includes systems and methods of removing truncation artifacts in magnetic resonance (MR) images of a subject using a deep learning model. As used herein, a subject is a human, an animal, or a phantom. Unlike signals, which represent the anatomies or structures of the subject, artifacts are visual anomalies in the medical images that are not present in the subject, which may be caused by the imaging modality such as partial sampling pulse sequences. Removing artifacts is reduction and/or removal of artifacts from an image. The systems and methods disclosed herein also synthesize missing data and interpolate high spatial frequency data, while removing truncation artifacts. Method aspects will be in part apparent and in part explicitly discussed in the following description.

In magnetic resonance imaging (MRI), a subject is placed in a magnet. When the subject is in the magnetic field generated by the magnet, magnetic moments of nuclei, such as protons, attempt to align with the magnetic field but precess about the magnetic field in a random order at the nuclei's Larmor frequency. The magnetic field of the magnet is referred to as B0 and extends in the longitudinal or z direction. In acquiring an MRI image, a magnetic field (referred to as an excitation field B1), which is in the x-y plane and near the Larmor frequency, is generated by a radio-frequency (RF) coil and may be used to rotate, or "tip," the net magnetic moment Mz of the nuclei from the z direction to the transverse or x-y plane. A signal, which is referred to as an MR signal, is emitted by the nuclei, after the excitation signal B1 is terminated. To use the MR signals to generate an image of a subject, magnetic field gradient pulses (Gx, Gy, and Gz) are used. The gradient pulses are used to scan through the k-space, the space of spatial frequencies or inverse of distances. A Fourier relationship exists between the acquired MR signals and an image of the subject, and therefore the image of the subject can be derived by reconstructing the MR signals.

<FIG> illustrates a schematic diagram of an exemplary MRI system <NUM>. In the exemplary embodiment, the MRI system <NUM> includes a workstation <NUM> having a display <NUM> and a keyboard <NUM>. The workstation <NUM> includes a processor <NUM>, such as a commercially available programmable machine running a commercially available operating system. The workstation <NUM> provides an operator interface that allows scan prescriptions to be entered into the MRI system <NUM>. The workstation <NUM> is coupled to a pulse sequence server <NUM>, a data acquisition server <NUM>, a data processing server <NUM>, and a data store server <NUM>. The workstation <NUM> and each server <NUM>, <NUM>, <NUM>, and <NUM> communicate with each other.

In the exemplary embodiment, the pulse sequence server <NUM> responds to instructions downloaded from the workstation <NUM> to operate a gradient system <NUM> and a radiofrequency ("RF") system <NUM>. The instructions are used to produce gradient and RF waveforms in MR pulse sequences. An RF coil <NUM> and a gradient coil assembly <NUM> are used to perform the prescribed MR pulse sequence. The RF coil <NUM> is shown as a whole body RF coil. The RF coil <NUM> may also be a local coil that may be placed in proximity to the anatomy to be imaged, or a coil array that includes a plurality of coils.

In the exemplary embodiment, gradient waveforms used to perform the prescribed scan are produced and applied to the gradient system <NUM>, which excites gradient coils in the gradient coil assembly <NUM> to produce the magnetic field gradients Gx, Gy, and Gz used for position-encoding MR signals. The gradient coil assembly <NUM> forms part of a magnet assembly <NUM> that also includes a polarizing magnet <NUM> and the RF coil <NUM>.

In the exemplary embodiment, the RF system <NUM> includes an RF transmitter for producing RF pulses used in MR pulse sequences. The RF transmitter is responsive to the scan prescription and direction from the pulse sequence server <NUM> to produce RF pulses of a desired frequency, phase, and pulse amplitude waveform. The generated RF pulses may be applied to the RF coil <NUM> by the RF system <NUM>. Responsive MR signals detected by the RF coil <NUM> are received by the RF system <NUM>, amplified, demodulated, filtered, and digitized under direction of commands produced by the pulse sequence server <NUM>. The RF coil <NUM> is described as a transmitter and receiver coil such that the RF coil <NUM> transmits RF pulses and detects MR signals. In one embodiment, the MRI system <NUM> may include a transmitter RF coil that transmits RF pulses and a separate receiver coil that detects MR signals. A transmission channel of the RF system <NUM> may be connected to a RF transmission coil and a receiver channel may be connected to a separate RF receiver coil. Often, the transmission channel is connected to the whole body RF coil <NUM> and each receiver section is connected to a separate local RF coil.

In the exemplary embodiment, the RF system <NUM> also includes one or more RF receiver channels. Each RF receiver channel includes an RF amplifier that amplifies the MR signal received by the RF coil <NUM> to which the channel is connected, and a detector that detects and digitizes the I and Q quadrature components of the received MR signal. The magnitude of the received MR signal may then be determined as the square root of the sum of the squares of the I and Q components as in Eq. (<NUM>) below: <MAT> and the phase of the received MR signal may also be determined as in Eq. (<NUM>) below: <MAT>.

In the exemplary embodiment, the digitized MR signal samples produced by the RF system <NUM> are received by the data acquisition server <NUM>. The data acquisition server <NUM> may operate in response to instructions downloaded from the workstation <NUM> to receive real-time MR data and provide buffer storage such that no data is lost by data overrun. In some scans, the data acquisition server <NUM> does little more than pass the acquired MR data to the data processing server <NUM>. In scans that need information derived from acquired MR data to control further performance of the scan, however, the data acquisition server <NUM> is programmed to produce the needed information and convey it to the pulse sequence server <NUM>. For example, during prescans, MR data is acquired and used to calibrate the pulse sequence performed by the pulse sequence server <NUM>. Also, navigator signals may be acquired during a scan and used to adjust the operating parameters of the RF system <NUM> or the gradient system <NUM>, or to control the view order in which k-space is sampled.

In the exemplary embodiment, the data processing server <NUM> receives MR data from the data acquisition server <NUM> and processes it in accordance with instructions downloaded from the workstation <NUM>. Such processing may include, for example, Fourier transformation of raw k-space MR data to produce two or three-dimensional images, the application of filters to a reconstructed image, the performance of a backprojection image reconstruction of acquired MR data, the generation of functional MR images, and the calculation of motion or flow images.

In the exemplary embodiment, images reconstructed by the data processing server <NUM> are conveyed back to, and stored at, the workstation <NUM>. In some embodiments, real-time images are stored in a database memory cache (not shown in <FIG>), from which they may be output to operator display <NUM> or a display <NUM> that is located near the magnet assembly <NUM> for use by attending physicians. Batch mode images or selected real time images may be stored in a host database on disc storage <NUM> or on a cloud. When such images have been reconstructed and transferred to storage, the data processing server <NUM> notifies the data store server <NUM>. The workstation <NUM> may be used by an operator to archive the images, produce films, or send the images via a network to other facilities.

MR signals are represented by complex numbers, where each location at the k-space is represented by a complex number, with I and Q quadrature MR signals being the real and imaginary components. Complex MR images may be reconstructed based on I and Q quadrature MR signals, using processes such as Fourier transform. Complex MR images are MR images with each pixel represented by a complex number, which also has a real component and an imaginary component.

In MRI, asymmetric sampling in the frequency and phase encoding directions or dimensions is referred to as fractional echo and partial number of acquisition (NEX), respectively, and is widely used in both 2D and 3D MR imaging. These undersampling techniques are typically used to shorten echo times (e.g. to increase SNR or alter tissue contrast), to shorten repetition times (e.g. to reduce scan time), and/or to suppress unwanted artifacts (such as fineline artifact in fast-spin echo (FSE) imaging or off-resonance artifacts in gradient recalled echo (GRE) and echo planar imaging (EPI)). Asymmetric sampling of k-space introduces truncation artifacts into the reconstructed images, both in the form of blurring and ringing. Various image reconstruction techniques have been devised, therefore, for reconstructing partial k-space data, such as conjugate synthesis, homodyne, and projection onto convex sets (POCS). These known techniques rely on some intrinsic estimate of the underlying image phase, which can be subsequently removed (or "corrected"), allowing the synthesis of the missing or unsampled data based on the principle of Hermitian symmetry of real-valued signals. This phase estimate is often derived from the central, symmetrically-sampled portion of k-space, and is limited in several important ways. First, the phase estimate is contaminated by thermal noise, which is especially problematic in low-signal image regions and/or when this phase estimate performed on a per-channel (or per-view) basis. Secondly, this phase estimate is inherently band-limited, and must be further low-pass filtered upon application to prevent the introduction of additional truncation artifacts. Therefore, high spatial frequency phase information is not corrected, leaving residual blurring in the final reconstructed image. The application of this low-frequency phase estimate also tends to bias the noise in the reconstructed image, which would otherwise tend to be normally distributed. The appearance of this biased noise signal in the reconstructed image degrades the image contrast, especially in low-signal regions, and the altered distribution of this noise degrades noise-averaging performance (as in multi-NEX EPI-diffusion) and/or complicate downstream denoising efforts, which are generally based on an assumed noise model. Further, the known partial k-space reconstruction techniques tend to exhibit various strengths and weaknesses, and the choice of method tends to result in various performance tradeoffs. POCS, for example, tends to localize reconstruction artifacts, whereas homodyne tends to result in contrast errors. Finally, in the case of homodyne and conjugate synthesis, the phase information is discarded during reconstruction, making them unsuitable for phase-sensitive applications, such as Dixon chemical shift imaging, phase-sensitive inversion recovery imaging, and generation of phase-sensitive maps based on the phases of the images.

Using deep learning to directly remove these asymmetric truncation artifacts provides superior performance to conventional methods. The deep learning approach involves no explicit phase correction, no low-pass filtering, and no conventional filtering of any kind. Unlike the conventional methods mentioned above, the deep learning approach makes use of all acquired data (versus a low-pass filtered phase estimate) and this results in a reconstructed image with sharper edges, truer contrast, and less noise bias. Moreover, the underlying phase of the image after truncation artifact removal is well-preserved, even at high frequencies, making this technique suitable for phase sensitive imaging applications. In addition to reduce truncation artifacts, the systems and methods described herein also increase or recover the missing high spatial frequency data caused by asymmetrical and/or symmetrical truncation.

<FIG> is a schematic diagram of an exemplary truncation artifact reduction system <NUM>. In the exemplary embodiment, the system <NUM> includes a truncation artifact reduction computing device <NUM> configured to reduce truncation artifacts and increase high spatial frequency data. The computing device <NUM> further includes a neural network model <NUM>. The system <NUM> may include a second truncation artifact reduction computing device <NUM>. The second truncation artifact reduction computing device <NUM> may be used to train the neural network model <NUM>, and the truncation artifact reduction computing device <NUM> may then use the trained neural network model <NUM>. The second truncation artifact reduction computing device <NUM> may be the same computing device as the truncation artifact reduction computing device <NUM> such that the training and use of the neural network model <NUM> are on one computing device. Alternatively, the second truncation artifact reduction computing device <NUM> may be a computing device separate from the truncation artifact reduction computing device <NUM> such that the training and use of the neural network model <NUM> are executed on separate computing devices. The truncation artifact reduction computing device <NUM> may be included in the workstation <NUM> of the MRI system <NUM>, or may be included on a separate computing device that is in communication with the workstation <NUM>.

<FIG> is a flow chart of an exemplary method <NUM>. The method <NUM> may be implemented on the truncation artifact reduction system <NUM>. In the exemplary embodiment, the method includes executing <NUM> a neural network model for analyzing MR images. The neural network model is trained with training images. The training images are pairs of pristine images and corrupted images, and the target output images of the neural network model are the pristine images. The corrupted images are images reconstructed based on partial k-space data from a partial k-space in one or more partial sampling patterns of the k-space. As used herein, partial sampling or truncation is partial sampling of the k-space in one or more dimensions of the k-space by truncating the k-space in those dimensions at locations corresponding to high spatial frequencies. High spatial frequencies are located at the peripheral of the k-space, compared to low-spatial frequencies, which are located at and around the center of the k-space. The truncation of the k-space causes truncation artifacts such as blurring and ringing in the corrupted images. The pristine images are images based on a full k-space corresponding to the partial k-space.

<FIG> is a schematic diagram of a partial sampling pattern or truncation pattern <NUM> of a full k-space <NUM>. A full k-space <NUM> is defined by the maximum kx or ky values kx,max and ky,max, which is defined by maximum frequency- or phase-encoding gradients. In partial sampling, part of the high spatial frequency data <NUM> is not acquired. Truncation may be in the kx dimension and/or the ky dimension, and may be in the kz dimension in a three-dimension (3D) acquisition. The full k-space <NUM> is truncated into a partial k-space <NUM>. The partial k-space <NUM> shown in <FIG> is the full k-space <NUM> truncated in the ky dimension, where negative high spatial frequency data are not acquired during the image acquisition of the partial k-space <NUM>. Truncation may be asymmetrical, where the k-space is truncated asymmetrically in a dimension. The partial k-space <NUM> shown in <FIG> is asymmetrical truncated in the ky dimension. The truncation may be symmetrical, where the k-space is truncated symmetrically at k-space locations of positive and negative spatial frequencies. Truncation may be symmetrical and asymmetrical in one dimension, where k-space is truncated at k-space locations for both positive and negative spatial frequencies but in an unequal amount. Truncation reduces high-spatial frequency data and causes truncation artifacts. Truncation along the axes of a 2D Cartesian coordinate system as shown in <FIG> is illustrated as an example only. The systems and methods described herein may also be used for removal of truncation artifacts in images based on k-space data from a k-space that is asymmetrically truncated along the axes of a 2D/3D Cartesian coordinate system, a 2D/3D non-Cartesian coordinate system such as a polar, spherical, or cylindrical coordinate system, or a combination thereof. For example, the partial sampling pattern is the k-space being asymmetrically truncated in a radial dimension. In another example, the k-space data are acquired as a stack of radial lines in the kx-ky planes along the kz direction and a partial sampling pattern is the k-space being asymmetrically truncated in a radial dimension in the kx-ky plane and asymmetrically truncated in the kz dimension.

In the exemplary embodiment, the corrupted images for training may be in various partial sampling patterns in various partial sampling factors or partial k-space factor. A partial k-space factor is the ratio between the partial k-space in the truncation dimension and the full k-space. For example, if the partial k-space factor is <NUM> in the ky dimension, only half of the k-space, the positive ky half or the negative ky half, is acquired. In some embodiments, the corrupted images and the pristine images are simulated image. The neural network model <NUM> may be trained with one partial sampling pattern and configured to remove truncation artifacts and increase high spatial frequency data for corrupted images based on MR k-space data from a partial k-space acquired in that partial sampling pattern. For example, the neural network model <NUM> is trained with pairs of corrupted images and pristine images for asymmetrical truncation in the kx dimension, the trained neural network model <NUM> is specialized in removing truncation artifacts and increase high spatial frequency data in the kx dimension for images acquired with asymmetrical truncation in the kx dimension. On the other hand, the neural network model <NUM> may be a general neural network model <NUM> that is configured to remove truncation artifacts and increase high spatial frequency data for partial k-space data acquired in various partial sampling patterns. A general neural network model <NUM> may be trained by pairs of corrupted images and pristine images for various partial sampling patterns. A specialized neural network model <NUM> takes less time and computation burden to train than a general neural network model <NUM>.

In some embodiments, the neural network model <NUM> includes one or more layers of neurons configured to reconstruct an image based on partial k-space data. During training, partial k-space data in various partial sampling patterns are used for training, where the partial k-space data are the inputs to the neural network model <NUM>.

Referring back to <FIG>, the method <NUM> further includes receiving <NUM> partial k-space data from a partial k-space that is truncated in at least one dimension. The method <NUM> also includes reconstructing <NUM> a crude image based on the partial k-space data. The crude image may be reconstructed by zero-filling the partial k-space data with zeros at locations corresponding to the skipped k-space locations to derive full k-space data, and then reconstructing the crude image based on the zero-filled k-space data. The full k-space data for the crude image may be reconstructed by methods other than zero-filling, such as interpolation. Reconstructing <NUM> the crude image may be carried out outside the neural network model <NUM> and the crude image is inputted into the neural network model. Alternatively, reconstructing <NUM> the crude image is conducted by the neural network model <NUM>, where partial k-space data are directly input into the neural network model <NUM>, and the neural network model <NUM> includes one or more layers of neurons configured to reconstruct a crude image based on the partial k-space data. Further, the method <NUM> includes analyzing <NUM> the crude image. In addition, the method <NUM> includes deriving <NUM> an improved image of the crude image based on the analysis. The neural network model <NUM> outputs an improved image, an image of improved image quality, corresponding to the crude image. The improved image has reduced truncation artifacts and increased high spatial-frequency data, compared to the crude image. In some embodiments, the neural network model <NUM> includes one or more layers of neurons configured to generate full k-space data by methods such as Fourier transforming the improved image inferenced by the neural network model <NUM>. The method <NUM> also includes outputting <NUM> the improved images.

<FIG> are schematic diagrams of exemplary neural network model <NUM>. The neural network model <NUM> may include a convolutional neural network <NUM>. The neural network <NUM> is trained with corrupted images <NUM> as inputs and output pristine images <NUM>. Compared to the corrupted images <NUM>, artifacts <NUM> such as truncation artifacts are reduced and missing high spatial frequency data <NUM> are recovered in the pristine images <NUM>. In the exemplary embodiment, partial k-space data <NUM> missing high spatial frequency data <NUM> is received. The differences among <FIG> are the different partial sampling patterns in acquiring partial k-space data <NUM>-a, <NUM>-b, <NUM>-c, <NUM>-d (collectively referred to as partial k-space data <NUM>). In <FIG>, for the partial k-space data <NUM>-a, the k-space <NUM> is truncated asymmetrically in one dimension, such as the kx dimension, where the positive kx portion of the k-space <NUM> is skipped while the negative kx portion is fully acquired. In <FIG>, for the partial k-space data <NUM>-b, the k-space <NUM> is truncated asymmetrically in two dimensions, such as the kx and the ky dimensions. In <FIG>, for the partial k-space data <NUM>-c, the k-space <NUM> is truncated asymmetrically in the kx dimension and symmetrically in the ky dimension. In <FIG>, for the partial k-space data <NUM>-d, the k-space <NUM> is truncated asymmetrically in the kx dimension and additionally symmetrically truncated in both the kx and ky dimensions. That is, the partial k-space data <NUM> has a varying partial sampling pattern. In the various partial sampling patterns, the partial sampling factor in the kx or ky dimension may vary. The neural network model is configured to reduce truncation artifacts and recover the missing k-space data for the partial k-space data in varying partial sampling patterns.

In some embodiments, the neural network <NUM> is trained with the corrupted images <NUM> as inputs and residual images <NUM> as target outputs. The residual images <NUM> are difference images between the corrupted images <NUM> and ground truth images <NUM>, which are based on full k-space data corresponding to the partial k-space data <NUM>-a, <NUM>-b, <NUM>-c, <NUM>-d. In <FIG> and <FIG>, the residual image <NUM> is an image of asymmetrical truncation artifacts of the corrupted image <NUM>. In <FIG> and <FIG>, the residual image <NUM> is an image of asymmetrical truncation artifacts and symmetrical truncation artifacts of the corrupted image <NUM> and of high spatial frequency data at spatial frequencies higher than those of the partial k-space data <NUM>-c, <NUM>-d.

The output of the neural network <NUM> may be a residual image or an improved image of the input to the neural network model <NUM>. When the output of the neural network <NUM> is a residual image, the neural network model <NUM> may include one or more layers of neurons configured to generate an improved image based on the output residual image. For example, the improved image is computed as the input image being subtracted by the residual image. As a result, the output image has reduced truncation artifacts and increased high spatial frequency data, compared to the input image to the neural network model <NUM>. Alternatively, the neural network model <NUM> outputs a residual image, and the generation of an improved image based on the residual image is carried out outside the neural network model. In one embodiment, a user is provided with options, such as outputting an improved image, a residual image, or both.

The neural network model <NUM> may be specialized such as being trained to reduce truncation artifacts and recover missing k-space data from asymmetrical truncation in one dimension. The neural network model <NUM> may be generalized such as being trained to reduce truncation artifacts and recover missing k-space data from asymmetrical truncation in one or more dimensions and/or symmetrical truncation in one or more dimensions. As more generalized the neural network model <NUM> gets, more training data is needed for training the neural network model <NUM> for the neural network model to be used to inference of improved images for partial k-space data in various truncation patterns and truncation factors. The computation burden therefore is increased. For example, to train the neural network model shown in <FIG>, asymmetrical partial k-space data in the same dimension of various partial sampling factors or corrupted images based on such partial k-space data are provided as inputs. In another example, to train the neural network model shown in <FIG>, corrupted images based on partial k-space data in various symmetrical partial sampling factors in the kx dimension, various symmetrical partial sampling factors in the ky dimension, and various asymmetrical partial sampling factors in the kx dimension are provided as inputs. Because the number of training image pairs are largely increased and complexity of partial sampling patterns is greatly increase, the complexity of the truncation artifacts increases and the training of the neural network model <NUM> in <FIG> is much more computationally intensive and time consuming that that of the neural network model <NUM> in <FIG>.

The neural network model <NUM> includes input layers for conjugate reflections of k-space data or conjugate reflection images reconstructed from the conjugate reflections (<FIG>). As described above, MRI signals/k-space data and an MR image are represented by complex numbers. A conjugate reflection of k-space data at k-space location k is a complex conjugate of the k-space data at k-space location - k, as shown in Eq. (<NUM>) below: <MAT> where Scj(k) is a conjugate reflection at k-space location k, S(-k) is the original k-space data at k-space location -k, and * represents a complex conjugate.

In other words, to synthesize a conjugate reflection of the original k-space data, each complex number at each k-space location is conjugated and reflected across the origin. For example, k-space data in the first quadrant in the conjugate reflection are complex conjugates of the original k-space data in the third quadrant. A conjugate reflection image is derived by Fourier transform of the conjugate reflections. Conjugate reflections or conjugate reflection images may be input into the neural network model <NUM> during the training as part of the training corrupted images, or during inferencing as being inputted together with the original partial k-space data or crude images based on the original partial k-space data. <FIG> shows a comparison of a real component <NUM>-o, <NUM>-vc, an imaginary component <NUM>-o, <NUM>-c, and a magnitude component <NUM>-o, <NUM>-vc of an original complex image <NUM>-o and the conjugate reflection image <NUM>-vc of the complex image <NUM>-o. The magnitude images <NUM>-o, <NUM>-vc are the same. <FIG> shows exemplary conjugate reflections <NUM>-a, <NUM>-b, <NUM>-c of original k-space data <NUM>-a, <NUM>-b, <NUM>-c. The original k-space data <NUM>-a, <NUM>-b, <NUM>-c were acquired with different k-space partial sampling patterns (also see <FIG>), where the k-space is asymmetrically truncated in the kx-dimension in k-space data <NUM>-a, asymmetrically truncated in both the kx- and ky dimensions in the k-space data <NUM>-b, and asymmetrical truncated in the kx-dimension and symmetrically truncated in the ky dimension in the original k-space data <NUM>-c.

<FIG> shows a comparison of an image <NUM> reconstructed with zero-filling, an image <NUM> output by the neural network model <NUM> having conjugate reflection input layers, and an image <NUM> output by the neural network model <NUM> without conjugate reflection input layers. The partial k-space data is from a partial k-space asymmetrically truncated in the left-right (kx) dimension and symmetrically truncated in both the kx and ky dimensions with a zero filling interpolation (ZIP) factor of <NUM>. A ZIP factor indicates the extent of symmetrical zero padding in the kx or ky dimension. The image resolution of the reconstructed image with zero padding is increased by a factor indicated by the ZIP factor. For example, if the image resolution before the zero padding is 128x128, the reconstructed image by zero padding with a ZIP factor of <NUM> in both dimensions has an image resolution of 256x256. In the neural network model with conjugate reflection input layers, conjugate reflections of the partial k-space data are provided as additional inputs to the neural network model <NUM>. The image <NUM>, <NUM> output by the neural network model <NUM> with or without additional inputs of conjugate reflections has reduced truncation artifacts <NUM>, compared to the image <NUM> reconstructed by zero-filling. Compared to the image <NUM>, the artifacts <NUM> in the image <NUM> output by the neural network model <NUM> with additional inputs of conjugate reflections is further reduced to a level of being not visually noticeable. Conjugate reflections <NUM> provide a different representation of the partial k-space data <NUM>, and improve the image quality output from the neural network model <NUM>.

<FIG> shows an embodiment of acquiring k-space data of various partial sampling patterns in a multi-acquisition pulse sequence. In the exemplary embodiment, four acquisitions are acquired. The multiple acquisitions may be acquired as multiple shots, multiple phases, or multiple number of acquisitions (NEX). The k-space is asymmetrically truncated in the kx and the ky dimensions. In acquisition <NUM>, positive kx and negative ky locations are truncated, where k-space data at those locations are not acquired. In acquisition <NUM>, negative kx and negative ky locations are truncated. In acquisition <NUM>, positive kx and positive ky locations are truncated. In acquisition <NUM>, negative kx and positive ky locations are truncated. To adjust the partial sampling pattern in the kx-dimension, the echo time may be adjusted to sample different portion of the echo. To adjust the partial sampling pattern in the ky-dimension, in Cartesian acquisition, ky lines of the truncated locations are not acquired, where truncated locations are locations in the k-space that k-space data are not acquired. The partial k-space data from the multiple acquisitions are input into the neural network model <NUM>. The k-space data from the multiple acquisitions are acquired with complementary partial sampling patterns, where k-space locations not sampled in one acquisition are sampled in at least one of the other acquisitions, and provide complementary information in the k-space data to each other. The complementary sampling patterns along axes of a 2D Cartesian coordinate system described above are illustrated as an example only. Similar to truncation patterns, complementary sampling patterns may be along axes of a 2D/3D Cartesian coordinate system, a 2D/3D non-Cartesian coordinate system such as a polar, spherical, or cylindrical coordinate system, or a combination thereof. The k-space data from the multiple acquisitions are jointly processed by the neural network model <NUM>, and in the meantime, the image quality of the image from each acquisition and a composite image from a combination of the multiple acquisitions is improved due to the complementary information.

<FIG> is a comparison of images of a digital phantom reconstructed with the deep learning (DL) methods described herein and with known methods. An image <NUM> is the target image. Images <NUM>-zf, <NUM>-dl, <NUM>-pocs, <NUM>-hd are images reconstructed by zero-filing, the methods described herein, POCS, and homodyne, respectively. Images <NUM>-dl, <NUM>-pocs, <NUM>-hd are the differences between the target image <NUM> and the reconstructed images <NUM>-zf, <NUM>-dl, <NUM>-pocs, <NUM>-hd. In this example, the neural network model <NUM> was trained to remove truncation artifacts in the left-right (kx) dimension only. The partial sampling factor was <NUM>. As shown in <FIG>, the systems and methods described herein outperform both iterative POCS and homodyne reconstruction methods in terms of edge sharpness and contrast preservation.

<FIG> shows axial abdominal images <NUM> (top row) and sagittal knee images <NUM> (bottom row) images reconstructed with zero-filling and the DL methods described herein. The Abdominal images <NUM> are acquired with a single-shot fast spin-echo sequence. The knee images <NUM> are acquired with a fast spin-echo sequence. Images <NUM>-zf, <NUM>-zf are magnitude images reconstructed by zero-filling. Images <NUM>-dl, <NUM>-dl are residual images output by the neural network model <NUM> that include truncation artifacts. Images <NUM>-dl, <NUM>-dl are magnitude images of images reconstructed by DL methods. Images <NUM>, <NUM> are phase images of images reconstructed by DL methods. As shown in <FIG>, the truncation artifacts in images <NUM>-dl, <NUM>-dl are largely reduced when reconstructed by DL methods, compared to images <NUM>-zf, <NUM>-zf when reconstructed by zero-filling. Phase information is substantially preserved, as shown in images <NUM>, <NUM>. In this example, the neural network model <NUM> is trained for half NEX and ZIP factor <NUM> in both phase- and frequency-encoding dimensions.

In some embodiments, k-space data is acquired by a multichannel/multi-coil RF coil, and the input to the neural network model <NUM> is k-space data or an image acquired by individual channels of the RF coil. The k-space data or images acquired by individual coils are input into the neural network model <NUM> separately and the outputs from the neural network model <NUM> are combined into one image. Coil sensitivity maps are applied in generating the combined image.

<FIG> depicts an exemplary artificial neural network model <NUM>. The exemplary neural network model <NUM> includes layers of neurons <NUM>, <NUM>-<NUM> to <NUM>-n, and <NUM>, including an input layer <NUM>, one or more hidden layers <NUM>-<NUM> through <NUM>-n, and an output layer <NUM>. Each layer may include any number of neurons, i.e., q, r, and n in <FIG> may be any positive integers. It should be understood that neural networks of a different structure and configuration from that depicted in <FIG> may be used to achieve the methods and systems described herein.

In the exemplary embodiment, the input layer <NUM> may receive different input data. For example, the input layer <NUM> includes a first input a<NUM> representing training images, a second input a<NUM> representing patterns identified in the training images, a third input a<NUM> representing edges of the training images, and so on. The input layer <NUM> may include thousands or more inputs. In some embodiments, the number of elements used by the neural network model <NUM> changes during the training process, and some neurons are bypassed or ignored if, for example, during execution of the neural network, they are determined to be of less relevance.

In the exemplary embodiment, each neuron in hidden layer(s) <NUM>-<NUM> through <NUM>-n processes one or more inputs from the input layer <NUM>, and/or one or more outputs from neurons in one of the previous hidden layers, to generate a decision or output. The output layer <NUM> includes one or more outputs each indicating a label, confidence factor, weight describing the inputs, and/or an output image. In some embodiments, however, outputs of the neural network model <NUM> are obtained from a hidden layer <NUM>-<NUM> through <NUM>-n in addition to, or in place of, output(s) from the output layer(s) <NUM>.

In some embodiments, each layer has a discrete, recognizable function with respect to input data. For example, if n is equal to <NUM>, a first layer analyzes the first dimension of the inputs, a second layer the second dimension, and the final layer the third dimension of the inputs. Dimensions may correspond to aspects considered strongly determinative, then those considered of intermediate importance, and finally those of less relevance.

In other embodiments, the layers are not clearly delineated in terms of the functionality they perform. For example, two or more of hidden layers <NUM>-<NUM> through <NUM>-n may share decisions relating to labeling, with no single layer making an independent decision as to labeling.

<FIG> depicts an example neuron <NUM> that corresponds to the neuron labeled as "<NUM>,<NUM>" in hidden layer <NUM>-<NUM> of <FIG>, according to one embodiment. Each of the inputs to the neuron <NUM> (e.g., the inputs in the input layer <NUM> in <FIG>) is weighted such that input a<NUM> through ap corresponds to weights w1 through wp as determined during the training process of the neural network model <NUM>.

In some embodiments, some inputs lack an explicit weight, or have a weight below a threshold. The weights are applied to a function α (labeled by a reference numeral <NUM>), which may be a summation and may produce a value z<NUM> which is input to a function <NUM>, labeled as f<NUM>,<NUM>(z<NUM>). The function <NUM> is any suitable linear or non-linear function. As depicted in FIG. 5B, the function <NUM> produces multiple outputs, which may be provided to neuron(s) of a subsequent layer, or used as an output of the neural network model <NUM>. For example, the outputs may correspond to index values of a list of labels, or may be calculated values used as inputs to subsequent functions.

It should be appreciated that the structure and function of the neural network model <NUM> and the neuron <NUM> depicted are for illustration purposes only, and that other suitable configurations exist. For example, the output of any given neuron may depend not only on values determined by past neurons, but also on future neurons.

The neural network model <NUM> may include a convolutional neural network (CNN), a deep learning neural network, a reinforced or reinforcement learning module or program, or a combined learning module or program that learns in two or more fields or areas of interest. Supervised and unsupervised machine learning techniques may be used. In supervised machine learning, a processing element may be provided with example inputs and their associated outputs, and may seek to discover a general rule that maps inputs to outputs, so that when subsequent novel inputs are provided the processing element may, based upon the discovered rule, accurately predict the correct output. The neural network model <NUM> may be trained using unsupervised machine learning programs. In unsupervised machine learning, the processing element may be required to find its own structure in unlabeled example inputs. Machine learning may involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data. Models may be created based upon example inputs in order to make valid and reliable predictions for novel inputs.

Additionally or alternatively, the machine learning programs may be trained by inputting sample data sets or certain data into the programs, such as images, object statistics, and information. The machine learning programs may use deep learning algorithms that may be primarily focused on pattern recognition, and may be trained after processing multiple examples. The machine learning programs may include Bayesian Program Learning (BPL), voice recognition and synthesis, image or object recognition, optical character recognition, and/or natural language processing - either individually or in combination. The machine learning programs may also include natural language processing, semantic analysis, automatic reasoning, and/or machine learning.

Based upon these analyses, the neural network model <NUM> may learn how to identify characteristics and patterns that may then be applied to analyzing image data, model data, and/or other data. For example, the model <NUM> may learn to identify features in a series of data points.

<FIG> is a block diagram of an exemplary CNN <NUM> that may be included in the neural network model <NUM>. The CNN <NUM> includes a convolutional layer <NUM>. In a convolutional layer, convolution is used in place of general matrix multiplication in a neural network model. In one example, a 1x1 convolution is used to reduce the number of channels in the neural network <NUM>. The neural network <NUM> includes one or more convolutional layer blocks <NUM>, a fully-connected layer <NUM> where the neurons in this layer is connected with every neuron in the prior layer, and an output layer <NUM> that provides outputs.

In the exemplary embodiment, the convolutional layer block <NUM> includes a convolutional layer <NUM> and a pooling layer <NUM>. Each convolutional layer <NUM> is flexible in terms of its depth such as the number of convolutional filters and sizes of convolutional filters. The pooling layer <NUM> is used to streamline the underlying computation and reduce the dimensions of the data by combining outputs of neuron clusters at the prior layer into a single neuron in the pooling layer <NUM>. The convolutional layer block <NUM> may further include a normalization layer <NUM> between the convolutional layer <NUM> and the pooling layer <NUM>. The normalization layer <NUM> is used to normalize the distribution within a batch of training images and update the weights in the layer after the normalization. The number of convolutional layer blocks <NUM> in the neural network <NUM> may depend on the image quality of training images, and levels of details in extracted features.

In operation, in training, training images and other data such as extracted features of the training images are inputted into one or more convolutional layer blocks <NUM>. Observed masks corresponding to the training images are provided as outputs of the output layer <NUM>. Neural network <NUM> is adjusted during the training. Once the neural network <NUM> is trained, an input image is provided to the one or more convolutional layer blocks <NUM> and the output layer <NUM> provides outputs that include a mask associated with the input image.

The workstation <NUM> and the truncation artifact reduction computing device <NUM>, <NUM> described herein may be any suitable computing device <NUM> and software implemented therein. <FIG> is a block diagram of an exemplary computing device <NUM>. In the exemplary embodiment, the computing device <NUM> includes a user interface <NUM> that receives at least one input from a user. The user interface <NUM> may include a keyboard <NUM> that enables the user to input pertinent information. The user interface <NUM> may also include, for example, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad and a touch screen), a gyroscope, an accelerometer, a position detector, and/or an audio input interface (e.g., including a microphone).

Moreover, in the exemplary embodiment, computing device <NUM> includes a display interface <NUM> that presents information, such as input events and/or validation results, to the user. The display interface <NUM> may also include a display adapter <NUM> that is coupled to at least one display device <NUM>. More specifically, in the exemplary embodiment, the display device <NUM> may be a visual display device, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light-emitting diode (LED) display, and/or an "electronic ink" display. Alternatively, the display interface <NUM> may include an audio output device (e.g., an audio adapter and/or a speaker) and/or a printer.

The computing device <NUM> also includes a processor <NUM> and a memory device <NUM>. The processor <NUM> is coupled to the user interface <NUM>, the display interface <NUM>, and the memory device <NUM> via a system bus <NUM>. In the exemplary embodiment, the processor <NUM> communicates with the user, such as by prompting the user via the display interface <NUM> and/or by receiving user inputs via the user interface <NUM>. The term "processor" refers generally to any programmable system including systems and microcontrollers, reduced instruction set computers (RISC), complex instruction set computers (CISC), application specific integrated circuits (ASIC), programmable logic circuits (PLC), and any other circuit or processor capable of executing the functions described herein. The above examples are exemplary only, and thus are not intended to limit in any way the definition and/or meaning of the term "processor.

In the exemplary embodiment, the memory device <NUM> includes one or more devices that enable information, such as executable instructions and/or other data, to be stored and retrieved. Moreover, the memory device <NUM> includes one or more computer readable media, such as, without limitation, dynamic random access memory (DRAM), static random access memory (SRAM), a solid state disk, and/or a hard disk. In the exemplary embodiment, the memory device <NUM> stores, without limitation, application source code, application object code, configuration data, additional input events, application states, assertion statements, validation results, and/or any other type of data. The computing device <NUM>, in the exemplary embodiment, may also include a communication interface <NUM> that is coupled to the processor <NUM> via the system bus <NUM>. Moreover, the communication interface <NUM> is communicatively coupled to data acquisition devices.

In the exemplary embodiment, the processor <NUM> may be programmed by encoding an operation using one or more executable instructions and providing the executable instructions in the memory device <NUM>. In the exemplary embodiment, the processor <NUM> is programmed to select a plurality of measurements that are received from data acquisition devices.

In operation, a computer executes computer-executable instructions embodied in one or more computer-executable components stored on one or more computer-readable media to implement aspects of the invention described and/or illustrated herein. The order of execution or performance of the operations in embodiments of the invention illustrated and described herein is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments of the invention may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the invention.

At least one technical effect of the systems and methods described herein includes (a) reduction of truncation artifacts; (b) increase of high spatial frequency information at the same time as reduction of truncation artifacts; (c) one neural network model for reduction of truncation artifacts caused by various partial sampling patterns; and (d) the use of conjugate reflection to increase image quality of images output from the neural network model.

Exemplary embodiments of systems and methods of truncation artifacts reduction are described above in detail. The systems and methods are not limited to the specific embodiments described herein but, rather, components of the systems and/or operations of the methods may be utilized independently and separately from other components and/or operations described herein. Further, the described components and/or operations may also be defined in, or used in combination with, other systems, methods, and/or devices, and are not limited to practice with only the systems described herein.

Although specific features of various embodiments of the invention may be shown in some drawings and not in others, this is for convenience only. In accordance with the principles of the invention, any feature of a drawing may be referenced and/or claimed in combination with any feature of any other drawing.

Claim 1:
A computer-implemented method of removing truncation artifacts in magnetic resonance (MR) images, comprising:
receiving a crude image that is based on partial k-space data (<NUM>) from a partial k-space (<NUM>) asymmetrically truncated in at least one k-space dimension at k-space locations corresponding to high spatial frequencies (<NUM>);
analyzing (<NUM>) the crude image using a neural network model (<NUM>, <NUM>), wherein the neural network model was trained with pairs of pristine images (<NUM>) and corrupted images (<NUM>), wherein the corrupted images are based on partial k-space data from partial k-spaces (<NUM>) truncated in one or more partial sampling patterns (<NUM>) at the k-space locations corresponding to the high spatial frequencies (<NUM>), the one or more partial sampling patterns including an asymmetrical truncation in at least one k-space dimension, the pristine images are based on full k-space data (<NUM>) corresponding to the partial k-space data of the corrupted images, and target output images (<NUM>) of the neural network model are the pristine images;
deriving (<NUM>) an improved image of the crude image based on the analysis, wherein the derived improved image includes reduced truncation artifacts (<NUM>) and increased high spatial frequency data, compared to the crude image; and
outputting (<NUM>) the improved image;
wherein the partial k-space data of the corrupted images (<NUM>) include a first set of k-space data and a second set of k-space data, wherein the second set of k-space data are conjugate reflections (<NUM>) of the first set of k-space data.