Patent Description:
Medical images are typically acquired for a specific medical procedure focusing on a specific body part of a patient. Such medical images are acquired having an initial field of view depicting an observed area, such as the specific body part of the patient. The initial field of view represents the region within an outer perimeter of the observed area of the medical image. However, the initial field of view of such medical images is often limited in scope.

Conventional approaches for augmenting images typically focus on image completion (i.e., inpainting), where one or more missing portions within the initial field of view of an image are reconstructed. Such conventional approaches to image completion are limited to generating imaging data for the missing portion within the initial field of view where the missing portion is surrounded on all sides by a priori imaging data. <NPL> discloses a data-driven method to predict the quality of an image completion method. The method uses automatically derived search space constraints for patch source regions to improved texture synthesis and an algorithm to crop and complete stitched panoramas. <NPL>discloses a method for developing a real-time and user interactive stitching application for x-ray images with different intensities and orientations.

In accordance with one or more embodiments, systems and methods are provided for generating an augmented medical imaging having an expanded field of view, which comprises an initial field of view of an initial medical image and an augmentation region outside of the initial field of view. Providing such an expanded field of view is beneficial in a number of use cases. For example, an expanded field of view would provide additional context to a clinician on anatomical structures depicted in the initial medical images. In another example, an expanded field of view would improve performance of medical imaging analysis algorithms that are designed for images having a larger field of view, such as, e.g., landmark detectors and image registration algorithms. There are no known conventional methods for providing an expanded field of view depicting an augmentation region that is outside of the initial field of view.

In accordance with one or more embodiments, systems and methods are provided for enhancing a medical image. An initial medical image having an initial field of view is received. An augmented medical image having an expanded field of view is generated using a trained machine learning model, wherein the trained machine learning model is a trained generative adversarial network comprising a generator, the generator comprises an encoder and a decoder, and the generator and a discriminator are used to train the generative adversarial network. The trained machine learning model is trained using pairs of training images, each pair of training images comprising an initial training image and a corresponding cropped training image, and the corresponding cropped training image is generated by cropping the initial training image. The expanded field comprises the initial field of view and an augmentation region. The augmentation region comprises a region immediately surrounding the initial field of view of the initial medical image. The augmented medical image is output.

The augmented medical image may be a different modality than a modality of the initial medical image. In one embodiment, the augmented medical image is generated based on a prior initial medical image and a prior augmented medical image.

The trained generative adversarial network may be based on at least one of a U-Net, a context encoder, and a variational auto-encoder. In one embodiment, the augmented medical image may be refined using another trained machine learning model.

In one embodiment, the initial field of view may be a high-quality region of an x-ray image and the expanded field of view comprises the high-quality region and the augmentation region, wherein the high-quality region results from high density-x-ray beams. In another embodiment, the initial medical image comprises an anatomical model.

The present invention generally relates to systems and methods for medical image enhancement. Embodiments of the present invention are described herein to give a visual understanding of such systems and methods. A digital image is often composed of digital representations of one or more objects (or shapes). The digital representation of an object is often described herein in terms of identifying and manipulating the objects. Such manipulations are virtual manipulations accomplished in the memory or other circuitry/hardware of a computer system. Accordingly, is to be understood that embodiments of the present invention may be performed within a computer system using data stored within the computer system.

Further, it should be understood that while the embodiments discussed herein may be discussed with respect to the enhancement of medical images, the present invention is not so limited. Embodiments of the present invention may be applied for the enhancement of any type of image.

<FIG> shows an initial DynaCT medical image <NUM> of a patient (or any other subject). Initial medical image <NUM> may be acquired to facilitate a clinical examination of the patient, such as, e.g., angiography. Initial medical image <NUM>, as initially acquired, has an initial field of view <NUM>, defined as the region within an outer perimeter <NUM> of the observed area of initial medical image <NUM>. Initial field of view <NUM> of initial medical image <NUM> may be limiting for some medical imaging analysis tasks.

Embodiments of the present invention employ one or more neural networks for generating an augmented medical image having an expanded field of view comprising initial field of view <NUM> and augmentation region <NUM>. Augmentation region <NUM> is the region between outer perimeter <NUM> of initial field of view <NUM> and outer perimeter <NUM> of the expanded field of view, and does not include any region within outer perimeter <NUM> (e.g., any region surrounded by imaging data of initial medical image <NUM>). By generating the augmented medical image, imaging data for augmentation region <NUM> is thereby generated to augment initial medical image <NUM>. Advantageously, an augmented medical imaging having an expanded field of view comprising initial field of view <NUM> and augmentation region <NUM> would, e.g., provide additional context to a clinician on anatomical structures depicted within initial field of view <NUM> of initial medical image <NUM> and improve performance of medical imaging analysis algorithms (e.g., landmark detectors and image registration algorithms). The generation of the augmented medical image is a complex task since augmentation region <NUM> only has imaging data of initial medical image <NUM> on, e.g., one side, and is not surrounded on all sides by the imaging data. The generation of the augmented medical image becomes more difficult as the distance between the pixels that it has to generate in augmentation region <NUM> and outer perimeter <NUM> becomes larger.

<FIG> and <FIG> will now be simultaneously discussed. <FIG> shows an illustrative high level workflow <NUM> for enhancing a medical image, in accordance with one or more embodiments. <FIG> shows a method <NUM> for implementing workflow <NUM> of <FIG>. Method <NUM> may be performed by any suitable computing device, such as, e.g., computer <NUM> of <FIG>.

At step <NUM> of <FIG>, an initial medical image having an initial field of view is received. An exemplary initial medical image <NUM> having initial field of view <NUM> is shown in workflow <NUM> of <FIG>. Initial field of view <NUM> is defined as the region within an outer perimeter <NUM> of the observed area of initial medical image <NUM>. The initial medical image <NUM> may be a two dimensional (2D) or three dimensional (3D) image of any suitable modality, such as, e.g., DynaCT, computed tomography (CT), x-ray, magnetic resonance imaging (MRI), ultrasound (US), single-photon emission computed tomography (SPECT), positron emission tomography (PET), or any other suitable modality or combination of modalities. Initial medical image <NUM> may be received directly from an image acquisition device (e.g., image acquisition device <NUM> of <FIG>), or may be received by loading initial medical image <NUM> from a memory or storage of a computer system, or by receiving initial medical image <NUM> at a computer system via a network transmission from another computer system.

At step <NUM> of <FIG>, an augmented medical image having an expanded field of view is generated from the initial medical image using a trained machine learning model. An exemplary augmented medical image <NUM> is shown in workflow <NUM> of <FIG> having an expanded field of view comprising initial field of view <NUM> and augmentation region <NUM>. Augmentation region <NUM> is the region between outer perimeter <NUM> of initial field of view <NUM> and outer perimeter <NUM> of the expanded field of view, and does not include any region within outer perimeter <NUM> (e.g., any area surrounded by imaging data of initial medical image <NUM>). Augmentation region <NUM> may be of any size or shape. Augmentation region <NUM> is the region immediately surrounding outer perimeter <NUM>, as illustratively shown in workflow <NUM>. By generating augmented medical image <NUM>, imaging data for augmentation region <NUM> is thereby generated to augment initial medical image <NUM>. In some embodiments, an entirely new augmented medical image <NUM> may be generated by generating augmentation region <NUM> and regenerating initial field of view <NUM>, while in other embodiments augmented medical image <NUM> is generated by only generating augmentation region <NUM> while copying the imaging data of initial field of view <NUM> from initial medical image <NUM>.

In one embodiment, augmented medical image <NUM> is generated using the trained machine learning model such that augmented medical image <NUM> is a different modality than the modality of the initial medical image <NUM>. Accordingly, the trained machine learning model generates augmented medical image <NUM> to simultaneously augment the initial medical image <NUM> in augmentation region <NUM> while also performing style transfer of initial medical image <NUM> from one modality to another (e.g., DynaCT to CT).

The trained machine learning model may be any suitable machine learning model, such as, e.g., a neural network. The trained machine learning model is a generative adversarial network (GAN), as described in more detail below with respect to <FIG>. The GAN may be implemented using a U-Net, a context encoder, and/or a variational auto-encoder (VAE).

The trained machine learning model is trained in a prior training stage using pairs of training images. The training images are generated by cropping initial training images such that each initial training image and its corresponding cropped training image form a pair of training images. The initial training image may be cropped according to various shapes and sizes. The initial training images may be patient-specific training images or synthetically generated training images.

At step <NUM> of <FIG>, optionally, augmented medical image <NUM> is refined using another trained machine learning model to generate refined medical image <NUM>. Refined medical image <NUM> may be generated by de-noising the augmented medical image <NUM>, applying super-resolution imaging techniques on augmented medical image <NUM>, or any other form of refinement or enhancement as is known in the art.

The other trained machine learning model may be any suitable machine learning model, such as, e.g., a neural network. The other trained machine learning model is trained in a prior training stage using pairs of training images. The training images may be generated by blurring initial training images such that each initial training image and its corresponding blurred training image form a pair of training images. The training images may be patient-specific training images or synthetic training images. In one embodiment, a same set of initial training images is used for training the other machine learning model and for training the machine learning network (applied at step <NUM>).

At step <NUM> of <FIG>, the refined medical image <NUM> is output. The refined medical image <NUM> can be output by displaying the refined medical image <NUM> on a display device of a computer system, storing the refined medical image <NUM> on a memory or storage of a computer system, or by transmitting the refined medical image <NUM> to a remote computer system, e.g., for further processing.

In accordance with one embodiment, method <NUM> may be applied to generate hyper-realistic collimated images. X-ray imaging is very sensitive to collimation. When x-ray beams targeting a patient are not perfectly aligned, the generated image has two regions: a high-quality region resulting from a higher density of x-ray beams, and a low-quality region resulting from a lower density of x-ray beams. Method <NUM> may be applied to solve the collimation issue by considering the high-quality region of the x-ray image to be the initial field of view <NUM> of the initial medical image <NUM> (received at step <NUM>). Method <NUM> continues at step <NUM> to generate an augmented medical image <NUM> having expanded field of view comprising the high-quality region of the x-ray region and augmentation region <NUM>, conditioned on the low-quality region of the x-ray image. For example, the low-quality region of the x-ray image may be used as an additional input into the machine learning models, or may be used in a post-process step to further refine the output of the machine learning models. Accordingly, method <NUM> may act as a collimation calibration step, resulting in a high-quality hyper-realistic collimated image.

In accordance with one embodiment, method <NUM> may be applied to enhance information and knowledge extracted from medical images, such as, e.g., an anatomical model. Given a medical image depicting, e.g., a portion of an arterial tree, an anatomical model (in one, two, or three dimensions) of the arterial tree may be reconstructed from the medical image. Method <NUM> may be applied to enhance the anatomical model by considering the anatomical model to be the initial field of view <NUM>. Accordingly, at step <NUM>, an augmented anatomical model is generated having an expanded field of view comprising initial field of view <NUM> and augmentation region <NUM>. In this embodiment, the refining step <NUM> may not be performed. The generated data may have a different dimensionality than the anatomical model. For example, the anatomical model may be 3D while the generated data may have zero dimensionality (e.g., resistances, compliances, etc.). The machine learning network applied at step <NUM> may be trained during a prior training stage starting with a large existing dataset of initial training images. To generate the training data, a cropped version of each initial training image is generated. A reconstructed anatomical model is generated from the cropped training images (with the desired dimensionality). A reconstruction anatomical model is generated for the augmentation region of the cropped training images (with the same or different dimensionality) using the initial training images. Thus, method <NUM> may be applied to generate an enhanced medical image and an enhanced anatomical model.

In accordance with one embodiment, where motion of anatomical structures is of interest, medical imaging techniques, such as, e.g., echocardiography and angiography, are applied to acquired multiple initial medical images of an object of interest over a period of time. Workflow <NUM> of <FIG> and method <NUM> of <FIG> may be modified to generate an augmented medical image and refined medical image based on one or more prior initial medical images and prior augmented medical images, which correspond to previous points in time, as shown in <FIG>.

<FIG> shows a high level workflow <NUM> for enhancing a medical image using one or more prior initial medical images and prior augmented medical images, in accordance with one or more embodiments. In workflow <NUM>, an initial medical image <NUM> having an initial field of view is received. An augmented medical image <NUM> having an expanded field of view is generated from an initial medical image <NUM> using a trained machine learning model (e.g., neural network), and based on prior initial medical images and prior augmented medical images <NUM>, which correspond to one or more previous points in time. Augmented image <NUM> is then refined using another machine learning model (e.g., neural network) based on prior initial medical images and prior augmented medical images <NUM> to generate refined medical image <NUM>.

In one embodiment, the one or more trained machine learning networks used to generate augmented medical image <NUM> and/or refined medical view image <NUM> in workflow <NUM> may each comprise a neural network (e.g., a GAN) implemented with a long short-term memory (LSTM) network, which provides long term memory controlled by opening or closing an input gate, an output gate, and/or a forget gate. Advantageously, the LSTM network enables prior initial medical images and prior augmented medical images <NUM> to be stored and subsequently used to generate more accurate images (i.e., augmented medical image <NUM> and refined medical image <NUM>). It should be understood that the present invention is not limited to LSTM networks; any type of recurrent neural network (RNN) architecture may be employed, such as, e.g., a gated recurrent unit (GRU).

<FIG> shows a GAN <NUM> for generating an augmented medical image, in accordance with one or more embodiments. In one embodiment, GAN <NUM> is the trained machine learning model applied at steps <NUM> and/or <NUM> of <FIG>. GANs represent a framework for creating generative models via an adversarial process. GAN <NUM> comprises two modules in the form of deep networks: a generator <NUM> for generation of a synthesized image <NUM> in a target modality and a discriminator <NUM> for distinguishing between a real image <NUM> and the synthesized image <NUM>. Generator <NUM> generates synthesized image <NUM> in the target modality (e.g., CT) from an input image <NUM> in an initial modality (e.g., DynaCT). Discriminator <NUM> inputs the synthesized image <NUM> generated by generator <NUM> and a real image <NUM> and classifies one image as real and the other image as fake (synthesized). Generator <NUM> and discriminator <NUM> are simultaneously trained such that while discriminator <NUM> is improving in terms of fake image detection, generator <NUM> is improving in terms of producing realistic looking images capable of fooling discriminator <NUM>. Accordingly, generator <NUM> and discriminator <NUM> are trained with adversarial loss to force generator <NUM> to learn the most meaningful features. Discriminator <NUM> is only used during the training stage, and is not used during the online or inference stage, e.g., to generate augmented medical images.

As shown in <FIG>, generator <NUM> comprises a style encoder <NUM> and content encoder <NUM>. Style encoder <NUM> receives a reference image <NUM> in the target domain and encodes reference image <NUM> to a style code representing low level features of reference image <NUM>. Content encoder <NUM> receives real image <NUM> in the initial domain and encodes real image <NUM> to a content code representing low level features of real image <NUM>. Decoder <NUM> generates synthesized image <NUM> based on the content code from content encoder <NUM> and weights based on style code from style encoder <NUM>. It should be understood that generator <NUM> may be implemented using any suitable network.

In one embodiment, generator <NUM> is implemented as a U-Net. A U-Net is a convolutional neural network comprising a contracting path and an expansive path. The contracting path reduces spatial information of the input image while increasing feature information. The expansive path combines the spatial and feature information through a sequence of up-convolutions and concatenations with high-resolution features from the contracting path. Due to its, skip connections, the U-Net is suitable for image completion tasks.

In another embodiment, generator <NUM> is implemented as a context encoder. A context encoder is a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings. Given an image, the network down-samples it to a specific dimension using an encoder, followed by a number of channel-wise fully-connected layers. Finally, a decoder produces the missing region of the image. Due to the fully connected layers, the mapping that generates the missing region of the image is highly conditioned on its surroundings.

In another embodiment, generator <NUM> is implemented as a VAE. The VAE comprises an encoder and a decoder. The encoder encodes an input image as a set of parameters of a statistical distribution (e.g., a mean and a variance). The decoder randomly samples a point from the statistical distribution to reconstruct the input image.

In one embodiment, the GAN shown in block diagram <NUM> may be a CycleGAN to enforce cycle consistency, such that an input image in a first modality translated to a second modality and then translated back to the first modality should return the original input image. <FIG> shows a functional block diagram <NUM> for training a CycleGAN for generating an augmented medical image, in accordance with one or more embodiments.

Functional block diagram <NUM> comprises generator GDCT<NUM>CT for generating synthesized augmented CT images (or images of any other target modality) from an input DynaCT (DCT) image (or an input image of any other initial modality) and generator GCT2DCT for generating synthesized augmented DCT images (or images of any other target modality) from an input CT image (or an input image of any other initial modality). Generators GDCT<NUM>CT and GCT2DCT are trained using DCT input image <NUM> and CT input image <NUM>. Images <NUM> and <NUM> are real images.

Generators GDCT<NUM>CT and GCT2DCT are trained with discriminators DCT and DDCT for adversarial loss. Discriminator DCT aims to distinguish between synthesized CT images generated by generator GDCT<NUM>CT and a real CT image (e.g., synthesized CT image <NUM> and real input CT image <NUM>), and classifies one image as real and the other as fake. Discriminator DDCT aims to distinguish between synthesized DCT images generated by generator GCT2DCT and a real DCT image (e.g., synthesized DCT image <NUM> and real input DCT image <NUM>), and classifies one image as real and the other as fake. Discriminators DCT and DDCT will guide generators GDCT<NUM>CT and GCT2DCT to generate synthesized images that are indistinguishable from the real images in their corresponding modality. Generators GDCT<NUM>CT and GCT2DCT and their discriminators DCT and DDCT are expressed as the objectives of Equations (<NUM>) and (<NUM>), respectively. <MAT> <MAT>.

Cycle consistency is introduced such that a synthesized image in the target domain could return back to the exact image in the source domain that it was generated from. Cycle consistency loss <MAT> compares a real CT image with a synthesized CT image (generated by translating a real CT image to a synthesized DCT image, and translating the synthesized DCT image to the synthesized CT image) (e.g., real CT image <NUM> and synthesized CT image <NUM>). Similarly, cycle consistent loss <MAT> compares a real DCT image with a synthesized DCT image (generated by translating a real DCT image to a synthesized CT image, and translating the synthesized CT image to the synthesized DCT image) (e.g., real DCT image <NUM> and synthesized DCT image <NUM>). Cycle consistency loss for generators GDCT<NUM>CT and GCT2DCT are defined by the following loss function.

Since the CT-like DCT image should be an enhanced version of the DCT image, identity loss is employed to regularize the training procedure. In other words, if a generator sees a real image in the target domain, it should make no changes to it. Identity loss for generators GDCT<NUM>CT and GCT2DCT are defined by the following loss function.

Generators GDCT<NUM>CT and GCT2DCT are also trained with supervision according to the following supervision loss.

A composite objective function is defined below in Equation (<NUM>), as a composite of Equations (<NUM>)-(<NUM>), to train generators GDCT<NUM>CT and GCT<NUM>DCT, where parameter λ represents weights for each loss.

Embodiments of the present invention were experimentally validated using two different configurations of GANs.

In a first experiment, a Pix2Pix-GAN implemented with a context encoder was applied for generating an augmented medical image. <FIG> shows an example of a network architecture for training a Pix2Pix-GAN <NUM>, which does not fall under the scope of the present invention. Pix2Pix-GAN <NUM> comprises generator <NUM> and discriminators <NUM> and <NUM>. Generator <NUM> is a fully convolutional context encoder and discriminators <NUM> and <NUM> are convolutional networks that output a probability indicating whether the image (generated by generator <NUM>) is real or fake (synthesized). Generator <NUM> receives as input a training image <NUM> having a missing region and outputs a synthetic image <NUM> having augmented region <NUM>. To improve the quality of results, two discriminators <NUM> and <NUM> are employed for recognizing fake images. Discriminator <NUM> is a global discriminator that takes as input the entire synthetic image <NUM> to ensure that at a global level the image <NUM> looks realistic, while discriminator <NUM> takes as input only augmented region <NUM> to improve the local coherence. <FIG> shows a comparison <NUM> of images resulting from applying embodiments of the present invention using a Pix2Pix-GAN implemented with a context encoder (i.e., Pix2Pix-GAN <NUM>) with ground truth images. Column <NUM> shows input images, column <NUM> shows ground-truth images, and column <NUM> shows generated images using Pix2Pix-GAN <NUM>.

In a second experiment, a Pix2Pix-GAN implemented with a U-Net was applied for generating an augmented medical image. In particular, in this configuration, the generator of the Pix2Pix-GAN was implemented using a U-Net architecture. The discriminator splits the input image into patches which are classified separately. In other words, the discriminator penalizes the structure only at the patch level, trying to determine whether each of the NxN patches in an image is real or fake (synthesized). The last layer of the network averages all responses to provide the output. <FIG> shows a comparison <NUM> images resulting from applying embodiments of the present invention using the Pix2Pix-GAN implemented with a U-Net with ground truth images. Column <NUM> shows input images, column <NUM> shows ground-truth images, and column <NUM> shows generated images using the Pix2Pix-GAN implemented with a U-Net.

Systems, apparatuses, and methods described herein may be implemented using digital circuitry, or using one or more computers using well-known computer processors, memory units, storage devices, computer software, and other components. Typically, a computer includes a processor for executing instructions and one or more memories for storing instructions and data. A computer may also include, or be coupled to, one or more mass storage devices, such as one or more magnetic disks, internal hard disks and removable disks, magneto-optical disks, optical disks, etc..

Systems, apparatus, and methods described herein may be implemented using computers operating in a client-server relationship. Typically, in such a system, the client computers are located remotely from the server computer and interact via a network. The client-server relationship may be defined and controlled by computer programs running on the respective client and server computers.

Systems, apparatus, and methods described herein may be implemented within a network-based cloud computing system. In such a network-based cloud computing system, a server or another processor that is connected to a network communicates with one or more client computers via a network. A client computer may communicate with the server via a network browser application residing and operating on the client computer, for example. A client computer may store data on the server and access the data via the network. A client computer may transmit requests for data, or requests for online services, to the server via the network. The server may perform requested services and provide data to the client computer(s). The server may also transmit data adapted to cause a client computer to perform a specified function, e.g., to perform a calculation, to display specified data on a screen, etc. For example, the server may transmit a request adapted to cause a client computer to perform one or more of the steps or functions of the methods and workflows described herein, including one or more of the steps or functions of <FIG>. Certain steps or functions of the methods and workflows described herein, including one or more of the steps or functions of <FIG>, may be performed by a server or by another processor in a network-based cloud-computing system. Certain steps or functions of the methods and workflows described herein, including one or more of the steps of <FIG>, may be performed by a client computer in a network-based cloud computing system. The steps or functions of the methods and workflows described herein, including one or more of the steps of <FIG>, may be performed by a server and/or by a client computer in a network-based cloud computing system, in any combination.

Systems, apparatus, and methods described herein may be implemented using a computer program product tangibly embodied in an information carrier, e.g., in a non-transitory machine-readable storage device, for execution by a programmable processor; and the method and workflow steps described herein, including one or more of the steps or functions of <FIG>, may be implemented using one or more computer programs that are executable by such a processor. A computer program is a set of computer program instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result.

A high-level block diagram of an example computer <NUM> that may be used to implement systems, apparatus, and methods described herein is depicted in <FIG>. Computer <NUM> includes a processor <NUM> operatively coupled to a data storage device <NUM> and a memory <NUM>. Processor <NUM> controls the overall operation of computer <NUM> by executing computer program instructions that define such operations. The computer program instructions may be stored in data storage device <NUM>, or other computer readable medium, and loaded into memory <NUM> when execution of the computer program instructions is desired. Thus, the method and workflow steps or functions of <FIG> can be defined by the computer program instructions stored in memory <NUM> and/or data storage device <NUM> and controlled by processor <NUM> executing the computer program instructions. For example, the computer program instructions can be implemented as computer executable code programmed by one skilled in the art to perform the method and workflow steps or functions of <FIG>. Accordingly, by executing the computer program instructions, the processor <NUM> executes the method and workflow steps or functions of <FIG>. Computer <NUM> may also include one or more network interfaces <NUM> for communicating with other devices via a network. Computer <NUM> may also include one or more input/output devices <NUM> that enable user interaction with computer <NUM> (e.g., display, keyboard, mouse, speakers, buttons, etc.).

Processor <NUM> may include both general and special purpose microprocessors, and may be the sole processor or one of multiple processors of computer <NUM>. Processor <NUM> may include one or more central processing units (CPUs), for example. Processor <NUM>, data storage device <NUM>, and/or memory <NUM> may include, be supplemented by, or incorporated in, one or more application-specific integrated circuits (ASICs) and/or one or more field programmable gate arrays (FPGAs).

Data storage device <NUM> and memory <NUM> each include a tangible non-transitory computer readable storage medium. Data storage device <NUM>, and memory <NUM>, may each include high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR RAM), or other random access solid state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices such as internal hard disks and removable disks, magneto-optical disk storage devices, optical disk storage devices, flash memory devices, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), digital versatile disc read-only memory (DVD-ROM) disks, or other non-volatile solid state storage devices.

Input/output devices <NUM> may include peripherals, such as a printer, scanner, display screen, etc. For example, input/output devices <NUM> may include a display device such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor for displaying information to the user, a keyboard, and a pointing device such as a mouse or a trackball by which the user can provide input to computer <NUM>.

An image acquisition device <NUM> can be connected to the computer <NUM> to input image data (e.g., medical images) to the computer <NUM>. It is possible to implement the image acquisition device <NUM> and the computer <NUM> as one device. It is also possible that the image acquisition device <NUM> and the computer <NUM> communicate wirelessly through a network. In a possible embodiment, the computer <NUM> can be located remotely with respect to the image acquisition device <NUM>.

Any or all of the systems and apparatus discussed herein may be implemented using one or more computers such as computer <NUM>.

One skilled in the art will recognize that an implementation of an actual computer or computer system may have other structures and may contain other components as well, and that <FIG> is a high level representation of some of the components of such a computer for illustrative purposes.

The foregoing Detailed Description is to be understood as being in every respect illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is not to be determined from the Detailed Description, but rather from the appended claims.

Claim 1:
A computer-implemented method for enhancing a medical image, comprising:
receiving (<NUM>) an initial medical image having an initial field of view;
generating (<NUM>) an augmented medical image having an expanded field of view using a trained machine learning model, the expanded field of view comprising the initial field of view and an augmentation region, the augmentation region comprising a region immediately surrounding the initial field of view of the initial medical image; and
outputting (<NUM>) the augmented medical image;
wherein the trained machine learning model is a trained generative adversarial network comprising a generator and a discriminator, the generator comprises an encoder and a decoder, and the generator and the discriminator are used to train the generative adversarial network, the trained machine learning model is trained using pairs of training images, each pair of training images comprising an initial training image and a corresponding cropped training image, and the corresponding cropped training image is generated by cropping the initial training image.