Patent Description:
The invention refers to a technology developed through the "<NPL>). This application is based on and claims priority under <NUM> U. §<NUM> to <CIT>, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.

When computed tomography (CT) or magnetic resonance imaging (MRI) is performed, contrast agents are administered to a patient to make specific tissues, blood vessels, or lesions more visible. For example, an abdominal CT scan may be taken before injecting contrast agents into a patient, and an abdominal CT scan may be taken again after injecting contrast agents into the patient. Medical images such as CT images may be divided into a plurality of phases (e.g., pre, artery, post, delay, etc.) according to the degree of contrast enhancement after administration of the contrast agents. A medical staff may give a diagnosis by comparing medical images of different phases. For example, a medical staff may diagnose liver cancer based on the size and location of liver lesions appearing in medical images of various phases. However, when CT or MRI is performed, the position and size of the liver and lesion appearing on a medical image of each phase may be different due to the movement or breathing of a patient. In this case, the diagnosis accuracy is low.

Patent publication <CIT> discloses a medical image prcessing method and computing device which comprise acquiring series of enhanced medical images, detecting a phase of each enhanced medical image in the series of enhanced medical images using a pre-trained 3D convolutional neural network model, selecting a plurality of target enhanced medical images from all the enhanced medical images according to the phases, obtaining a plurality of interest images by segmenting an interest region in each of the plurality of target enhanced medical images, and registering the plurality of interest images.

It is the technical problem underlying the invention to provide a medical image registration method and apparatus which avoid or alleviate shortcomings of the prior art as well as a related computer-readable recording medium.

The invention solves this problem by providing a medical image registration method having the features of claim <NUM> and a medical image registration apparatus having the features of claim <NUM> as well as a computer-readable recording medium having the features of claim <NUM>. Advantageous embodiments of the invention are mentioned in the dependent claims the wording of which is herewith incorporated into the description by reference to avoid unnecessary text repetitions.

One or more embodiments of the invention include a method and apparatus for registering medical images captured after administering a contrast agent to a patient.

According to the invention, a medical image registration method includes receiving a medical image captured using a contrast agent, determining a phase of the medical image by using a classification model, and registering the medical image to a reference image by using a registration model, wherein the classification model includes an artificial intelligence model that outputs the phase of the medical image, and the registration model includes an artificial intelligence model for registering the medical image to the reference image.

According to the invention, a medical image registration apparatus includes an image input unit configured to receive a medical image captured using a contrast agent, a phase determination unit configured to determine a phase of the medical image by using a classification model, and a registration unit configured to register the medical image to a reference image by using a registration model, wherein the classification model includes an artificial intelligence model that outputs the phase of the medical image, and the registration model includes an artificial intelligence model for registering the medical image to the reference image.

The above and other aspects, features, and advantages of the invention will be more apparent from the following description of certain embodiments taken in conjunction with the accompanying drawings, in which:.

Hereinafter, a medical image registration method and apparatus according to an embodiment will be described in detail with reference to the attached drawings.

<FIG> is a diagram illustrating examples of a phase of a medical image captured by administering a contrast agent to a patient.

Referring to <FIG>, a medical image may be captured by administering a contrast agent to a patient so that a specific tissue, blood vessel, or lesion is more visible. The medical image may be a computed tomography (CT) or magnetic resonance imaging (MRI) image, and hereinafter, for convenience of description, description will be based on a CT image.

The degree of contrast enhancement varies as time passes after a contrast agent is administered to a patient, and phases of medical images are divided according to the degree of contrast enhancement. For example, a medical image captured by administering a contrast agent may be classified into an arterial phase <NUM>, a portal phase <NUM>, and a delayed phase <NUM>. A medical image captured before administration of a contrast agent may be classified as an unenhanced phase <NUM>.

The present embodiment is only an example for explaining that the phases (<NUM>, <NUM>, <NUM>, <NUM>) before and after the administration of a contrast agent may be divided into a plurality of sections according to the degree of contrast enhancement, and the number and type of phases may be set variously according to embodiments. For example, the portal phase <NUM> may be further subdivided into a plurality of sections, such as an early portal phase, a middle portal phase, and a late portal phase. However, hereinafter, for convenience of description, the phase of a medical image will be described with reference to <FIG>.

<FIG> illustrates an example of a medical image registration apparatus according to an embodiment.

Referring to <FIG>, upon receiving a medical image <NUM> and a reference image <NUM>, a medical image registration apparatus <NUM> registers the medical image <NUM> to the reference image <NUM> (<NUM>). The medical image <NUM> is an image to which movement, rotation, or resizing is applied for registration, and may be an image captured before administration of a contrast agent or an image captured after administration of a contrast agent. The reference image <NUM> is a fixed image and may be an image captured before administration of a contrast agent or an image captured after administration of a contrast agent.

When images of various phases are captured, the medical image <NUM> and the reference image <NUM> may be selected in various manners. For example, in the example of <FIG>, the medical image registration apparatus <NUM> may select an image of the portal phase as the reference image <NUM> and receive an image of the arterial phase as the medical image <NUM>.

As described with reference to <FIG>, in a medical image for each phase, the shapes of various tissues in an image appear differently according to the degree of contrast enhancement. Thus, if an image registration algorithm according to the related art is used as it is, registration accuracy may be low. For example, when a registration algorithm used to register two images of the arterial phase and the portal phase, respectively, is used to register the two images of the portal phase and the delayed phase, the registration accuracy may decrease. Accordingly, according to the present embodiment, a registration method using a plurality of registration models that perform registration between images of different phases is suggested. A registration model according to the present embodiment will be described again with reference to <FIG>.

When registration between a plurality of different phases is performed using the registration model according to the present embodiment, it may be important to distinguish exactly of what phase the medical image <NUM> and the reference image <NUM> are. The degree of contrast enhancement changes over time after administration of a contrast agent, which may differ from patient to patient. For example, a first patient may reach the portal phase after passage of time t1 after administration of a contrast agent, and a second patient may reach the portal phase after passage of time t2. Thus, only accurate identification of the phase of a medical image allows use of an accurate registration model suitable for the phase. Accordingly, in the present embodiment, a classification model for determining a phase of a medical image is included, which will be described with reference to <FIG>.

<FIG> illustrates an example of medical image according to an embodiment.

Referring to <FIG>, a medical image <NUM> may include a plurality of cross-sectional images <NUM>, <NUM>, and <NUM>. For example, when taking an abdominal CT scan of a patient, the medical image <NUM> may include the plurality of cross-sectional images <NUM>, <NUM>, and <NUM> captured at regular intervals. For example, a medical image of the arterial phase may be composed a plurality of cross-sectional images obtained by photographing the abdomen in the arterial phase at regular intervals after administration of a contrast agent.

When the medical image <NUM> includes the plurality of cross-sectional images <NUM>, <NUM>, and <NUM>, registration of images of different phases may be a process of registration all of the plurality of cross-sectional images. For example, when a medical image is composed of N cross-sectional images of the arterial phase (where N is a natural number of <NUM> or more), and a reference image is composed of M cross-sectional images of the delayed phase (M may be the same as or different from N), the medical image registration apparatus may register all N cross-sectional images of the arterial phase to M cross-sectional images of the delayed phase.

According to the present embodiment, the medical image <NUM> including the plurality of cross-sectional images <NUM>, <NUM>, and <NUM> is suggested, but this is only an example, and the medical image and the reference image to be registered may also be each composed of a single cross-sectional image. However, hereinafter, for convenience of description, a case in which both the medical image and the reference image are composed of a plurality of cross-sectional images will be described.

<FIG> illustrates an example of classification model according to an embodiment.

Referring to <FIG>, a classification model <NUM> is an artificial intelligence model that predicts and outputs a phase <NUM> upon receiving a medical image <NUM> (or a reference image). The classification model <NUM> may be implemented using various artificial neural networks such as a Convolutional Neural Network (CNN). For example, the classification model <NUM> may be implemented by a Visual Geometry Group (VGG).

The classification model <NUM> may be generated by training by using learning data obtained by labeling medical images of various phases, by phase. For example, the classification model <NUM> may be trained by a supervised method by using various medical images for the four phases of the unenhanced phase <NUM>, the arterial phase <NUM>, the portal phase <NUM>, and the delayed phase <NUM>, as learning data. Since the method of training and generating an artificial intelligence model using learning data is already a well-known method, additional description thereof will be omitted.

<FIG> illustrates an example of registration model according to an embodiment.

Referring to <FIG>, a registration model <NUM> is an artificial intelligence model that outputs a registered medical image <NUM> when images of different phases are input. The registration model <NUM> may be implemented by various artificial neural networks such as CNN. The registration model <NUM> may be generated by training by using learning data including images of two types of phases (e.g., a medical image <NUM> and a reference image <NUM>) and a registered medical image <NUM>. For example, the registration model <NUM> that registers two medical images of a portal phase and a delayed phase may be generated using learning data including the two medical images of the portal phase and the delayed phase and a registration medical image of the two medical images of the portal phase and the delayed phase.

Upon receiving the medical image <NUM> and the reference image <NUM>, the registration model <NUM> that has been trained moves, rotates, or resizes the medical image <NUM> such that the medical image <NUM> matches the reference image <NUM> and outputs a result. For example, if the phase of the medical image <NUM> is the portal phase and the phase of the reference image <NUM> is the delayed phase, the registration model <NUM> adjusts a position or size of the medical image <NUM> of the portal phase to match the reference image <NUM> and outputs a result. It is difficult for one registration model <NUM> to perform all registration between various phases. Accordingly, in the present embodiment, a plurality of registration models for registration between each phase is included, and an example thereof is shown in <FIG>.

<FIG> illustrates an example of a plurality of registration models according to an embodiment.

Referring to <FIG>, there are a plurality of registration models <NUM> for registration of images of different phases. For example, a first registration model may be a model that performs registration between images of a first phase and a second phase, and a second registration model may be a model that performs registration between images of the second phase and a third phase, and an Nth registration model may be a model that performs registration between images of the Nth phase and the first phase. The number of registration models may vary according to the number of phases. The plurality of registration models <NUM> may be generated by the method described with reference to <FIG>.

<FIG> is a flowchart of an example of a medical image registration method according to an embodiment.

Referring to <FIG>, the medical image registration apparatus <NUM> receives a medical image in operation S700. The medical image may include a medical image captured using a contrast agent.

The medical image registration apparatus <NUM> determines a phase of the medical image in operation S710. For example, the medical image registration apparatus <NUM> may determine the phase of the medical image by using the classification model <NUM> of <FIG>. As another embodiment, the medical image registration apparatus <NUM> may additionally determine a phase of a reference image. For example, when the exact phase of the reference image is not known, the medical image registration apparatus <NUM> may determine the phase of the reference image by using the classification model <NUM>. If phase information of the reference image is known in advance, the determining of the phase of the reference image may be omitted.

The medical image registration apparatus <NUM> registers a medical image to a previously designated reference image by using a registration model in operation S720. When registration between a plurality of phases is required, there is the plurality of registration models <NUM> described with reference to <FIG>. In this case, the medical image registration apparatus <NUM> selects a registration model to be used for registration, by using the phase of the medical image and the phase of the reference image. For example, in the example of <FIG>, if the phase of the medical image is the second phase and the phase of the reference image is the third phase, the medical image registration apparatus performs registration between the medical image and the reference image by using the second registration model.

<FIG> illustrates a configuration of an example of a medical image registration apparatus according to an embodiment.

Referring to <FIG>, the medical image registration apparatus <NUM> includes an image input unit <NUM>, a phase determination unit <NUM>, a model selection unit <NUM>, a registration unit <NUM>, a classification model <NUM>, and at least one registration model <NUM>. The medical image registration apparatus <NUM> may be implemented using a computing device including a memory, a processor, an input/output device, and the like. In this case, each component may be implemented as software and loaded in a memory, and then performed by a processor.

The image input unit <NUM> receives a medical image and a reference image. The medical image and the reference image may be three-dimensional medical images captured using a contrast agent.

The phase determination unit <NUM> determines a phase of the medical image by using a classification model. When a phase of the reference image is not known, the phase determination unit <NUM> may additionally determine the phase of the reference image by using a classification model.

The model selection unit <NUM> selects one registration model to be used for registration from among a plurality of registration models <NUM> based on the phase of the medical image and the phase of the reference image.

The registration unit <NUM> registers the medical image to the reference image by using the registration model.

<FIG> is a diagram illustrating an example of a result of image registration by using a medical image registration method according to an embodiment.

Referring to <FIG>, an image <NUM> is shown in which a medical image <NUM> of the delayed phase is modified to be registered to a reference image <NUM> of the portal phase by the medical image registration method according to the present embodiment. In the present embodiment, one image of each phase is shown for convenience of description, but an image of each phase may also be composed of a plurality of cross-sectional images as described with reference to <FIG>.

As evident from square boxes <NUM>, <NUM>, and <NUM> of the images <NUM>, <NUM>, and <NUM>, the medical image <NUM> after registration is accurately compared with the reference image <NUM> compared to the medical image <NUM> in the delay phase. For example, when using phase-specific medical images for liver cancer diagnosis, medical staff may more accurately compare the size and location of liver cancer, thereby increasing the accuracy of diagnosis.

The disclosure may also be implemented as computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include read only memory (ROM), random access memory (RAM), a compact disk ROM (CD-ROM), a solid state disk (SSD), and optical data storage devices. In addition, the computer-readable recording medium is distributed in a network-connected computer system so that the computer-readable code can be stored and executed in a distributed manner.

While the disclosure has been particularly shown and described with reference to embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as defined by the appended claims. The disclosed embodiments should be considered in a descriptive sense only and not for purposes of limitation. Therefore, the scope of the disclosure is defined not by the detailed description of the disclosure but by the appended claims, and all differences within the scope will be construed as being included in the disclosure.

According to an embodiment, the accuracy of medical image registration may be increased by using a registration model suitable for a phase of a medical image. In addition, the accuracy of diagnosis may be improved through registration of medical images of various phases. For example, the accuracy of liver cancer diagnosis may be increased by registering the size and location of liver lesions appearing in medical images of various phases.

Claim 1:
Medical image registration method comprising:
- receiving a medical image captured using a contrast agent;
- determining a phase of the medical image by using a classification model which comprises an artificial intelligence model that outputs the phase of the medical image; and
- registering the medical image to a reference image by using a registration model which comprise an artificial intelligence model for registering the medical image to the reference image,
- wherein a plurality of registration models is provided for registering images of different phases and the registering comprises selecting one of the registration models to be used for registration, based on the phase of the medical image and a phase of the reference image.