SYSTEMS AND METHODS FOR IMAGE PROCESSING

The present disclosure may provide a system. The system may obtain a first image of a first region of a subject and a second image of a second region of the subject. The first region and the second region may have an overlapping region. The system may generate a first corrected image by correcting the first image based on the second image and a second corrected image by correcting the second image based on the first image. The system may generate a combined image by combining the first corrected image and the second corrected image.

TECHNICAL FIELD

The present disclosure generally relates to image processing, and in particular, to systems and methods for generating a combined image by combining different images.

BACKGROUND

With the development of image stitching technology, a plurality of images of different regions of a subject with overlapping regions may be acquired and combined to form a global image of the subject. However, since deformation information of the plurality of images may be different due to different acquisition parameters (e.g., magnetic inhomogeneity), there may be content discontinuities in the combined global image. Thus, it is desirable to provide systems and methods for accurately generating a combined image by combining different images, thereby improving a content continuity of the combined image.

SUMMARY

According to one aspect of the present disclosure, a system may be provided. The system may include at least one storage device including a set of instructions and at least one processor in communication with the at least one storage device. When executing the set of instructions, the at least one processor may be configured to cause the system to: obtain a first image of a first region of a subject; obtain a second image of a second region of the subject, the first region and the second region having an overlapping region; generate a first corrected image by correcting the first image based on the second image; generate a second corrected image by correcting the second image based on the first image; and generate a combined image by combining the first corrected image and the second corrected image.

In some embodiments, wherein to generate the first corrected image by correcting the first image based on the second image, the at least one processor may be directed to: generate a first rigid registration image corresponding to the first image and a second rigid registration image corresponding to the second image by performing a rigid registration on the first image and the second image; and generate the first corrected image by correcting the first rigid registration image based on the second rigid registration image; and to generate the second corrected image by correcting the second image based on the first image, the at least one processor is directed to: generate the second corrected image by correcting the second rigid registration image based on the first rigid registration image.

In some embodiments, wherein to generate the first rigid registration image corresponding to the first image and the second rigid registration image corresponding to the second image by performing the rigid registration on the first image and the second image, the at least one processor may be directed to: determine a first image section of the first image corresponding to the overlapping region; determine a second image section of the second image corresponding to the overlapping region; and generate the first rigid registration image corresponding to the first image and the second rigid registration image corresponding to the second image by performing the rigid registration on the first image section and the second image section.

In some embodiments, wherein to generate the first corrected image by correcting the first rigid registration image based on the second rigid registration image, the at least one processor may be directed to: generate a first non-rigid registration result by performing a non-rigid registration on the first rigid registration image based on the second rigid registration image; and generate the first corrected image based on the first non-rigid registration result; and to generate the second corrected image by correcting the second rigid registration image based on the first rigid registration image, the at least one processor may be directed to: generate a second non-rigid registration result by performing a non-rigid registration on the second rigid registration image based on the first rigid registration image; and generate the second corrected image based on the second non-rigid registration result.

In some embodiments, the non-rigid registration may be performed based on a transformation function associated with a similarity between the first rigid registration image and the second rigid registration image.

In some embodiments, wherein to generate the first corrected image based on the first non-rigid registration result, the at least one processor may be directed to: determine a first correction weight function corresponding to a first image section of the first rigid registration image corresponding to the overlapping region; and generate the first corrected image based on the first non-rigid registration result and the first correction weight function; and to generate the second corrected image based on the second non-rigid registration result, the at least one processor may be directed to: determine, based on the first correction weight function, a second correction weight function corresponding to a second image section of the second rigid registration image corresponding to the overlapping region; and generate the second corrected image based on the second non-rigid registration result and the second correction weight function.

In some embodiments, the first correction weight function may be associated with a height of the first image section and distances of pixels in the first image section to an edge of the first image section. A sum of the first correction weight function and the second correction weight function is 1.

In some embodiments, the farther a pixel in the first image section from an upper edge of the first image section is, the smaller a first correction weight function of the pixel may be.

In some embodiments, wherein to generate the combined image by combining the first corrected image and the second corrected image, the at least one processor may be directed to: determine a first fusion weight function corresponding to a first image section of the first corrected image corresponding to the overlapping region; determine, based on the first fusion weight function, a second fusion weight function corresponding to a second image section of the second corrected image corresponding to the overlapping region; determine a fused image corresponding to the overlapping region based on the first image section, the first fusion weight function, the second image section, and the second fusion weight function; and generate the combined image by combining remainder sections of the first corrected image, remainder sections of the second corrected image, and the fused image.

In some embodiments, the first fusion weight function may be associated with distances of pixels in the first image section to an edge of the first image section. A sum of the first fusion weight function and the second fusion weight function may be 1.

According to another aspect of the present disclosure, a method may be provided. The method may be implemented on a computing device including at least one processor, at least one storage medium, and a communication platform connected to a network. The method may include: obtaining a first image of a first region of a subject; obtaining a second image of a second region of the subject, the first region and the second region having an overlapping region; generating a first corrected image by correcting the first image based on the second image; generating a second corrected image by correcting the second image based on the first image; and generating a combined image by combining the first corrected image and the second corrected image.

In some embodiments, the generating the first corrected image by correcting the first image based on the second image may include: generating a first rigid registration image corresponding to the first image and a second rigid registration image corresponding to the second image by performing a rigid registration on the first image and the second image; and generating the first corrected image by correcting the first rigid registration image based on the second rigid registration image; and the generating the second corrected image by correcting the second image based on the first image may include: generating the second corrected image by correcting the second rigid registration image based on the first rigid registration image.

In some embodiments, the generating the first rigid registration image corresponding to the first image and the second rigid registration image corresponding to the second image by performing the rigid registration on the first image and the second image may include: determining a first image section of the first image corresponding to the overlapping region; determining a second image section of the second image corresponding to the overlapping region; and generating the first rigid registration image corresponding to the first image and the second rigid registration image corresponding to the second image by performing the rigid registration on the first image section and the second image section.

In some embodiments, the generating the first corrected image by correcting the first rigid registration image based on the second rigid registration image may include: generating a first non-rigid registration result by performing a non-rigid registration on the first rigid registration image based on the second rigid registration image; and generating the first corrected image based on the first non-rigid registration result; and the generating the second corrected image by correcting the second rigid registration image based on the first rigid registration image may include: generating a second non-rigid registration result by performing a non-rigid registration on the second rigid registration image based on the first rigid registration image; and generating the second corrected image based on the second non-rigid registration result.

In some embodiments, the non-rigid registration may be performed based on a transformation function associated with a similarity between the first rigid registration image and the second rigid registration image.

In some embodiments, the generating the first corrected image based on the first non-rigid registration result may include: determining a first correction weight function corresponding to a first image section of the first rigid registration image corresponding to the overlapping region; and generating the first corrected image based on the first non-rigid registration result and the first correction weight function; and the generating the second corrected image based on the second non-rigid registration result may include: determining, based on the first correction weight function, a second correction weight function corresponding to a second image section of the second rigid registration image corresponding to the overlapping region; and generating the second corrected image based on the second non-rigid registration result and the second correction weight function.

In some embodiments, the first correction weight function may be associated with a height of the first image section and distances of pixels in the first image section to an edge of the first image section. A sum of the first weight function and the second weight function may be 1.

In some embodiments, the farther a pixel in the first image section from an upper edge of the first image section is, the smaller a first correction weight function of the pixel may be.

In some embodiments, the generating the combined image by combining the first corrected image and the second corrected image may include: determining a first fusion weight function corresponding to a first image section of the first corrected image corresponding to the overlapping region; determining, based on the first fusion weight function, a second fusion weight function corresponding to a second image section of the second corrected image corresponding to the overlapping region; determining a fused image corresponding to the overlapping region based on the first image section, the first fusion weight function, the second image section, and the second fusion weight function; and generating the combined image by combining remainder sections of the first corrected image, remainder sections of the second corrected image, and the fused image.

In some embodiments, the first fusion weight function may be associated with distances of pixels in the first image section to an edge of the first image section. A sum of the first fusion weight function and the second fusion weight function may be 1.

According to another aspect of the present disclosure, a non-transitory computer readable medium may be provided. The non-transitory computer readable medium may include executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method. The method may include: obtaining a first image of a first region of a subject; obtaining a second image of a second region of the subject, the first region and the second region having an overlapping region; generating a first corrected image by correcting the first image based on the second image; generating a second corrected image by correcting the second image based on the first image; and generating a combined image by combining the first corrected image and the second corrected image.

DETAILED DESCRIPTION

The modules (or units, blocks, units) described in the present disclosure may be implemented as software and/or hardware modules and may be stored in any type of non-transitory computer-readable medium or other storage devices. In some embodiments, a software module may be compiled and linked into an executable program. It will be appreciated that software modules can be callable from other modules or from themselves, and/or can be invoked in response to detected events or interrupts. Software modules configured for execution on computing devices can be provided on a computer-readable medium, such as a compact disc, a digital video disc, a flash drive, a magnetic disc, or any other tangible medium, or as a digital download (and can be originally stored in a compressed or installable format that requires installation, decompression, or decryption prior to execution). Such software code can be stored, partially or fully, on a memory device of the executing computing device, for execution by the computing device. Software instructions can be embedded in a firmware, such as an EPROM. It will be further appreciated that hardware modules (e.g., circuits) can be included of connected or coupled logic units, such as gates and flip-flops, and/or can be included of programmable units, such as programmable gate arrays or processors. The modules or computing device functionality described herein are preferably implemented as hardware modules, but can be software modules as well. In general, the modules described herein refer to logical modules that can be combined with other modules or divided into units despite their physical organization or storage.

In the present disclosure, a subject may include a biological object and/or a non-biological object. The biological subject may include a human being, an animal, a plant, or a specific portion, organ, and/or tissue thereof, such as the head, the neck, the thorax, the heart, the stomach, a blood vessel, soft tissue, a tumor, a nodule, or the like, or any combination thereof. The term “object” or “subject” are used interchangeably in the present disclosure.

As used herein, a representation of a subject (e.g., a patient, an object, or a portion thereof) in an image may be referred to the subject for brevity. For instance, a representation of an organ or tissue (e.g., the heart, the liver, a lung, etc., of a patient) in an image may be referred to as the organ or tissue for brevity. An image including a representation of a subject may be referred to as an image of the subject or an image including the subject for brevity.

An aspect of the present disclosure relates to systems and methods for generating a combined image by combining different images. The system may obtain a first image of a first region of a subject and a second image of a second region of the subject. The first region and the second region may have an overlapping region. The system may generate a first rigid registration image corresponding to the first image and a second rigid registration image corresponding to the second image by performing a rigid registration on the first image and the second image. The system may generate a first corrected image by correcting (e.g., by performing a non-rigid registration) the first rigid registration image based on the second rigid registration image and a second corrected image by correcting (e.g., by performing a non-rigid registration) the second rigid registration image based on the first rigid registration image. Further, the system may generate a combined image by combining the first corrected image and the second corrected image.

According to systems and methods of the present disclosure, firstly, the rigid registration may be performed on the first image and the second image to preliminarily register the first image and the second image; secondly, the non-rigid registration may be further performed on the first rigid registration image or the second rigid registration image, with the other image as a reference, accordingly, local information of a first image section of the first image corresponding to the overlapping region coincides with local information of a second image section of the second image corresponding to the overlapping region. Accordingly, a content continuity and the quality of the combined image can be improved. In some embodiments, the rigid registration and/or the non-rigid registration may be performed on the first image section of the first image corresponding to the overlapping region and the second image section of the second image corresponding to the overlapping region, thereby saving computing time and/or computing resource for generating the combined image.

FIG.1is a schematic diagram illustrating an exemplary medical system according to some embodiments of the present disclosure. As illustrated, the medical system100may include a medical device110, a network120, a terminal device130, a processing device140, and a storage device150. The components of the medical system100may be connected in one or more of various ways. Merely by way of example, as illustrated inFIG.1, the medical device110may be connected to the processing device140through the network120. As another example, the medical device110may be connected to the processing device140directly as indicated by the bi-directional arrow in dotted lines linking the medical device110and the processing device140. As a further example, the storage device150may be connected to the processing device140directly or through the network120. As still a further example, the terminal device130may be connected to the processing device140directly (as indicated by the bi-directional arrow in dotted lines linking the terminal device130and the processing device140) or through the network120.

The medical device110may scan a subject located within its detection region and generate imaging data relating to the subject or a portion thereof. In some embodiments, the imaging data may be two-dimensional (2D) imaging data, three-dimensional (3D) imaging data, four-dimensional (4D) imaging data, or the like, or any combination thereof. In some embodiments, the scanning device110may include a single modality imaging device. In some embodiments, the medical device110may include a magnetic resonance (MR) (e.g., a diffusion weighted MR, a dynamic contrast-enhanced MR (DCE-MR)), an X-ray system, a computed tomography (CT) (e.g., a cone beam CT (CBCT), a helical CT, a multi-slice CT), a positron emission tomography (PET) (e.g., a single-photon emission computed tomography (SPECT), a fluoroscopic device, an ultrasound device, a radiotherapy portal device, a PET-CT, or the like, or any combination thereof.

The network120may include any suitable network that can facilitate the exchange of information and/or data for the medical system100. In some embodiments, one or more components (e.g., the medical device110, the terminal device130, the processing device140, or the storage device150) of the medical system100may communicate information and/or data with one or more other components of the medical system100via the network120. For example, the processing device140may obtain a first image of a first region of a subject and a second image of a second region of the subject acquired by the medical device110via the network120, wherein the first image and the second image may have an overlapping region. In some embodiments, the network120may be any type of wired or wireless network, or a combination thereof. The network120may be and/or include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN), a wide area network (WAN))), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.11 network, a Wi-Fi network), a cellular network (e.g., a long term evolution (LTE) network), a frame relay network, a virtual private network (VPN), a satellite network, a telephone network, routers, hubs, switches, server computers, and/or any combination thereof. Merely by way of example, the network120may include a cable network, a wireline network, a fiber-optic network, a telecommunications network, an intranet, a wireless local area network (WLAN), a metropolitan area network (MAN), a public telephone switched network (PSTN), a Bluetooth™ network, a ZigBee™ network, a near field communication (NFC) network, or the like, or any combination thereof. In some embodiments, the network120may include one or more network access points. For example, the network120may include wired and/or wireless network access points such as base stations and/or internet exchange points through which one or more components of the medical system100may be connected to the network120to exchange data and/or information.

The terminal device130may include a mobile device131, a tablet computer132, a laptop computer133, or the like, or any combination thereof. In some embodiments, the mobile device131may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. The smart home device may include a smart lighting device, a control device of an intelligent electrical apparatus, a smart monitoring device, a smart television, a smart video camera, an interphone, or the like, or any combination thereof. The wearable device may include a smart bracelet, a smart footgear, a pair of smart glasses, a smart helmet, a smart watch, smart clothing, a smart backpack, a smart accessory, or the like, or any combination thereof. The smart mobile device may include a smartphone, a personal digital assistant (PDA), a gaming device, a navigation device, a point of sale (POS) device, or the like, or any combination thereof. The virtual reality device and/or the augmented reality device may include a virtual reality helmet, a virtual reality glass, a virtual reality patch, an augmented reality helmet, an augmented reality glass, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device and/or the augmented reality device may include a Google™ Glass, an Oculus Rift, a Hololens, a Gear VR, etc. In some embodiments, the terminal device130may remotely operate the medical device110and/or the processing device140. In some embodiments, the terminal device130may operate the medical device110and/or the processing device140via a wireless connection. In some embodiments, the terminal device130may receive information and/or instructions inputted by a user, and send the received information and/or instructions to the medical device110or to the processing device140via the network120. In some embodiments, the terminal device130may receive data and/or information from the processing device140. In some embodiments, the terminal device130may be part of the processing device140. In some embodiments, the terminal device130may be omitted.

The processing device140may process data and/or information obtained from the medical device110, the terminal device130, and/or the storage device150. For example, the processing device140may generate a first corrected image by correcting a first image based on a second image. As another example, the processing device140may generate a second corrected image by correcting a second image based on a first image. As a further example, the processing device140may generate a combined image by combining a first corrected image and a second corrected image. In some embodiments, the processing device140may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processing device140may be local or remote. For example, the processing device140may access information and/or data stored in or acquired by the medical device110, the terminal device130, and/or the storage device150via the network120. As another example, the processing device140may be directly connected to the medical device110(as illustrated by the bidirectional arrow in dashed lines connecting the processing device140and the medical device110inFIG.1), the terminal device130(as illustrated by the bidirectional arrow in dashed lines connecting the processing device140and the terminal device130inFIG.1), and/or the storage device150to access stored or acquired information and/or data. In some embodiments, the processing device140may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof. In some embodiments, the processing device140or a portion of the processing device140may be integrated into the medical device110. In some embodiments, the processing device140may be implemented by a computing device200including one or more components as described inFIG.2.

The storage device150may store data and/or instructions. In some embodiments, the storage device150may store data obtained from the medical device110, the terminal device130, and/or the processing device140. For example, the processing device140may generate a combined image by combining a first corrected image and a second corrected image, and then the combined image may be stored in the storage device150for further use or processing. In some embodiments, the storage device150may store data and/or instructions that the processing device140may execute or use to perform exemplary methods described in the present disclosure. For example, the storage device150may store instructions that the processing device140may execute or use to generate a first corrected image and a second corrected image based on a first image and a second image of a subject acquired by the medical device110. As another example, the storage device150may store instructions that the processing device140may execute or use to generate a combined image by combining a first corrected image and a second corrected image. In some embodiments, the storage device150may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM), or the like, or any combination thereof. Exemplary mass storage may include a magnetic disk, an optical disk, a solid-state drive, etc. Exemplary removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. Exemplary volatile read-and-write memory may include a random access memory (RAM). Exemplary RAM may include a dynamic RAM (DRAM), a double date rate synchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), a thyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM), etc. Exemplary ROM may include a mask ROM (MROM), a programmable ROM (PROM), an erasable programmable ROM (PEROM), an electrically erasable programmable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digital versatile disk ROM, etc. In some embodiments, the storage device150may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof.

In some embodiments, the storage device150may be connected to the network120to communicate with one or more components (e.g., the medical device110, the terminal device130, the processing device140) of the medical system100. One or more components of the medical system100may access the data or instructions stored in the storage device150via the network120. In some embodiments, the storage device150may be directly connected to or communicate with one or more components (e.g., the medical device110, the processing device140, the terminal device130) of the medical system100. In some embodiments, the storage device150may be part of the processing device140.

In some embodiments, the medical system100may further include one or more power supplies (not shown inFIG.1) connected to one or more components (e.g., the medical device110, the terminal device130, the processing device140, the storage device150) of the medical system100.

This description is intended to be illustrative, and not to limit the scope of the present disclosure. Many alternatives, modifications, and variations will be apparent to those skilled in the art. The features, structures, methods, and other characteristics of the exemplary embodiments described herein may be combined in various ways to obtain additional and/or alternative exemplary embodiments. However, those variations and modifications do not depart the scope of the present disclosure.

FIG.2is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure. In some embodiments, the processing device140may be implemented on the computing device200. As illustrated inFIG.2, the computing device200may include a processor210, a storage220, an input/output (I/O)230, and a communication port240.

The processor210may execute computer instructions (program code) and perform functions of the processing device140in accordance with techniques described herein. The computer instructions may include routines, programs, objects, components, signals, data structures, procedures, modules, and functions, which perform particular functions described herein. In some embodiments, the processor210may include a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application specific integrated circuits (ASICs), an application-specific instruction-set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of executing one or more functions, or the like, or any combinations thereof.

Merely for illustration purposes, only one processor is described in the computing device200. However, it should be noted that the computing device200in the present disclosure may also include multiple processors, and thus operations of a method that are performed by one processor as described in the present disclosure may also be jointly or separately performed by the multiple processors. For example, if in the present disclosure the processor of the computing device200executes both operations A and B, it should be understood that operations A and B may also be performed by two different processors jointly or separately in the computing device200(e.g., a first processor executes operation A and a second processor executes operation B, or the first and second processors jointly execute operations A and B).

The storage220may store data/information obtained from the medical device110, the terminal device130, the storage device150, or any other component of the medical system100. In some embodiments, the storage220may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM), or the like, or any combination thereof. For example, the mass storage device may include a magnetic disk, an optical disk, a solid-state drive, etc. The removable storage device may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. The volatile read-and-write memory may include a random access memory (RAM). The RAM may include a dynamic RAM (DRAM), a double date rate synchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), a thyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM), etc. The ROM may include a mask ROM (MROM), a programmable ROM (PROM), an erasable programmable ROM (PEROM), an electrically erasable programmable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digital versatile disk ROM, etc. In some embodiments, the storage220may store one or more programs and/or instructions to perform exemplary methods described in the present disclosure. For example, the storage220may store a program for the processing device140for generating a first corrected image and a second corrected image based on a first image and a second image.

The I/O230may input or output signals, data, or information. In some embodiments, the I/O230may enable user interaction with the processing device140. In some embodiments, the I/O230may include an input device and an output device. Exemplary input devices may include a keyboard, a mouse, a touch screen, a microphone, a trackball, or the like, or a combination thereof. Exemplary output devices may include a display device, a loudspeaker, a printer, a projector, or the like, or a combination thereof. Exemplary display devices may include a liquid crystal display (LCD), a light-emitting diode (LED)-based display, a flat panel display, a curved screen, a television device, a cathode ray tube (CRT), or the like, or a combination thereof.

Merely by way of example, a user (e.g., an operator) of the processing device140may input data related to a subject (e.g., a patient) that is being/to be imaged/scanned through the I/O230. The data related to the subject may include identification information (e.g., the name, age, gender, medical history, contact information, physical examination result) and/or test information including the nature of the scan that must be performed. The user may also input parameters needed for the operation of the medical device110, such as image contrast and/or ratio, a region of interest (ROI), or the like, or any combination thereof. The I/O230may also display an image (or videos) generated based on the imaging/scan data.

The communication port240may be connected to a network (e.g., the network120) to facilitate data communications. The communication port240may establish connections between the processing device140and the medical device110, the terminal device130, or the storage device150. The connection may be a wired connection, a wireless connection, or a combination of both that enables data transmission and reception. The wired connection may include an electrical cable, an optical cable, a telephone wire, or the like, or any combination thereof. The wireless connection may include Bluetooth, Wi-Fi, WiMax, WLAN, ZigBee, mobile network (e.g., 3G, 4G, 5G), or the like, or a combination thereof. In some embodiments, the communication port240may be a standardized communication port, such as RS232, RS485, etc. In some embodiments, the communication port240may be a specially designed communication port. For example, the communication port240may be designed in accordance with the digital imaging and communications in medicine (DICOM) protocol.

FIG.4is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure. The processing device140may include an image obtainment module410, an image correction module420, and an image combination module430.

The image obtainment module410may be configured to obtain a first image of a first region of a subject and a second image of a second region of the subject. In some embodiments, the first region and the second region may have an overlapping region. For example, the first region may include the head and the neck of the subject; the second region may include the neck and the chest of the subject. In some embodiments, the overlapping region may correspond to different places of the first image and/or the second image. For example, the overlapping region may correspond to a lower part of the first image and correspond to an upper part of the second image. As another example, the overlapping region may correspond to a right side of the first image and correspond to a left side of the second image.

The image correction module420may be configured to generate a first corrected image by correcting the first image based on the second image and a second corrected image by correcting the second image based on the first image. In some embodiments, the image correction module420may generate a first rigid registration image corresponding to the first image and a second rigid registration image corresponding to the second image by performing a rigid registration on the first image and the second image. Further, the image correction module420may generate the first corrected image by correcting the first rigid registration image based on the second rigid registration image; and generate the second corrected image by correcting the second rigid registration image based on the first rigid registration image. For example, the image correction module420may generate a first non-rigid registration result by performing a registration (e.g., a non-rigid registration) on the first rigid registration image using the second rigid registration image as a reference. Further, the image correction module420may generate the first corrected image based on the first non-rigid registration result. Similarly, the image correction module420may generate a second non-rigid registration result by performing a registration (e.g., a non-rigid registration) on the second rigid registration image using the first rigid registration image as a reference. Further, the image correction module420may generate the second corrected image based on the second non-rigid registration result.

In some embodiments, the image correction module420may directly perform the non-rigid registration on the first image (or the first image section) based on the second image (or the second image section) and perform the non-rigid rigid registration on the second image (or the second image section) based on the first image (or the first image section) without performing the rigid registration on the first image and the second image. More descriptions regarding the first corrected image and the second corrected image may be found elsewhere in the present disclosure (e.g.,FIG.5and the descriptions thereof).

The image combination module430may be configured to generate a combined image by combining the first corrected image and the second corrected image. In some embodiments, the image combination module430may generate the combined image using an image combination algorithm. In some embodiments, the image combination module430may determine a first fusion weight function corresponding to a first image section (i.e., the first corrected image section) of the first corrected image corresponding to the overlapping region and a second fusion weight function corresponding to a second image section (i.e., the second corrected image section) of the second corrected image corresponding to the overlapping region. In some embodiments, for a first pixel in the first image section and a second pixel in the second image section (the first pixel and the second pixel correspond to a same position in the overlapping region), a sum of the first fusion weight function (e.g., a value thereof) and a second fusion weight function (e.g., a value thereof) may be a constant value (e.g., 1). In some embodiments, the first fusion weight function may be associated with distances of pixels in the first image section of the first corrected image to an edge (e.g., an upper edge) of the first image section (as illustrated inFIG.6AorFIG.6B, we can assume that the first image section is located above the second image section).

In some embodiments, the image combination module430may determine a fused image corresponding to the overlapping region based on the first image section, the first fuse weight function, the second image section, and the second fuse weight function. Further, the image combination module430may generate the combined image by combining remainder sections of the first corrected image, remainder sections of the second corrected image, and the fused image. More descriptions regarding the combined image may be found elsewhere in the present disclosure (e.g.,FIG.5and the descriptions thereof).

The modules in the processing device140may be connected to or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, or the like, or any combination thereof. The wireless connection may include a Local Area Network (LAN), a Wide Area Network (WAN), a Bluetooth, a ZigBee, a Near Field Communication (NFC), or the like, or any combination thereof.

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. For example, the processing device140may include a storage module (not shown) configured to store data generated by the above-mentioned modules. As another example, two or more of the above-mentioned modules may be combined into a single module or any one of the above-mentioned modules may be divided into two or more units.

FIG.5is a flowchart illustrating an exemplary process for generating a combined image according to some embodiments of the present disclosure. In some embodiments, the process500may be implemented as a set of instructions stored in the storage220. The processor210and/or the modules inFIG.4may execute the set of instructions, and when executing the instructions, the processor210and/or the modules may be configured to perform the process500. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process500may be accomplished with one or more additional operations not described and/or without one or more of the operations herein discussed. Additionally, the order in which the operations of the process as illustrated inFIG.5and described below is not intended to be limiting.

In510, the processing device140(e.g., the image obtainment module410) (e.g., the interface circuits of the processor320) may obtain a first image of a first region of a subject.

In520, the processing device140(e.g., the image obtainment module410) (e.g., the interface circuits of the processor320) may obtain a second image of a second region of the subject.

In some embodiments, the first region or the second region of the subject may include an abdomen, a chest, a leg, a head, a neck, a lung, a spine, a scapula, a heart, or a portion thereof, or the like, or any combination thereof. In some embodiments, the first region and the second region may have an overlapping region. For example, the first region may include the head and the neck of the subject; the second region may include the neck and the chest of the subject. In some embodiments, the overlapping region may correspond to different places of the first image and/or the second image. For example, the overlapping region may correspond to a lower part of the first image and correspond to an upper part of the second image. As another example, the overlapping region may correspond to a right side of the first image and correspond to a left side of the second image.

In some embodiments, the first image and the second image may be acquired by a same medical device (e.g., the medical device110). For example, the first image and the second image may be two adjacent images of the subject sequentially acquired by the same medical device. In some embodiments, the processing device140may obtain the first image and the second image directly from the medical device110or a storage device (e.g., the storage device150). In some embodiments, the processing device140may obtain the first image and the second image from an external source connected to the medical system100.

In some embodiments, the first image or the second image may include a two-dimensional (2D) image, a three-dimensional (3D) image, a four-dimensional (4D) image, etc. In some embodiments, the first image or the second image may include an MR image (e.g., a diffusion MR image (e.g., a diffusion-weighted MR image), a DCE-MR image, a streaming MR image, a volumetric MR image, a cine MR image), an X-ray image, a CT image (e.g., a CBCT image), a PET image (e.g., a SPECT image), a fluoroscopic image, an ultrasound image, a radiotherapy portal image, or the like, or any combination thereof. Merely by way of example, the first image and the second image may be two diffusion-weighted MR images.

In530, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may generate a first corrected image by correcting the first image based on the second image.

In540, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may generate a second corrected image by correcting the second image based on the first image.

In some embodiments, the processing device140may generate a first rigid registration image corresponding to the first image and a second rigid registration image corresponding to the second image by performing a rigid registration on the first image and the second image. Further, the processing device140may generate the first corrected image by correcting the first rigid registration image based on the second rigid registration image; and generate the second corrected image by correcting the second rigid registration image based on the first rigid registration image. For example, the processing device140may generate a first non-rigid registration result by performing a registration (e.g., a non-rigid registration) on the first rigid registration image using the second rigid registration image as a reference. Further, the processing device140may generate the first corrected image based on the first non-rigid registration result. Similarly, the processing device140may generate a second non-rigid registration result by performing a registration (e.g., a non-rigid registration) on the second rigid registration image using the first rigid registration image as a reference. Further, the processing device140may generate the second corrected image based on the second non-rigid registration result. More details will be described below.

In some embodiments, the processing device140may perform the rigid registration using a rigid registration algorithm. For example, the rigid registration algorithm may include a mutual information (MI) algorithm, a sum of squared differences (SSD) algorithm, a mean absolute difference (MAD) algorithm, a sum of absolute differences (SAD) algorithm, a mean square difference (MSD) algorithm, a normalized cross correlation (NCC) algorithm, a sequential similarity detection algorithm (SSDA), a sum of absolute transformed difference (SATD) algorithm, or the like, or any combination thereof.

In some embodiments, the processing device140may perform the rigid registration to align the first image and the second image. In some embodiments, the processing device140may determine at least one transformation parameter (e.g., a rotation angle, a translation distance) associated with the first image and the second image. Then the processing device140may determine an evaluation parameter (e.g., an SSD, an MAD, an SAD, an MSD, an NCC, an SSDA, a SATD, MI)) of the rigid registration based on the at least one transformation parameter. In response to determining that the evaluation parameter satisfies a preset condition, the processing device140may generate the first rigid registration image and the second rigid registration image based on the at least one transformation parameter. In response to determining that the evaluation parameter fails to satisfy the preset condition, the processing device140may update at least one of the at least one transformation parameter until the evaluation parameter satisfies the preset condition. Further, the processing device140may generate the first rigid registration image and the second rigid registration image based on the at least one updated transformation parameter. Merely by way of example, the processing device140may displace or rotate the first image or the second image based on the at least one (updated) transformation parameter, thereby aligning the first image and the second image. In some embodiments, the preset condition may include that the SSD is smaller than a threshold, the MAD is smaller than a threshold, the MI is larger than a threshold, or the like, or any combination thereof.

In some embodiments, the processing device140may determine a first image section of the first image corresponding to the overlapping region and a second image section of the second image corresponding to the overlapping region. Further, the processing device140may generate the first rigid registration image corresponding to the first image and the second rigid registration image corresponding to the second image by performing the rigid registration on the first image section and the second image section. For example, the processing device140may perform the rigid registration on the first image section and the second image section, determine spatial transformation parameters of the rigid registration, and generate the first rigid registration image corresponding to the first image and the second rigid registration image corresponding to the second image based on the spatial transformation parameters.

In some embodiments, the processing device140may determine the first image section and the second image section based on location information of the overlapping region in the first image and the second image. In some embodiments, the processing device140may determine the location information of the overlapping region in the first image and the second image based on scanning information (e.g., a scanning region) of the subject. The processing device140may obtain the scanning information of the subject based on an imaging protocol (e.g., a DICOM protocol) of the subject.

In some embodiments, as described above, after generating the first rigid registration image and the second rigid registration image, the processing device140may generate a first non-rigid registration result (which can be considered as a “first deformation field”) by performing a non-rigid registration (e.g., a two-dimensional (2D) registration, a three-dimensional (3D) registration) on the first rigid registration image using the second rigid registration image as a reference and generate the first corrected image based on the first non-rigid registration result. Specifically, the processing device140may determine a first transformation function associated with a similarity between the second rigid registration image and the first rigid registration image. In some embodiments, the processing device140may optimize the first transformation function by adjusting at least one parameter of the first transformation function, for example, to minimize a cost value of the first transformation function. The processing device140may perform the non-rigid registration on the first rigid registration image based on the first (optimized) transformation function to generate the first corrected image.

In some embodiments, similarly, the processing device140may generate a second non-rigid registration result (which can be considered as a “second deformation field”) by performing a non-rigid registration on the second rigid registration image using the first rigid registration image as a reference and generate the second corrected image based on the second non-rigid registration result. The processing device140may determine a second transformation function associated with a similarity between the first rigid registration image and the second rigid registration image. In some embodiments, the processing device140may optimize the second transformation function by adjusting at least one parameter of the second transformation function, for example, to minimize a cost value of the second transformation function. The processing device140may perform the non-rigid registration on the second rigid registration image based on the second (optimized) transformation function to generate the second corrected image.

In some embodiments, the processing device140may perform the non-rigid registration based on a non-rigid registration algorithm. For example, the non-rigid registration algorithm may include a linear elastic algorithm, a viscous fluid flow algorithm, a diffusion Demons algorithm, a differential homeomorphic diffusion algorithm, a curvature registration algorithm, a Fourier basis algorithm, a spline basis algorithm, a wavelet basis algorithm, a geometric feature algorithm, an image characteristic feature algorithm, a statistical feature algorithm, or the like, or any combination thereof.

In some embodiments, the processing device140may determine a first image section (also referred to as “first rigid registration image section”) of the first rigid registration image corresponding to the overlapping region and a second image section (also referred to as “second rigid registration image section”) of the second rigid registration image corresponding to the overlapping region. Further, the processing device140may perform the non-rigid registration on the first rigid registration image section and the second rigid registration image section. Take the first rigid registration image section as an example, the processing device140may determine (or update) a first transformation function associated with a similarity between the second rigid registration image section and the first rigid registration image section. Then the processing device140may perform the non-rigid registration on the first rigid registration image section based on the first (updated) transformation function to generate a first corrected image section. Accordingly, the first corrected image section may be used to replace the first rigid registration image section in the first rigid registration image and remainder sections are kept unchanged.

In some embodiments, the processing device140may determine a first correction weight function corresponding to the first image section of the first rigid registration image corresponding to the overlapping region, and then generate the first corrected image (or the first corrected image section) based on the first non-rigid registration result and the first correction weight function. The processing device140may also determine a second correction weight function corresponding to the second image section of the second rigid registration image corresponding to the overlapping region based on the first correction weight function, and then generate the second corrected image (or the second corrected image section) based on the second non-rigid registration result and the second correction weight function. In some embodiments, for a first pixel in the first image section and a second pixel in the second image section (the first pixel and the second pixel correspond to a same position in the overlapping region), a sum of the first correction weight function (e.g., a value thereof) and the second correction weight function (e.g., a value thereof) may be a constant value (e.g., 1).

In some embodiments, the first correction weight function may be associated with a height of the first image section and distances of pixels in the first image section to an edge (e.g., an upper edge) of the first image section (as illustrated inFIG.6AorFIG.6B, we can assume that the first image section is located above the second image section). In some embodiments, the farther a pixel in the first image section from the upper edge of the first image section is, that is, the larger a distance of the pixel in the first image section to the upper edge of the first image section is, the smaller the first correction weight function of the pixel may be. For example, the processing device140may determine the first correction weight function and the second correction weight function according to formulas (1) and (2) below:

where wArefers to the first correction weight function, wBrefers to the second correction weight function, H refers to the height of the first image section, and d refers to a distance between a pixel and the upper edge of the first image section.

In some embodiments, after determining the first correction weight function and the second correction weight function, the processing device140may generate the first corrected image (or the first corrected image section) and the second corrected image (or the second corrected image section) respectively according to formulas (3) and (4) below:

where wArefers to the first correction weight function, wBrefers to the second correction weight function, disA refers to the first non-rigid registration result, disB refers to the second non-rigid registration result, IA(x, y) refers to a pixel (x, y) in the first image section of the first rigid registration image, IB(x, y) refers to a pixel (x, y) in the second image section of the second rigid registration image, I′A(x, y) refers to a pixel (x, y) in the first corrected image (or the first corrected image section), and I′B(x, y) refers to a pixel (x, y) in the second corrected image (or the second corrected image section).

It should be noted that the above descriptions are for illustration purposes and be non-limiting. In some embodiments, the processing device140may directly perform the non-rigid registration on the first image (or the first image section) based on the second image (or the second image section) and perform the non-rigid rigid registration on the second image (or the second image section) based on the first image (or the first image section) without performing the rigid registration on the first image and the second image.

In550, the processing device140(e.g., the image combination module430) (e.g., the processing circuits of the processor320) may generate a combined image by combining the first corrected image and the second corrected image.

In some embodiments, the processing device140may generate the combined image using an image combination algorithm. For example, the image combination algorithm may include a weighted average algorithm (e.g., a stepwise weighted average algorithm), a simple maximum algorithm, a simple minimum algorithm, a simple average algorithm, a principal component analysis (PCA) algorithm, a discrete wavelet transform (DWT) algorithm, or the like, or any combination thereof.

In some embodiments, the processing device140may determine a first fusion weight function corresponding to a first image section (i.e., the first corrected image section) of the first corrected image corresponding to the overlapping region and a second fusion weight function corresponding to a second image section (i.e., the second corrected image section) of the second corrected image corresponding to the overlapping region. In some embodiments, for a first pixel in the first image section and a second pixel in the second image section (the first pixel and the second pixel correspond to a same position in the overlapping region), a sum of the first fusion weight function (e.g., a value thereof) and a second fusion weight function (e.g., a value thereof) may be a constant value (e.g., 1).

In some embodiments, the first fusion weight function may be associated with distances of pixels in the first image section of the first corrected image to an edge (e.g., an upper edge) of the first image section (as illustrated inFIG.6AorFIG.6B, we can assume that the first image section is located above the second image section). In some embodiments, the farther a pixel in the first image section from the upper edge of the first image section is, that is, the larger a distance of the pixel in the first image section to the upper edge of the first image section is, the smaller the first fusion weight function corresponding to the pixel may be. For example, the processing device140may determine the first fusion weight function and the second fusion weight function according to formulas (5) and (6) below:

where a refers to the first fusion weight function, b refers to the second fusion weight function, H refers to the height of the first image section, and d refers to a distance between a pixel and the upper edge of the first image section.

In some embodiments, the processing device140may determine a fused image corresponding to the overlapping region based on the first image section, the first fuse weight function, the second image section, and the second fuse weight function. For example, the processing device140may determine the fused image according to formulas (7) and (8) below:

where a refers to the first fusion weight function, b refers to the second fusion weight function, I′A(x, y) refers to a pixel (x, y) in the first image section of the first corrected image, I′B(x, y) refers to a pixel (x, y) in the second image section of the first corrected image, and I(x, y) refers to a pixel (x, y) in the fused image.

Further, the processing device140may generate the combined image by combining remainder sections of the first corrected image, remainder sections of the second corrected image, and the fused image.

FIGS.6A and6Bare a schematic diagram illustrating an exemplary first image and an exemplary second image of a subject according to some embodiments of the present disclosure.

As shown inFIG.6A, a first image610may include a first region of a subject and a second image620may include a second region of the subject. The first region and the second region may have an overlapping region. Accordingly, the first image610includes a first image section612corresponding to the overlapping region and the second image620includes a second image section622corresponding to the overlapping region.

As shown inFIG.6B, a first image630may include a first region of a subject and a second image640may include a second region of the subject. The first region and the second region may have an overlapping region650. As described in connection withFIG.5, take a pixel652in the first image630(or in a first image section of the first image630corresponding to the overlapping region) as an example, a first correction weight function of the pixel652may be associated with a height (e.g., H) of the first image section corresponding to the overlapping region and a distance (e.g., d) of the pixel652to an upper edge654of the first image section.

FIG.7Ais a schematic diagram illustrating an exemplary combined image determined based on a rigid registration.FIG.7Bis a schematic diagram illustrating an exemplary combined image according to some embodiments of the present disclosure.

As illustrated inFIG.7A, there are six images (e.g., a first image711, a second image712, a third image713, a fourth image714, a fifth image715, and a sixth image716) to be combined, wherein any adjacent two images have an overlapping region. Further, a combined image710was generated by performing a rigid registration on the six images. As illustrated inFIG.7B, a combined image720was generated according to the method (e.g., process500) described in the present disclosure. It can be seen that a content continuity of the combined image720is clearly better than a content continuity of the combined image710.

FIG.8is a flowchart illustrating an exemplary process for generating a combined image according to some embodiments of the present disclosure. In some embodiments, the process800may be implemented as a set of instructions stored in the storage220. The processor210and/or the modules inFIG.4may execute the set of instructions, and when executing the instructions, the processor210and/or the modules may be configured to perform the process800. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process800may be accomplished with one or more additional operations not described and/or without one or more of the operations herein discussed. Additionally, the order in which the operations of the process as illustrated inFIG.8and described below is not intended to be limiting.

In810, the processing device140(e.g., the image obtainment module410) (e.g., the interface circuits of the processor320) may obtain a first image of a first region of a subject and a second image of a second region of the subject. The first region and the second region may have an overlapping region. More descriptions regarding the first image and the second image may be found elsewhere in the present disclosure (e.g., operations510and520inFIG.5and the descriptions thereof).

In820, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may generate a first corrected image corresponding to the first image by performing a non-rigid registration on the first image using the second image as a reference.

In830, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may generate a second corrected image corresponding to the second image by performing a non-rigid registration on the second image using the first image as a reference. More descriptions regarding the first corrected image and the second corrected image may be found elsewhere in the present disclosure (operations530and540inFIG.5and the descriptions thereof).

In840, the processing device140(e.g., the image combination module430) (e.g., the processing circuits of the processor320) may generate a combined image based on the first corrected image and the second corrected image. More descriptions regarding the generation of the combined image may be found elsewhere in the present disclosure (operation550inFIG.5and the descriptions thereof).

FIGS.9A and9Bare a flowchart illustrating an exemplary process for generating a combined image according to some embodiments of the present disclosure. In some embodiments, the process900may be implemented as a set of instructions stored in the storage220. The processor210and/or the modules inFIG.4may execute the set of instructions, and when executing the instructions, the processor210and/or the modules may be configured to perform the process900. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process900may be accomplished with one or more additional operations not described and/or without one or more of the operations herein discussed. Additionally, the order in which the operations of the process as illustrated inFIG.9and described below is not intended to be limiting.

In910, the processing device140(e.g., the image obtainment module410) (e.g., the interface circuits of the processor320) may obtain a first image of a first region of a subject.

In920, the processing device140(e.g., the image obtainment module410) (e.g., the interface circuits of the processor320) may obtain a second image of a second region of the subject.

In some embodiments, the first region and the second region may be adjacent and have an overlapping region. Operations910and920may be similar to operations510and520, the descriptions of which are not repeated here.

In930, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may determine a first image section of the first image corresponding to the overlapping region based on a DICOM protocol of the subject.

In940, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may determine a second image section of the second image corresponding to the overlapping region based on the DICOM protocol of the subject.

In950, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may determine a first rigid registration image section corresponding to the first image section and a second rigid registration image section corresponding to the second image section by performing a rigid registration on the first image section and the second image section.

In960, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may perform a first non-rigid registration on the first rigid registration image section using the second rigid registration image section as a reference and generate a first non-rigid registration result based on the first non-rigid registration. The processing device140may also perform a second non-rigid registration on the second rigid registration image section using the first rigid registration image section as a reference and generate a second non-rigid registration result based on the second non-rigid registration.

In970, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may determine a first correction weight function corresponding to the first rigid registration image section. The processing device140may also determine a second correction weight function corresponding to the second rigid registration image section.

In980, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may generate a first corrected image corresponding to the first image based on the first non-rigid registration result and the first correction weight function. The processing device140may also generate a second corrected image corresponding to the second image based on the second non-rigid registration result and the second correction weight function. Operations930-980may be similar to operations530-540, the descriptions of which are not repeated here.

In990, the processing device140(e.g., the image correction module420) (e.g., the processing circuits of the processor320) may generate a combined image by combing the first corrected image and the second corrected image. Operation990may be similar to operation550, the descriptions of which are not repeated here.