# Application of Image Fusion in Diagnosis and Treatment of Liver Cancer
## 6 Discussion on the Limitations and Prospects of Medical Image Fusion Technology
These features would allow wider usage of this technology by individuals with less experience in image fusion. With further development of follow-up physical diagnosis technologies, improved fusion imaging may be widely applied in clinical practice to achieve an early diagnosis and treatment of tumors in the future. Another interesting research topic is to reduce the computational time of the registration/fusion algorithm, or to speed up the fusion procedure. The relatively large computational time puts limitation of the implementation of several fusion algorithms in specific clinical applications based on the requirement of these medical studies. For example, as discussed in Section 5.1, for the real-time fusion of US and MRI or CT for liver lesion diagnosis, current clinical application can only use fusion results by a simple arithmetic combination due to its fast processing speed. Several research directions could be useful to overcome this challenge, such as (i) applying high performance computation, which utilizes parallel computing, to obtain efficient image fusion process; (ii) using a pretrained deep learning network for high speed image fusion processing. As shown in Section 4.10, the deep learning method becomes more and more popular for the registration/fusion of medical images. However, the robustness and availability of the dataset still constrain the usage of such a method in clinical image fusion. For instance, the size of available data is often very limited due to the privacy of these clinical data. Secondly, the need for medical experts to label the available dataset is very time- and finance-consuming. At last, the quality of the data, especially pathological data, cannot be guaranteed. To solve the problem of data shortage, researchers have proposed and applied data augmentation in medical image processing, which increase the diversity and size of the dataset without obtaining new data. The simplest data augment function such as random image rotations or nonlinear deformations are easy to implement, but lack the ability to emulate real variations. The more advanced methods, such as few-shot/one-shot learning [187] and attribute-guided augmentation (AGA) [188], could produce a wide variety of realistic new images for the deep learning-based image fusions with little supervised data samples. Unsupervised learning is another important research direction to overcome the small dataset challenge. For instance, stack autoencoders is one of the popular feature extraction unsupervised learning algorithms used in the image registration/fusion process. A newly developed learning module, Spatial Transformer (ST) [189],can make explicit use of the data's spatial information and can be inserted into CNNs. This makes CNNs invariant to translation, scaling, rotation and common distortions without additional training, and thus to be able to register medical images without training datasets. Another relative new and popular algorithm, Generative Adversarial Networks (GANs) [190], creates a generative network and a discriminative network at the same time. The network can receive end-to-end training and learn representative features in a completely unsupervised way which provides a research direction in deep learning-based image registration/fusion. As discussed in the previous sections, all of the existing image fusion techniques possess their own strength or weakness. With efficient combination of different image fusion methods, the advantages of different fusion methods could be combined for higher image qualities, while the weakness of these methods could be avoided. At the same time, a new and more efficient algorithm still needs to be developed to improve the quality and visualization of the fusion image, while reducing the errors due to resolution difference between images, image noise and the dimension difference between images. Noise effects due to signal noise could affect the image fusion process in a negative way.