Face hallucination using multisource references and cross‐scale dual residual fusion mechanism

International Journal of Intelligent SystemsEarly View RESEARCH ARTICLE Face hallucination using multisource references and cross-scale dual residual fusion mechanism Rui Wang, Rui Wang School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, ChinaSearch for more papers by this authorMuwei Jian, Corresponding Author Muwei Jian jianmuweihk@163.com orcid.org/0000-0002-4249-2264 School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China School of Information Science and Technology, Linyi University, Linyi, China Correspondence Muwei Jian, School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China. Email: jianmuweihk@163.comSearch for more papers by this authorHui Yu, Hui Yu School of Creative Technologies, University of Portsmouth, Portsmouth, UKSearch for more papers by this authorLin Wang, Lin Wang Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan, China QuanCheng Laboratory, Jinan, ChinaSearch for more papers by this authorBo Yang, Bo Yang Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan, China QuanCheng Laboratory, Jinan, ChinaSearch for more papers by this author Rui Wang, Rui Wang School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, ChinaSearch for more papers by this authorMuwei Jian, Corresponding Author Muwei Jian jianmuweihk@163.com orcid.org/0000-0002-4249-2264 School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China School of Information Science and Technology, Linyi University, Linyi, China Correspondence Muwei Jian, School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China. Email: jianmuweihk@163.comSearch for more papers by this authorHui Yu, Hui Yu School of Creative Technologies, University of Portsmouth, Portsmouth, UKSearch for more papers by this authorLin Wang, Lin Wang Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan, China QuanCheng Laboratory, Jinan, ChinaSearch for more papers by this authorBo Yang, Bo Yang Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan, China QuanCheng Laboratory, Jinan, ChinaSearch for more papers by this author First published: 23 August 2022 https://doi.org/10.1002/int.23024Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract There is an increasing interest in enhancing the quality of low-resolution (LR) facial images for various social life applications. Existing methods often use domain-specific prior knowledge, which is effective in improving the face super-resolution model's performance. However, it is challenging to obtain rich and accurate prior information from LR inputs in real-world scenarios, which can limit the robustness and generalization ability of the developed face super-resolution model. In this paper, a multisource reference-based face super-resolution Network, namely MSRNet, is proposed. Without considering the prior knowledge of faces, the network can reconstruct a LR face image with a magnitude factor of 8 under the guidance of multiple reference face images of different identities. By constructing an "appearance-alike" reference data set Face_Ref, the designed MSRNet aims to fully exploit the local and spatially similar high frequency information between the distinct references and the current face. More specifically, to effectively combine the information from multiple references, a cross-scale and cross-space feature fusion mechanism is introduced for external and internal references, and then the enhanced local semantics are finally incorporated into the high-resolution face reconstruction. The robustness of face image super-resolution is increased compared to current correlation approaches, since it not only eliminates the need for face prior knowledge but also avoids performing alignment operations on reference faces with multiple expressions and different poses. Experimental results show that the proposed model is able to produce results for face super-resolution that are satisfying and dependable and outperforms the state-of-the-art methods in terms of visual perceptual quality and quantity evaluation. Open Research DATA AVAILABILITY STATEMENT Research data are not shared. Early ViewOnline Version of Record before inclusion in an issue RelatedInformation