Semantic‐aware automatic image colorization via unpaired cycle‐consistent self‐supervised network

International Journal of Intelligent SystemsVolume 37, Issue 2 p. 1222-1238 RESEARCH ARTICLE Semantic-aware automatic image colorization via unpaired cycle-consistent self-supervised network Yuxuan Xiao, Yuxuan Xiao orcid.org/0000-0001-9726-4034 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, ChinaSearch for more papers by this authorAiwen Jiang, Corresponding Author Aiwen Jiang jiangaiwen@jxnu.edu.cn orcid.org/0000-0002-5979-7590 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China Correspondence Aiwen Jiang, School of Computer and Information Engineering, Jiangxi Normal University, No. 99, Ziyang Ave, Nanchang 330022, China. Email: jiangaiwen@jxnu.edu.cnSearch for more papers by this authorChanghong Liu, Changhong Liu orcid.org/0000-0003-4673-5806 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, ChinaSearch for more papers by this authorMingwen Wang, Mingwen Wang orcid.org/0000-0002-8118-9612 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, ChinaSearch for more papers by this author Yuxuan Xiao, Yuxuan Xiao orcid.org/0000-0001-9726-4034 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, ChinaSearch for more papers by this authorAiwen Jiang, Corresponding Author Aiwen Jiang jiangaiwen@jxnu.edu.cn orcid.org/0000-0002-5979-7590 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China Correspondence Aiwen Jiang, School of Computer and Information Engineering, Jiangxi Normal University, No. 99, Ziyang Ave, Nanchang 330022, China. Email: jiangaiwen@jxnu.edu.cnSearch for more papers by this authorChanghong Liu, Changhong Liu orcid.org/0000-0003-4673-5806 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, ChinaSearch for more papers by this authorMingwen Wang, Mingwen Wang orcid.org/0000-0002-8118-9612 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, ChinaSearch for more papers by this author First published: 19 September 2021 https://doi.org/10.1002/int.22667Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Automatic image colorization without manual interventions is an ill-conditioned and inherently ambiguous problem. Most of existing methods focus on formulating colorization as a regression problem and learn parametric mappings from grayscale to color through deep neural networks. Due to the multimodalities of color-grayscale space, in many applications, it is not required to recover exact ground-truth color. Pair-wise pixel-to-pixel learning-based algorithms lack rationality. Techniques such as color space conversion techniques are then proposed to avoid such direct pixel learning. However, the coloring results after color space conversion are blunt and unnatural. In this paper, we hold viewpoints that a reasonable solution is to generate some colorized result that looks natural. No matter what color a region is to be assigned, the colorized region should be semantically and spatially consistent. In this paper, we propose an effective semantic-aware automatic colorization model via unpaired cycle-consistent self-supervised network. Low-level monochrome loss, perceptual identity loss and high-level semantic-consistence loss, together with adversarial loss, are introduced to guide network self-training. We train and test our model on randomly selected subsets from PASCAL VOC 2012. The experimental results including human subjective studies demonstrate that, compared with state-of-the-art methods, our proposed model can achieve more convincing and superior results. Relevant source code is available at https://github.com/YuSuen/ACCycleGAN. Volume37, Issue2February 2022Pages 1222-1238 RelatedInformation