An artistic analysis model based on sequence cartoon images for scratch

International Journal of Intelligent SystemsVolume 37, Issue 11 p. 9598-9619 RESEARCH ARTICLE An artistic analysis model based on sequence cartoon images for scratch Xiaolin Chai, Xiaolin Chai orcid.org/0000-0002-7447-3367 Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, ChinaSearch for more papers by this authorYan Sun, Corresponding Author Yan Sun [email protected] orcid.org/0000-0002-8192-9545 Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China Correspondence Yan Sun, Department of Computer Science, Beijing University of Posts and Telecommunications, 100876 Beijing, China. Email: [email protected]Search for more papers by this authorHong Luo, Hong Luo Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, ChinaSearch for more papers by this authorMohsen Guizani, Mohsen Guizani Department of Electrical and Computer Engineering, University of Idaho, Moscow, Idaho, USASearch for more papers by this author Xiaolin Chai, Xiaolin Chai orcid.org/0000-0002-7447-3367 Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, ChinaSearch for more papers by this authorYan Sun, Corresponding Author Yan Sun [email protected] orcid.org/0000-0002-8192-9545 Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China Correspondence Yan Sun, Department of Computer Science, Beijing University of Posts and Telecommunications, 100876 Beijing, China. Email: [email protected]Search for more papers by this authorHong Luo, Hong Luo Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, ChinaSearch for more papers by this authorMohsen Guizani, Mohsen Guizani Department of Electrical and Computer Engineering, University of Idaho, Moscow, Idaho, USASearch for more papers by this author First published: 23 August 2022 https://doi.org/10.1002/int.23017Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinkedInRedditWechat Abstract With the development of visual programming languages, researchers pay attention to the automatic evaluation of visual projects. Previous work focus on the code evaluation but ignored another essential part—the visualization results. Scratch is a widely used programming platform, and projects created on it are displayed in the form of cartoon clips. It is valuable to explore the visual aesthetics embodied in these clips to fill the gap in the assessment system. We propose a model that predicts the human view scores of cartoon clips created on Scratch. Our method is divided into two steps to evaluate the aesthetic of the sequence images that compose cartoon clips. First, we train an image classification network to predict the relative aesthetics of individual images. Then we construct an aesthetic space for the sequence image and improve the rating within a specific range. We put forward ScratchGAN to generate a Scratch-cartoon-style aesthetic analysis data set for training the classification network. Experimental results show that our Generative Adversarial Network framework can well transform photos into a Scratch-cartoon style. The single image assessment network can generate predictions that fit human cartoon aesthetic opinions. Our method achieves satisfactory results in the aesthetic evaluation of sequence cartoon images. Open Research DATA AVAILABILITY STATEMENT Research data are not shared. REFERENCES 1Resnick M, Maloney J, Monroy-Hernández A, et al. Scratch: programming for all. Commun ACM. 2009; 52(11): 60-67. 2Wilson A, Hainey T, Connolly T. Evaluation of computer games developed by primary school children to gauge understanding of programming concepts. In: European Conference on Games Based Learning. Academic Conferences International Limited; 2012:549. 3Brennan K, Resnick M. New frameworks for studying and assessing the development of computational thinking. In: Proceedings of the 2012 Annual Meeting of the American Educational Research Association, Vancouver, Canada. Vol 1; 2012: 25. 4Seiter L, Foreman B. Modeling the learning progressions of computational thinking of primary grade students. In: Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research; 2013: 59-66. 5Moreno-León J, Robles G, Román-González M. Dr. Scratch: automatic analysis of scratch projects to assess and foster computational thinking. RED—Rev Educ Distancia. 2015; 46: 1-23. 6Chang Z, Sun Y, Wu TY, Guizani M. Scratch analysis tool (SAT): a modern scratch project analysis tool based on ANTLR to assess computational thinking skills. In: 2018 14th International Wireless Communications & Mobile Computing Conference (IWCMC). IEEE; 2018: 950-955. 7Chai X, Sun Y, Luo H, Guizani M. DWES: a dynamic weighted evaluation system for scratch based on computational thinking. IEEE Trans Emerging Top Comput. 2021; 10(2): 917-932. 8Murray N, Marchesotti L, Perronnin F. AVA: A Large-Scale Database for Aesthetic Visual Analysis. IEEE; 2012: 2408-2415. 9 Pascoal DMANdR. Aesthetic Analysis of Images. Ph.D. Thesis, Universidade de Coimbra; 2015. 10Lu X, Lin Z, Jin H, Yang J, Wang JZ. Rating image aesthetics using deep learning. IEEE Trans Multimedia. 2015; 17(11): 2021-2034. 11Kong S, Shen X, Lin Z, Mech R, Fowlkes C. Photo Aesthetics Ranking Network with Attributes and Content Adaptation. Springer; 2016: 662-679. 12Deng Y, Loy CC, Tang X. Image aesthetic assessment: an experimental survey. IEEE Signal Proc Mag. 2017; 34(4): 80-106. 13Ma S, Liu J, WenChen C. A-lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017: 4535-4544. 14Sheng K, Dong W, Ma C, Mei X, Huang F, Hu BG. Attention-based multi-patch aggregation for image aesthetic assessment. In: Proceedings of the 26th ACM International Conference on Multimedia; 2018: 879-886. 15Schwarz K, Wieschollek P, Lensch HP. Will People Like Your Image? Learning the Aesthetic Space. IEEE; 2018: 2048-2057. 16Talebi H, Milanfar P. NIMA: neural image assessment. IEEE Trans Image Process. 2018; 27(8): 3998-4011. 17Jin X, Wu L, Li X, et al. Predicting aesthetic score distribution through cumulative jensen–shannon divergence. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol 32; 2018: 77-84. 18Pan B, Wang S, Jiang Q. Image aesthetic assessment assisted by attributes through adversarial learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol 33; 2019: 679-686. 19Ungvarsky D, Oliver L. DPChallenge—a digital photogra. Accessed February 2, 2022. Available at https://www.dpchallenge.com/ 20Henderson C. Flickr: Find Your Inspiration. Accessed February 8, 2022. Available at https://www.flickr.com/ 21Kamitani T. Pixiv. Accessed February 2, 2022. Available at https://www.pixiv.net/ 22Datta R, Li J, Wang JZ. Algorithmic Inferencing of Aesthetics and Emotion in Natural Images: an Exposition. IEEE; 2008: 105-108. 23Tang X, Luo W, Wang X. Content-based photo quality assessment. IEEE Trans Multimedia. 2013; 15(8): 1930-1943. 24Chen Y, Lai YK, Liu YJ. Cartoongan: generative adversarial networks for photo cartoonization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018: 9465-9474. 25Chen J, Liu G, Chen X. Animegan: A Novel Lightweight GAN for Photo Animation. Springer; 2019: 242-256. 26Wang X, Yu J. Learning to cartoonize using white-box cartoon representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020: 8090-8099. 27Lu X, Lin Z, Jin H, Yang J, Wang JZ. Rapid: Rating Pictorial Aesthetics Using Deep Learning. In: Proceedings of the 22nd ACM International Conference on Multimedia; 2014: 457-466. 28Lu X, Lin Z, Shen X, Mech R, Wang JZ. Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision; 2015: 990-998. 29He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal. 2015; 37(9): 1904-1916. 30Mai L, Jin H, Liu F. Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 497-506. 31Wang Z, Chang S, Dolcos F, Beck D, Liu D, Huang TS. Brain-inspired deep networks for image aesthetics assessment. arXiv preprint arXiv:1601.04155. 2016. 32Chatterjee A. Prospects for a cognitive neuroscience of visual aesthetics. Bull Psychol Arts. 2003; 4: 55-60. 33Hosu V, Goldlucke B, Saupe D. Effective aesthetics prediction with multi-level spatially pooled features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019: 9375-9383. 34Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representations. 2015. 35Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 0 2818-2826. 36Howard AG, Zhu M, Chen B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861. 2017. 37Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision; 2017: 0 2223-2232. 38Kanezaki A. Unsupervised Image Segmentation by Backpropagation. IEEE; 2018: 1543-1547. 39Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012; 25: 1097-1105. 40Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence; 2017; 0: 4278-4284. 41Hou L, Yu CP, Samaras D. Squared earth mover's distance-based loss for training deep neural networks. arXiv preprint arXiv:1611.05916. 2016. 42Hoffer E, Ailon N. Deep Metric Learning Using Triplet Network. Springer; 2015: 84-92. 43He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 0 770-778. 44Abadi M, Barham P, Chen J, et al. Tensorflow: A system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16); 2016: 0 265-283. 45Smith LN. Cyclical Learning Rates for Training Neural Networks. IEEE; 2017: 464-472. Volume37, Issue11November 2022Pages 9598-9619 ReferencesRelatedInformation