# Online Simulation Quality Assessment of Illustration Patterns Based on Digital Art Design in Neural Network Perspective

Authors: Lin Yang
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Digital art illustration as new media is a major advancement in the history of illustration. It has broadened the application of graphic arts and the way of communication. At present, digital illustrators are constantly exploring new artistic expressions for digital illustration in order to pursue new digital effects. A wide range of integrated materials and experimental design concepts are gradually emerging, making digital illustration evolve in the competition. The artistic style of digital illustration has also shown the development trend of “diversification.” With the trend of commercialization of digital illustration, the quality of illustration patterns is gradually neglected. The traditional illustration pattern assessment relies on manual subjective judgment, with backward evaluation means and poor accuracy. This study proposes an illustration pattern evaluation method based on a deep neural network. In particular, this study proposes a reference-free image evaluation model with multiple feature fusion; specifically, we use CNN and information entropy-based methods for feature extraction and regularization methods for information fusion to solve the problem of missing reference images in applications. The chunking process is performed on the basis of considering the influence of information entropy on image quality, and the information entropy of multiple chunked features is calculated as importance weights, representing the degree of their influence on distorted image quality. The experimental results on the digital illustration pattern database show that the method in this study has strong robustness and can output a reasonable and reliable quality assessment score for distorted patterns.


---

## Body

## 1. Introduction

The art of digital illustration has matured with the development of digital technology. Digital technology has developed at an amazing rate in just a few decades. Driven by digital technology, illustration art has derived new visual effects and greatly enriched the language of illustration art. Digital visual communication stands rightfully in front of the public as one of the most applicable means to realize human life. It has profoundly changed the way people access information. At the same time, powerful computer software systems have revolutionized the illustrator’s own industrial revolution. Efficient digital illustration art has also become a key part of the “age of reading picture” [1]. In addition, the digital age has a broader impact on human work, thought, and so on. Illustration art is an important way of visual language, which has the advantages of being intuitive, realistic, and infectious. It is loved by the general public. The impact of digitalization on the art of illustration is multifaceted; for example, it changes the means of creating illustration, makes the art of illustration more diverse, and makes the art of illustration more attractive to the audience. In such a social context, digital illustration has a large “consumer base.” This potential commercial value was exploited, resulting in the art of illustration having a strong market value, while making it active, popular, and greatly stimulating its development. Digital technology has started to fully intervene in various industries of commercial art, such as graphic design, commercial illustration, exhibition display, advertising and packaging, industrial styling design, environmental art design, and architectural design [2].Compared with traditional illustration, digital illustration is more diversified and richer in terms of form and content. Specifically, first, according to the performance language and style criteria, it can be divided into conceptual digital illustration, hybrid digital illustration, vector digital illustration, and abstract digital illustration. Second, according to the performance content standard, it can be divided into digital illustration of people, digital illustration of animals, digital illustration of landscape, and so on. Third, according to the market perspective, it can be divided into commercial digital illustration and noncommercial digital illustration. Although with the aid of computer software, illustration has become easier to draw. However, to create excellent digital illustration works, it is also necessary to follow certain drawing methods and strict image quality assessment criteria. The merits of the drawing affect the overall aesthetic and artistic standards of digital illustration to a large extent. Good drawing standards will make the creation of the entire illustration appear harmonious and smooth and create a sense of beauty to the audience after completion. Inappropriate composition will reduce the beauty of the entire work and lower its artistic standards. However, with the increasing proportion of commercialization, many hand-drawn designers choose to learn software to adapt to this fast-paced society, so “fast food design” has emerged. People seem to be more inclined to the pursuit of technology but ignore the pursuit of pattern art and quality.Image quality assessment has been a fundamental problem in the field of image processing and computer vision. Image quality assessment models are also widely used in the fields of image coding, super-resolution reconstruction, image quality enhancement, and other related fields [3]. Image quality assessment mainly includes full-reference image quality assessment, half-reference image quality assessment, and no-reference image quality assessment [4]. Full-reference image quality assessment and half-reference image quality assessment refer to the full availability and partial availability of reference information for predicting image quality, respectively, while no-reference image quality assessment refers to the unavailability of reference information for predicting image quality. Although full-reference and half-reference image quality assessment models are more reliable, they must rely on reference information in the calculation process, which makes the application extremely limited. The reference-free image quality assessment model has been a hot topic of research in the field of image quality assessment because it is not dependent on reference information and has strong applicability.Digital illustrations, as images generated by information technology, are easy to obtain a large amount of raw image data. However, annotating images is time consuming and laborious, so it is difficult to collect reference images for training models. The main contributions of this study are as follows: (1) in this study, we propose a quality assessment method for reference-free illustration patterns based on multifeature fusion. For an illustration pattern, this study first chunks it into multiple blocks. (2) Then, the information entropy of each block is calculated as the importance weight of the block, which represents its impact on the quality of the distorted image. The loss function of the convolutional neural network is adjusted according to this weight. (3) The experimental results show that the reference-free image quality assessment method proposed in this study can accurately assess the quality of digital illustration patterns, and the assessment results are close to the subjective visual assessment of humans.


## 2. Related Works

### 2.1. Research Status of Illustration Pattern Based on Digital Art Design

Digital illustration art has the characteristics of digital creativity and digital media display and interaction. Digital illustration art is a product of the digital age and is a branch of digital art. Digital illustration art is a visual art that is created and designed to achieve certain purposes through “digital” technology. This is a broad definition of digital illustration art. As long as there is a digital component involved in the creation and final display process of illustration, it can be summarized as digital illustration art, for example, preinformation collection, postsynthesis, diversity of creators, and multimedia presentation. Digital illustration art is a comprehensive digital creation art form. It has the distinctive features of fusion of digital technology and art as well as artistic characteristics and aesthetic features of the digital era. It breaks with the traditional means of creation and communication medium of illustration and has a unique aesthetic characteristic of “multiple synthesis” [5].In the information age, different fields have encountered new opportunities for development, and this is also true for the field of illustration. With the help of powerful information technology and tools, digital illustration art can exhibit many characteristics, especially in terms of performance language and display space [6]. Digital illustration art is very different from traditional illustration art. However, in essence, digital illustration art in the information age is also developed on the basis of traditional flower arrangement art. Therefore, although they have different qualities, digital illustration art adheres to many characteristics of traditional illustration art. For example, traditional illustration art focuses on innovation and inspiration. This has not changed in the digital illustration art of the information age. In addition, traditional illustration art attaches importance to the use of natural elements, thus reflecting the harmony and unity between man and nature. Likewise, many natural elements can be properly used in the creation of digital illustration art of information age. Unlike traditional illustration, which creates virtual through cloth, paint, and so on, digital illustration mainly realizes virtual through various digital techniques and devices.Of course, the virtual aspect of digital illustration also has the following two meanings: one meaning is “simulation.” Digital illustration presents reality to the viewer through digital technology, which can only simulate a visual image of reality on the screen. The other meaning is “breakthrough.” Due to the strong simulation ability of digital illustration, digital illustration can go beyond the confines of “reality” and create images that do not exist in reality at will. Interactivity is also one of the characteristics of digital technology and the direction of digital illustration art. Traditional illustration is a static presentation, information dissemination is one-way and passive, and interactivity is not comparable to digital illustration. The interactivity of digital illustration is mainly reflected in the following ways: first, the digital illustration is made more dynamic through digital technology, which makes the audience feel as if they are in the scene when viewing it. Second, digital technology has built a free and open platform for digital illustration, breaking the limitation of time and space, so that viewers can enjoy illustration art anytime and anywhere. Third, the audience has the opportunity to give their own understanding and assessment of the art of illustration. Thus, digital illustration has an interactive, two-way, peer-to-peer exchange of information [7].As digital technology continues to develop, the scope of application of digital illustration artworks becomes broader and broader. The new digital technology provides a strong support for the creators. The artistic concept and thinking of the creators have also become more extensive. The current digital illustration art shows a diversified development trend. On the one hand, in the development of information society, the use of digital illustration artworks has become a habit. At present, different types of illustrations have different illustrators, which is a reflection of the professionalism of digital illustration. At present, some online games are more popular. Digital illustration has also started to enter the game field and has a surprising market development space. In addition, during the development of digital illustration art, traditional elements such as color, culture, and pattern have been incorporated into digital illustration art, and over time, have taken on a deeper cultural and spiritual dimension. For this reason, in the subsequent social development process, the art of digital illustration needs to be given a diversified development.In terms of visual expression, the vectorized style of images is the unique artistic style of digital illustration art compared to traditional illustration art. Graphics with vectorized visual characteristics can be scaled infinitely without distortion. Its lines are smooth and the graphics are complete and highly accurate. In particular, transparency and different overlay modes can create artistic styles with industrialized products. As a result, vector technology has become the best way to convey information in mass media such as printing, packaging, framing, publicity, advertising, posters, web, and video [8]. The traditional illustration technique of drawing is a linear creation and reworking on paper media with various brushes, while digital illustration is created in a more integrated way, with the emergence of the concept of nonlinear “synthesis.” In computer graphics software, the drawing material is redesigned and reprocessed, which not only greatly improves the efficiency of illustration creation but also enriches the expression form of illustration. The clarity of the digital illustration pattern directly affects the quality of the illustration. The use of digital tools has made art and design easy, and this development of high technology has made everyone an artist. The “threshold” of mediocrity has led to an increase in the number of practitioners, while the quality of drawings has decreased significantly. Therefore, there is a need for rigorous monitoring and assessment of the quality of digital illustration patterns.


### 2.2. Research Status of Image Quality Assessment

Image quality assessment is a measure of the visual quality of an image [9]. Digital images are disturbed by many factors during acquisition, compression storage, and transmission, resulting in distortion or degradation, which affects the human visual experience or the postprocessing effect of the image. Only with a correct assessment of the image quality, subsequent enhancement or control methods can be determined. Depending on whether or not there is human involvement, it can be divided into subjective image quality assessment and objective image quality assessment. Subjective quality assessment relies on human subjective perception to judge the quality of images. Since different people may have different perceptions of image quality, it is common practice for multiple people to evaluate distorted images and then take the average value. Objective image quality assessment requires the use of computers to build mathematical models. Then, to calculate and output digital measurement results, a high degree of consistency is required between objective image quality assessment results and subjective quality scores [10–13].According to the presence or absence of reference information, digital image quality assessment methods canbedivided into three categories: full-reference image quality assessment (FR-IQA) [14], reduced-reference image quality assessment (RR-IQA) [15], and no-reference image quality assessment (NR-IQA) [16]. The FR-IQA model and RR-IQA model calculate the visual quality of distorted images by analyzing the visual features of the images and quantifying the differences between the reference images and the distorted images. Compared with the FR-IQA model and RR-IQA model, the NR-IQA model does not require any reference image information for calculating the visual quality of distorted images and has a wider application prospect in practical systems. In this study, we focus on the current research status of NR-IQA model.Xue et al. [17] proposed an NR-IQA method for quality-aware clustering based on image blocks. First, the quality score of each image block is calculated by the FR-IQA algorithm, and all image blocks are divided into L major classes according to their quality score ranges. Then, the major image blocks are divided into K subclasses by the K-means algorithm based on the extracted structural features. Among them, each subclass corresponds to a cluster center, namely, an image block (structural features and visual quality score). Given a distorted image, we first extract the image block and calculate the distance between the image block and the cluster center of each subclass in L major classes. The scores of the image blocks are then calculated by fusing the visual scores of the distance from the center of the smallest cluster in each major class. Finally, the visual quality scores of the distorted images were obtained using the averaging weighting strategy. Ye et al. [18] proposed an unsupervised IQA model BLISS (blind learning of image quality using synthetic scores). The authors obtained a single synthetic quality score by combining multiple valid FR-IQA models and used the synthetic quality score with high correlation as a subjective score approximation to further train the corresponding IQA model. Among them, the fractions calculated by different FR-IQA methods were fused by unsupervised sorting aggregation and the corresponding performance of the synthesized obtained quality fractions exceeded the performance of a single FR-IQA method. For screen images, Gu et al. [19] proposed a reference-free screen image quality assessment method. By extracting image features related to four aspects, including image complexity, screen content statistics, global brightness, and detail sharpness, a quality score is calculated using an existing high-performance full-reference screen image quality assessment method. The quality score is used as the training dataset label to obtain the final screen image quality assessment model.The unsupervised IQA model based on traditional machine learning algorithms measures the difference in image quality by constructing a statistical model. The unsupervised IQA model based on deep learning achieves the mapping between image and quality scores by constructing effective training data. This type of method solves the problem of lack of training data in data-driven models to a certain extent, and the performance is much higher than that of NR-IQA models based on traditional machine learning.KangCNN [20] model was proposed by Kang et al. It is the first model that employs the convolutional neural network (CNN) to solve the NR-IQA problem. It incorporates feature extraction and fractional regression into a unified framework for implementation. CNN has 5 layers, including 1 convolutional layer, 1 pooling layer, 2 fully connected layers, and 1 output node layer. The quality score of the whole image is based on the local quality assessment score, which is obtained by taking the average of the scores. The pooling layer performs maximum pooling and minimum pooling operations on the full-size feature maps of the convolutional layer respectively to obtain two feature vectors of dimension 50. The more popular ReLU unit is used between the fully connected layers. The loss function used is similar to SVR and can be seen formally as SVR taking ε to zero. DeepIQA proposed by Bosse et al. [21] is also an IQA model based on an end-to-end framework. DeepIQA combines full-reference IQA tasks and NR-IQA tasks in a single network implementation. Full-reference IQA is trained using the Siamese network, and a branch of the Siamese network can be used for NR-IQA with minor modifications. DeepIQA has achieved good results on both artificially simulated distortion datasets and natural distortion datasets.Ma et al. proposed a bilinear pooling-based CNN structure for NR-IQA [22]. The network consists of two branching networks and a bilinear pooling module. High-quality images from the dataset are used to synthesize different types of distorted images with different degrees of distortion, and the branch network is trained with classification. Another branch network uses VGG16, which is pretrained on ImageNet datasets to improve the perception of natural distortion across the network. Finally, the bilinear pooling module is used to fuse the fully connected layer features of the two networks as the depth features of the images, and then, the mapping relationship between the depth features and the quality scores is fitted. This method achieves the highest performance on both synthetic distortion and natural distortion IQA datasets, bringing new ideas to the current NR-IQA research efforts.Although the current NR-IQA method can achieve good test results in artificial simulation datasets, it is not very scalable. So, how to design IQA models with robustness and apply them to the task of digital illustration quality assessment deserves further research.


## 2.1. Research Status of Illustration Pattern Based on Digital Art Design

Digital illustration art has the characteristics of digital creativity and digital media display and interaction. Digital illustration art is a product of the digital age and is a branch of digital art. Digital illustration art is a visual art that is created and designed to achieve certain purposes through “digital” technology. This is a broad definition of digital illustration art. As long as there is a digital component involved in the creation and final display process of illustration, it can be summarized as digital illustration art, for example, preinformation collection, postsynthesis, diversity of creators, and multimedia presentation. Digital illustration art is a comprehensive digital creation art form. It has the distinctive features of fusion of digital technology and art as well as artistic characteristics and aesthetic features of the digital era. It breaks with the traditional means of creation and communication medium of illustration and has a unique aesthetic characteristic of “multiple synthesis” [5].In the information age, different fields have encountered new opportunities for development, and this is also true for the field of illustration. With the help of powerful information technology and tools, digital illustration art can exhibit many characteristics, especially in terms of performance language and display space [6]. Digital illustration art is very different from traditional illustration art. However, in essence, digital illustration art in the information age is also developed on the basis of traditional flower arrangement art. Therefore, although they have different qualities, digital illustration art adheres to many characteristics of traditional illustration art. For example, traditional illustration art focuses on innovation and inspiration. This has not changed in the digital illustration art of the information age. In addition, traditional illustration art attaches importance to the use of natural elements, thus reflecting the harmony and unity between man and nature. Likewise, many natural elements can be properly used in the creation of digital illustration art of information age. Unlike traditional illustration, which creates virtual through cloth, paint, and so on, digital illustration mainly realizes virtual through various digital techniques and devices.Of course, the virtual aspect of digital illustration also has the following two meanings: one meaning is “simulation.” Digital illustration presents reality to the viewer through digital technology, which can only simulate a visual image of reality on the screen. The other meaning is “breakthrough.” Due to the strong simulation ability of digital illustration, digital illustration can go beyond the confines of “reality” and create images that do not exist in reality at will. Interactivity is also one of the characteristics of digital technology and the direction of digital illustration art. Traditional illustration is a static presentation, information dissemination is one-way and passive, and interactivity is not comparable to digital illustration. The interactivity of digital illustration is mainly reflected in the following ways: first, the digital illustration is made more dynamic through digital technology, which makes the audience feel as if they are in the scene when viewing it. Second, digital technology has built a free and open platform for digital illustration, breaking the limitation of time and space, so that viewers can enjoy illustration art anytime and anywhere. Third, the audience has the opportunity to give their own understanding and assessment of the art of illustration. Thus, digital illustration has an interactive, two-way, peer-to-peer exchange of information [7].As digital technology continues to develop, the scope of application of digital illustration artworks becomes broader and broader. The new digital technology provides a strong support for the creators. The artistic concept and thinking of the creators have also become more extensive. The current digital illustration art shows a diversified development trend. On the one hand, in the development of information society, the use of digital illustration artworks has become a habit. At present, different types of illustrations have different illustrators, which is a reflection of the professionalism of digital illustration. At present, some online games are more popular. Digital illustration has also started to enter the game field and has a surprising market development space. In addition, during the development of digital illustration art, traditional elements such as color, culture, and pattern have been incorporated into digital illustration art, and over time, have taken on a deeper cultural and spiritual dimension. For this reason, in the subsequent social development process, the art of digital illustration needs to be given a diversified development.In terms of visual expression, the vectorized style of images is the unique artistic style of digital illustration art compared to traditional illustration art. Graphics with vectorized visual characteristics can be scaled infinitely without distortion. Its lines are smooth and the graphics are complete and highly accurate. In particular, transparency and different overlay modes can create artistic styles with industrialized products. As a result, vector technology has become the best way to convey information in mass media such as printing, packaging, framing, publicity, advertising, posters, web, and video [8]. The traditional illustration technique of drawing is a linear creation and reworking on paper media with various brushes, while digital illustration is created in a more integrated way, with the emergence of the concept of nonlinear “synthesis.” In computer graphics software, the drawing material is redesigned and reprocessed, which not only greatly improves the efficiency of illustration creation but also enriches the expression form of illustration. The clarity of the digital illustration pattern directly affects the quality of the illustration. The use of digital tools has made art and design easy, and this development of high technology has made everyone an artist. The “threshold” of mediocrity has led to an increase in the number of practitioners, while the quality of drawings has decreased significantly. Therefore, there is a need for rigorous monitoring and assessment of the quality of digital illustration patterns.


## 2.2. Research Status of Image Quality Assessment

Image quality assessment is a measure of the visual quality of an image [9]. Digital images are disturbed by many factors during acquisition, compression storage, and transmission, resulting in distortion or degradation, which affects the human visual experience or the postprocessing effect of the image. Only with a correct assessment of the image quality, subsequent enhancement or control methods can be determined. Depending on whether or not there is human involvement, it can be divided into subjective image quality assessment and objective image quality assessment. Subjective quality assessment relies on human subjective perception to judge the quality of images. Since different people may have different perceptions of image quality, it is common practice for multiple people to evaluate distorted images and then take the average value. Objective image quality assessment requires the use of computers to build mathematical models. Then, to calculate and output digital measurement results, a high degree of consistency is required between objective image quality assessment results and subjective quality scores [10–13].According to the presence or absence of reference information, digital image quality assessment methods canbedivided into three categories: full-reference image quality assessment (FR-IQA) [14], reduced-reference image quality assessment (RR-IQA) [15], and no-reference image quality assessment (NR-IQA) [16]. The FR-IQA model and RR-IQA model calculate the visual quality of distorted images by analyzing the visual features of the images and quantifying the differences between the reference images and the distorted images. Compared with the FR-IQA model and RR-IQA model, the NR-IQA model does not require any reference image information for calculating the visual quality of distorted images and has a wider application prospect in practical systems. In this study, we focus on the current research status of NR-IQA model.Xue et al. [17] proposed an NR-IQA method for quality-aware clustering based on image blocks. First, the quality score of each image block is calculated by the FR-IQA algorithm, and all image blocks are divided into L major classes according to their quality score ranges. Then, the major image blocks are divided into K subclasses by the K-means algorithm based on the extracted structural features. Among them, each subclass corresponds to a cluster center, namely, an image block (structural features and visual quality score). Given a distorted image, we first extract the image block and calculate the distance between the image block and the cluster center of each subclass in L major classes. The scores of the image blocks are then calculated by fusing the visual scores of the distance from the center of the smallest cluster in each major class. Finally, the visual quality scores of the distorted images were obtained using the averaging weighting strategy. Ye et al. [18] proposed an unsupervised IQA model BLISS (blind learning of image quality using synthetic scores). The authors obtained a single synthetic quality score by combining multiple valid FR-IQA models and used the synthetic quality score with high correlation as a subjective score approximation to further train the corresponding IQA model. Among them, the fractions calculated by different FR-IQA methods were fused by unsupervised sorting aggregation and the corresponding performance of the synthesized obtained quality fractions exceeded the performance of a single FR-IQA method. For screen images, Gu et al. [19] proposed a reference-free screen image quality assessment method. By extracting image features related to four aspects, including image complexity, screen content statistics, global brightness, and detail sharpness, a quality score is calculated using an existing high-performance full-reference screen image quality assessment method. The quality score is used as the training dataset label to obtain the final screen image quality assessment model.The unsupervised IQA model based on traditional machine learning algorithms measures the difference in image quality by constructing a statistical model. The unsupervised IQA model based on deep learning achieves the mapping between image and quality scores by constructing effective training data. This type of method solves the problem of lack of training data in data-driven models to a certain extent, and the performance is much higher than that of NR-IQA models based on traditional machine learning.KangCNN [20] model was proposed by Kang et al. It is the first model that employs the convolutional neural network (CNN) to solve the NR-IQA problem. It incorporates feature extraction and fractional regression into a unified framework for implementation. CNN has 5 layers, including 1 convolutional layer, 1 pooling layer, 2 fully connected layers, and 1 output node layer. The quality score of the whole image is based on the local quality assessment score, which is obtained by taking the average of the scores. The pooling layer performs maximum pooling and minimum pooling operations on the full-size feature maps of the convolutional layer respectively to obtain two feature vectors of dimension 50. The more popular ReLU unit is used between the fully connected layers. The loss function used is similar to SVR and can be seen formally as SVR taking ε to zero. DeepIQA proposed by Bosse et al. [21] is also an IQA model based on an end-to-end framework. DeepIQA combines full-reference IQA tasks and NR-IQA tasks in a single network implementation. Full-reference IQA is trained using the Siamese network, and a branch of the Siamese network can be used for NR-IQA with minor modifications. DeepIQA has achieved good results on both artificially simulated distortion datasets and natural distortion datasets.Ma et al. proposed a bilinear pooling-based CNN structure for NR-IQA [22]. The network consists of two branching networks and a bilinear pooling module. High-quality images from the dataset are used to synthesize different types of distorted images with different degrees of distortion, and the branch network is trained with classification. Another branch network uses VGG16, which is pretrained on ImageNet datasets to improve the perception of natural distortion across the network. Finally, the bilinear pooling module is used to fuse the fully connected layer features of the two networks as the depth features of the images, and then, the mapping relationship between the depth features and the quality scores is fitted. This method achieves the highest performance on both synthetic distortion and natural distortion IQA datasets, bringing new ideas to the current NR-IQA research efforts.Although the current NR-IQA method can achieve good test results in artificial simulation datasets, it is not very scalable. So, how to design IQA models with robustness and apply them to the task of digital illustration quality assessment deserves further research.


## 3. Quality Assessment of Illustration Patterns Based on Multifeature Fusion

### 3.1. Information Entropy

The convolutional neural network (CNN) is a kind of common deep learning network. In this study, the IQA-CNN model combined with this network and image quality assessment (IQA) method is selected as the deep learning network to realize the reference-free assessment of digital illustration pattern quality. In the process of creating the training set of IQA-CNN model, the dataset is usually divided by the chunking process, and each chunk is labeled with the quality score of each chunk, so as to effectively solve the problem of the lack of the number of samples in the training set. The information entropy of the illustration pattern is rich in information about the structure of the image, and it is possible to measure the sharpness of part of the illustration pattern by such information. In general, it is customary to use the high acuity part to evaluate the image quality. Thus, the accuracy of the annotation is improved by the information entropy annotation of each chunk after chunking. Suppose the illustration pattern is represented byIx,y and the image has k levels, then its information entropy can be expressed as follows:(1)H=∑i=0k−1Pilog10Pi,where Pi represents the percentage of pixels with gray value i in the image, which is set at Pi=0, PilogPi=0 .This information entropy is used to label the information content of each chunk, which can represent the quality of each chunk.


### 3.2. Normalization of Fused Multipartition Features

The improved IQA-CNN model obtained by introducing information entropy into the IQA-CNN model is shown in Figure1. Before convolution, the image chunks are preprocessed with local normalization, which is helpful to remove the redundant features of the image that are weakly associated with the image quality.Figure 1
Improved IQA-CNN model.The local normalization method used is as follows:(2)I^i,j=Ii,j−μi,jσi,j+1i∈1,M,j∈1,N,μi,j=∑k=−KK∑l=−LLωk,lIi+k,j+l,σi,j=∑k=−KK∑l=−LLωk,lIi+k,j+l−μi,j2,where Ix,y is a distorted image. M and N represent the height and width of the image, respectively. P and Q represent the size of the normalized window, and the maximum value cannot exceed the size of the input image. ω is the weight of the Gaussian function window, and a window of size 3 × 3 is used in this study. Using 40 convolution kernels of size 7 × 7 on a 32 × 32 input image block with a sliding step of 1 for convolution, 40 feature maps of size 26 × 26 are obtained. Then, in the sampling layer, 40 feature maps are processed using the nonoverlapping large window sampling method. Unlike the small window sampling approach for classification tasks, this study uses 26 × 26 window sampling to map each feature map to an activation unit. In the output layer, the 40 output values are used as the input values of the next layer. In this study, two fully connected layers of 400 and 100 are used, and ReLU is used as its activation function, which helps to speed up the decrease of prediction error during training. Finally, the quality values of the images are obtained by a nonlinear regression loss function network based on information entropy, and the average of the quality of each chunk is taken to generate the quality fraction of the large image. The information entropy of each large image corresponding to a chunk is normalized as in equation (3) and used as the importance weight W of each chunk in the training process. N = 100 indicates that each large image is divided into 100 nonoverlapping chunks:(3)Wk=H∑i=1N2H.According to equation (3), the different distortion type plots in Figure 2(a) are chunked and their importance weights W are calculated. Figure 2(b) can be obtained by visualization. The importance weights W in Figure 2(b) are represented by different colors. From blue to red, the darker the color, the higher the weight (electronic version). Comparing Figures 2(a) and 2(b), it can be found that in the case that the whole image in Figure 2(a) is blurred, and the chunks with rich contents are still able to obtain higher weights in Figure 2(b). This indicates that the weights obtained by equation (3) can locate the content features of the image more accurately in the case of image distortion. Therefore, the chunks with higher weights are more representative in the training process of the convolutional network.Figure 2
Distorted illustration patterns and their normalized information entropy. (a) Distorted illustration patterns. (b) Normalized information entropy corresponding to different distortion patterns.
(a)(b)


### 3.3. Loss Function

Since the higher the weight of the image chunk, the higher its representation. Therefore, the loss function within the initial convolutional network is transformed as follows:(4)L=1N∑n=1Nωnfxn−ynl2,where label value of each block of image is represented by yn. Input image block is represented by xn. The convolutional network prediction score of xn is represented by fxn. In this way, when the information entropy is high, the block fraction that can better represent the large image quality deviates from the actual label value, and the convolutional network will generate a higher loss function than the original due to the high weight assigned by these blocks, so as to compensate for the fraction deviation. The adjustment of the loss function can effectively enhance the role played by the chunks that can represent the whole image features in the training process, while weakening the role of the less representative chunks and making the training process more effective.


### 3.4. Algorithm Steps

The process of illustration pattern quality assessment algorithm with multifeature fusion proposed in this study is detailed in Figure3. The specific process is as follows:(1)
Normalized preprocessing of the digitized illustration patterns in the input dataset and dividing the images into severalN × N chunks with no overlap between the chunks.(2)
The IQA-CNN model is trained to optimize the parameters of the model.(3)
The steps in the training stage are as follows: each subjective quality score is assigned to each image block according to the standard score of the original image. The training samplesxn,yn are input to the QA-CNN model and the objective function used is shown in equation (5).The fraction function predicted by the network with weight ω is represented by fω,xn. The objective function ω'=minωJ is minimized by combining the back propagation method with the gradient descent method. The normalized information entropy of all image chunks is used as the weight. This weight is used to adjust the loss function to increase the weight of the chunks that are more representative of the overall image quality and to optimize the parameters of the IQA-CNN model.(5)Jω;xn,yn=1N∑m=1Nfω,xn−yn2.(4)
Tests are performed on any given image. The image quality score is given directly by the IQA-CNN model in this study, and then, a weighted average is used to generate the predicted quality of the large image.Figure 3
Overall algorithm implementation flow.


## 3.1. Information Entropy

The convolutional neural network (CNN) is a kind of common deep learning network. In this study, the IQA-CNN model combined with this network and image quality assessment (IQA) method is selected as the deep learning network to realize the reference-free assessment of digital illustration pattern quality. In the process of creating the training set of IQA-CNN model, the dataset is usually divided by the chunking process, and each chunk is labeled with the quality score of each chunk, so as to effectively solve the problem of the lack of the number of samples in the training set. The information entropy of the illustration pattern is rich in information about the structure of the image, and it is possible to measure the sharpness of part of the illustration pattern by such information. In general, it is customary to use the high acuity part to evaluate the image quality. Thus, the accuracy of the annotation is improved by the information entropy annotation of each chunk after chunking. Suppose the illustration pattern is represented byIx,y and the image has k levels, then its information entropy can be expressed as follows:(1)H=∑i=0k−1Pilog10Pi,where Pi represents the percentage of pixels with gray value i in the image, which is set at Pi=0, PilogPi=0 .This information entropy is used to label the information content of each chunk, which can represent the quality of each chunk.


## 3.2. Normalization of Fused Multipartition Features

The improved IQA-CNN model obtained by introducing information entropy into the IQA-CNN model is shown in Figure1. Before convolution, the image chunks are preprocessed with local normalization, which is helpful to remove the redundant features of the image that are weakly associated with the image quality.Figure 1
Improved IQA-CNN model.The local normalization method used is as follows:(2)I^i,j=Ii,j−μi,jσi,j+1i∈1,M,j∈1,N,μi,j=∑k=−KK∑l=−LLωk,lIi+k,j+l,σi,j=∑k=−KK∑l=−LLωk,lIi+k,j+l−μi,j2,where Ix,y is a distorted image. M and N represent the height and width of the image, respectively. P and Q represent the size of the normalized window, and the maximum value cannot exceed the size of the input image. ω is the weight of the Gaussian function window, and a window of size 3 × 3 is used in this study. Using 40 convolution kernels of size 7 × 7 on a 32 × 32 input image block with a sliding step of 1 for convolution, 40 feature maps of size 26 × 26 are obtained. Then, in the sampling layer, 40 feature maps are processed using the nonoverlapping large window sampling method. Unlike the small window sampling approach for classification tasks, this study uses 26 × 26 window sampling to map each feature map to an activation unit. In the output layer, the 40 output values are used as the input values of the next layer. In this study, two fully connected layers of 400 and 100 are used, and ReLU is used as its activation function, which helps to speed up the decrease of prediction error during training. Finally, the quality values of the images are obtained by a nonlinear regression loss function network based on information entropy, and the average of the quality of each chunk is taken to generate the quality fraction of the large image. The information entropy of each large image corresponding to a chunk is normalized as in equation (3) and used as the importance weight W of each chunk in the training process. N = 100 indicates that each large image is divided into 100 nonoverlapping chunks:(3)Wk=H∑i=1N2H.According to equation (3), the different distortion type plots in Figure 2(a) are chunked and their importance weights W are calculated. Figure 2(b) can be obtained by visualization. The importance weights W in Figure 2(b) are represented by different colors. From blue to red, the darker the color, the higher the weight (electronic version). Comparing Figures 2(a) and 2(b), it can be found that in the case that the whole image in Figure 2(a) is blurred, and the chunks with rich contents are still able to obtain higher weights in Figure 2(b). This indicates that the weights obtained by equation (3) can locate the content features of the image more accurately in the case of image distortion. Therefore, the chunks with higher weights are more representative in the training process of the convolutional network.Figure 2
Distorted illustration patterns and their normalized information entropy. (a) Distorted illustration patterns. (b) Normalized information entropy corresponding to different distortion patterns.
(a)(b)


## 3.3. Loss Function

Since the higher the weight of the image chunk, the higher its representation. Therefore, the loss function within the initial convolutional network is transformed as follows:(4)L=1N∑n=1Nωnfxn−ynl2,where label value of each block of image is represented by yn. Input image block is represented by xn. The convolutional network prediction score of xn is represented by fxn. In this way, when the information entropy is high, the block fraction that can better represent the large image quality deviates from the actual label value, and the convolutional network will generate a higher loss function than the original due to the high weight assigned by these blocks, so as to compensate for the fraction deviation. The adjustment of the loss function can effectively enhance the role played by the chunks that can represent the whole image features in the training process, while weakening the role of the less representative chunks and making the training process more effective.


## 3.4. Algorithm Steps

The process of illustration pattern quality assessment algorithm with multifeature fusion proposed in this study is detailed in Figure3. The specific process is as follows:(1)
Normalized preprocessing of the digitized illustration patterns in the input dataset and dividing the images into severalN × N chunks with no overlap between the chunks.(2)
The IQA-CNN model is trained to optimize the parameters of the model.(3)
The steps in the training stage are as follows: each subjective quality score is assigned to each image block according to the standard score of the original image. The training samplesxn,yn are input to the QA-CNN model and the objective function used is shown in equation (5).The fraction function predicted by the network with weight ω is represented by fω,xn. The objective function ω'=minωJ is minimized by combining the back propagation method with the gradient descent method. The normalized information entropy of all image chunks is used as the weight. This weight is used to adjust the loss function to increase the weight of the chunks that are more representative of the overall image quality and to optimize the parameters of the IQA-CNN model.(5)Jω;xn,yn=1N∑m=1Nfω,xn−yn2.(4)
Tests are performed on any given image. The image quality score is given directly by the IQA-CNN model in this study, and then, a weighted average is used to generate the predicted quality of the large image.Figure 3
Overall algorithm implementation flow.


## 4. Experiments

### 4.1. Experimental Preparation

The dataset used in the training process of the network is the illustration pattern dataset collected by network retrieval. The dataset contains 1000 distorted images (from 20 reference images) with distortion types of JPEG compression (JPEG), white noise (WN), and Gaussian blur (GB). The database provides DMOS (differential mean opinion score) values for each image. The values range from 0 to 100, with higher DMOS values indicating lower image quality. The dataset is divided according to a ratio of training set:test set = 80%:20%. All comparison experiments in this study are based on Windows 10 CPU Intel Corei7 3.2 GHz with 64 GB of RAM and 1 Nvidia RTX2080Ti graphics card.Judging the performance of the image quality assessment algorithm requires experimental validation on a database, mainly by calculating the correlation between the objective quality scores of the assessment algorithm for distorted images and their corresponding subjective quality scores. If the correlation degree between them both is high, the algorithm performs better. The evaluation metrics used in this study are as follows:


#### 4.1.1. Linear Correlation Coefficient

The linear correlation coefficient (LCC) aims to calculate the linear correlation between two variables, as shown in equation (6). xi denotes the quality score predicted by the algorithm for the ith image, yi denotes the subjective quality score corresponding to the ithith image in the database, x¯ is the average of the algorithm’s prediction scores x1,x2,x3,…xn for all test images, y¯ is the average of the subjective quality fraction y1,y2,y3,…yn of all test images in the database, and n represents the number of test images. The linear correlation coefficient, as an index to evaluate the accuracy of the algorithm, is mainly used to calculate the linear correlation between the quality fraction predicted by the algorithm and the quality fraction in the corresponding database:(6)LCC=∑i=1nxi−x¯yi−y¯∑i=1nxi−x¯2yi−y¯2.


#### 4.1.2. Spearman’s Rank-Order Correlation Coefficient

Spearman’s rank-order correlation coefficient (SROCC) measures the closeness of the association between two variables, as shown in equation (7). rix is the ranking position of the ithith image in the sequence after xi has been arranged into a sequence by size and riy is the ranking position of the ithith image in the sequence after yi has been arranged into a sequence by size:(7)SROCC=6∑i−1nrix−riy2nn2−1.


#### 4.1.3. Pearson Linear Correlation Coefficient (PLCC)

The operator of the PLCC index is shown in equation (8). In this equation, the mean value obtained after averaging the algorithm-predicted quality scores of all test images is expressed as x¯. The mean value obtained after averaging the subjective quality scores of all test images is expressed as y¯:(8)PLCC=∑i=1Nxi−x¯yi−y¯∑i=1Nxi−x¯2∑i=1Nyi−y¯2.


### 4.2. Effect of the Number of Convolution Kernels on Model Performance

The evaluation performance of the algorithm was examined for different numbers of convolutional cores from 10 to 70 to determine the number of convolutional cores of the algorithm. The results are shown in Figure4. When the number of convolutional cores ranges from 10 to 40, the values of SROCC and PLCC show an increasing trend, but when the number of convolutional cores reaches 50, the values of both evaluation performance indicators of this algorithm decrease gradually. It can be seen that the number of convolutional kernels has a direct influence on the evaluation performance of this algorithm, and the number of convolutional kernels is selected as 40 for this algorithm.Figure 4
Effect of the number of convolution kernels on the model.


### 4.3. Effect of Sampling Step Size on Model Performance

The evaluation performance of the algorithm is also affected by the sampling step length selected when sampling the image chunks, so the evaluation performance of the algorithm is tested for 10 sampling steps in the range of 30 to 300. The obtained test results are shown in Figure5. The evaluation performance of this algorithm is the highest when the sampling step length is 90, and then, the evaluation performance of this algorithm decreases as the sampling step length continues to grow. The reason is that the image feature information that can be used to evaluate the image quality decreases with the increase of the sampling step, so the higher the step size the lower the evaluation performance of this algorithm. Therefore, the sampling step length of this algorithm is selected as 90.Figure 5
Effect of different sampling steps on model performance.


### 4.4. Evaluation Performance Test

With the selected number of convolution kernels and sampling steps, the algorithm proposed in this study is applied to evaluate the quality of digital illustration patterns in the dataset. In order to present the evaluation effect of this algorithm more clearly, three experimental images are presented as examples of these three illustration patterns. The quality of the three experimental images is demonstrated by DMOS scores, and the obtained subjective observations are compared with the actual evaluation results of each method to further test the evaluation effect of this algorithm. The three digital illustration patterns are shown in Figure6. The results of the quality evaluation scores of the three experimental images derived from the four algorithms are listed in Table 1. Combining Figure 6 and Table 1, it can be concluded that the visual quality of the three experimental images, from high to low, is defined once for images A, B, and C, respectively. The quality evaluation scores of BLISS [18] algorithm for the three images are close to each other. The quality evaluation scores of KangCNN [20] and DeepIQA [21] for images A and C are higher, while the evaluation score of image B is lower. The quality evaluation scores of the three experimental images by the algorithm in this study are similar to the visual subjective observation results from the highest to the lowest order. This indicates that the evaluation results obtained by this algorithm are more reasonable, with higher evaluation accuracy, and have higher practical application value.Figure 6
Distorted digital illustration pattern. (a) DMOS = 68.743. (b) DMOS = 53.271. (c) DMOS = 42.197.
(a)(b)(c)Table 1
Evaluation effects of different algorithms on distorted digital illustration patterns.
ModelImage AImage BImage CBLISS49.32847.21144.269KangCNN75.98969.40335.092DeepIQA77.39865.80437.925Proposed65.43850.10738.721Figure7 shows the predicted quality scores versus the original DMOS for both GB + JPEG and GB + WN mixed distortions. It can be seen that there is a linear correlation between the predicted scores of distorted images and the original DMOS values. Table 2 lists the evaluation scores of the proposed model for different distortion cases in this study. A better image quality evaluation model is to have a higher LCC and SROCC, and the method in this study achieves the desired results.Figure 7
GB + JPEG and GB + WN mixed distortion.Table 2
Comparison of LCC and SROCC with different distortions.
DistortionLCCSROCCJPEG0.8750.899WN0.8940.874GB0.8260.865JPEG + WN0.9030.901JPEG + GB0.9120.937WN + GB0.9050.912JPEG + WN + GB0.9430.956


## 4.1. Experimental Preparation

The dataset used in the training process of the network is the illustration pattern dataset collected by network retrieval. The dataset contains 1000 distorted images (from 20 reference images) with distortion types of JPEG compression (JPEG), white noise (WN), and Gaussian blur (GB). The database provides DMOS (differential mean opinion score) values for each image. The values range from 0 to 100, with higher DMOS values indicating lower image quality. The dataset is divided according to a ratio of training set:test set = 80%:20%. All comparison experiments in this study are based on Windows 10 CPU Intel Corei7 3.2 GHz with 64 GB of RAM and 1 Nvidia RTX2080Ti graphics card.Judging the performance of the image quality assessment algorithm requires experimental validation on a database, mainly by calculating the correlation between the objective quality scores of the assessment algorithm for distorted images and their corresponding subjective quality scores. If the correlation degree between them both is high, the algorithm performs better. The evaluation metrics used in this study are as follows:


### 4.1.1. Linear Correlation Coefficient

The linear correlation coefficient (LCC) aims to calculate the linear correlation between two variables, as shown in equation (6). xi denotes the quality score predicted by the algorithm for the ith image, yi denotes the subjective quality score corresponding to the ithith image in the database, x¯ is the average of the algorithm’s prediction scores x1,x2,x3,…xn for all test images, y¯ is the average of the subjective quality fraction y1,y2,y3,…yn of all test images in the database, and n represents the number of test images. The linear correlation coefficient, as an index to evaluate the accuracy of the algorithm, is mainly used to calculate the linear correlation between the quality fraction predicted by the algorithm and the quality fraction in the corresponding database:(6)LCC=∑i=1nxi−x¯yi−y¯∑i=1nxi−x¯2yi−y¯2.


### 4.1.2. Spearman’s Rank-Order Correlation Coefficient

Spearman’s rank-order correlation coefficient (SROCC) measures the closeness of the association between two variables, as shown in equation (7). rix is the ranking position of the ithith image in the sequence after xi has been arranged into a sequence by size and riy is the ranking position of the ithith image in the sequence after yi has been arranged into a sequence by size:(7)SROCC=6∑i−1nrix−riy2nn2−1.


### 4.1.3. Pearson Linear Correlation Coefficient (PLCC)

The operator of the PLCC index is shown in equation (8). In this equation, the mean value obtained after averaging the algorithm-predicted quality scores of all test images is expressed as x¯. The mean value obtained after averaging the subjective quality scores of all test images is expressed as y¯:(8)PLCC=∑i=1Nxi−x¯yi−y¯∑i=1Nxi−x¯2∑i=1Nyi−y¯2.


## 4.1.1. Linear Correlation Coefficient

The linear correlation coefficient (LCC) aims to calculate the linear correlation between two variables, as shown in equation (6). xi denotes the quality score predicted by the algorithm for the ith image, yi denotes the subjective quality score corresponding to the ithith image in the database, x¯ is the average of the algorithm’s prediction scores x1,x2,x3,…xn for all test images, y¯ is the average of the subjective quality fraction y1,y2,y3,…yn of all test images in the database, and n represents the number of test images. The linear correlation coefficient, as an index to evaluate the accuracy of the algorithm, is mainly used to calculate the linear correlation between the quality fraction predicted by the algorithm and the quality fraction in the corresponding database:(6)LCC=∑i=1nxi−x¯yi−y¯∑i=1nxi−x¯2yi−y¯2.


## 4.1.2. Spearman’s Rank-Order Correlation Coefficient

Spearman’s rank-order correlation coefficient (SROCC) measures the closeness of the association between two variables, as shown in equation (7). rix is the ranking position of the ithith image in the sequence after xi has been arranged into a sequence by size and riy is the ranking position of the ithith image in the sequence after yi has been arranged into a sequence by size:(7)SROCC=6∑i−1nrix−riy2nn2−1.


## 4.1.3. Pearson Linear Correlation Coefficient (PLCC)

The operator of the PLCC index is shown in equation (8). In this equation, the mean value obtained after averaging the algorithm-predicted quality scores of all test images is expressed as x¯. The mean value obtained after averaging the subjective quality scores of all test images is expressed as y¯:(8)PLCC=∑i=1Nxi−x¯yi−y¯∑i=1Nxi−x¯2∑i=1Nyi−y¯2.


## 4.2. Effect of the Number of Convolution Kernels on Model Performance

The evaluation performance of the algorithm was examined for different numbers of convolutional cores from 10 to 70 to determine the number of convolutional cores of the algorithm. The results are shown in Figure4. When the number of convolutional cores ranges from 10 to 40, the values of SROCC and PLCC show an increasing trend, but when the number of convolutional cores reaches 50, the values of both evaluation performance indicators of this algorithm decrease gradually. It can be seen that the number of convolutional kernels has a direct influence on the evaluation performance of this algorithm, and the number of convolutional kernels is selected as 40 for this algorithm.Figure 4
Effect of the number of convolution kernels on the model.


## 4.3. Effect of Sampling Step Size on Model Performance

The evaluation performance of the algorithm is also affected by the sampling step length selected when sampling the image chunks, so the evaluation performance of the algorithm is tested for 10 sampling steps in the range of 30 to 300. The obtained test results are shown in Figure5. The evaluation performance of this algorithm is the highest when the sampling step length is 90, and then, the evaluation performance of this algorithm decreases as the sampling step length continues to grow. The reason is that the image feature information that can be used to evaluate the image quality decreases with the increase of the sampling step, so the higher the step size the lower the evaluation performance of this algorithm. Therefore, the sampling step length of this algorithm is selected as 90.Figure 5
Effect of different sampling steps on model performance.


## 4.4. Evaluation Performance Test

With the selected number of convolution kernels and sampling steps, the algorithm proposed in this study is applied to evaluate the quality of digital illustration patterns in the dataset. In order to present the evaluation effect of this algorithm more clearly, three experimental images are presented as examples of these three illustration patterns. The quality of the three experimental images is demonstrated by DMOS scores, and the obtained subjective observations are compared with the actual evaluation results of each method to further test the evaluation effect of this algorithm. The three digital illustration patterns are shown in Figure6. The results of the quality evaluation scores of the three experimental images derived from the four algorithms are listed in Table 1. Combining Figure 6 and Table 1, it can be concluded that the visual quality of the three experimental images, from high to low, is defined once for images A, B, and C, respectively. The quality evaluation scores of BLISS [18] algorithm for the three images are close to each other. The quality evaluation scores of KangCNN [20] and DeepIQA [21] for images A and C are higher, while the evaluation score of image B is lower. The quality evaluation scores of the three experimental images by the algorithm in this study are similar to the visual subjective observation results from the highest to the lowest order. This indicates that the evaluation results obtained by this algorithm are more reasonable, with higher evaluation accuracy, and have higher practical application value.Figure 6
Distorted digital illustration pattern. (a) DMOS = 68.743. (b) DMOS = 53.271. (c) DMOS = 42.197.
(a)(b)(c)Table 1
Evaluation effects of different algorithms on distorted digital illustration patterns.
ModelImage AImage BImage CBLISS49.32847.21144.269KangCNN75.98969.40335.092DeepIQA77.39865.80437.925Proposed65.43850.10738.721Figure7 shows the predicted quality scores versus the original DMOS for both GB + JPEG and GB + WN mixed distortions. It can be seen that there is a linear correlation between the predicted scores of distorted images and the original DMOS values. Table 2 lists the evaluation scores of the proposed model for different distortion cases in this study. A better image quality evaluation model is to have a higher LCC and SROCC, and the method in this study achieves the desired results.Figure 7
GB + JPEG and GB + WN mixed distortion.Table 2
Comparison of LCC and SROCC with different distortions.
DistortionLCCSROCCJPEG0.8750.899WN0.8940.874GB0.8260.865JPEG + WN0.9030.901JPEG + GB0.9120.937WN + GB0.9050.912JPEG + WN + GB0.9430.956


## 5. Conclusions

With the increasingly widespread use of computer technology, traditional illustration is gradually transforming into digital illustration. This study mainly analyzes the characteristics and development trend of digital illustration and proposes a quality evaluation method of digital illustration based on the deep neural network to solve the problems of backward evaluation methods and poor accuracy. Specifically, this study proposes a multifeature fusion method for reference-free pattern evaluation, which fuses the information entropy of multiple chunked features in a single image and trains IQF-CNN networks using features and image subjective quality to obtain an objective quality assessment model. The experimental results on the digitized illustration pattern dataset show that this model can achieve high accuracy assessment of digitized illustration patterns, and the results obtained are more reasonable and reliable. In our future research program, we plan to use recurrent neural networks and knowledge graphs for online simulation studies of illustration patterns for digital art design.



---

*Source: 7152454-2022-06-27.xml*