# Study on Digital Painting Art Based on Computer Virtual Technology

Authors: Kan Guo
Journal: Mobile Information Systems
 (2022)

---

## Abstract

To improve the comprehensive level of artificial intelligence robot painting art creation, the software and hardware of the robot are designed from the perspective of robot copying painting. In this study, a more refined 6-axis robot is used to cooperate with the 2-axis slider connecting rod platform for performing painting operation. Its hardware uses a 6-axis robot system grafted on the 2-axis screw slider platform, and its software uses machine learning artificial intelligence algorithms. These algorithms include multi-layer spatial convolution neural network and multi-column fuzzy neural network. It is found that the subjective evaluation results and market transaction value of the robot copying paintings completed by the system are equivalent to the creative level of senior undergraduate and graduate students majoring in fine arts. It is far higher than the high-precision inkjet system that has completed market transformation. Finally, it is considered that the robot copying painting system has basically met the requirements of the robot art creation system and is separated from the technical category of the picture printer.


---

## Body

## 1. Introduction

In the relevant literature, there are two research directions under the requirements of computer virtual painting technology [1]. The machine learning algorithm of artificial intelligence is used to realize the function of aesthetic comparison. The functions of computer composition, color composition, character performance, and narration are realized in combination with exhaustive algorithm and fusion algorithm [2]. Use the artificial intelligence algorithm to copy the artworks (such as previous paintings and photos) that have completed the composition creativity and has focused on improving the pixel decomposition, stroke selection, and driving ability of the artificial intelligence to the painting hardware.The stroke control matrix is formed and is used to form the control instruction matrix of computer painting hardware equipment [2]. In the second stage of stroke control matrix processing, the stroke diameter, stroke color, stroke strength, and moving direction of the stroke will be assessed and converted into the action instruction of the painting hardware.Compared with the two, the latter is closer to the category of fine arts, while the former is closer to the category of aesthetics. The relevant research of the former is still in the stage of theoretical research and conceptual research, whereas the research of the latter has entered the stage of applied research [3]. This research is, therefore, based on later research direction, focusing on the driving algorithm of computer painting hardware.After importing the original image, the technological path of this research is divided into numerous branches for convolution recognition based on two-dimensional image dot matrix. The original image includes black-and-white binarization where the edge was found, the stroke diameter was calculated, and the stroke force point was decomposed. It also obtains the color result by combining the stroke matrix with the original image and then obtains the stroke strength by combining the color results with the original image.The virtual reality technology is introduced to deal with virtual space. In study [4], using VR technology, the photo sprayer and a virtual painting tool were used to create digital art in virtual technology. By using digital art in virtual technology, the photos of landscape and other objects can be transformed into colors.In [5], the primary technology of virtual reality technology is observed. The SketchUp virtual modelling software was applied to reveal the certain issues. The different phases of virtual reality technology-aided landscape design were also observed. The findings show that by using the computer virtual technology into the landscape design teaching in universities is helpful to enhance teaching efficiency. The SketchUp modelling can be used for virtual modelling of buildings.


## 2. Basic Algorithm Flow and Mathematical Principle

As mentioned above, the basic logic of the algorithm is divided into two processing stages. The algorithm must go through three parallel two-dimensional image convolution processes in the first stage of image processing, and the creation procedure and generation algorithm for the last two columns of convolution kernel also employ a fuzzy neural network. In the second stage of stroke control matrix processing, the stroke diameter, stroke color, stroke strength, and moving direction of the stroke are converted into the action instruction of the painting hardware [3]. The logic of the algorithm is shown in Figure 1.Figure 1
Logic architecture diagram of computer drawing driving algorithm.In Figure1, four convolution kernels are shared. The black and white convolution kernel and the diffusion edge convolution kernel are fixed convolution kernels, both of which have 3 × 3 structure [6]. The expression is(1)B=000010000;D=11110.21111,where B is the black white revolution kernel and D is the diffusion edge revolution kernel.Both the color extraction convolution kernel (C convolution kernel) and power extraction convolution kernel (P convolution kernel) are dynamic control convolution kernels. The stroke extraction module needs to determine the value of each node of the convolution kernel according to the image data matrix output from the convolution results of the preceding B and D convolution kernels.In each convolution process, the continuous space convolution method is used, as shown in the following formula:(2)F=∫−∞+∞Mt−xNx,where M is the original image sequence; N is the convolution kernel sequence; t is the control pointer which is used to control the convolution position of data; and x is the moving pointer variable.The first stage (above the dotted line in Figure1) involves three calculation modules as follows.


### 2.1. Design of Stroke Extraction Module

Stroke extraction module generates a color extraction convolution kernel through a multicolumn fuzzy neural network. The convolution kernel is a 3 × 3 structure, a total of 9 output requirements, and the input data is the spatial convolution results of the first two steps 2048 × 1080 monochrome images [7]. The data structure of each image is 2211840 16 bit integer data. The logical structure of the module is shown in Figure 2.Figure 2
Logical structure diagram of stroke extraction module.In Figure2, a total of 11 fuzzy convolution modules are used. Fuzzy convolution is to use a certain convolution rule to form a deep iterative regression relationship for bulk data and finally gives an output on a double precision variable containing the hiding law of bulk data. It includes two depth fuzzy convolution modules and nine multi-column fuzzy convolution modules [8]. The statistical importance of the depth fuzzy convolution module is that it convolutes bulk data into double precision data while preserving the data properties of bulk data. Therefore, the depth iterative regression algorithm based on polynomial is generally used to save the data characteristics into the variables to be regressed. Polynomial depth iterative regression basis function is shown in the following formula:(3)Y=∑i=1n∑j=15AjXij,where Y is the node output value; X is the input value provided by the i-th node in the upper layer neural network node; j is the polynomial order; Aj is the coefficient to be regressed of the j-th order polynomial; and n is the number of nodes of the previous neural network.In stroke extraction module, multi-column fuzzy neural network convolutes two double precision variables into one double precision variable, integrating and sorting out the data of two input image arrays according to different data training objectives. Its basis function uses logarithmic convolution function, as shown in the following formula:(4)Y=∑i=1nAlnXi+B,where A and B are coefficients to be regressed; other mathematical symbols have the same meaning as in formula (3).


### 2.2. Color Extraction Module and Power Extraction Module

The color extraction module and power extraction module have the same structure. Data training is carried out according to different training objectives to get different functions. After deep fuzzy convolution of image data, multi-column fuzzy neural network is used to process it into convolution kernel. The convolution kernel output is a 3 × 3 matrix which means it contains 9 output nodes [9]. The logic of the algorithm is shown in Figure 3.Figure 3
Logical structure diagram of color extraction module and power extraction module.The most significant distinction between Figures3 and 2 is that Figure 3 inputs the image array data in one group, while Figure 2 deals with incoming image array data. Both of them are entirely compatible with other logical nodes. The depth fuzzy neural network, for example, uses the polynomial depth iterative regression algorithm as in formula (3). The multi-column fuzzy neural network adopts logarithmic depth iterative regression algorithm, which is the same as formula (4). The difference is that in the multicolumn herdsman neural network, the two modules are one input layer node, while the two modules above are two input layer nodes.


## 2.1. Design of Stroke Extraction Module

Stroke extraction module generates a color extraction convolution kernel through a multicolumn fuzzy neural network. The convolution kernel is a 3 × 3 structure, a total of 9 output requirements, and the input data is the spatial convolution results of the first two steps 2048 × 1080 monochrome images [7]. The data structure of each image is 2211840 16 bit integer data. The logical structure of the module is shown in Figure 2.Figure 2
Logical structure diagram of stroke extraction module.In Figure2, a total of 11 fuzzy convolution modules are used. Fuzzy convolution is to use a certain convolution rule to form a deep iterative regression relationship for bulk data and finally gives an output on a double precision variable containing the hiding law of bulk data. It includes two depth fuzzy convolution modules and nine multi-column fuzzy convolution modules [8]. The statistical importance of the depth fuzzy convolution module is that it convolutes bulk data into double precision data while preserving the data properties of bulk data. Therefore, the depth iterative regression algorithm based on polynomial is generally used to save the data characteristics into the variables to be regressed. Polynomial depth iterative regression basis function is shown in the following formula:(3)Y=∑i=1n∑j=15AjXij,where Y is the node output value; X is the input value provided by the i-th node in the upper layer neural network node; j is the polynomial order; Aj is the coefficient to be regressed of the j-th order polynomial; and n is the number of nodes of the previous neural network.In stroke extraction module, multi-column fuzzy neural network convolutes two double precision variables into one double precision variable, integrating and sorting out the data of two input image arrays according to different data training objectives. Its basis function uses logarithmic convolution function, as shown in the following formula:(4)Y=∑i=1nAlnXi+B,where A and B are coefficients to be regressed; other mathematical symbols have the same meaning as in formula (3).


## 2.2. Color Extraction Module and Power Extraction Module

The color extraction module and power extraction module have the same structure. Data training is carried out according to different training objectives to get different functions. After deep fuzzy convolution of image data, multi-column fuzzy neural network is used to process it into convolution kernel. The convolution kernel output is a 3 × 3 matrix which means it contains 9 output nodes [9]. The logic of the algorithm is shown in Figure 3.Figure 3
Logical structure diagram of color extraction module and power extraction module.The most significant distinction between Figures3 and 2 is that Figure 3 inputs the image array data in one group, while Figure 2 deals with incoming image array data. Both of them are entirely compatible with other logical nodes. The depth fuzzy neural network, for example, uses the polynomial depth iterative regression algorithm as in formula (3). The multi-column fuzzy neural network adopts logarithmic depth iterative regression algorithm, which is the same as formula (4). The difference is that in the multicolumn herdsman neural network, the two modules are one input layer node, while the two modules above are two input layer nodes.


## 3. Driving Algorithm of Drawing Equipment

Robot plotter is a bionic robot system for painting techniques. Under the conventional layout, the six-axis robot system can automatically move in a large range under theX and Y coordinate system, holding the brush dip in the pigment and completing the conventional painting operation in the drawing paper area [5, 6]. The above system is shown in Figure 4.Figure 4
Structure diagram of art creation robot system.In Figure4, the robot is mainly composed of three parts: 2-axis positioning slide, 6-axis pen holding robot, and pigment control system. The design driving ideas of the three parts are as follows.


### 3.1. 2-Axis Positioning Slide

The positioning slide rail system simulates the action of arm shoulder joint and elbow joint in painting. Its main function is to drive the brush to reach a certain area of the drawing paper or take some pigment. The manual operation process of Chinese tubular brush, Western flat brush, Western umbrella brush, and other common brushes is stimulated by the system. The positioning slide provides and compensates for the six-axis robot’s contradiction between high-precision control and large-scale movement by fully simulating the manual operation process of the brush in larger range. The movement assistance of the high-precision six-axis robot is provided in this way.


### 3.2. 6-Axis Pen Holding Robot

The six-axis robot based on the stepping motor control system can provide extremely high-precision motion control. In this design, the high-precision control part is limited to the control range of the six-axis robot. The control radius of the six-axis robot is about 80∼320 mm, the control accuracy is ±0.5 mm, and the control angle is 150°. A sector angle of 150°, an inner circle radius of 80 mm, and a pen drop control area with outer circle radius of 320 mm are provided. The maximum paper amplitude controlled by the system is 1200∼2400 mm. The six-axis robot in this area is assisted by the two-axis slider track system to form a control accuracy of ±1.0 mm.


### 3.3. Pigment Control System

Pigment mixing and modulation skills are the most important color usage skills in the painting process. Water quantity, oil content, emulsification degree, and percentage of primary color pigments are all non-quantitative criteria. The color modulation in Western oil painting differs significantly from Chinese color ink painting. Western oil painting pursues an uneven color mixing mode when mixing two colors of pigments, and complete mixing is forbidden between pigments, but a semi-mixed state is retained [10]. Chinese color ink pursues to immerse different colors in different positions of the pen which requires auxiliary pigment mixing equipment to assist in painting. In the related research, there are more in-depth studies on the pen edge [11], stroke operation accuracy, and bionic degree, but there is less research on the pigment mixing system. In this study, the liquid 3D printing technology is used to form an ink disc [12]. The semi-mixed pigment state in the process of bionic painting is shown in Figure 5.Figure 5
Top view of segment control system structure.In Figure5, four working areas are arranged on the rotary table. Two of these areas are responsible for recycling the used ink disc or placing a new blank ink disc [13], one area is used for dipping the brush, and another area is a pigment nozzle for spraying or extruding the pigments with different pressures and viscosities into the ink disc. The paint nozzle is controlled by another six-axis robot. If the pigment control system is designed to meet the needs of different kinds of painting, such as color ink traditional Chinese painting, coke ink traditional Chinese painting, oil painting, gouache painting, watercolor painting, color lead painting, and so on [12, 14–16], the pigment nozzle of the pigment control system will be very complex. However, the control process of optional pigments for a work generally only needs one nozzle. Limited by the length, this study only examines the control scheme and does not go deep into the detailed design of mechanical equipment.


## 3.1. 2-Axis Positioning Slide

The positioning slide rail system simulates the action of arm shoulder joint and elbow joint in painting. Its main function is to drive the brush to reach a certain area of the drawing paper or take some pigment. The manual operation process of Chinese tubular brush, Western flat brush, Western umbrella brush, and other common brushes is stimulated by the system. The positioning slide provides and compensates for the six-axis robot’s contradiction between high-precision control and large-scale movement by fully simulating the manual operation process of the brush in larger range. The movement assistance of the high-precision six-axis robot is provided in this way.


## 3.2. 6-Axis Pen Holding Robot

The six-axis robot based on the stepping motor control system can provide extremely high-precision motion control. In this design, the high-precision control part is limited to the control range of the six-axis robot. The control radius of the six-axis robot is about 80∼320 mm, the control accuracy is ±0.5 mm, and the control angle is 150°. A sector angle of 150°, an inner circle radius of 80 mm, and a pen drop control area with outer circle radius of 320 mm are provided. The maximum paper amplitude controlled by the system is 1200∼2400 mm. The six-axis robot in this area is assisted by the two-axis slider track system to form a control accuracy of ±1.0 mm.


## 3.3. Pigment Control System

Pigment mixing and modulation skills are the most important color usage skills in the painting process. Water quantity, oil content, emulsification degree, and percentage of primary color pigments are all non-quantitative criteria. The color modulation in Western oil painting differs significantly from Chinese color ink painting. Western oil painting pursues an uneven color mixing mode when mixing two colors of pigments, and complete mixing is forbidden between pigments, but a semi-mixed state is retained [10]. Chinese color ink pursues to immerse different colors in different positions of the pen which requires auxiliary pigment mixing equipment to assist in painting. In the related research, there are more in-depth studies on the pen edge [11], stroke operation accuracy, and bionic degree, but there is less research on the pigment mixing system. In this study, the liquid 3D printing technology is used to form an ink disc [12]. The semi-mixed pigment state in the process of bionic painting is shown in Figure 5.Figure 5
Top view of segment control system structure.In Figure5, four working areas are arranged on the rotary table. Two of these areas are responsible for recycling the used ink disc or placing a new blank ink disc [13], one area is used for dipping the brush, and another area is a pigment nozzle for spraying or extruding the pigments with different pressures and viscosities into the ink disc. The paint nozzle is controlled by another six-axis robot. If the pigment control system is designed to meet the needs of different kinds of painting, such as color ink traditional Chinese painting, coke ink traditional Chinese painting, oil painting, gouache painting, watercolor painting, color lead painting, and so on [12, 14–16], the pigment nozzle of the pigment control system will be very complex. However, the control process of optional pigments for a work generally only needs one nozzle. Limited by the length, this study only examines the control scheme and does not go deep into the detailed design of mechanical equipment.


## 4. Evaluation of Actual Effect of Computer Painting

As artworks are emotional consumer commodities, and the quality assessment criteria for artworks is the audience’s perceptual evaluation result, it investigates that the actual physical quantities in robot painting are of no direct consequence. Furthermore, the research is not confined to the performance outcomes of a certain task or kind of labor [19], so the computer-aided painting system (robot painting system) designed by the research is used to draw many works of common painting types. Through social visit investigation and online auction, the painting forms such as manual creation and high-precision spray painting are compared to evaluate the practical application effect of the system.First, look at the subjective evaluation findings. The technique of evaluation is online voting. A total of 1000 works are included in the voting system, comprising 400 works drawn by the system, 400 high-precision inkjet works, and 200 hand drawn works. In these types of works, color ink painting and coke ink painting are examples of Chinese painting, whereas oil painting, watercolor painting, and gouache painting are examples of Western painting. The number of five categories of works is same. All works are scanned at 600 dpi to create electronic images, which are then shown in high definition in the voting system. Students from art colleges and universities voted. A total of 2000 students from 55 colleges and universities took part in voting. The voting results are computed using linear re-projection on the [0, 1] interval, which is the interval with the highest number of votes. Then, the lowest number of votes of this type of works is calculated before the difference between both is calculated. The difference between the actual voting results of each work and the lowest number of votes and similarly the difference ratio between the above-mentioned highest and lowest votes are taken as the statistical results. The mean value and standard deviation rate of the statistical results are calculated and included in Table1.Table 1
Distribution of voting results of artwork value.
Painting systemForm of expressionHand drawnInkjet printerRobot paintingChinese paintingColor ink traditional0.803 ± 0.2320.617 ± 0.0850.806 ± 0.093Coke ink traditional0.769 ± 0.2080.596 ± 0.0720.759 ± 0.084Western paintingOil painting0.734 ± 0.2140.531 ± 0.0690.742 ± 0.096Gouache0.835 ± 0.2460.626 ± 0.0730.826 ± 0.075Watercolor0.851 ± 0.2630.647 ± 0.0790.857 ± 0.074Table1 shows the following three data characteristics:(1) The mean value of subjective evaluation of hand drawn works and robot painting works is high, and the difference between them is about 5%. Each of this has its own advantages and disadvantages. However, the standard deviation of robot painting works is much smaller than that of hand drawn works. The evaluators’ evaluation opinions on robot painting works are relatively concentrated. From the perspective of fine arts, the artistic skill level of robot painting works is relatively stable.(2) The standard deviation results of inkjet printer works and robot painting works are relatively close. It can be, therefore, considered that both techniques are relatively stable; however, the average evaluation value of inkjet printer works is much lower than that of robot painting works and hand drawn works. Inkjet printer creation mode has been fully commercialized in the early computer painting and has become an important implementation form of current computer artworks. However, it is important to note that it is mostly concentrated in low-end art consumer goods. In other words, inkjet printer works are not separated from the technical category of printing pictures.(3) Although the creators of the hand drawn works participating in the evaluation do not include well-known contemporary painters at home and abroad, the creators of all works are senior undergraduate and graduate students majoring in art in colleges and universities and their creative ability should represent the level of grass-roots art creators. The mean value of subjective evaluation of robot painting works is basically the same as that of hand drawn works, and the standard deviation is much lower than that of hand drawn works. Therefore, the actual evaluation results of works should be split to analyze the distribution of works in different sections, as shown in Figure6.Figure 6
Distribution of work evaluation results.Although the subjective assessment results of manually drawn works are essentially the same as those of robot-painted works in Figure6, robot-painted works are more dominating in the distribution of evaluation results, as shown below:(1) The falling point peak value of robot painting works is in the range of 0.80∼0.85, which is higher than that of hand drawn works by 0.75∼0.80.(2) The falling point range of hand drawn works is wide which is distributed in the range of 0.55∼0.10, while the falling point range of robot painting works is narrow and is basically concentrated in the range of 0.65∼0.90. It can be therefore evaluated that although the robot painting works have far more landing points in high score areas than robot painting works due to its unstable technical play, there are many landing points in low score areas.To strengthen the above subjective evaluation results, in addition to the subjective evaluation, the online virtual auction link is designed to provide each participant with an equal amount of simulation currency [15]. It evaluates the total value of their works, performs linear re-projection calculation on the simulation auction price of each work, and then calculates the mean value and standard deviation of similar works (Table 2).Table 2
Distribution of auction transaction price of works.
Painting systemForm of expressionHand drawnInkjet printerRobot paintingChinese paintingColor ink traditional0.781 ± 0.2480.403 ± 0.0360.763 ± 0.114Coke ink traditional0.736 ± 0.2250.356 ± 0.0320.712 ± 0.120Western paintingOil painting0.715 ± 0.2360.487 ± 0.0270.694 ± 0.117Gouache0.827 ± 0.2610.514 ± 0.0290.803 ± 0.105Watercolor0.846 ± 0.2790.533 ± 0.0360.816 ± 0.119In Table2, the transaction price data and their comparison with Table 1 have the following characteristics:(1)
The average actual transaction price of robot painting works is lower than that of hand drawn works, and its standard deviation has been enlarged, but its standard deviation is still much lower than that of hand drawn works. Also, the average transaction price of inkjet printer works is much lower than the voting result of its work value. When compared with the voting evaluation results, the actual transaction value of robot painting works has decreased, but inkjet printer works have decreased significantly. It can be concluded that the actual exchange value of robot painting works is much greater than inkjet printer works and can share the market with hand drawn works.(2)
There are also differences between the transaction value and voting results of hand drawn works. Some types of works have increased, and some types of works have decreased. When compared with the other two works, there is a strong coupling between the transaction value and voting value of hand drawn works.


## 5. Conclusions

In this study, a more refined 6-axis robot is used to cooperate with the 2-axis slider connecting rod platform for performing painting operation. Spatial convolution neural network and multi-column fuzzy neural network are introduced to realize high-precision artificial intelligence control. Robot copying painting is realized. Although the system cannot realize the independent artistic creation of the robot under the existing technical system, the ability of the robot to copy painting is greatly improved. Combined with digital photography technology, the system can also realize robot sketching. According to the results of the social survey, robot painting works created with this technology have essentially achieved the same subjective evaluation results and market transaction value as paintings created by senior undergraduate and graduate students majoring in fine arts. This indicates that this technology has moved away from digital printing and is closer to robot art creation.



---

*Source: 5687835-2022-08-13.xml*