# Pose Estimation under Visual Sensing Technology and Its Application in Art Design

Authors: Bing Song; Meng Li; Liping Lou
Journal: Journal of Sensors
 (2021)

---

## Abstract

The study is aimed at solving the problem of large measurement errors caused by the binocular camera in traditional 3D art design, which leads to inaccurate 3D information of the target. The contour information extraction in the process of human motion pose reconstruction is easily affected by the noise in the image. Therefore, a binocular stereo vision system is built first and it integrates image acquisition, camera calibration, and image processing. The dedistortion method is used to process the image because it can reduce errors. Second, a three-dimensional human motion pose reconstruction model is implemented, the Gaussian template is used to remove the noise in the image frame, and the change detection template (CDM) is used to solve the problem of background “exposure” and “occlusion.” Finally, simulation experiments are designed to verify the system and model designed. Since the research on the application of pose estimation based on visual sensing technology in art design is still blank, such research has great significance and provides a reference for the research in the field. The literature analysis is used to expound and analyze the application of pose estimation based on visual sensing technology in visual communication design and environmental art design: (1) although the binocular stereo vision system causes some errors in the measurement, the overall error is controlled within 2% and the accuracy is high, which proves that it can be applied to the acquisition of three-dimensional information of the target in art design; (2) there is a high degree of fitting between the video sequence data created by the three-dimensional human motion pose reconstruction model designed and the real motion data, which indicates that this method has high accuracy in processing image sequences and the feasibility of applying it to human pose reconstruction in three-dimensional art design is high; (3) through the analysis of the existing literature, it is found that most of the current visual-based attitude assessment studies are carried out by using network cameras combined with computers, and the quality of the obtained images is low. The combination of binocular stereo sensor and attitude estimation technology can be applied to the design of advertising, animation, games, and packaging, making the behavior of virtual characters in animation and games more vivid. The combination provides convenience for the collection of environmental spatial information and object attitude information, the formulation of a design scheme, and real-time monitoring of construction in environmental art design. The purpose of this study is to provide an important theoretical basis for the technical upgrading of art design.


---

## Body

## 1. Introduction

Art belongs to the social superstructure, and it is an important part of people’s spiritual life after their needs of material life are met. If the economy is more prosperous, the higher living standards are and the greater demand for art is becoming [1]. The art design is a process in which artists express their inspiration, experience, and feelings through artworks and communicate with the public. The traditional art design process needs to go through tedious design steps and takes a lot of time, which cannot meet the practical needs of today’s society. In response to the problem, the research of art design combined with science and technology attracts more and more attention of people [2]. No matter what type of art design, it needs to be conveyed through vision, such as color, pattern, and text on clothing; the quality and 3D effect of animation; the composition of product packaging; and the shape and size of landscape [3]. With the development of society, people have higher and higher requirements for artistic products. The quality of 2D photos or video images taken by ordinary cameras is poor, and they are unable to meet practical needs. The equipment specially used for shooting ultra-high-quality films, television dramas, and animations are generally bulky, expensive, and not suitable for people to use in artistic design work.Visual sensor based on visual sensing technology has a series of advantages, such as small size, low price, and long life. It can draw the surface signal of the object after computer processing and present it in front of the researchers. For example, the most popular binocular stereo vision sensor at present is widely used in three-dimensional modeling, three-dimensional measurement, intelligent monitoring, and other research fields [4]. Visual sensors have great advantages in image processing compared with ordinary cameras, but there are also some shortcomings, such as being vulnerable to complex background and color, occlusion, and irregular movement of the target [5]. Pose estimation refers to the estimation of the position and attitude of the target to be tested through the detection and tracking of key points. Combined with deep learning, 3D pose estimation can be realized without the interference of background and color [6]. However, traditional human motion pose reconstruction methods are susceptible to image noise when contour information is extracted. After the literature is reviewed, it is found that the current research on pose assessment based on visual sensing technology mainly focuses on human pose assessment and UAV (Unmanned Aerial Vehicle) pose estimation, but there is little literature on its application in art design.Based on the above problems, a binocular stereo vision system and a 3D human motion pose reconstruction model are built first. Second, simulation experiments are designed to verify the system and model designed. Finally, the application of pose estimation based on visual sensing technology in visual communication design and environmental art design is analyzed. The purpose of this study is to provide an important theoretical basis for the technical upgrading of art design.


## 2. Materials and Methods

### 2.1. Analysis of Visual Sensing Technology

Vision is the most important feeling of human beings. Through vision, the size, color, and action of objects can be perceived to obtain information about the surrounding environment. However, human visual perception is vulnerable to emotional, physical, and light, and it has certain restrictions [7]. In recent years, with the development of science and technology, visual sensors gradually replace human beings and are used in various fields, solving the problems existing in the human visual. Here, product detection is taken as an example to compare the human vision and visual sensing technology, as shown in Table 1.Table 1
Comparison of human vision and visual sensing technology.
Human visionVisual sensing technologyAccuracyNeed magnifying glass or microscope assistance and the accuracy is lowAccuracy can reach one-thousandth of an inch without physical constraintsReproducibilityEasy to fatigue and has some subtle differences when testing identical products. The repeatability is poorUsing the same method and parameters to test the product. It will not fatigue and has good repeatabilitySpeedSlow detection speed, especially when the product has a certain speed displacementFaster and better detection of high-speed moving objects on production lines to improve production efficiencyObjectivityThe subjective judgment which is easily influenced by emotion when testing productsThe test results are objective and reliableCostFatigue, illness, or other subjective factors may lead to lower productivity and higher costsNo illness, no need to stop, high efficiency, production efficiency, cost savingsThe visual sensor is one of the fastest-growing and most widely used technologies in recent years. Its essence is image processing technology. It mainly uses optical devices and imaging devices to capture the image information of the external environment; that is, the image is drawn and presented in front of the researchers by intercepting the signal of the object surface [8]. The basic working principle of visual sensors is shown in Figure 1.Figure 1
Working principle of visual sensors.Visual sensing technology is mainly divided into two categories: 3D visual sensing technology and intelligent visual sensing technology. The objects seen by human eyes are hierarchical stereo images, and the shooting and display effects of 2D cannot meet the needs of people. Therefore, 3D stereo imaging becomes a research topic in recent years. Different applications such as multimedia mobile phones, robot visual navigation, automobile safety system, virtual reality, and monitoring, industrial detection are all based on 3D visual image sensor technology. The density of the reflected light on the surface of the object is different in general three-dimensional space. The three-dimensional image of the object can be obtained by detecting the density of the reflected light on the object. At present, 3D images can be obtained from solid image sensors, such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensors, to detect the density of the reflected light of the object. CCD is a detecting element that transmits signals by coupling mode. It has the functions of photoelectric conversion, information storage, and transmission. CCD has such advantages as self-scanning, wide sensing spectrum range, small distortion, small volume, low system noise, low power consumption, and high reliability [9]. CMOS is a kind of chip in the computer system, and it has a low cost. Although its imaging quality is slightly lower than CCD, CMOS has the advantages of small size, low power consumption, and low cost [10]. The structure of visual image sensors is shown in Figure 2.Figure 2
Structure of visual image sensors.Chmiel et al. applied visual sensing technology to the intelligent traffic control system. A 5 million resolution CCD camera is designed for real-time analysis of the traffic situation at various intersections of urban roads and calculation of the traffic flow of each lane and queue. It can provide real-time data for the signal control system to configure the dynamic signal parameters, and realize the intelligent single-point light signal control, trunk control, and regional control. This system can be widely used in intelligent signal lamp control and traffic information collection [11].Intelligent vision sensors under intelligent vision sensing technology, also known as intelligent cameras, are a new technology with the fastest development in the field of machine vision in recent years. The intelligent camera has the functions of image acquisition, image processing, and information transmission. It integrates image sensors, digital processors, communication modules, and other peripherals into a single camera. The integrated design makes it easy to learn, use, and maintain and has high reliability and stability, which greatly broadens the application field of visual sensing technology. The structure of the smart camera is shown in Figure3.Figure 3
Structure of smart cameras.


### 2.2. Construction of the Binocular Stereo Vision System

A complete binocular stereo vision system needs to include multiple functional modules, such as image acquisition, camera parameter calibration, image correction, and processing. Each module needs different algorithms to achieve its functions. Although a lot of previous studies have been carried out, there are also many new methods to improve the accuracy of the disparity map, solving the problems such as the obscured area and boundary between two given images.


#### 2.2.1. Geometric Principle and Depth Relationship of the Binocular Vision Sensor

Binocular stereo vision technology is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions. The three-dimensional geometric information of the object is obtained by calculating the position deviation between the corresponding points of the image. The principle is similar to the imaging principle of human vision. Two CCD/CMOS cameras with relatively fixed positions are used to obtain the image pairs of the measured object. The geometric model of the measured object can be established by extracting and matching the image feature points [12]. Based on the parallax principle and similar triangle principle, the three-dimensional information of the object is obtained through multiple images. The geometric principle of the binocular vision sensor is shown in Figure 4.Figure 4
Geometric principle of the binocular vision sensor.In Figure4, B is baseline distance, namely, the center distance of the left and right camera; Q1 and Q2 are the image point of a point Q in the space on the imaging plane of the left and right cameras; x1 and x2 are the distance between left and right image points and boundary of camera imaging plane; A is the distance from point Q to camera imaging plane; and f is focus on the camera.The distance between a point in the space and the camera imaging plane can be calculated according to the proportional relationship using the principle of a similar triangle. The parallaxd of point Q on the left and right cameras can be calculated by the following equation:
(1)d=x1−x2.The distance betweenQ1 and Q2 is set by y, and y can be obtained by the following equation:
(2)y=B−x1−x2.From the principle of the similar triangle,(3)B−x1−x2A−f=BA.The distance from pointQ to the imaging plane of the camera can be calculated by the following equation:
(4)A=fBx1−x2.Equation (4) shows that the distance from a point in space to the imaging plane of the camera is inversely proportional to the parallax of the point on the left and right cameras, as shown in Figure 5.Figure 5
Relationship between disparity and depth.


#### 2.2.2. Triangulation

In general, after the calibration of the binocular camera, the image is first processed by the method of dedistortion, and the nonlinear factors causing the subsequent experimental error are excluded. After processing, the linear equations are established by using the optical triangulation method to calculate the spatial coordinates. The imaging process of an object includes several coordinate transformations:(1)
The world coordinate system is transformed into the camera coordinate system, which can be described by a rotation matrix R and translation vector T:(5)xcyczc=Rxwywzw+T⟶xcyczc1=RT01xwywzw1In equation (5), xw,yw,zw is the world coordinate system and xc,yc,zc is the camera coordinate system.
(1)
The camera coordinate system is converted to the image coordinate system. This equation is obtained from the triangular proportional relationship:(6)xu=fxczc,yu=fyczcIn the above equation,fxc and fyc are the focus in the direction of xc and yc.The matrix is expressed as(7)zcxuyu1=f0000f000010xcyczc1.In the equation,xu,yu,zu is the image coordinate system.After the first-order radial distortion is considered, the distortion model is established:(8)xd=1+k1rd2xn,yd=1+k1rd2yn.In equation (8), xd,yd are the coordinates of a point on an image plane in a plane coordinate system. k1 and k2 are distortion parameters.
(9)rd2=xd2+yd2.The matrix is expressed as(10)xdyd1=1+k1rd2xuyu1.(1)
The image coordinate system is converted to the pixel coordinate systemIn the coordinate system in Figure6, the coordinate is u,v, and O1u0,v0 is the main point coordinate, so the following relationship is satisfied:
(11)u=xddx+u0,v=yddy+v0.Figure 6
The pixel coordinate system.In the equation,dx and dy are the physical sizes of pixels in the directions x and y. The pixel coordinate system is shown in Figure 6.The matrix is expressed as(12)uv1=1dx0u001dyv0001xdyd1.In summary, the relationship betweenxw−yw−zw coordinate system and the u−v coordinate system is obtained:
(13)zuv1=fx0u00fyv0001RTxwywzw1=KRTxwywzw1.K is the internal parameter matrix of the camera, which needs to be obtained in the calibration:
(14)K=fx0u00fyv0001.In equation (14), fx and fy are the focal length in the directions x and y.
(1)
Errors in camera calibrationDue to the radial distortion and tangential distortion of the camera, further correction is needed for the calibration results to reduce the error caused by distortion. Since the tangential distortion has little effect on the camera calibration results, the error caused by radial distortion is only considered here. Radial distortion is expressed as(15)x0=x1+k1r2+k2r4+k3r6,y0=y1+k1r2+k2r4+k3r6.In the above equation,x0,y0 are the point coordinates after distortion correction, k1, k2, and k3 are radial distortion parameters, and r is the distance between x,y and the imaging center.The coordinate relationship between the optical center and the imaging point is(16)x02+y02=r0,x2+y2=r,r0=k3r7+k2r5+k1r3+r.Constructive functions can be obtained:(17)fr=k3r7+k2r5+k1r3+r−r0.


### 2.3. Construction of 3D Human Motion Pose Reconstruction Model for Binocular Sensing Technology

3D human motion pose reconstruction is based on the human contour information in the video to obtain the initial three-dimensional information of human postdeformation. Then, the initial three-dimensional post is refined to obtain the accurate three-dimensional structure of human information corresponding to the video. Finally, the spatiotemporal model of the human motion database is reconstructed. The construction process of a 3D human motion pose based on the content of videos is shown in Figure7.
(1)
Image preprocessingFigure 7
3D human motion pose reconstruction.Due to the influence of the external environment, there are many noise points in the obtained videos and images, which will greatly reduce the accuracy of video object segmentation. Therefore, it is necessary to filter the image before segmentation to remove the influence of noise. Because the noise in the image frame is often cluttered and randomly distributed, the Gaussian template is often used to smooth the Gaussian noise. The mathematical expression of 2D Gaussian functionGx,y is shown in the following equation:
(18)Gx,y=12πσexp−x2+y22σ2.In equation (18), x,y represents the template coordinates of pixels. σ is the standard deviation of normal distribution.Gaussian smoothing is conducted by Gaussian template in practice:(19)ftx,y=W∗Itx,y.In the equation,W is the Gaussian smoothing window, Itx,y is the original image frame, and ftx,y is the smoothed image frame.
(2)
Human contour extraction in the static stateExtracting foreground contour from human motion video is an indispensable step in contour-based motion analysis technology. The video does not contain the motion information of the motion camera, so in the process of processing, the static background technology is used to segment the human motion, which is easier to process. The method based on Change Detection Mask (CDM) is used to solve the video frame difference. Through the pairwise difference of the continuous three-frame video sequence images, the difference results are binarized, and then, the “and” operations are performed. The shape contour of the moving target in the middle frame can be well detected, and the problem of background “exposure” and “occlusion” can be better solved.The three consecutive input frames are set asft−1x,y and ft+1x,y, respectively, and the absolute gray difference between the two adjacent source images is calculated:
(20)Dt−1,tx,y=ft−1x,y−ftx,y,Dt,t+1x,y=ft+1x,y−ftx,y.Two gray difference imagesDt−1,tx,y and Dt,t+1x,y are obtained. The difference image must be binarized first, and the appropriate threshold value is selected, which is a very important step. The approximate optimal threshold is obtained by histogram statistics:
(1)
For the obtained difference image, the gray histogram is calculated to obtain the minimum gray valueImin and the maximum gray value Imax, and then, the initial threshold T0 is taken:(21)T0=Imin+Imax2(2)
Tk is set as the threshold obtained after the kth iteration (k=0,1,2…) and used to get the gray value of the image which is initially divided into the two sets of A and B, and the average gray value of the two sets are calculated:(22)ZA=∑Ix,y<TkIx,y×Nx,y∑Ix,y<TkNx,y,ZB=∑Ix,y≥TkIx,y×Nx,y∑Ix,y≥TkNx,yIn the equation,Ix,y is the gray value of pixels. Nx,y is the weight value of pixel x,y, which is 1.
(3)
Take the average of the new thresholdsZA and ZB:(23)Tk+1=ZA+ZB2(4)
IfTk=Tk+1, the algorithm ends and returns the obtained threshold, otherwise it goes to step 2. The threshold T obtained by the above method are used to binarize Dt−1,tx,yandDt,t+1x,y to obtain two binary images Bt−1,tx,y and Bt,t+1x,y:(24)Bx,y=0,ifIx,y<T,1,ifIx,y≥TThen, the “and” operation is performed on each corresponding pixel position to obtain the final differential binary image.


### 2.4. Simulation Experiment Method

#### 2.4.1. Measurement Experiment of the Binocular Vision Sensor

In the measurement experiment of binocular vision sensors, a scene is randomly selected. First, several points with the same distance but different positions or different positions and distances are selected as the key points of measurement. Then, the left and right images of this scene are captured by the binocular camera. After the disparity map of the image is obtained by the binocular stereo vision system, the distance of the key points in the image is measured. The measurement results are compared with the actual distance, and the error is calculated to explore the accuracy of the measurement of the binocular vision sensor system designed. The hardware components of the system designed mainly include a computer and a binocular camera. And an economically applicable binocular camera is selected, and it can restore the real color of the object well and support multiple resolutions. This camera can be applied to 3D modeling, 3D ranging, and depth detection. Therefore, the camera can meet the basic requirements of this experiment. The hardware structure of the system is shown in Figure8.Figure 8
Hardware structure of the system.The experimental configuration and parameter settings are as follows:(1)
Hardware configurationIn the experiment, the binocular camera is fixed on the whiteboard, and its data line is connected with the computer USB interface. The laptop used in this experiment is a Macbook pro. The processor is 2.3 GHz, four cores are Intel Core i5, and the memory is 8 GB. The specific parameters of the camera used are shown in Table2.
(2)
Software development environmentsTable 2
Specific parameters of the camera.
CameraRER-1MP2CAM002-V90SensorsAE0330Pixel dimension5.07μm∗0.038μmOutput image formatMJPEGSignal-to-noise ratio39 dBFrame rate30 fpsResolution640∗480 or 320∗240Sensitivity2.0 V/lux-sec@550 nmFocal length60 mmColorColor imagesApertureHand operateWeight140 gLowest illumination0.1 luxPyCharm 2019.1 professional version, Matlab 2018a, OpenCV3.6, and Python are selected as the basic software development environment for experiments. In PyCharm, the classic OpenCV library is used to process images based on Python. The NumPy module also needs to be used in image processing.During the experiment, NumPy is repeatedly combined with SciPy and Matplotlib. In-network training, the image format of the dataset needs to be converted to a faster .npy format. When the parallax of the self-shot image is predicted, the image format should also be processed first.(3)
Human motion pose reconstruction experimentThe Poser software is used to synthesize the video data sequence. First, the Gaussian template is used to remove the noise of the videos and images. Then, the static background human contour extraction method designed is used to obtain the human contour information in the video and the initial three-dimensional information of the human pose deformation The accurate three-dimensional structure of human information corresponding to the video is obtained, and the three-dimensional human motion posture reconstruction is completed. The reconstruction results are quantitatively compared with the real motion data.


### 2.5. Analysis of Pose Estimation Based on Visual Sensing Technology

Pose estimation refers to the problem of determining the orientation of a three-dimensional target object, which has theoretical significance and application value in tracking, control, navigation, and positioning in military and civil fields [13]. With the development of information technology, pose estimation based on visual sensor technology has gradually entered into all aspects of people’s lives, and its importance has become increasingly prominent, attracting many scholars and research institutions in China and foreign countries to participate in this field [14]. In recent years, the research on pose estimation based on visual sensing technology mainly focuses on the pose estimation of humans and UAVs, which has broad application prospects.Human pose estimation is a basic problem in the field of computer vision research, and it is also a director of concern. The main functions are as follows: (1) action recognition: it is mainly used in human-computer interaction, intelligent monitoring, and sports medicine research; (2) animation and motion capture and augmented reality: they are mainly used in 3D movies, animation, and game production; (3) training of robots: it is mainly used in the development of intelligent robots [15].UAV is increasingly replacing the manned system to cope with dangerous, remote, or difficult situations for manned aircraft. Accurate pose estimation is the key requirement of the UAV autonomous driving system, especially for the rotary-wing aircraft during hovering. In recent years, using pose estimation based on visual sensing technology to control UAV is a very active research field [16]. UAV usually relies on IMU (Inertial Measurement Unit) and global positioning system to provide pose and velocity information. However, the low-cost IMU has the problems of sensor bias and drift, which is vulnerable to noise interference. The high-precision IMU is usually expensive and cumbersome, which limits its application in UAV [17]. Tehrani et al. proposed a new pose estimation method based on panoramic vision sensors to reduce the drift of the aircraft. A new camera system is designed, and it consists of a CCD camera, an ultraviolet filter, and a panoramic lens. The panoramic image for pose estimation is presented in the form of UV wavelength to enhance the contrast between the sky and the ground. The effectiveness of this method is verified by comparing the visual system with IMU [18]. Visual sensors have the advantages of being lightweight, low cost, and passiveness, and they can provide information about UAV motion and the surrounding environment. Therefore, the use of visual sensors can effectively compensate for the shortcomings of inertial sensors and better obtain the pose information of UAV.


### 2.6. The Influence of Technological Development on Art Design

The art design is the product of comprehensive psychological activities such as human knowledge, emotion, idea, and thinking. It is the process of transmitting a certain plan, idea, and problem-solving method through art visual language. Art design involves many fields of knowledge; it not only contains the performance of aesthetics but also includes logical thinking in philosophy. It is not only the process of artists’ creation but also the process of logical thinking. Everyone has different life experiences, knowledge connotations, and different understandings of things, so the products of art design are random and perceptual and cannot be formalized [19]. From the perspective of material environment and human behavior, art design is mainly divided into three categories: (1) design for communication-visual communication design; (2) design-product design for use; and (3) design for life–environmental art design. Specific classifications are shown in Figure 9.Figure 9
Classification of the art design.In the traditional way of art design, a series of complex processes such as obtaining inspiration, drawing sketches, fine creation, and coloring are needed to produce a work of art. The whole design process takes a lot of time. At present, with the development of science and technology, the application of modern science and technology in art design becomes a research topic in recent years. Liu et al. proposed a computer-based citrus peel art design to solve the problem of finding the best cutting line in citrus peel art. A designed input shape is mapped to the citrus, trying to cover the entire citrus, and the mapping boundary is used to generate the cutting path. Five customized interaction methods are developed to correct the input shape and make it suitable for citrus peel art. A large number of experiments have proved that the design and implementation method of citrus peel art based on the computer has good practicability [20]. Artificial intelligence technology can help designers get rid of tedious design steps in some ways, save design time, and improve efficiency.


## 2.1. Analysis of Visual Sensing Technology

Vision is the most important feeling of human beings. Through vision, the size, color, and action of objects can be perceived to obtain information about the surrounding environment. However, human visual perception is vulnerable to emotional, physical, and light, and it has certain restrictions [7]. In recent years, with the development of science and technology, visual sensors gradually replace human beings and are used in various fields, solving the problems existing in the human visual. Here, product detection is taken as an example to compare the human vision and visual sensing technology, as shown in Table 1.Table 1
Comparison of human vision and visual sensing technology.
Human visionVisual sensing technologyAccuracyNeed magnifying glass or microscope assistance and the accuracy is lowAccuracy can reach one-thousandth of an inch without physical constraintsReproducibilityEasy to fatigue and has some subtle differences when testing identical products. The repeatability is poorUsing the same method and parameters to test the product. It will not fatigue and has good repeatabilitySpeedSlow detection speed, especially when the product has a certain speed displacementFaster and better detection of high-speed moving objects on production lines to improve production efficiencyObjectivityThe subjective judgment which is easily influenced by emotion when testing productsThe test results are objective and reliableCostFatigue, illness, or other subjective factors may lead to lower productivity and higher costsNo illness, no need to stop, high efficiency, production efficiency, cost savingsThe visual sensor is one of the fastest-growing and most widely used technologies in recent years. Its essence is image processing technology. It mainly uses optical devices and imaging devices to capture the image information of the external environment; that is, the image is drawn and presented in front of the researchers by intercepting the signal of the object surface [8]. The basic working principle of visual sensors is shown in Figure 1.Figure 1
Working principle of visual sensors.Visual sensing technology is mainly divided into two categories: 3D visual sensing technology and intelligent visual sensing technology. The objects seen by human eyes are hierarchical stereo images, and the shooting and display effects of 2D cannot meet the needs of people. Therefore, 3D stereo imaging becomes a research topic in recent years. Different applications such as multimedia mobile phones, robot visual navigation, automobile safety system, virtual reality, and monitoring, industrial detection are all based on 3D visual image sensor technology. The density of the reflected light on the surface of the object is different in general three-dimensional space. The three-dimensional image of the object can be obtained by detecting the density of the reflected light on the object. At present, 3D images can be obtained from solid image sensors, such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensors, to detect the density of the reflected light of the object. CCD is a detecting element that transmits signals by coupling mode. It has the functions of photoelectric conversion, information storage, and transmission. CCD has such advantages as self-scanning, wide sensing spectrum range, small distortion, small volume, low system noise, low power consumption, and high reliability [9]. CMOS is a kind of chip in the computer system, and it has a low cost. Although its imaging quality is slightly lower than CCD, CMOS has the advantages of small size, low power consumption, and low cost [10]. The structure of visual image sensors is shown in Figure 2.Figure 2
Structure of visual image sensors.Chmiel et al. applied visual sensing technology to the intelligent traffic control system. A 5 million resolution CCD camera is designed for real-time analysis of the traffic situation at various intersections of urban roads and calculation of the traffic flow of each lane and queue. It can provide real-time data for the signal control system to configure the dynamic signal parameters, and realize the intelligent single-point light signal control, trunk control, and regional control. This system can be widely used in intelligent signal lamp control and traffic information collection [11].Intelligent vision sensors under intelligent vision sensing technology, also known as intelligent cameras, are a new technology with the fastest development in the field of machine vision in recent years. The intelligent camera has the functions of image acquisition, image processing, and information transmission. It integrates image sensors, digital processors, communication modules, and other peripherals into a single camera. The integrated design makes it easy to learn, use, and maintain and has high reliability and stability, which greatly broadens the application field of visual sensing technology. The structure of the smart camera is shown in Figure3.Figure 3
Structure of smart cameras.


## 2.2. Construction of the Binocular Stereo Vision System

A complete binocular stereo vision system needs to include multiple functional modules, such as image acquisition, camera parameter calibration, image correction, and processing. Each module needs different algorithms to achieve its functions. Although a lot of previous studies have been carried out, there are also many new methods to improve the accuracy of the disparity map, solving the problems such as the obscured area and boundary between two given images.


### 2.2.1. Geometric Principle and Depth Relationship of the Binocular Vision Sensor

Binocular stereo vision technology is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions. The three-dimensional geometric information of the object is obtained by calculating the position deviation between the corresponding points of the image. The principle is similar to the imaging principle of human vision. Two CCD/CMOS cameras with relatively fixed positions are used to obtain the image pairs of the measured object. The geometric model of the measured object can be established by extracting and matching the image feature points [12]. Based on the parallax principle and similar triangle principle, the three-dimensional information of the object is obtained through multiple images. The geometric principle of the binocular vision sensor is shown in Figure 4.Figure 4
Geometric principle of the binocular vision sensor.In Figure4, B is baseline distance, namely, the center distance of the left and right camera; Q1 and Q2 are the image point of a point Q in the space on the imaging plane of the left and right cameras; x1 and x2 are the distance between left and right image points and boundary of camera imaging plane; A is the distance from point Q to camera imaging plane; and f is focus on the camera.The distance between a point in the space and the camera imaging plane can be calculated according to the proportional relationship using the principle of a similar triangle. The parallaxd of point Q on the left and right cameras can be calculated by the following equation:
(1)d=x1−x2.The distance betweenQ1 and Q2 is set by y, and y can be obtained by the following equation:
(2)y=B−x1−x2.From the principle of the similar triangle,(3)B−x1−x2A−f=BA.The distance from pointQ to the imaging plane of the camera can be calculated by the following equation:
(4)A=fBx1−x2.Equation (4) shows that the distance from a point in space to the imaging plane of the camera is inversely proportional to the parallax of the point on the left and right cameras, as shown in Figure 5.Figure 5
Relationship between disparity and depth.


### 2.2.2. Triangulation

In general, after the calibration of the binocular camera, the image is first processed by the method of dedistortion, and the nonlinear factors causing the subsequent experimental error are excluded. After processing, the linear equations are established by using the optical triangulation method to calculate the spatial coordinates. The imaging process of an object includes several coordinate transformations:(1)
The world coordinate system is transformed into the camera coordinate system, which can be described by a rotation matrix R and translation vector T:(5)xcyczc=Rxwywzw+T⟶xcyczc1=RT01xwywzw1In equation (5), xw,yw,zw is the world coordinate system and xc,yc,zc is the camera coordinate system.
(1)
The camera coordinate system is converted to the image coordinate system. This equation is obtained from the triangular proportional relationship:(6)xu=fxczc,yu=fyczcIn the above equation,fxc and fyc are the focus in the direction of xc and yc.The matrix is expressed as(7)zcxuyu1=f0000f000010xcyczc1.In the equation,xu,yu,zu is the image coordinate system.After the first-order radial distortion is considered, the distortion model is established:(8)xd=1+k1rd2xn,yd=1+k1rd2yn.In equation (8), xd,yd are the coordinates of a point on an image plane in a plane coordinate system. k1 and k2 are distortion parameters.
(9)rd2=xd2+yd2.The matrix is expressed as(10)xdyd1=1+k1rd2xuyu1.(1)
The image coordinate system is converted to the pixel coordinate systemIn the coordinate system in Figure6, the coordinate is u,v, and O1u0,v0 is the main point coordinate, so the following relationship is satisfied:
(11)u=xddx+u0,v=yddy+v0.Figure 6
The pixel coordinate system.In the equation,dx and dy are the physical sizes of pixels in the directions x and y. The pixel coordinate system is shown in Figure 6.The matrix is expressed as(12)uv1=1dx0u001dyv0001xdyd1.In summary, the relationship betweenxw−yw−zw coordinate system and the u−v coordinate system is obtained:
(13)zuv1=fx0u00fyv0001RTxwywzw1=KRTxwywzw1.K is the internal parameter matrix of the camera, which needs to be obtained in the calibration:
(14)K=fx0u00fyv0001.In equation (14), fx and fy are the focal length in the directions x and y.
(1)
Errors in camera calibrationDue to the radial distortion and tangential distortion of the camera, further correction is needed for the calibration results to reduce the error caused by distortion. Since the tangential distortion has little effect on the camera calibration results, the error caused by radial distortion is only considered here. Radial distortion is expressed as(15)x0=x1+k1r2+k2r4+k3r6,y0=y1+k1r2+k2r4+k3r6.In the above equation,x0,y0 are the point coordinates after distortion correction, k1, k2, and k3 are radial distortion parameters, and r is the distance between x,y and the imaging center.The coordinate relationship between the optical center and the imaging point is(16)x02+y02=r0,x2+y2=r,r0=k3r7+k2r5+k1r3+r.Constructive functions can be obtained:(17)fr=k3r7+k2r5+k1r3+r−r0.


## 2.2.1. Geometric Principle and Depth Relationship of the Binocular Vision Sensor

Binocular stereo vision technology is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions. The three-dimensional geometric information of the object is obtained by calculating the position deviation between the corresponding points of the image. The principle is similar to the imaging principle of human vision. Two CCD/CMOS cameras with relatively fixed positions are used to obtain the image pairs of the measured object. The geometric model of the measured object can be established by extracting and matching the image feature points [12]. Based on the parallax principle and similar triangle principle, the three-dimensional information of the object is obtained through multiple images. The geometric principle of the binocular vision sensor is shown in Figure 4.Figure 4
Geometric principle of the binocular vision sensor.In Figure4, B is baseline distance, namely, the center distance of the left and right camera; Q1 and Q2 are the image point of a point Q in the space on the imaging plane of the left and right cameras; x1 and x2 are the distance between left and right image points and boundary of camera imaging plane; A is the distance from point Q to camera imaging plane; and f is focus on the camera.The distance between a point in the space and the camera imaging plane can be calculated according to the proportional relationship using the principle of a similar triangle. The parallaxd of point Q on the left and right cameras can be calculated by the following equation:
(1)d=x1−x2.The distance betweenQ1 and Q2 is set by y, and y can be obtained by the following equation:
(2)y=B−x1−x2.From the principle of the similar triangle,(3)B−x1−x2A−f=BA.The distance from pointQ to the imaging plane of the camera can be calculated by the following equation:
(4)A=fBx1−x2.Equation (4) shows that the distance from a point in space to the imaging plane of the camera is inversely proportional to the parallax of the point on the left and right cameras, as shown in Figure 5.Figure 5
Relationship between disparity and depth.


## 2.2.2. Triangulation

In general, after the calibration of the binocular camera, the image is first processed by the method of dedistortion, and the nonlinear factors causing the subsequent experimental error are excluded. After processing, the linear equations are established by using the optical triangulation method to calculate the spatial coordinates. The imaging process of an object includes several coordinate transformations:(1)
The world coordinate system is transformed into the camera coordinate system, which can be described by a rotation matrix R and translation vector T:(5)xcyczc=Rxwywzw+T⟶xcyczc1=RT01xwywzw1In equation (5), xw,yw,zw is the world coordinate system and xc,yc,zc is the camera coordinate system.
(1)
The camera coordinate system is converted to the image coordinate system. This equation is obtained from the triangular proportional relationship:(6)xu=fxczc,yu=fyczcIn the above equation,fxc and fyc are the focus in the direction of xc and yc.The matrix is expressed as(7)zcxuyu1=f0000f000010xcyczc1.In the equation,xu,yu,zu is the image coordinate system.After the first-order radial distortion is considered, the distortion model is established:(8)xd=1+k1rd2xn,yd=1+k1rd2yn.In equation (8), xd,yd are the coordinates of a point on an image plane in a plane coordinate system. k1 and k2 are distortion parameters.
(9)rd2=xd2+yd2.The matrix is expressed as(10)xdyd1=1+k1rd2xuyu1.(1)
The image coordinate system is converted to the pixel coordinate systemIn the coordinate system in Figure6, the coordinate is u,v, and O1u0,v0 is the main point coordinate, so the following relationship is satisfied:
(11)u=xddx+u0,v=yddy+v0.Figure 6
The pixel coordinate system.In the equation,dx and dy are the physical sizes of pixels in the directions x and y. The pixel coordinate system is shown in Figure 6.The matrix is expressed as(12)uv1=1dx0u001dyv0001xdyd1.In summary, the relationship betweenxw−yw−zw coordinate system and the u−v coordinate system is obtained:
(13)zuv1=fx0u00fyv0001RTxwywzw1=KRTxwywzw1.K is the internal parameter matrix of the camera, which needs to be obtained in the calibration:
(14)K=fx0u00fyv0001.In equation (14), fx and fy are the focal length in the directions x and y.
(1)
Errors in camera calibrationDue to the radial distortion and tangential distortion of the camera, further correction is needed for the calibration results to reduce the error caused by distortion. Since the tangential distortion has little effect on the camera calibration results, the error caused by radial distortion is only considered here. Radial distortion is expressed as(15)x0=x1+k1r2+k2r4+k3r6,y0=y1+k1r2+k2r4+k3r6.In the above equation,x0,y0 are the point coordinates after distortion correction, k1, k2, and k3 are radial distortion parameters, and r is the distance between x,y and the imaging center.The coordinate relationship between the optical center and the imaging point is(16)x02+y02=r0,x2+y2=r,r0=k3r7+k2r5+k1r3+r.Constructive functions can be obtained:(17)fr=k3r7+k2r5+k1r3+r−r0.


## 2.3. Construction of 3D Human Motion Pose Reconstruction Model for Binocular Sensing Technology

3D human motion pose reconstruction is based on the human contour information in the video to obtain the initial three-dimensional information of human postdeformation. Then, the initial three-dimensional post is refined to obtain the accurate three-dimensional structure of human information corresponding to the video. Finally, the spatiotemporal model of the human motion database is reconstructed. The construction process of a 3D human motion pose based on the content of videos is shown in Figure7.
(1)
Image preprocessingFigure 7
3D human motion pose reconstruction.Due to the influence of the external environment, there are many noise points in the obtained videos and images, which will greatly reduce the accuracy of video object segmentation. Therefore, it is necessary to filter the image before segmentation to remove the influence of noise. Because the noise in the image frame is often cluttered and randomly distributed, the Gaussian template is often used to smooth the Gaussian noise. The mathematical expression of 2D Gaussian functionGx,y is shown in the following equation:
(18)Gx,y=12πσexp−x2+y22σ2.In equation (18), x,y represents the template coordinates of pixels. σ is the standard deviation of normal distribution.Gaussian smoothing is conducted by Gaussian template in practice:(19)ftx,y=W∗Itx,y.In the equation,W is the Gaussian smoothing window, Itx,y is the original image frame, and ftx,y is the smoothed image frame.
(2)
Human contour extraction in the static stateExtracting foreground contour from human motion video is an indispensable step in contour-based motion analysis technology. The video does not contain the motion information of the motion camera, so in the process of processing, the static background technology is used to segment the human motion, which is easier to process. The method based on Change Detection Mask (CDM) is used to solve the video frame difference. Through the pairwise difference of the continuous three-frame video sequence images, the difference results are binarized, and then, the “and” operations are performed. The shape contour of the moving target in the middle frame can be well detected, and the problem of background “exposure” and “occlusion” can be better solved.The three consecutive input frames are set asft−1x,y and ft+1x,y, respectively, and the absolute gray difference between the two adjacent source images is calculated:
(20)Dt−1,tx,y=ft−1x,y−ftx,y,Dt,t+1x,y=ft+1x,y−ftx,y.Two gray difference imagesDt−1,tx,y and Dt,t+1x,y are obtained. The difference image must be binarized first, and the appropriate threshold value is selected, which is a very important step. The approximate optimal threshold is obtained by histogram statistics:
(1)
For the obtained difference image, the gray histogram is calculated to obtain the minimum gray valueImin and the maximum gray value Imax, and then, the initial threshold T0 is taken:(21)T0=Imin+Imax2(2)
Tk is set as the threshold obtained after the kth iteration (k=0,1,2…) and used to get the gray value of the image which is initially divided into the two sets of A and B, and the average gray value of the two sets are calculated:(22)ZA=∑Ix,y<TkIx,y×Nx,y∑Ix,y<TkNx,y,ZB=∑Ix,y≥TkIx,y×Nx,y∑Ix,y≥TkNx,yIn the equation,Ix,y is the gray value of pixels. Nx,y is the weight value of pixel x,y, which is 1.
(3)
Take the average of the new thresholdsZA and ZB:(23)Tk+1=ZA+ZB2(4)
IfTk=Tk+1, the algorithm ends and returns the obtained threshold, otherwise it goes to step 2. The threshold T obtained by the above method are used to binarize Dt−1,tx,yandDt,t+1x,y to obtain two binary images Bt−1,tx,y and Bt,t+1x,y:(24)Bx,y=0,ifIx,y<T,1,ifIx,y≥TThen, the “and” operation is performed on each corresponding pixel position to obtain the final differential binary image.


## 2.4. Simulation Experiment Method

### 2.4.1. Measurement Experiment of the Binocular Vision Sensor

In the measurement experiment of binocular vision sensors, a scene is randomly selected. First, several points with the same distance but different positions or different positions and distances are selected as the key points of measurement. Then, the left and right images of this scene are captured by the binocular camera. After the disparity map of the image is obtained by the binocular stereo vision system, the distance of the key points in the image is measured. The measurement results are compared with the actual distance, and the error is calculated to explore the accuracy of the measurement of the binocular vision sensor system designed. The hardware components of the system designed mainly include a computer and a binocular camera. And an economically applicable binocular camera is selected, and it can restore the real color of the object well and support multiple resolutions. This camera can be applied to 3D modeling, 3D ranging, and depth detection. Therefore, the camera can meet the basic requirements of this experiment. The hardware structure of the system is shown in Figure8.Figure 8
Hardware structure of the system.The experimental configuration and parameter settings are as follows:(1)
Hardware configurationIn the experiment, the binocular camera is fixed on the whiteboard, and its data line is connected with the computer USB interface. The laptop used in this experiment is a Macbook pro. The processor is 2.3 GHz, four cores are Intel Core i5, and the memory is 8 GB. The specific parameters of the camera used are shown in Table2.
(2)
Software development environmentsTable 2
Specific parameters of the camera.
CameraRER-1MP2CAM002-V90SensorsAE0330Pixel dimension5.07μm∗0.038μmOutput image formatMJPEGSignal-to-noise ratio39 dBFrame rate30 fpsResolution640∗480 or 320∗240Sensitivity2.0 V/lux-sec@550 nmFocal length60 mmColorColor imagesApertureHand operateWeight140 gLowest illumination0.1 luxPyCharm 2019.1 professional version, Matlab 2018a, OpenCV3.6, and Python are selected as the basic software development environment for experiments. In PyCharm, the classic OpenCV library is used to process images based on Python. The NumPy module also needs to be used in image processing.During the experiment, NumPy is repeatedly combined with SciPy and Matplotlib. In-network training, the image format of the dataset needs to be converted to a faster .npy format. When the parallax of the self-shot image is predicted, the image format should also be processed first.(3)
Human motion pose reconstruction experimentThe Poser software is used to synthesize the video data sequence. First, the Gaussian template is used to remove the noise of the videos and images. Then, the static background human contour extraction method designed is used to obtain the human contour information in the video and the initial three-dimensional information of the human pose deformation The accurate three-dimensional structure of human information corresponding to the video is obtained, and the three-dimensional human motion posture reconstruction is completed. The reconstruction results are quantitatively compared with the real motion data.


## 2.4.1. Measurement Experiment of the Binocular Vision Sensor

In the measurement experiment of binocular vision sensors, a scene is randomly selected. First, several points with the same distance but different positions or different positions and distances are selected as the key points of measurement. Then, the left and right images of this scene are captured by the binocular camera. After the disparity map of the image is obtained by the binocular stereo vision system, the distance of the key points in the image is measured. The measurement results are compared with the actual distance, and the error is calculated to explore the accuracy of the measurement of the binocular vision sensor system designed. The hardware components of the system designed mainly include a computer and a binocular camera. And an economically applicable binocular camera is selected, and it can restore the real color of the object well and support multiple resolutions. This camera can be applied to 3D modeling, 3D ranging, and depth detection. Therefore, the camera can meet the basic requirements of this experiment. The hardware structure of the system is shown in Figure8.Figure 8
Hardware structure of the system.The experimental configuration and parameter settings are as follows:(1)
Hardware configurationIn the experiment, the binocular camera is fixed on the whiteboard, and its data line is connected with the computer USB interface. The laptop used in this experiment is a Macbook pro. The processor is 2.3 GHz, four cores are Intel Core i5, and the memory is 8 GB. The specific parameters of the camera used are shown in Table2.
(2)
Software development environmentsTable 2
Specific parameters of the camera.
CameraRER-1MP2CAM002-V90SensorsAE0330Pixel dimension5.07μm∗0.038μmOutput image formatMJPEGSignal-to-noise ratio39 dBFrame rate30 fpsResolution640∗480 or 320∗240Sensitivity2.0 V/lux-sec@550 nmFocal length60 mmColorColor imagesApertureHand operateWeight140 gLowest illumination0.1 luxPyCharm 2019.1 professional version, Matlab 2018a, OpenCV3.6, and Python are selected as the basic software development environment for experiments. In PyCharm, the classic OpenCV library is used to process images based on Python. The NumPy module also needs to be used in image processing.During the experiment, NumPy is repeatedly combined with SciPy and Matplotlib. In-network training, the image format of the dataset needs to be converted to a faster .npy format. When the parallax of the self-shot image is predicted, the image format should also be processed first.(3)
Human motion pose reconstruction experimentThe Poser software is used to synthesize the video data sequence. First, the Gaussian template is used to remove the noise of the videos and images. Then, the static background human contour extraction method designed is used to obtain the human contour information in the video and the initial three-dimensional information of the human pose deformation The accurate three-dimensional structure of human information corresponding to the video is obtained, and the three-dimensional human motion posture reconstruction is completed. The reconstruction results are quantitatively compared with the real motion data.


## 2.5. Analysis of Pose Estimation Based on Visual Sensing Technology

Pose estimation refers to the problem of determining the orientation of a three-dimensional target object, which has theoretical significance and application value in tracking, control, navigation, and positioning in military and civil fields [13]. With the development of information technology, pose estimation based on visual sensor technology has gradually entered into all aspects of people’s lives, and its importance has become increasingly prominent, attracting many scholars and research institutions in China and foreign countries to participate in this field [14]. In recent years, the research on pose estimation based on visual sensing technology mainly focuses on the pose estimation of humans and UAVs, which has broad application prospects.Human pose estimation is a basic problem in the field of computer vision research, and it is also a director of concern. The main functions are as follows: (1) action recognition: it is mainly used in human-computer interaction, intelligent monitoring, and sports medicine research; (2) animation and motion capture and augmented reality: they are mainly used in 3D movies, animation, and game production; (3) training of robots: it is mainly used in the development of intelligent robots [15].UAV is increasingly replacing the manned system to cope with dangerous, remote, or difficult situations for manned aircraft. Accurate pose estimation is the key requirement of the UAV autonomous driving system, especially for the rotary-wing aircraft during hovering. In recent years, using pose estimation based on visual sensing technology to control UAV is a very active research field [16]. UAV usually relies on IMU (Inertial Measurement Unit) and global positioning system to provide pose and velocity information. However, the low-cost IMU has the problems of sensor bias and drift, which is vulnerable to noise interference. The high-precision IMU is usually expensive and cumbersome, which limits its application in UAV [17]. Tehrani et al. proposed a new pose estimation method based on panoramic vision sensors to reduce the drift of the aircraft. A new camera system is designed, and it consists of a CCD camera, an ultraviolet filter, and a panoramic lens. The panoramic image for pose estimation is presented in the form of UV wavelength to enhance the contrast between the sky and the ground. The effectiveness of this method is verified by comparing the visual system with IMU [18]. Visual sensors have the advantages of being lightweight, low cost, and passiveness, and they can provide information about UAV motion and the surrounding environment. Therefore, the use of visual sensors can effectively compensate for the shortcomings of inertial sensors and better obtain the pose information of UAV.


## 2.6. The Influence of Technological Development on Art Design

The art design is the product of comprehensive psychological activities such as human knowledge, emotion, idea, and thinking. It is the process of transmitting a certain plan, idea, and problem-solving method through art visual language. Art design involves many fields of knowledge; it not only contains the performance of aesthetics but also includes logical thinking in philosophy. It is not only the process of artists’ creation but also the process of logical thinking. Everyone has different life experiences, knowledge connotations, and different understandings of things, so the products of art design are random and perceptual and cannot be formalized [19]. From the perspective of material environment and human behavior, art design is mainly divided into three categories: (1) design for communication-visual communication design; (2) design-product design for use; and (3) design for life–environmental art design. Specific classifications are shown in Figure 9.Figure 9
Classification of the art design.In the traditional way of art design, a series of complex processes such as obtaining inspiration, drawing sketches, fine creation, and coloring are needed to produce a work of art. The whole design process takes a lot of time. At present, with the development of science and technology, the application of modern science and technology in art design becomes a research topic in recent years. Liu et al. proposed a computer-based citrus peel art design to solve the problem of finding the best cutting line in citrus peel art. A designed input shape is mapped to the citrus, trying to cover the entire citrus, and the mapping boundary is used to generate the cutting path. Five customized interaction methods are developed to correct the input shape and make it suitable for citrus peel art. A large number of experiments have proved that the design and implementation method of citrus peel art based on the computer has good practicability [20]. Artificial intelligence technology can help designers get rid of tedious design steps in some ways, save design time, and improve efficiency.


## 3. Results

### 3.1. Scene Measurement Results

In the experiment, five representative points are selected from the scene disparity map for distance measurement, as shown in Figure10.Figure 10
Scene disparity map.In Figure10, the detailed texture of the object is clear. The selected five points are located in different directions and at different distances from the lens. The measurement results are shown in Figure 11.Figure 11
Scene measurement results.The above figure shows that the error is small when shooting at points 1-4 near the measurement distance, and the error is controlled within 0.6%, which has high accuracy. When point 5 is measured far away, the measurement error reaches 1.67%, which increases. The reason may be that when objects are shot far away, the deviation of measurement results is largely due to the decrease of illumination conditions and clarity. However, in general, the measurement error is also controlled within 2%, which has high accuracy and is feasible.


### 3.2. 3D Human Motion Pose Reconstruction Based on Video Content

The quantitative comparison between reconstruction results and real motion data is shown in Figure12.Figure 12
Comparison of motion data.The above figure shows that the three-dimensional human motion pose reconstruction method designed in this study has a high fitting degree between the data created by the video sequence and the real motion data. This shows that this method has high accuracy in dealing with video sequences, and the technology used in this study is based on the general processing framework, which can be used to refine various types of objects with known initial 3D information.


### 3.3. Application Analysis of Pose Estimation Based on Visual Sensing Technology in Visual Communication Design

Art design involves a wide range of areas, but all types of artworks need to be achieved through basic elements such as color and graphics, and this must be conveyed through vision [21]. Therefore, in art design, the vision has the advantages and status that the other four senses do not have. Visual communication design is an important part of the modern art design. It refers to the design of information communication using visual images. It is the formal rule of exploring and explaining the function, purpose, and aesthetic feeling of the art design. The three elements are text, image, and color [22]. Visual communication designs most close to people’s lives are advertising design, packaging design, animation, and game design, and the specific classification is shown in Figure 13.Figure 13
Classification of visual communication design.With the progress and development of the times, people’s requirements for the picture and image quality of advertising, film, and television works are gradually increasing. However, the spectral range of ordinary cameras is only suitable for human vision. And after compression, the image quality is poor, which is not conducive to analysis and processing, and the dimension stays at the 2D level, which cannot meet the needs of modern society [23]. In recent years, 3D technology has been widely used in people’s production and life. The market size of 3D cameras increases year by year between 2015 and 2021, as shown in Figure 14.Figure 14
Market size of 3D cameras from 2015 to 2021.The 3D technology and sensors are combined recently, like binocular stereo vision sensors which are widely studied at present. Wei et al. proposed a 3D human motion capture method based on MobilePose, which is a supervised learning method for real-time detection of 2D bone joints. Due to the short time of two-dimensional joint detection and three-dimensional reconstruction, this method can realize the real-time acquisition of three-dimensional human motion. A simple human animation application program is made using the captured 3D motion information [24]. The three-dimensional space information of the target is obtained by key point detection, and the three-dimensional model is established by combining visual sensor with binocular stereo vision technology, which can be applied to the production and packaging of 3D advertising, animation, and games for better visual effects. After the two-dimensional joints are taken as the key point to estimate human posture in real time, the obtained three-dimensional position information of human joints is imported into the computer, and they should correspond to the joints of virtual characters in animation and game production. The synchronization of the two actions can make the behavior of virtual characters more vivid. Moreover, based on the characteristics of lightweight and low-cost visual sensors, it is more convenient and practical and has broad application prospects compared with professional shooting equipment.


### 3.4. The Application Analysis of Pose Estimation Based on Visual Sensing Technology in Environmental Art Design

With the improvement of people’s living standards, people have higher and higher requirements for environmental quality. The concept and practice of environmental art have gradually risen and developed, becoming an extremely important scientific, cultural, and artistic achievement in China in recent years [25]. The environmental art design is to use artistic means to design human living space, coordinate the relationship between “people-building-environment,” provide comfortable and beautiful space for human beings, and meet the needs of production and life in people’s daily life [26]. The elements of environmental art design are shown in Figure 15.Figure 15
Elements of environmental art design.In most of the environmental art design work, the acquisition of environmental spatial information is mainly through visual observation, manual mapping, and aerial calculation, which require a large number of human and material resources, and is also affected by factors such as weather or light. The use of double/multivisual stereo vision sensors can quickly and accurately collect environmental spatial information and object posture information and finally present the image in two or three-dimensional form, which is convenient for designers to formulate design schemes according to accurate information. In the process of landscape and building construction, there may be some uncertainty in the construction due to the technical differences between designers and builders [27]. For example, most artificial buildings should be built vertically, but the human eye and ordinary tools are difficult to judge some subtle differences.Ozaki and Kuroda proposed a landscape attitude estimation method based on EKF (Extended Kalman Filter). This method uses DNN (deep neural networks) to learn terrain information and outputs the camera image in the form of the average vector and covariance matrix of gravity, which can predict the gravity vector from the single-shot image. The experimental results show that the method can predict the gravity vector from a single-shot image and obtain the attitude information of the landscape. At the same time, the use of the simulator breaks the limit of real data acquisition on the ground [28]. The EKF architecture is shown in Figure 16.Figure 16
EKF architecture.Based on the inspiration of the above method, visual sensing technology (like binocular stereo vision sensor) is combined with deep learning. The target image information is obtained by a binocular stereo vision sensor, and the three-dimensional model is established. Combined with the terrain information of deep neural network learning, the gravity vector is predicted, which can estimate the target attitude more accurately and comprehensively. This method can be applied to the monitoring of artificial landscape and building construction in environmental art design work, which ensures the safety and makes the artistic design achievements of designers perfectly presented.In short, the combination of the binocular stereo vision sensor and attitude estimation technology can accurately obtain the three-dimensional information of the human body or the target and construct the three-dimensional model. It can also estimate the real-time target pose through the detection of the key points of the target, and it can be applied to advertising, animation, game production, and packaging design to make it have better visual effects and make the virtual characters more vivid. Its application in environmental art design can provide technical support for the collection of environmental spatial information and object pose information, the design scheme of designers, and the real-time monitoring of scenes.


## 3.1. Scene Measurement Results

In the experiment, five representative points are selected from the scene disparity map for distance measurement, as shown in Figure10.Figure 10
Scene disparity map.In Figure10, the detailed texture of the object is clear. The selected five points are located in different directions and at different distances from the lens. The measurement results are shown in Figure 11.Figure 11
Scene measurement results.The above figure shows that the error is small when shooting at points 1-4 near the measurement distance, and the error is controlled within 0.6%, which has high accuracy. When point 5 is measured far away, the measurement error reaches 1.67%, which increases. The reason may be that when objects are shot far away, the deviation of measurement results is largely due to the decrease of illumination conditions and clarity. However, in general, the measurement error is also controlled within 2%, which has high accuracy and is feasible.


## 3.2. 3D Human Motion Pose Reconstruction Based on Video Content

The quantitative comparison between reconstruction results and real motion data is shown in Figure12.Figure 12
Comparison of motion data.The above figure shows that the three-dimensional human motion pose reconstruction method designed in this study has a high fitting degree between the data created by the video sequence and the real motion data. This shows that this method has high accuracy in dealing with video sequences, and the technology used in this study is based on the general processing framework, which can be used to refine various types of objects with known initial 3D information.


## 3.3. Application Analysis of Pose Estimation Based on Visual Sensing Technology in Visual Communication Design

Art design involves a wide range of areas, but all types of artworks need to be achieved through basic elements such as color and graphics, and this must be conveyed through vision [21]. Therefore, in art design, the vision has the advantages and status that the other four senses do not have. Visual communication design is an important part of the modern art design. It refers to the design of information communication using visual images. It is the formal rule of exploring and explaining the function, purpose, and aesthetic feeling of the art design. The three elements are text, image, and color [22]. Visual communication designs most close to people’s lives are advertising design, packaging design, animation, and game design, and the specific classification is shown in Figure 13.Figure 13
Classification of visual communication design.With the progress and development of the times, people’s requirements for the picture and image quality of advertising, film, and television works are gradually increasing. However, the spectral range of ordinary cameras is only suitable for human vision. And after compression, the image quality is poor, which is not conducive to analysis and processing, and the dimension stays at the 2D level, which cannot meet the needs of modern society [23]. In recent years, 3D technology has been widely used in people’s production and life. The market size of 3D cameras increases year by year between 2015 and 2021, as shown in Figure 14.Figure 14
Market size of 3D cameras from 2015 to 2021.The 3D technology and sensors are combined recently, like binocular stereo vision sensors which are widely studied at present. Wei et al. proposed a 3D human motion capture method based on MobilePose, which is a supervised learning method for real-time detection of 2D bone joints. Due to the short time of two-dimensional joint detection and three-dimensional reconstruction, this method can realize the real-time acquisition of three-dimensional human motion. A simple human animation application program is made using the captured 3D motion information [24]. The three-dimensional space information of the target is obtained by key point detection, and the three-dimensional model is established by combining visual sensor with binocular stereo vision technology, which can be applied to the production and packaging of 3D advertising, animation, and games for better visual effects. After the two-dimensional joints are taken as the key point to estimate human posture in real time, the obtained three-dimensional position information of human joints is imported into the computer, and they should correspond to the joints of virtual characters in animation and game production. The synchronization of the two actions can make the behavior of virtual characters more vivid. Moreover, based on the characteristics of lightweight and low-cost visual sensors, it is more convenient and practical and has broad application prospects compared with professional shooting equipment.


## 3.4. The Application Analysis of Pose Estimation Based on Visual Sensing Technology in Environmental Art Design

With the improvement of people’s living standards, people have higher and higher requirements for environmental quality. The concept and practice of environmental art have gradually risen and developed, becoming an extremely important scientific, cultural, and artistic achievement in China in recent years [25]. The environmental art design is to use artistic means to design human living space, coordinate the relationship between “people-building-environment,” provide comfortable and beautiful space for human beings, and meet the needs of production and life in people’s daily life [26]. The elements of environmental art design are shown in Figure 15.Figure 15
Elements of environmental art design.In most of the environmental art design work, the acquisition of environmental spatial information is mainly through visual observation, manual mapping, and aerial calculation, which require a large number of human and material resources, and is also affected by factors such as weather or light. The use of double/multivisual stereo vision sensors can quickly and accurately collect environmental spatial information and object posture information and finally present the image in two or three-dimensional form, which is convenient for designers to formulate design schemes according to accurate information. In the process of landscape and building construction, there may be some uncertainty in the construction due to the technical differences between designers and builders [27]. For example, most artificial buildings should be built vertically, but the human eye and ordinary tools are difficult to judge some subtle differences.Ozaki and Kuroda proposed a landscape attitude estimation method based on EKF (Extended Kalman Filter). This method uses DNN (deep neural networks) to learn terrain information and outputs the camera image in the form of the average vector and covariance matrix of gravity, which can predict the gravity vector from the single-shot image. The experimental results show that the method can predict the gravity vector from a single-shot image and obtain the attitude information of the landscape. At the same time, the use of the simulator breaks the limit of real data acquisition on the ground [28]. The EKF architecture is shown in Figure 16.Figure 16
EKF architecture.Based on the inspiration of the above method, visual sensing technology (like binocular stereo vision sensor) is combined with deep learning. The target image information is obtained by a binocular stereo vision sensor, and the three-dimensional model is established. Combined with the terrain information of deep neural network learning, the gravity vector is predicted, which can estimate the target attitude more accurately and comprehensively. This method can be applied to the monitoring of artificial landscape and building construction in environmental art design work, which ensures the safety and makes the artistic design achievements of designers perfectly presented.In short, the combination of the binocular stereo vision sensor and attitude estimation technology can accurately obtain the three-dimensional information of the human body or the target and construct the three-dimensional model. It can also estimate the real-time target pose through the detection of the key points of the target, and it can be applied to advertising, animation, game production, and packaging design to make it have better visual effects and make the virtual characters more vivid. Its application in environmental art design can provide technical support for the collection of environmental spatial information and object pose information, the design scheme of designers, and the real-time monitoring of scenes.


## 4. Conclusions

In recent years, with the rapid development of three-dimensional technology, three-dimensional art design attracts widespread attention. At present, the binocular stereo vision sensor and attitude estimation technology are widely used in this field. The former has large measurement errors and inaccurate three-dimensional information acquisition. The latter is easily affected by image noise when the contour information of the target is extracted. Based on the above problems, a binocular stereo vision system and a 3D human motion pose reconstruction model are built first. Second, simulation experiments are designed to verify the system and model designed. Finally, the application of pose estimation based on visual sensing technology in visual communication design and environmental art design is analyzed by literature analysis. The results show that (1) although there are some errors caused by the binocular stereo vision system in the measurement, the overall error is controlled within 2% and the accuracy is high, which can be applied to the acquisition of three-dimensional information of the target in art design; (2) there is a high degree of fitting between the video sequence data created by the three-dimensional human motion pose reconstruction model designed and the real motion data, which indicates that this method has high accuracy in processing video sequences and the feasibility of applying it to human pose reconstruction in three-dimensional art design is high; (3) through the analysis of the existing literature, it is found that most of the current visual-based attitude assessment studies are carried out by using network cameras combined with computers, and the quality of the obtained images is low. The combination of the binocular stereo sensor and attitude estimation technology can be applied to the design of advertising, animation, games, and packaging, which can make the design have better visual effects and the behavior of virtual characters in animation and games more vivid. It also provides technical support for the collection of environmental spatial information and objects attitude information, the formulation of design schemes, and real-time monitoring of construction in environmental art design. The deficiency of the study is that all the analysis and ideas proposed are only at the theoretical stage, and whether they are suitable for practical work remains to be further verified. The purpose of this study is to provide an important theoretical basis for the technical upgrading of art design.



---

*Source: 2255730-2021-11-22.xml*