# Haptic Decorating on the Surface of Virtual Clay Model

Authors: Huang Lei; Hou Zeng-xuan; Guo Chao; Zhang Wei; Xu Jun
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

A novel haptic decorating method on the surface of virtual clay model is proposed. The relationship between brush deformation and endured force is researched for the first time by applying the spring-mass model to construct the 3D brush model. Then, the collision detection between virtual hairy brush and virtual clay model is researched based on collision algorithm of weighted average distance. When the hairy brush initially collides with 3D exterior, the tactility is simulated and the interactive virtual painting on the 3D exterior is carried out practically. The 3D brush stroke is formed by superimposing 3D brush footprints along the painting direction and controlling the stress of the brush. The ink quantity in the brush footprint is determined according to the proposed positive correlation between the exerted pressure on brush and outflow ink quantity of the brush. A painting storage method is also presented for storing and displaying 3D stroke painting results. The proposed method has been successfully applied in the 3D virtual painting system based on real-time force feedback technology. With this system, the 3D brush strokes with 3D half-dry and ink diffusion results can be painted with a Phantom Desktop haptic device, which effectively enhance reality to users.


---

## Body

## 1. Introduction

Product exterior design, of which main research fields include product form design and exterior decoration, plays a more and more important role in the modern product development. The exterior decoration is usually carried out by using computer aided industrial design (CAID). Traditionally, in this process, the texture mapping is applied for transforming 2D graphics onto the 3D object exterior. However, the process of texture mapping is complex and occupies more computer resources, so it is difficult for designers to catch design inspiration. With the development of virtual painting technology, direct painting and decoration on 3D product exterior are introduced. In this paper, the novel haptic decorating method on the surface of virtual clay model is proposed. This method has been realized in the 3D virtual painting system based on the real-time force feedback technology with higher performance.


## 2. Related Works

Igarashi and Cosgrove [1] proposed an adaptive unwrapping mechanism where their system can dynamically create a tailored UV-mapping for painting polygons during the interactive painting process. Nevertheless, the repeated reprojection of the existing texture gradually degrades the quality of the bitmap, so their system is only suitable for simple paintings with a limited number of strokes.ArtNova system [2, 3] used a user-centric viewing technique that integrated haptic and visual presentation by taking into account the user’s haptic manipulation in the dynamic determination of the new viewpoint locations. However, a 3D brush model was not constructed in their method that has a negative impact on the reality of the interactive painting.Fu and Chen [4] presented a haptic painting system which provided an easy way for users to accurately add color on the mesh models. Their painting system provided virtual brushes based on a haptic device with an input of 3 degrees of freedom. Since a real brush has 6 DOFs, their system cannot provide various brush effects.The artist and designer can use the system of Kim et al. [5–7] to create and refine a 3D multiresolution polygonal mesh. However, a 6 DOFs haptic device was not integrated into their system.The painting system in [8] for 3D objects provided intuitive user interaction. However, their method failed to simulate real hairy brush features that include brush flattening and bristle spreading produced by exerting real-time force on a hairy brush.The system of Chen et al. [9] can not only simulate the motion of brush bristle and painting liquid, but also imitate the liquid transfer processes among different presentations. However, their method does not mention force feedback technology, so the sense of reality was reduced during the painting process of virtual interaction.The brush in [10] is modeled with a spring-mass particle system skeleton and a subdivision surface. In their feedback force model, the stretch springs are used as brush skeletons to deform subdivision surface, and the force computation is separated from the brush deformation computation, so this approach makes limited use of the haptic feedback device’s ability to recreate an accurate haptic sensation of the virtual brush.In this paper, a new decorating method on the exterior of virtual clay model is proposed based on force feedback technology with six degrees of freedom input device. The relationship between brush deformation and endured force is researched for the first time by applying the spring-mass model to construct the 3D brush model. Then, the collision detection between virtual hairy brush and virtual clay model is researched based on collision algorithm of weighted average distance. When the hairy brush initially collides with 3D exterior, the tactility is simulated and the interactive virtual painting on the 3D exterior is carried out practically. In the virtual painting process, the collision related points can be found out based on the theory of minimum bounding ball, and then it can calculate the average normal vector of the collision point to determine the projection plane. In our 3D brush model, a virtual spring which is perpendicular to the projection plane and deforms along the normal of the projection plane is used for calculating the exerted force on the brush, and then the deformation of the brush such as the brush flattening and bristle spreading on the projection plane can be simulated. According to the deformation of the brush model at a sampling point, the 2D brush footprint between the brush and the projection plane is calculated in real-time. Then, the 3D brush footprint can be obtained by projecting the 2D brush footprint onto the 3D object surface in real-time. At the same time, along the sampling points, the 3D brush stroke is formed by superimposing 3D brush footprints of different sizes and shapes along the painting direction and controlling the stress of the brush. In the meantime, force information is sent back to the force feedback device to simulate the feeling that user touches the surface of 3D object. Users can also adjust the painting to the desired effects according to the feedback force. Therefore, the dynamic painting of the 3D brush with deformation is implemented. In our simulation system, the ink quantity in the brush footprint is determined according to the proposed positive correlation between the exerted pressure on brush and outflow ink quantity of the brush. A painting storage method is also presented for storing and displaying 3D stroke painting results.


## 3. Force Feedback Model of Hairy Brush

It is necessary to establish a virtual 3D brush model with rich representation in order to realize the haptic decorating of 3D model surface [11, 12]. According to the actual characteristics of the brush, the 3D brush models which include geometric model and mechanical model are constructed. This model can reflect the physical characteristics of the actual brush and meet the requirements of decorating of the virtual 3D model surface as well.


### 3.1. Geometric Model

The geometry is closely related to the mechanical dynamics. A well-structured geometry can not only reduce the computational complexity and improve the real-time performance, but also simulate the deformations of various brushes. We represent the geometry (Figure1 [11]) in two layers: the skeleton and the surface.Figure 1
Sketch map of the geometric model of the brush{Figure 1 reproduced from Guo et al. (2015), [under the Creative Commons Attribution License/public domain]}.The skeleton handles the general bending of the brush. We represent the skeleton as a connected sequence of line segments (spine segments) that become progressively shorter toward the tip. Since the brush tip is usually much softer than the brush root, it bends much more. In fact, usually only the tip and the belly are used to paint. Therefore, for modeling efficiency, progressively shorter segments are used towards the brush tip so as to dedicate higher resolution to tip.Suppose the skeleton hasn+1 nodes, P0,P1,…,Pn, with P0 as the tip node and Pn as the root node. The length of the line segment Pi-1Pi is denoted by Li. L1,L2,…,Ln form an arithmetic progression, and the general formula is shown as follows:(1)Li=L1+i-1DD=2Lspine-nL1nn-1,where Li stands for the length of the spine between the control node Pi and Pi+1, n refers to the number of segments of the spine, and L1 is related to the soft and hard degree of the brush: when the same force is exerted on the brush, the softer the brush is, the easier the bristles near the brush tip are deformed; thus, the value of L1 is smaller; on the contrary, the value of L1 is larger. The value of L1 is given according to the experiments. The common difference d is determined according to the initial length of the skeleton (Lspine) and the value of L1.When the brush bends, all the spine segments are in the same plane (Figure2 [11]). The inclination angle of brush holder (the angle between the brush holder and the paper plane) is denoted by θ(θ∈(0,π)). The angle between the spine segment Pi-1Pi and the paper plane (virtual projection plane) is denoted by αi(i∈1,n). When the brush is unbent, αi=θ. In the virtual painting, in order to control the outline of the brush stroke when it is painted with the brush, we define the cross-section which passes the node Pi(i∈[1,n)) as the outline controlling plane of the brush (the plane bisects the angle between the two adjacent spine segments and is perpendicular to the plane of spine segments). The deformation of the brush can be simulated by controlling the positions and sizes of the outline controlling planes.Figure 2
The deformation of brush skeleton{Figure 2 reproduced from Guo et al. (2015), [under the Creative Commons Attribution License/public domain]}.The brush surface is represented as a triangular mesh surface defined by the skeleton and the varying outline controlling planes of the brush. When the brushes are moistened and unbent, the shape of the brush is similar to a cone. Therefore, the outline controlling planes of the brush are circles along the entire skeleton. When the brush is moved down and deformed under pressure, the spine of the nib begins to be bent and deformed. The outline controlling is deformed from circle to ellipse by pressure and surface friction, as shown in Figure3 [11]. With the increasing in pressure, the short axis of the elliptical surface is getting shorter and shorter, and the long axis is getting longer and longer. The elliptical surface will become more flat, and the entire brush head is changed from a cone to an elliptical cone. This representation is computationally efficient and does not differ much from the real brush deformation.Figure 3
Sketch of deformation of outline controlling surfaceFigure 3 reproduced from Guo et al. (2015), [under the Creative Commons Attribution License/public domain]}.When the brush head gets the force ofF=0, Ria=Rib=Ri can be deduced. As the pressure increases, the formula for calculating Rib is as follows:(2)Ria=Ri×1+dp+epf,where d and e are given by empirical values; p is the pressure factor, which is defined as the ratio of the pressure F to the maxim output force of the force feedback device. In our system, the maximum output force provided by the Phantom Desktop haptic device is 7.9 N. Therefore, p=F/7.9, with a range of 0,1; pf is frictional factor which can be obtained by the formula pf=μ×p with a range of 0,1. In order to adjust the sizes of ellipses in different painting conditions, we set the adjustment factors b and c, which are determined by painting experiments to simulate the most realistic brush deformations. The algorithm of this paper assumes that the area of the outline controlling plane remains unchanged in the deformation before and after, so the short axis can be calculated as follows:(3)Rib=Ri2Ria.


### 3.2. Mechanical Model

The aim of the brush mechanical model is to simulate brush flattening and bristle spreading due to the force exerted on the brush during the painting process.We use a mass-spring model (Figure4) to simulate the brush dynamic deformation when real-time force is exerted on the brush. In the process of virtual interactive painting, a compression spring which deforms in the normal plane is placed between Pm and its projection Pm′. That is to say, the spring is perpendicular to the paper plane (virtual projection plane) and deforms along the normal of the paper plane (virtual projection plane). When the brush just contacts the paper plane, and it is unbent, the point Pm′ coincides with the brush tip node P0, and the initial length of the skeleton is denoted by LE. The spring moves downward when the pressure is exerted on the brush, and the positions of nodes change, while the length of each spine segment remains unchanged.Figure 4
Brush spring-mass model.In the initial condition of painting, the feedback pressure (Fp) is related to the downward displacement of the brush. Fp is given in(4)FP=λc·S·Ds.A proper output force will be transferred to the user for the purpose of simulating; a coefficient (λc) of feedback force is adopted to control the magnitude of force. The λc (unit N/mm) coefficient is obtained from many painting experiments using the Phantom Desktop haptic device. The value of the brush softness coefficient (S) is given in(5)S=Gd48Dm3n,where G is the Elastic Modulus of the brush spring, d is the diameter of the brush spring wire, Dm is the middle diameter of the brush spring, and n is the effective number of turns of the brush spring. The brush is much softer with the decrease in S, so the downward displacement is much larger when the same force is exerted on the brush. Ds (unit mm) refers to the amount of deformation in the spring, which is also the downward displacement of the brush.Friction can help users to perceive the surface roughness of 3D objects and control the three-dimensional brush. Whenγ is the angle between the normal vector of the surface of the 3D model and the normal vector of the projective plane, and its range is γ∈0,π/2, we can compute the friction (Ff) which is proportional to Fp (unit N) according to(6)Ff=μFpcos⁡γ.The friction coefficient (μ) of the brush is given in (7), where RH is the humidity factor of the brush which is obtained experimentally.(7)μ=RHFp.


## 3.1. Geometric Model

The geometry is closely related to the mechanical dynamics. A well-structured geometry can not only reduce the computational complexity and improve the real-time performance, but also simulate the deformations of various brushes. We represent the geometry (Figure1 [11]) in two layers: the skeleton and the surface.Figure 1
Sketch map of the geometric model of the brush{Figure 1 reproduced from Guo et al. (2015), [under the Creative Commons Attribution License/public domain]}.The skeleton handles the general bending of the brush. We represent the skeleton as a connected sequence of line segments (spine segments) that become progressively shorter toward the tip. Since the brush tip is usually much softer than the brush root, it bends much more. In fact, usually only the tip and the belly are used to paint. Therefore, for modeling efficiency, progressively shorter segments are used towards the brush tip so as to dedicate higher resolution to tip.Suppose the skeleton hasn+1 nodes, P0,P1,…,Pn, with P0 as the tip node and Pn as the root node. The length of the line segment Pi-1Pi is denoted by Li. L1,L2,…,Ln form an arithmetic progression, and the general formula is shown as follows:(1)Li=L1+i-1DD=2Lspine-nL1nn-1,where Li stands for the length of the spine between the control node Pi and Pi+1, n refers to the number of segments of the spine, and L1 is related to the soft and hard degree of the brush: when the same force is exerted on the brush, the softer the brush is, the easier the bristles near the brush tip are deformed; thus, the value of L1 is smaller; on the contrary, the value of L1 is larger. The value of L1 is given according to the experiments. The common difference d is determined according to the initial length of the skeleton (Lspine) and the value of L1.When the brush bends, all the spine segments are in the same plane (Figure2 [11]). The inclination angle of brush holder (the angle between the brush holder and the paper plane) is denoted by θ(θ∈(0,π)). The angle between the spine segment Pi-1Pi and the paper plane (virtual projection plane) is denoted by αi(i∈1,n). When the brush is unbent, αi=θ. In the virtual painting, in order to control the outline of the brush stroke when it is painted with the brush, we define the cross-section which passes the node Pi(i∈[1,n)) as the outline controlling plane of the brush (the plane bisects the angle between the two adjacent spine segments and is perpendicular to the plane of spine segments). The deformation of the brush can be simulated by controlling the positions and sizes of the outline controlling planes.Figure 2
The deformation of brush skeleton{Figure 2 reproduced from Guo et al. (2015), [under the Creative Commons Attribution License/public domain]}.The brush surface is represented as a triangular mesh surface defined by the skeleton and the varying outline controlling planes of the brush. When the brushes are moistened and unbent, the shape of the brush is similar to a cone. Therefore, the outline controlling planes of the brush are circles along the entire skeleton. When the brush is moved down and deformed under pressure, the spine of the nib begins to be bent and deformed. The outline controlling is deformed from circle to ellipse by pressure and surface friction, as shown in Figure3 [11]. With the increasing in pressure, the short axis of the elliptical surface is getting shorter and shorter, and the long axis is getting longer and longer. The elliptical surface will become more flat, and the entire brush head is changed from a cone to an elliptical cone. This representation is computationally efficient and does not differ much from the real brush deformation.Figure 3
Sketch of deformation of outline controlling surfaceFigure 3 reproduced from Guo et al. (2015), [under the Creative Commons Attribution License/public domain]}.When the brush head gets the force ofF=0, Ria=Rib=Ri can be deduced. As the pressure increases, the formula for calculating Rib is as follows:(2)Ria=Ri×1+dp+epf,where d and e are given by empirical values; p is the pressure factor, which is defined as the ratio of the pressure F to the maxim output force of the force feedback device. In our system, the maximum output force provided by the Phantom Desktop haptic device is 7.9 N. Therefore, p=F/7.9, with a range of 0,1; pf is frictional factor which can be obtained by the formula pf=μ×p with a range of 0,1. In order to adjust the sizes of ellipses in different painting conditions, we set the adjustment factors b and c, which are determined by painting experiments to simulate the most realistic brush deformations. The algorithm of this paper assumes that the area of the outline controlling plane remains unchanged in the deformation before and after, so the short axis can be calculated as follows:(3)Rib=Ri2Ria.


## 3.2. Mechanical Model

The aim of the brush mechanical model is to simulate brush flattening and bristle spreading due to the force exerted on the brush during the painting process.We use a mass-spring model (Figure4) to simulate the brush dynamic deformation when real-time force is exerted on the brush. In the process of virtual interactive painting, a compression spring which deforms in the normal plane is placed between Pm and its projection Pm′. That is to say, the spring is perpendicular to the paper plane (virtual projection plane) and deforms along the normal of the paper plane (virtual projection plane). When the brush just contacts the paper plane, and it is unbent, the point Pm′ coincides with the brush tip node P0, and the initial length of the skeleton is denoted by LE. The spring moves downward when the pressure is exerted on the brush, and the positions of nodes change, while the length of each spine segment remains unchanged.Figure 4
Brush spring-mass model.In the initial condition of painting, the feedback pressure (Fp) is related to the downward displacement of the brush. Fp is given in(4)FP=λc·S·Ds.A proper output force will be transferred to the user for the purpose of simulating; a coefficient (λc) of feedback force is adopted to control the magnitude of force. The λc (unit N/mm) coefficient is obtained from many painting experiments using the Phantom Desktop haptic device. The value of the brush softness coefficient (S) is given in(5)S=Gd48Dm3n,where G is the Elastic Modulus of the brush spring, d is the diameter of the brush spring wire, Dm is the middle diameter of the brush spring, and n is the effective number of turns of the brush spring. The brush is much softer with the decrease in S, so the downward displacement is much larger when the same force is exerted on the brush. Ds (unit mm) refers to the amount of deformation in the spring, which is also the downward displacement of the brush.Friction can help users to perceive the surface roughness of 3D objects and control the three-dimensional brush. Whenγ is the angle between the normal vector of the surface of the 3D model and the normal vector of the projective plane, and its range is γ∈0,π/2, we can compute the friction (Ff) which is proportional to Fp (unit N) according to(6)Ff=μFpcos⁡γ.The friction coefficient (μ) of the brush is given in (7), where RH is the humidity factor of the brush which is obtained experimentally.(7)μ=RHFp.


## 4. The Collision Detection between Virtual Hairy Brush and Virtual Clay Model

To determine the relative location between the hairy brush and 3D product, a contact force in a small tolerance range is introduced in this paper. When the distance between the hairy brush and product exterior of virtual clay model is in the tolerance range, a contact force is transferred to users by Phantom Desktop haptic device and the collision between virtual hairy brush and virtual clay model can be detected.The distance between the hairy brush and product exterior of virtual clay model is set asddep. In this paper, the exteriors of hairy brush and virtual clay model are described as triangle mesh models, and then ddep is represented as the weighted average distance between brush tip P0 and M triangle mesh vertexes which are closest to P0. (8)ddep=∑i=1Mwi×ni→×xi→-x→wi=dmax-di∑j=1Mdmax-dj,where wi, ni→, xi→, and x→ are weighted coefficient, the normal vector of product exterior vertexes, coordinate vector of product exterior vertexes, and the vector of brush tip P0, respectively. di=xi→-x→. dmax is the maximum of di.Whenddep is in the tolerance range, the hairy brush collides with the product exterior of virtual clay model. A contact force is transferred to users by Phantom Desktop haptic device, and the tactility in the real painting is simulated. Then, the button in the stylus of Phantom Desktop haptic device is pressed and the contact force is invalid and the interactive virtual painting on the 3D exterior is begun practically. The coordinate vector of brush tip P0 is denoted as (9)x→⟵x→+ddep×n→n→,where the weighted average vector n→ is denoted as (10)n→=∑i=1Mwi×ni→.


## 5. Virtual Painting on 3D Object Exterior

### 5.1. Modeling a Footprint Profile of the Virtual Brush

A local mapping method is presented for mapping the 2D brush footprint into the 3D brush footprint on 3D object exterior. The 3D brush stroke is obtained by superimposing the 3D brush footprints along the painting direction.The brush interacts with the plane when the brush is under pressure. The nib of the brush begins to deform when subjected to force, and it contacts with the plane to form a “raindrops” type of strokes [13]. This paper chooses the orthographic projection of the contact area between the brush and the plane as the brush stroke. In the process of virtual 3D painting, the actual brush strokes are simplified and formed by two symmetrical B-spline curves in order to simulate the footprint profile, as shown in Figure 5. When the brush is subjected to different pressures, the size and shape of the stroke can be changed by controlling the number and position of the control points of the brush strokes.Figure 5
The generation of brush footprint.According to the geometric relation, the lineNiNi′ is the intersection of the contour circle and the plane whose center of circle is Pi. The coordinates of Md are obtained by intersecting the left outline of the brush with the plane in which the stroke is located. Then, the coordinates of the control node P0 of brush tip can be obtained, and the coordinates of the control node Pi of contour central line of the brush can be obtained by the coordinates of P0, and then the coordinates of the control points of the brush stroke contour can be calculated. The specific algorithms are shown as follows.The coordinates of pointP0 are as follows:(11)x0=∑i=1nLicos⁡αisin⁡βy0=0z0=∑i=1nLicos⁡αicos⁡β,where the range of β is 0,2π. The coordinates of the contour control points Pi on the spine of the brush are as follows:(12)xPi=sin⁡β∑t=1iLicos⁡αiyPi=yPi+1+Li+1sin⁡αi+1zPi=cos⁡β∑t=1iLicos⁡αi,i∈0,n.The contour control point of the brush strokesNi, Ni′ is the intersection point of the control circle of brush outline and the drawing plane at the control node Pi; then the coordinates of the point Ni can be calculated by(13)xNi=xPi-yPitan⁡αi+αi+12sin⁡β-NiMicos⁡βyNi=0zNi=zPi-yPitan⁡αi+αi+12cos⁡β+NiMisin⁡β,where(14)NiMi=RibRiaRib2-yPi2cos2αi+αi+1/2.SinceNi and Ni′ are symmetrical about the line Pn′P0, the coordinate expressions about Ni′ are as follows:(15)xNi′=xPi-yPitan⁡αi+αi+12sin⁡β+NiMicos⁡βyNi′=0zNi′=zPi-yPitan⁡αi+αi+12cos⁡β-NiMisin⁡β.Then a B-spline curve is used to form a closed two-dimensional brush footprint byP0, Ni, Ni′, and Md; Figure 5(b) is a two-dimensional brush footprint generated by the system.


### 5.2. The Establishment of Projective Plane

According to the smallest enclosing ball algorithm [14], we compute the bounding sphere of the bent brush according to an effective ball expanding operation until all the vertices of the exterior of the bent brush are in the bounding sphere.In Figure6, vertex A is an arbitrary vertex of the bent brush. On the exterior of the bent brush, the farthest vertex B from vertex A is obtained by traversing all the vertices of the exterior of the bent brush [15]. In the same way, the farthest vertex C from vertex B is obtained.The midpoint and half-length of line segment BC are denoted by D0 and r0, respectively. The initial bounding sphere is defined as (D0,r0).Figure 6
The bounding sphere of the bent brush.The length of the line segmentD0E, which is obtained by connecting an outlier vertex E and D0, is denoted by l1. D1 is positioned along the line segment D0E at distance r1 from vertex E, and the updated sphere is defined as (D1,r1). In the initial sphere, the diameter FG is perpendicular to the line segment D0E. Vertex E and diameter FG form an isosceles triangle EFG with base 2r0, and the radius of the updated sphere r1 is obtained by(16)r1=r02+l122l1.Next, the length of line segmentD0D1 (ld1) is computed below:(17)ld1=r12-r02.The coordinate values of the updated center of the sphere are calculated according told1 and the coordinate values of D0.Repeat the above steps until the smallest bounding sphere is obtained, which encloses all the vertices of the bent brush. When the distance between the center of the smallest bounding sphere and an object exterior vertex is less than or equal to the radius of the smallest bounding sphere, these object exterior vertices are defined as effective vertices. The normal of the 2D brush footprint plane is defined by the average normal of these effective vertices (Figure7), and this plane went through the projection point of the brush root node Pm′ on the object exterior. Then, the 3D brush footprint profile is obtained by mapping the 2D brush footprint onto the 3D object exterior.Figure 7
The determination of the 2D brush footprint plane.


### 5.3. The Formation of 3D Brush Stroke

In general, the ink quantity of the 3D brush footprint at a sampling point is proportional to the pressure exerted on the brush and the current ink quantity of the brush. Suppose thatZ is the initial brush ink quantity, and pei is the pressure factor of the ith sampling point. The 3D brush footprint ink quantity at the first sampling (Q1) is determined by(18)Q1=η×pe1×Z,where η is the ink quantity factor, which is used for controlling the ink quantity of the 3D brush footprint under different painting conditions, and given by painting experiments.The ink quantity of the 3D brush footprint at theith sampling (Qi) is determined by(19)Qi=η×pei×Z-∑t=1i-1Qt.When the brush moves along a certain path, the ink contained in the nib is transferred to the surface of the three-dimensional model from the brush. The ink quantity is converted into color intensity using theory of KM [16], and the 3D stroke is formed on the surface of 3D model. The specific steps are shown in Figure 8.Figure 8
3D brush stroke formed by superimposing 3D brush footprints.During the virtual 3D painting, the 3D brush stroke is simulated by superimposing 3D brush footprints along the sampling points (Figure9).Figure 9
3D brush stroke formed by superimposing 3D brush footprints.The methods of a two-dimensional semidry painting and ink diffusion based on the force feedback are also presented in our previous papers [17, 18]. In the painting process of virtual 3D, the preset value and corresponding parameters of rice paper fiber model are determined according to the 3D product exterior property, and then the half-dry stroke and the ink diffusion on 3D product exterior (useful out of shape) are simulated by the mapping of 2D half-dry stroke and ink diffusion, respectively, which support designers to catch design inspiration.


## 5.1. Modeling a Footprint Profile of the Virtual Brush

A local mapping method is presented for mapping the 2D brush footprint into the 3D brush footprint on 3D object exterior. The 3D brush stroke is obtained by superimposing the 3D brush footprints along the painting direction.The brush interacts with the plane when the brush is under pressure. The nib of the brush begins to deform when subjected to force, and it contacts with the plane to form a “raindrops” type of strokes [13]. This paper chooses the orthographic projection of the contact area between the brush and the plane as the brush stroke. In the process of virtual 3D painting, the actual brush strokes are simplified and formed by two symmetrical B-spline curves in order to simulate the footprint profile, as shown in Figure 5. When the brush is subjected to different pressures, the size and shape of the stroke can be changed by controlling the number and position of the control points of the brush strokes.Figure 5
The generation of brush footprint.According to the geometric relation, the lineNiNi′ is the intersection of the contour circle and the plane whose center of circle is Pi. The coordinates of Md are obtained by intersecting the left outline of the brush with the plane in which the stroke is located. Then, the coordinates of the control node P0 of brush tip can be obtained, and the coordinates of the control node Pi of contour central line of the brush can be obtained by the coordinates of P0, and then the coordinates of the control points of the brush stroke contour can be calculated. The specific algorithms are shown as follows.The coordinates of pointP0 are as follows:(11)x0=∑i=1nLicos⁡αisin⁡βy0=0z0=∑i=1nLicos⁡αicos⁡β,where the range of β is 0,2π. The coordinates of the contour control points Pi on the spine of the brush are as follows:(12)xPi=sin⁡β∑t=1iLicos⁡αiyPi=yPi+1+Li+1sin⁡αi+1zPi=cos⁡β∑t=1iLicos⁡αi,i∈0,n.The contour control point of the brush strokesNi, Ni′ is the intersection point of the control circle of brush outline and the drawing plane at the control node Pi; then the coordinates of the point Ni can be calculated by(13)xNi=xPi-yPitan⁡αi+αi+12sin⁡β-NiMicos⁡βyNi=0zNi=zPi-yPitan⁡αi+αi+12cos⁡β+NiMisin⁡β,where(14)NiMi=RibRiaRib2-yPi2cos2αi+αi+1/2.SinceNi and Ni′ are symmetrical about the line Pn′P0, the coordinate expressions about Ni′ are as follows:(15)xNi′=xPi-yPitan⁡αi+αi+12sin⁡β+NiMicos⁡βyNi′=0zNi′=zPi-yPitan⁡αi+αi+12cos⁡β-NiMisin⁡β.Then a B-spline curve is used to form a closed two-dimensional brush footprint byP0, Ni, Ni′, and Md; Figure 5(b) is a two-dimensional brush footprint generated by the system.


## 5.2. The Establishment of Projective Plane

According to the smallest enclosing ball algorithm [14], we compute the bounding sphere of the bent brush according to an effective ball expanding operation until all the vertices of the exterior of the bent brush are in the bounding sphere.In Figure6, vertex A is an arbitrary vertex of the bent brush. On the exterior of the bent brush, the farthest vertex B from vertex A is obtained by traversing all the vertices of the exterior of the bent brush [15]. In the same way, the farthest vertex C from vertex B is obtained.The midpoint and half-length of line segment BC are denoted by D0 and r0, respectively. The initial bounding sphere is defined as (D0,r0).Figure 6
The bounding sphere of the bent brush.The length of the line segmentD0E, which is obtained by connecting an outlier vertex E and D0, is denoted by l1. D1 is positioned along the line segment D0E at distance r1 from vertex E, and the updated sphere is defined as (D1,r1). In the initial sphere, the diameter FG is perpendicular to the line segment D0E. Vertex E and diameter FG form an isosceles triangle EFG with base 2r0, and the radius of the updated sphere r1 is obtained by(16)r1=r02+l122l1.Next, the length of line segmentD0D1 (ld1) is computed below:(17)ld1=r12-r02.The coordinate values of the updated center of the sphere are calculated according told1 and the coordinate values of D0.Repeat the above steps until the smallest bounding sphere is obtained, which encloses all the vertices of the bent brush. When the distance between the center of the smallest bounding sphere and an object exterior vertex is less than or equal to the radius of the smallest bounding sphere, these object exterior vertices are defined as effective vertices. The normal of the 2D brush footprint plane is defined by the average normal of these effective vertices (Figure7), and this plane went through the projection point of the brush root node Pm′ on the object exterior. Then, the 3D brush footprint profile is obtained by mapping the 2D brush footprint onto the 3D object exterior.Figure 7
The determination of the 2D brush footprint plane.


## 5.3. The Formation of 3D Brush Stroke

In general, the ink quantity of the 3D brush footprint at a sampling point is proportional to the pressure exerted on the brush and the current ink quantity of the brush. Suppose thatZ is the initial brush ink quantity, and pei is the pressure factor of the ith sampling point. The 3D brush footprint ink quantity at the first sampling (Q1) is determined by(18)Q1=η×pe1×Z,where η is the ink quantity factor, which is used for controlling the ink quantity of the 3D brush footprint under different painting conditions, and given by painting experiments.The ink quantity of the 3D brush footprint at theith sampling (Qi) is determined by(19)Qi=η×pei×Z-∑t=1i-1Qt.When the brush moves along a certain path, the ink contained in the nib is transferred to the surface of the three-dimensional model from the brush. The ink quantity is converted into color intensity using theory of KM [16], and the 3D stroke is formed on the surface of 3D model. The specific steps are shown in Figure 8.Figure 8
3D brush stroke formed by superimposing 3D brush footprints.During the virtual 3D painting, the 3D brush stroke is simulated by superimposing 3D brush footprints along the sampling points (Figure9).Figure 9
3D brush stroke formed by superimposing 3D brush footprints.The methods of a two-dimensional semidry painting and ink diffusion based on the force feedback are also presented in our previous papers [17, 18]. In the painting process of virtual 3D, the preset value and corresponding parameters of rice paper fiber model are determined according to the 3D product exterior property, and then the half-dry stroke and the ink diffusion on 3D product exterior (useful out of shape) are simulated by the mapping of 2D half-dry stroke and ink diffusion, respectively, which support designers to catch design inspiration.


## 6. Painting Storage Method for Virtual Clay Model

In virtual clay modeling, the compressed voxel model is defined by three orthogonal dexel ray groups, perpendicular toXOY, YOZ, and XOZ plane, respectively [19–21]. Each dexel ray is located on the intersection between square grids (Figure 10). A list of intersections between dexel rays and 3D object is denoted by nodes in linked list. Each node in linked list stores position (P), material (M), and normal vector (N) (Figure 11).Figure 10
Voxel and dexel model.Figure 11
Node information of dexel model.Example data structure of the compressed voxel model is shown in Algorithm1, where dimension denotes the division space of voxel grid. X plane describes the information of the dexels parallel to the X axis. This paper extends the structure of the compressed voxel model, and the painting information (color intensities RGB) on virtual 3D product exterior is saved directly in this model (Algorithm 2). For example, RGB (209, 207, and 210) at the intersection (X=156.2440) between 3D object and the dexel (Y=20, Z=13) describes the color intensities of this point.Algorithm 1:Data structure of the compressed voxel model.
SXF2a
Dimension =
160,60,80
O
ffset =
0,0,0
Voxelsize
=
1,1,1
GridType
= 0

Xplane
20,13
3,,12.5128
,,156.244
⋯
Yplane
0,56
3,,26.5227
,,29.5227
⋯
Zplane
0,27
3,,55.2843
,,58.663
⋯Algorithm 2:Painting information in the compressed voxel model.
SXF2a
Dimension =
72,72,110
Offset =
0,0,0
Voxelsize =
1,1,1
GridType = 0

Xplane
20,13
3,,12.5128
,,156.244
0,209,207,210
⋯
Yplane
0,56
3,,26.5227
,,29.5236,209,207,210
⋯
Zplane
0,27
3,,55.2843
,,58.663
0,209,207,210
⋯


## 7. Experimental Verification

In this paper, the 3D virtual painting system based on the real-time force feedback technology has been developed with Visual Studio 2005, Qt, and Open Inventor. The 3D painting is implemented in this system running on the HP Xw8600 workstation with a 6DOF Phantom Desktop device.In our simulation system, the spring-mass model is applied to simulate the brush deformation as long as real-time force is exerted on the brush. A two-dimensional brush footprint between the brush and the brush plane is simulated according to the brush’s deformation. Next, the 3D brush footprint is obtained by projecting the 2D brush footprint onto the outer surface of the 3D object in real-time. Finally, the 3D brush stroke is simulated by superimposing 3D brush footprints along the direction of painting. The schematic diagram of the system for haptic decorating is shown in Figure12; the haptic device provides the brush motion and location information that can compute the force exerted on the brush. Some of the input devices commonly used in haptic decorating are listed in Table 1. The Phantom Desktop (Figure 13) uses the automatic calibration mechanism that the internal virtual model movements are controlled by the motor. The phantom haptic device is able to deliver an arbitrary force to the user under programmatic control compared with the Wacom Intuos Tablet from Table 1. The interactive haptic decorating systems are shown in Figure 14.Table 1
Some of the input devices commonly used in haptic decorating.
Input device
Input DOF
Output DOF
Haptics
Mouse
2
0
None
Wacom Intuos Tablet
5
0
Static
Phantom Desktop
6
3
ProgrammaticFigure 12
The schematic diagram of the system for haptic decorating.Figure 13
The haptic device named Phantom Desktop.Figure 14
The interactive haptic decorating system.In a simulation experiment, using the hairy brush model in Table2, we apply the corresponding parameters listed in the table to simulate the Chinese brush. When we used different pressures (Fp) on the brush, then the brush deformation is shown in Figure 15.Table 2
Main parameters of virtual hairy brush.
Parameter
H
l
1 (mm)
n
μ
b
c
L (mm)
Value
0.7
1
10
0.28
1.5
0.75
40Figure 15
Brush deformation is presented when exerting different types of pressure on the brush.As a simple example, the Chinese character “七上八下” is drawn on the exterior of a bowl (Figure16).Figure 16
Chinese character “七上八下” is drawn on the exterior of a bowl with our haptic decorating system.Some concrete examples of the interactive haptic decorating system created on the surface of the 3D objects are shown in Figure17.Figure 17
Some concrete examples of the interactive haptic decorating system.In another simulation experiment, using the hairy brush model in Table1, Chinese character “漫步人生” is drawn on the exterior of a bowl (Figure 18(a)). Compared with the painting results on the 2D painting plane in “Moxi” system (Figure 18(b)), the 3D brush stroke with ink diffusion effects is displayed with our method.Figure 18
The Chinese character “漫步人生” created with our system (a) and the “MoXi” system (b).
(a)
(b)In order to verify the effectiveness of the haptic feedback, a simulation experiment is implemented by six users which include two art students and four ordinary volunteers. In our system, all the users are able to pick up the haptic stylus and start painting immediately, with little training or detailed instruction. The same Chinese character “漫步人生” is painted by these users under the same painting condition with two devices: (I) mouse and keyboard; (II) Phantom Desktop device, and the two devices are randomly used in sequence. The used time of the painting process with the two devices is shown in Figure 19. The used time with the mouse and keyboard is more than double that with Phantom Desktop device. After the experiment, six users are also asked to use a pen to check which painting manner is more attractive in accomplishing this task, whether it is painted with (I) or (II). The survey results show that five out of six users express their preference for painting with (II) (Phantom Desktop device) compared to (I) (mouse and keyboard). We can conclude that painting with haptic feedback is better than that without haptic sensation.Figure 19
The used time of the painting process with the two devices ((I) and (II)).


## 8. Conclusion

In this paper, a new haptic decorating method on the exterior of virtual clay model is proposed. Firstly, the force feedback model of hairy brush is constructed to simulate the deformation of hairy brush. The 2D brush footprint is simulated according to the brush deformation. Then, a local mapping method is presented for mapping the 2D brush footprint into the 3D brush footprint on 3D object exterior, and the 3D brush stroke is created by superimposing 3D brush footprints. A painting storage method is also presented for storing and displaying 3D stroke painting results. The proposed methods have been applied in the 3D virtual painting system based on the real-time force feedback technology. With this system, 3D brush strokes with 3D half-dry and ink diffusion results can be painted with a Phantom Desktop haptic device, which effectively enhance reality to users.



---

*Source: 7939706-2018-01-03.xml*