# Computer-Aided Design Methods in Virtual Reality Product Development

Authors: Shan Guan
Journal: Advances in Multimedia
 (2022)

---

## Abstract

In order to improve the effectiveness of virtual reality product design, this paper combines the computer-aided design method to construct a virtual reality product development system and analyzes the elements, architecture, and calculation steps of the virtual reality model in detail. Moreover, from the research on the color appearance phenomenon, this paper leads to the improvement direction of improving the virtual reality model. In addition, this paper discusses the advantages of the virtual reality product design model and its feasibility as a PCS link space to undertake the task of color appearance transformation. The experimental research analysis shows that the virtual reality product design system based on computer-aided technology proposed in this paper has good effects and can effectively improve the effect of virtual reality product design.


---

## Body

## 1. Introduction

At present, the way to improve the design level of new products is to strengthen the process innovation of design and use virtual reality technology to study the usability of industrial design products in the process of product design, so as to improve the success rate of product development. After usability research, product design is no longer the result of rigid and mechanical design, but after fully understanding people’s usage habits and needs, it brings convenience and pleasure to people from all aspects. Moreover, machines and systems are no longer things that humans have produced for themselves, increasingly complex and difficult to navigate and master. At the same time, due to the application of usability research, the human-computer interaction interface is more effective, easy to use, and comfortable, and the user experience is comprehensively improved. However, although some companies have realized the value of usability, it still needs a process of nurturing for every employee of the company to understand and agree. From the perspective of the entire product development process, many companies still do not take usability research as a necessary part of the project development process. Therefore, it is impossible to fundamentally guarantee the consistency of the usability of all products in the company. Therefore, a process change is a prerequisite for good usability research. In addition to this, it is easy for businesses to confuse usability research with market research. Moreover, usability research is not a panacea, it must have its scope, and everything related to users cannot be regarded as the scope of usability.The rapid development of science and technology has realized increasingly perfect product functions, and at the same time, it has also deepened the contradiction between people’s demand for the complexity of product functions and the ease of use, which leads to the widespread availability of products [1]. People are the starting point and ultimate destination of design, and product designers have the responsibility to ensure that products will not endanger the health and safety of users, protect users from bad design factors, and better meet users’ deeper needs [2]. Therefore, the user-centered product usability research aimed at improving this contradiction must have important research value and application significance [3].Usability design originated from the US Air Force weapons during World War II and has been widely used in the design of industrial products since then. Researchers put forward the concept of user interface usability (Usability) [4] and began to study its evaluation method and its application. For the concept of usability, researchers have proposed various explanations [5]. The introduction of the concept of user interface usability has changed people’s understanding of the interaction process, and the user-centered interface design idea has gradually gained popularity [6]. With the popularity of the user-centric concept, businesses are increasingly realizing the significance of usability research. Enterprises feel that usability research can make it easier for people to face the system and pay more attention to their own work, rather than focusing on the tools or the system itself [7].The development of modern information technology and the widespread use of computers have fundamentally changed the form and connotation of modern industrial design while changing our way of life [8]. In recent years, CAID technology has provided strong support for enterprises in the process of new product development, but at present, software combination is mostly used in virtual product design to complete the product design process [9]. For example, for the product modeling of complex curved surfaces, a combination of software such as Rhino and Pro/Engineer is used to complete the virtual product design model. In essence, it does not free the designers from the two-dimensional mouse and keyboard, and the design staff does not [10]. To really participate in virtual product design, to a certain extent, we limit the enthusiasm and creativity of designers [11].Restricted by the current technical conditions, the help provided by the computer to the design process in the conception and sketch stage is relatively limited [12]. Because the design thinking process is an intuitive, nonverbal image thinking process, while traditional computers are based on logical operations and arithmetic operations, the current computer system is not good at generating ideas and concepts, nor can it be used for aesthetics. Judging choices, it is not yet possible for computers to completely replace the role of humans at this stage of industrial design. Sketches are still one of the most intuitive and quick expressions for designers in the concept stage to capture design ideas instantly [13]. The three-dimensional modeling function provided by the computer cannot provide enough help for design thinking because it is relatively complicated and difficult to operate. At present, the auxiliary role of computers in design is mainly reflected in the in-depth development stage of the design scheme and in the process of revising, improving, and expressing a certain scheme [14]. There are two different working modes in current design practice: one is the design mode based on the digital model, that is, the designer directly operates the computer-aided industrial design (CAID) system for modeling [15]. Designers first input the hand-painted design sketches, three views, etc., into the computer by scanning, or they can directly draw the design plan in two-dimensional drawing software such as StudioPaint and 3DPaint. This kind of software provides tools such as pencil and airbrush. With the application of graphic tablets, designers can draw sketches as naturally as using traditional pen and paper. Whether it is a sketch entered into a computer by scanning or a sketch drawn in 3DPaint, it acts as a link between 2D and 3D, because they will be imported into 3D modeling software as a reference [16]. In 3D CAID software, designers extract characteristic curves from background sketches to ensure that the design intent embodied in the sketches is preserved during the modeling process. The extraction of the characteristic curve is the basic work of modeling because the characteristic curve is not only the skeleton of the product model but also the most important manifestation of the product modeling characteristics [17]. 3D modeling constitutes the core task of CAID. To express design thinking into a digital model, designers require professional CAID software with strong modeling capabilities to ensure the complete and accurate expression of the design scheme [18]. Another design pattern is based on a traditional mock-up, where designers can create a mock-up in a way they are familiar with, carry out their design work, and then digitize this model through reverse engineering. Physical models are still an important means of industrial design, and hand-made models are not only for design expression but also a process of deliberation on design [19]. Some subtle curves in product modeling are difficult to express clearly with other media (such as drawings) and can only be determined by adjusting the morphological proportions of the actual product model in three-dimensional space. Especially now, products tend to be perceptual free-form surface modeling. Even when drawing renderings from multiple angles, it is difficult to express design ideas clearly. Clay models are still the working medium of many designers and the final basis for evaluation and decision-making [20]. The next work is the process of digitizing the physical model to convert it into a 3D data model, reverse engineering.A 3D CMM is used to collect the coordinates of points on the model to generate a point cloud, which is then filtered. After that, the computer system is used to further define the characteristic curve of the model according to the processed point data, and then, the product model is further formed. Through the reading of the point by the 3D coordinate measuring instrument, the original design intent can be faithfully retained and the data model can be easily modified and its size can be precisely controlled.In order to improve the effectiveness of virtual product development, this paper combines the computer-aided design method to construct a virtual reality product development system to improve the design interactivity of virtual reality products.


## 2. Calibration of the Structured Light System Based on Planar Target

### 2.1. System Calibration

Since the imaging principle of the projector can be regarded as the inverse process of the camera imaging, the pinhole model of the camera can be used to approximate the projector model, as shown in Figure1. The three-dimensional coordinates of any point q=uvT on the object plane of the projector projected into the space are Q=X Y ZT, and their homogeneous coordinates are q˜=uv1T and Q˜=XYZ1T.Figure 1
Projector pinhole model.Based on the perspective projection model, the projection mapping of the spatial pointQ˜ to the image coordinate q˜ on the object plane can be expressed as(1)q˜=sHQ˜.Among them,q˜ and Q˜ correspond to the homogeneous coordinate forms of q and Q, respectively, and the parameter s is the scale factor. H is a homography matrix, which consists of two parts: (1) The external parameters of the projector, which are composed of a rotation matrix R and a translation vector T, can be represented by homogeneous coordinates W = [R T]. (2) The internal parameters of the projector M=fx0cx0fycy001. Among them, cx,cy is the coordinate of the principal point of the object plane, fx=f×sx and fy=f×sy are the focal lengths along the two coordinate axes of the object plane, f is the focal length of the projection lens, and sx and sy represent the number of pixels per unit distance along the two coordinate axes in the object plane. Therefore, the ideal pinhole model of the projector can be expressed as(2)q˜=sMRTQ˜.Due to the inevitable nonlinear distortion of the lens in the process of designing, processing, and assembling optical lenses, the influence of lens distortion should be considered in high-precision measurement, as shown in Figure2.Figure 2
Radial and tangential distortion of the camera.This paper mainly considers the radial distortion from the lens shape and the tangential distortion from the whole assembly process, because other distortions have less influence on the measurement accuracy. Among them, the radial distortion model can be expressed as(3)x=u1+k1r2+k2r4+k3r6,y=v1+k1r2+k2r4+k3r6.The tangential distortion model can be expressed as(4)x=u+2p1v+p2r2+2u2,y=v+p1r2+2v2+2p2u.In (3) and (4), (x, y) is the corrected pixel coordinate point, (u, v) is the original coordinate of the distortion point on the object plane, and r=u2+v2. Thus, projector lens distortion can be described by five parameters of radial distortion and tangential distortion.(5)sq˜=MRTQ˜+δr+δd.Among them,δr=xryrT represents radial distortion and δd=xdydT represents tangential distortion.The calibration of the camera is done using the planar calibration method of the two-step parameter optimization technique whenq˜ and Q˜ are known.When we assumeZ = 0, the specific internal parameter solution process is as follows:(6)xyz=sMr1r2r5TXY01=sMr1r2TXY1.A mappingH=Mr1r2T is established between the point on the calibration plane and its object plane, and the homography matrix is(7)H=h11h12h13h21h22h23h31h33h33=h1h2h3.That is,(8)h1h2h3=Mr1r2T.Becauser1 and r2 are unit orthogonal vectors, rT/1r1=rT/2r2=1 and r1T2=0. From the above formula, we can get(9)h1TM−TM−1h2=0,(10)h1TM−TM−1h1=h2TM−TM−1h2.We set(11)B=M−TM−1=B11B12B13B21B22B23B31B33B33.According to constraints (9) and (10), the two homogeneous equations of vector b can be obtained as(12)v12Tv11−v12Tb=0.Here,B=B11B12B22B13B23B33T;(13)vij=hi1hj1hi1hj2+hi2hj1hi2hj2hi3hj2+hi1hj3hi3hj1+hi2hj3hi3hj3.After acquiringn plane images, the linear equation system can be obtained according to formula (13):(14)Vb=0.In (14), b is calculated by formula (12). B can be calculated according to b, and then, the inverse operation of B is performed. Finally, the various parameters of the camera can be obtained by decomposing the result of the inverse operation. The specific calculation formula is as follows:(15)cy=B12B13−B11B23B11B22−B122,k=B33−B132+cyB12B13−B11B23B11.fx=kB11fy=kB11B11B22−B122,s=−B12fx2fyk,cx=scyfx−B13fx2k.


### 2.2. Nonlinear Optimization of Camera Parameters Based on AWMPSO

Particle swarm optimization (PSO) is a metaheuristic method obtained by studying the predation behavior of birds, due to its advantages of easy implementation, high precision, and fast convergence speed. The formula for particle update position and velocity is as follows:(16)vidt+1=wvidt+1+c1r1pbest−xidt+c2r2gbest−xidt,xidt+1=xidt+1+vidt+1.For general particle swarm optimization, due to the lack of dynamic speed adjustment ability, its global search ability needs to be further improved. Moreover, since all particles eventually converge to a position, particles get stuck in a local optimum. Therefore, this paper proposes an adaptive weight variation particle swarm optimization algorithm (AWMPSO) to improve the searchability of the algorithm.t(17)ISAid=xid−pbestpbest−dbest+ε,d=1,2……,D,(18)wid=1.4−a11+e−ISAid,(19)σ2=∑i=1nfi−favgδ2,δ=maxfi−favg,maxfi−favg>1,1,others.This operation can effectively improve the algorithm’s ability to jump out of the local optimum. For gbest satisfying the mutation condition, the calculation formula of mutation probability pm is as follows:(20)pm=k,σ2=0andfgbest>fd,0,others.Among them,fd is a constant between [0,1]. f is the desired optimal solution. For the mutation operation of gbest, the method of random perturbation is adopted.(21)gbestd=1+0.01∗η×gbestd.Among them,η represents a random variable obeying a Gaussian distribution. To sum up, in view of the shortcomings of particle swarm optimization, an ISA-based AWMPSO is proposed to dynamically adjust the inertia weight w and adaptively mutate gbest based on the population fitness variance.AWMPSO introduces dynamic adjustment weightw and mutation operation, which improves the global searchability.(22)fobj=min∑i=1n∑j=1m||pij−pijA,k1,k2,k3,p1,p2,Ri,Ti,Mij.pij(A, k1, k2, k3, p1, p2, Ri, Ti, Mij) are the two-dimensional pixel coordinates obtained by reprojecting the three-dimensional coordinates Mij by formula (22). The specific optimization steps are shown in Figure 3.Figure 3
Overall flow chart of the improved calibration method.Step 1.
Population initialization: the algorithm takes the linear solution of the camera parameters without considering lens distortion as the mean of the initial positions of N particles so that the particles are evenly distributed around the linear solution.Step 2.
Build a fitness function: the fitness function is:(23)fitness=1n×m∑i=1n∑j=1mUij−uij2+Vij−vij2.Step 3.
Dynamically adjust the inertia weight: the algorithm dynamically adjusts the inertia weightw according to formulas (18) and (19).Step 4.
Update population position: the algorithm updatesxid and vid according to formula (16).Step 5.
Mutation operation: the algorithm determines the mutation probability according to the population fitness varianceσ2 and the expected optimal value fd.Step 6.
Adjust gbest again: for the gbest that satisfies the mutation condition, the algorithm adjusts gbest again according to formula (21).Step 7.
Determine the termination condition: when the maximum number of iterations is reached, the loop ends and the result is output; otherwise, the algorithm jumps to step 3 to continue searching.
The calibrated camera parameters are used to solve the relationship between the calibration plate and the projected image.
The calibrated camera extrinsic parameter matrix is(24)r11r12r13r21r22r23r31r33r33txtytz.
The third column vectorr13r23r33T of the camera external parameter matrix in formula (24) provides a normal vector n of the calibration plane, and the last column vector txtytzT provides the coordinates of the origin P of the calibration plane. The schematic diagram of the origin coordinates and normal vector in the plane target is shown in Figure 4.
According to the mathematical formula of the plane equation in space, it can be known that the calibration plane is the set of all pointsr. The origin o represents the coordinates of a known point on the calibration plane, and n represents the nonzero normal vector of the calibration plane. Finally, the three linear equations of the calibration plate plane obtained by the characteristics of the plane equation are(25)n⋅r−o=0,n=ax^+by^+cz^,r=xx^+yy^+zz^.
In the formula,x^,y^,z^ is the unit vector and d is the dot product.(26)d=−n⋅p.
From this, the equation for the planeπ can be derived:(27)ax+by+cz+d=0.
In the formula, a, b, c, and d are known real numbers and a, b, and c are not all 0. Through the above expression, the plane equation of the calibration plate in the camera coordinate system can be obtained. The three-dimensional world coordinate point of the corner point can be obtained by solving the intersection of the ray and the plane.
In order to obtain a three-dimensional ray passing through an angleCx,Cy on the image plane of the projected image, the following transformations are made using the obtained camera parameters:(28)RxRyRz1=sKintKext−1CxCy1.
In the formula,s is the scaling factor for the rays (Rx, Ry, Rz).
As shown in Figure5, the three-dimensional position of the projection angle is obtained by determining the intersection of the ray and the calibration plane. The value of the ray coordinates is brought into the calibration plane equation to get(29)asRx+bsRy+csRz+d=0.
In the formula, except for the unknown parameters, the other parameters are known and the value of s can be obtained from this. After the value of s is determined, the three-dimensional world coordinate value corresponding to the projection angle can be obtained, so that the projector can be calibrated using an advanced camera calibration method.Figure 4
Calibration plate plane.Figure 5
Correspondence in the camera coordinate system.


### 2.3. N-Step Phase Shift Algorithm

The fringe projection system is shown in Figure6.Figure 6
Schematic diagram of structured light measurement.Structured light (FPT) measurement uses multifrequency extrapolation or Fourier transform to solve the absolute phase of the object surface by projecting multiple phase-shifted pictures. Any pixel on the plane is expressed by the phase shift formula as(30)Inx,y=I+I′=a+bcosφx,y+2πn−1N+I′.Among them, I is the cosine modulated light intensity projected by the system andI′ is the ambient light intensity component captured by the camera. 2πn−1/N is the movement amount for each shot, a is the background component, b is the fringe component, and φx,y is the phase to be solved.From formula (30), it can be concluded that the phase value at any pixel on the plane consists of two parts: one part is the phase value l that we need to solve, and the other part is the interference value I′. At present, there are relatively mature solutions for the solution of phase value 1 and the compensation of errors such as the nonlinear noise of the system. However, for the interference phase I′, less literature is involved. During measurement, inaccurate measurement of the interference phase will cause the overall light intensity Inx,y at any pixel position to change, which will ultimately affect the height measurement result. For any fixed grating projection system, since parameters such as phase shift error and system nonlinearity have been corrected (assuming the object remains stationary and ignoring intensity saturation), the height measurement error mainly comes from illumination jitter caused by changes in the measurement environment.(31)Inx,y,t=a+bcosφx,y+2πn−1N+I′x,y,t.In the formula,t is time. At this time, the external light intensity I′(x, y, t) is no longer a fixed value, but a quantity that changes with time and space. At this time, I′(x, y, t) can be divided into two categories: a is indoor natural light, and b is common indoor lighting sources (such as fluorescent lamps, etc.). We assume uniform reflections from the surface of the object. For the first type of indoor natural light, its intensity I′(x, y, t) is weaker than that of the grating structured light projected by the system and its light intensity varies randomly with time and space. Therefore, the phase can be calculated according to the above-mentioned joint formula of the ideal model, and the calculation formula remains unchanged at this time.For the second type of indoor common lighting source such as fluorescent lamps, they often carry multiple periodic harmonic components, which will interfere with the image acquisition of the target object. Moreover, compared with noise in the image, this interference is more insidious and difficult to remove by multistep phase shifting method.In order to explore the influence of ambient light intensity on the system, this paper uses the optical path shown in Figure7 to build an experimental platform. In the experiment, the light emitted by the active light source projection device first passes through the beam expander to expand the beam and then passes through the half-wave plate and the polarization beam splitter filter realizes the modulation of the spatial phase. Finally, the light is projected on the target plane through the quarter-wave plate.Figure 7
Construction of stray light intensity optical path based on polarization information.As shown in Figure7, the distances between the focus positions O1 and O2 from the test plane are both z and the horizontal distance between the two points is d. The focal plane of the main lens is set to X1,Y1, and the test plane is set to (X, Y). The Z axes of the two planes are coincident. If the point O1 is located at the origin of the X1,Y1 plane, then the coordinates of O1 and O2 in the X1,Y1 plane are O1 = (0,0,0) and O2 =(-d, 0,0), respectively. From this, it can be obtained that the complex amplitude distribution of the spherical wave divergent from points O1 and O2 in the test plane is(32)U1x,y=a0zexpjkzexpjk2zx2+y2.(33)U2x,y=a0zexpjkzexpjk2zx+d2+y2.Among them, K is the wave number anda0 is the amplitude. Formula (32) and formula (33) show that the optical field phase difference values of two orthogonally polarized lights satisfy a linear distribution. The light field at the test plane is(34)E=U1x,yU2x,y=U1x,y1expjΔψx,y.After the light field passes through the 1/4 wave plate, the phase-shifted picture is captured by the camera. Each pixel in the image plane of the camera can obtain four polarized pictures with polarization directions of 0°, 45°, 90°, and 135°. The obtained four light intensity expressions are(35)E0=1210001−j−j1E,E45=12211111−j−j1E,E90=1200011−j−j1E,E135=1221−1−111−j−j1E.If it is assumed that the imaging magnification of the light intensity received by the camera isM, the sinusoidal fringes of 0°, 45°, 90°, and 135° are finally obtained as(36)I0x,y=a02zz1+sinkxdMz+kd22z,I90x,y=a02zz1+coskxdMz+kd22z,I180x,y=a02zz1−sinkxdMz+kd22z,I270x,y=a02zz1−coskxdMz+kd22z.It can be seen from the above formula that the obtained fringe period is related to bothz and d. The calculated phase value at each pixel is(37)φx,y=artanI0x,y−I180x,yI90x,y−I270x,y.The phase obtained at this time is the wrapping phase with a value between [-π, π], which needs to be unwrapped later to obtain the continuous phase on the test plane. When there is interference from other light sources on the test plane, if the light intensity of the stray light reflected by the test plane is gx,y, the intensity captured by the camera is Ax,y=gx/M,y/M. From this, the light intensity captured by the camera can be obtained when the test plane has no polarization effect on the stray light intensity.Since the stray light will produce different polarization states after being reflected by the test plane in a complex environment, the Stokes variable of the stray light at each pixel can be calculated as(38)S0=121100110000000000M^Ax,y000,S45=121010000010100000M^Ax,y000,S90=121−100−110000000000M^Ax,y000,S135=1210−100000−10100000M^Ax,y000.Among them,M^=1m01/m00m02/m00m03/m00m10/m00m11/m00m12/m00m13/m00m20/m00m21/m00m22/m00m23/m00m30/m00m31/m00m32/m00m33/m00 is the Mueller matrix normalized to the test plane. The numerator of the matrix element represents the reflection ability of the test object to incident light, and the denominator represents the polarization ability of incident unpolarized light. From this, the light intensities of the four-phase shift diagrams can be obtained as(39)I0x,y=a02zz1+sinkxdMz+kd22z+Ax,y21+m10m00,I90x,y=a02zz1+coskxdMz+kd22z+Ax,y21+m20m00,I180x,y=a02zz1−sinkxdMz+kd22z+Ax,y21−m10m00,I270x,y=a02zz1−coskxdMz+kd22z+Ax,y21−m20m00.In order to suppress the interference of stray light in the system, it is necessary to filter the DC component of the fringes in the frequency domain of the fringes to enhance the contrast of the fringes. First, the obtained four total light intensity distribution maps are subtracted two by two to obtainI0x,y-I180x,y and I90x,y-I270x,y, which are combined into complex domain images for processing. The combined image expression is(40)cx,y=bx,y+jax,y=2a02zzexpjkxdMz+kd22z+m20m00+jm10m00Ax,y.The Fourier transform of the above formula can get F[A(x, y)] corresponding to A(x, y). The modulus value of F[A(x, y)] is defined as a quantitative index described as the suppression of stray light intensity.(41)0≤FAx,y=m102+m202m00≤1.When the value of|FAx,y is 0, it corresponds to the situation that the test plane object has no polarization effect on the stray light and the stray light interference can be completely suppressed at this time. When FAx,y is not 0 and the frequency-domain distribution of stray light overlaps with the fringe frequency to a certain extent, the subsequent fringe processing process and the final solution result will be disturbed by the stray light.


## 2.1. System Calibration

Since the imaging principle of the projector can be regarded as the inverse process of the camera imaging, the pinhole model of the camera can be used to approximate the projector model, as shown in Figure1. The three-dimensional coordinates of any point q=uvT on the object plane of the projector projected into the space are Q=X Y ZT, and their homogeneous coordinates are q˜=uv1T and Q˜=XYZ1T.Figure 1
Projector pinhole model.Based on the perspective projection model, the projection mapping of the spatial pointQ˜ to the image coordinate q˜ on the object plane can be expressed as(1)q˜=sHQ˜.Among them,q˜ and Q˜ correspond to the homogeneous coordinate forms of q and Q, respectively, and the parameter s is the scale factor. H is a homography matrix, which consists of two parts: (1) The external parameters of the projector, which are composed of a rotation matrix R and a translation vector T, can be represented by homogeneous coordinates W = [R T]. (2) The internal parameters of the projector M=fx0cx0fycy001. Among them, cx,cy is the coordinate of the principal point of the object plane, fx=f×sx and fy=f×sy are the focal lengths along the two coordinate axes of the object plane, f is the focal length of the projection lens, and sx and sy represent the number of pixels per unit distance along the two coordinate axes in the object plane. Therefore, the ideal pinhole model of the projector can be expressed as(2)q˜=sMRTQ˜.Due to the inevitable nonlinear distortion of the lens in the process of designing, processing, and assembling optical lenses, the influence of lens distortion should be considered in high-precision measurement, as shown in Figure2.Figure 2
Radial and tangential distortion of the camera.This paper mainly considers the radial distortion from the lens shape and the tangential distortion from the whole assembly process, because other distortions have less influence on the measurement accuracy. Among them, the radial distortion model can be expressed as(3)x=u1+k1r2+k2r4+k3r6,y=v1+k1r2+k2r4+k3r6.The tangential distortion model can be expressed as(4)x=u+2p1v+p2r2+2u2,y=v+p1r2+2v2+2p2u.In (3) and (4), (x, y) is the corrected pixel coordinate point, (u, v) is the original coordinate of the distortion point on the object plane, and r=u2+v2. Thus, projector lens distortion can be described by five parameters of radial distortion and tangential distortion.(5)sq˜=MRTQ˜+δr+δd.Among them,δr=xryrT represents radial distortion and δd=xdydT represents tangential distortion.The calibration of the camera is done using the planar calibration method of the two-step parameter optimization technique whenq˜ and Q˜ are known.When we assumeZ = 0, the specific internal parameter solution process is as follows:(6)xyz=sMr1r2r5TXY01=sMr1r2TXY1.A mappingH=Mr1r2T is established between the point on the calibration plane and its object plane, and the homography matrix is(7)H=h11h12h13h21h22h23h31h33h33=h1h2h3.That is,(8)h1h2h3=Mr1r2T.Becauser1 and r2 are unit orthogonal vectors, rT/1r1=rT/2r2=1 and r1T2=0. From the above formula, we can get(9)h1TM−TM−1h2=0,(10)h1TM−TM−1h1=h2TM−TM−1h2.We set(11)B=M−TM−1=B11B12B13B21B22B23B31B33B33.According to constraints (9) and (10), the two homogeneous equations of vector b can be obtained as(12)v12Tv11−v12Tb=0.Here,B=B11B12B22B13B23B33T;(13)vij=hi1hj1hi1hj2+hi2hj1hi2hj2hi3hj2+hi1hj3hi3hj1+hi2hj3hi3hj3.After acquiringn plane images, the linear equation system can be obtained according to formula (13):(14)Vb=0.In (14), b is calculated by formula (12). B can be calculated according to b, and then, the inverse operation of B is performed. Finally, the various parameters of the camera can be obtained by decomposing the result of the inverse operation. The specific calculation formula is as follows:(15)cy=B12B13−B11B23B11B22−B122,k=B33−B132+cyB12B13−B11B23B11.fx=kB11fy=kB11B11B22−B122,s=−B12fx2fyk,cx=scyfx−B13fx2k.


## 2.2. Nonlinear Optimization of Camera Parameters Based on AWMPSO

Particle swarm optimization (PSO) is a metaheuristic method obtained by studying the predation behavior of birds, due to its advantages of easy implementation, high precision, and fast convergence speed. The formula for particle update position and velocity is as follows:(16)vidt+1=wvidt+1+c1r1pbest−xidt+c2r2gbest−xidt,xidt+1=xidt+1+vidt+1.For general particle swarm optimization, due to the lack of dynamic speed adjustment ability, its global search ability needs to be further improved. Moreover, since all particles eventually converge to a position, particles get stuck in a local optimum. Therefore, this paper proposes an adaptive weight variation particle swarm optimization algorithm (AWMPSO) to improve the searchability of the algorithm.t(17)ISAid=xid−pbestpbest−dbest+ε,d=1,2……,D,(18)wid=1.4−a11+e−ISAid,(19)σ2=∑i=1nfi−favgδ2,δ=maxfi−favg,maxfi−favg>1,1,others.This operation can effectively improve the algorithm’s ability to jump out of the local optimum. For gbest satisfying the mutation condition, the calculation formula of mutation probability pm is as follows:(20)pm=k,σ2=0andfgbest>fd,0,others.Among them,fd is a constant between [0,1]. f is the desired optimal solution. For the mutation operation of gbest, the method of random perturbation is adopted.(21)gbestd=1+0.01∗η×gbestd.Among them,η represents a random variable obeying a Gaussian distribution. To sum up, in view of the shortcomings of particle swarm optimization, an ISA-based AWMPSO is proposed to dynamically adjust the inertia weight w and adaptively mutate gbest based on the population fitness variance.AWMPSO introduces dynamic adjustment weightw and mutation operation, which improves the global searchability.(22)fobj=min∑i=1n∑j=1m||pij−pijA,k1,k2,k3,p1,p2,Ri,Ti,Mij.pij(A, k1, k2, k3, p1, p2, Ri, Ti, Mij) are the two-dimensional pixel coordinates obtained by reprojecting the three-dimensional coordinates Mij by formula (22). The specific optimization steps are shown in Figure 3.Figure 3
Overall flow chart of the improved calibration method.Step 1.
Population initialization: the algorithm takes the linear solution of the camera parameters without considering lens distortion as the mean of the initial positions of N particles so that the particles are evenly distributed around the linear solution.Step 2.
Build a fitness function: the fitness function is:(23)fitness=1n×m∑i=1n∑j=1mUij−uij2+Vij−vij2.Step 3.
Dynamically adjust the inertia weight: the algorithm dynamically adjusts the inertia weightw according to formulas (18) and (19).Step 4.
Update population position: the algorithm updatesxid and vid according to formula (16).Step 5.
Mutation operation: the algorithm determines the mutation probability according to the population fitness varianceσ2 and the expected optimal value fd.Step 6.
Adjust gbest again: for the gbest that satisfies the mutation condition, the algorithm adjusts gbest again according to formula (21).Step 7.
Determine the termination condition: when the maximum number of iterations is reached, the loop ends and the result is output; otherwise, the algorithm jumps to step 3 to continue searching.
The calibrated camera parameters are used to solve the relationship between the calibration plate and the projected image.
The calibrated camera extrinsic parameter matrix is(24)r11r12r13r21r22r23r31r33r33txtytz.
The third column vectorr13r23r33T of the camera external parameter matrix in formula (24) provides a normal vector n of the calibration plane, and the last column vector txtytzT provides the coordinates of the origin P of the calibration plane. The schematic diagram of the origin coordinates and normal vector in the plane target is shown in Figure 4.
According to the mathematical formula of the plane equation in space, it can be known that the calibration plane is the set of all pointsr. The origin o represents the coordinates of a known point on the calibration plane, and n represents the nonzero normal vector of the calibration plane. Finally, the three linear equations of the calibration plate plane obtained by the characteristics of the plane equation are(25)n⋅r−o=0,n=ax^+by^+cz^,r=xx^+yy^+zz^.
In the formula,x^,y^,z^ is the unit vector and d is the dot product.(26)d=−n⋅p.
From this, the equation for the planeπ can be derived:(27)ax+by+cz+d=0.
In the formula, a, b, c, and d are known real numbers and a, b, and c are not all 0. Through the above expression, the plane equation of the calibration plate in the camera coordinate system can be obtained. The three-dimensional world coordinate point of the corner point can be obtained by solving the intersection of the ray and the plane.
In order to obtain a three-dimensional ray passing through an angleCx,Cy on the image plane of the projected image, the following transformations are made using the obtained camera parameters:(28)RxRyRz1=sKintKext−1CxCy1.
In the formula,s is the scaling factor for the rays (Rx, Ry, Rz).
As shown in Figure5, the three-dimensional position of the projection angle is obtained by determining the intersection of the ray and the calibration plane. The value of the ray coordinates is brought into the calibration plane equation to get(29)asRx+bsRy+csRz+d=0.
In the formula, except for the unknown parameters, the other parameters are known and the value of s can be obtained from this. After the value of s is determined, the three-dimensional world coordinate value corresponding to the projection angle can be obtained, so that the projector can be calibrated using an advanced camera calibration method.Figure 4
Calibration plate plane.Figure 5
Correspondence in the camera coordinate system.


## 2.3. N-Step Phase Shift Algorithm

The fringe projection system is shown in Figure6.Figure 6
Schematic diagram of structured light measurement.Structured light (FPT) measurement uses multifrequency extrapolation or Fourier transform to solve the absolute phase of the object surface by projecting multiple phase-shifted pictures. Any pixel on the plane is expressed by the phase shift formula as(30)Inx,y=I+I′=a+bcosφx,y+2πn−1N+I′.Among them, I is the cosine modulated light intensity projected by the system andI′ is the ambient light intensity component captured by the camera. 2πn−1/N is the movement amount for each shot, a is the background component, b is the fringe component, and φx,y is the phase to be solved.From formula (30), it can be concluded that the phase value at any pixel on the plane consists of two parts: one part is the phase value l that we need to solve, and the other part is the interference value I′. At present, there are relatively mature solutions for the solution of phase value 1 and the compensation of errors such as the nonlinear noise of the system. However, for the interference phase I′, less literature is involved. During measurement, inaccurate measurement of the interference phase will cause the overall light intensity Inx,y at any pixel position to change, which will ultimately affect the height measurement result. For any fixed grating projection system, since parameters such as phase shift error and system nonlinearity have been corrected (assuming the object remains stationary and ignoring intensity saturation), the height measurement error mainly comes from illumination jitter caused by changes in the measurement environment.(31)Inx,y,t=a+bcosφx,y+2πn−1N+I′x,y,t.In the formula,t is time. At this time, the external light intensity I′(x, y, t) is no longer a fixed value, but a quantity that changes with time and space. At this time, I′(x, y, t) can be divided into two categories: a is indoor natural light, and b is common indoor lighting sources (such as fluorescent lamps, etc.). We assume uniform reflections from the surface of the object. For the first type of indoor natural light, its intensity I′(x, y, t) is weaker than that of the grating structured light projected by the system and its light intensity varies randomly with time and space. Therefore, the phase can be calculated according to the above-mentioned joint formula of the ideal model, and the calculation formula remains unchanged at this time.For the second type of indoor common lighting source such as fluorescent lamps, they often carry multiple periodic harmonic components, which will interfere with the image acquisition of the target object. Moreover, compared with noise in the image, this interference is more insidious and difficult to remove by multistep phase shifting method.In order to explore the influence of ambient light intensity on the system, this paper uses the optical path shown in Figure7 to build an experimental platform. In the experiment, the light emitted by the active light source projection device first passes through the beam expander to expand the beam and then passes through the half-wave plate and the polarization beam splitter filter realizes the modulation of the spatial phase. Finally, the light is projected on the target plane through the quarter-wave plate.Figure 7
Construction of stray light intensity optical path based on polarization information.As shown in Figure7, the distances between the focus positions O1 and O2 from the test plane are both z and the horizontal distance between the two points is d. The focal plane of the main lens is set to X1,Y1, and the test plane is set to (X, Y). The Z axes of the two planes are coincident. If the point O1 is located at the origin of the X1,Y1 plane, then the coordinates of O1 and O2 in the X1,Y1 plane are O1 = (0,0,0) and O2 =(-d, 0,0), respectively. From this, it can be obtained that the complex amplitude distribution of the spherical wave divergent from points O1 and O2 in the test plane is(32)U1x,y=a0zexpjkzexpjk2zx2+y2.(33)U2x,y=a0zexpjkzexpjk2zx+d2+y2.Among them, K is the wave number anda0 is the amplitude. Formula (32) and formula (33) show that the optical field phase difference values of two orthogonally polarized lights satisfy a linear distribution. The light field at the test plane is(34)E=U1x,yU2x,y=U1x,y1expjΔψx,y.After the light field passes through the 1/4 wave plate, the phase-shifted picture is captured by the camera. Each pixel in the image plane of the camera can obtain four polarized pictures with polarization directions of 0°, 45°, 90°, and 135°. The obtained four light intensity expressions are(35)E0=1210001−j−j1E,E45=12211111−j−j1E,E90=1200011−j−j1E,E135=1221−1−111−j−j1E.If it is assumed that the imaging magnification of the light intensity received by the camera isM, the sinusoidal fringes of 0°, 45°, 90°, and 135° are finally obtained as(36)I0x,y=a02zz1+sinkxdMz+kd22z,I90x,y=a02zz1+coskxdMz+kd22z,I180x,y=a02zz1−sinkxdMz+kd22z,I270x,y=a02zz1−coskxdMz+kd22z.It can be seen from the above formula that the obtained fringe period is related to bothz and d. The calculated phase value at each pixel is(37)φx,y=artanI0x,y−I180x,yI90x,y−I270x,y.The phase obtained at this time is the wrapping phase with a value between [-π, π], which needs to be unwrapped later to obtain the continuous phase on the test plane. When there is interference from other light sources on the test plane, if the light intensity of the stray light reflected by the test plane is gx,y, the intensity captured by the camera is Ax,y=gx/M,y/M. From this, the light intensity captured by the camera can be obtained when the test plane has no polarization effect on the stray light intensity.Since the stray light will produce different polarization states after being reflected by the test plane in a complex environment, the Stokes variable of the stray light at each pixel can be calculated as(38)S0=121100110000000000M^Ax,y000,S45=121010000010100000M^Ax,y000,S90=121−100−110000000000M^Ax,y000,S135=1210−100000−10100000M^Ax,y000.Among them,M^=1m01/m00m02/m00m03/m00m10/m00m11/m00m12/m00m13/m00m20/m00m21/m00m22/m00m23/m00m30/m00m31/m00m32/m00m33/m00 is the Mueller matrix normalized to the test plane. The numerator of the matrix element represents the reflection ability of the test object to incident light, and the denominator represents the polarization ability of incident unpolarized light. From this, the light intensities of the four-phase shift diagrams can be obtained as(39)I0x,y=a02zz1+sinkxdMz+kd22z+Ax,y21+m10m00,I90x,y=a02zz1+coskxdMz+kd22z+Ax,y21+m20m00,I180x,y=a02zz1−sinkxdMz+kd22z+Ax,y21−m10m00,I270x,y=a02zz1−coskxdMz+kd22z+Ax,y21−m20m00.In order to suppress the interference of stray light in the system, it is necessary to filter the DC component of the fringes in the frequency domain of the fringes to enhance the contrast of the fringes. First, the obtained four total light intensity distribution maps are subtracted two by two to obtainI0x,y-I180x,y and I90x,y-I270x,y, which are combined into complex domain images for processing. The combined image expression is(40)cx,y=bx,y+jax,y=2a02zzexpjkxdMz+kd22z+m20m00+jm10m00Ax,y.The Fourier transform of the above formula can get F[A(x, y)] corresponding to A(x, y). The modulus value of F[A(x, y)] is defined as a quantitative index described as the suppression of stray light intensity.(41)0≤FAx,y=m102+m202m00≤1.When the value of|FAx,y is 0, it corresponds to the situation that the test plane object has no polarization effect on the stray light and the stray light interference can be completely suppressed at this time. When FAx,y is not 0 and the frequency-domain distribution of stray light overlaps with the fringe frequency to a certain extent, the subsequent fringe processing process and the final solution result will be disturbed by the stray light.


## 3. Virtual Reality Products Based on Computer-Aided Design

User experience-based product innovation information resource integration requires user experience design on the basis of information resource integration. The information resource integration model based on user experience can be summarized in Figure8(a).Figure 8
Virtual reality product design model. (a) Virtual simulation resource integration model. (b) Integrated model for virtual simulation design system evaluation.
(a)(b)Borrowing the cost-benefit theory in economics, this paper analyzes the motivation of users to interact, the satisfaction of virtual simulation experience, and the relationship between these elements and constructs a system evaluation of virtual simulation design based on product innovation, as shown in Figure8(b).On the basis of the above research, the effect of the virtual reality product development system proposed in this paper is verified and the effect is evaluated through multiple groups of designs. The statistical test results are shown in Table1 and Figure 9.Table 1
Design effects of virtual products.
NumDesign effectNumDesign effectNumDesign effect179.881291.072381.17280.441384.532480.38380.861483.072587.88488.671589.682680.22587.811680.462784.92691.281784.012879.35784.121886.452981.76891.901990.723089.05987.892083.503185.181085.172183.203283.201183.482288.993387.52Figure 9
Evaluation of virtual product design.Through the above experimental research and analysis, it can be seen that the virtual reality product design system based on computer-aided technology proposed in this paper has good effects and can effectively improve the effect of virtual reality product design.


## 4. Conclusion

With the return of user personalization, users have higher and higher requirements for experience design and product design needs to meet users’ requirements for “interaction, real-time, precision, and personalization.” However, the traditional design cannot easily meet the user’s experience needs from the technical and conceptual level. With the continuous development of virtual simulation technology, building a product innovation design based on a virtual simulation system can improve human-computer interaction and provide deep user experience. This paper combines the computer-aided design method to build a virtual reality product development system to improve the design interactivity of virtual reality products. The experimental research analysis shows that the virtual reality product design system based on computer-aided technology proposed in this paper has good effects and can effectively improve the effect of virtual reality product design.



---

*Source: 9489650-2022-08-31.xml*