# Three-Dimensional Modeling Design and Color Correction Algorithm of Packaging Structure Based on Human Visual Characteristics

Authors: Jinping Liu
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Product packaging design is a work that integrates graphic design and three-dimensional design. 3D visualization technology and digital virtual interaction technology in virtual reality are developing rapidly in packaging design applications. On the one hand, this study discusses the application of digital technology in the three-dimensional design of packaging structure, including packaging carton structure design, packaging container modeling design, and packaging two-dimensional visual design. On the other hand, this study studies the color correction algorithm of packaging design and proposes a color correction based on the visual characteristics of human eyes. The pattern search algorithm finds a local optimum, and the initial value affects the correction result. Therefore, inspired by the color perception properties of the human visual system, the image is corrected by combining visual multichannel and visual nonlinearity. The experimental results of the algorithm are more consistent with human visual perception.


---

## Body

## 1. Introduction

In the process of product development, excellent packaging design is essential. By comprehensively considering the function, shape, and material of the product, choosing appropriate packaging materials, graphic design, and matching with ingenious craftsmanship, the image of the entire product is more attractive [1]. It can be said that packaging design is another mean to enhance the affinity between products and consumers in addition to the functional characteristics of products. Excellent packaging design can significantly enhance the value and image of products and stand out among similar products. The three-dimensional modeling design of packaging by means of computer technology is the current mainstream research direction [2]. Compared with graphic design, three-dimensional modeling can better show the characteristics of packaging and make the design process more scientific. However, in the existing research, the three-dimensional modeling design of the packaging structure has the problem of poor image quality [3]. In this study, starting from the characteristics of human eye brightness adaptability, visual nonlinearity, contrast sensitivity, masking effect, and color perception and brightness perception in visual perception, a research on color correction and highlight removal based on human visual characteristics is proposed for color and illumination uniformity. Algorithms make the improved images more in line with the characteristics of human eyes, better visual effects, and more suitable for computer processing.


## 2. 3D Packaging Design and Implementation

The application field of visualization technology is very wide, and there are many technical categories [4]. The visualization technology in this study is researched in the field of 3D model technology. There are many types of technology platforms in this field; commonly used ones include 3DS MAX, MAYA, LightWave 3D, Softimage XSI, SolidEdge, and Solidwork [5]. The 3DS MAX software platform is selected for this study, the reason is that the model data of this software is relatively stable when exported to Sun3d, and there are many data types that can be exported, and the speed is fast. 3DS MAX is one of the three-dimensional modeling software programs in the world. It has functions such as modeling, rendering, and animation effects and is widely used in the performance of three-dimensional visual effects [6].


### 2.1. Packaging 2D Visual Design

Packaging two-dimensional visual design is to design the external graphics, text, and color elements according to the three-dimensional packaging model structure. The design of the two-dimensional textures in this article is all drawn in Photoshop, which can be used for both drawing and typesetting.


#### 2.1.1. Textured Square Box Design

The stickers designed for the square box are the festive series and the flavor series. The specific drawing method is to arrange visual elements such as graphics, text, and color, according to the structure position on the expanded view of the packaging box, and the size ratio is correct, so that after the 3D model is completed, the UV Map positioning can accurately paste the model and complete the three-dimensional packaging design. Choose to typeset in PS software here. The design of the Chinese Spring Festival theme texture is shown in Figure1, and other texture design methods are similar.Figure 1
Map drawing: spring festival.


#### 2.1.2. Column Container Texture Design

The texture design of the alien container design is mainly the arrangement of point elements, and the cute, lively, and interesting visual effects are in line with the characteristics of chocolate products. First draw various element graphics, arrange them according to the rules of formal beauty in the layout space, and form a color series through the combination of color changes to express different flavors of chocolate. Take the milk chocolate packaging texture as an example (Figure2).Figure 2
Map design.


#### 2.1.3. Variable Shape Texture Design

Be especially careful when designing textures for changing shapes, because the shape is irregular, and the design of the unfold must be accurate. The shape and structure design should be reasonable.


### 2.2. Model Design

In the chocolate packaging design, the carton shape structure design and the container shape design are the most important and important modeling contents [7]. There are many types of carton shapes and structures. In addition to geometric shapes, there are many special-shaped boxes. In the design of the three-dimensional packaging teaching system, square box packaging, cylindrical box packaging, and variable modeling packaging are mainly selected. In the future teaching process, both modeling and mapping can be expanded. The object model is made by 3DMAX software, referring to the precise data of the packaging design, and the length, width, and height are used to make the finished product.


#### 2.2.1. Square Box Model

First, create a box in the standard Primitives panel to make the box size and shape we need. Because of the material thickness of the carton, the corners have an arc, which is called chamfering in 3DMAX. In the edit ploy editing mode, enter the face editing mode, select the top face of the model, and find the Bevel command in the editing panel to chamfer the face on the model. After the chamfering is completed, perform a mirror copy. Copy out the lower half of the model and scale it manually to ensure that the box is 1–2% smaller than the above and finally get the shape of the box we need, as shown in Figure3.Figure 3
Square box.


#### 2.2.2. Columnar Packaging Container Model

The model includes two parts: the column body and the cover. Make a barrel shape from Tube in the standard Primitives panel. In the online editing mode, select the bottom face of the model, select a bottom edge and select the loop command to get all the lines on the inner edge, perform infinite scaling until all the lines become a point, and finally make a chamfered column body. Create chamfer in the Extended Primitives panel to make the shape of the bucket lid we need. The complete cylindrical packaging container model design is shown in Figure4.Figure 4
Make chamfer.


#### 2.2.3. Multivariable Packaging Container Model

The multivariable packaging container shape includes a tapered body part and a special-shaped lid part. Create the cylinder in the standard Primitives panel to make the desired barrel shape, edit the nodes, perform zooming and moving, adjust the position of the nodes according to the shape analysis of the special-shaped part, complete the general outer surface, and smooth the outer surface through MeshSmooth to obtain a smooth shape. The special-shaped model is completed, and the edited special-shaped packaging container model is shown in Figure5.Figure 5
Special-shaped packaging container models.


#### 2.2.4. Model of a Triangular Box

The model of the triangular box is modeled in a way that approximates the square box, as shown in Figure6.Figure 6
Triangle box.


#### 2.2.5. Model of Chocolate Ball

The model of the chocolate ball has wrinkled wrapping paper. Use line in the spline panel to draw the outer vertical edge line of the chocolate wrapping paper, use the extrude command to pull out a certain height, and select edit ploy in the modifier list to scale the lower row of nodes to obtain the toothed edge shape. The shape of the chocolate ball is round and slightly irregular, as shown in Figure7.Figure 7
Chocolate model of the ball.


#### 2.2.6. Modeling of Square Inner Packaging Container

The inner packing box is a container that can hold 16 chocolate balls. The material is plastic and the color is golden yellow. The method of making base map by reference design is used to model the inner packaging container. First cut the model with the shape of chocolate, set out the individual model details, and gradually edit ploy to copy and make the remaining models one by one. After completing the model editing, carry out Boolean operation to dig out the shape of chocolate box, as shown in Figure8.Figure 8
Equipment production.After the model modeling is completed in 3DMAX, it is necessary to give the model positioning material map, which can not only make the map stick correctly, but also continue to add the design map of the same modeling structure and standard size to the system in the future. The map can be changed to the model through interactive control. The method is to edit the UVWs of the POLY object to make the model map in a reasonable position. UVWs editing of packaging map here takes the square box as an example, and the UVWs editing method of other modeling models is similar. Find the texture that has been made, find Unwrap UVW in the Modifier List drop-down menu, and enter the texture editing mode. Select open UV Editor to bring up the panel on the left for editing UVs. Select the base map that has been made, and edit the positions of each point of the model in place. After editing, the mapping on the model is completed, and the final model mapping is obtained, as shown in Figure9.Figure 9
UVWs editing.The final effect of the square box model after UVWs editing is shown in Figure10.Figure 10
UVWs editing.After UVWs editing, the models of various shapes are correctly mapped. The inner packaging model is shown in Figure11.Figure 11
Three-dimensional design of inner packaging container.


## 2.1. Packaging 2D Visual Design

Packaging two-dimensional visual design is to design the external graphics, text, and color elements according to the three-dimensional packaging model structure. The design of the two-dimensional textures in this article is all drawn in Photoshop, which can be used for both drawing and typesetting.


### 2.1.1. Textured Square Box Design

The stickers designed for the square box are the festive series and the flavor series. The specific drawing method is to arrange visual elements such as graphics, text, and color, according to the structure position on the expanded view of the packaging box, and the size ratio is correct, so that after the 3D model is completed, the UV Map positioning can accurately paste the model and complete the three-dimensional packaging design. Choose to typeset in PS software here. The design of the Chinese Spring Festival theme texture is shown in Figure1, and other texture design methods are similar.Figure 1
Map drawing: spring festival.


### 2.1.2. Column Container Texture Design

The texture design of the alien container design is mainly the arrangement of point elements, and the cute, lively, and interesting visual effects are in line with the characteristics of chocolate products. First draw various element graphics, arrange them according to the rules of formal beauty in the layout space, and form a color series through the combination of color changes to express different flavors of chocolate. Take the milk chocolate packaging texture as an example (Figure2).Figure 2
Map design.


### 2.1.3. Variable Shape Texture Design

Be especially careful when designing textures for changing shapes, because the shape is irregular, and the design of the unfold must be accurate. The shape and structure design should be reasonable.


## 2.1.1. Textured Square Box Design

The stickers designed for the square box are the festive series and the flavor series. The specific drawing method is to arrange visual elements such as graphics, text, and color, according to the structure position on the expanded view of the packaging box, and the size ratio is correct, so that after the 3D model is completed, the UV Map positioning can accurately paste the model and complete the three-dimensional packaging design. Choose to typeset in PS software here. The design of the Chinese Spring Festival theme texture is shown in Figure1, and other texture design methods are similar.Figure 1
Map drawing: spring festival.


## 2.1.2. Column Container Texture Design

The texture design of the alien container design is mainly the arrangement of point elements, and the cute, lively, and interesting visual effects are in line with the characteristics of chocolate products. First draw various element graphics, arrange them according to the rules of formal beauty in the layout space, and form a color series through the combination of color changes to express different flavors of chocolate. Take the milk chocolate packaging texture as an example (Figure2).Figure 2
Map design.


## 2.1.3. Variable Shape Texture Design

Be especially careful when designing textures for changing shapes, because the shape is irregular, and the design of the unfold must be accurate. The shape and structure design should be reasonable.


## 2.2. Model Design

In the chocolate packaging design, the carton shape structure design and the container shape design are the most important and important modeling contents [7]. There are many types of carton shapes and structures. In addition to geometric shapes, there are many special-shaped boxes. In the design of the three-dimensional packaging teaching system, square box packaging, cylindrical box packaging, and variable modeling packaging are mainly selected. In the future teaching process, both modeling and mapping can be expanded. The object model is made by 3DMAX software, referring to the precise data of the packaging design, and the length, width, and height are used to make the finished product.


### 2.2.1. Square Box Model

First, create a box in the standard Primitives panel to make the box size and shape we need. Because of the material thickness of the carton, the corners have an arc, which is called chamfering in 3DMAX. In the edit ploy editing mode, enter the face editing mode, select the top face of the model, and find the Bevel command in the editing panel to chamfer the face on the model. After the chamfering is completed, perform a mirror copy. Copy out the lower half of the model and scale it manually to ensure that the box is 1–2% smaller than the above and finally get the shape of the box we need, as shown in Figure3.Figure 3
Square box.


### 2.2.2. Columnar Packaging Container Model

The model includes two parts: the column body and the cover. Make a barrel shape from Tube in the standard Primitives panel. In the online editing mode, select the bottom face of the model, select a bottom edge and select the loop command to get all the lines on the inner edge, perform infinite scaling until all the lines become a point, and finally make a chamfered column body. Create chamfer in the Extended Primitives panel to make the shape of the bucket lid we need. The complete cylindrical packaging container model design is shown in Figure4.Figure 4
Make chamfer.


### 2.2.3. Multivariable Packaging Container Model

The multivariable packaging container shape includes a tapered body part and a special-shaped lid part. Create the cylinder in the standard Primitives panel to make the desired barrel shape, edit the nodes, perform zooming and moving, adjust the position of the nodes according to the shape analysis of the special-shaped part, complete the general outer surface, and smooth the outer surface through MeshSmooth to obtain a smooth shape. The special-shaped model is completed, and the edited special-shaped packaging container model is shown in Figure5.Figure 5
Special-shaped packaging container models.


### 2.2.4. Model of a Triangular Box

The model of the triangular box is modeled in a way that approximates the square box, as shown in Figure6.Figure 6
Triangle box.


### 2.2.5. Model of Chocolate Ball

The model of the chocolate ball has wrinkled wrapping paper. Use line in the spline panel to draw the outer vertical edge line of the chocolate wrapping paper, use the extrude command to pull out a certain height, and select edit ploy in the modifier list to scale the lower row of nodes to obtain the toothed edge shape. The shape of the chocolate ball is round and slightly irregular, as shown in Figure7.Figure 7
Chocolate model of the ball.


### 2.2.6. Modeling of Square Inner Packaging Container

The inner packing box is a container that can hold 16 chocolate balls. The material is plastic and the color is golden yellow. The method of making base map by reference design is used to model the inner packaging container. First cut the model with the shape of chocolate, set out the individual model details, and gradually edit ploy to copy and make the remaining models one by one. After completing the model editing, carry out Boolean operation to dig out the shape of chocolate box, as shown in Figure8.Figure 8
Equipment production.After the model modeling is completed in 3DMAX, it is necessary to give the model positioning material map, which can not only make the map stick correctly, but also continue to add the design map of the same modeling structure and standard size to the system in the future. The map can be changed to the model through interactive control. The method is to edit the UVWs of the POLY object to make the model map in a reasonable position. UVWs editing of packaging map here takes the square box as an example, and the UVWs editing method of other modeling models is similar. Find the texture that has been made, find Unwrap UVW in the Modifier List drop-down menu, and enter the texture editing mode. Select open UV Editor to bring up the panel on the left for editing UVs. Select the base map that has been made, and edit the positions of each point of the model in place. After editing, the mapping on the model is completed, and the final model mapping is obtained, as shown in Figure9.Figure 9
UVWs editing.The final effect of the square box model after UVWs editing is shown in Figure10.Figure 10
UVWs editing.After UVWs editing, the models of various shapes are correctly mapped. The inner packaging model is shown in Figure11.Figure 11
Three-dimensional design of inner packaging container.


## 2.2.1. Square Box Model

First, create a box in the standard Primitives panel to make the box size and shape we need. Because of the material thickness of the carton, the corners have an arc, which is called chamfering in 3DMAX. In the edit ploy editing mode, enter the face editing mode, select the top face of the model, and find the Bevel command in the editing panel to chamfer the face on the model. After the chamfering is completed, perform a mirror copy. Copy out the lower half of the model and scale it manually to ensure that the box is 1–2% smaller than the above and finally get the shape of the box we need, as shown in Figure3.Figure 3
Square box.


## 2.2.2. Columnar Packaging Container Model

The model includes two parts: the column body and the cover. Make a barrel shape from Tube in the standard Primitives panel. In the online editing mode, select the bottom face of the model, select a bottom edge and select the loop command to get all the lines on the inner edge, perform infinite scaling until all the lines become a point, and finally make a chamfered column body. Create chamfer in the Extended Primitives panel to make the shape of the bucket lid we need. The complete cylindrical packaging container model design is shown in Figure4.Figure 4
Make chamfer.


## 2.2.3. Multivariable Packaging Container Model

The multivariable packaging container shape includes a tapered body part and a special-shaped lid part. Create the cylinder in the standard Primitives panel to make the desired barrel shape, edit the nodes, perform zooming and moving, adjust the position of the nodes according to the shape analysis of the special-shaped part, complete the general outer surface, and smooth the outer surface through MeshSmooth to obtain a smooth shape. The special-shaped model is completed, and the edited special-shaped packaging container model is shown in Figure5.Figure 5
Special-shaped packaging container models.


## 2.2.4. Model of a Triangular Box

The model of the triangular box is modeled in a way that approximates the square box, as shown in Figure6.Figure 6
Triangle box.


## 2.2.5. Model of Chocolate Ball

The model of the chocolate ball has wrinkled wrapping paper. Use line in the spline panel to draw the outer vertical edge line of the chocolate wrapping paper, use the extrude command to pull out a certain height, and select edit ploy in the modifier list to scale the lower row of nodes to obtain the toothed edge shape. The shape of the chocolate ball is round and slightly irregular, as shown in Figure7.Figure 7
Chocolate model of the ball.


## 2.2.6. Modeling of Square Inner Packaging Container

The inner packing box is a container that can hold 16 chocolate balls. The material is plastic and the color is golden yellow. The method of making base map by reference design is used to model the inner packaging container. First cut the model with the shape of chocolate, set out the individual model details, and gradually edit ploy to copy and make the remaining models one by one. After completing the model editing, carry out Boolean operation to dig out the shape of chocolate box, as shown in Figure8.Figure 8
Equipment production.After the model modeling is completed in 3DMAX, it is necessary to give the model positioning material map, which can not only make the map stick correctly, but also continue to add the design map of the same modeling structure and standard size to the system in the future. The map can be changed to the model through interactive control. The method is to edit the UVWs of the POLY object to make the model map in a reasonable position. UVWs editing of packaging map here takes the square box as an example, and the UVWs editing method of other modeling models is similar. Find the texture that has been made, find Unwrap UVW in the Modifier List drop-down menu, and enter the texture editing mode. Select open UV Editor to bring up the panel on the left for editing UVs. Select the base map that has been made, and edit the positions of each point of the model in place. After editing, the mapping on the model is completed, and the final model mapping is obtained, as shown in Figure9.Figure 9
UVWs editing.The final effect of the square box model after UVWs editing is shown in Figure10.Figure 10
UVWs editing.After UVWs editing, the models of various shapes are correctly mapped. The inner packaging model is shown in Figure11.Figure 11
Three-dimensional design of inner packaging container.


## 3. Color Correction Method of Three-Dimensional Packaging Design

Affected by the light source, camera, and color temperature, there will be differences between the 3D design image and what the human eye sees in reality, which will affect the subsequent image processing to a certain extent, such as feature extraction and classification, so the quality of 3D design imaging is particularly critical [8]. Compared with sound, text, and even gray image, the information is richer, and the visual effect is better. Considering the cost, when the image obtained by three-dimensional design software is determined, the obtained original image will generally be color corrected to obtain better imaging effect.


### 3.1. Reasons Affecting Color Distortion

In general, the packaging image designed by 3D software will be different from that seen by human eyes in reality, the spatial brightness distribution is uneven, and the color will deviate [9]. To a certain extent, the authenticity of the color will affect the later image processing, such as feature extraction, classification, and so on [10]. Therefore, we need to first understand why the acquired image color distortion can be targeted to solve and finally obtain the real image closest to the human eye. In terms of optical image imaging principle, the imaging paradigm used in 3D design software is the same as that of traditional camera, which is imaged on the focal plane through reflected light. However, because the photosensitive medium is different, it will be different in actual imaging. In addition, some of the sensors of the camera use CCD and some use CMOS, resulting in different sensing characteristics, different optical characteristics of the lens, focal length, field angle, and relative aperture, different characteristics of light source chromaticity, and different image scanning direction, which will directly affect the quality of the image and make the collected industrial image have color deviation from that seen by human eyes in reality.


### 3.2. Classic Color Correction Method

#### 3.2.1. Color Correction Based on Polynomial Regression

Polynomial regression analysis, the most widely used method in statistics, has solved many practical problems, large and small. Through continuous observation, experiment, test, and summary, regression analysis can find out the interdependence of different variables. Regression analysis is generally divided into nonlinear and linear.This study adopts the form of linear regression, and the mathematical model can be expressed as(1)Y=Xβ+Z,where Y is the standard value of the color card standard target of the observation object, β=β1,β2,…,βp is the linear regression parameter, X is the independent variable, the size is of order N×P, and Z is the random value. Among them,(2)X=x11x12x13…x1px21x22x23…x2p……………xN1xN2xN3…xNp.Since the expected value of the random disturbance vectorZ can be put into the mathematical model Xβ, it is assumed that(3)EZ=0.It is equivalent to(4)EY=Xβ.Note that, in a linear regression model, once a parameter is derived, the derivative has nothing to do with the parameter [11]. When using the linear regression model to solve the color correction problem, we must first know the color value of the standard color card, which is the vector Y in the regression model.The required parameterβ should satisfy the following formula:(5)minY−βX.Among them,⋅ represents the second-order norm, and the solution β obtained by the least square method is the parameter matrix required in this study.Taking the matrix of size3×3 as a reference to illustrate how to obtain the model parameters, the first is(6)X1Y1Z1X2Y2Z2………XNYNZN=R1G1B1R2G2B2………RNGNBNTa11a12a13a21a22a23a31a32a33,where the parameters satisfy(7)X=R1G1B1R2G2B2R3G3B3TY=X1X2X3β=a11a21a31T.First, taking the first column of the model coefficient matrixβ1 as an example, there are(8)Y=Xβ1.The solutionβ1 obtained from the above formula should conform to the following equation:(9)XTY=XTXβ1,which is(10)β1=XTX−1XTY.Similarly, the values of other columns of the model coefficient matrix can be solved.From the above introduction, the most classic color correction process can be summarized as follows:(1)
In the experiment process, it is necessary to design the color card standard target first. The selection of the standard target should include all possible color gamuts as much as possible to ensure the accuracy of the experimental results.(2)
Determine the specific chromaticity value of the color card, that is, Y in the analysis process of the above formula. This article selects the xRite 24-color color card ColorChecker as the standard value.(3)
Get the RGB value to be corrected(4)
The model parameter matrixβ is obtained by the method of polynomial regression


#### 3.2.2. Constrained Least Squares Color Correction

Although the least squares method can ensure that all colors are as close as possible, it cannot guarantee that the white block area will remain white after correction, and a primary consideration for the success of color correction is the accurate reproduction of the white block [12]. Therefore, this study adds the constraint that the white block remains unchanged before and after correction.In the least squares method,X¯=x¯,y¯,z¯ is the q×3 matrix, and in this method it is the set of corrected XYZ values. X=x,y,z is also a q×3 matrix, which is the set of raw RGB values to be corrected. In the case of a linear map, the residual sum of squares can be used to minimize the linear map, as follows:(11)F=X¯−XM22Among them,M is the linear parameter matrix of 3×3 to be solved, and the above formula is equivalent to minimizing the residual, as follows:(12)ε2=∑i=13X¯i−∑k=13mi,kXk2.If a column element of linear parameterM is required, the following formula shall be minimized:(13)∂ε2∂mi,k=0,k=1,2,3Therefore, the linear parameter matrixM can be summarized as follows:(14)∑i=1qXi12∑i=1qXi1Xi2∑i=1qXi1Xi3∑i=1qXi2Xi1∑i=1qXi22∑i=1qXi2Xi3∑i=1qXi3Xi1∑i=1qXi3Xi2∑i=1qXi32m1,km2,km3,k=∑i=1qXi1X¯ik∑i=1qXi2X¯ik∑i=1qXi3X¯ik.The above formula can be rewritten into a more concise form:BM=d, where B means(15)B=XTX,X=X11X12X13………Xq1Xq2Xq3.where d can be expressed as(16)d=XTX¯,X¯=X¯11X¯12X¯13………X¯q1X¯q2X¯q3.Therefore, the linear parameter matrixM required in this study is as follows:(17)M=XTX−1XTX¯.However, the classical least square method cannot remove the noise, resulting in inaccurate correction, and the corrected white block area cannot remain white. The primary consideration of the success of color correction is the accurate reproduction of white blocks. Therefore, this study adds the constraint that the white block remains unchanged before and after correction.In the constrained least squares method,X¯ represents the set of XYZ values after correction. X represents the set of original RGB values to be corrected. For the constraint term, the RGB value of the original white block is represented by points, and the XYZ value after correction of the corresponding color block is represented by x¯C. Lagrange multiplier is added to make the constrained white area correct accurately, as shown in the following equation:(18)F=X¯−XM22+μ∑k=13x¯kC−∑j=13xjCmjk.μ represents the Lagrange multiplier, which ensures that the constrained white area xc can be accurately corrected to x¯C. At the same time, the following formula must be met:(19)x¯CT−xCTM=0.


### 3.3. Color Correction Based on the Pattern Search Optimization Algorithm

In order to obtain a more consistent color image with human visual perception, this study combines polynomial regression analysis with CIEDE2000 color difference calculation to create a new color correction algorithm [13]. The algorithm constructs a linear parameter matrix m to obtain the relationship between the original RGB three-channel value and the corresponding standard XYZ three-channel value.


#### 3.3.1. Select Color Card Standard Target

In real life, for a fixed color object, some people feel green, and others feel yellow, which varies from person to person. In the process of calibration, a standard value is needed as the calibration, so this standard value cannot be seen by human eyes and can only be used as the quantitative analysis standard by using the inherent properties of color obtained by specific instruments in the specified environment. This study selects the xRite Aiseli 24-color card ColorChecker under D65 light source as the standard.


#### 3.3.2. CIEDE2000 Color Difference Calculation

The commonly used color difference calculation, such as Euclidean distance, can not truly reflect the reality because of its inherent nonuniformity and other defects. Therefore, in this algorithm, a new CIEDE2000 color difference equation closest to human visual perception is used. CIEDE2000 color difference calculation method needs to convert the initial color space from RGB to CIEXYZ color space by default.CIEDE2000 color difference formula is as follows:(20)ΔE=ΔL′KLSL2+ΔCab′KCSC2+ΔHab′KHSH2+RTΔCab′KCSCΔHab′KHSH.


#### 3.3.3. Build Model

Establish Correction Model. Here, a polynomial regression method is used to establish the relationship from the initial measurement of the sample color to the standard value. The actual practice is to use the matrix Q=X,Y,Z to represent the standard XYZ value, the size is q×3, and q is the total number of color swatches participating in the calibration. Similarly, an q×3 matrix P=B,G,R is used to represent the initial RGB values obtained by the camera. Studies have shown that there is a linear regression parameter M obeying the relationship from the initial measurement of the sample color to the standard value, as follows:(21)X,Y,Z=B,G,RM.Among them, the form of matrixM is(22)M=a11a12a13a21a22a23a31a32a33.Typically, polynomial regression is used to solve the above equations. If the image corrected by the linear parameter matrixM is to be most in line with human visual perception, in terms of the overall visual perception difference, the difference between the two images must be reduced, so that the corrected XYZ value and the standard XYZ value are two. The smaller the difference between them the better. Therefore, when establishing the objective function, the average value of the color difference between the corrected XYZ value and the standard XYZ value should be used as an optimization item.In order to remove the effect of dark current, the color correction model is redefined as follows:(23)X,Y,Z=B−τb,G−τg,R−τrM.Among them,τb, τg, and τr are the average values of dark current, generally in the RGB color space.Establish Objective Function. After the correction model has been determined, the determined objective function needs to be established. Based on the in-depth study of the visual characteristics of human eyes, a color correction method based on pattern search is proposed. Compared with the classical color correction method, the new method is more consistent with the color perception of human eyes. From the above analysis, in order to solve the linear regression parameter matrix M, we need to know the matrices P and Q on both sides of the equation. Therefore, this study establishes an objective function, in which matrix Q¯ is q×3 matrix, which is the standard XYZ value, and is also the matrix, which is the corrected XYZ value. Finally, this study constructs a linear parameter matrix M of 3∗3, which minimizes the standard XYZ value and the corrected XYZ value.If the image corrected by the linear parameter matrix is to be most in line with human visual perception, in terms of the overall visual perception difference, the difference between the two images must be reduced to make the difference in XYZ values as small as possible. Therefore, when establishing the objective function, the average value of the color difference between the corrected XYZ value and the standard XYZ value should be used as an optimization item. However, in this study, the average value of color difference is only used as one of the optimization items, and the first-order normQ¯−Ql to be optimized is also added, and the infinite norm Q¯−Q∞ is also added to make the maximum error that has occurred as small as possible. Finally, this study also adds the standard deviation into the stdevQ¯−Q optimization term, which can reduce noise. So the final form of the objective function establishment is as follows:(24)η=α×∑i=1qΔE00,iQ¯,Qq+β×Q¯−QI+δQ¯−Q∞+μ×stdevQ¯−Q.Among them,α, β, δ are μ weighting factors, and the default value is 1, which is used to easily modify the weight of each equation later.


#### 3.3.4. Pattern Search

Adaptive Brightness Adjustment. The standard value of the color card of the standard sample is usually obtained under a standard illuminant such as D65 light source. However, when industrial cameras acquire images, they will be affected by factors such as light sources, cameras, image sensors, and lenses, resulting in not only color shifts, but also differences in brightness in the acquired original images. In this way, there are also phenomena such as color shift and different brightness from the initial RGB value that has a difference to the corrected RGB value.In order to obtain a more accurate image and ensure that the brightness of the image after correction is the same as that before correction, this study introduces the brightness adjustment parameterλ to act on the standard value of the color card of the sample, and the initial value of λ is 1 by default. At this time, the matrix Q¯ representing the standard XYZ values is redefined as Q¯=λX,λY,λZ. When calculating the linear parameter matrix M, first set the initial value λ of the brightness adjustment parameter, then use the pattern search method to solve the linear regression parameter M at this time, and then correct the original RGB value through the correction matrix M to obtain the corrected sample color value. At this time, it is necessary to judge whether the brightness constraint conditions are met. If not, it is necessary to fine-tune the brightness adjustment parameter λ and recalculate the correction matrix M at this time until the brightness change is within the allowable range. Usually white block brightness is selected as the brightness constraint. Additional luminance constraints are as follows:(25)Yt−Ym<ε,where Yt is the corrected brightness of the white patch in the CIEXYZ color space, Ym is the initial brightness of the white patch in the CIEXYZ color space, and ε represents the maximum allowable brightness adaptation range. According to the sensitivity of the human eye to brightness in the visual characteristics of the human eye, the ε value is set to 5 in this algorithm, and the linear parameter matrix M to be solved finally must satisfy this constraint.Pattern Search Algorithm. Pattern search methods are often used to optimize problems with N independent variables. It does not require the derivative of the objective function in the calculation, so it is very effective in solving optimization problems of nonderivable functions or functions that are extremely cumbersome to derive.The pattern search method will first search for the independent variablesX0, X1, X2,…, make these independent variables get closer and closer to the optimal point, and end the search to obtain the optimal solution when the termination condition is encountered.The specific method is as follows: first, the initial valuexi of the matrix M is randomly given, generally set as the unit matrix, and the objective function value fXi when M is the initial value is obtained. Then increase or decrease the value of the independent variable, and calculate the objective function value fXi+Vj∗L at this time, among which j∈1,2,…,2N. If the new objective function value is better than the previous objective function value, the search is successful, and the independent variable is updated, that is, Xi+1=Xi+Vj∗L. Then change the search center and expand the step size to be increased or decreased in the next search. Take Xi+1 as the search center and L=L∗δ as the step size (δ>1 means increase the step size to increase the search range) if the new objective function value is not optimal. If the value of the previous objective function is exceeded, the search fails. At this time, continue to take the previous Xi as the center and reduce the search step size, and take L=L∗λ as the step size (λ<1 means to reduce the search step size to reduce the search range); repeat the previous operation until the cutoff condition is reached, the cutoff condition is generally set as the number of iterations reaches a specified value.So far, the color correction method based on the pattern search optimization algorithm has been introduced, and the entire process is shown in Figure12.Figure 12
Flowchart of the color correction method based on the pattern search optimization algorithm.


### 3.4. Color Correction Based on Human Vision Characteristics

#### 3.4.1. Human Vision Models and Features

The human eye first observes the real scene, that is, the visual organ. The human eye processes the information, and then the information is transmitted through the visual pathway, and finally the visual sensory and visual perception information is processed in the visual sensory and visual perception center. Among them, visual perception processes information in parallel, while visual perception processes it serially. The overall visual process from the generation of information to the final response of the brain is as follows: the external light stimuli are received by the retina of the human eye and processed by the cone cells on the retina. This part is processed by the symmetrical left and right eyes at the same time, and then the information enters the visual channel. Cerebral cortex will first store image information and then produce the final response to its feature extraction, decision-making judgment, and so on.Based on the understanding of the physiological structure of the human visual system, the characteristics of the human visual system are proposed, including brightness adaptability, visual multichannel characteristics, visual nonlinearity, contrast sensitivity, and masking effects. Its simple model is shown in Figure13.Figure 13
Simple model of the vision system.


#### 3.4.2. Color Correction Based on Human Vision Characteristics

The human eye’s visual perception performance is very advanced; it can adapt to the changes of different color light sources and see the color of the object itself under different light sources. When a white object is illuminated with a low temperature light source, the object will appear red, while the same object is illuminated with a high temperature light source, and the object will appear blue instead. But the human eye can still see that the color of the object itself is white, and this property of human eye perception is color constancy. However, machine imaging systems do not have the ability to adapt to changes in light sources as intelligently as described above. To mimic human color vision perception, machine vision systems need to apply transformations such that the output image matches the scene observed by human vision. This is called white balance, and the transformation used to perform white balance is called a color-adaptive transformation.A color correction algorithm based on pattern search optimization will find a local optimum, and the initial value affects the correction result. Therefore, inspired by the color perception of the human visual system, a color correction based on the visual characteristics of the human eye is proposed, using a subtraction operation to subtract the hue value of the white point from the hue value of the pixel of each original image, which is at the same tristimulus values captured in the environment. The above expression can be expressed by the following formula:(26)fIcorrected=f1ΘfWF.Among them,Θ is a subtraction operation, which can be regarded as a color filter.The specific steps of the color correction method based on the visual characteristics of the human eye are as follows:(1)
Convert from RGB to LMS. The image is first transformed from the original color space to the cone optical space (L,M, S), assuming the original color space is (R, G, B); then from RGB to LMS there are(27)LMS=0.36320.59280.03940.13830.78270.07190.00300.08390.3721.(2)
In digital image processing, image intensity is quantized to 256 gray levels, so the scaling factorM0 is set to 256, and the tone function fc is defined as follows:(28)c=fcC=M0−C−l.(3)
Calculate outputl˜,m,s˜ from lms to nonlinear response.(4)
Calculate froml˜,m,s˜ to logarithmic achromatic functions a˜, logarithmic antihue functions rg and yb.(5)
The logarithmic functiona˜ is mapped back to a using the following inverse operation:(29)a˜rgyb=PCLIPl˜ms˜.(6)
In the above formula:(30)PCLIP=αβγα′β′γ′α″β″γ″.(7)
Among them,α,β,γ represent achromatic channels, the value is greater than 0, α′,γ′ are greater than 0, β′ is less than 0, indicating red and green channels, α″,β″ are greater than 0, and γ″ is less than 0, indicating yellow and blue channels.(8)
Calculates the tonal value of the white point on each image. Calculate with the maximum value of all channels of the image until the image pixels reach a certain percentagep, p is set to 1%, and then take the average value of these pixels in each channel.(9)
Calculate the corrected hue functionacor,rgcor,ybcor(10)
The corrected tone function is inversely transformed according to the above process to obtain the corrected RGB value


## 3.1. Reasons Affecting Color Distortion

In general, the packaging image designed by 3D software will be different from that seen by human eyes in reality, the spatial brightness distribution is uneven, and the color will deviate [9]. To a certain extent, the authenticity of the color will affect the later image processing, such as feature extraction, classification, and so on [10]. Therefore, we need to first understand why the acquired image color distortion can be targeted to solve and finally obtain the real image closest to the human eye. In terms of optical image imaging principle, the imaging paradigm used in 3D design software is the same as that of traditional camera, which is imaged on the focal plane through reflected light. However, because the photosensitive medium is different, it will be different in actual imaging. In addition, some of the sensors of the camera use CCD and some use CMOS, resulting in different sensing characteristics, different optical characteristics of the lens, focal length, field angle, and relative aperture, different characteristics of light source chromaticity, and different image scanning direction, which will directly affect the quality of the image and make the collected industrial image have color deviation from that seen by human eyes in reality.


## 3.2. Classic Color Correction Method

### 3.2.1. Color Correction Based on Polynomial Regression

Polynomial regression analysis, the most widely used method in statistics, has solved many practical problems, large and small. Through continuous observation, experiment, test, and summary, regression analysis can find out the interdependence of different variables. Regression analysis is generally divided into nonlinear and linear.This study adopts the form of linear regression, and the mathematical model can be expressed as(1)Y=Xβ+Z,where Y is the standard value of the color card standard target of the observation object, β=β1,β2,…,βp is the linear regression parameter, X is the independent variable, the size is of order N×P, and Z is the random value. Among them,(2)X=x11x12x13…x1px21x22x23…x2p……………xN1xN2xN3…xNp.Since the expected value of the random disturbance vectorZ can be put into the mathematical model Xβ, it is assumed that(3)EZ=0.It is equivalent to(4)EY=Xβ.Note that, in a linear regression model, once a parameter is derived, the derivative has nothing to do with the parameter [11]. When using the linear regression model to solve the color correction problem, we must first know the color value of the standard color card, which is the vector Y in the regression model.The required parameterβ should satisfy the following formula:(5)minY−βX.Among them,⋅ represents the second-order norm, and the solution β obtained by the least square method is the parameter matrix required in this study.Taking the matrix of size3×3 as a reference to illustrate how to obtain the model parameters, the first is(6)X1Y1Z1X2Y2Z2………XNYNZN=R1G1B1R2G2B2………RNGNBNTa11a12a13a21a22a23a31a32a33,where the parameters satisfy(7)X=R1G1B1R2G2B2R3G3B3TY=X1X2X3β=a11a21a31T.First, taking the first column of the model coefficient matrixβ1 as an example, there are(8)Y=Xβ1.The solutionβ1 obtained from the above formula should conform to the following equation:(9)XTY=XTXβ1,which is(10)β1=XTX−1XTY.Similarly, the values of other columns of the model coefficient matrix can be solved.From the above introduction, the most classic color correction process can be summarized as follows:(1)
In the experiment process, it is necessary to design the color card standard target first. The selection of the standard target should include all possible color gamuts as much as possible to ensure the accuracy of the experimental results.(2)
Determine the specific chromaticity value of the color card, that is, Y in the analysis process of the above formula. This article selects the xRite 24-color color card ColorChecker as the standard value.(3)
Get the RGB value to be corrected(4)
The model parameter matrixβ is obtained by the method of polynomial regression


### 3.2.2. Constrained Least Squares Color Correction

Although the least squares method can ensure that all colors are as close as possible, it cannot guarantee that the white block area will remain white after correction, and a primary consideration for the success of color correction is the accurate reproduction of the white block [12]. Therefore, this study adds the constraint that the white block remains unchanged before and after correction.In the least squares method,X¯=x¯,y¯,z¯ is the q×3 matrix, and in this method it is the set of corrected XYZ values. X=x,y,z is also a q×3 matrix, which is the set of raw RGB values to be corrected. In the case of a linear map, the residual sum of squares can be used to minimize the linear map, as follows:(11)F=X¯−XM22Among them,M is the linear parameter matrix of 3×3 to be solved, and the above formula is equivalent to minimizing the residual, as follows:(12)ε2=∑i=13X¯i−∑k=13mi,kXk2.If a column element of linear parameterM is required, the following formula shall be minimized:(13)∂ε2∂mi,k=0,k=1,2,3Therefore, the linear parameter matrixM can be summarized as follows:(14)∑i=1qXi12∑i=1qXi1Xi2∑i=1qXi1Xi3∑i=1qXi2Xi1∑i=1qXi22∑i=1qXi2Xi3∑i=1qXi3Xi1∑i=1qXi3Xi2∑i=1qXi32m1,km2,km3,k=∑i=1qXi1X¯ik∑i=1qXi2X¯ik∑i=1qXi3X¯ik.The above formula can be rewritten into a more concise form:BM=d, where B means(15)B=XTX,X=X11X12X13………Xq1Xq2Xq3.where d can be expressed as(16)d=XTX¯,X¯=X¯11X¯12X¯13………X¯q1X¯q2X¯q3.Therefore, the linear parameter matrixM required in this study is as follows:(17)M=XTX−1XTX¯.However, the classical least square method cannot remove the noise, resulting in inaccurate correction, and the corrected white block area cannot remain white. The primary consideration of the success of color correction is the accurate reproduction of white blocks. Therefore, this study adds the constraint that the white block remains unchanged before and after correction.In the constrained least squares method,X¯ represents the set of XYZ values after correction. X represents the set of original RGB values to be corrected. For the constraint term, the RGB value of the original white block is represented by points, and the XYZ value after correction of the corresponding color block is represented by x¯C. Lagrange multiplier is added to make the constrained white area correct accurately, as shown in the following equation:(18)F=X¯−XM22+μ∑k=13x¯kC−∑j=13xjCmjk.μ represents the Lagrange multiplier, which ensures that the constrained white area xc can be accurately corrected to x¯C. At the same time, the following formula must be met:(19)x¯CT−xCTM=0.


## 3.2.1. Color Correction Based on Polynomial Regression

Polynomial regression analysis, the most widely used method in statistics, has solved many practical problems, large and small. Through continuous observation, experiment, test, and summary, regression analysis can find out the interdependence of different variables. Regression analysis is generally divided into nonlinear and linear.This study adopts the form of linear regression, and the mathematical model can be expressed as(1)Y=Xβ+Z,where Y is the standard value of the color card standard target of the observation object, β=β1,β2,…,βp is the linear regression parameter, X is the independent variable, the size is of order N×P, and Z is the random value. Among them,(2)X=x11x12x13…x1px21x22x23…x2p……………xN1xN2xN3…xNp.Since the expected value of the random disturbance vectorZ can be put into the mathematical model Xβ, it is assumed that(3)EZ=0.It is equivalent to(4)EY=Xβ.Note that, in a linear regression model, once a parameter is derived, the derivative has nothing to do with the parameter [11]. When using the linear regression model to solve the color correction problem, we must first know the color value of the standard color card, which is the vector Y in the regression model.The required parameterβ should satisfy the following formula:(5)minY−βX.Among them,⋅ represents the second-order norm, and the solution β obtained by the least square method is the parameter matrix required in this study.Taking the matrix of size3×3 as a reference to illustrate how to obtain the model parameters, the first is(6)X1Y1Z1X2Y2Z2………XNYNZN=R1G1B1R2G2B2………RNGNBNTa11a12a13a21a22a23a31a32a33,where the parameters satisfy(7)X=R1G1B1R2G2B2R3G3B3TY=X1X2X3β=a11a21a31T.First, taking the first column of the model coefficient matrixβ1 as an example, there are(8)Y=Xβ1.The solutionβ1 obtained from the above formula should conform to the following equation:(9)XTY=XTXβ1,which is(10)β1=XTX−1XTY.Similarly, the values of other columns of the model coefficient matrix can be solved.From the above introduction, the most classic color correction process can be summarized as follows:(1)
In the experiment process, it is necessary to design the color card standard target first. The selection of the standard target should include all possible color gamuts as much as possible to ensure the accuracy of the experimental results.(2)
Determine the specific chromaticity value of the color card, that is, Y in the analysis process of the above formula. This article selects the xRite 24-color color card ColorChecker as the standard value.(3)
Get the RGB value to be corrected(4)
The model parameter matrixβ is obtained by the method of polynomial regression


## 3.2.2. Constrained Least Squares Color Correction

Although the least squares method can ensure that all colors are as close as possible, it cannot guarantee that the white block area will remain white after correction, and a primary consideration for the success of color correction is the accurate reproduction of the white block [12]. Therefore, this study adds the constraint that the white block remains unchanged before and after correction.In the least squares method,X¯=x¯,y¯,z¯ is the q×3 matrix, and in this method it is the set of corrected XYZ values. X=x,y,z is also a q×3 matrix, which is the set of raw RGB values to be corrected. In the case of a linear map, the residual sum of squares can be used to minimize the linear map, as follows:(11)F=X¯−XM22Among them,M is the linear parameter matrix of 3×3 to be solved, and the above formula is equivalent to minimizing the residual, as follows:(12)ε2=∑i=13X¯i−∑k=13mi,kXk2.If a column element of linear parameterM is required, the following formula shall be minimized:(13)∂ε2∂mi,k=0,k=1,2,3Therefore, the linear parameter matrixM can be summarized as follows:(14)∑i=1qXi12∑i=1qXi1Xi2∑i=1qXi1Xi3∑i=1qXi2Xi1∑i=1qXi22∑i=1qXi2Xi3∑i=1qXi3Xi1∑i=1qXi3Xi2∑i=1qXi32m1,km2,km3,k=∑i=1qXi1X¯ik∑i=1qXi2X¯ik∑i=1qXi3X¯ik.The above formula can be rewritten into a more concise form:BM=d, where B means(15)B=XTX,X=X11X12X13………Xq1Xq2Xq3.where d can be expressed as(16)d=XTX¯,X¯=X¯11X¯12X¯13………X¯q1X¯q2X¯q3.Therefore, the linear parameter matrixM required in this study is as follows:(17)M=XTX−1XTX¯.However, the classical least square method cannot remove the noise, resulting in inaccurate correction, and the corrected white block area cannot remain white. The primary consideration of the success of color correction is the accurate reproduction of white blocks. Therefore, this study adds the constraint that the white block remains unchanged before and after correction.In the constrained least squares method,X¯ represents the set of XYZ values after correction. X represents the set of original RGB values to be corrected. For the constraint term, the RGB value of the original white block is represented by points, and the XYZ value after correction of the corresponding color block is represented by x¯C. Lagrange multiplier is added to make the constrained white area correct accurately, as shown in the following equation:(18)F=X¯−XM22+μ∑k=13x¯kC−∑j=13xjCmjk.μ represents the Lagrange multiplier, which ensures that the constrained white area xc can be accurately corrected to x¯C. At the same time, the following formula must be met:(19)x¯CT−xCTM=0.


## 3.3. Color Correction Based on the Pattern Search Optimization Algorithm

In order to obtain a more consistent color image with human visual perception, this study combines polynomial regression analysis with CIEDE2000 color difference calculation to create a new color correction algorithm [13]. The algorithm constructs a linear parameter matrix m to obtain the relationship between the original RGB three-channel value and the corresponding standard XYZ three-channel value.


### 3.3.1. Select Color Card Standard Target

In real life, for a fixed color object, some people feel green, and others feel yellow, which varies from person to person. In the process of calibration, a standard value is needed as the calibration, so this standard value cannot be seen by human eyes and can only be used as the quantitative analysis standard by using the inherent properties of color obtained by specific instruments in the specified environment. This study selects the xRite Aiseli 24-color card ColorChecker under D65 light source as the standard.


### 3.3.2. CIEDE2000 Color Difference Calculation

The commonly used color difference calculation, such as Euclidean distance, can not truly reflect the reality because of its inherent nonuniformity and other defects. Therefore, in this algorithm, a new CIEDE2000 color difference equation closest to human visual perception is used. CIEDE2000 color difference calculation method needs to convert the initial color space from RGB to CIEXYZ color space by default.CIEDE2000 color difference formula is as follows:(20)ΔE=ΔL′KLSL2+ΔCab′KCSC2+ΔHab′KHSH2+RTΔCab′KCSCΔHab′KHSH.


### 3.3.3. Build Model

Establish Correction Model. Here, a polynomial regression method is used to establish the relationship from the initial measurement of the sample color to the standard value. The actual practice is to use the matrix Q=X,Y,Z to represent the standard XYZ value, the size is q×3, and q is the total number of color swatches participating in the calibration. Similarly, an q×3 matrix P=B,G,R is used to represent the initial RGB values obtained by the camera. Studies have shown that there is a linear regression parameter M obeying the relationship from the initial measurement of the sample color to the standard value, as follows:(21)X,Y,Z=B,G,RM.Among them, the form of matrixM is(22)M=a11a12a13a21a22a23a31a32a33.Typically, polynomial regression is used to solve the above equations. If the image corrected by the linear parameter matrixM is to be most in line with human visual perception, in terms of the overall visual perception difference, the difference between the two images must be reduced, so that the corrected XYZ value and the standard XYZ value are two. The smaller the difference between them the better. Therefore, when establishing the objective function, the average value of the color difference between the corrected XYZ value and the standard XYZ value should be used as an optimization item.In order to remove the effect of dark current, the color correction model is redefined as follows:(23)X,Y,Z=B−τb,G−τg,R−τrM.Among them,τb, τg, and τr are the average values of dark current, generally in the RGB color space.Establish Objective Function. After the correction model has been determined, the determined objective function needs to be established. Based on the in-depth study of the visual characteristics of human eyes, a color correction method based on pattern search is proposed. Compared with the classical color correction method, the new method is more consistent with the color perception of human eyes. From the above analysis, in order to solve the linear regression parameter matrix M, we need to know the matrices P and Q on both sides of the equation. Therefore, this study establishes an objective function, in which matrix Q¯ is q×3 matrix, which is the standard XYZ value, and is also the matrix, which is the corrected XYZ value. Finally, this study constructs a linear parameter matrix M of 3∗3, which minimizes the standard XYZ value and the corrected XYZ value.If the image corrected by the linear parameter matrix is to be most in line with human visual perception, in terms of the overall visual perception difference, the difference between the two images must be reduced to make the difference in XYZ values as small as possible. Therefore, when establishing the objective function, the average value of the color difference between the corrected XYZ value and the standard XYZ value should be used as an optimization item. However, in this study, the average value of color difference is only used as one of the optimization items, and the first-order normQ¯−Ql to be optimized is also added, and the infinite norm Q¯−Q∞ is also added to make the maximum error that has occurred as small as possible. Finally, this study also adds the standard deviation into the stdevQ¯−Q optimization term, which can reduce noise. So the final form of the objective function establishment is as follows:(24)η=α×∑i=1qΔE00,iQ¯,Qq+β×Q¯−QI+δQ¯−Q∞+μ×stdevQ¯−Q.Among them,α, β, δ are μ weighting factors, and the default value is 1, which is used to easily modify the weight of each equation later.


### 3.3.4. Pattern Search

Adaptive Brightness Adjustment. The standard value of the color card of the standard sample is usually obtained under a standard illuminant such as D65 light source. However, when industrial cameras acquire images, they will be affected by factors such as light sources, cameras, image sensors, and lenses, resulting in not only color shifts, but also differences in brightness in the acquired original images. In this way, there are also phenomena such as color shift and different brightness from the initial RGB value that has a difference to the corrected RGB value.In order to obtain a more accurate image and ensure that the brightness of the image after correction is the same as that before correction, this study introduces the brightness adjustment parameterλ to act on the standard value of the color card of the sample, and the initial value of λ is 1 by default. At this time, the matrix Q¯ representing the standard XYZ values is redefined as Q¯=λX,λY,λZ. When calculating the linear parameter matrix M, first set the initial value λ of the brightness adjustment parameter, then use the pattern search method to solve the linear regression parameter M at this time, and then correct the original RGB value through the correction matrix M to obtain the corrected sample color value. At this time, it is necessary to judge whether the brightness constraint conditions are met. If not, it is necessary to fine-tune the brightness adjustment parameter λ and recalculate the correction matrix M at this time until the brightness change is within the allowable range. Usually white block brightness is selected as the brightness constraint. Additional luminance constraints are as follows:(25)Yt−Ym<ε,where Yt is the corrected brightness of the white patch in the CIEXYZ color space, Ym is the initial brightness of the white patch in the CIEXYZ color space, and ε represents the maximum allowable brightness adaptation range. According to the sensitivity of the human eye to brightness in the visual characteristics of the human eye, the ε value is set to 5 in this algorithm, and the linear parameter matrix M to be solved finally must satisfy this constraint.Pattern Search Algorithm. Pattern search methods are often used to optimize problems with N independent variables. It does not require the derivative of the objective function in the calculation, so it is very effective in solving optimization problems of nonderivable functions or functions that are extremely cumbersome to derive.The pattern search method will first search for the independent variablesX0, X1, X2,…, make these independent variables get closer and closer to the optimal point, and end the search to obtain the optimal solution when the termination condition is encountered.The specific method is as follows: first, the initial valuexi of the matrix M is randomly given, generally set as the unit matrix, and the objective function value fXi when M is the initial value is obtained. Then increase or decrease the value of the independent variable, and calculate the objective function value fXi+Vj∗L at this time, among which j∈1,2,…,2N. If the new objective function value is better than the previous objective function value, the search is successful, and the independent variable is updated, that is, Xi+1=Xi+Vj∗L. Then change the search center and expand the step size to be increased or decreased in the next search. Take Xi+1 as the search center and L=L∗δ as the step size (δ>1 means increase the step size to increase the search range) if the new objective function value is not optimal. If the value of the previous objective function is exceeded, the search fails. At this time, continue to take the previous Xi as the center and reduce the search step size, and take L=L∗λ as the step size (λ<1 means to reduce the search step size to reduce the search range); repeat the previous operation until the cutoff condition is reached, the cutoff condition is generally set as the number of iterations reaches a specified value.So far, the color correction method based on the pattern search optimization algorithm has been introduced, and the entire process is shown in Figure12.Figure 12
Flowchart of the color correction method based on the pattern search optimization algorithm.


## 3.3.1. Select Color Card Standard Target

In real life, for a fixed color object, some people feel green, and others feel yellow, which varies from person to person. In the process of calibration, a standard value is needed as the calibration, so this standard value cannot be seen by human eyes and can only be used as the quantitative analysis standard by using the inherent properties of color obtained by specific instruments in the specified environment. This study selects the xRite Aiseli 24-color card ColorChecker under D65 light source as the standard.


## 3.3.2. CIEDE2000 Color Difference Calculation

The commonly used color difference calculation, such as Euclidean distance, can not truly reflect the reality because of its inherent nonuniformity and other defects. Therefore, in this algorithm, a new CIEDE2000 color difference equation closest to human visual perception is used. CIEDE2000 color difference calculation method needs to convert the initial color space from RGB to CIEXYZ color space by default.CIEDE2000 color difference formula is as follows:(20)ΔE=ΔL′KLSL2+ΔCab′KCSC2+ΔHab′KHSH2+RTΔCab′KCSCΔHab′KHSH.


## 3.3.3. Build Model

Establish Correction Model. Here, a polynomial regression method is used to establish the relationship from the initial measurement of the sample color to the standard value. The actual practice is to use the matrix Q=X,Y,Z to represent the standard XYZ value, the size is q×3, and q is the total number of color swatches participating in the calibration. Similarly, an q×3 matrix P=B,G,R is used to represent the initial RGB values obtained by the camera. Studies have shown that there is a linear regression parameter M obeying the relationship from the initial measurement of the sample color to the standard value, as follows:(21)X,Y,Z=B,G,RM.Among them, the form of matrixM is(22)M=a11a12a13a21a22a23a31a32a33.Typically, polynomial regression is used to solve the above equations. If the image corrected by the linear parameter matrixM is to be most in line with human visual perception, in terms of the overall visual perception difference, the difference between the two images must be reduced, so that the corrected XYZ value and the standard XYZ value are two. The smaller the difference between them the better. Therefore, when establishing the objective function, the average value of the color difference between the corrected XYZ value and the standard XYZ value should be used as an optimization item.In order to remove the effect of dark current, the color correction model is redefined as follows:(23)X,Y,Z=B−τb,G−τg,R−τrM.Among them,τb, τg, and τr are the average values of dark current, generally in the RGB color space.Establish Objective Function. After the correction model has been determined, the determined objective function needs to be established. Based on the in-depth study of the visual characteristics of human eyes, a color correction method based on pattern search is proposed. Compared with the classical color correction method, the new method is more consistent with the color perception of human eyes. From the above analysis, in order to solve the linear regression parameter matrix M, we need to know the matrices P and Q on both sides of the equation. Therefore, this study establishes an objective function, in which matrix Q¯ is q×3 matrix, which is the standard XYZ value, and is also the matrix, which is the corrected XYZ value. Finally, this study constructs a linear parameter matrix M of 3∗3, which minimizes the standard XYZ value and the corrected XYZ value.If the image corrected by the linear parameter matrix is to be most in line with human visual perception, in terms of the overall visual perception difference, the difference between the two images must be reduced to make the difference in XYZ values as small as possible. Therefore, when establishing the objective function, the average value of the color difference between the corrected XYZ value and the standard XYZ value should be used as an optimization item. However, in this study, the average value of color difference is only used as one of the optimization items, and the first-order normQ¯−Ql to be optimized is also added, and the infinite norm Q¯−Q∞ is also added to make the maximum error that has occurred as small as possible. Finally, this study also adds the standard deviation into the stdevQ¯−Q optimization term, which can reduce noise. So the final form of the objective function establishment is as follows:(24)η=α×∑i=1qΔE00,iQ¯,Qq+β×Q¯−QI+δQ¯−Q∞+μ×stdevQ¯−Q.Among them,α, β, δ are μ weighting factors, and the default value is 1, which is used to easily modify the weight of each equation later.


## 3.3.4. Pattern Search

Adaptive Brightness Adjustment. The standard value of the color card of the standard sample is usually obtained under a standard illuminant such as D65 light source. However, when industrial cameras acquire images, they will be affected by factors such as light sources, cameras, image sensors, and lenses, resulting in not only color shifts, but also differences in brightness in the acquired original images. In this way, there are also phenomena such as color shift and different brightness from the initial RGB value that has a difference to the corrected RGB value.In order to obtain a more accurate image and ensure that the brightness of the image after correction is the same as that before correction, this study introduces the brightness adjustment parameterλ to act on the standard value of the color card of the sample, and the initial value of λ is 1 by default. At this time, the matrix Q¯ representing the standard XYZ values is redefined as Q¯=λX,λY,λZ. When calculating the linear parameter matrix M, first set the initial value λ of the brightness adjustment parameter, then use the pattern search method to solve the linear regression parameter M at this time, and then correct the original RGB value through the correction matrix M to obtain the corrected sample color value. At this time, it is necessary to judge whether the brightness constraint conditions are met. If not, it is necessary to fine-tune the brightness adjustment parameter λ and recalculate the correction matrix M at this time until the brightness change is within the allowable range. Usually white block brightness is selected as the brightness constraint. Additional luminance constraints are as follows:(25)Yt−Ym<ε,where Yt is the corrected brightness of the white patch in the CIEXYZ color space, Ym is the initial brightness of the white patch in the CIEXYZ color space, and ε represents the maximum allowable brightness adaptation range. According to the sensitivity of the human eye to brightness in the visual characteristics of the human eye, the ε value is set to 5 in this algorithm, and the linear parameter matrix M to be solved finally must satisfy this constraint.Pattern Search Algorithm. Pattern search methods are often used to optimize problems with N independent variables. It does not require the derivative of the objective function in the calculation, so it is very effective in solving optimization problems of nonderivable functions or functions that are extremely cumbersome to derive.The pattern search method will first search for the independent variablesX0, X1, X2,…, make these independent variables get closer and closer to the optimal point, and end the search to obtain the optimal solution when the termination condition is encountered.The specific method is as follows: first, the initial valuexi of the matrix M is randomly given, generally set as the unit matrix, and the objective function value fXi when M is the initial value is obtained. Then increase or decrease the value of the independent variable, and calculate the objective function value fXi+Vj∗L at this time, among which j∈1,2,…,2N. If the new objective function value is better than the previous objective function value, the search is successful, and the independent variable is updated, that is, Xi+1=Xi+Vj∗L. Then change the search center and expand the step size to be increased or decreased in the next search. Take Xi+1 as the search center and L=L∗δ as the step size (δ>1 means increase the step size to increase the search range) if the new objective function value is not optimal. If the value of the previous objective function is exceeded, the search fails. At this time, continue to take the previous Xi as the center and reduce the search step size, and take L=L∗λ as the step size (λ<1 means to reduce the search step size to reduce the search range); repeat the previous operation until the cutoff condition is reached, the cutoff condition is generally set as the number of iterations reaches a specified value.So far, the color correction method based on the pattern search optimization algorithm has been introduced, and the entire process is shown in Figure12.Figure 12
Flowchart of the color correction method based on the pattern search optimization algorithm.


## 3.4. Color Correction Based on Human Vision Characteristics

### 3.4.1. Human Vision Models and Features

The human eye first observes the real scene, that is, the visual organ. The human eye processes the information, and then the information is transmitted through the visual pathway, and finally the visual sensory and visual perception information is processed in the visual sensory and visual perception center. Among them, visual perception processes information in parallel, while visual perception processes it serially. The overall visual process from the generation of information to the final response of the brain is as follows: the external light stimuli are received by the retina of the human eye and processed by the cone cells on the retina. This part is processed by the symmetrical left and right eyes at the same time, and then the information enters the visual channel. Cerebral cortex will first store image information and then produce the final response to its feature extraction, decision-making judgment, and so on.Based on the understanding of the physiological structure of the human visual system, the characteristics of the human visual system are proposed, including brightness adaptability, visual multichannel characteristics, visual nonlinearity, contrast sensitivity, and masking effects. Its simple model is shown in Figure13.Figure 13
Simple model of the vision system.


### 3.4.2. Color Correction Based on Human Vision Characteristics

The human eye’s visual perception performance is very advanced; it can adapt to the changes of different color light sources and see the color of the object itself under different light sources. When a white object is illuminated with a low temperature light source, the object will appear red, while the same object is illuminated with a high temperature light source, and the object will appear blue instead. But the human eye can still see that the color of the object itself is white, and this property of human eye perception is color constancy. However, machine imaging systems do not have the ability to adapt to changes in light sources as intelligently as described above. To mimic human color vision perception, machine vision systems need to apply transformations such that the output image matches the scene observed by human vision. This is called white balance, and the transformation used to perform white balance is called a color-adaptive transformation.A color correction algorithm based on pattern search optimization will find a local optimum, and the initial value affects the correction result. Therefore, inspired by the color perception of the human visual system, a color correction based on the visual characteristics of the human eye is proposed, using a subtraction operation to subtract the hue value of the white point from the hue value of the pixel of each original image, which is at the same tristimulus values captured in the environment. The above expression can be expressed by the following formula:(26)fIcorrected=f1ΘfWF.Among them,Θ is a subtraction operation, which can be regarded as a color filter.The specific steps of the color correction method based on the visual characteristics of the human eye are as follows:(1)
Convert from RGB to LMS. The image is first transformed from the original color space to the cone optical space (L,M, S), assuming the original color space is (R, G, B); then from RGB to LMS there are(27)LMS=0.36320.59280.03940.13830.78270.07190.00300.08390.3721.(2)
In digital image processing, image intensity is quantized to 256 gray levels, so the scaling factorM0 is set to 256, and the tone function fc is defined as follows:(28)c=fcC=M0−C−l.(3)
Calculate outputl˜,m,s˜ from lms to nonlinear response.(4)
Calculate froml˜,m,s˜ to logarithmic achromatic functions a˜, logarithmic antihue functions rg and yb.(5)
The logarithmic functiona˜ is mapped back to a using the following inverse operation:(29)a˜rgyb=PCLIPl˜ms˜.(6)
In the above formula:(30)PCLIP=αβγα′β′γ′α″β″γ″.(7)
Among them,α,β,γ represent achromatic channels, the value is greater than 0, α′,γ′ are greater than 0, β′ is less than 0, indicating red and green channels, α″,β″ are greater than 0, and γ″ is less than 0, indicating yellow and blue channels.(8)
Calculates the tonal value of the white point on each image. Calculate with the maximum value of all channels of the image until the image pixels reach a certain percentagep, p is set to 1%, and then take the average value of these pixels in each channel.(9)
Calculate the corrected hue functionacor,rgcor,ybcor(10)
The corrected tone function is inversely transformed according to the above process to obtain the corrected RGB value


## 3.4.1. Human Vision Models and Features

The human eye first observes the real scene, that is, the visual organ. The human eye processes the information, and then the information is transmitted through the visual pathway, and finally the visual sensory and visual perception information is processed in the visual sensory and visual perception center. Among them, visual perception processes information in parallel, while visual perception processes it serially. The overall visual process from the generation of information to the final response of the brain is as follows: the external light stimuli are received by the retina of the human eye and processed by the cone cells on the retina. This part is processed by the symmetrical left and right eyes at the same time, and then the information enters the visual channel. Cerebral cortex will first store image information and then produce the final response to its feature extraction, decision-making judgment, and so on.Based on the understanding of the physiological structure of the human visual system, the characteristics of the human visual system are proposed, including brightness adaptability, visual multichannel characteristics, visual nonlinearity, contrast sensitivity, and masking effects. Its simple model is shown in Figure13.Figure 13
Simple model of the vision system.


## 3.4.2. Color Correction Based on Human Vision Characteristics

The human eye’s visual perception performance is very advanced; it can adapt to the changes of different color light sources and see the color of the object itself under different light sources. When a white object is illuminated with a low temperature light source, the object will appear red, while the same object is illuminated with a high temperature light source, and the object will appear blue instead. But the human eye can still see that the color of the object itself is white, and this property of human eye perception is color constancy. However, machine imaging systems do not have the ability to adapt to changes in light sources as intelligently as described above. To mimic human color vision perception, machine vision systems need to apply transformations such that the output image matches the scene observed by human vision. This is called white balance, and the transformation used to perform white balance is called a color-adaptive transformation.A color correction algorithm based on pattern search optimization will find a local optimum, and the initial value affects the correction result. Therefore, inspired by the color perception of the human visual system, a color correction based on the visual characteristics of the human eye is proposed, using a subtraction operation to subtract the hue value of the white point from the hue value of the pixel of each original image, which is at the same tristimulus values captured in the environment. The above expression can be expressed by the following formula:(26)fIcorrected=f1ΘfWF.Among them,Θ is a subtraction operation, which can be regarded as a color filter.The specific steps of the color correction method based on the visual characteristics of the human eye are as follows:(1)
Convert from RGB to LMS. The image is first transformed from the original color space to the cone optical space (L,M, S), assuming the original color space is (R, G, B); then from RGB to LMS there are(27)LMS=0.36320.59280.03940.13830.78270.07190.00300.08390.3721.(2)
In digital image processing, image intensity is quantized to 256 gray levels, so the scaling factorM0 is set to 256, and the tone function fc is defined as follows:(28)c=fcC=M0−C−l.(3)
Calculate outputl˜,m,s˜ from lms to nonlinear response.(4)
Calculate froml˜,m,s˜ to logarithmic achromatic functions a˜, logarithmic antihue functions rg and yb.(5)
The logarithmic functiona˜ is mapped back to a using the following inverse operation:(29)a˜rgyb=PCLIPl˜ms˜.(6)
In the above formula:(30)PCLIP=αβγα′β′γ′α″β″γ″.(7)
Among them,α,β,γ represent achromatic channels, the value is greater than 0, α′,γ′ are greater than 0, β′ is less than 0, indicating red and green channels, α″,β″ are greater than 0, and γ″ is less than 0, indicating yellow and blue channels.(8)
Calculates the tonal value of the white point on each image. Calculate with the maximum value of all channels of the image until the image pixels reach a certain percentagep, p is set to 1%, and then take the average value of these pixels in each channel.(9)
Calculate the corrected hue functionacor,rgcor,ybcor(10)
The corrected tone function is inversely transformed according to the above process to obtain the corrected RGB value


## 4. Experimental Results and Evaluation

During the experiment, the xRite 24-color color card ColorCheck was used as the test object. Quantitative analysis uses Euclidean distance to calculate the difference between the two. The smaller the difference, the better the correction effect. In the actual use process, this study first obtains the correction matrixM according to the 24-color blocks and then obtains the image to be corrected in the same environment. The corrected image is obtained by using the correction matrix.For the same color card image, as shown in Figure14(a), different algorithms are used for processing, and the experimental results are shown in Figure 14. Figure 14(a) is the original color chart, Figure 14(b) is the result of the color correction based on the least squares method, Figure 14(c) is the result of the color correction based on the least squares method with constraints, and Figure 14(d) is the color correction results based on the pattern search optimization algorithm, and Figure 14(e) is the color correction results based on the visual characteristics of the human eye. Taking the white block in the lower left corner as an example, it can be seen that the white block in the original image is blue, and the color correction results based on the least square method and the constrained least square method are the worst. Although the blue of the white block is corrected, the color deviation of other color blocks is serious, such as the blue block in the upper right corner, the whole picture is still blue, and the pattern based search effect is relatively better than the first two. However, the color correction result based on human visual characteristics in Figure 14(e) is closer to what human eyes see in real life.Figure 14
Color correction results.
(a)(b)(c)(d)(e)The classical Euclidean distance is used as the quantitative evaluation method. The smaller the distance is, the better the correction effect is. The maximum value, minimum value, mean value, median value, standard deviation, and the percentage of mean value less than 3 are added. As given in Table1, it can be seen that the maximum, minimum, mean, median, and standard deviation of color difference in this study are lower than other methods, and the percentage of color block mean percentage less than 3 reaches 48.1, which is much higher than other methods. Therefore, color correction based on human visual characteristics can achieve better results.Table 1
Color correction evaluation.
MinMaxMeanMedianSD%ΔE<3Constrained multiplication2.009817.21819.53039.62895.103820.3Polynomial0.874113.84836.82775.28814.028130.2Pattern search0.662811.39815.98277.31043.749141.94Human vision0.38349.29484.62993.72932.837248.1The image of Figure12 after color correction using the color correction method based on human visual characteristics is shown in Figure 15. It can be seen that the packaging image corrected by the method proposed in this study is more consistent with what human eyes really see.Figure 15
Packaging design image after color correction.


## 5. Conclusion

With the development of computer software and hardware technology, the function of virtual reality language is more powerful, and the application characteristics and forms of virtual reality technology will be more abundant and magical. It will play a more important role in product packaging design and virtual display. This study provides a three-dimensional design method of packaging structure based on virtual reality technology, which has high application value. In order to further improve the color authenticity of packaging design, this study focuses on the color correction method based on computer vision. The research shows that the pattern search algorithm can find the local optimal solution, and the initial value affects the correction result. Therefore, inspired by the color perception characteristics of human visual system, the image is corrected by combining visual multichannel and visual nonlinearity. The experimental results of the algorithm are more consistent with human visual perception.Although the proposed color correction method based on human visual characteristics is better than the traditional method, it can also be improved. So far, color correction algorithms can be divided into two categories, one based on spectrum and the other based on target color standard target. The color correction method based on human visual characteristics in this study is based on the target color standard target, so it can be studied based on spectral response.



---

*Source: 2738900-2022-07-12.xml*