# Image-Driven Panel Design via Feature-Preserving Mesh Deformation

Baojun Li, Xiuping Liu, Yanqi Liu, Ping Hu, Mingzeng Liu, Changsheng Wang

► To cite this version:

Baojun Li, Xiuping Liu, Yanqi Liu, Ping Hu, Mingzeng Liu, et al. Image-Driven Panel Design via Feature-Preserving Mesh Deformation. 4th Conference on Computer and Computing Technologies in Agriculture (CCTA), Oct 2010, Nanchang, China. pp.30-40, 10.1007/978-3-642-18336-2_5. hal-01562795

HAL Id: hal-01562795

https://inria.hal.science/hal-01562795v1

Submitted on 17 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Image-driven Panel Design via Feature-preserving
Mesh Deformation

Baojun Li¹, Xiuping Liu², Yanqi Liu², Ping Hu¹*, Mingzeng Liu¹, Changsheng Wang¹

¹ School of Automotive Engineering, Faculty of Vehicle Engineering and Mechanics,
State Key Laboratory of Structural Analysis for Industrial Equipment,
Dalian University of Technology, 116024 Dalian, China

² School of Mathematical Sciences, Dalian University of Technology, 116024 Dalian, China
{bjli, xpliu, pinghu}@dlut.edu.cn

**Abstract.** In this paper, we propose an image-driven 3D modeling technique for rapid panel design. Our semi-automatic approach is based on template technique and mesh volume deformation controlled by a special cage. We designed our modeling system to be interactive in 2D, automating the process of shape generation while relying on the user to provide image samples. Once a parametric model template is given, using the contour extracted from images, the new control cage corresponding to mesh models generated. Then the geometry of new panel is automatically recovered from the deformable template model. Our system also allows the user to easily reconstruct other 3D objects in a similar manner, such as realistic-looking plant modeling from images. We show realistic reconstructions of a variety of panels, automobile shapes and demonstrate examples of plant editing.

**Keywords:** Image-driven, Mesh deformation, Panel Design, Deformable template.

# 1 Introduction

Nowadays polygon meshes are widely used in both geometric modeling and finite element analysis fields. Mesh deformation is useful for providing various shapes of meshes for CAE tools, especially in very early phases of conceptual automotive design. Most of 3D objects are largely dominated by a few typical features which include contours, even and engineered meanings. Thus, to create 3D models with new appearances by processing and reusing the existing models is becoming an extremely important way to ease the efficiency problem of geometric design in computer-aided design and computer graphics.

Space deformations (volume deformation) are to this day the method of choice for shape deformation due to its independence of surface representation [1]. A space deformation is defined via a (usually simple) control cage or grid; user-defined deformation of this object is interpolated to the 3D space and evaluated at the input surface points. Space deformations are popular since they can handle various object

<footnote>* Corresponding author.</footnote>
representations, including parametric surfaces, polygonal meshes with multiple connected components, etc. In addition, space deformations are simple to implement, and they are highly efficient and robust, because the cost of the deformation is mainly dependent on the complexity of the control object and not on the deformed shape. Early space deformations used lattices as control objects, and then had also been explored as FFD[2], EFFD[3], and DFFD[4] etc. Later work proposed the use of so-called cages as control objects for shape deformations. Typically, the cage is a very coarse and offsetted version of the input shape. Various coordinate functions have been designed to carry over the deformation of the cage to the entire space, such as mean-value coordinates[5], harmonic coordinates[6], Green coordinates[7]. However, cage-based deformation schemes in references [5][6][7] which preserve differential properties, so far, cannot support direct manipulations very well. Cages generation and its manipulations are more complicate. Thus, in this work, for more adaptive for different kind of FEM meshes and easy to edit in 1D, we adapt modified DFFD [4] method as our main deformation technique.

The core techniques of the reuse of existing 3D mesh models using mesh deformation, the key property of mesh editing is interaction techniques and more input examples, such as images, 2D sketches. Masuda et al. proposed a combination method to manually specify varying surface stiffness for panel design [10], [11]. Gal et al. introduce a so called iWIRES, a novel approach based on the argument that man-made models can be distilled using a few special 1D wires and their mutual relations⁰. In this paper, a simple 1D editing method based on symmetric projection is developed, which deals with the control box of DFFD easily and can support the image examples very well.

In this paper, we introduce an image-driven mesh morphing framework for rapid design of the automobile panels, and also generalize this method to other applications, such as realistic-looking planting modeling from images. Section 2 presents the whole pipeline of the method. Then the detailed algorithm of the mesh deformation and contour extraction from images are introduced in Section 3. Section 4 gives the numerical examples and discussion.

## 2 Overview of Image-driven Framework

For production of high quality panels in a short period of time, the more freely and rapid design technique is widely studied in the past few years. Inspired by IWires method [1] and image morphing [12] method, our main motivation is to generate new models rapidly from images; meanwhile, users can edit the shapes in 1D.

In this section, we will introduce our framework of the image-driven panel design based on mesh deformation. Fig. 1 shows the whole process of our method proposed in this paper. There are three main parts of this framework, which include contour extraction from images, generation of initial control box $CB_0$ and new control box $CB_1$, deformation and related verification via CAE software and aesthetics respectively. The pipeline of the framework illustrated in Fig. 1 is given briefly as follows.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Flowchart of our method.</figcaption></figure>

Pre-processing (Control box generation)

In the pre-processing phase, the deformable mesh template *M* should be chosen and constructed properly according to needs of the practical panels. The first step of the *CB*₀ generation algorithm is to compute the bounding box of the initial dense mesh model *M* by principal component analysis (PCA); the so constructed bounding box captures the major geometric shape of the mesh *M*. For the simplicity of edition-interaction, we consider the main contour of the model via projection mapping, see Fig. 2. Then the corresponding contour and the control box *CB*₀ associated with DFFD method should be constructed automatically.

<figure><img src="image_3.png" /><figcaption>Fig. 2. (a-d) Control box generation.</figcaption></figure>
Fig. 2(a-d) illustrates the construction process of the control box corresponding to
the given model in Fig. 2 (a). More details are described in Section 3 and Section 4.

New control box constructed from images

For the purpose of the image-driven mesh deformation, the key point is to create
new control box from input image examples. Moreover, there are two core steps
should be done carefully, which are the contour extraction from images with high
precision and subsequent reconstruction of the new control box respectively. This
stage will be described particularly in one full section due to its complicity with many
techniques, see Section 4.

Panel design via mesh deformation

With the deformable template and the control boxes CB₀ and CB₁ are given, the
next task is to generate the new panel or model using mesh deformation. The final
mesh models are generated to meet requirements of panel CAE analysis, such as crash,
NVH, durability and formability. Thus, a general deformation method which is
independent of surface representation will be developed. In this paper, FFD method is
used. In the next section, the mesh deformation techniques are described in detail.

3. Design via mesh deformation

The rapid design of panels or carbody shapes is constrained by the conflicting
requirements of multiple objectives. For example, designers need a more conventional
CAD tool to modify the original models to obtain the shape changes required and
time constants necessary for these changes. However, the final design is determined
by CAE engineers and designers to verify the artistic and reliability of the shapes.
State-of-the-art commercial CAE software, such as LS-Dyna, Nastran, Abaqus, Pam-
Crash, Fluent, etc. are all based on meshes. Thus, this paper focuses on the platform
which enables the user to rapidly change an existing FE / CFD mesh into a new target
shape without having to redraw it in the CAD system. In this section, the mesh
deformation method used in this paper will be introduced in detail.

**3.1. Pre-processing**

Firstly, in order to rapidly generate a panel model, it is important that the suitable
deformable template should be chosen and constructed. Thus, a series of panels or
carbody database is constructed to meet the requirement of the different products.

Once the template model was constructed, the following step is to generate a
corresponding control box, which can design new shapes via space deformation
method. In this paper, for the sake of edition simplicity, the 2D edition approach will
be adopted. Now we introduce our main methods used in this work in brief. The
automatic *CB* generation steps are described as following:

1) Compute the principle axis of the template model by Principal Component Analysis (PCA)[17];
2) Rotate and project the model into the principle plane, then obtain the contour polygon, see Fig. 2;

3) Automatic generation of the control polygon feature points from the contour; see Fig. 2b;

4) Semi-automatic generation of control box CB₀ by tensor production to the principle axis, see Fig. 2 c-d.

It is noted that the suitable CB₀ should be constructed carefully, which affects the deformable template and the final design. Meanwhile, control boxes CB₀ and the corresponding CB₁ also provide the feature-preserving of the template through space deformation method.

In this work, we use the projection method in order to simplify the user-interaction process, that is, designers can modify the shapes in 2D principle plane but in the complicated 3D model directly. Taking the advantage of 2D editing, our method can generate a new model just from an input image or image examples, which is so called image-driven panel design method. Fig. 3 illustrates a new control box generated by our method.

<figure><img src="image_4.png" /><figcaption>Fig. 3. New control box generation by projection.</figcaption></figure>

## 3.2. New control box generated from images

Due to editing the control box CB₀ in 2D while in 3D directly, our method can generate a new control box from a new contour with some constraints. So, an image-driven modeling method is proposed based on the mesh space deformation method. This part is of the most importance in this framework, and will be given in Section 4.

## 3.3. Mesh deformation

In order to make use of surface-based techniques for deforming automobile sheet-metal panels, Masuda et al. [10][11] develop the soft and hard constraints on the mesh and propose a framework which can preserve the form features of the sheet-metal panel while deforming the model. Huang et al. [13] proposed a morphing method with feature-preserved for panel design, which use the DFFD method on NURBS surface directly. However, as described in Section 3, the rapid panel design method should provide different meshes to satisfy the need of subsequent CAE analysis. Thus, in this work, the mesh space deformation theory is used rather than the surface-based method.
## 4. New CB1 constructed from images

In this section, we introduce the detailed techniques to reconstruct new control box $CB_1$ which shares the same topological structure as the original one $CB_0$, see Fig. 4

<figure><img src="image_5.png" /><figcaption>Fig. 4. Pipeline of $CB_1$ reconstruction from an image.</figcaption></figure>

### 4.1 Contour extraction from images

First of all we introduce how to extract the exact contour information from an ideal panel image which is the key point to generate new control box.

As shown in Fig. 4, the input images should be pre-processed to obtain a contour with high precision firstly. There are two necessary operations to improve the extracted result in the pre-processing phase, which are image resizing and image smoothing respectively. In this work, for higher effency, the bi-linear interpolation method is used to resize an image for ideal; In order to obtain an initial contour from the input image, Guassian smoothing filter is used. Fig. 5 or an imput image example, and the Fig. 5 obtained after the pre-processing stage.

As the pipeline shown in Fig. 4, the following step is to extract the contour from the image, as shown in Fig. 5b-c. The improved gradient vector flow snake (GVF) [14] method is used in this paper. Now we introduce this method in brief. A GVF model is the vector field

$$v(x, y) = [u(x, y), v(x, y)],$$

which minimizes the energy functional

$$\epsilon = \iint \mu(u_x^2 + u_y^2 + v_x^2 + v_y^2) + |\nabla f|^2 |v - \nabla f|^2 dx dy.$$

This variational formulation follows a standard principle, that of making the result smooth when there is no noisy data. In particular, when $|\nabla f|$ is small, the energy is dominated by sum of the squares of the partial derivatives of the vector field, yielding a slowly varying field. On the other hand, when $|\nabla f|$ is large, the second term
dominates the integrand, and is minimized by setting $v=|∇f|$. This produces the desired effect of keeping $v$ nearly equal to the gradient of the edge map when it is large, but forcing the field to be slowly-varying in homogeneous regions. The parameter $\mu$ is a regularization parameter governing the tradeoff between the first term and the second term in the integrand. This parameter should be set according to the amount of noise present in the image (more noise, increase $\mu$).

<figure><img src="image_6.png" /><figcaption>Fig. 5. Contour extraction from an image.</figcaption></figure>

## 4.2. Post-processing of contour

In the above subsection, the initial contour is obtained from an image, as illustrated in Fig. 4a-c. However, the initial contour cannot match with the contour of original control box $CB_0$ very well. There are two necessary operations to solve the problem, which are contour assessment and feature point assignment respectively.

### Contour assessment

Once the template model is given, through the pre-processing operation in Section 3, a constant original contour is obtained. In order to generate a final control box $CB_1$ with the same topology and features as $CB_0$, the original contour and contours extracted from images should be assessed exactly. Fig. 6 shows the geometric assessment result for a given carbody model, and Fig. 6b illustrates assessment result of the contour shown in Fig. 4c. It is should be noted that this step can be an automatic process for a given specific panel case.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Geometric assessment of contours.</figcaption></figure>

### Feature point assignment

There is another important point of contour post-processing, i.e., feature point assignment. It is obvious that there are no any apparent relationships between the original contours and contours extracted from images. Thus, to bridge this gap, we should assign some specific feature points on the contours which determine the panel shapes and main corresponding engineering meanings. In this paper, we assignment
the feature points semi-manually. As shown in Fig. 7. Coding the feature points, the coding of the feature points for contour in Fig. 4c is given.

<figure><img src="image_8.png" /><figcaption>Fig. 7. Coding the feature points.</figcaption></figure>

### 4.3. Reconstruction of new control box $CB_1$

By extraction from images and suitable post-processing, a coded contour with geometric assessment is obtained. In this step, a new control box $CB_1$ with the same topology with $CB_0$ will be constructed using the above contour information.

The geometric transformation between the contour from image and the contour associated with $CB_0$ is used. Due to the difference of geometric information between two contours, we adjust the control polygon of the contour extracted from image, according to the feature calibration and geometric information of the original contour. Fig. 8 illustrates an adjustment for the contour shown in Fig. 6 (in blue), and the red one denotes contour associated with the template. After this operation, the control polygon and contour of $CB_1$ are obtained.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Adjustment of the control polygon.</figcaption></figure>

In order to construct a new control box $CB_1$ with high precision and the same topology with $CB_0$, a resampling operation with high precision is necessary. However, the contour extracted from an image directly usually has a low-resolution. In this paper, cubic spline interpolation method [16] is used for the initial extracted contour. Then the final contour of the $CB_1$ is obtained by resampling from the interpolated spline curve with the same number of points with $CB_0$. Fig. 9 shows the final contour of the $CB_1$ corresponding to Fig. 5a-c.

<figure><img src="image_10.png" /><figcaption>Fig. 9. The final contour of the $CB_1$.</figcaption></figure>
**5. Implementation and numerical examples**

We have applied our method to a large variety of panel and carbody models, and obtained very ideal results. All models are computed on a double 2.80-GHz Pentium-R (2G RAM) machine, using VC++ 6.0 and KMAS/COMX development platform.

In this section, three images of cars from Internet are given, and the associated carbodies are computed rapidly by our framework. Fig. 10 is the template model used in this paper, and the final corresponding designs are given as following, such as Fig. 11-13.

<figure><img src="image_11.png" /><figcaption>**Fig. 10.** The template carbody model in this paper.</figcaption></figure>

Based on the template model shown in Fig. 10, a new carbody design from image in Fig. 5 is obtained as following figures using the new control box illustrated in Fig. 3 and Fig. 9. Fig. 11 bottom shows the final FEM mesh of the new design, and the others shows the shading one from different views.

<figure><img src="image_12.png" /><figcaption>**Fig. 11.** The final design corresponding to Fig.5-9.</figcaption></figure>

We also provide two carbody designs from images directly using our method, see Fig. 12 and Fig. 13.

<figure><img src="image_13.png" /><figcaption>**Fig. 12.** New design generated using our method.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Fig.13. New SUV design generated using our method.</figcaption></figure>

## 6. Conclusions

In this paper, we proposed a framework for rapid panel design based on mesh space deformation. Furthermore, we also explored the approach to create any other kinds of man-made, engineered objects and realistic shapes. This approach generates so many different kinds of shapes due to image example input which is abundant from internet. Numerical examples show that our framework is effective and able to reconstruct visually pleasing objects. The final output finite element meshes can be utilized by CAE software. It should be noted that this study has examined only for auto-body design in the conceptual design stage.

For the future work, we will improve current contour-extraction method from images, in order to obtain the feature information with higher precision. A more automatic registration algorithm to reconstruct new control boxes $CB_1$ from images will be also considered. Furthermore, a more parametric framework will be constructed to generate panels or carbody self-adaptively to meet designer's needs. More applications under this framework will be also considered.

## 7. Acknowledgement

This work was funded by the Key Project of the NSFC (No. 10932003, u0935004), NSFC (No. 60873181), "863" Project of China (No. 2009AA04Z101), "973" National Basic Research Project of China (No. 2010CB832700) and the Fundamental Research Funds for the Central Universities. The model in Fig.10 is provided by DEP.

## 8. References

[1] Gal R., Sorkine O., Mitra N.J., Cohen-Or D.: iWIRES: An Analyze-and-Edit Approach to Shape Manipulation. ACM Trans. Graph., 28(3):1--10 (2009)

[2] Sederberg T.W., Parry S.R.: Free-form Deformation of Solid Geometric Models. In: Proc. of ACM SIGGRAPH 1986, pp. 151--160, ACM, New York (1986)

[3] Coquillart S.: Extended Free-form Deformation: A Sculpturing Tool for 3D Geometric Modeling. In: Proc. of ACM SIGGRAPH 1990, pp. 187--196, ACM, New York (1990)

[4] Hu S., Zhang H., Tai C.: Direct manipulation of FFD: Efficient explicit solutions and decomposable multiple point constraints. Visual Computer 17 (6), 370--379 (2001)

[5] Ju T., Schaefer S., Warren J.: Mean Value Coordinates for Closed Triangular Meshes. ACM Trans. Graph. 24(3), 561--566 (2005)
[6] Joshi P., Meyer M., DeRose T., Green B., Sanocki T.: Harmonic Coordinates for Character Articulation. ACM Trans. Graph. 26(3), #71 (2007)

[7] Jiang N., Tan P., Cheong L.F.: Symmetric Architecture Modeling with a Single Image. ACM Trans. Graph. 28(5), 1--8 (2009)

[8] Tan P., Zeng G., Wang J., Kang S.B., Quan L.: Image-based Tree Modleing. ACM Trans. Graph. 26(3), 87--83 (2007)

[9] Lipman Y., D Levin., Cohen-Or D.: Green Coordinates. ACM Trans. Graph. 27(3), 1--10 (2008)

[10] Masuda H., Ogawa K.: Application of Interactive Deformation to Assembled Mesh Models for CAE Analysis. In: ASME Int. Design Engineering Technical Conferences, 2007.

[11] Masuda H., Yoshioka Y., Furukawa Y.: Preserving Form Features in Interactive Mesh Deformation. Computer Aided Design 39(5), 361--368 (2007)

[12] Chen L.L., Wang G.F., Hsiao K.A.: Affective Product Shapes through Image Morphing. In: Proceedings of the International Conference on Designing Pleasurable Products and Interfaces, pp. 11--16, ACM, New York (2003)

[13] Huang Q., Li B.J., Liu. M.Z., Bao J.R.: Feature-preserved Morphing Method for Panel Design. Mathematical and Computer Modelling 51, 1417--1420 (2010)

[14] Xu C.Y., Jerry L.P.: Snake, Shapes and Gradient Vector Flow. IEEE Trans. on Image Processing 7, 359--369 (1998)

[15] Xu C.Y., Jerry L.P.: Gradient Vector Flow Deformable Models. Handbook of medical imaging, pp. 159--169 (2000)

[16] Wang R.H.: Numerical Approximation. Higher Education Press, Beijing (1999)

[17] David L.: Linear Algebra and Its Applications. Addison-Wesley, New York (2000)