user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Part-Based 3D Face Morphable Model with Anthropometric Local Control

Paper content:
We propose an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attribute
editing workflow.
Current face modeling methods using 3DMM suffer from the lack of local control.
We thus create a 3DMM by
combining local part-based 3DMM for the eyes, nose, mouth, ears, and facial mask regions.
Our local PCA-based approach
uses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressive
while allowing accurate reconstruction.
The editing controls we provide to the user are intuitive as they are extracted from
anthropometric measurements found in the literature.
Out of a large set of possible anthropometric measurements, we filter the
ones that have meaningful generative power given the face data set.
We bind the measurements to the part-based 3DMM through
mapping matrices derived from our data set of facial scans.
Our part-based 3DMM is compact yet accurate, and compared to
other 3DMM methods, it provides a new trade-off between local and global control.
We tested our approach on a data set of 135
scans used to derive the 3DMM, plus 19 scans that served for validation.
The results show that our part-based 3DMM approach
has excellent generative properties and allows intuitive local control to the user.
Authoring realistic 3D faces with intuitive controls is used in a broad range of computer graphics applications such as video games, person identification, facial plastic surgery, and virtual reality.
This process is particularly time-consuming given the intricate details found in the eyes, nose, mouth, and ears.
Consequently, it would be convenient to use high-level controls, such as anthropometric measurements, to edit human-like character heads.
Many methods use 3D morphable face models (3DMM) for animation (blend shapes), face capture, and face editing.
Even though face animation concerns are important, our work focuses on the editing of facial meshes.
3DMMs are typically constructed by computing a Principal Component Analysis (PCA) on a data set of scans sharing the same mesh topology.
New 3D faces are generated by changing the relative weights of the individual eigenvectors.
These methods are popular due to the simplicity and efficiency of the ap-proach, but suffer from two fundamental limitations: they impose global control to the new generated meshes, making it impossible to edit a localized region of the face, and they control mechanism is very unintuitive.
Some methods compute localized 3DMM, but they focus on facial animation instead of face modeling.
Furthermore, we compared our approach with such methods and saw that their automatic localized basis construction works well for animation purposes (considering a data set composed of animations for a single person), but perform worst than our approach for modeling purposes (considering a data set made of neutral faces from different persons).
We propose an approach to constructs realistic 3DMMs.
We increase the controllability of our faces by segmenting them into independent subregions and selecting the most dominant eigenvectors per part.
Furthermore, we rely on facial anthropometric measurements to derive useful controls to use our 3DMM for editing faces.
We propose a measurement selection technique to bind the essential measurements to the 3DMM eigenvectors.
Our method allows the user to edit faces by adjusting the facial parts using sliders controlling the values of anthropometric measurements.
The measurements are mapped to eigenvector weights, allowing us to compute the individual parts matching the values selected by the user.
Finally, the reconstructed parts are seamlessly blended together to generate the desired 3D face.
We present experimental evidence to demonstrate how these tailored 3DMMs are preferred over the global PCA models.
3D morphable models are powerful statistical models widely used in many applications in Computer Vision and Computer Graphics.
One of the most well-known previous work is proposed by Blanz and Vetter [BV*99] .
Their pioneer work proposes a model using PCA from face scans.
Although they propose a multi-segment model and decompose a face into four parts to augment expressiveness, the PCA decomposition is computed globally on the whole face.
Other global PCA methods have been proposed [ACP03; CWZ*14; BRZ*16; LBB*17; BRP*18; LGJV18].
A downside of global PCAbased methods is that they exhibit global support: when we adjust the eye, the nose may also undergo undesirable changes.
Another downside is a lack of intuitive user control for face editing.
While the eigenvectors are good at extracting the dominant modes of variation of the data, they provide weak intuitive interpretation.
To address the former problem, several local models have been proposed that segment the face into independent sub-regions and select the most dominant eigenvectors per part.
Tena et al. [TDM11] propose a method to create localized clustered PCA models for animation.
They select the location of the basis using a spectral clustering on the geodesic distance and correlation of vertex displacement considering variation in the expressions.
Their method requires a manual step to adjust the boundaries of the segments, making it not so different than ours where the parts are user specified.
Chi et al. [CGZ17] adaptively segments the face model into soft regions based on user-interaction and coherency coefficients.
Afterwards, they estimate the blending weights which satisfy the user constraints as well as the spatio-temporal properties of the face set.
Here too, the required user intervention renders the segmentation not so different than our user provided segments.
SPLOCS [NVW*13] propose the theory of sparse matrix decompositions to produce localised deformation from an animated mesh sequence.
They use vertex displacement in the Euclidean coordinates to select the basis in a greedy fashion.
We noticed that when considering variation in identity instead of variation in expression, the greedy selection leads to basis which are far less local than those from the method of Tena et al. [TDM11] and ours.
These papers address facial animation instead of face modeling and therefore assume large yet localized deformations caused by facial expressions which are different from our context where each face is globally significantly different from the others.
Cao et al. [CCWL18] segment the face with the same spectral clustering followed by manual adjustment as Tena et al. [TDM11] .
While their method focuses mostly on expression, they also provide some identity modeling, as they rely on the FaceWarehouse [CWZ*14] global model, which they decompose using the segments defined by the spectral clustering.
In their case, the goal is to do the realtime adaptation of a 3DMM to a face from a video feed.
While their method works remarkably well for the real-time application of "virtual makeup", it does not provide a very detailed facial model compared to ours, and it does not support a face editing workflow.
Other papers supplement decomposition approaches with the extraction of fine details, allowing to reconstruct a faithful facial model [CBZB15; GZC*16; SWTC14].
The major problem with these approaches is that they work for a specific person and do not provide editing capabilities.
The Phace [IKKP17] method allows the user to edit fat or muscle maps in texture space on the face.
While this provides a physically-based adjustment, the control is implicit.
The user modifies the texture and then the system simulates muscles and fat to get the result.
Wu et al. [WBGB16] propose an anatomically-constrained local deformation model to improve the fidelity of monocular facial animation.
Their model uses 1000 overlapping parts and then decouple the rigid pose of the part from its non-rigid deformation.
While this approach works particularly well for reconstruction, the parts are too small for editing semantic face parts such as the nose or the eyes.
As opposed to the methods described so far, the methods of Allen et al. [ACP03] and BodyTalk [SQH*16] greatly ease the editing by mapping intuitive features to modifications of global 3DMM eigenvector weights.
In particular, BodyTalk [SQH*16] relates transformations of the meshes to keywords such as "fit" and "sturdy".
While the mapping between the words and the deformations is not perfect, it still makes it reasonably intuitive to edit the mesh of the body.
One problem with this method, is that it provides words for bodies, not faces.
A second major problem is the inability to do local adjustments; adjustments that increase the length of the legs will result in changes to other regions such as the torso and arms.
In contrast, for our approach, we aim at providing local control in the editing.
A downside of global PCA based methods is that they exhibit global support: adjusting parameters to change one part has unwanted effects on other unrelated parts.
To address this problem, we segment the face into independent sub-regions and provide a process to select the best set of eigenvectors given a target number of eigenvectors.
Methods that segment the face in subregions target facial animation instead of modeling.
We will demonstrate that our approach is better suited to the task of face editing than these methods.
Another problem of most of the previous work is that it does not allow facial model editing through the adjustment of objective measurements.
In contrast, our method relies on anthropometric measurements used as controls for the editing.
Furthermore, we propose a process to select the right set of anthropometric measurements for each facial part.
In this paper, we introduce a pipeline for constructing a 3DMM.
We separate the face into regions and compute independent PCA decomposition on each region.
We then combine the per region 3DMMs, paying particular attention to select the most dominant eigenvectors across the eigenvectors of the different regions.
While the eigenvectors are good at extracting the dominant modes of variation of the data, they provide weak intuitive interpretation.
We thus use anthropometric measurements to provide human understandable adjustments of the face.
The reconstruction from the measurements is done through a mapping from the measurements to the weights that need to be applied to each eigenvector.
Out of the set of measurements we extracted from our survey of the literature, we selected a subset which proved to provide the least reconstruction error.
The overview of our approach can be found in Fig. 1 .
The remainder of this paper is organized as follows:
Sec. 4 describes how 3DMMs are constructed, including the face decomposition and selecting the most dominant eigenvectors.
Afterwards, we discuss how to reconstruct a face by smooth blending of different facial parts (Sec. 5).
In Sec. 6, the selection of the anthropometric measurements, and the mapping between these measurements and the PCA eigenvectors are discussed.
We demonstrate the results in Sec. 7 and discuss them in Sec. 8.
We employ PCA on a data set of scans (Sec. 8.3) to construct our 3DMMs.
We propose to segment the face into different parts in order to focus the decomposition on a part-by-part basis instead of computing the PCA decomposition on the whole face.
We compute the decomposition separately for the male and female subsets.
As is shown in Fig. 1 , we decompose the face into five parts: eyes, nose, mouth, ears, and what we refer to as the facial mask (grouping the remaining areas such as cheeks, jaws, forehead, and chin).
We further discuss this design choice in Sec. 8.2.
This face decomposition allows us to have eigenvectors for each part.
The geometry of the facial parts is presented with a shape-vector
where n v is the number of vertices of dth facial part, d ∈ 1, . . .
, 5 , and V i = x i y i z i ∈ R 3 defines the x, y, and z coordinates of the ith vertex.
After applying PCA, each facial part d is reconstructed as:
where S d is the mean shape of dth facial part, n e is its number of eigenvectors, P j is an eigenvector of size 3n v , b is a n e × 1 vector containing the weights of the corresponding eigenvectors, and S d is the reconstruction, which will be an approximation when not using all eigenvectors.
Our approach selects the smallest set of eigenvectors that still reconstructs accurately the shape.
We accomplish this by incrementally adding the eigenvectors in the order of their significance to the reconstruction until a certain accuracy is met.
Even though we rely on the eigenvalues to sort the eigenvectors for each part, we provide the user with a measurable error (in mm) which is more precise than relying solely on eigenvalues across different parts.
We determine the best set of eigenvector to achieve a balance between the quality of the per part reconstruction and the whole face reconstruction.
To evaluate the accuracy of our selection, we construct the facial parts (Eq. 1) and blend them together (Sec. 5) to generate the whole face.
Afterwards, we assess the accuracy of reconstruction by calculating the average of the geometric error D GE between the ground truth and the blended face.
We first do a rigid alignment step (rotation and translation) between the facial parts of the ground truth and the blended result.
We then record the average per-vertex Euclidean distance over all vertices and per part:
where n all is the number of vertices of the face mesh, n d is the number of vertices for part d, V j is on the ground truth and V j is the corresponding point on the blended face.
We compute averages over all vertices and per part to ensure that parts with more vertices do not end up using most of the eigenvector budget at the expense of parts with fewer vertices.
We do so for the entire data set and for a set of 19 validation faces that were not part of the training data set.
We compute the median data set error as well as the median validation error, and we average the two in a global error.
At each step of our incremental eigenvector selection, we decide which of the five parts will get a new eigenvector added to its set.
We compare the geometric errors resulting from each of the five candidate eigenvectors, and we select a candidate eigenvector which has a great impact on decreasing the error.
When eigenvectors from multiple parts result in similar decreases in error, instead of systematically picking the eigenvector based on lowest error, we select by sampling from a discrete probability density function (PDF) created from the respective decreases in error of the five candidate eigenvectors.
This PDF selection process creates a more even distribution of eigenvectors across the parts and it maintains a low error.
As we iterate, the reconstruction error decreases.
For the female and male data set faces, the average reconstruction errors are 2.00 and 2.13 mm when considering zero eigenvector.
The errors decrease to 0.75 and 0.74 mm after 80 iterations, and when considering all eigenvectors the errors are 0 mm.
We chose an error threshold of 1 mm which balances out the cost associated with considering too many eigenvectors and the accuracy of the reconstruction.
Table 1 shows the resulting eigenvector distribution after achieving our 1 mm reconstruction accuracy.
This step focuses on the problem of constructing a realistic new face through blending the five segmented parts together.
As opposed to methods such as those of Tena et al. [TDM11] and Cao et al. [CCWL18] which handle the transition between the parts by relying on the adjustment of a single strip of vertices, we diffuse the transition across three strips of vertices.
In contrast to other methods that adjust the transition by vertex averaging [CCWL18] or leastsquares fitting [TDM11] , we use Laplacian blending [SCL*04] of the parts and the transition, resulting in a smooth yet faithful global surface.
The vertex positions are solved by an energy minimization which reduces the surface curvature discontinuities at the junction between the parts while maintaining the desired surface curvature.
To this end, we define a transition zone made of quadrilateral strips around the parts.
In our experiments, a band of two quadrilaterals (three rings of green vertices in Fig. 2 ) provides good results.
We interpolate the Laplacian L (the cotangent weights) of the five facial parts weighted by β d , which has values of β d = 1 inside the part, β d ∈ {0.75, 0.5, 0.25} going outward of the part in the transition zone, β d = 0 elsewhere.
We normalize these weights so that they sum to one for each vertex.
These soft constraints allow for some leeway in the transition zone.
The boundary conditions of our system are set to the ring of blue vertices in Fig. 2 , and we solve for the remaining vertices.
To this end, we minimize the following energy function: where "inner" is the set of vertices of the five parts excluding the vertices of the boundary conditions.
T i is an appropriate transformation for vertex V i based on the eventual new configuration of vertices V i and R d is the rotation of part d.
We solve Eq. 5 in a similar fashion to ARAP [SA07] by alternating solving for the vertex position and rotation matrices until the change is small.
Fig. 3 shows that the rotation quickly converges as the Frobenius norm of consecutive rotations is large only for the first few iterations.
Given our experiments, we decided to stop iterating when the Frobenius norm falls below 0.01 or after 6 iterations.
Fig. 4 shows an example of blended face.
In this case, the Frobenius norm was below 0.01 after five iterations.
Fig. 5 , shows the evolution of the geometric error D GE between the ground truth parts and their blended counterparts for the example of Fig. 4 .
As can be seen, the error quickly reaches a plateau as the rotation stabilizes.
PCA eigenvectors characterize the space of the data variation but they do not provide a clear intuitive interpretation.
In this paper, we mainly focus on constructing linear regression models from data using a set of intuitive facial anthropometric measurements.
means of measurements taken between specific surface landmarks defined with respect to anatomical features.
We use 33 parameters listed in Table 2 .
Each measurement corresponds to either an Euclidean distance or ratios of Euclidean distances between surface positions as specified in each paper cited in Table 2 .
In this section, we propose a measurement selection technique which assesses the accuracy of each measurement, resulting in the most relevant ones for each facial part.
We evaluate the measurements on the facial parts of the data set, yielding f d,i = f i 1 . . .
f i nm for the dth facial part of scan S i considering n m measures.
The measures for all of the scans are combined into an n m × n s matrix,
, where n s is the number of scans.
We learn how to adjust the weights of the PCA eigenvectors to reconstruct faces having specific characteristics corresponding to the measures.
While Allen et al. [ACP03] learn a global mapping that adjusts the whole face, we will learn per part local mappings.
Furthermore, in Sec. 6.2 we will derive a process to select the best measures out of the set of all measures f i1 . . .
f in m , doing independently for each of the d parts.
The mapping is represented as an (n e ) × (n m + 1) matrix, M d :
where a row of ones is appended to the measurement matrix F d for y-intercepts in the regression and F + d is the pseudoinverse of F d .
B d is a (n e ) × (n s + 1) matrix containing the corresponding eigenvector of the related facial part.
Afterwards, we can solve for a weight vector b for a set of measurement values:
To construct a new facial part based on specific measurements, we use b in Eq. 1, as follows:
Moreover, we can define delta-feature vectors of the form:
where each ∆ f contains the user prescribed differences in measurement values.
Afterwards, by adding ∆b = M d ∆ f d to the related eigenvector weights, it is possible to adjust the measure such as making a face slimmer or fatter.
We propose a novel technique for automatically detecting the most effective and relevant anthropometric measurements.
Some might be redundant with respect to others, some might not make sense for a specific part (e.g., the "Ear Height" might not be relevant for the mouth), and some might even lead to mapping matrices that generate worst results.
During our investigations, we discovered that considering more anthropometric measurements does not necessarily lead to a lower reconstruction error.
Fig. 6 illustrates that a higher error occurs considering all the measurements in comparison with our selected combination.
Fig. 7 shows two odd-looking examples from using all the measurements of nose (a) and facial mask (b).
We thus evaluate the set of relevant measurements; we do so separately for each part.
We begin with an empty set of selected measurements, and we iteratively test which measurement we should add to the set by evaluating the quality of the reconstructed faces when creating the mapping matrix considering the currently selected measurements together with the candidate measurement.
We reconstruct a face using the mapping matrix (Eq. 7, 8) based only on its measurement values.
The reconstructed face is considered as a prediction, and thus we evaluate the prediction quality in a fashion very similar to the one used for the eigenvector selection, by reconstructing all of the faces found in the data set of facial scans, as well as the 19 validation faces.
Each candidate measurement is used together with the current set of selected measurement, and we compute the candidate mapping matrix from this set of measurements.
We use the mapping matrix with the data set and validation faces, and reconstruct all of the instances of the part under consideration (e.g., all of the mouths).
We then evaluate a geometric error, D GE , with the per vertex distance between each predicted instance and its corresponding ground truth instance.
The distance is calculated after doing a rigid alignment of the predicted instance to the ground truth instance.
This way, we ensure that we evaluate the fidelity of the shape and not its pose.
If one or few faces result in a large error, this could lead to the rejection of a measurement which might still be beneficial for the prediction of most faces.
To avoid this, we also measure the percentage D NI of faces that see an improvement of their error.
We count the number of faces for which their geometric errors have been decreased by considering the candidate measurement.
We then normalize D GE and D NI to the range [0, 1] and combine them into a single reconstruction quality measure:
Considering the combined geometric error and percentage of improvement of all candidate measurements, we pick the one which will be added to the set of selected measurements.
We stop adding measurements when we observe an increase of D GE and a value D NI below 50%.
We conduct this same process for each part (eyes, nose, mouth, etc.)
The selected anthopometric measurements are enumerated in Table 3 .
The description of each measurement as well as the reference to the literature where we took the measurement are shown in Table 2 , where we also list the measurements we rejected (measurements which were never selected for any of the segments).
Defining the correlation between the measurements is important for the adjustment of faces.
Accordingly, if the user adjusts one measurement, the system automatically calculates the adjustment of the other measurements.
This greatly helps to create realistic faces by maintaining the correlation observed in the data set.
Similarly to Body Talk [SQH*16], we use Pearson's correlation coefficient on F to evaluate the relationship between the anthropometric measurements.
Considering a facial part d, the Pearson's correlation coefficient Cor jk for measurements j and k is expressed as:
where f i j , f ik ∈ f d,i are measurements of scan S i , f j and f k are the mean values of measurements j and k respectively, and n s is the number of scans.
The coefficient is a value between −1 and 1 that represents the correlation.
When adjusting measurement k Notice how the automatic clustering lead to non-symmetrical clusters (left eye with one cluster vs right eye with two clusters) for 13 clusters and required us to manually check which other clustering would be usable.
by ∆ f k , we get the change of other measures as ∆ f j = Cor jk ∆ f k .
Accordingly, we can evaluate the influence of one measurements on the others as well as conditioning on one or more measurements and creating the most likely ratings of the other measurements.
Compared to global 3DMM methods that compute one set of eigenvectors for the whole face, our 3DMM computes a set of eigenvectors for each part.
This brings one of the advantages of our approach: its ability to locally adjust the faces.
We compare our approach to other methods that rely on local 3DMM.
We created mapping matrices (Eq. 6) for global 3DMMs, SPLOCS [NVW*13], clustered PCA [TDM11] , as well as our part-based 3DMMs, and tested the adjustment of measurements with these models.
We used 46 eigenvectors for global 3DMM, SPLOCS, and our part-based 3DMM.
For clustered PCA, we first tested using 13 clusters as is reported in the paper, but found that it lead to a non-symmetrical result (Fig. 8b) .
By checking other clusterings, we selected 12 clusters (Fig. 8a) .
As clustered PCA does not allow for different number of eigenvectors for each cluster, and to avoid having too few eigenvectors per part, we used 46 eigenvectors for each cluster (selecting the 46 with largest eigenvalues).
To compare our approach and the use of measurements with other methods, we derived a way to use our measurements with SPLOCS and clustered PCA.
We furthermore demonstrate that with SPLOCS and our approach, we can have more local or global control.
For our approach, Table 3 shows that some measures influence more than one part.
For example, the "Lip Length" is found in the lists for both mouth and nose.
When a measurement is shared between different facial parts, our method allows to decide to have more localized changes by adjusting the measure for only one part, or have more coherence across the parts by adjusting all of the parts that involve the measurement.
When comparing with SPLOCS, we can also balance between locality and globality of the influence of the measurement.
Each measurement is based on computations involving specific measurement vertices (such as the corner of the mouth and the tip of the nose).
To enforce locality, when considering a measurement, we check which SPLOCS "eigenvectors" infer significant movement at the related measurement vertices.
We compute this by checking if the eigenvector displacement vector at a measurement vertex is large enough compared to the maximum displacement vector of the eigenvector (we check if it is larger than 1% of the maximum displacement of all vertices of the eigenvector).
A SPLOCS eigenvector is considered for a measurement only if it meets the criterion for one of the measurement vertices of a specific measurement.
To enforce more globality with SPLOCS, we use the mapping matrices for all of the eigenvectors.
Fig. 9 compares global PCA eigenvectors, local and global SPLOCS, clustered PCA, and our local and global approaches.
Note how our strategy for global SPLOCS and our global approach create a useful continuum from the global PCA to our local approach.
We will now focus on local editing.
Fig. 10-12 show the adjustment of the same anthopometric measurement using global 3DMM, local SPLOCS, clustered PCA, and our local approach.
The color coding shows the per vertex Euclidean distance.
Note that the colors do not represent errors, but vertex movement.
Thus, the goal is to have warmer colors around the location where the editing is intended, and colder colors in unrelated regions.
In Fig. 10f and 10g , we can see that even though we wanted to adjust the "Nose Breadth", the adjustment using the global eigenvectors and local SPLOCS resulted in significant deformation all over the face while clustered PCA and our approach can focus the deformation around the nose as expected (Figs. 10h and 10i) .
We can observe similar unwanted global deformation of the face in Fig. 11f and 11g .
Also note the automatic segmentation of clustered PCA does not provide desired deformation for some cases such as Figs. 11h and 12h.
We consistently outperform clustered PCA in terms of local deformation where expected.
The results shown in Fig. 10-12 highlight the difficulty of locally controlling the face deformation, and the power of our approach in locally adjusting the face with respect to the anthropometric measurements.
In the accompanying video, we show multiple edits on multiple parts starting from the average face, while Fig. 13 shows edits starting from four real faces.
We can see that our approach allows to capture the essence of the anthropometric measurements, providing an easy to use workflow.
In this section we discuss different aspects of our approach.
We present different comparisons highlighting the impact of the eigenvector and measurement selection.
We then discuss the choice of face segmentation, and we end by describing the procedure used to bring all of our scans to a common face mesh.
To verify the robustness of our 3DMMs and of our set of selected measurements, we reconstruct real faces relying on their anthropometric measurements to compute their eigenvector weights (Eq. 7).
We then get the face with our approach, including the blending procedure (Sec. 5) and compute its resulting anthropometric measurements.
We compute the quality of the reconstruction through the absolute value of the difference between the ground truth measurement and the measurement from the reconstructed face.
Since measurements correspond to either an Euclidean distance or ratios of Euclidean distances, we normalized all the measurements to the range [0%, 100%].
Fig. 14 shows that the average percentage of error is low when using "our measurements".
This means that both the selection of eigenvectors and the mapping matrix work Starting from one face, we adjust one of its measurements to match the value of that measurement for another face.
We then compute the difference between the prescribed measurement value and the measurement value calculated from the mesh.
We do so for 1, 000 such edits.
Our approach leads to smaller error (percentage) compared to cluster PCA and slightly better results when compared to local SPLOCS.
well.
Furthermore, it shows that when using "all measurements" to compute the mapping matrix (Eq. 6), we get larger average errors compared to ground truth measurements.
When calculating the error in Fig. 14 for "our measurements", we calculate the average error over our selected measurements only (Table 3 ).
The error shown in Fig. 14 for "all measurements" also considers only our selected measurements (if considering the error across all of the measurements, the comparison is even more in favor of using our selected measurements).
We evaluated how our approach compared to SPLOCS and clustered PCA with respect to achieving measurement values prescribed by edit operations.
We created a set of 1, 000 random edits on 135 face meshes.
We take the resulting edited face mesh from our approach, SPLOCS, and clustered PCA, and evaluate the difference between the measurement value prescribed by the editing and the measurement value calculated from the edited mesh.
Overall, our approach is the one that performed the best with the resulting measurement being closest to the prescribed measurement.
SPLOCS was second and clustered PCA had the greatest differences (see Fig. 15 ).
Our face segmentation was motivated by several facial animation artists with whom we worked that strongly prefer that they have control over the face patches in order to make sure they match the morphology of the face and muscle locations.
This type of control is impossible to achieve with an automatic method that is typically agnostic to the underlying anatomical structure.
It is important to note that this manual way of selecting the regions is not more cumbersome than the current state of the art methods.
According to the authors and confirmed by our own experiments, the state of the art method of Tena et al. [TDM11] requires a post-processing step to fix occasional artifacts in their segmentation method.
Furthermore, as illustrated in Fig. 8b , the segmentation boundaries can occasionally occur across important semantic regions such as the eyes, leading to complications further down the pipeline.
The quality of the input mesh data set is important to the reconstruction of good 3D face models.
As many existing methods, we assume that the meshes share a common mesh topology.
Mapping the raw 3D scans to a common base mesh is typically done by a surface mapping method [RGdL*18; ZB13; ARV07].
We established this correspondence with the commercial solution R3DS WRAP.
In this paper, we designed a new local 3DMM used for face editing.
We demonstrated the difficulty to locally edit the faces with global 3DMMs; we thus segmented the face in five parts and combined the 3DMMs for each part into a single 3DMM by selecting the best eigenvectors through prediction error measurements.
We then proposed the use of established anthropometric measurements as a basis for the face editing.
We mapped the anthropometric measurements to the 3DMM through a mapping matrix.
We proposed a process to select the best set of anthropometric measurements, leading to an improve reconstruction accuracy and the removal of conflicting measurements.
From the list of 33 anthropometric measurements that we surveyed from the literature, we identified 31 which lead to an improvement of the reconstruction and we rejected 2 as they decreased the quality of the reconstruction.
Note that the anthropometric measurement selection process would apply as well even if using a different 3DMM than the one proposed in this paper, as well as if considering a different set of anthropometric measurements.
We demonstrated this by applying our set of measurement to both SPLOCS [NVW*13] and clustered PCA [TDM11] .
This also demonstrated that our approach produces results superior to those of established methods proposing automatic segmentation and different ways to construct the eigenvector basis.
We also presented different experimental evidence to show the superiority of our approach, especially in terms of local control, compared to the typical global 3DMM.
A limitation of our approach is the mapping matrices assuming a linear relationship between anthropometric measurements and the eigenvector weights.
An interesting avenue for future work would be to apply machine learning to identify non-linear mappings.
Our set of anthropometric measurements contains too few measures for the ears due to the scarcity of measurements within the ear compared to the nose.
It would be interesting to identify more anthropometric measurements for ears, as well as, considering the measurements that specify the distribution of curvature over the face, such as the measurement specifying the angle formed at the tip of the chin.
Another limitation comes from the blending of the different parts that.
Compared to global 3DMMs, our fixed boundary does not allow as much deformation for the shape of the head.
An additional avenue for future work would be to reconstruct a skull based on the anthropometric measurements, and then generate the facial mask based on an energy minimization of the skin thickness considering the skull and the measurements.
Another avenue for future research is to create textures that would plausibly have the facial structure of generated 3DMMs.
Using a Generative Adversarial Network that gets the 3DMM as well as some details such as anthropometric measurements to create a texture that fit to the generated face.

assistant: text: This paper describes a method of part-based morphable facial model allowing for localized user control.