<figure><img src="image_1.png" /></figure>

HAL
open science

# Data-driven modeling of subtle eye region deformations

Glenn Kerbiriou, Quentin Avril, Maud Marchal

► To cite this version:

Glenn Kerbiriou, Quentin Avril, Maud Marchal. Data-driven modeling of subtle eye region deformations. Computers and Graphics, 2025, 133, pp.104425. 10.1016/j.cag.2025.104425. hal-05477608

HAL Id: hal-05477608

https://inria.hal.science/hal-05477608v1

Submitted on 26 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Special Section on RAGI 2025

# Data-driven modeling of subtle eye region deformations☆

Glenn Kerbiriou a, b, Quentin Avril b, c,*, Maud Marchal a, c

a University Rennes, INSA, IRISA, France
b Interdigital, Rennes, France
c IUF, France

## ARTICLE INFO

### Keywords:

Modeling/geometry
Facial modeling
Digital human

## ABSTRACT

Eyes are extremely important in face-to-face communication due to their critical role in conveying emotions, mental state and non-verbal cues. The eye region has a wide variety of possible shapes and deformations, which play a key role in the modeling of faithful digital double. In this paper, we present DREAM (Data-dRiven Eye Area Modeling) the first data-driven eye region model that covers all the possible eye poses, allowing to recreate precise identities, gaze directions, eyelid shapes and aperture, and eye region deformations. We set up an innovative pipeline with eyelash removal, automatic landmarking and mesh registration to generate high quality training data. Comparative results show that DREAM outperforms existing models and allows to cover the modeling of a larger range of morphologies, gaze directions and eyelids apertures. We also present a new scan fitting methodology specific to eye region. We eventually show that DREAM can be used to extract a high-quality, animation-ready eye region rig from a single textured scan.

## 1. Introduction

Digitizing real humans with a very high level of faithfulness and realism has gained of interest these last years for a large range of applications such as virtual [1–3] and augmented reality [4], visual effects [5], gaming (e.g. the *MetaHuman* technology), social network [6], telepresence [7,8], caricature [9,10] and many more. The complexity of the human appearance, both static and dynamic, requires to develop dedicated acquisition, modeling and simulation systems in order to capture, model or simulate as accurately as possible the large spectrum of morphologies and movements (see Fig. 1).

Among the various anatomical features of the human head, the eyes play a crucial role in non-verbal face-to-face communication, expressing emotions and reflecting mental states. Moreover, the need for robust and anatomically precise eye models is reinforced by the utility of gaze tracking technologies in real-world applications [11] and the raising use of AI in ophthalmology for extracting eyes features [12]. However, due to their complexity, accurately modeling and animating eye region remains challenging. Specifically, changes in gaze direction and eyelid aperture are accompanied by subtle eyelid deformations. Moreover, the motion of the eyeball indirectly stretches the inner corner tissues which can occupy up to 13% of the visible eyeball area [13]. To address these complexities, person-specific eye region models have been developed by leveraging many multi-view images

of a same person [13–15]. However, these methods are not applicable in cases with limited data, such as a single scan or image. In this work, we propose a data-driven eye region model that combines identity and eye pose (expression) dimensions, enabling accurate modeling of eye region deformations for a new, unseen individual. This model exposes semantic parameters that directly monitor the gaze direction and eyelid aperture. Acting as a rig, our model allows for high-quality eye region animation.

In this paper, we first design a fully automatic pipeline for high-quality eye region scan registration that involves eyelash matting, 3D landmarks prediction, and texture consistency preservation. Then we present DREAM (Data-dRiven Eye Area Modeling) the proposed eye region model and how we learn it with the obtained data. Lastly, we fit DREAM to a single scan using a dedicated optimization-based strategy and show that it accurately predicts the subtle deformations that come with gaze and eye aperture variations.

The contributions presented in this paper are:

* An innovative pipeline composed of eyelash inpainting, automatic landmarking and mesh registration to generate high quality training data.

* A data-driven model of eye region that models the subtle skin deformations for any gaze direction, eyelid aperture and eye morphology.

<footnote>☆ This article is part of a Special issue entitled: 'RAGI 2025' published in Computers & Graphics.
* Corresponding author.
E-mail address: quentin.avril@interdigital.com (Q. Avril).</footnote>
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Scan fitting with DREAM allows to generate an animation-ready eye region rig that models person-specific subtle eyelid deformations. Given an input scan (top left), we can register DREAM (top middle) with a small error (top right). With the semantic parameters, we can then retarget the pose of the fitted model (bottom middle) to another pose matching closely the ground truth (bottom left and right).</figcaption></figure>

* A scan fitting methodology specific for the case of eye region.
* A qualitative and quantitative assessment of our model against existing models of the literature.

The following of the paper is organized as follows. In Section 2 we describe the existing eye region reconstruction, modeling and animation approaches. All details on how we have processed the scan datasets is provided in Appendix A. Section 4 describes eye region model DREAM and results of the scan fitting application are presented in Section 5.

## 2. Related work

**Human face modeling.** 3D analysis of the human face has attracted a lot of research interest these last decades. Blanz and Vetter's seminal work [16] first introduced 3D morphable models, representing the human face as a multivariate normal distribution using Principal Component Analysis (PCA). Since then, morphable models have been extremely popular for many applications including image face fitting and tracking [16–20], face recognition [21,22], face scan fitting [23–26] or synthetic face generation [27–30]. The reader may refer to the survey published by Egger et al. [31] for more insights on morphable face models. Among the different anatomical components of the human face, some are more complex than others and are often tackled independently from the full face, including the teeth [32], the neck [33], the tongue [34], the eyelids [35–37], the eyeballs [38–41] or, more recently, the eyelashes [42]. In particular, the research community has paid a lot of attention this last decade to the modeling of eye region, given its key role in the human face modeling and simulation.

**Eye region modeling.** Modeling the subtle features of human eye region, such as the inner eye corner or the skin bulging effect due to globe orientation is still a complex challenge as of today. Among the different models proposed in the literature, semantic models provide interesting properties since they can incorporate the parameters essential to model subtle deformations of the complex anatomy of eye region. For instance, Wood et al. [27] proposed a semantic model built on an articulated eye region mesh coupled with PCA identity components. The proposed articulation angle is a convenient knob for tuning the vertical angle of the look, as the upper eyelid height is strongly correlated with the vertical gaze angle. Yet robust and lightweight, this model cannot simulate eyelid deformations associated with horizontal eyeball rotations. In the same vein, Li et al. proposed FLAME [25] which models the full head with linear and joint-based deformations. The eyeballs are explicitly modeled but their motion has no impact on the surrounding skin. In contrast, Neog et al. [36] studied the impact of gaze direction changes on eyelids shape using an

elastodynamic model. They were able to simulate the soft tissues of the eyelids, but mostly for horizontal eyeball motion.

A semantic linear eyelids model has been successfully used for real-time eyelids tracking [37]. Eyelid creases and folds are detected in a RGB-D stream and a 3D-to-2D edge fitting scheme reconstructs the eyelids shape and pose. However, they chose to use a *blendshapes* model, thus generating several unrealistic shapes when the parameters are modified [43]. Additionally, the identity and expression modes were not extracted from data, but designed by artists, which limits their capability to represent real human biometric data. Even if their method is not real-time, Bermano et al. [35] managed to reconstruct the eyelids with unprecedented level of detail, including the upper eyelid fold inside by using a dedicated multi-view video setup and several manual steps. However, the system provides no eyelids posing control, such limiting its use for designing a model of eye region.

Deep learning methods have also been explored to represent eye region. Deep Appearance Models (DAM) [44] generate a view-dependant texture on a 3D mesh to produce realistic face renderings. Schwartz et al. [45] improved eye regions of DAMs with the help of explicit eyeball geometry, and truly made possible photorealistic avatar rendering in virtual reality. While promising, DAMs are person-specific and require an important amount of data in the training phase, which limits their large scale applicability. If the dynamic textures allow impressive visual quality, they also hide a relatively coarse underlying geometry since wrinkles and other details are baked in the texture. More recently, EyeNeRF [14] went a step further by disentangling the reflectance of the skin and eyeball from the scene illumination. Their Neural Radiance Field based technique allows for relighting, novel view synthesis and gaze retargeting. However, the trained models are person-specific and do not incorporate eyelid aperture variations, while they are a core feature of the human eye.

**Positioning of our approach.** In our work, we intend to propose a semantic data-driven model that could be able to reproduce the subtle deformations of eye regions whatever the eyelids and gaze motions. We chose a data-driven approach to capture the human diversity of the deformations. For that purpose, we leveraged an existing dataset that had the advantage of recording several eye expressions with a high accuracy [13]. Compared to other existing datasets of the literature, this dataset provides a sufficient level of detail so that we could learn the various possible deformations, such as the inner eye corner or the skin bulging effect due to globe orientation for instance. We chose to formalize our model using a multilinear approach. Multilinear models [17,23,46,47] borrow the robustness of linear models [16,22] and the semantic disentanglement of recent deep learning geometric models [29,48,49]. Compared to linear models, they have the advantage of
<figure><img src="image_4.png" /><figcaption>Fig. 2. Frontal photo and scan of a participant performing six different eye poses from the leveraged dataset.</figcaption></figure>

being able to combine together robustness and semantic information. In return they are not compact and require specifically formatted data to be learnt. For that reason, we designed an innovative pipeline to format the data issued from the dataset of Kerbiriou et al. [13] (see Appendix A) before designing our multilinear geometrical model (see Section 4).

## 3. Overview

To capture the wide diversity of eye region deformations, we leveraged an existing dataset [13] of more than 2000 scans of 57 people performing 36 eye expressions, reconstructed from 12 cameras. The eye expressions are defined as the Cartesian product of four eye aperture levels (closed, half-open, open and wide open), three horizontal gaze values (left, center and right) and three vertical gaze values (top, middle and bottom). A few data samples are presented in Fig. 2.

We performed several operations on the data before designing our model. Fig. 3 presents an overview of our skin registration process. The three presented poses (neutral, pose 1 and pose 2) belong to the same person's set of 36 eye expressions. The different steps of our pipeline are described in Appendix. As the original scans contain a lot of mesh artifacts due to incorrect reconstruction of eyelashes, Appendix A provides a detailed description on the eyelashes cleaning step we performed.

To build our model and extract all highly subtle skin variations, we need to find skin correspondences between scans. To do so, we used a template mesh registration approach. Our skin template mesh is a subset of the ICT-FaceKit topology [28] focusing on the left eye containing $N_s = 1407$ vertices and $F_s = 2664$ triangles, as shown, in the annex, on the left side of Fig. B.16. In the appendix, Appendix B provides all technical information on the registration process we performed with sparse and dense correspondence computations.

## 4. Eye region model

DREAM is a multipart eye region morphable model with explicit skin and eyeball textured meshes. The vertices and texture of the skin part are computed in a multilinear (quadrilinear) manner from tensors learnt from data. These tensors contain both identity and eye dimensions. To account for the spherical nature of the eyeball, the eyelid deformations are primarily described with Linear Blend Skinning (LBS), and secondly, with corrective linear displacements. The eyeball position is obtained by interpolating in a tensor with identity and pose dimensions, insisting on the fact that the eyeball center motion varies from person to person [40]. Its scale and rotation are arbitrary within bounds observed from data. To encourage a plausible size and position

of the eyeball, the position is made relative to the eye center, and the scale is made relative to the eye width. The eyeball texture is computed from a PCA basis learnt beforehand (see Fig. 5). The model also exposes the eyelid margin thickness as a parameter (see Fig. 4), due to its variability among humans. Measured from its posterior border to the posterior lash line, the eyelid margin is approximately 2.5 mm [52], but it varies among people and increases with age [53].

### 4.1. Model formulation

DREAM can be synthesized as a function:

$$M(\vec{\beta}, \vec{\gamma}, \vec{\psi}, \vec{\mu}, \vec{\tau}, h, s) : \mathbb{R}^{|\vec{\beta}|} \times |\vec{\gamma}| \times |\vec{\psi}| \times |\vec{\mu}| \times |\vec{\tau}| \times 2 \rightarrow \mathbb{R}^{3N_s \times 3N_e \times 3S_s^2 \times 3S_e^2} \quad (1)$$

DREAM takes as input the vectors that encode the shape $\vec{\beta} \in \mathbb{R}^{|\vec{\beta}|}$, the color $\vec{\gamma} \in \mathbb{R}^{|\vec{\gamma}|}$ and the eye pose i.e. the expression $\vec{\psi} = \{a, \theta, \phi\} \in \mathbb{R}^3$ with $a \in [0, 1]$ the eye aperture that ranges from $a = 0$ (eye closed) to $a = 1$ (eye wide open) with intermediate values $a = \frac{1}{3}$ (eyes half closed) and $a = \frac{2}{3}$ (eyes normally open). $\theta \in [-37^\circ, 19^\circ]$ is the vertical gaze angle and $\phi \in [-26^\circ, 42^\circ]$ the horizontal gaze angle. The gaze angle bounds were extracted from data. $\vec{\tau} \in \mathbb{R}^{|\vec{\tau}|}$ encodes the position of the eyeball that depends slightly on the eye pose. $h$ and $s$ are scalars that drive the eyelid margin thickness and the eyeball relative scale, respectively (see Fig. 4).

DREAM then returns the vertex positions of the skin mesh $V_s \in \mathbb{R}^{3N_s}$, the vertex positions of the eyeball mesh $V_e \in \mathbb{R}^{3N_e}$ as well as the skin and eyeball textures $I_s, I_e \in \mathbb{R}^{3S_s^2}$, respectively. The skin mesh is an eye region sample from ICT-FaceKit [28] totaling $N_s = 1407$ vertices. The eyeball mesh topology is borrowed from Wood et al.'s eye region 3DMM [27] with $N_e = 353$ vertices. $S_s^2$ and $S_e^2$ being the sizes of the skin and eyeball textures respectively.

**Eyeball modeling.** The eyeball is represented both implicitly and explicitly. The implicit representation consists in a signed distance function (SDF) to two spheres (one for the sclera and one for the cornea bulge) and is used to fill the gap between the eyelid and the eyeball and to ensure a consistent eyelid wiper width. The eyelid wiper being the thin moist structure located at the posterior edges of the eyelids. The explicit, mesh-based, eyeball representation is used for rendering and to distinguish the visible eyeball part from the obstructed one.

The eyeball transform is obtained with axis-aligned rotations by gaze angles $\{\theta, \phi\}$, a scale factor $s$ and a translation $\vec{i}$ computed with a code $\vec{\delta}$ and PCA tensor $\mathcal{T} \in \mathbb{R}^{|\vec{\delta}| \times 4 \times 3 \times 3}$:

$$\vec{i} = (\vec{T} + \mathcal{T} \times_1 \vec{\delta}) \times_2 f_4(a) \times_3 f_3(y) \times_4 f_3(x) \\ \text{with } y = \frac{\theta - \theta_{\min}}{\theta_{\max} - \theta_{\min}} \quad \text{and} \quad x = \frac{\phi - \phi_{\min}}{\phi_{\max} - \phi_{\min}} \quad (2)$$
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Overview of our skin registration system. Left: The neutral pose mesh is first registered using NRICP [50] and sparse landmarks only. The sparse landmarks are automatically predicted using an off-the-shelf method [51]. Right: In a second step, other scan meshes are augmented with dense correspondences retrieved from textures, before also getting registered with NRICP. Textures maps (diffuse color, height, normal) are then generated using a ray casting based approach.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Example of eyeball and margin size variation. Both are expressed in DREAM as scalar parameters ($h$ and $s$).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Base textures (left) and blendshapes (right) of the eyeball. The eyeball texture is computed from a previously learned PCA basis. The ring blackness parameter (middle) controls the strength of the outer black border of the iris. Eyeball blendshapes control the pupil dilation and the cornea collapse.</figcaption></figure>

where $\times_n$ denotes the tensor n-mode product and $f_k(u)$ is the linear interpolation kernel function that takes an input $u \in [0, 1]$ and outputs a vector of length $k$, distributing the value of $t$ between two adjacent components based on the fractional position $k \cdot u$:

$$f_k : [0, 1] \rightarrow [0, 1]^k$$

$$u \mapsto \begin{cases} \vec{u}_i = [(k-1)u] - (k-1)u & \text{if } i = [(k-1)u] \\ \vec{u}_i = (k-1)u - [(k-1)u] & \text{if } i = [(k-1)u] \\ \vec{u}_i = 0 & \text{else} \end{cases} \quad (3)$$

This model-based formulation of the eyeball center position $\vec{t}$ allows considering the eyeball center as not fixed but rather dependant on the

gaze direction [40]. It must be noted that the position of the eyeball is relative to the eye corners midpoint such that it depends more on one's eye region morphology.

**Skin modeling.** To construct the final skin shape, DREAM first generates neutral skin vertices with the given shape code $\vec{\beta}$. Then, they are transformed with Linear Blend Skinning to match the target pose $\vec{\psi}$ and a residual displacement conditioned by both $\vec{\beta}$ and $\vec{\psi}$. A post-processing step eventually fills the potential gaps between the eyeball and the eyelids and ensures that the globe does not pop out of the skin.

First, DREAM generates the skin vertices for a given shape code $\vec{\beta}$ in the neutral pose $\vec{\psi}^* = \{\frac{2}{3}, 0, 0\}$ i.e. eye normally open and gaze forward which we refer to as the “neutral pose”. As illustrated in Fig. 6, this is
<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Example on the dependence between shape and pose in DREAM. First (left) the model defines the neutral shape using a linear combination via the PCA base of neutral shapes. Then (right) it determines the pose via a trilinear interpolation of learned tensor.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Fig. 7.** The skin color is represented with RGB textures. The skin texture depends on the color identity code (left) but also on the eye pose (right). It is obtained via linear interpolation in the tensor.</figcaption></figure>

done by linear combination of the neutral shape PCA base $V^* \in \mathbb{R}^{|\vec{\beta}| \cdot 3N_s}$ with the shape code $\vec{\beta}$:

$$V_s^* = \vec{V}^* + \vec{\beta} \cdot V^* \quad (4)$$

The resulting shape in neutral pose $V_s^*$ is then retargeted to the input pose $\vec{\psi}$ using generic skinning function $W(V, P, A, W)$, where $P, A \in \mathbb{R}^{3|\vec{\psi}|}$, $W \in \mathbb{R}^{|\vec{\psi}| \times 3N_s}$ are respectively the 3D positions, axes and vertex blendweights of the $|\vec{\psi}| = 3$ joints. All these values are obtained with trilinear interpolation (see Fig. 6) with learnt tensors $P, A \in \mathbb{R}^{4 \times 3 \times 3}$, $W \in \mathbb{R}^{4 \times 3 \times 3 \times |N_s|}$:

$$\mathbf{P} = C(\mathbf{V}) + \mathbf{P} \times_1 f_4(a) \times_2 f_3(y) \times_3 f_3(x) \quad (5)$$

$$\mathbf{A} = \mathcal{A} \times_1 f_4(a) \times_2 f_3(y) \times_3 f_3(x) \quad (6)$$

$$\mathbf{W} = \mathcal{W} \times_1 f_4(a) \times_2 f_3(y) \times_3 f_3(x) \quad (7)$$

where $C$ computes the eye corners midpoint given vertices $V$. Once the retargeted mesh vertices $\hat{V}_s = W(V_s^*, P, A, W)$ are obtained, the residual is added using another trilinear interpolation:

$$\mathbf{V}_s = \hat{V}_s + (\bar{D} + D \times_1 \vec{\beta}) \times_2 f_4(a) \times_3 f_3(y) \times_4 f_3(x) \quad (8)$$

where $D$ is the residual PCA tensor learnt after removing the low frequency skinning deformations from the training data. The final skin shape is obtained by *shrinkwarping* the eye contour vertices on the eyeball to fill potential gaps. In addition, all the vertices closer than the eyelid thickness $h$ to the eyeball are projected back to this minimum distance.

**Color modeling.** The skin and eyeball colors are represented with rgb textures $I_s$ and $I_e$ of sizes $S_s \times S_s$ and $S_e \times S_e$, respectively. The skin texture depends on the color identity code $\vec{\gamma}$ but also on the eye pose $\vec{\psi} = \{a, \theta, \phi\}$ (see Fig. 7). It is obtained via linear interpolation in the tensor $I_s \in \mathbb{R}^{|\vec{\gamma}| \times 4 \times 3 \times 3 \times 3S_e^2}$

$$I_s = (\bar{I}_s + I_s \times_1 \vec{\gamma}) \times_2 f_4(a) \times_3 f_3(y) \times_4 f_3(x) \quad (9)$$

The eyeball color is directly obtained with a linear combination of its code vector $\vec{\mu}$ with the base eyeball texture tensor $I_e \in \mathbb{R}^{|\vec{\mu}| \times 3S_e^2}$:

$$I_e = \bar{I}_e + \vec{\mu} \cdot I_e \quad (10)$$

## 4.2. Training

Training the model consists in learning the previously mentioned tensors $V, T, P, A, W$ and $I$. The tensors $V, T$, and $I$ are obtained

by performing PCA on the first dimension of data tensors $V_{data} \in \mathbb{R}^{n \times 4 \times 3 \times 3 \times 3N_s}$, $T_{data} \in \mathbb{R}^{n \times 4 \times 3 \times 3 \times 3}$, $I_{data} \in \mathbb{R}^{n \times 4 \times 3 \times 3 \times 3S_e^2}$ respectively, where $n$ is the number of samples in the training dataset. The tensors $P, A, W$ are optimized to minimize the $L_1$ distance between the retargeted neutral shapes and the ground truth expressive shapes:

$$\arg \min_{P, A, W} |V_{data} - W(V_{data}^*, P, A, W)| \quad (11)$$

Convergence was achieved after ~500 iterations with the Adam optimizer [54] shipped in Pytorch library, taking about 2 min on a single NVIDIA Quadro P3200 GPU. It must be noted that $D$ mentioned in Eq. (8) is learnt on the residual $V_{data} - W(V_{data}^*, P, A, W)$.

## 4.3. Evaluation

Compared to existing face Models with eye and eyeball integration, DREAM is the only one that represents conjointly the morphology from the eye pose i.e. the cartesian product of the gaze direction and eyelid aperture. This quality is essential to extract an animation-ready rig from limited data. FLAME [25] has been shown very reliable for geometric and photometric fitting thanks to its large expressivity. However This kind of model does not expose semantic knobs to edit or animate one's face with great fidelity. DREAM's philosophy leans more towards the semantics of the parameters and the link between the morphology and the deformation space.

The eye region model from Wood et al. [41] has been designed for synthetic generation purposes. In this context, plausibility of generated samples is crucial. With the help of rotations, Wood et al.effectively model the deformations of the eyelids adapted to the morphology. However, these rotations only concern the vertical gaze angle and omit other aspects such as lateral displacements. As depicted in Fig. 8, DREAM also uses rotation joints learnt from data, which allows for spherical eyelid motion. In particular Fig. 9 shows a comparison between linear interpolation [13] and our rotation-based interpolation.

# 5. Using DREAM for scan fitting

DREAM models conjointly the shape and color with the eye pose. In this section, we propose a practical application of DREAM: extracting an animation-ready eye rig from a single static textured scan. To do so, we optimize the model parameters to fit an unseen scan with known pose (typically, the neutral pose) using the strategy described in the next paragraph.
<figure><img src="image_10.png" /><figcaption>**Fig. 8.** Joints position (base of the arrows) and axes (direction of the arrows) in frontal view (left), zoomed frontal view (middle) and top down view (right). Blue arrow represents both the joints corresponding to eyelid aperture and vertical gaze angle while green arrow corresponds to the horizontal gaze direction joint.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>**Fig. 9.** Illustration of the rotation-based interpolation (green) versus linear interpolation (red). The colored lines represent the path of a point on the upper eyelid when transitioning from look down to look up.</figcaption></figure>

## 5.1. Fitting strategy

We optimize DREAM with a numerical gradient descent by seeking for correspondences from the model surface to the scan surface.

**Inner and outer regions.** In the process of fitting DREAM to a scan, we want to focus the accuracy on the inner part of the eye i.e. giving less importance to the static areas which are the forehead, the nose and the cheek. To do so, we spatially and smoothly blend DREAM with traditional PCA models for the shape and textures on the outer areas.

**Source points sampling.** First, we sample $P_s$ points on the skin surface and $P_e$ points on the eyeball surface. Sampling surface points, instead of using mesh vertices, has the advantage to better control the number and distribution of the sampled points. For the skin part, we sample substantially more points on the eyelids than on the forehead, nose and cheek to focus on the inner eye region. This is done by reusing the vertex weight mask as a probability in the sampling. Concerning the eyeball, half the points are sampled on the sclera and the other half on the cornea.

**Eyeball visibility.** The target scan only contains the outer surface of the person's eye, so the obstructed part of the model eyeball should not intervene in the optimization objective function. To distinguish between the visible and the obstructed part of the eyeball, we project the eyelid contour vertices on the eyeball mesh and compute their texture coordinates, so they form a closed loop with 2D segments. For any point on the eyeball, we take its texture coordinates and compute its signed distance to the closed loop. If it is negative, the point is visible, else it is obstructed and can be discarded from the objective function.

**Target point matching.** The sampled points are matched against their closest point on the target mesh surface. We compute their associated normal and color by sampling the scan texture.

**Eye segmentation.** Even if fitting our model to a scan produces satisfactory results with no prior annotations, we saw that leveraging spatial information on the eye eyelid interface, iris contour and pupil contour can effectively guide the optimization to a better solution. We propose to leverage a semantic segmentation of the target eyeball including the sclera, the iris and the pupil. Contours of these structures can easily be extracted from image gradients and computed back in 3D on the surface of the scan. For the iris and pupil contours, we fit 3D circles to the obtained 3D points.

**Rigid transform.** In addition to optimizing DREAM's parameters, we learn a rigid transform with a scale, a translation and a rotation represented as a rotation logarithm map. Optimizing conjointly the rigid and non-rigid aspects has been shown successful in 3D face registration [55]. We initialize the rigid transform by Procrustes analysis w.r.t the landmarks described in Appendix B of the appendix. This rough alignment is enough to guide the optimization to a good solution even with no spatial supervision. The semantic segmentation maps are hand-made here, but current state-of-the-art in eye semantic techniques could be used without harming much the segmentation accuracy.

## 5.2. Objective function

In the scan fitting process, we minimize the following energy:

$$E_{\text{tot}} = E_{\text{skin}} + E_{\text{eyeball}} + E_{\text{seg}} + \lambda_{\text{tc}} E_{\text{tc}}$$

with

$$E_{\text{skin}} = \lambda_{\text{sg}} E_{\text{sg}} + \lambda_{\text{sn}} E_{\text{sn}} + \lambda_{\text{sp}} E_{\text{sp}} + E_{\text{sr}}$$

$$E_{\text{eyeball}} = \lambda_{\text{eg}} E_{\text{eg}} + \lambda_{\text{ep}} E_{\text{ep}} + E_{\text{er}}$$

$$E_{\text{seg}} = \lambda_{\text{int}} E_{\text{int}} + \lambda_{\text{iri}} E_{\text{iri}} + \lambda_{\text{pup}} E_{\text{pup}}$$
In the following, we refer to the 3D positions, normal and color of the source sampled points as $\vec{x}_i$, $\vec{n}_i$ and $\vec{c}_i$. For their match on the target scan, we use the notation $\vec{X}_i$, $\vec{N}_i$ and $\vec{C}_i$. For simplicity, the notation does not depend on whether the points are sampled from the skin or the eyeball surface. $P_s$ are the sampled points on the skin surface and $P_e$ on the eyeball surface.

The **skin geometric energy** $E_{sg}$ is the mean distance between the sampled points $\vec{x}_i$ and their corresponding matches $\vec{X}_i$ on the target scan surface. The distance is weighted by the normal cosine of the normals $\vec{n}_i \cdot \vec{N}_i$ and the color similarity $1 - \|\vec{c}_i - \vec{C}_i\|_1$ to discourage merge between points that differ in orientation and color.

$$E_{sg} = \frac{1}{P_s} \sum_{i}^{P_s} \|\vec{x}_i - \vec{X}_i\|_2 (\vec{n}_i \cdot \vec{N}_i) (1 - \|\vec{c}_i - \vec{C}_i\|_1) \quad (12)$$

The **skin normal energy** $E_{sn}$ encourages the model to orient the sampled point normal $\vec{n}_i$ like their matches $\vec{N}_i$.

$$E_{sn} = \frac{1}{P_s} \sum_{i}^{P_s} \vec{n}_i \cdot \vec{N}_i \quad (13)$$

The **skin photometric energy** $E_{sp}$ makes it possible to adjust the skin texture $I_s$ given the color difference between source and matched points.

$$E_{sp} = \frac{1}{P_s} \sum_{i}^{P_s} \|\vec{c}_i - \vec{C}_i\|_1 \quad (14)$$

The **skin regularization energy** $E_{sr}$ limits the norm of the code vectors $\vec{\beta}$ and $\vec{\gamma}$ to preserve the plausibility of the generated shape and texture.

$$E_{sr} = \lambda_{\beta} \frac{\|\vec{\beta}\|_2^2}{|\vec{\beta}|} + \lambda_{\gamma} \frac{\|\vec{\gamma}\|_2^2}{|\vec{\gamma}|} \quad (15)$$

The **eyeball geometric energy** $E_{eg}$ computes the distance between the eyeball visible sampled points and their matches on the scan. The cornea area is poorly reconstructed due to light refraction and creates a pit [38]. We add a “iris collapse” blendshape, as shown in Fig. 5, to account specifically for this recurring artifact.

$$E_{eg} = \frac{1}{P_e} \sum_{i}^{P_e} \text{visibility}(\vec{x}_i) \|\vec{x}_i - \vec{X}_i\|_2 \quad (16)$$

The **eyeball photometric energy** $E_{ep}$ is there to optimize the iris color. It only concerns the visible points sampled on the cornea part.

$$E_{ep} = \frac{1}{P_e} \sum_{i}^{P_e} (\text{visibility}_{\text{cornea}})(\vec{x}_i) \|\vec{c}_i - \vec{C}_i\|_1 \quad (17)$$

The **eyeball regularization energy** $E_{er}$ serves the same purpose as $E_{sr}$ but for the eyeball part.

$$E_{er} = \lambda_{\mu} \frac{\|\vec{\mu}\|_2^2}{|\vec{\mu}|} + \lambda_{\tau} \frac{\|\vec{\tau}\|_2^2}{|\vec{\tau}|} \quad (18)$$

The **interface consistency energy** $E_{int}$ helps aligning the eyeball eyelid interface of the model to the one of the scan thank to an input semantic segmentation. Interface points $\vec{X}_{int,j}$ are computed as the image gradient of the segmentation. $E_{int}$ is the average minimum distance between $\vec{X}_{int,j}$ and the $n_{int}$ vertices $\vec{v}_{int,i}$ of the model eyelid contour.

$$E_{int} = \frac{1}{n_{int}} \sum_{i}^{n_{int}} \min_{j} \|\vec{v}_{int,i} - \vec{X}_{int,j}\|_2 \quad (19)$$

The **iris and pupil consistency energies** $E_{iri}$ and $E_{pup}$ have the same effect than the $E_{int}$ but 3D circles have been fitted to the 3D points of iris and pupil contour, and the vertices are from the eyeball this time. They are the mean distance to these respective 3D circles $\mathscr{C}_{iri}$ and $\mathscr{C}_{pup}$.

$$E_{iri} = \frac{1}{n_{iri}} \sum_{i}^{n_{iri}} \min_{\vec{X} \in \mathscr{C}_{iri}} \|\vec{v}_{iri,i} - \vec{X}\|_2 \quad (20)$$

$$E_{pup} = \frac{1}{n_{pup}} \sum_{i}^{n_{pup}} \min_{\vec{X} \in \mathscr{C}_{pup}} \|\vec{v}_{pup,i} - \vec{X}\|_2 \quad (21)$$

We decided not to use the iris structure (as proposed in the work of Bérard et al. [38]) in the energy function, because it is not possible to reconstruct it with the dataset we used. It requires a varying controlled lighting on the images that we do not have.

Finally, the **eyelid margin thickness consistency energy** $E_{tc}$ makes sure that the distance from the eyelid ridges and the eyeball remain equal to $h$, DREAM's parameter for the eyelid thickness that is also optimized in the scan fitting process. The distance to the eyeball is computed with its implicit signed distance function representation $SDF_e$.

$$E_{tc} = \frac{1}{n_{rid}} \sum_{i}^{n_{rid}} |SDF_e(\vec{v}_{rid,i}) - h| \quad (22)$$

### 5.3. Results

We show qualitative results of scan fitting and pose retargeting to assess the strengths of our model.

**Implementation and performances.** The proposed scan fitting algorithm is implemented in Python using the Pytorch library for the numerical gradient descent. Pytorch3D is also used for closest point queries and operations on 3D transforms. With 20k sampled point, the fitting converges in about 100 iterations with the Adam optimizer, taking ~40s on a NVIDIA Quadro P3200. Inference is real time on GPU which leaves room for online optimization in future research.

**Test set.** To effectively assess the performance of DREAM in the scan fitting application, we formed a validation set of 5 persons that have been removed from the training set.

**Scan fitting** We show qualitative results of DREAM fitting. Fig. 10 shows the progressive fitting of DREAM on an unseen identity. The error shown in false colors is computed independently for the skin and eyeball, meaning that a bad alignment of the eye contours would produce important errors, which is not the case here. The mean error is depicted below each thumbnail images and its reduction shows the progressive improvement of the fitting.

We also provide an ablation study to observe the effect of the different energies involved in the optimization procedure in Fig. 11. The histograms of each thumbnails is presented below each of them and the mean error value is indicated below. Quantitatively, the default setup produces the best result and qualitatively We can see that it induces the least error on the eyelids and periocular region. Without the segmentation map, there is substantially more error in the eye contour since the some skin is wrongly interpreted as eyeball and vice-versa. Removing $E_{sn}$ has an important negative impact on the error. Interestingly, removing the geometric term does converge to a somewhat coherent result, probably thank to the information provided by the segmentation map. “Naive $E_{sg}$” corresponds to a simple mean distance without weighting by the normal and color similarity. Using this energy leads to slightly more error, especially on the eyelid ridges. Finally, removing the photometric terms does not have an immense impact on the geometric error, which could be expected.

In Fig. 12, we show samples of model-to-scan fitting using FLAME [25]. The colormap used is the same as all the other figures, showing comparable results with DREAM fits. We used a fitting procedure that aims to minimize the distance between the model and the scan given a certain threshold to cancel the impact of the non-eye region vertices. An $L_2$ regularization loss is added to the PCA identity vector to enforce the anatomical correctness of the result.

**Pose retargeting** The main strength of DREAM comes when extracting, from a single data sample (here, a scan), a fully animatable rig with highly personalized deformations. We fit DREAM to the neutral pose (“look forward, eye normally open”) and retarget the result to new poses “look forward, eye closed”, “look top right, eye normally open” and “look left, eye half closed”. The results are shown in Figs. 13 and
<figure><img src="image_12.png" /><figcaption>Fig. 10. Fitting results at different number of iterations.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 11. Results of the ablation study on DREAM scan fitting. We depicted the corresponding color pixels histogram and mean error stats for each thumbnails.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 12. Fitting results with FLAME [25] on our Eye Region Scans.</figcaption></figure>

14 and illustrate that DREAM can successfully adapt to a new, unseen morphology to predict the whole eye pose space. In these figures, we can see that sometimes the eye poses are very different from person to person and thus, DREAM struggles to replicate the exact same pose. This is partially due to the nature of the dataset in use, which relies on subjective indications such as “eye half closed” or “look as much to right as possible”. This introduces variability in the dataset that does not depend on the person’s morphology, but more on an understanding of indications.

## 6. Discussion and future work

We presented a new model of eye region trained on more than 2000 3D registered meshes. In order to achieve this, we set-up an innovative pipeline composed of eyelash inpainting, automatic land- marking and mesh registration to generate high quality training data. The DREAM model is a multipart 3DMM with explicit skin and eyeball

textured meshes. DREAM is the first data-driven eye region model that covers all the possible eye poses, allowing us to recreate precise identities, gaze directions, eyelid shapes and aperture, and eye region deformations. We also showed a novel scan fitting methodology specific for eye region. We performed a qualitative and quantitative study of DREAM with existing models of the literature and showed that DREAM outperforms existing models and enables the modeling of a larger range of morphologies, gaze directions and eyelids apertures. Extending validation of DREAM on wilder and different datasets is a point we plan to work on. The objective being to highlight the generalization of DREAM beyond the training data we presented.

On different datasets, our proposed segmentation-guided fitting process might have some limitations. It may suffer from accuracy where the segmentation is noisy, incomplete, or inaccurate. It might also show a semantic misalignment where the target meshes use different segmen- tation conventions, thus it can be hard to establish correspondences.
<figure><img src="image_15.png" /><figcaption>Fig. 13. Fitting and pose retargeting results.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Fig. 14. Fitting and pose retargeting results.</figcaption></figure>

And it can suffer from mesh artifacts, such as holes, noise, or irregular triangulation which may affect both segmentation and fitting.

On the rendering side, we do think that better rendering pipelines would have brought nicer visuals. Even if our main contributions in this paper is on the modeling and fitting side of our model, combining our final results with different rendering approaches could lead to an interesting study. Rendering photorealistic human eyes is one of the most challenging tasks in computer graphics due to the complex interplay of geometry, materials, and light. To achieve realism, you typically need a combination of specialized shaders that simulate the anatomical and optical properties of the eye such as for cornea, sclera (white part), iris, pupil, tear film and eyelids.

DREAM is designed to statically model as accurately as possible eye region, however we currently do not capture dynamic information. In a future version, this dynamic information would be beneficial for a dynamic eye rig. It would also help the model to fit video sequences. As Bérard et al. [40] showed, the dynamics of the eye are biometric. This means that we each have our own way of moving our eye and that the eye saccades are different, in terms of frequency and amplitude, from one person to another. Integrating scalar measures (in angle) for gaze direction and eyelid aperture could also be an interesting improvement for this dataset to keep improving the quality of the fitting. DREAM is connected to a nose, a cheek and an eyebrow sharing the same topology as the ICT-FaceKit face model. It will be of high interest, in a near future, to combine these two models, enabling to better capture both the eye and face together. Extending it to a body model such as SMPL [56] would also be of interest as the body tends to follow the eyes.

It would also be of high interest to represent both eyes in the model to encode impacts that both eyes have on each other (it is difficult to move one eye without moving the second one). So far, the dataset of

Kerbiriou et al. [13] only contains left eyes. The link between eyes cannot be highlighted with this dataset. Setting up a specific capture session with both eyes coupled with different eyelids aperture and gaze directions would be a great benefit for the eye modeling community.

There are many possible ways for extending the DREAM model. It could be on integrating additional ocular features such as tear film or eyelashes. It could also be on a coupling it with facial blendshapes such as cheek raiser, nose wrinkler etc. with are impacting on eye regions.

Finally, and more importantly, to improve the model fitting feature, it would be of high interest to investigate the coupling between our DREAM model and existing model of the iris [38] and other fine-grained ocular structures [35]. Merging an iris surface detection approach to DREAM would probably provide better fitting performances.

## 7. Conclusion

We presented a new model of the eye region trained on more than 2000 3D registered meshes. In order to achieve this, we set-up an innovative pipeline composed of eyelash inpainting, automatic landmarking and mesh registration to generate high quality training data. The DREAM model is a multipart 3DMM with explicit skin and eyeball textured meshes. DREAM is the first data-driven eye region model that covers all the possible eye poses, enabling the recreation of precise identities, gaze directions, eyelid shapes and aperture, and eye region deformations. We also showed a novel scan fitting methodology specific for the eye region. We performed a qualitative and quantitative study of DREAM with existing models of the literature and showed that DREAM outperforms existing models and enables the modeling of a larger range of morphologies, gaze directions and eyelids apertures.
## CRediT authorship contribution statement

**Glenn Kerbiriou:** Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Data curation.
**Quentin Avril:** Writing – review & editing, Writing – original draft, Supervision, Project administration, Methodology, Data curation, Conceptualization.
**Maud Marchal:** Writing – review & editing, Writing – original draft, Supervision, Resources, Project administration, Methodology, Investigation, Conceptualization.

## Declaration of competing interest

**1. Support for the Reported Work:** This work has been founded by Interdigital and INSA of Rennes, France.

**2. Relevant Support Outside this Work:** None

**3. Intellectual Property:** The three authors (and only) are involved in the two following patents that are relevant to the work in the manuscript.

* “Eyelashes Semantic Model” 2023P00467EP

* “Semantic Multi-linear Geometrical Eye Model” 2024P00647EP

**4. Other Activities:** None

## Appendix A. Eyelashes cleaning

Eyelashes are highly challenging to reconstruct faithfully through a photogrammetry process [57]. In practice, photogrammetry based reconstruction generates undesired artifacts for dense eyelashes.

These artifacts are clearly visible on the mesh registration output, thus generating mesh issues for the design of eye region model. The recent work EyelashNet [58] tackles this specific issue proposing a dedicated hair segmentation network [59] adapted to eyelashes. This network estimates a continuous mask of eyelashes presence confidence in the source images, so that an inpainting algorithm can remove them softly. The authors showed that feeding a standard photogrammetry pipeline with the inpainted images leads to much better eye reconstruction quality. We therefore leveraged EyelashNet to segment the eyelashes in all source images of the scans dataset. We used a popular inpainting method [60] implemented in OpenCV [61], and reconstructed again the 57 × 36 = 2052 scans. This process fixed almost all the unwanted bulges (see Fig. A.15 for an illustration of the eyelashes cleaning process). EyelashNet segmentation result is a continuous mask from 0 to 255, 255 being that the current pixel is confidently an eyelash pixel. A thresholding step is necessary to use the inpainting algorithm. If the threshold value is too high, the eyelashes are not inpainted enough and the artifacts remain. If, on the contrary, this threshold is too small, the inpainting process generates blurry areas that result in too flat areas on the reconstructed surface. In our experiments, we found that a threshold value of 50 is a good trade-off between the two, even if eyelashes are still slightly visible in source images (see Fig. A.15).

## Appendix B. Template registration

We describe in this section of the appendix the registration process we use on all the scans.

<caption>**Table B.1**</caption>

Comparisons between different input types for the 3D landmarking method [51] in terms of mean error (mm).

<table><thead><tr><td>Input type</td><td>Mean (l)</td><td>Std (l)</td></tr></thead><tbody><tr><td>Depth</td><td>1.032</td><td>0.721</td></tr><tr><td>Geometry</td><td>0.909</td><td>0.504</td></tr><tr><td>RGB</td><td>0.887</td><td><strong>0.490</strong></td></tr><tr><td>Geometry + Depth</td><td>1.003</td><td>0.601</td></tr><tr><td>RGB + Depth</td><td>0.886</td><td>0.616</td></tr><tr><td>RGB + Geometry</td><td><strong>0.815</strong></td><td>0.518</td></tr></tbody></table>

### B.1. Sparse correspondences

First, sparse correspondences are computed using an automatic landmarking process. We automatically predict landmarks' position by adapting an existing mesh landmarking method [51]. This strategy consists in training a Convolutional Neural Network (CNN) on rendered scans while varying the camera position and the shading. To predict the 3D position of a landmark, a ray is casted from each point of view through the 2D prediction location and a consensus point is computed with the help of a RANSAC least squares fitting algorithm. We adapted this technique to the specific case of the eye region by varying eyelid aperture (two possibilities: at least half open and closed eye) and gaze directions (three possibilities: down, middle, up). We trained four individual networks each for a different cluster of eye poses. The clusters are “eye at least half open + look up”, “eye at least half open + look middle”, “eye at least half open + look down” and “closed eye”. In practice, the dataset was annotated in a semi-automatic manner. Only the poses “eye open look up”, “eye open look forward”, “eye open look down” and “eye closed look forward” were manually annotated. Variance introduced by manual annotation can be significant and depends on many factors. We respected the following rules to reduce it as much as possible: (1) a clear landmark definition with guidelines, (2) an annotation protocol with specific order to follow, (3) a precise annotating tool with zoom on the images, (4) an high level of quality on the images and (5) no non expert annotators with only a few annotations a day to reduce fatigue. Four networks were again trained on the fully annotated dataset (one for each cluster), leading to a fully automatic system for eye region 3D landmarking.

In the original dataset [13], the last 11 landmarks are placed on mostly static locations so we decided to predict the 21 first landmarks only. This new landmarks layout results in even better estimation as illustrated in Fig. B.17.

To evaluate the performances of this landmarking system, we used the mean error in millimeters and split our dataset into 47 training samples and 10 test samples. In this experiment, 96 view points per scan were uniformly distributed around the center of the eye. The training phase ran for 100 epoch in the same settings as in the original work [51] on a NVidia Tesla V100 GPU. In Table B.1, six different combinations of three input types are compared. The three types are “RGB” (texture only), “Geometry” (white color with diffuse lighting) and “Depth” (depth buffer output). The pair combinations are “Geometry + Depth”, “RGB + Depth” and “RGB + Geometry” where images of different types are concatenated following the channel axis. The “RGB” input appears to be the best candidate as it minimizes the NME over the test set. Surprisingly, combining it with depth or geometry information did not lead to significantly better results. In more details, “RGB” leads to a better landmarking of the tissues, it is faster to train and easier to use in practice.

### B.2. Dense correspondences

Once sparse correspondences are computed, the neutral eye pose (Eye open, look forward) was used to register our template mesh.
<figure><img src="image_17.png" /><figcaption>**Fig. A.15.** Illustration of the impact of eyelash removal on eye region reconstruction and parametrization pipeline. Left: Eyelash segmentation and inpainting. Right: comparison of reconstruction and parametrization with and without eyelash removal. The reader can notice how the bulges formed by the eyelashes are smoothed out when the eyelashes are inpainted in the first place.</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>**Fig. B.16.** Information related to our topology. (a) Our topology as a subset of ICT-FaceKit topology [28] focusing on the left eye with $N_e = 1407$ vertices and $F_s = 2664$ triangles. (b) Landmarks located near the eyelid/eyeball interface, used in model evaluation. (c) Weights mask used during the scan fitting optimization.</figcaption></figure>

Based on this registration, the $N_s = 1407$ vertices are placed in the texture space of the scan. At the same time, we seek for warps that map every textures to the neutral texture. First, SIFT [62] features were used to find the best rigid transform for each texture. Then, we leveraged RAFT, a state-of-the-art optical flow estimator [63] to warp locally the rigidly aligned textures onto the neutral texture. In this manner, all the template mesh vertices can be found on the surface of “expressive” scans. The quality of these new matches are assessed with Spatio-temporal Edge Distance (SED) metric [64], and vertices with an associated SED below an empirically fixed threshold value of 0.06 are counted as additional landmarks.

Template registration is then achieved on all the expressive poses with the help of both the sparse and dense landmarks. Resulting mesh vertices are finally arranged in a tensor $V_{data} \in \mathbb{R}^{N_i \times 4 \times 3 \times 3 \times N_s}$ where $N_i = 57$ is the number of captured identities and $N_s$ the number of vertices of the registered skin.

## Appendix C. Supplementary data

Supplementary material related to this article can be found online at https://doi.org/10.1016/j.cag.2025.104425.
<figure><img src="image_19.png" /><figcaption>**Fig. B.17.** Qualitative results of the 3D scan landmarking system trained on *N* = 32 landmarks (green) and *N* = 21 landmarks (blue) with ground truth (red). Notice how the *N* = 21 version is better at tracking the inner corner tissues border than the *N* = 32 one. The 11 other green landmarks are located outside this viewport.</figcaption></figure>

## Data availability

The authors do not have permission to share data.

## References

[1] Danieau Fabien, Gubins Iija, Olivier Nicolas, Dumas Olivier, Denis Bernard, Lopez Thomas, Mollet Nicolas, Frager Brian, Avril Quentin. Automatic generation and stylization of 3d facial rigs. IEEE Conf Virtual Real 2019;784-92.
[2] Bagautdinov Timur, Wu Chenglei, Simon Tomas, Prada Fabián, Shiratori Takaaki, Wei Shih En, Xu Weipeng, Sheikh Yaser, Saragih Jason. Driving-signal aware full-body avatars. ACM Trans Graph 2021;40(4):17.
[3] Bi Sai, Lombardi Stephen, Saito Shunsuke, Simon Tomas, Wei Shih En, McPhail Kevyn, Ramamoorthi Ravi, Sheikh Yaser, Saragih Jason. Deep relightable appearance models for animatable faces. ACM Trans Graph 2021;40(4).
[4] Genay Adelaide, Lecuyer Anatole, Hachet Martin. Being an Avatar 'for Real': A Survey on Virtual Embodiment in Augmented Reality. IEEE Trans Vis Comput Graphics 2022;28(12):5071-90.
[5] Choi Byungkuk, Eom Haekwang, Mouscadet Benjamin, Cullingford Stephen, Ma Kurt, Gassel Stefanie, Kim Suzi, Moffat Andrew, Maier Millicent, Revelant Marco, Letteri Joe, Singh Karan. Animatomy: an Animator-centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer. ACM Trans Graph 2022.
[6] Sang Shen, Zhi Tiancheng, Song Guoxian, Liu Minghao, Lai Chunpong, Liu Jing, Wen Xiang, Davis James, Luo Linjie. AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging. ACM Trans Graph 2022.
[7] Wang Ting Chun, Mallya Arun, Liu Ming Yu. One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing. In: IEEE computer society conference on computer vision and pattern recognition. 2021, p. 10034-44.
[8] Avril Quentin, Regateiro João, Gosselin Philippe-Henri, Le Clerc François, Guillotel Philippe. MORGAN: MPEG original reference geometric avatar neutral. In: ACM conference on 3D web technology. 2023, p. 1-10.
[9] Olivier Nicolas, Hoyet Ludovic, Danieau Fabien, Argelaguet Ferran, Avril Quentin, Lecuyer Anatole, Guillotel Philippe, Multon Franck. The impact of stylization on face recognition. In: ACM symposium on applied perception. 2020, p. 1-9.
[10] Olivier Nicolas, Kerbiriou Glenn, Arguelaguet Ferran, Avril Quentin, Danieau Fabien, Guillotel Philippe, Hoyet Ludovic, Multon Franck. Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning. Front Virtual Real 2022;2.

[11] Hussien Sura Abed Sarab, Abed Basim Najim Al-din, Ibrahim Kamaran Adil. Encrypting text messages via iris recognition and gaze tracking technology. Mesopotamian J CyberSecurity 2025;5(1):90-103.
[12] Ali Guma, Eid Marwa M, Ahmed Omar G, Abotaleb Mostafa, Alaabdin Anas M Zein, Buruga Bosco Apparatus. Artificial intelligence in corneal topography: A short article in enhancing eye care. Mesopotamian J Artif Intell Heal 2023;2023:31-4.
[13] Kerbiriou Glenn, Avril Quentin, Danieau Fabien, Marchal Maud. Detailed Eye Region Capture and Animation. Comput Graph Forum 2022.
[14] Li Gengyan, Meka Abhimitra, Mueller Franziska, Buehler Marcel C, Hilliges Otmar, Beeler Thabo. EyeNeRF: A hybrid representation for photorealistic synthesis, animation and relighting of human eyes. ACM Trans Graph 2022;41(4).
[15] Li Gengyan, Sarkar Kripasindhu, Meka Abhimitra, Buehler Marcel, Mueller Franziska, Gotardo Paulo, Hilliges Otmar, Beeler Thabo. ShellNeRF: Learning a Controllable High-resolution Model of the Eye and Periocular Region. Comput Graph Forum 2024.
[16] Blanz Volker, Vetter Thomas. A morphable model for the synthesis of 3D faces. ACM Trans Graph 1999;187-94.
[17] Vlasic Daniel, Brand Matthew, Pfister Hanspeter, Popović Jovan. Face transfer with multilinear models. ACM Trans Graph 2005;24(3):426-33.
[18] Garrido Pablo, Valgaerts Levi, Wu Chenglei, Theobalt Christian. Reconstructing detailed dynamic face geometry from monocular video. ACM Trans Graph 2013;32(6):1581-15810.
[19] Kim Hyeongwoo, Garrido Pablo, Tewari Ayush, Xu Weipeng, Thies Jus-tus, Nießner Matthias, Pérez Patrick, Richardt Christian, Zollöfer Michael, Theobalt Christian. Deep video portraits. ACM Trans Graph 2018;37(4):163.
[20] R Mallikarjun B, Tewari Ayush, Seidel Hans-Peter, Elgharib Mohamed, Theobalt Christian. Learning complete 3D morphable face models from images and videos. IEEE/CVF Conf Comput Vis Pattern Recognit 2021;3360-70.
[21] Blanz V, Vetter T. Face recognition based on fitting a 3D morphable model. IEEE Trans Pattern Anal Mach Intell 2003;25(9):1063-74.
[22] Paysan Pascal, Knothe Reinhard, Amberg Brian, Romdhani Sami, Vetter Thomas. A 3D face model for pose and illumination invariant face recognition. In: IEEE international conference on advanced video and signal based surveillance. 2009, p. 296-301.
[23] Cao Chen, Weng Yanlin, Zhou Shun, Tong Yiying, Zhou Kun. FaceWarehouse: A 3D facial expression database for visual computing. IEEE Trans Vis Comput Graphics 2014;20(3):413-25.
[24] Bolkart Timo, Wuhrer Stefanie. A groupwise multilinear correspondence optimization for 3D faces. IEEE Int Conf Comput Vis 2015;3604-12.
[25] Li Tianye, Bolkart Timo, Black Michael J, Li Hao, Romero Javier. Learning a model of facial shape and expression from 4D scans. ACM Trans Graph 2017;36(6):1941-19417.
[26] Gerig Thomas, Forster Andreas, Blumer Clemens, Egger Bernhard, Lüthi Marcel, Schönborn Sandro, Vetter Thomas. Morphable face models - an open framework. IEEE Int Conf Autom Face & Gesture Recognit 2017;75-82.

[27] Wood Erroll, Baltrušaitis Tadas, Morency Louis-Philippe, Robinson Peter, Bulling Andreas. Learning an appearance-based gaze estimator from one million synthesised images. In: ACM symposium on eye tracking research and applications. 2016, p. 131-8.

[28] Li R, Bladin K, Zhao Y, Chinara C, Ingraham O, Xiang P, Ren X, Prasad P, Kishore B, Xing J, Li H. Learning formation of physically-based face attributes. IEEE/CVF Conf Comput Vis Pattern Recognit 2020;3407-16.

[29] Chandran P, Bradley D, Gross M, Beeler T. Semantic deep face models. Int Conf 3D Vis 2020;345-54.

[30] Wood Erroll, Baltrušaitis Tadas, Hewitt Charlie, Dziedzio Sebastian, Cashman Thomas J, Shotton Jamie. Fake it till you make it: face analysis in the wild using synthetic data alone. IEEE/CVF Int Conf Comput Vis 2021;3681-91.

[31] Egger Bernhard, Smith W, Tewari Ayush, Wuhrer Stefanie, Zollhöfer Michael, Beeler Thabo, Bernard Florian, Bolkart Timo, Kortylewski Adam, Romdhani Sami, Theobalt Christian, Blanz Volker, Vetter Thomas. 3D morphable face models—Past, present, and future. ACM Trans Graph 2020;39:1-38.

[32] Wu C, Bradley D, Garrido P, Zollhöfer M, Theobalt C, Gross M, Beeler T. Model-Based Teeth Reconstruction. ACM Trans Graph 2016;35(6).

[33] Liu Yilong, Zheng Chengwei, Xu Feng, Tong Xin, Guo Baining. Data-driven 3D neck modeling and animation. IEEE Trans Vis Comput Graphics 2021;27(7):3226-37.

[34] Ploumpis Stylianos, Moschoglou Stylianos, Triantafyllou Vasileios, Zafeiriou Stefanos. 3D human tongue reconstruction from single" in-the-wild" images. IEEE/CVF Conf Comput Vis Pattern Recognit 2022;2771-80.

[35] Bermano Amit, Beeler Thabo, Kozlov Yeara, Bradley Derek, Bickel Bernd, Gross Markus. Detailed spatio-temporal reconstruction of eyelids. ACM Trans Graph 2015;34(4).

[36] Neog Debanga R, Cardoso João L, Ranjan Anurag, Pai Dinesh K. Interactive gaze driven animation of the eye region. In: International conference on web3D technology. 2016, p. 51-9.

[37] Wen Quan, Xu Feng, Lu Ming, Yong Jun-Hai. Real-time 3D eyelids tracking from semantic edges. ACM Trans Graph 2017;36(6).

[38] Bérard Pascal, Bradley Derek, Nitti Maurizio, Beeler Thabo, Gross Markus. High-quality capture of eyes. ACM Trans Graph 2014;33(6).

[39] Bérard Pascal, Bradley Derek, Gross Markus, Beeler Thabo. Lightweight eye capture using a parametric model. ACM Trans Graph 2016;35(4).

[40] Bérard Pascal, Bradley Derek, Gross Markus, Beeler Thabo. Practical person-specific eye rigging. Comput Graph Forum 2019;38.

[41] Wood Erroll, Baltrušaitis Tadas, Morency Louis-Philippe, Robinson Peter, Bulling Andreas. A 3D Morphable Eye Region model for gaze estimation. In: European conference on computer vision. 2016, p. 297-313.

[42] Kerbiriou Glenn, Avril Quentin, Marchal Maud. 3D reconstruction and semantic modeling of eyelashes. Comput Graph Forum 2024;43.

[43] Lewis JP, ichi Anjyo Ken, Rhee Taehyun, Zhang Mengjie, Pighin Frédéric H, Deng Zhigang. Practice and theory of blendshape facial models. Comput Graph Forum 2014.

[44] Lombardi Stephen, Saragih Jason, Simon Tomas, Sheikh Yaser. Deep appearance models for face rendering. ACM Trans Graph 2018;37(4):68:1-68:13.

[45] Schwartz Gabriel, Wei Shih-En, Wang Tenia, Lombardi Stephen, Simon Tomas, Saragih Jason M. The eyes have it: An integrated eye and face model for photorealistic facial animation. ACM Trans Graph 2020;39(4).

[46] Abrevaya Victoria Fernandez, Wuhrer Stefanie, Boyer Edmond. Multilinear autoencoder for 3D face model learning. In: IEEE winter conference on applications of computer vision. 2018, p. 1-9.

[47] Yang Haotian, Zhu Hao, Wang Yanru, Huang Mingkai, Shen Qiu, Yang Ruigang, Cao Xun. FaceScape: a large-scale high quality 3D face dataset and detailed riggable 3D face prediction. IEEE Conf Comput Vis Pattern Recognit 2020.

[48] Jiang Zi-Hang, Wu Qianyi, Chen Keyu, Zhang Juyong. Disentangled representation learning for 3D face shape. IEEE/CVF Conf Comput Vis Pattern Recognit 2019;11949-58.

[49] Olivier Nicolas, Baert Kelian, Danieau Fabien, Multon Franck, Avril Quentin. FaceTuneGAN: Face autoencoder for convolutional expression transfer using neural generative adversarial networks. Comput Graph 2023.

[50] Sumner Robert W, Popović Jovan. Deformation transfer for triangle meshes. ACM Trans Graph 2004;23(3):399-405.

[51] Paulsen Rasmus R, Juhl Kristine Aavild, Haspang Thilde Marie, Hansen Thomas, Ganz Melanie, Einarsson Gudmundur. Multi-view consensus CNN for 3D facial landmark placement. Asian Conf Comput Vis 2018;706-19.

[52] Bron Anthony, Benjamin Larry, Snibson Grant. Meibomian gland disease. Classification and grading of lid changes. Eye (London, England) 1991;5 (Pt 4):395-411.

[53] Wang Da-Hu, Li Zhao-Yan, Liu Xin-Quan. Measurement of the Lower Lid Margin Thickness by Anterior Segment Optical Coherence Tomography. Ophthalmic Res 2020;64(1):22-7.

[54] Kingma Diederik P, Ba Jimmy. Adam: A method for stochastic optimization. In: Bengio Yoshua, LeCun Yann, editors. Int Conf Learn Represent 2015.

[55] Schneider David C, Eisert Peter. Fast nonrigid mesh registration with a data-driven deformation prior. In: IEEE international conference on computer vision workshops, ICCV workshops. 2009, p. 304-11.

[56] Loper Matthew, Mahmood Naureen, Romero Javier, Pons-Moll Gerard, Black Michael J. SMPL: A skinned multi-person linear model. ACM Trans Graph 2015;34(6):248:1-248:16.

[57] Nam Giljoo, Wu Chenglei, Kim Min H, Sheikh Yaser. Strand-accurate multi-view hair capture. IEEE/CVF Conf Comput Vis Pattern Recognit 2019.

[58] Xiao Qinjie, Zhang Hanyuan, Zhang Zhaorui, Wu Yiqian, Wang Luyuan, Jin Xiaogang, Jiang Xinwei, Yang Yong-Liang, Shao Tianjia, Zhou Kun. EyelashNet: A dataset and a baseline method for eyelash matting. ACM Trans Graph 2021;40(6).

[59] Li Yaoyi, Lu Hongtao. Natural image matting via guided contextual attention. AAAI Conf Artif Intell 2020;34:11450-7.

[60] Telea Alexandru. An image inpainting technique based on the fast marching method. J Graph Tools 2004;9.

[61] Bradski G. The OpenCV Library. Dr Dobb's J Softw Tools 2000.

[62] Lowe David G. Object recognition from local scale-invariant features. IEEE Int Conf Comput Vis 1999;2:1150-7.

[63] Teed Zachary, Deng Jia. Raft: Recurrent all-pairs field transforms for optical flow. Eur Conf Comput Vis 2020;402-19.

[64] Vasa Libor, Skala Vaclav. A perception correlated comparison method for dynamic meshes. IEEE Trans Vis Comput Graphics 2011;17(2):220-30.