Video Face Re-Aging: Toward Temporally Consistent Face Re-Aging

## Abstract

Abstract Video face re-aging deals with altering the apparent age of a person to the target age in videos. This problem is challenging due to the lack of paired video datasets maintaining temporal consistency in identity and age. Most re-aging methods process each image individually without considering the temporal consistency of videos. While some existing works address the issue of temporal coherence through video facial attribute manipulation in latent space, they often fail to deliver satisfactory performance in age transformation. To tackle the issues, we propose (1) a novel synthetic video dataset that features subjects across a diverse range of age groups; (2) a baseline architecture designed to validate the effectiveness of our proposed dataset, and (3) the development of novel metrics tailored explicitly for evaluating the temporal consistency of video re-aging techniques. Our comprehensive experiments on public datasets, including VFHQ and CelebA-HQ, show that our method outperforms existing approaches in age transformation accuracy and temporal consistency. Notably, in user studies, our method was preferred for temporal consistency by 48.1% of participants for the older direction and by 39.3% for the younger direction.

###### Keywords:

## 1 Introduction

Video Face re-aging aims to transform the apparent age in facial videos while ensuring the temporal consistency in both age and identity. This field holds significance relevance across diverse domains, including computer graphics, forensics, entertainment, and advertising. Despite the extensive research conducted in this domain, the challenge remains largely unexplored when it comes to videos. One of the remaining challenges is that existing image-based methods yield inconsistent identities when applied to videos or consecutive frames featuring varying expressions, viewpoints, and lighting conditions.

Recent studies [, ] have leveraged StyleGAN-based [] frameworks to develop techniques for manipulating facial attributes in videos, aiming for greater attribute consistency. However, these approaches frequently fall short in precisely executing the desired age transformations. They commonly rely on assuming a linear path in the latent space for explicit age control, which is not always accurate. As a result, noticeable artifacts frequently emerge in the output when dealing with input data featuring significant age gaps.

Lately, [] have utilized the synthetic images labeled with existing re-aging techniques [] to curate a paired dataset for re-aging. Their study shows that supervised training on synthetic dataset yields favorable outcomes for still images. However, it is well-known that models trained on static images often suffer from temporal inconsistencies.

These observations suggest that training face re-aging on video datasets is beneficial for addressing temporal consistency issues in video re-aging tasks. Therefore, we propose a pipeline to generate a synthetic video dataset. This dataset comprises paired data for supervised training, consisting of various ages, poses, and expressions. Creating this dataset involves three major steps.

Firstly, we utilize StyleGAN to synthesize a face image of a specific individual. Then we apply existing re-aging method SAM [] to generate images of the same person with varying ages (Sec. 3.1.1). Next, we generate key frames which consist of various poses and expressions of that individual (Sec. 3.1.2). Lastly, we produce a natural and continuous motion through key frames (Sec. 3.1.3).

As a result of our pipeline, we obtain paired videos that feature individuals of different ages and in various poses and expressions. For the first time, this has enabled us to construct a video re-aging dataset for supervised training. Our work can be viewed as an extension of the existing research [], which previously introduced a synthetic image dataset for re-aging.

In addition to the dataset, we introduce a baseline architecture designed to utilize the temporal coherence inherent in our proposed video dataset. This architecture primarily comprises recurrent blocks, employing a fusion-based approach that leverages concatenated inputs to exploit temporal consistency. Drawing inspiration from seminal works in video generation [, , ], we incorporate a video discriminator equipped with 3D convolutional layers to ensure both realism and natural motion in the generated videos.

Recognizing that existing aging metrics are not well-suited for video-based methods, we address this gap by developing novel metrics to assess temporal continuity in video-based re-aging. Through extensive experiments, we demonstrate that our video-based architecture produces remarkable results and outperforms existing state-of-the-art methods across various public datasets.

Figure: Figure 1: Our proposed pipeline to construct the video dataset for re-aging. Firstly, high-resolution synthetic facial images are created using StyleGAN [] Subsequently, images of individuals at different target ages are generated using SAM [] for age transformation. Next, key frames are produced by employing OSFV, which alters the pose and expression of these synthetic images. This is achieved without relying on driving images, instead using random values for rotation, translation, and expression keypoints. Finally, motion is added to these key frames using FILM [], creating smooth and high-fidelity motion videos of subjects at different ages.
Refer to caption: x1.png

Given the challenges and limitations of current methods, our work introduces several contributions in video re-aging as follows.

- 1.
For the first time, we introduce a pipeline designed to generate a synthetic video dataset specifically for video re-aging. This dataset features videos of individuals across various ages, poses, and expressions.
- 2.
We present a baseline network architecture custom-designed for our synthesized video dataset. Our generator is built upon a combination of recurrent blocks and U-Net modules, and it utilizes both 2D image-based and 3D video-based conditional discriminators.
- 3.
We propose novel metrics tailored to assess the temporal consistency of re-aging methods. These include Temporal Regional Wrinkle Consistency (TRWC) and Temporally Age Preservation. These metrics provide a robust framework for evaluating the quality of age transformations over time.

## 2 Related Works

### 2.1 Face Image Re-Aging

The study in [] pioneered the use of a conditional GAN for face aging. Subsequently, several influential works such as [, , , , , ] emerged, expanding on this concept. Similar to [], SAM [] emphasizes continuous age progression. SAM is a StyleGAN-based model []
capable of generating high-resolution images. In contrast to other methods, it does not use an age classifier to estimate the input age. FRAN []
trains a simple encoder-decoder network in a supervised manner with a generated synthetic dataset. Rather than adopting age classifiers or embedding, it extends the input to a 5-channel image, including two binary masks for input and target ages. CUSP [] introduces style and content encoders to disentangle the style and content of the input. This approach also incorporates the GB algorithm [] within the CUSP module, ensuring that only age-relevant features are processed. AgeTransGAN [] disentangles the encoded image into identity and age components with two independent modules. PADA [] and FADING [] adopt a text-driven approach and integrates the pre-trained CLIP [] and diffusion models. However, these works only focus on images, and directly applying these methods on individual frames does not consider temporal consistency, affecting the quality of re-aging quality over time.

### 2.2 Face Video Re-Aging

Most video face re-aging methods transform the face by manipulating the age in latent space, except [] that proposes a reinforcement learning method for the sequence of video frames. [] proposes editing the vectors in the StyleGAN latent space with various pre-processing steps that are independently applied to every single frame of an input video. [] identify the inconsistencies in PTI [] and suggest e4e [] encoder with it for finding the pivots. Video editing methods often crop faces as a prepossessing step. To overcome this problem, [] addresses this issue by proposing changes in the initial StyleGAN layers to overcome the cropping problem. [] proposes a diffusion-based editing approach by using [] that disentangles the video into time-dependent features (such as motion) that are applied to each frame and time-independent features (such as identity) which are shared across all the frames. While improving temporal consistency over image-based methods, they still struggle to accurately transform faces to the target age.

### 2.3 Re-Aging Dataset

Video face re-aging presents significant challenges, primarily due to the lack of dedicated video datasets. Existing image-based face re-aging datasets are typically labeled either automatically using an age classifier or manually via crowdsourcing [, , , ]. However, these datasets do not provide paired ages for supervised training. [] proposed a technique to generate a synthetic dataset that is labelled through SAM []. [] also labelled a video dataset, which remains private.

## 3 Video Face Re-Aging Framework

We first describe the pipeline of synthesizing the proposed video dataset. This is followed by introducing our baseline architecture and loss functions. Lastly, we present novel metrics specifically designed for video re-aging methods to quantify their re-aging performance over time.

### 3.1 Re-Aging Video Dataset

#### 3.1.1 Longitudinal Aging Data

Creating a high-quality image re-aging dataset is a crucial in our pipeline because it directly influences the quality of our subsequent video re-aging dataset. However, obtaining these paired image datasets is a challenging task. To overcome this limitation, we turn to insights from [], which show that training on synthetic datasets can yield realistic results on real images. For instance, the neural network can learn how wrinkles change from the synthetic images and apply this knowledge to real images. We refer to these learned changes as delta images $D_{t}$, as illustrated in Fig. 2.

Firstly, we leverage StyleGAN [] which takes a random noise as input and generate high-resolution synthetic image. Then we utilize SAM [] that manipulates the latent vector with StyleGAN for age transformation. The intuition behind choosing SAM over other existing methods is its re-aging performance in terms of age error, that is also evident through our experiments. Using this approach, we construct a synthetic facial image dataset:

$$ $I^{tar}=SAM(G_{z}(z);a^{tar})$ (1) $$

Given a random noise $z$, StyleGAN $G_{z}(\cdot)$ produces a sample image, which is then used as input for SAM [] along with a target age $a^{tar}$. As a result we get an image $I^{tar}$ with apparent age $a^{tar}$ as shown in Fig. 1 (a). Readers can refer to [] for more details about image-level re-aging as this process is greatly inspired by [].

#### 3.1.2 Key Frames Generation

The subsequent step in the outlined pipeline involves key frames generation. This key frames capture specific snapshots or moments in videos to enable seamless transitions between sequences. The challenge lies in acquiring diverse facial images to adequately encapsulate the video’s dynamics, including various poses and expressions. To address this, we utilize a recent face reenactment technique [] that modifies the pose and expression of a source image based on a driving image. Incorporating this method into our pipeline allows us to generate multiple images of an individual with varying poses and expressions. These resulting images serve as key frames.

In this study, we employ the off-the-shelf model OSFV(^1^11Note that we trained OSFV on VFHQ [] dataset for a resolution of 512$\times$512. The training process on VFHQ dataset requires 30 days on 8 A100 GPUs. Additional details are provided in the supplementary materials.)[] to produce synthetic key frames using our image dataset. Our approach diverges from the original methodology in that we rely solely on source images from our dataset. Instead of utilizing driving images, which can be challenging to gather, we employ random values for the rotation matrix $R$, translation matrix $Q$, and expression keypoints $\delta$ to generate various poses and expressions as shown in Fig. 1 (b). We can write this process as follows:

$$ $K=G_{kp}(I^{tar},R,Q,\delta),$ (2) $$

where $K$ represents generated key frames and $G_{kp}$ denotes key frames generator. We generate eight different key frames through this method. While it is also possible to repeat this procedure to create a motion video, but we have observed a degradation of the quality in the resultant videos. Therefore, we come up with an alternative approach to address this problem in next section.

#### 3.1.3 Motion Generation

The final step of our pipeline is motion generation. We leverage the recent work in frame interpolation methods to ensure smooth and high-fidelity motion. Specifically, we employ the method presented in [] to the eight key frames generated in the previous step by recursively generating intermediate frames between them. This iterative process is executed for two consecutive frames and is repeated for every subject across all ages as follows:

$$ ${I}_{t}=FI(I_{t-1},I_{t+1}),$ (3) $$

where $FI$ is motion generation network [] and $I_{t}$ is the frame at $t$ time-step. As a result, we obtain smooth and high-fidelity paired videos for every subject of different ages as shown in Fig. 1 (c).

Figure: Figure 2: Overview of our generator for video re-aging.
Refer to caption: extracted/5470085/Figures/fig2.png

### 3.2 Network Architecture

#### Generator

To fully utilize the advantage of our novel video re-aging dataset, we carefully design the generation scheme to accurately transform faces over time. Following the approach in [], we adopt an alternative method for incorporating input and target ages into our model structure. Recent methods [, ] tend to use pre-trained age classifiers to estimate the input age. We concatenate the input frame $I_{t}$ at $t$ time-step with two spatial masks (one for input age and one for target age) over channel dimensions. These two masks contain constant values representing input and output ages, ranging from 0 to 100 normalized between 0 and 1. This results in a 5-channel masked frame $I^{mask}_{t}$. Mathematically,

$$ $I_{t}^{mask}=[I_{t},M^{inp},M^{tar}],$ (4) $$

where $M^{inp}$ and $M^{tar}$ are spatial masks for input and target ages. $[\cdot]$ denotes the channel-wise concatenation operation. If $V^{mask}=\{I^{mask}_{1},I^{mask}_{2},\dots,I^{mask}_{N}\}$ refers to any input masked sequence $V^{mask}$ for $N$ total number of frames, our generator $G$ produces the output video $\hat{V}$.

$$ $\tilde{V}=G(V^{mask}),$ (5) $$

The proposed $G$ adopts a recursive scheme to consider the temporal information of videos, inspired by [, , ], namely recurrent block function $RB(\cdot)$. We employ the U-Net architecture within the $RB$ function. Please refer to Sec. 9 for the details of U-Net network architecture. This $RB$ stacks the multiple arguments. First, we concatenate the consecutive input frames $[I^{mask}_{t-\Delta t},I^{mask}_{t},I^{mask}_{t+\Delta t}]$. Here, ${I^{mask}_{t-\Delta t}}$ and ${I^{mask}_{t+\Delta t}}$ refers to the two adjacent frames of $I^{mask}_{t}$ with interval step $\Delta t$. We further concatenate the resulting stacked frames with previous hidden state $h_{t-\Delta t}$ and previous output frame $\hat{I}_{t-\Delta t}$. We formulate this overall process as follows:

$$ $h_{t},{D}_{t}=RB([{I^{mask}_{t-\Delta t}},{I^{mask}_{t}},{I^{mask}_{t+\Delta t }},\tilde{I}_{t-\Delta t},h_{t-\Delta t}])$ (6) $$

As a result, we obtain hidden state $h_{t}$ and delta image ${D}_{t}$. Once the delta image is attained, we can easily obtain output frame $\hat{I}_{t}$ through element-wise summation between input $I_{t}$ and $D_{t}$:

$$ $\hat{I}_{t}={D}_{t}+{I_{t}}.$ (7) $$

We have now processed three frames to get the re-aged output of the middle frame. For the remaining consecutive frames, we can pass the hidden state $h_{t}$ and output frame $\hat{I}_{t}$ along with the next consecutive frames in Eq. 6 for the next iteration. This process is repeated for all $N$ frames of the input video $V^{mask}$ to generate the output video $\hat{V}$, as illustrated in Fig. 2.

#### Discriminator

In addition to the image discriminator, we introduce a video discriminator to assess the consistency of age-related high frequency details across consecutive frames. We observed that using only image discriminator produces inconsistent output, leading to flickering. The comparison is presented in our ablation studies. The image discriminator employs a PatchGAN architecture [] to differentiate realistic images from synthetic ones, while for the video discriminator, we adopt a spatio-temporal convolutional network that utilizes 3D convolution layers, similar to the approach used in previous works such as [] and []. Specifically, we exploit the spatio-temporal information from consecutive generated frames in the form of delta images $D_{t}$ as inputs to ensure the preservation of high-level details over time, rather than relying on facial images.

#### Loss Functions

We use the similar loss function as [] by including $L1$ loss, LPIPS loss [], and adversarial loss functions []. We also use adversarial loss functions for image and video discriminators $\mathcal{L}_{adv,I}$ and $\mathcal{L}_{adv,V}$. Our total objective is obtained as:

$$ $\displaystyle\mathcal{L}=\lambda_{L1}\mathcal{L}_{L1}(\hat{V},V_{gt})+\lambda_ {adv,V}\mathcal{L}_{adv,V}(\hat{V},M^{tar})+$ $\displaystyle\ \lambda_{adv,I}\mathcal{L}_{adv,I}(\hat{V},M^{tar})+\lambda_{p} \mathcal{L}_{LPIPS}(\hat{V},V_{gt}),$ $$

where $V_{gt}$ is the ground-truth video corresponding to the target age obtained from dataset building pipeline. Here, we set the lambda values as $\lambda_{L1}=1.0$, $\lambda_{adv,I}=0.025$, $\lambda_{adv,V}=0.025$, $\lambda_{p}=1.0$.

### 3.3 Proposed Metrics

Evaluating age transformation performance in a video context is both essential and challenging. According to the best of our knowledge, there is no specific metric to assess the temporal consistency of re-aging methods. Most existing evaluation metrics are designed for image-based re-aging methods as they measure the age transformation performance on individual images [] or use metrics designed to evaluate the continuity of identity drift []. Regarding this, we propose two metrics, TRWC and T-Age, capable of effectively assessing temporal continuity in terms of age in video-based re-aging task.

#### TRWC

Figure: Figure 3: Conceptual overview of proposed TRWC.
Refer to caption: extracted/5470085/Figures/fig3.jpg

Te evaluate the consistency of video re-aging methods, we need to consider the aging-related details in facial areas such as wrinkles where it matters. This is because these regions are most vulnerable and simultaneously important in terms of human perception. Therefore, we focus on aging-related facial areas, especially Crow’s feet and Nasolabial folds, motivated from [, ]. Due to their active role in expressions and speech, these regions are highly susceptible to aging. We extract the region-of-interest (ROI) around these areas to observe changes over time. However, when analyzing perceptual differences between frames, factors such as facial angles and expressions can be sensitive. To address this, we calculate the LPIPS of the synthesized image and normalize it using the LPIPS of the real image, ensuring we consider differences in each image over time. Fig. 3 explains the conceptual schematic of TRWC. Mathematically, we define TRWC as follows.

$$ $\begin{multlined}\text{TRWC}_{\Delta t}=\frac{1}{3(N-{\Delta t})}\\ \sum_{r=1}^{3}\sum_{t=1}^{N-\Delta t}\frac{lpips(f^{r}_{roi}(\hat{I}_{t}),f^{r }_{roi}(\hat{I}_{t+{\Delta t}}))}{lpips(f^{r}_{roi}(I_{t}),f^{r}_{roi}(I_{t+{ \Delta t}}))},\end{multlined}\text{TRWC}_{\Delta t}=\frac{1}{3(N-{\Delta t})} \\ \sum_{r=1}^{3}\sum_{t=1}^{N-\Delta t}\frac{lpips(f^{r}_{roi}(\hat{I}_{t}),f^{r }_{roi}(\hat{I}_{t+{\Delta t}}))}{lpips(f^{r}_{roi}(I_{t}),f^{r}_{roi}(I_{t+{ \Delta t}}))},$ (8) $$

where $N$ is the number of frames and ${\Delta t}$ is time interval between consecutive frames. The function $f^{r}_{roi}(\cdot)$ is ROI function that includes eyes and mouth obtained by facial landmark detector []. Here, we consider only three ROI regions, left-eye, right-eye, and mouth, indexed by $r$.

#### T-Age

Drawing inspiration from the TL-ID metric [], which proposes metrics for identity consistency in videos, we introduce T-Age for video re-aging. T-Age measures the age difference between two adjacent frames using cosine similarity, utilizing an off-the-shelf age classifier []. A lower T-Age value indicates a more consistent age representation across the frames.

## 4 Experiments

### 4.1 Experiment Setup

In this section, we show the superiority of our method by comparing with all existing state-of-the-art re-aging methods. This includes HRFAE [], SAM [], AgeTransGAN [], Diffusion AE [], STIT [], StyleGANEX [], Diffusion VAE [], and FRAN [], totaling 8 methodologies.

We have trained our methods on the proposed synthetic videos generated through our pipeline. For the test set, we choose the CelebV-HQ [] and VFHQ dataset [] as video test set. We consider three target age groups, $(18,25,35)$ with input age as ${85}$ for $Old\rightarrow Young$ task and three age groups, $(65,75,85)$ with input age as ${18}$ for $Young\rightarrow Old$ task. We provide the additional details in the supplementary materials (Sec. 9).

### 4.2 Metrics

We evaluate the performance of re-aging models based on their ability to transform to the target age while ensuring temporal consistency in the re-aged image. Given that we do not have access to ground-truth for real videos, we employ metrics that do not rely on ground-truth. Specifically, we use four metrics to evaluate the results. We calculate mean absolute error (MAE) between the estimate ages, computed with the pre-trained age classifier to quantify the age transformation quality whereas temporal consistency is measured by TRWC and T-Age. Despite knowing that many existing methods employ DEX that might bias the test results, we opted for DEX due to its accuracy and stability. However, it’s important to note that we did not use any age estimation network in our training process, including DEX.

**Table 1: Quantitative comparison on CelebV & VFHQ datasets. The best results are highlighted in bold.**
| Dataset | Models | Young $\rightarrow$ Old | Old $\rightarrow$ Young |  |  |
| --- | --- | --- | --- | --- | --- |
| $\text{TRWC}_{1}$ | T-Age | $\text{TRWC}_{1}$ | T-Age |  |  |
| CelebV-HQ | AgeTransGAN | 4.25 | 1.26 | 3.12 | 2.29 |
| CUSP | - | - | 1.90 | 2.19 |  |
| FRAN | 1.55 | 1.01 | 0.74 | 0.93 |  |
| OURS | 1.38 | 0.84 | 0.70 | 1.03 |  |
| VFHQ | AgeTransGAN | 4.26 | 1.61 | 2.92 | 2.04 |
| CUSP | - | - | 1.76 | 2.14 |  |
| FRAN | 1.54 | 1.52 | 0.72 | 1.47 |  |
| OURS | 1.35 | 1.16 | 0.69 | 1.34 |  |

**Table 2: User Study Results. Participants evaluated the methods according to four criteria: Age Accuracy (AA), Identity Preservation (IP), Temporal Consistency (TC), and Overall Naturalness (ON). The highest-scoring results are highlighted in bold. Each score is represented as percentage (%).**
| Models | Young $\rightarrow$ Old | Old $\rightarrow$ Young |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| AA | IP | TC | ON | Avg | AA | IP | TC | ON | Avg |  |
| AgeTransGAN | 7.6 | 9.3 | 7.9 | 9.0 | 8.45 | 14.8 | 19.8 | 17.4 | 16.3 | 17.08 |
| CUSP | 15.7 | 36.3 | 14.9 | 21.1 | 22.0 | 8.3 | 8.9 | 7.6 | 8.3 | 8.28 |
| FRAN | 28.1 | 21.2 | 29.1 | 27.3 | 26.42 | 33.9 | 37.8 | 35.7 | 35.4 | 35.7 |
| OURS | 48.7 | 33.2 | 48.1 | 42.6 | 43.15 | 43.0 | 33.5 | 39.3 | 40.0 | 38.95 |

## 5 Comparison Results

### 5.1 Quantitative Results

We quantitatively compare our method with state-of-the-art methods in Table 1. Note that we omit the CUSP result for $Young\rightarrow Old$ due to its maximum target age limitation (65)[]. Our results indicate that model trained on videos show higher TRWC, suggesting that it maintains greater temporal stability and exhibits lower perceptual differences between the consistent frames. On other side, When evaluating age transformation continuity from a broader perspective using metrics like T-AGE, our method performs comparable or better performance. These results suggest that these metrics’ performances are strongly correlated with the target ages. Therefore, we show the MAE for all the target ages in Fig. 4. We can observe that the age transformation performance of our method is consistently better than the state-of-the-art method [], despite being trained on synthetic videos. Additionally, our method also improves overall temporal consistency. It is worth mentioning that all the results we presented are in line with user studies (Table. 2), which further affirm our newly proposed metrics.

Figure: Figure 4: Performance comparison for all target ages. We compare our method with FRAN [] for all target ages.
Refer to caption: extracted/5470085/Figures/fig4.png

### 5.2 User Study

We conducted a subjective evaluation via a user study, presenting 15 videos to a total of 43 anonymous participants for the older direction and 36 participants for the younger direction. This study compares our method with state-of-the-art techniques such as CUSP [], AgeTransGAN [], and FRAN [] on $Young\rightarrow Old$ and $Old\rightarrow Young$ tasks.

As illustrated in Table. 2, a clear majority of participants found our method superior, particularly in terms of age accuracy, temporal consistency, and naturalness. These findings highlight the effectiveness of our video-based approach in enhancing both temporal consistency and age transformation capabilities, thereby demonstrating that proposed metrics are reliable indicators of cognitive temporal consistency. For example, the significance of a 0.18 (±1) difference in TRWC in Table. 1 is evident by the users’ choices.

Figure: Figure 5: Qualitative comparison with existing state-of-the-art methods. The target age is set to 85.
Refer to caption: extracted/5470085/Figures/fig5.jpg

### 5.3 Qualitative Results

We present our qualitative comparison with the existing state-of-the-art in Fig. 5. It is evident that SAM is effective in age transformation, while STIT and StyleGANEX struggle in this aspect. However, SAM fails to preserve attributes such as identity, pose, and expression because it strongly depends on age classifier loss, which is proficient at altering age but not at maintaining other key attributes. On the other hand, STIT and StyleGANEX, which utilize average vectors related to age changes, fail to properly consider the individual samples. As a result, they encounter difficulties in age transformation, particularly when dealing with wild samples. In such cases, their methods become more susceptible due to their strong reliance on trained data for constructing the latent space. Diffusion AE achieves successful age transformation but introduces artifacts such as glasses, leading to identity drift. While the Diffusion VAE works for small changes, it fails entirely in age transformation when dealing with a significant age gap for all subjects.Image-based methodologies, such as HRFAE and AgeTransGAN, inherently exhibit lower age transformation capabilities in wild scenarios. In contrast, while CUSP achieves successful age transformation in certain samples, it often tends to produce artifacts. In contrast, our method successfully performs age transformation while maintaining stable results in terms of identity, expression, and pose. While FRAN also achieves similar quality, our results possess more natural wrinkle details, with the differences being particularly pronounced around the eyes.

Furthermore, Fig. 6 presents a performance comparison of different frames to demonstrate the efficacy of our approach against other video editing methods. It can be observed that, despite producing temporally consistent results, their ability to transform age is quite limited. Overall, these comparison results indicate that leveraging a video dataset enables our method to outperform the existing methods, yielding natural age progression and high-quality results, even under extreme conditions. Similar to every method, our method has some limitations which are discussed in Sec. 12, along with possible future directions. More results can be found in Sec. 10.

Figure: Figure 6: Comparison with video editing methods.
Refer to caption: extracted/5470085/Figures/fig6.jpg

## 6 Ablation Study

In this section, we ablate the selection of the modules of proposed pipeline except the choice of SAM [] which is already validated in [].

#### Key Frame Generation

**Table 3: Ablation studies: We ablate the (a) key frame generation and (b) different frame interpolation methods to select the highest performance method based on higher CPBD and lower warping error.**
| Method<br>MAE ($\downarrow$)<br>ID ($\uparrow$)<br>18<br>85<br>18<br>85<br>StyleHEAT<br>8.06<br>4.41<br>0.64<br>0.50<br>StyleMask<br>7.23<br>9.98<br>0.61<br>0.37<br>OSFV<br>1.62<br>0.71<br>0.90<br>0.90 | Methods<br>DQBC<br>EMA-VFI<br>FILM<br>CPBD $\uparrow$<br>0.5115<br>0.4213<br>0.6061<br>Warp Error $\downarrow$<br>0.00118<br>0.00119<br>0.00104 |
| --- | --- |

In Table 3 (a), we compare the performance of three reenactment methods: OSFV [], StyleMask [], and StyleHeat [], for key frame generation. The best method is selected for our re-aging task based on lower MAE and higher identity similarity (ID). For ID, we calculate the cosine similarity between the vectors of the face image, extracted using ArcFace []. Given different pose and expressions, it is crucial to maintain the age of a person without losing its identity. The results show that OSFV [] successfully preserves both the identity and age of the person, while other methods fail to maintain the person’s identity and age. We further investigate the training configurations of [] in Sec. 11.

#### Motion Generation

In this ablation study, we experiment with state-of-the-art frame interpolation models [, , ] to determine the most effective motion generation method. We evaluate image quality and motion consistency using non-reference-based metrics such as CPBD [, ] for image quality and warping error [] for motion consistency. Table 3 (b) suggests that all methods exhibits lower error in terms of temporal consistency. However, FILM [] outperforms other models by producing sharp results, while the outputs of the remaining models tend to be blurry.

**Table 4: Ablating our method with image (I) and video (V) discriminators on VFHQ dataset.**
| Task | Method | $\text{TRWC}_{1}$ $\downarrow$ | MAE $\downarrow$ |
| --- | --- | --- | --- |
| Old$\rightarrow$Young | I | 0.72 | 10.7 |
| I + V | 0.69 | 10.6 |  |
| Young$\rightarrow$Old | I | 1.53 | 16.9 |
| I+V | 1.35 | 19.7 |  |

#### Video Discriminator

In Table 4, we present ablation studies to assess the effect of the video discriminator. These results indicate that incorporating both image and video discriminators consistently demonstrates superior performance as compared to solely relying on the $I$ discriminator. Whereas, for the Old to Young task, the difference is marginal; this is attributed to the fact that the consistency of wrinkles is often consistent in younger images. Overall, the results suggest that using two discriminators, I + V, improves the consistency of generated videos with negligible effect on MAE.

## 7 Ethical statement

Our proposed video dataset comprises synthetic videos which heavily relies on StyleGAN images, trained on FFHQ dataset []. We acknowledge the potential biases inherited from StyleGAN and FFHQ. Recognizing the societal threat or risk of misuse of our work, we explicitly disapprove of any malicious applications of our research. Our primary intent is to contribute positively for the production and advertisement industry.

## 8 Conclusion

In this paper, we introduced a new video dataset tailored for the supervised learning of video re-aging. This dataset encompasses subjects from a wide range of age groups. Shifting from the conventional model-centric focus, we adopted a data-centric approach. Inspired by recent advancements in face reenactment and frame interpolation, we encompasses various facial poses and expressions in the proposed data. Based on this dataset, we demonstrated the capabilities of the proposed method, which features a baseline network architecture that emphasize both temporal consistency and the quality of age transformations. Furthermore, we formulated two novel metrics to evaluate temporal consistency in video re-aging, which consider age-relevant features such as facial wrinkles over time. We validated our method with comprehensive experiments.

## 9 Experimental Settings

### 9.1 Network Architecture

(Line 204-205: Please refer to Sec. 9 for the details of U-Net network architecture.)

#### Generator

Our network generator consists of a shared recurrent block, which is essentially a U-Net-based network which comprises both up-sampling and down-sampling layers. For every set of 3 input frames, we mask the frames using input and target ages, thereby increasing the number of input channels from 3 to 5. Each of these input frames has a resolution of $512\times 512$. In the first layer of the recurrent block, we concatenate three input frames with a 64-channel hidden state and a 3-channel output image from the previous iteration. This combination results in a spatial image with a total of 82 channels. Such a detailed configuration ensures that the spatial image accurately captures both the current input data and the information processed in the preceding steps. Finally, we obtain a 67-channel spatial image that is subsequently split into a delta image and a hidden state. For detailed architectures of these layers and blocks, please refer to Table 5 for down-sampling, Table 6 for up-sampling, Table 7 for the recurrent block, and Table 8 for generator architecture.

**Table 5: Down-sampling layer.**
| Layer | Output |
| --- | --- |
| Input | $w\times h\times c$ |
| MaxBlurPool | $w/2\times h/2\times c$ |
| $3\times 3$ Conv + LeakyReLU | $w/2\times h/2\times 2c$ |
| $3\times 3$ Conv + LeakyReLU | $w/2\times h/2\times 2c$ |
| Output | $w/2\times h/2\times 2c$ |

**Table 6: Up-sampling layer.**
| Layer | Output |
| --- | --- |
| Input | $w\times h\times c$ |
| BlurUpSample | $2w\times 2h\times c$ |
| $3\times 3$ Conv + LeakyReLU | $2w\times 2h\times c/2$ |
| $3\times 3$ Conv + LeakyReLU | $2w\times 2h\times c/2$ |
| Output | $2w\times 2h\times c/2$ |

**Table 7: Recurrent block.**
| Layer | Output |
| --- | --- |
| Input (Video) | $512\times 512\times 3\times 5$ |
| Reshape | $512\times 512\times 15$ |
| Previous Hidden State | $512\times 512\times 64$ |
| Previous Output | $512\times 512\times 3$ |
| Concatenation | $512\times 512\times 82$ |
| $3\times 3$ Conv + LeakyReLU | $512\times 512\times 64$ |
| $3\times 3$ Conv + LeakyReLU | $512\times 512\times 64$ |
| DownSampleLayer | $256\times 256\times 128$ |
| DownSampleLayer | $128\times 128\times 256$ |
| DownSampleLayer | $64\times 64\times 512$ |
| DownSampleLayer | $32\times 32\times 1024$ |
| UpSampleLayer | $64\times 64\times 512$ |
| UpSampleLayer | $128\times 128\times 256$ |
| UpSampleLayer | $256\times 256\times 128$ |
| UpSampleLayer | $512\times 512\times 64$ |
| $1\times 1$ Conv | $512\times 512\times 67$ |
| Output Delta Image | $512\times 512\times 3$ |
| Output Hidden State + LeakyReLU | $512\times 512\times 64$ |

**Table 8: Generator architecture. $N$ represents the number of frames in the input sequence.**
| Layer | Output |
| --- | --- |
| Input (Video) | $512\times 512\times N\times 3$ |
| Recurrent Blocks ($\times$ N) | $512\times 512\times N\times 67$ |
| Output (Video) | $512\times 512\times N\times 3$ |

#### Discriminator

In our architecture, we use two discriminators. The first discriminator, specifically focused on image quality, is based on PatchGAN []. This consists of three downsampling layers followed by two convolution layers. The output frames are concatenated by their target age masks in a channel-wise manner, resulting in a 4-channel input image. Additionally, we process all output frames independently by concatenating them batch-wise as shown in Table 9.

We also incorporate another discriminator with 3D convolutions, referred to as the video discriminator, to evaluate the realism of motion. Our video discriminator consists of three downsampling layers. The input is created by concatenating the target input age mask and the output images corresponding to three consecutive frames, resulting in a four-channel input. The architecture of the video discriminator is illustrated in Table 10. It is noted that both of our discriminators consist of LeakyReLU in every layer, which is not shown in Table 9 and Table 10.

**Table 9: Architecture of image discriminator.**
| Layer | Output |
| --- | --- |
| Video with Target Mask | $512\times 512\times 4$ |
| $4\times 4$ Conv | $256\times 256\times 64$ |
| $4\times 4$ Conv | $128\times 128\times 128$ |
| $4\times 4$ Conv | $64\times 64\times 256$ |
| $4\times 4$ Conv (Stride = 1) | $64\times 64\times 512$ |
| $4\times 4$ Conv (Stride = 1) | $64\times 64\times 1$ |

**Table 10: Architecture of video discriminator.**
| Layer | Output |
| --- | --- |
| Video with Target Mask | $512\times 512\times 3\times 4$ |
| $4\times 4$ 3D Conv | $256\times 256\times 32\times 4$ |
| $4\times 4$ 3D Conv | $128\times 128\times 64\times 4$ |
| $4\times 4$ 3D Conv | $64\times 64\times 128\times 4$ |
| $4\times 4$ 3D Conv (Stride = 1) | $64\times 64\times 256\times 4$ |
| $4\times 4$ 3D Conv (Stride = 1) | $64\times 64\times 1\times 4$ |

### 9.2 Implementation settings

(Line 283-284: We provide the additional details in the supplementary document)

In this section, we describe our experimental setup for video re-aging training data. We utilized a total of 4,248 subjects to train our network, generating 14 videos per subject. These subjects were divided into 14 classes, covering a wide age range from 18 to 85. Each video consists of 57 frames for training. We applied a cumulative probability of blur detection(CPBD) threshold of 0.5 to ensure sharpness, reducing the prevalence of blurry videos, especially in those with lower CPBD values. Higher CPBD videos, exhibiting fewer dynamic poses, led us to maintain this threshold to balance sharpness and dynamic representation. The sample images of our generated dataset are shown in Fig. 7. For testing, we selected 20 videos from VFHQ [] and 85 videos from CelebV-HQ [] for each Young $\rightarrow$ Old direction (target ages: 65, 75, 85) and Old $\rightarrow$ Young direction (target ages: 18, 25, 35).

During training, we set input and output ages randomly without imposing any conditions that both ages cannot be equal in the same iteration. This approach enables the reconstruction of the input image when the input and target ages are the same. We used a learning rate of 0.0001 for 250K iterations with a batch size of 4. Additionally, we introduced temporal augmentation in our training. We randomly selected a frame interval $\Delta{t}$ from $[3,5,7]$. Reverse augmentation is applied to every frame sequence with a 0.5 probability. We implemented our code in the PyTorch framework and trained our model with a single A100 GPU.

Figure: Figure 7: Data samples in our proposed video dataset with ages 18, 50, and 85.
Refer to caption: extracted/5470085/Figures/fig7.png

#### Experimental Settings for Ablation Study

For the ablation study on reenactment methods in Table 3 (a), we employed 5 videos of 10 subjects, each with 2 target ages (18 and 85), totaling 100 videos. Table 3 (b), which shows the different interpolation methods, utilizes 21 videos from [] for testing. In the experiments exploring various training configurations of OSFV [] (Table 11), we use a relatively large test set comprising 100 videos, as this step is crucial for generating quality data. Lastly, Our ablation study for discriminators in Table 4 follows the same settings as presented in Table 1.

## 10 Additional Qualitative Comparison

In this section, we provide additional comparison results to evaluate our method against state-of-the-art approaches.

Young $\rightarrow$ Old. Fig. 8 shows the comparison results for $Young\rightarrow Old$. The results indicate that SAM [] emphasizes on target age and does not retrain the image fidelity especially for side-poses. This tendency is also observed for HRFAE []. The diffusion based model DIFF-AE [] and CUSP [] often produce artifacts in output images. Recently proposed AgeTransGAN [] generate artistic images that appears to be unnatural. We observe that, compared to FRAN [], our method is robust in handling profile shots and side-view angles.

Old $\rightarrow$ Young. We also present our comparison results for $Old\rightarrow Young$ task in Fig. 9. These results show the similar tendency in which HRFAE [] and SAM [] fail to recover the facial details and suffer from significant artifacts. The results indicate that [] lacks control on target ages and consists various artifacts in their results with variant ages. [] focuses solely on wrinkles removal whereas our method generates smoother images, appearing more natural. In addition, it is quite evident that [] often fail to remove forehead wrinkles and Nasolabial folds.

Comparison to FRAN. In Fig. 10, We conduct a more detailed comparison with FRAN using enlarged images. The rows represent each subject, and the left nine images and right nine images depict Young $\rightarrow$ Old and Old $\rightarrow$ Young transformations, respectively. The results show that our method consistently performs better in age transformation while maintaining natural outcomes across all results. In the Old $\rightarrow$ Young category, it can be observed that the wrinkles around the eyes are unnaturally generated in the results produced by FRAN. Particularly in the transition to a younger appearance, it is noticeable that wrinkles on the forehead are removed more effectively with our method compared to FRAN.

Figure: Figure 8: Qualitative comparison with existing state-of-the-art methods on CelebV-HQ dataset. The target age is set to 85.
Refer to caption: extracted/5470085/Figures/fig8.png

Figure: Figure 9: Qualitative comparison with existing state-of-the-art methods on CelebV-HQ dataset. The target age is set to 18.
Refer to caption: x2.png

Figure: Figure 10: Detailed comparison between FRAN and OURS. Left: $Young\rightarrow Old$, Right: $Old\rightarrow Young$.
Refer to caption: extracted/5470085/Figures/fig10.png

## 11 Ablation of Training Configurations of OSFV

(Line 162: (Footnote) Additional details are provided in the supplementary materials.)
For generation of key frames, we utilize the unofficial implementation(^2^22[https://github.com/zhengkw18/face-vid2vid](https://github.com/zhengkw18/face-vid2vid)) of OSFV [] trained on VoxCeleb [] at 256${}^{2}$ resolution. We denote this configuration as model ‘A’. We evaluated the performance on 100 []’s test set For the training at 512${}^{2}$, we experimented with four different training configurations, as detailed in Table 11. One of the approaches involved a naive upscaling of the images to 512${}^{2}$, resulting in a notably poor CPBD. Afterward, we fine-tuned the existing pretrained model on VFHQ [] at 512${}^{2}$, resulting in improved quality and designated as model ‘B’. We also conducted training from scratch at $256^{2}$ on the VFHQ dataset, following []. While SSIM and PNSR showed improvement compared to model ‘A’, CPBD remained lower. This model is referred to as model ‘C’. Therefore, we fine-tuned model ‘C’ at $512^{2}$ and observed an overall improvement in the dataset’s quality, designated as model ‘D’. For the test set, we randomly selected 100 videos from []’s test set. Our configurations are summarized as follows:

| A: Publicly available model trained at resolution 256${}^{2}$ |
| --- |
| B: Fine-tuning model A on VFHQ at resolution 512${}^{2}$ |
| C: Training [] from scratch on VFHQ at resolution 256${}^{2}$ |
| D: Fine-tuning C on VFHQ at resolution 512${}^{2}$ |

**Table 11: Ablating performance of OSFV [] with different training configurations.**
| Method | PSNR$\uparrow$ | SSIM$\uparrow$ | CPBD$\uparrow$ |
| --- | --- | --- | --- |
| A | 17.553 | 0.657 | 0.149 |
| B | 18.476 | 0.665 | 0.442 |
| C | 19.200 | 0.697 | 0.223 |
| D | 19.519 | 0.683 | 0.487 |

## 12 Limitations and Future Works

(Line 354-355: our method has some limitations …)
(Line 368-369: We further investigate the training configurations …)

Figure: Figure 11: Limitations of our approach.
Refer to caption: extracted/5470085/Figures/fig11.png

Despite of our work surpasses current state-of-the-art methods, we have observed that our method often struggles to preserve the facial hairs. The limitation of our methods are shown in Fig. 11. Our empirical results found that this problem arises due to SAM []. Therefore, leveraging alternative methodologies that are closely aligned to our aspirations may improve the performance. Additionally, we employ simple encoder-decoder within recurrent block architecture. One can explore more advanced network architectures, focusing on refining the transformation of facial shapes. This will likely lead to enhancements in age transformation capabilities and temporal consistency. Furthermore, we have utilized [] for face reenactment and [] for frame interpolation, integrating models such as [] could potentially lead to the creation of even more realistic videos and enhancing our performance. These tasks require further investigation by future researchers.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 3
questions: 1.	Did you conduct quantitative comparisons with methods such as Diffusion VAE, and did you train these methods using your own dataset to evaluate their performance? If so, could you provide the relevant experimental results and analysis?
2.	Can you add new experiments to demonstrate the effectiveness of your newly constructed dataset? For example, by using a currently common technique to conduct experiments on both the existing dataset and the dataset you provided, and using the corresponding metrics to show that your newly constructed dataset can achieve better training results.
3.	Can you clearly articulated the specific benefits of training on videos for the face age reset method, which is essential for understanding the motivation behind choosing video training over static image training? The paper seems to imply the importance of temporal consistency, but it does not explicitly state the advantages of this approach in the context of videos. Could you please elaborate on these benefits to strengthen the motivational aspect of the research and to clarify why this method is innovative and significant compared to traditional static image training methods?
soundness: 3
strengths: 1.	Innovative Data Generation Pipeline: The authors designed a comprehensive pipeline for generating a synthetic dataset specifically for model training in video face re-aging. This pipeline addresses the challenge of obtaining paired video data with consistent identities and varying ages, thereby enhancing the quality and applicability of the training data.

2.	Introduction of New Evaluation Metrics: The development of two novel metrics, Time Region Wrinkle Consistency (TRWC) and Time-Age Preservation (T-Age), provides a more effective means of assessing the quality of age transformations in videos. These metrics focus on maintaining temporal coherence, offering a more nuanced evaluation compared to traditional methods, and contributing to the advancement of the field.
summary: The paper presents a novel approach to video face re-aging, focusing on altering the apparent age of individuals in videos while maintaining temporal consistency. Key contributions include the creation of a synthetic video dataset, a baseline architecture leveraging recurrent blocks for temporal coherence, and the introduction of new metrics for evaluating age transformation quality.
weaknesses: 1.	One significant shortcoming of the paper lies in its experimental section, which lacks thoroughness and depth. Specifically, the evaluation of the proposed new metrics includes only three baselines, and the quantitative comparisons in the User Study Results are limited to just two baselines. While the paper presents qualitative comparisons with various methods, these are not sufficiently persuasive without robust quantitative backing. Furthermore, the authors do not demonstrate the performance of their architecture using their own dataset to evaluate past methods, which undermines claims of superiority for their network design. This lack of comprehensive evaluation limits the credibility of the results and the overall impact of the proposed approach.
2.	Another weakness is the poor discussion of the related works. The authors merely list all current works without providing a comprehensive discussion on them. It is unclear how many methods currently exist in the video-based face re-aging area and why those methods perform poorly.
3.	The motivational section of the paper requires enhancement to better articulate the significance of the proposed method. Currently, the paper does not provide a sufficiently detailed explanation of the practical benefits and implications of the technique. The authors should elaborate on the unique challenges in video face re-aging and how their method specifically addresses these issues, thereby clarifying the motivation behind the research and its importance in the field.
4.	The paper does not explicitly address the rationale for comparing the proposed method with image-based methods. It would be beneficial for the authors to clearly state the reasons behind this comparison. The review suggests that the paper lacks an explanation of why the video approach is being contrasted with image-based techniques, and what specific advantages or insights are to be gained from this comparison. Providing this information would strengthen the paper’s argument and help the reader understand the significance of the methodological choice.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: Please address the concerns in Weaknesses section
soundness: 2
strengths: - The paper addresses temporal consistency factor of video face aging. This is a challenging factor in this topic.
- The paper has introduced both data generation; architecture and metrics for video face aging.
summary: In this paper, the authors address the Temporal consistency issue in Video Face-Aging Approaches.
To tackle this issue, the authors introduce:
(1) A video data generation pipeline to obtain a synthetic video dataset;
(2) A video face aging framework with recurrent U-Net structure; and
(3) Temporal Regional Wrinkle Consistency (TRWC) and Temporally Age Preservation metrics to validate the temporal consistency factor as well as age transformation over time.

Experiments are employed on CelebV-HQ and VFHQ datasets to show the advantages of the proposed approach.
weaknesses: The novelty of the paper is limited as most sections are "inspired" or "motivated" from previous approaches. 
Particularly:
- For data generation process, it relied on StyleGAN and SAM to generate aging results for single frames. Then OSFV technique is adopted to generate faces at different poses and expressions for key frames and motion generation for temporal smoothing.
- For video aging architecture, it is not novel as it is just a recurrent U-Net with commonly used losses.
The structure of the paper is more on putting multiple (previous) approaches together in an engineering manner rather than emphasizing on the novelty.

1. In line 475, "SAM fails to preserve attributes such as identity, pose and expression". However, the data generation relied on images generated by SAM (Eqn. (1)) and learn from that data. Moreover, this error will be further accumulated with OSFV method when poses and expressions presented. 
- What are technical details or modifications in the proposed approach that helps to mitigate the limitation of SAM in the data generation process? 
- Moreover, the authors should provide more discussions/ analysis on how the proposed approach can improve on attribute preservation. Quantitative comparison demonstrating these improvements over SAM is recommended.

2. There is no explicit constraint for consistency between frames during learning process. How can the trained network achieve the consistency when generating age-progressed frames?
Particularly:
- How can temporal consistency be enforced in the proposed approach? The authors should discuss about the details on architecture/loss functions that maintain this factor during learning/inference stages?
- Can we adopt TRWC metric as loss function for this ? 

3. In Eqn. (8), why do we need to validate on the generate image rather than Delta image? In other words, can Delta images be used directly to validate the similarity rather than compute that similarity on \hat{I} and normalize with real image.
The authors should analyze on the choice of using generated images instead of delta images and its effect on the metric values. An ablation study to compare the similarity/difference between these choices is recommended.

---

confidence: 5
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. Synthetic Dataset Details:
    a. Could you please provide more information about the size of your synthetic dataset, specifically the number of videos it contains?
    b. What is the average length of the generated videos in the dataset?
2. Motion Generation:
    a. In Section 3.1.3, you mention generating intermediate frames between keyframes. How many intermediate frames are typically generated?
    b. Is there a specific criterion or stopping condition that determines when to stop generating intermediate frames?
3. Spatial Masks:
    a. What is the purpose of the spatial masks M^inp and M^tar?
    b. Could you provide a visual example or description of these masks?
    c. Are they the same size as the input image I_t, and do they have the same value for all pixels?
4. Limitations in Age Progression: I noticed that the generated videos primarily show age-related changes in the facial area. Why does the proposed approach not generate changes in other regions, such as hair and neck skin? How might this limitation be addressed in future work?
5. Typos: I came across a few typos in the text, such as on line 311 and some symbols in Figure 2. Please ensure a thorough proofread to correct these errors.
soundness: 3
strengths: 1. Establishes a Strong Baseline: It introduces a new baseline for video re-aging, with novel contributions to architecture, dataset creation, and evaluation metrics. This provides a valuable foundation for future research in this area.
2. Demonstrates the Effectiveness of Synthetic Data: The proposed approach, while architecturally simple, effectively leverages synthetic video datasets to achieve compelling results. This highlights the potential of synthetic data for training re-aging models.
3. Provides Comprehensive Evaluation: Through extensive experiments, the authors convincingly demonstrate the realism and temporal coherence of their framework, using both qualitative and quantitative analysis.
summary: The paper presents a simple GAN-based approach to generate a video of a subject at the target age. To maintain temporal consistency, the generator employs a recurrent architecture with U-Net blocks. This structure leverages both previous hidden states and generated frames, ensuring smooth transitions between ages. The model is trained using a combination of image and video discriminators, enhancing realism and temporal coherence.  Furthermore, the authors develop a pipeline for generating synthetic aging datasets and propose two new metrics for evaluating the temporal consistency of video re-aging methods.
weaknesses: 1. Lack of Detail Regarding the Synthetic Dataset: The authors provide insufficient information about their synthetic dataset. To enable a comprehensive evaluation, the authors should provide detailed information about the dataset's size, diversity (including the range of ages, facial features, and other relevant attributes), and visual samples. This would allow reviewers to assess the dataset's quality and its potential impact on the reported results.
2. Missing Information on Motion Generation: Section 3.1.3 on motion generation lacks clarity regarding the stopping condition for generating intermediate frames. A more precise explanation of this process is necessary for readers to fully understand the method.
3. Unclear Availability of Resources: The authors do not explicitly state their intentions regarding the availability of the proposed dataset, pipeline code, or trained models. To enhance reproducibility and facilitate further research, it is strongly recommended that the authors publicly release these resources. Providing a link to a project page or repository, even if it's currently empty, would provide a clear indication of their commitment to open science.
4. Limited Scope of Age Progression: The generated videos primarily exhibit age-related changes in the facial area, neglecting other important regions like hair and neck skin. This inconsistency detracts from the overall realism, as subjects appear to have mismatched facial and other features.
See more detailed questions about the above weaknesses in the next section.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: There are some questions need to be clarified from authors.
1. In line 292, for image and video discriminator loss , how to explain there is no ground truth in total objective function when updating the discriminator loss ? 
2. In Table 1,  three image-based face re-aging methods are compared, is there no comparison with SAM? and how about video-based method, such as diffusion autoencoders (Preechakul et al.) ?
3. In Figure 4 (b), how to explain there is no CUSP results ?
4. In Table 2, how to explain there is no video-based face re-aging method in user study ?
5. In line 468, please give more detailed explanation about the sentence “ the significance of a 0.18 in TRWC in Table. 1 is evident by the user’s choices”
6. Overall, I can’t find more detailed meta information about the proposed video face re-aging dataset, such as how many identities or subjects, total duration of dataset and so on.
soundness: 2
strengths: 10 existing state-of-the-art re-aging methods are compared in order to validate the efficacy of proposed synthesis video dataset and baseline architecture on public datasets, such as VFHQ and CelebV-HQ, as well as necessary ablation experiments. The paper is overall well written.
summary: In this paper, authors focus on video face re-aging task considering the temporal consistency. Most re-aging methods processed each image individually without integrating temporal dimension of videos due to the lack of paired video datasets for supervised training. Thus, an important contribution from authors is a novel synthesis video dataset created via proposed pipeline, it features many subjects with covering a diverse range of age groups.  Then, a baseline video face re-aging architecture is designed to validate the effectiveness of the proposed video dataset. Last but not least, two tailored novel metrics are developed for evaluating the temporal consistency of video face re-aging task.
weaknesses: Although, a new paired video face re-aging dataset is essential for enhancing face re-aging technique and motivating relevant community. Overall, lack of novelty is disadvantage of this manuscript. First, video face re-aging dataset is constructed by a pipeline with three stages. Each of them focuses on off-the-shelf method, such as Style-based Age Manipulation (SAM) is chosen for image-based face re-aging, OSFV is chosen for key frame generation and FILM is chosen for motion generation. It is a general pipeline for constructing video dataset. Second, the proposed baseline architecture of video face re-aging is composed of off-the-shelf building block stacks. Such as recurrent block (RB) and Unet-based Encoder-Decoder. Even the input fashion of the proposed architecture is borrowed from Zoss et al, such as 5 channels with age masks, let alone the discriminator with PatchGAN proposed by Isola et al. Last but not least, the proposed Temporal-Age (T-Age) metric measures the age difference between two adjacent frames utilizing an off-the-shelf age classifier from Rothe at al.  In a short, this manuscript can be considered as a regular technical report, it has a gap to meet the novelty requirement for acceptance.

Meta-review: The authors have not been able to convince three reviewers (duBS, 9tWA, azHF) towards the positive side; all these three reviewers agreed this work needs extra efforts to reach the acceptance bar of the ICLR. Thus I am inclined towards not accepting this draft at this stage. Thank you for your effort. It is an interesting work. I hope input from the reviewers will help you improve this work further.

Decision: Reject