REACT 2025: the Third Multiple Appropriate Facial
Reaction Generation Challenge

Siyang Song, Micol Spitale, Xiangyu Kong, Hengde Zhu, Cheng Luo, Cristina

Palmero, German Barquero, Sergio Escalera, Michel Valstar, Mohamed

Daoudi, et al.

► To cite this version:

Siyang Song, Micol Spitale, Xiangyu Kong, Hengde Zhu, Cheng Luo, et al. REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challenge. MM '25: The 33rd ACM International Conference on Multimedia, Oct 2025, Dublin, Ireland. pp.13979-13984, 10.1145/3746027.3762244 .
hal-05430604

HAL Id: hal-05430604

https://hal.science/hal-05430604v1

Submitted on 23 Dec 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challenge

Siyang Song*

University of Exeter
United Kingdom
s song@exeter.ac.uk

Hengde Zhu

University of Leicester
United Kingdom
hz204@le.ac.uk

German Barquero

Universitat de Barcelona
Barcelona, Spain
germanbarquero@ub.edu

Mohamed Daoudi

IMT Nord Europe
Villeneuve d'Ascq, France
mohamed.daoudi@imt-nord-europe.fr

Andrew Howes

University of Exeter
Exeter, United Kingdom
andrew.howes@exeter.ac.uk

Micol Spitale

Politecnico di Milano
Milan, Italy
micol.spitale@polimi.it

Cheng Luo

King Abdullah University of Science
and Technology
Saudi Arabia
cheng.luo@kaust.edu.sa

Sergio Escalera

Universitat de Barcelona
Barcelona, Spain
sergio@maia.ub.es

Tobias Baur

University of Augsburg
Augsburg, Germany
tobias.baur@uni-a.de

Elisabeth Andrè

University of Augsburg
Augsburg, Germany
andre@uni-a.de

Xiangyu Kong

University of Exeter
United Kingdom
xk219@exeter.ac.uk

Cristina Palmero

King's College London
London, United Kingdom
cristina.palmero@kcl.ac.uk

Michel Valstar

University of Nottingham
Nottingham, United Kingdom
michel.valstar1@nottingham.ac.uk

Fabien Ringeval

Université Grenoble Alpes
Grenoble, France
fabien.ringeval@imag.fr

Hatice Gunes

University of Cambridge
Cambridge, United Kingdom
hatice.gunes@cl.cam.ac.uk

Abstract

In dyadic interactions, a broad spectrum of human facial reactions might be *appropriate* for responding to each human speaker behaviour. Following the successful organisation of the REACT 2023 and REACT 2024 challenges, we are proposing the REACT 2025 challenge encouraging the development and benchmarking of Machine Learning (ML) models that can be used to generate multiple *appropriate, diverse, realistic and synchronised* human-style facial reactions expressed by human listeners in response to an input stimulus (i.e., audio-visual behaviours expressed by their corresponding speakers). As a key of the challenge, we provide challenge participants with the first natural and large-scale multi-modal Multiple Appropriate Facial Reaction Generation (MAFRG) dataset (called MARS) recording 136 human-human dyadic interactions containing a total of 2856 interaction sessions covering

five different topics. In addition, this paper also presents the challenge guidelines and the performance of our baselines on the two proposed sub-challenges: Offline MAFRG and Online MAFRG, respectively. The challenge baseline code is publicly available at https://github.com/reactmultimodalchallenge/baseline_react2025.

CCS Concepts

• Human-centered computing → HCI theory, concepts and models.

Keywords

Human-computer Interaction, Multiple Appropriate Facial Reaction Generation, Generative AI

<figure><img src="image_2.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License.
MM '25, Dublin, Ireland

© 2025 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-2035-2/2025/10
https://doi.org/10.1145/3746027.3762244

ACM Reference Format:

Siyang Song, Micol Spitale, Xiangyu Kong, Hengde Zhu, Cheng Luo, Cristina Palmero, German Barquero, Sergio Escalera, Michel Valstar, Mohamed Daoudi, Tobias Baur, Fabien Ringeval, Andrew Howes, Elisabeth Andrè, and Hatice Gunes. 2025. REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challenge. In *Proceedings of the 33rd ACM International Conference on Multimedia (MM '25), October 27–31, 2025, Dublin, Ireland*. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3746027.3762244

<footnote>*Corresponding Author</footnote>
# 1 Introduction

Recent advances in generative models have boosted numerous automatic human behaviour synthesis systems, including dialogue systems (e.g., Large Language Models (LLMs)) [19, 28], audio/text-driven talking face solutions [4, 41] and human body motion generation [35]. While facial reactions play a key role for conveying human intentions and emotions in human-human interactions, they are characterized by complex and variable nature of interpersonal communications [13]. Due to lack of comprehensive benchmarks and datasets, automatic human facial reaction generation (FRG) is still an underexplored research area, despite the fact that the ability to generate realistic and contextually appropriate human-style facial reactions is essential for developing advanced humanoid virtual agents and robots [30, 37]. Early FRG approaches [7, 10, 16, 17, 27, 31, 42] initially posited that each behaviour (e.g., facial behaviour or audio-visual behaviour) expressed by a speaker (called speaker behaviour) could only trigger human listeners to express a single facial reaction, assuming FRG to be ‘one-to-one mapping’ problem. These efforts aim to accurately replicate the ground truth (GT) listener facial reactions, such as facial reaction sketches [12], photo-realistic expressions [16], facial landmark sequences [10, 31], or full face videos [24], derived from the observed facial or audio-visual behaviours of the speaker.

In real-world human-human interactions, an identical speaker behaviour may elicit varied facial reactions expressed from different listeners or the same listener under different contexts [23], suggesting that FRG is fundamentally a ‘one-to-many mapping’ task [11]. As a result, the recently introduced Multiple Appropriate Facial Reaction Generation (MAFRG) task hypothesizes that each speaker behaviour may trigger human listeners to express different but realistic and appropriate facial reactions (AFRs) depending on external and internal contexts [11]. The organization of the REACT 2023 and REACT 2024 challenges [33, 34] facilitated the creation of several successful solutions [3, 14, 15, 18, 20, 22, 25, 26, 39, 40, 43?] for both online and offline MAFRG tasks. Specifically, they focused on addressing the ill-posed training problem (i.e., **one input speaker behaviour** may correspond to **multiple labels (AFRs)**) by learning either a distribution [14, 18, 22, 39] or a codebook [20, 26] to represent multiple plausible AFRs, from the input speaker behaviour, which reformulate the ‘one-to-many training’ problem as the ‘one-to-one training’ task.

However, both REACT 2023 and REACT 2024 challenges only included modified/segmented audio-visual clips originally collected by the NoXI [6], UDIVA [9] and RECOLA [8] datasets, all of which were recorded for the purposes other than the MAFRG task. Consequently, their data and labels limit the development of more advanced MAFRG systems. Following the conclusion of both challenges, many researchers approached the organisers to request a natural and well-annotated multi-modal dataset specifically recorded for the MAFRG task. In response to this interest, we propose the third MAFRG challenge - REACT 2025, aiming to advance the current state-of-the-art MAFRG solutions using **the first specifically collected MAFRG dataset containing various natural human-human interaction scenarios and objective AFR labels**.

The REACT 2025 challenge follows previous REACT challenges to invite participants to attend offline MAFRG and online MAFRG

<figure><img src="image_3.png" /><figcaption>**Figure 1:** The MARS dataset collection involves two phases: preparation (designing protocols, obtaining ethical approval, scheduling, consent, and personality questionnaires) and data collection (a speaker and listener interact via video conference on predefined tasks, while experimenters monitor from room 3 to provide support).</figcaption></figure>

tasks, both requiring to generate multiple AFRs in response to each given speaker audio-visual behaviour. Each generated AFR clip is represented by a face video and its corresponding multi-channel facial primitive time-series consisting of 25 frame-level facial attributes, i.e., 15 action units (AUs), 8 facial expressions, as well as valence and arousal intensities. It employs the metrics defined in [11] to evaluate four aspects of the submitted models in terms of generated AFRs, namely: appropriateness, diversity, realism and synchrony. Participants are required to submit their developed model, checkpoints and well-explained source code, accompanied by a paper describing their proposed methodologies and the achieved results. Only contributions that meet the pre-determined requirements, terms and conditions <fnref n="1" /> are eligible for participation. The organisers do not engage in active participation themselves, but instead undertake a re-evaluation of the findings of the systems submitted to both sub-challenges. The ranking of the submitted models depend on two metrics: correlation (FRCorr) of the generated facial reaction attributes and facial reaction realism (FRRea) of the generated facial reaction video clips, for both sub-challenges. In summary, the main contributions and novelties of this challenge are listed as follows:

* Introducing and sharing the novel and well-annotated REACT 2025 Multi-modal Challenge Dataset (called Multi-modal Multiple Appropriate Reaction in Social Dyads (MARS) Dataset), together with the recorded multi-modal audio, visual and newly introduced EEG features, as well as objectively- annotated ground-truth (GT) AFRs and self-reported personality traits of every given speaker behaviour.
* Presenting first open-source baseline models on MARS for generating multiple AFRs in response to each multi-modal speaker behaviour under various dyadic interaction scenarios, where the duration of speaker behaviours are not equal. Consequently, the REACT 2025 challenge requires the developed MAFRG models to be able to process variable-length inputs, aiming to simulate real-world human-human interactions where speakers may express the same intention with behaviours of different ways and rates, which is different from previous REACT challenges that only require to predict equal-length AFRs from equal-length speaker behaviours.

<footnote n="1">¹https://sites.google.com/view/react2025/home</footnote>
<table><caption>**Table 1:** Demographics of participants in MARS dataset. *: The highest levels of educational attainment are categorised as B: Below Undergraduate, U: Undergraduate Degree, M: Master's Degree, D: Doctoral Degree.</caption><thead><tr><th rowspan="2">Role</th><th colspan="3">Gender</th><th colspan="4">Age Group</th><th colspan="3">Highest Degree*</th><th colspan="2">Mother Language</th><th rowspan="2">Total</th></tr><tr><th>Male</th><th>Female</th><th>Others</th><th>≤20</th><th>21-30</th><th>31-40</th><th>≥ 41</th><th>B</th><th>U</th><th>M</th><th>D</th><th>English</th><th>Others</th></tr></thead><tbody><tr><td>Speaker</td><td>10</td><td>13</td><td>0</td><td>2</td><td>18</td><td>2</td><td>1</td><td>3</td><td>4</td><td>14</td><td>2</td><td>11</td><td>12</td><td>23</td></tr><tr><td>Listener</td><td>70</td><td>65</td><td>1</td><td>18</td><td>80</td><td>26</td><td>12</td><td>26</td><td>29</td><td>57</td><td>24</td><td>46</td><td>90</td><td>136</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>**Figure 2:** Statistics of participants' ethnic groups.</figcaption></figure>

<table><caption>**Table 2:** Statistics of different interaction topics of the MARS dataset. 's' denotes seconds and 'm' denotes minutes.</caption><thead><tr><th rowspan="2">Topics</th><th colspan="3">Time</th></tr><tr><th>Min</th><th>Max</th><th>Average</th></tr></thead><tbody><tr><td>Cultural differences</td><td>195s</td><td>530s</td><td>339s</td></tr><tr><td>Movie scene sharing</td><td>143s</td><td>573s</td><td>321s</td></tr><tr><td>Policy changes</td><td>153s</td><td>726s</td><td>336s</td></tr><tr><td>Quizzes and games</td><td>191s</td><td>415s</td><td>311s</td></tr><tr><td>Scenario-based interviews</td><td>161s</td><td>518s</td><td>305s</td></tr><tr><td>Overall interactions</td><td>19m'50s</td><td>40m'49s</td><td>27m'16s</td></tr></tbody></table>

## 2 Challenge Tasks

Given a short-term behaviour $b(S_n)^{t_1, t_2}$ expressed by a speaker $S_n$ at the time period $[t_1, t_2]$, the REACT 2025 challenge follows the similar purpose and form as the REACT 2023 [33] and REACT 2024 [34] challenges, focusing on two MAFRG tasks (e.g., offline Multiple Appropriate Facial Reaction Generation (offline MAFRG) and online Multiple Appropriate Facial Reaction Generation (online MAFRG)). The formal definitions for these are provided in [11].

**Task 1: Offline Appropriate Facial Reaction Generation:** The offline MAFRG task aims to develop a machine learning model $\mathcal{H}$ that takes the entire speaker behaviour sequence $b(S_n)^{t_1, t_2}$ as the input, and generates multiple (M) realistic spatio-temporal appropriate facial reactions (AFRs) $p_f(L|b(S_n)^{t_1, t_2})_1, \dots, p_f(L|b(S_n)^{t_1, t_2})_M$, where $p_f(L|b(S_n)^{t_1, t_2})_m$ is a multi-channel time-series consisting of AUs, facial expressions, valence and arousal states representing the $m_{th}$ predicted AFR. As a result, M facial reactions are required to be generated for the task given each input speaker behaviour.

**Task 2: Online Appropriate Facial Reaction Generation:** The online MAFRG task aims to develop a machine learning model $\mathcal{H}$ that estimates each frame (i.e., $\gamma_{th} \in [t_1, t_2]$ frame) of the listener's facial reaction by only considering the $\gamma_{th}$ frame and

its previous frames expressed by the corresponding speaker (i.e., $t_{1th}$ to $\gamma_{th}$ frames in $b(S_n)^t$), rather than taking all $t_{1th}$ to $t_{2th}$ frames into consideration. The model is expected to gradually generate all facial reaction frames to form multiple (M) appropriate and realistic / naturalistic spatio-temporal facial reactions $p_f(L|b(S_n)^{t_1, t_2})_1, \dots, p_f(L|b(S_n)^{t_1, t_2})_M$, where $p_f(L|b(S_n)^{t_1, t_2})_m$ is a multi-channel time-series consisting of AUs, facial expressions, valence and arousal state representing the $m_{th}$ predicted facial reaction. As a result, M facial reactions are required to be generated for the task given each input speaker's behaviour.

## 3 Challenge Corpora

**Dataset.** The Multi-modal Multiple Appropriate Reaction in Social Dyads (MARS) Dataset is the first multi-modal dataset that is specifically collected for MAFRG tasks. It comprises 136 human-human dyadic interaction multi-modal (audio, visual and EEG) clips recorded from 23 human speakers and 136 human listeners (the details of these 154 participants are provided in Table 1 and Fig. 2), where each speaker is randomly and individually matched with 2 to 8 listeners. Specifically, each clip captures a human speaker and a human listener's behaviours (i.e., audio, facial, and EEG) in a pair of files, where the multi-modal recording in each file contains 23 distinct sessions covering five main topics conducted in a fixed order, including **cultural differences (four sessions per clip)**, **movie scene sharing (four sessions per clip)**, **policy changes (four sessions per clip)**, **quizzes and games (eight sessions per clip)**, as well as **scenario-based interviews (three sessions per clip)**. In total, 272 multi-modal recordings (136 clips) whose durations range from 19m50s to 40m49s are obtained (the statistics of our MARS dataset are provided in Table. 2). We split all clips into 2856 multi-modal session pairs (5712 sessions). During the recording of each clip, a pair of speaker and listener are situated in separate rooms to interact with each other on Microsoft Teams through screens. Prior to that, two human experimenters assist both speaker and listener in setting up their cameras, microphones and EEG sensors (MUSE-2), while adjusting the webcam to optimally capture them within the recording frame. Then, the speaker is informed to start the conversation once the recordings of audio, video and EEG signal commence, where each speaker is responsible for initiating and directing the discussions with their corresponding listener, maintaining consistent semantic contexts through the pre-designed conversational topics (illustrated in Fig. 1). Consequently, a set of diverse verbal and non-verbal behaviour reactions expressed by different listeners under a consistent context are considered appropriate in response to each speaker behaviour, which shapes the designed semantic context. Throughout the conversation session, the two experimenters monitor the interaction from a third room
to ensure that the conversation is under the designed interaction control, and to immediately handle the unexpected interruptions (i.e., network interruption). At the end of the conversation, the experimenters go back to the recording rooms to stop the recordings and assist participants in removing their wearable equipments. All clips were collected from students and staff at the University of Leicester, UK, in the period of June - October 2024.

**Provided data:** Our MARS dataset includes the following **raw and pre-processed data**: (1) original videos; (2) cropped face videos that still include expressions and head movements; (3) original audio signals; (5) textual transcripts; (6) EEG clips; and (7) the age, gender, race, and education level of each participant. In addition, we also provide **frame-level facial and audio descriptors** including 15 AUs extracted using [21, 32], eight facial expression probabilities and valence/arousal intensities extracted using EmoNet [36], 58-dimensional 3D Morphable Model (3DMM) coefficients (i.e., 52 facial expression coefficients, 3 pose coefficients and 3 translation coefficients) extracted using Faceverse [38] as well as 768-dimensional audio features extracted using wav2vec 2.0 [2].

**Ground-truth labels.** During data recording, the semantic contexts are carefully controlled through the 23 distinct sessions, each of which is guided by a few pre-defined sentences posted by the speaker. This provides a consistent session-specific context across dyadic interactions between different speakers and listeners. More specifically, for each speaker behaviour expressed in a specific session, we define all facial reactions expressed by different listeners under the same session to be appropriate facial reactions (i.e., ground-truth) for responding to it. As a result, the length of different appropriate facial reactions (i.e., different listeners' facial behaviours expressed in a specific session) for responding to a speaker behaviour can be varied. In summary, our MARS dataset includes the following labels: (1) ground truth for the appropriate facial reactions (GT real AFRs) for each speaker behaviour clip; and (2) self-reported personality traits based on the 10 question Big-Five inventory [29].

**Ethical consideration.** Ethical approval for our MARS dataset was obtained at the University of Leicester. Prior to data acquisition, participants are required to read and sign the consent form. Subsequently, they complete the personality questionnaire and are informed about the recording equipments and their use. Finally, all participants are compensated with Amazon vouchers for their participation in the study. During the challenge, all participating teams are required to sign an EULA to access the dataset.

# 4 Evaluation Metrics

In this challenge, the submitted models are expected to generate two types of outputs for representing each facial reaction: (i) 25 facial attribute time-series; and (ii) a 2D facial image sequence. We followed [11, 33] to comprehensively evaluate three aspects of the generated facial reaction attributes: (i) **Appropriateness** based on two metrics, **FRCorr**: Concordance Correlation Coefficient (CCC) and **FRDist**: Dynamic Time Warping (DTW); (ii) **Diversity**: **FRVar**, and **FRDiv**; and (iii) **Synchrony**: the Time Lagged Cross Correlation (TLCC), called **FRSyn** in this challenge. Also, the **Realism** of the generated facial reaction video clips is assessed using the Fréchet Inception Distance (FID), denoted as **FRRea**.

<figure><img src="image_5.png" /><figcaption>**Figure 3:** Overview of the Trans-VAE baseline.</figcaption></figure>

As detailed in Sec. 3, the lengths of real AFRs expressed by different listeners for responding to a speaker behaviour can be varied. In this sense, we trained a Transformer-based variational encoder-decoder framework to summarise every arbitrary-length real AFRs into a fixed-length sequence of tokens, allowing to represent all variable-length real facial reactions with fixed-length token sequences in a latent space. This way, each generated AFR can be directly compared with all corresponding real AFRs of varied lengths in this latent space for computing **FRDist** and **FRCorr** metrics. Then, the transformer decoder can decode the token sequence as facial reaction image sequences of any required length.

# 5 Baseline Models

To enable fair and easy comparison, we first re-employ an open-source Trans-VAE designed for both online and offline MAFRG tasks in previous challenges [33, 34]. We also provide the generic MAFRG model of the state-of-the-art diffusion-based MAFRG approach (i.e., PerFDiff [43]) as both online and offline MAFRG baselines. In addition, we provide the REGNN [39] as the offline baseline.

**Trans-VAE [22, 33, 34]:** The Trans-VAE baseline follows a similar architecture to TEACH [1]. As shown in Fig. 3, it is made up of (i) a **CNN encoder** extracting facial reaction-related features from the input speaker facial image sequence; (ii) a **Transformer encoder** combining the learned facial and baseline audio features (768-dimensional features) extracted from the speaker audio behaviours using wav2vec 2.0 [2], based on which a Gaussian Distribution is learned to describe multiple predicted AFRs for responding to the input speaker behaviour; and (iii) a **Transformer decoder** that samples two types of facial reaction representations from the learned distribution: 1) a set of 3DMM coefficients [38]; and 2) a multi-channel facial attribute time-series (Refer to [33] for details).

**PerFDiff [43]:** We re-train the generic MAFRG model (i.e., a transformer-based conditional diffusion model) to separately conduct online task and offline MAFRG task. As demonstrated in Fig. 4, it employs three modality-specific linear projectors to individually map the input multi-modal speaker behaviours including audio features extracted from Wav2vec 2.0 [2], raw multi-channel facial attribute time-series and raw 3DMM coefficients sequence, to a shared latent space, serving as conditional inputs to the diffusion Transformer. Then, the model follows the standard denoising diffusion process to gradually denoise clean AFRs, where cross-attention operations are used to process the conditions (i.e., speaker behaviour features). To accommodate variable-length speaker behavior sequences across batches, we apply standard padding masks
<figure><img src="image_6.png" /><figcaption>Figure 4: Overview of the PerFRDiff baseline.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 5: Overview of the REGNN baseline.</figcaption></figure>

in the attention layers to prevent information leakage from padded positions, following common practices in sequential modeling.

**Reversible Graph Neural Network (REGNN) [39]:** We additionally employ the REGNN [39] as the offline MAFRG baseline. As illustrated in Fig. 5, it consists of three main modules: (i) a **Perceptual Processor** that encodes the input speaker audio-facial behaviour as a pair of latent audio and facial representations; (ii) a **Cognitive Processor (Cog)** that predicts a Gaussian Mixture Graph Distribution describing all AFRs in response to the input speaker behaviour; and (iii) a Reversible GNN-based **Motor Processor** that samples an appropriate facial reaction from the predicted appropriate facial reaction distribution. During training, the Reversible GNN encodes all real AFRs of each input speaker behaviour as a ground-truth real AFR distribution, enforcing the perceptual and cognitive processors to predict the same distribution from the input speaker behaviour. As a result, the *one-to-many mapping* training problem is re-formulated as a *one-to-one mapping* problem.

# 6 Baseline Results

As shown in Table 3, all baselines largely outperformed the B_Random and B_MeanFr, showing that they can to some extent generate meaningful AFRs from different speaker behaviours. Here, the B_Mime setting achieved surprising FRCorr performance by simply generating the corresponding speaker facial behaviours as human

<table><caption>Table 3: Baseline results achieved on the test set of MARS dataset, where B_Random randomly samples $\alpha = 10$ facial reaction sequences from a Gaussian distribution; B_Mime generates facial reactions by mimicking the corresponding speaker facial behaviours; B_MeanSeq and B_MeanFr generate facial reactions by average the sequence- and frame-wise facial reactions in the training set, respectively.</caption><thead><tr><th rowspan="2">Method</th><th colspan="2">Appropriateness</th><th colspan="2">Diversity</th><th colspan="2">Realism</th><th colspan="2">Synchrony</th></tr><tr><th>FRCorr (↑)</th><th>FRDist (↓)</th><th>FRDiv (↑)</th><th>FRVar (↑)</th><th>FRRea (↓)</th><th>FRSyn (↓)</th><th></th><th></th></tr></thead><tbody><tr><td>GT</td><td>10</td><td>0.00</td><td>0.1876</td><td>0.0669</td><td>-</td><td>48.66</td><td></td><td></td></tr><tr><td>B_Random</td><td>0.03</td><td>474.68</td><td>0.3342</td><td>0.1671</td><td>-</td><td>46.64</td><td></td><td></td></tr><tr><td>B_Mime</td><td>0.52</td><td>206.02</td><td>0.0000</td><td>0.0766</td><td>-</td><td>43.70</td><td></td><td></td></tr><tr><td>B_MeanFr</td><td>0.00</td><td>205.65</td><td>0.0000</td><td>0.0000</td><td>-</td><td>49.00</td><td></td><td></td></tr><tr><td colspan="9"><strong>Offline Results</strong></td></tr><tr><td>Trans-VAE</td><td>0.24</td><td>158.97</td><td>0.0079</td><td>0.0067</td><td>-</td><td>49.00</td><td></td><td></td></tr><tr><td>PerFRDiff</td><td>0.56</td><td>177.76</td><td>0.1386</td><td>0.0706</td><td>-</td><td>48.54</td><td></td><td></td></tr><tr><td>REGNN</td><td>0.46</td><td>141.93</td><td>0.0010</td><td>0.0036</td><td>-</td><td>45.52</td><td></td><td></td></tr><tr><td colspan="9"><strong>Online Results</strong></td></tr><tr><td>Trans-VAE</td><td>0.27</td><td>178.31</td><td>0.0879</td><td>0.0475</td><td>-</td><td>49.00</td><td></td><td></td></tr><tr><td>PerFRDiff</td><td>0.51</td><td>214.20</td><td>0.1598</td><td>0.0888</td><td>-</td><td>47.32</td><td></td><td></td></tr></tbody></table>

listeners often exhibit facial behaviours that mirror those of the corresponding speakers, i.e., facial mimicry [5].

**Offline MAFRG task:** The generic MAFRG model of PerFRDiff and the REGNN outperformed the Trans-VAE. Specifically, the PerFRDiff achieved the highest FRCorr (0.58) with best diversity (more than 10 times for both metrics), indicating that it can generate multiple diverse facial reactions that show relatively similar trend as GT real AFRs. Meanwhile, the REGNN achieved similar FRCorr (0.54) but the lowest FRDist (155.49) and FRSyn, suggesting that the AFRs generated by it not only have similar trend but also closely aligned with the GT real AFRs.

**Online MAFRG task:** Similarly, the PerFRDiff also achieved clearly better performances over the Trans-VAE baseline in terms of all metrics except the slightly higher FRDist (222.69 vs. 212.22). Importantly, the PerFRDiff still achieved much higher FRCorr (0.55 vs. 0.25) with better facial reaction diversities and the synchrony between the input speaker behaviours and the generated facial reactions, validating its capability in simulating multiple appropriate human-style facial temporal patterns for responding conversational partners, while maintaining their diversity.

# 7 Participation and Conclusion

This paper introduces REACT 2025 Challenge organised in conjunction with the ACM Multimedia 25 Conference, which focuses on MAFRG tasks under various video conference-based dyadic interactions scenarios. A total of 23 teams registered for the challenge, with 21 and 20 teams participated in the offline and online sub-challenges, respectively, where 8 teams have successfully submitted valid codes/models. Challenge participants were given access to the training and validation sets to develop their ML models, with a challenge guideline and a baseline paper released to provide more details. Each participating group was allowed to submit their models and results for the test set, which were automatically evaluated. Our evaluation protocol strictly ranked all participant models under the same settings by evaluating two aspects of their generated
facial reactions: FRCorr and FRRea. We hope that the new MAFRG dataset, baseline code, methodologies and results of the participating teams, will serve as a valuable stepping stone for researchers and practitioners interested in automatic facial reaction generation.

## Acknowledgements

M. Spitale is supported by PNRR-PE-AI FAIR project funded by the NextGeneration EU program. H. Gunes is supported by the EPSRC/UKRI under grant ref. EP/R030782/1 (ARoEQ). M. DAOUDI is partially supported by a French government grant under the France 2030 program - ANR-22-EXEN-0004 (PEPR eNSEMBLE / PC3).

## References

[1] Nikos Athanasiasiou, Mathis Petrovich, Michael J Black, and Gül Varol. 2022. Teach: Temporal action composition for 3d humans. In *2022 International Conference on 3D Vision (3DV)*. IEEE.

[2] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. *Advances in neural information processing systems* 33 (2020), 12449–12460.

[3] Quang Tien Dam, Tri Tung Nguyen Nguyen, Dinh Tuan Tran, and Joo-Ho Lee. 2024. Finite Scalar Quantization as Facial Tokenizer for Dyadic Reaction Generation. In *2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG)*. IEEE, 1–5.

[4] Kaijun Deng, Dezhi Zheng, Jindong Xie, Jinbao Wang, Weicheng Xie, Linlin Shen, and Siyang Song. 2025. DEGTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis. In *ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE, 1–5.

[5] Ulf Dimberg. 1982. Facial reactions to facial expressions. *Psychophysiology* 19, 6 (1982), 643–647.

[6] Cafaro et al. 2017. The NoXi database: multimodal recordings of mediated novice-expert interactions. In *ICMI 2017*.

[7] Evonne Ng et al. 2022. Learning to listen: Modeling non-deterministic dyadic facial motion. In *IEEE/CVF CVPR 2022*. 20395–20405.

[8] Fabien Ringeval et al. 2013. Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. In *IEEE FG 2013*.

[9] Palmero et al. 2021. Context-aware personality inference in dyadic scenarios: Introducing the udiva dataset. In *IEEE/CVF WACV 2021*.

[10] Song et al. 2022. Learning Person-specific Cognition from Facial Reactions for Automatic Personality Recognition. *IEEE Transactions on Affective Computing* (2022).

[11] Song et al. 2023. Multiple Appropriate Facial Reaction Generation in Dyadic Interaction Settings: What, Why and How? https://arxiv.org/abs/2302.06514 (2023).

[12] Yuchi et al. 2017. Dyadgan: Generating facial expressions in dyadic interactions. In *IEEE CVPR Workshops 2017*.

[13] Ursula Hess, Pierre Phillipott, and Sylvie Blairy. 1998. Facial reactions to emotional facial expressions: Affect or cognition? *Cognition & Emotion* 12, 4 (1998), 509–531.

[14] Ximi Hoque, Adamay Mann, Gulshan Sharma, and Abhinav Dhall. 2023. BEAMER: Behavioral Encoder to Generate Multiple Appropriate Facial Reactions. In *Proceedings of the ACM International Conference on Multimedia*. 9536–9540.

[15] Guanyu Hu, Jie Wei, Siyang Song, Dimitrios Kollias, Xinyu Yang, Zhonglin Sun, and Odysseus Kaloidas. 2024. Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation. In *2024 IEEE International Joint Conference on Biometrics (IJCB)*. IEEE, 1–10.

[16] Yuchi Huang and Saad M Khan. 2018. Generating Photorealistic Facial Expressions in Dyadic Interactions. In *BMVC* 2011.

[17] Yuchi Huang and Saad M Khan. 2018. A generative approach for dynamically varying photorealistic facial expressions in human-agent interactions. In *Proceedings of the 20th ACM International Conference on Multimodal Interaction*. 437–445.

[18] Cong Liang, Jiahe Wang, Haofan Zhang, Bing Tang, Junshan Huang, Shangfei Wang, and Xiaoping Chen. 2023. UniFaRN: Unified Transformer for Facial Reaction Generation. In *Proceedings of the ACM International Conference on Multimedia*. 9506–9510.

[19] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. *arXiv preprint arXiv:2412.19437* (2024).

[20] Zhenjie Liu, Cong Liang, Jiahe Wang, Haofan Zhang, Yadong Liu, Caichao Zhang, Jialin Gui, and Shangfei Wang. 2024. One-to-Many Appropriate Reaction Mapping Modeling with Discrete Latent Variable. In *2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG)*. IEEE, 1–5.

[21] Cheng Luo and et al. 2022. Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition. *IJCAI* (2022).

[22] Cheng Luo, Siyang Song, Weicheng Xie, Micol Spitale, Zongyuan Ge, Linlin Shen, and Hatice Gunes. 2024. ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactions. *IEEE Transactions on Visualization and Computer Graphics* (2024).

[23] Albert Mehrabian and James A Russell. 1974. *An approach to environmental psychology*. the MIT Press.

[24] Evonne Ng, Hanbyul Joo, Liwen Hu, Hao Li, Trevor Darrell, Angjoo Kanazawa, and Shiry Ginosar. 2022. Learning To Listen: Modeling Non-Deterministic Dyadic Facial Motion. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*. 20395–20405.

[25] Dang-Khanh Nguyen, Prabesh Paudel, Seung-Won Kim, Ji-Eun Shin, Soo-Hyung Kim, and Hyung-Jeong Yang. 2024. Multiple Facial Reaction Generation Using Gaussian Mixture of Models and Multimodal Bottleneck Transformer. In *2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG)*. IEEE, 1–5.

[26] Minh-Duc Nguyen, Hyung-Jeong Yang, Ngoc-Huynh Ho, Soo-Hyung Kim, Seung-won Kim, and Ji-Eun Shin. 2024. Vector Quantized Diffusion Models for Multiple Appropriate Reactions Generation. In *2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG)*. IEEE, 1–5.

[27] Behnaz Nojavanasghari, Yuchi Huang, and Saad Khan. 2018. Interactive generative adversarial networks for facial expression generation in dyadic interactions. *arXiv preprint arXiv:1801.09092* (2018).

[28] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. (2018).

[29] Beatrice Rammstedt and Oliver P John. 2007. Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. *Journal of research in Personality* 41, 1 (2007), 203–212.

[30] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. 2019. Habitat: A platform for embodied ai research. In *Proceedings of the IEEE/CVF international conference on computer vision*. 9339–9347.

[31] Zilong Shao, Siyang Song, Shashank Jaiswal, Linlin Shen, Michel Valstar, and Hatice Gunes. 2021. Personality recognition by modelling person-specific cognitive processes using graph representation. In *Proceedings of the 29th ACM international conference on multimedia*.

[32] Siyang Song, Yuxin Song, Cheng Luo, Zhiyuan Song, Selim Kuzucu, Xi Jia, Zhi-jiang Guo, Weicheng Xie, Linlin Shen, and Hatice Gunes. 2022. Gratis: Deep learning graph representation with task-specific topology and multi-dimensional edge features. *arXiv preprint arXiv:2211.12482* (2022).

[33] Siyang Song, Micol Spitale, Cheng Luo, Germán Barquero, Cristina Palmero, Sergio Escalera, Michel Valstar, Tobias Baur, Fabien Ringeval, Elisabeth André, et al. 2023. React2023: The first multiple appropriate facial reaction generation challenge. In *Proceedings of the 31st ACM International Conference on Multimedia*. 9620–9624.

[34] Siyang Song, Micol Spitale, Cheng Luo, Cristina Palmero, Germán Barquero, Hengde Zhu, Sergio Escalera, Michel Valstar, Tobias Baur, Fabien Ringeval, et al. 2024. React 2024: the second multiple appropriate facial reaction generation challenge. *arXiv preprint arXiv:2401.05166* (2024).

[35] Mingze Sun, Chao Xu, Xinyu Jiang, Yang Liu, Baigui Sun, and Ruqi Huang. 2025. Beyond talking—generating holistic 3d human dyadic motion for communication. *International Journal of Computer Vision* 133, 5 (2025), 2910–2926.

[36] Antoine Toisoul and et al. 2021. Estimation of continuous valence and arousal levels from faces in naturalistic conditions. *Nature Machine Intelligence* 3, 1 (2021), 42–50.

[37] Isaac Wang and Jaime Ruiz. 2021. Examining the use of nonverbal communication in virtual agents. *International Journal of Human-Computer Interaction* 37, 17 (2021), 1648–1673.

[38] Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang Ma, Liang Li, and Yebin Liu. 2022. Faceverse: a fine-grained and detail-controllable 3d face morphable model from a hybrid dataset. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*. 20333–20342.

[39] Tong Xu, Micol Spitale, Hao Tang, Lu Liu, Hatice Gunes, and Siyang Song. 2023. Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation. *arXiv preprint arXiv:2305.15270* (2023).

[40] Jun Yu, Ji Zhao, Guochen Xie, Fengxin Chen, Ye Yu, Liang Peng, Minglei Li, and Zonghong Dai. 2023. Leveraging the Latent Diffusion Models for Offline Facial Multiple Appropriate Reactions Generation. In *Proceedings of the ACM International Conference on Multimedia*. 9561–9565.

[41] Chenxu Zhang, Yifan Zhao, Yifei Huang, Ming Zeng, Saifeng Ni, Madhukar Budagavi, and Xiaohu Guo. 2021. Facial: Synthesizing dynamic talking face with implicit attribute learning. In *Proceedings of the IEEE/CVF international conference on computer vision*. 3867–3876.

[42] Mohan Zhou, Yalong Bai, Wei Zhang, Ting Yao, Tiejun Zhao, and Tao Mei. 2022. Responsive listening head generation: a benchmark dataset and baseline. In *European conference on computer vision*. Springer, 124–142.

[43] Hengde Zhu, Xiangyu Kong, Weicheng Xie, Xin Huang, Linlin Shen, Lu Liu, Hatice Gunes, and Siyang Song. 2024. Perfrdiff: Personalised weight editing for multiple appropriate facial reaction generation. In *Proceedings of the 32nd ACM International Conference on Multimedia*. 9495–9504.