# The Impact of Pedagogical Agents' Gender on Academic Learning: A Systematic Review

Marjorie Armando, Magalie Ochs, Isabelle Régner

► To cite this version:

Marjorie Armando, Magalie Ochs, Isabelle Régner. The Impact of Pedagogical Agents' Gender on Academic Learning: A Systematic Review. Frontiers in Artificial Intelligence, 2022, 5, 10.3389/frai.2022.862997. [https://hal.science/hal-03700888v2](https://hal.science/hal-03700888v2)

HAL Id: hal-03700888

https://hal.science/hal-03700888v2

Submitted on 15 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# The impact of pedagogical agents' gender on academic learning: a systematic review.

Marjorie Armando<sup>1,2</sup>, Magalie Ochs<sup>1,*</sup> and Isabelle Régner<sup>2,*</sup>

Correspondence*:

Corresponding Author

## 2 ABSTRACT

Virtual learning environments often use virtual characters to facilitate and improve the learning process. These characters, known as pedagogical agents, can take on different roles, such as tutors or companions. Research has highlighted the importance of various characteristics of virtual agents, including their voice or non-verbal behaviors. Little attention has been paid to the gender-specific design of pedagogical agents, although gender has an important influence on the educational process. In this article, we perform an extensive review of the literature regarding the impact of the gender of pedagogical agents on academic outcomes. Based on a detailed review of 59 articles, we analyze the influence of pedagogical agents' gender on students' academic self-evaluations and achievements to answer the following questions: (1) Do students perceive virtual agents differently depending on their own gender and the gender of the agent? (2) Does the gender of pedagogical agents influence students' academic performance and self-evaluations? (3) Are there tasks or academic situations which a male virtual agent is better suited to than a female virtual agent, and vice versa, according to empirical evidence? (4) How do a virtual agent's pedagogical roles impact these results? (5) How do a virtual agent's appearance and interactive capacities impact these results? (6) Are androgynous virtual agents a potential solution to combatting gender stereotypes? This review provides important insight to researchers on how to approach gender when designing pedagogical agents in virtual learning environments.

**Keywords:** Virtual Agent, Gender, Pedagogical Agent, Learning Environment, Gender Stereotypes, Systematic Review

## 1 INTRODUCTION

Pedagogical agents are virtual characters in digital environments used to improve learning in educational settings (Mohtadi et al., 2014; Schroeder et al., 2017). They can take on different roles, such as expert, mentor, or motivator (Baylor and Kim, 2005). As shown in a meta-analytic review of 43 studies by Schroeder et al. (2013), pedagogical agents can have a positive effect on students' free recall ability, knowledge retention, and transfer of prior knowledge to new situations or problems. However, some characteristics of pedagogical agents may impact the learning process: for instance, how realistic the virtual agents' appearance is (Baylor and Kim, 2004), the way they communicate with learners, verbally or nonverbally, positively or negatively (Gratch et al., 2007; Pecune et al., 2016), or the way they deliver feedback, using voice, text, or both (Kim and Baylor, 2016).

Virtual agents' gender is another feature that users can perceive from the agents' appearance (Lee, 2003).

Yet few studies have evaluated the impact of pedagogical agents' gender, which is surprising considering

<footnote n="1"><sup>1</sup>Aix Marseille Univ, Université de Toulon, CNRS, LIS, Marseille, France</footnote>

<footnote n="2"><sup>2</sup>Aix Marseille Univ, CNRS, LPC, Marseille, France</footnote>
the amount of research in Social Cognition documenting the impact of the gender of both learners and teachers on academic learning. Social Cognition and human-to-human studies are particularly interesting in the domain of virtual agents, as research shows that individuals have a propensity to interact with virtual agents as if they were human (Nass and Moon, 2000). Research in Social Cognition and Cognitive Psychology can therefore be enlightening for understanding users' perception of virtual characters and the effect of these perceptions on their performance. This is why we present some major Social Cognition research on the impact of learners and teachers' gender on learners' academic outcomes. For instance, Sansone (2019) conducted a survey on the link between high school students' beliefs about women's abilities in math and science and their teacher's gender, finding that students were less likely to report that men are better than women in math/science when assigned to female teachers. Teachers' behavior can also impact girls' and boys' learning differently: a large scale survey conducted by Forgasz and Leder (1996) showed that students who perceived their math teachers to be interested in them as individuals were more likely to have functional beliefs about themselves in mathematics, and this was more critical for female learners than male learners. Core beliefs represent general and strongly held views about ourselves, others, and the world; they influence the way we react in different circumstances. Functional beliefs are rational thought patterns which are generally useful for individuals to achieve their goals (Ellis, 1962). In the forementioned study, math teachers' behaviors seemed to favor boys over girls: boys had more interactions with their teachers, teachers were more tolerant of boys' misbehavior, and they had higher expectations of boys (Forgasz and Leder, 1996). A meta-analysis conducted by Lindberg et al. (2010) from 242 studies published between 1990 and 2007 indicated that while male and female learners performed similarly in mathematics, female students reported higher anxiety, more discomfort, and lower interest and self-efficacy in math classes than male students. Parents themselves tend to attribute different explanations for their children's academic performance depending on their gender: they explain their sons' mathematical success as due to their natural talent, whereas they explain their daughters' as due to their effort (Yee and Eccles, 1988). These results were replicated by Räty et al. (2002) who also found that parents of boys evaluated their child's mathematical competence as higher than parents of girls, and parents of girls perceived them as surpassing boys in reading. Despite this, parents still attributed competence in reading as resulting from effort for girls but from natural talent for boys. By explaining their daughters' success in math as due to effort, the authors suggested that parents may undermine both their own and their daughters' estimation of their daughters' success in mathematics, hence raising possible doubts about their future success in a domain that they think gets increasingly complicated; meanwhile, they may encourage boys to develop greater confidence in their future success (Yee and Eccles, 1988).

All these differences reflect the influence of gender stereotypes that lead people to consider men to be better at math than women, and women to be better in liberal arts -such as literature, for example- than men. In addition, studies have shown that the fear of being negatively stereotyped in a skill area produces negative thoughts, which in turn reduce individuals' working memory capacity and impair learning and performance (Schmader and Johns, 2003). This phenomenon, called *Stereotype Threat* (Steele and Aronson, 1995), applies to different stereotypes and social groups, such as boys in reading tests (Pansu et al., 2016) and girls and women in math tests (Régner et al., 2014). The effects of Stereotype Threat can be reduced using different strategies, such as reading a story about a successful role model before taking a test (Bagès and Martinot, 2011; Bagès et al., 2016).

Presenting pedagogical agents as role models could be a potential solution for reducing the effects of Stereotype Threat. Researchers designing agents should take into account the gender of both learners and pedagogical agents to adapt the agent to the learners. The advantages of adapting virtual agents to participants have been demonstrated in several studies. For instance, in Vilaro et al. (2021), participants (all
Black women) liked Black female agents for being artificial, hence creating a sense of trust and freedom where participants could avoid inherent biases and racism. In virtual learning environments, research has shown the impact of virtual agents' gender on human-agent interactions (see Section 3.4). However, the gender of *pedagogical* virtual agents is rarely considered as an important characteristic in the design of virtual learning environments, whereas most pedagogical agents are human-like, and their gender can have an impact on academic outcomes (see Section 3.5). In terms of perception, various studies have shown that male virtual agents are rated as more powerful (Nunamaker et al., 2011), more expert (Nunamaker et al., 2011), and more knowledgeable (Baylor and Kim, 2004), whereas female agents are rated as more likable (Nunamaker et al., 2011) and more attractive (Lunardo et al., 2016). These attributes are important in learning environments, as competent and expert agents improve learners' performance (Baylor and Kim, 2004), and likable and attractive agents improve learners' self-perception including their self-efficacy (feeling of achievement) (Rosenberg-Kima et al., 2008), which may help improve their performance (Plant et al., 2009).

In this article, we present an extensive state of the art focusing on the *effects of pedagogical agents' gender in virtual learning environments*. We explore the impact of gender on the users' perceptions of agents and on their learning.

This article is organized as follows. In the next section, we explain our methodology used to conduct the state of the art and particularly how we used the PRISMA method to select relevant articles (Webster and Watson, 2002). In Section 3, the selected articles are summarized in Table 1 and 2 to provide a comprehensive review of research on the impact of pedagogical agents' gender on learners' performance and self-perception in academic domains. We discuss the articles summarized in the tables in Section 3.4 and Section 3.5. In Section 3.4, we address research highlighting the impact of virtual agents' gender on users' perceptions. In Section 3.5, we focus on pedagogical agents and the impact of their gender on learners' academic outcomes. The last section discusses what could be done in future research on virtual learning environments to reduce gender stereotypes and to improve learners' performance, and the important research questions that arise from this review.

# 2 METHODS

## 2.1 Search strategy

This article examines research on the impact of virtual agents' gender on learners but also more generally on users' behavior and perceptions. For this purpose, we reviewed articles from the Web of Science database over a twenty-one-year period from 2000 to 2021. To collect the relevant studies, we conducted an online database search with the query gender+("virtual agent*" OR "virtual character*" ). This systematic review was conducted according to the PRISMA guidelines presented in Figure 1 (Webster and Watson, 2002) as follows: (1) scanning databases and starting with the major contributions in the leading journals, (2) reviewing the citations for the articles identified in step 1 to determine prior articles that should be considered, and (3) identifying articles citing the key articles identified in the previous steps. We used Google Scholar for the last step. A total of 120 articles were retained after following these steps.

## 2.2 Selection of articles

From this set of articles, we selected empirical studies analyzing the effect of virtual agents' gender on users' perceptions, behaviors, and academic outcomes. We only took into account embodied virtual agents (i.e. we excluded studies on vocal assistants). We focused on Western culture and thus only selected
<figure><img src="image_2.png" /><figcaption>**Figure 1.** PRISMA 2020 flow diagram for new systematic reviews.</figcaption></figure>

117 papers relating to this culture. We eliminated articles only about avatars (users embodying a virtual agent) which were mainly about video games. In the end we retained a set of 59 articles. We distinguished two types of research articles: Perceptive studies of virtual agents depending on their gender, regardless of the application domain, and research studies on the impact of gendered virtual agents in the context of a learning task.

122 This systematic review focuses on how virtual agents are designed, and the impact of their gender on different academic outcomes (motivation, learning, interest), but also on participants' perceptions of the agents. The research questions guiding this review are as follows:

125 1. Do students perceive virtual agents differently depending on their own gender and the gender of the agent?
126
127 2. Does the gender of pedagogical agents influence students' academic performance and self-evaluations?

128 3. Are there tasks or academic situations which a male virtual agent is better suited to than a female virtual agent, and vice versa, according to empirical evidence?

130 4. How do a virtual agent's pedagogical roles impact these results?

131 5. How do a virtual agent's appearance and interactive capacities impact these results?

132 6. Are androgynous virtual agents a potential solution to combatting gender stereotypes?

## 3 SYSTEMATIC REVIEW OF VIRTUAL AGENTS' GENDER AND ITS IMPACT ON USERS' PERCEPTIONS AND ACADEMIC OUTCOMES

133 In this section, we first review the different measures used to assess users' perceptions of agents, users' learning and self-evaluations. We then highlight the persistence of gender stereotypes in human-machine interactions by presenting research on users' perceptions of virtual agents depending on their gender (Section 3.4). Secondly, we focus on pedagogical agents and discuss research that shows the effect of their gender on learners (Section 3.5).

### 3.1 Subjective measures of users' perceptions of agents

139 Most of the studies used post-experience questionnaires to assess users' perceptions of virtual agents.
140 Likert scale items were used to determine participants' stereotyped attributions of the agents, corresponding
141 to communal traits stereotypically associated with women (e.g., affectionate, compassionate, sensitive,
142 inviting, helpful), agency traits stereotypically associated with men (e.g., arrogant, ambitious, aggressive,
143 courageous, and decisive), and competence traits associated more often with men (e.g., knowledgeable,
144 intelligent, expert, credible, creative, innovative and organized) (Lee, 2003; Nunamaker et al., 2011; Feng
145 et al., 2017; Khashe et al., 2017; van der Lubbe and Bosse, 2017; Kantharaju et al., 2018; Sczesny et al.,
146 2018). Another questionnaire was sometimes used to determine which stereotypical gendered traits users
147 applied to the agents (Kulms et al., 2011). This scale, the Bem Sex Role Inventory (BSRI) developed by
148 Bem (1974), measures the construction of the gender schema of individuals, aims to highlight androgyny,
149 and questions the usual dichotomy of female/male gendered traits stereotypically attributed to people. The
150 BSRI consists of 20 positive items stereotypically associated with men (e.g., independent, analytical), 20
151 other positive items stereotypically associated with women (e.g., compassionate, loves children), and 20
152 other positive neutral items (e.g., tactful, reliable). The agents' gender perception was evaluated with a 5-
153 point Likert sliding scale, for example with men=1, androgynous=4, and women=7 (Lee, 2003; McDonnell
154 et al., 2009; Niculescu et al., 2009; Nag and Yalçın, 2020). Other social attitude perceptions were also
155 assessed with Likert scale items, such as the perceived friendliness, trustworthiness, likability, and social
156 presence of the agent (Lee, 2003; Guadagno et al., 2007; Nunamaker et al., 2011; Lunardo et al., 2016;
157 Khashe et al., 2017; Akbar et al., 2018). Social presence is particularly important as it provides individuals
158 with the possibility of developing a relationship or having a social interaction with one another, as they
159 recognize each other as "social beings" (Biocca et al., 2003). Social presence is commonly defined as
160 the sensation of being in the presence of a real person and having access to their feelings (Biocca, 1997),
161 and can be assessed with a 5-item survey (e.g. "I feel that the person is watching me and is aware of my
162 presence") (Bailenson et al., 2001) and the Networked Minds Questionnaire (e.g. "The other individual
163 didn't notice me in the room") (Biocca et al., 2001), as used by Kulms et al. (2011).
164

## 3.2 Objective measures of learning

165

166

167

168

169

170

171

172

173

174

The impact of pedagogical agents on users' learning can be assessed by measuring users' performance in an exercise by comparing different conditions: for example, virtual agents with different behaviors (Chang et al., 2019), the presence of gendered virtual agents (Kim and Wei, 2011), or virtual agents with different genders (Kim, 2013). Performance can be measured with different problem-solving tests: using knowledge retention (using past knowledge to solve a problem, Sajjadi et al., 2020), recall (the ability to remember items, Wirzberger et al., 2019), or transfer learning (using past knowledge to solve new problems, Makransky et al., 2019). In addition to performance, researchers can also evaluate response times and effort. Effort can be measured by comparing the number of problems solved (that are not necessarily correct) in different problem-solving tests (Krämer et al., 2016). Response times correspond to the duration required to solve a problem (Hayes et al., 2010).

175

## 3.3 Users' self-evaluations

176

177

178

179

180

181

182

183

184

185

186

187

188

In learning situations, other more subjective measures than performance are rated using Likert-scale items. These measures include the interest in a task or a domain, (e.g., "I will take a hard sciences course as an elective", Rosenberg-Kima et al., 2008), beliefs about the utility of a task or a domain (e.g., "I would have many good career opportunities if I was a hard science major", Plant et al., 2009), learners' self-efficacy as in feeling capable of performing a task (e.g., "I can achieve high grades in math", Kim and Wei, 2011), learners' self-regulation to regulate their behaviors to succeed in a task (e.g., "I kept track of my progress", Baylor and Kim, 2004), learners' motivation assessed with the Situational Motivation Scale (SIMS) (Guay et al., 2000) which includes 16 items about the motivation to work on tasks (e.g., "Because I am doing it for my own good", Krämer et al., 2016), learners' enjoyment (e.g., "How much did you enjoy preparing for the exam?", Shiban et al., 2015), their perceived learning effectiveness (e.g., "I gained a good understanding of the basic concepts of the materials", Sajjadi et al., 2020), and their mental demand to know how much mental and perceptual activity was required (thinking, deciding, calculating, etc.), e.g., "Was the task easy or demanding?" (Hart and Staveland, 1988; Pezzullo et al., 2017).

189

## 3.4 Evidence of the persistence of gender stereotypes in human-machine interactions

190

191

192

193

194

We have summarized the selected studies on users' perceptions of virtual agents depending on their gender in each line of the following table. We stated agents' characteristics, number of male and female participants with their average age, tasks of the study, the observed measures and the study's results. Some acronyms are present in this table. We used MA for Male Agent(s) and FA for Female Agent(s). In the same logic, MP is used for Male Participant(s), and FP for Female Participant(s).
<table><caption>Table 1 Summary of articles on perceptive studies of virtual agents depending on their gender, regardless of the application domain. Articles are listed from oldest to most recent. FA = female agent, MA = male agent, FP = female participants, MP = male participants. The agents column describes the number of agents depending on their gender, their dimension (2-D or 3-D), their appearance (realist or cartoon), and their role. The participants column describes the number of men and women who participated in the study, and the average age. In the result(s) column, "MP > FP" means it impacted more the male participants than the female participants. "FA > MA" means the female agent has more impact than the male agent. Explanations are in section 3.4.</caption><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Lee, 2003)</td><td>• 1 MA<br>• 1 FA<br>• 2-D<br>• Text<br>• Cartoon<br>• Adviser</td><td>• 28 MP<br>• 88 FP<br>• avg age N/A</td><td>Playing a multiple-choice game with an agent. Participants could change their answer after they were told the agent's answer. It was specified that the agent's answer might not be correct.</td><td>• Masculinity<br>• Attractiveness<br>• Competence<br>• Trustworthiness<br>• Persuasiveness (sport or fashion questions)</td><td>• Masculinity: MA > FA<br>• Attractiveness, competence: FA > MA<br>• Persuasiveness (sport): MA > FA<br>• Persuasiveness (fashion): FA > MA</td></tr><tr><td>(Zanbaka et al., 2006)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 41 MP<br>• 97 FP<br>• avg age 20.6</td><td>Listening to agents deliver a message to change participants' attitudes about university-wide comprehensive exams.</td><td>• Persuasiveness</td><td>• Persuasiveness:<br>- MP: FA > MA<br>- FP: MA > FA</td></tr><tr><td>(Guadagno et al., 2007)</td><td>• 1 MA<br>• 1 FA<br>• 1 neutral<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 37 MP<br>• 29 FP<br>• avg age N/A</td><td>Listening to agents talk about changes to university security policy.</td><td>• Likeability<br>• Credibility<br>• Presentation quality<br>• Persuasiveness</td><td>• Likeability, credibility, presentation quality:<br>- MP: not significant<br>- FP: FA > MA<br>• Persuasiveness:<br>- MP: MA > FA<br>- FP: FA > MA</td></tr><tr><td>(Guadagno et al., 2007)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 85 MP<br>• 89 FP<br>• avg age N/A</td><td>Listening to agents talk about changes to university security policy.</td><td>• Likeability<br>• Credibility<br>• Presentation quality<br>• Social presence<br>• Persuasiveness</td><td>• Likeability: FA > MA<br>• Credibility, presentation quality: not significant<br>• Persuasiveness:<br>- MP: MA > FA<br>- FP: not significant</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Gulz et al., 2007)</td><td><ul><li>2 MA</li><li>2 FA</li><li>2-D</li><li>Voice</li><li>Realist</li><li>Presenter</li></ul></td><td><ul><li>72 MP</li><li>86 FP</li><li>avg age N/A</li></ul></td><td>Listening to agents present university program engineering.</td><td><ul><li>Favorite agent</li><li>Interest</li></ul></td><td><ul><li>Favorite agent:<ul><li>MP: less feminine & less masculine agents > more masculine agent > more feminine agent</li><li>FP: less feminine & less masculine agents > more feminine agent > more masculine agent</li><li>Interest:<ul><li>MP: more feminine & more masculine agents > less feminine & less masculine agents</li><li>FP: more feminine & more masculine & less feminine agents > less masculine agent</li></ul></li></ul></td></tr><tr><td>(Dill et al., 2008)</td><td><ul><li>16 MA</li><li>16 FA</li><li>3-D</li><li>Realist</li><li>Video game characters</li></ul></td><td><ul><li>61 MP</li><li>120 FP</li><li>avg age 18.82</li></ul></td><td>Watching a PowerPoint presentation opposing still pictures of video game characters and male or female US senators. Reading a real-life story about the sexual harassment of a female student by a male professor.</td><td><ul><li>Tolerance for sexual harassment</li><li>Rape-supportive attitudes</li></ul></td><td><ul><li>Tolerance for sexual harassment: MP > FP</li><li>Rape-supportive attitudes: MP > FP</li></ul></td></tr><tr><td>(Rosenberg-Kima et al., 2008)</td><td><ul><li>1 MA</li><li>1 FA</li><li>3-D</li><li>Voice</li><li>Realist</li><li>Speaker</li></ul></td><td><ul><li>89 FP</li><li>avg age 19.7</li></ul></td><td>Listening to an agent describe four female engineers and the benefits of engineering, with or without the agent present.</td><td><ul><li>Interest</li><li>Self-efficacy</li><li>Utility for engineering</li></ul></td><td><ul><li>Interest, self-efficacy, utility for engineering: not significant</li></ul></td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Rosenberg-Kima et al., 2008)</td><td>• 4 MA<br>• 4 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 111 FP<br>• avg age 19.72</td><td>Listening to an agent describe four female engineers and the benefits of engineering, with or without the agent present.</td><td>• Utility for engineering<br>• Fewer engineering gender stereotypes</td><td>• Utility for engineering:<br>MA &gt; FA (not significant)<br>• Fewer engineering gender stereotypes: FA &gt; MA</td></tr><tr><td>(Niculescu et al., 2009)</td><td>• 3 MA<br>• 3 FA<br>• 1 neutral<br>• 3-D<br>• Voice<br>• Cartoon<br>• Assistant</td><td>• 24 MP<br>• 24 FP<br>• avg age N/A</td><td>Interacting with agents about medical queries, evaluating an androgynous agent's gender either after or before rating non-androgynous agents.</td><td>• Androgynous agent's perceived gender</td><td>• Androgynous agent's perceived gender:<br>- Non-androgynous agents rated first:<br>- FP: more feminine<br>- MP: more masculine<br>- Androgynous agent rated first:<br>- FP: more masculine<br>- MP: more feminine</td></tr><tr><td>(McDonnell et al., 2009)</td><td>• 1 MA<br>• 1 FA<br>• 2 neutral<br>• 3-D<br>• Realist<br>• Subject</td><td>• 22 MP<br>• 19 FP<br>• avg age N/A</td><td>Watching a video of agents walking.</td><td>• Agents' perceived gender</td><td>• Agents' perceived gender:<br>- FA (male walk): ambiguous<br>- FA (neutral walk): female<br>- MA (female walk): ambiguous<br>- MA (neutral walk): male<br>- Genderless agents: ambiguous<br>- Genderless agents (female walk): female<br>- Genderless agents (male walk): male<br>- Genderless agents (neutral walk): female</td></tr></tbody></table>
<table><thead><tr><th>Author(s)</th><th>Agent(s)</th><th>Participant(s)</th><th>Task(s)</th><th>Measure(s)</th><th>Result(s)</th></tr></thead><tbody><tr><td>(McDonnell et al., 2009)</td><td><ul><li>3 MA</li><li>3 FA</li><li>3-D</li><li>Realist</li><li>Subject</li></ul></td><td><ul><li>33 MP</li><li>5 FP</li><li>avg age N/A</li></ul></td><td>Watching a video of agents walking.</td><td><ul><li>Agents' perceived gender</li></ul></td><td><ul><li>Agents' perceived gender: - FA rated 'most female': FA (bigger hips and breast size) &gt; FA (smallest hips and breast size) - MA rated 'most male': no difference</li><li>Agents rated 'most ambiguous': FA (male walk) &amp; MA (female walk)</li></ul></td></tr><tr><td>(Fox and Bailenson, 2009)</td><td><ul><li>4 FA</li><li>3-D</li><li>Realist</li><li>Subject</li></ul></td><td><ul><li>43 MP</li><li>40 FP</li><li>avg age 20.82</li></ul></td><td>Participants encountered an agent (low gaze (LG) or high gaze (HG), masculine or feminine clothes) via virtual reality, then made judgments about them.</td><td><ul><li>Rape myth acceptance</li><li>Benevolent sexism</li><li>Hostile sexism</li></ul></td><td><ul><li>Rape myth acceptance: masculine LG agent &gt; feminine HG agent &gt; masculine HG agent &gt; feminine LG agent</li><li>Benevolent sexism: masculine LG agent &gt; feminine LG agent &gt; masculine HG agent &gt; feminine HG agent</li><li>Benevolent sexism: LG agent &gt; HG agent</li><li>Hostile sexism: feminine HG agent &gt; masculine HG agent</li></ul></td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Cloud-Buckner et al., 2009)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Guide</td><td>• 19 MP<br>• 16 FP<br>• avg age N/A</td><td>Watching an agent introducing a college campus as an online tour guide.</td><td>• Friendliness<br>• Anger<br>• Cooperation<br>• Self consciousness<br>• Adventurousness<br>• Sympathy<br>• Sociability<br>• Assertiveness<br>• Cooperation<br>• Self consciousness<br>• Self discipline</td><td>• Friendliness, anger, cooperation, self consciousness, adventurousness, sympathy: Outgoing personality: MA > FA<br>• Sociability, assertiveness, cooperation, self consciousness, self discipline: Introverted personality: FA > MA</td></tr><tr><td>(Niculescu et al., 2010)</td><td>• 1 MA<br>• 1 FA<br>• 1 neutral<br>• 3-D<br>• Text<br>• Cartoon<br>• Tutor</td><td>• 4 MP<br>• 4 FP<br>• avg age N/A</td><td>Asking an agent medical questions.</td><td>• Comfortable<br>• Confident<br>• Less tense<br>• Preferred agent</td><td>• Comfortable, confident, less tense: FA > MA &amp; androgynous agent<br>• Preferred agent: FA > MA &amp; androgynous agent</td></tr><tr><td>(Rosenberg-Kima et al., 2010)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 119 FP<br>• avg age 21.49</td><td>Listening to an agent describe four female engineers and the benefits of engineering.</td><td>• Interest<br>• Self-efficacy<br>• Utility<br>• Agent's likeability<br>• Fewer engineering gender stereotypes</td><td>• Interest:<br>- Black FP: Black FA > others<br>- White FP: FA > MA<br>• Self-efficacy, utility, agent's likeability:<br>- Black FP (Black agents): FA > MA<br>- Black FP (White agents): MA > FA<br>• Fewer engineering gender stereotypes:<br>- Black FP: Black agents > White agents<br>- White FP: FA > MA</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Astrid et al., 2010)</td><td>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Questioner</td><td>• 41 MP<br>• 42 FP<br>• avg age 37.27</td><td>Answering personal questions from an agent.</td><td>• Weak<br>• Shy<br>• Naive<br>• Compassionate<br>• Inviting</td><td>• Weak, shy, naive, compassionate, inviting: not significant</td></tr><tr><td>(Nunamaker et al., 2011)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Questioner</td><td>• 53 MP<br>• 35 FP<br>• avg age 25.45</td><td>Answering questions from an agent simulating an airport screening.</td><td>• Power<br>• Trustworthiness<br>• Expertise<br>• Likability</td><td>• Power, trustworthiness, expertise: MA &gt; FA<br>• Likability: FA &gt; MA</td></tr><tr><td>(Kulms et al., 2011)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Questioner</td><td>• 32 MP<br>• 40 FP<br>• avg age 35.03</td><td>Answering casual questions asked by an agent, either in a low gaze (LG) or a high gaze (HG) condition.</td><td>• Masculinity<br>• Positive evaluation<br>• Social presence</td><td>• Masculinity: HG MA &gt; LG MA<br>• Positive evaluation: FA &gt; MA<br>• Social presence: MA &gt; FA</td></tr><tr><td>(Brahnam and De Angeli, 2012)</td><td>• 8 MA<br>• 8 FA<br>• 3 neutral<br>• 2-D<br>• Text<br>• Cartoon<br>• Chatbot</td><td>• 127 MP<br>• 73 FP<br>• avg age N/A</td><td>Chatting over text with a chatbot.</td><td>• Sexual discourse<br>• Avg number of words about money/job, and physical appearance</td><td>• Sexual discourse, avg number of words (physical appearance): FA &gt; MA<br>• Avg number of words (money/jobs): MA &gt; FA (among adult agents)</td></tr><tr><td>(Ozogul et al., 2013)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Cartoon<br>• Tutor</td><td>• 35 MP<br>• 42 FP<br>• avg age 12.83</td><td>Rating pictures of agents.</td><td>• Gender preference<br>• Preferred agent to learn about engineering from</td><td>• Gender preference:<br>- FP: FA &gt; MA<br>- MP: MA &gt; FA<br>• Preferred agent to learn about engineering from: young FA &gt; young MA &gt; old MA &gt; old FA</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Payne et al., 2013)</td><td><ul><li>4 MA</li><li>4 FA</li><li>2-D & 3-D</li><li>Cartoon</li><li>Assistant</li></ul></td><td><ul><li>220 MP</li><li>358 FP</li><li>avg age 35.56</li></ul></td><td>Choosing an agent to assist in self-service checkouts.</td><td><ul><li>Preferred agent</li></ul></td><td><ul><li>Preferred agent:<ul><li>FP: FA &gt; MA</li><li>MP: MA &gt; FA</li></ul></li></ul></td></tr><tr><td>(Lunardo et al., 2016)</td><td><ul><li>2 MA</li><li>2 FA</li><li>2-D</li><li>Text</li><li>Realist</li><li>Assistant</li></ul></td><td><ul><li>107 MP</li><li>147 FP</li><li>avg age N/A</li></ul></td><td>Interacting with an agent over text at fnac.com.</td><td><ul><li>Attractiveness</li></ul></td><td><ul><li>Attractiveness:<ul><li>Agents (corporate clothes):<ul><li>FA &gt; MA</li></ul></li><li>Agents (casual clothes):<ul><li>FA &gt; MA (not significant)</li></ul></li></ul></li></ul></td></tr><tr><td>(van der Lubbe and Bosse, 2017)</td><td><ul><li>2 MA</li><li>2 FA</li><li>3-D</li><li>Voice</li><li>Realist</li><li>Employee</li></ul></td><td><ul><li>55 MP</li><li>38 FP</li><li>avg age N/A</li></ul></td><td>Interacting with an agent employee to negotiate the agent's salary (assertive agent or non-assertive agent).</td><td><ul><li>Appropriate language</li><li>Sensitive</li><li>No deal reached</li><li>Persuasiveness</li></ul></td><td><ul><li>Appropriate language:<ul><li>assertive FA &gt; assertive MA</li></ul></li><li>Sensitive: non-assertive MA &gt; non-assertive FA</li><li>No deal reached:<ul><li>assertive MA &gt; assertive FA &gt; non-assertive FA &gt; non-assertive MA (not significant)</li></ul></li><li>Persuasiveness:<ul><li>assertive FA &gt; assertive MA</li></ul></li></ul></td></tr><tr><td>(Feng et al., 2017)</td><td><ul><li>1 MA</li><li>1 FA</li><li>3-D</li><li>Voice</li><li>Realist</li><li>Instructor</li></ul></td><td><ul><li>31 MP</li><li>32 FP</li><li>avg age 21.37</li></ul></td><td>Acting out a scene in presence of an agent giving negative feedback.</td><td><ul><li>Inspiration</li><li>Self-blame</li><li>Helpfulness</li><li>Preferred agent</li></ul></td><td><ul><li>Inspiration, self-blame, helpfulness, preferred agent:<ul><li>FA &gt; MA</li></ul></li></ul></td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Mell et al., 2017)</td><td>• 1 FA<br>• 2-D<br>• Text<br>• Realist<br>• Assistant</td><td>• 241 MP<br>• 140 FP<br>• avg age 35.13</td><td>Answering questions from a chatbot about sensitive information, either with a picture of a real woman, a picture of a female virtual agent, or no picture.</td><td>• Reported lies<br>• Allowing the system to do a credit check<br>• Providing their address</td><td>• Reported lies: human > no presence > agent<br>• Allowing the system to do a credit check:<br>- FP: agent > human > no presence<br>- MP: no presence > agent > human<br>• Providing their address:<br>- FP: equal across conditions<br>- MP: human > agent > no presence</td></tr><tr><td>(Khashe et al., 2017)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 98 MP<br>• 116 FP<br>• avg age N/A</td><td>Requested to switch off the lights and open the window by a manager, either voice only, text only, or a virtual agent).</td><td>• Affectionate<br>• Friendly<br>• Likable<br>• Persuasiveness</td><td>• Affectionate, friendly, likable: female (agent and voice only) > male (agent and voice only)<br>• Persuasiveness: female (agent, voice only, text only) > male (agent, voice only, text only)</td></tr><tr><td>(Kantharaju et al., 2018)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• 2 experts<br>• 2 motivators</td><td>• 113 MP<br>• 92 FP<br>• avg age N/A</td><td>Listening to a persuasive conversation about cinema between agents.</td><td>• Distant<br>• Arrogant<br>• Forceful<br>• Credible<br>• Persuasiveness</td><td>• Distant, arrogant, forceful, credible, persuasiveness: MP > FP</td></tr><tr><td>(Akbar et al., 2018)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Text<br>• Realist<br>• Interviewer</td><td>• 158 MP<br>• 158 FP<br>• avg age N/A</td><td>Interviewed by an agent over text for a job in a financial firm.</td><td>• Agreeableness<br>• Trustworthiness</td><td>• Agreeableness: opposite gender agent > matching-gender agent<br>• Trustworthiness: matching-gender agent > opposite gender agent</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Mousas et al., 2018)</td><td>• 2 MA<br>• 3-D<br>• Realist<br>• Subject</td><td>• 56 MP<br>• 16 FP<br>• avg age 23.24</td><td>Answering questions about the agents (e.g., "Would you feel uneasy if this virtual character communicated with you?") by the experimenter while the agent walked towards the participant.</td><td>• Easiness<br>• Comfortableness<br>• Readiness for interaction<br>• Likeability</td><td>• Easiness, comfortableness, readiness for interaction: MP > FP<br>• Likeability:<br>- zombie agent: MP > FP<br>- MA: not significant</td></tr><tr><td>(Ait Challal and Grynszpan, 2018)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Realist<br>• Subject</td><td>• 12 MP<br>• 12 FP<br>• avg age 23.6</td><td>Watched virtual agents sit in front of them (in gaze following, gaze avoidance, high direct gaze, and low direct gaze conditions). Judging their personalities.</td><td>• Neuroticism<br>• Agreeableness</td><td>• Neuroticism: FA > MA<br>• Agreeableness (high direct gaze condition): MA > FA</td></tr><tr><td>(ter Stal et al., 2020)</td><td>• 4 MA<br>• 4 FA<br>• 2-D<br>• Cartoon<br>• 4 experts, 4 peers</td><td>• 67 MP<br>• 69 FP<br>• avg age 51.36</td><td>Observing and rating 8 agents.</td><td>• Friendliness<br>• Expertise<br>• Authority</td><td>• Friendliness: FA > MA<br>• Expertise, authority: MA > FA</td></tr><tr><td>(ter Stal et al., 2020)</td><td>• 4 MA<br>• 4 FA<br>• 2-D<br>• Cartoon<br>• 4 experts, 4 peers</td><td>• 35 MP<br>• 30 FP<br>• avg age 67.85</td><td>Observing and rating 8 agents.</td><td>• Friendliness<br>• Authority</td><td>• Friendliness: not significant<br>• Authority: MA > FA</td></tr><tr><td>(Zibrek et al., 2020)</td><td>• 2 neutral<br>• 3-D<br>• Realist<br>• Subject</td><td>• 10 MP<br>• 10 FP<br>• avg age N/A</td><td>Pressing a button as soon as they felt uncomfortable with the distance between themselves and an agent walking towards them.</td><td>• Genderless agents' perceived gender</td><td>• Genderless agents' perceived gender:<br>- female motions = female<br>- male motions = male</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Richards et al., 2020)</td><td>• 6 MA<br>• 6 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Assistant</td><td>• 43 MP<br>• 146 FP<br>• avg age 21.7</td><td>Watching 12 videos of 12 different agents introducing themselves.</td><td>• Favorite agent (before and after watching the videos)</td><td>• Favorite agent (before):<br>- FP: gender does not matter > FA > MA<br>- MP: gender does not matter > MA > FA<br>• Favorite agent (after): Mediterranean FA > Asian FA > White FA</td></tr><tr><td>(Nag and Yalçın, 2020)</td><td>• 1 MA<br>• 1 FA<br>• 1 neutral<br>• 3-D<br>• Realist<br>• Subject</td><td>• 41 MP<br>• 31 FP<br>• avg age 21.7</td><td>Looking at pictures of agents and rating them.</td><td>• Communion<br>• Agency<br>• Competence</td><td>• Communion: FA > MA (not significant)<br>• Agency, competence: not significant</td></tr><tr><td>(Esposito et al., 2021)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Assistant</td><td>• 22 MP<br>• 24 FP<br>• avg age 71.59</td><td>Watching a video of an agent talking about daycare facilities for the elderly.</td><td>• Willingness to interact with the agent<br>• Attractiveness<br>• Usefulness<br>• Presentable<br>• Professional<br>• Of good taste<br>• Pleasant<br>• Original<br>• Creative<br>• Captivating</td><td>• Willingness to interact with the agent, attractiveness, usefulness, presentable, professional, of good taste, pleasant, original, creative, captivating: FA > MA</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Esposito et al., 2021)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Assistant</td><td>• 20 MP<br>• 25 FP<br>• avg age 71.22</td><td>Watching a video of an agent talking about daycare facilities for the elderly. (2nd experiment).</td><td>• Willingness to interact with the agent<br>• Attractiveness<br>• Usefulness<br>• Presentable<br>• Professional<br>• Of good taste<br>• Pleasant<br>• Original<br>• Creative<br>• Captivating</td><td>• Presentable, professional, of good taste, pleasant: FA > MA<br>• Willingness to interact with the agent, attractiveness, usefulness, original, creative, captivating: Not significant</td></tr><tr><td>(Vilaro et al., 2021)</td><td>• 3 FA<br>• 3-D<br>• Voice, text<br>• Realist<br>• Assistant, expert</td><td>• 53 FP<br>• avg age 60.90</td><td>Watching an agent deliver colorectal cancer screening messages.</td><td>• Trustworthiness<br>• Expertise</td><td>• Trustworthiness: not significant<br>• Expertise: agents (white medical coat) > agent (casual clothes)</td></tr><tr><td>(Antonio Gómez-Jáuregui et al., 2021)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Interviewer</td><td>• 16 MP<br>• 16 FP<br>• avg age 29.95</td><td>Introducing themselves to a blurred-face virtual agent for a job interview.</td><td>• Dominance<br>• Warmth</td><td>• Dominance: not significant<br>• Warmth: FA (mirrored movements) > FA (random movements)</td></tr></tbody></table>
<table><thead><tr><th>Author(s)</th><th>Agent(s)</th><th>Participant(s)</th><th>Task(s)</th><th>Measure(s)</th><th>Result(s)</th></tr></thead><tbody><tr><td>(Świdrak et al., 2021)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Player</td><td>• 15 MP<br>• 19 FP<br>• avg age 25</td><td>Playing a negotiation/ decision-making game with a female and a male agent.</td><td>• Touch pleasantness<br>• Touch awkwardness<br>• Touch adequacy<br>• Persuasiveness</td><td>• Touch pleasantness: FA > MA<br>• Touch awkwardness: FP > MP<br>• Touch adequacy: FA perceived as more masculine > FA perceived as less masculine<br>• Persuasiveness:<br>- MP: agents perceived as more masculine > agents perceived as less masculine<br>- FP: depends on the offer</td></tr><tr><td>(Świdrak et al., 2021)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Player</td><td>• 40 MP<br>• avg age 23</td><td>Playing a negotiation/ decision-making game with two female and two male agents.</td><td>• Masculinity<br>• Touch pleasantness<br>• Touch awkwardness<br>• Touch adequacy<br>• Persuasiveness</td><td>• Masculinity: masculine FA > feminine MA<br>• Touch pleasantness: FA > feminine MA<br>• Touch awkwardness: feminine MA > others (not significant)<br>• Touch adequacy: others > feminine MA (not significant)<br>• Persuasiveness: masculine-perceived agents > feminine-perceived agents</td></tr></tbody></table>
The studies presented in Table 1 show that gender stereotypes persist in human-machine interactions. Users' behavior varies according to the gendered appearance of virtual agents. For example, in De Angeli and Brahnam (2006), the female virtual agent received a number of violent sexual propositions and even rape threats; the male virtual agent received only a few sexual propositions, none of them violent ("gently presses my lips to yours into a small kiss"), and the other sexual comments made during the interactions with the male virtual agent targeted his girlfriend. In a similar study by Brahnam and De Angeli (2012), users interacted with several pairs of female/male agents, including child agents, White agents, Black agents, and 'old' agents. The female agents were the target of significantly more sexual discourse, comments on their appearance, and swear words than the male agents; this was even true for the pair of child agents. Other features of agents influenced the conversational topics, such as their age and appearance: users talked more about jobs, achievement and money with old agents dressed in formal clothing than with any other pair of agents. However, gender stereotypes still applied to this category, since users interacted more with the older male agent about these topics than with the older female agent.

An agent's gender also has a direct influence on participants' decisions. For instance, Lee (2003) reported that users followed more advice from virtual agents when their gender stereotypically matched the topic (e.g., a female agent and cosmetics, a male agent and sports). In this study, the female virtual agent presented as particularly feminine. This result should thus be verified in a separate study using a female virtual agent presenting a sport-oriented appearance to determine whether these results are due solely to gender and not to the agents' presentation (clothes and make-up). In Guadagno et al. (2007), the male virtual agent was more persuasive when perceived to be computer-controlled rather than human-controlled. The opposite was true for the female virtual agent. The authors concluded that these results may have been due to gender stereotypes, specifically by the "participants' expectations for interacting with a computer being more consistent with masculine stereotypes (e.g., competent), whereas expectations for interacting with a human are more consistent with feminine stereotypes (e.g., warm)."Not only the gender of a virtual agent but even their perceived masculinity can influence participants' decisions. In a decision-making game where virtual agents made a monetary offer to male participants, the number of offers accepted were higher with the agents perceived as more masculine (Świdrak et al., 2021). The same results were obtained in a similar study for male participants; in contrast, female participants accepted more offers from the agents than male participants but were only influenced by the offer itself, not by the agents' perceived masculinity (Świdrak et al., 2021).

An agent's gender also has an impact on how users perceive the agent in terms of stereotypical traits attributed to men and women (Sczesny et al., 2018). In a study by Nunamaker et al. (2011), the male agent was perceived as more powerful, whereas the female agent was perceived as more likable. Even when male and female agents wore the same clothes, exhibited the same verbal and non-verbal behaviors, and had their faces blurred (thus lacking salient indicators of gender), the female agent was rated higher for warmth than the male agent; however, they were rated similarly for dominance, a trait typically associated with men (Antonio Gómez-Jáuregui et al., 2021). In contrast Kulms et al. (2011), found in their main experiment that participants did not ascribe more masculine traits to the male agents nor more feminine traits to the female agents, unlike in their pretest with 14 participants using still pictures of the same virtual agents. The authors concluded that stereotyped attributions became less important when participants could interpret the behavior of the agents. However, a study by Ait Challal and Grynszpan (2018) contradicts this conclusion: the female agent was rated as less agreeable than the male agent when using high direct gaze. The authors suggested that participants were less tolerant of dominance when expressed by a female agent. Gender stereotypes associated with users' gender can also impact the ratings of virtual agents. In a study by Mousas et al. (2018), male participants reported feeling more at ease and comfortable with a zombie
agent than female participants; they also liked the zombie agent more than the female participants did. The authors concluded that gender stereotypes may have influenced the results, because stereotypes call for men to be calmer in the face of fear and embarrassment/disgust and to report milder emotional reactions.

Contexts stereotypically associated with one gender may also have an impact on participants' preferences as to the gender of agents: in two experiments conducted by ter Stal et al. (2020), elderly participants preferred still pictures of female agents in a healthcare context. According to the authors, this result could be due to the task - health coaching - being associated with female gender stereotypes. In addition, male agents were rated as more authoritarian and expert than female agents. In a study by Gulz et al. (2007), when virtual agents presented university programs in computer engineering, participants' interest was higher with feminine and masculine agents as compared to 'neutral' agents (a less feminine female agent and a less masculine male agent). However, participants who ranked the less feminine female agent as the best presenter chose her because they believed that she could make more girls interested in computer engineering ("she seems young and nice, and I think she would make more girls interested"); and participants who ranked the feminine female agent as the worst presenter chose her because she was a woman who did not seem to belong in that context ("as I said, a woman feels more welcoming than a man, but she looked so styled, which I don't like"). These results show that gender stereotypes apply to the appearance of female agents.

In addition to context, agents' roles can also influence how users perceive them. When female agents were presented as assistants to elderly people in their daily life, participants found them to be more worth interacting with, more useful, efficient and well designed, and more captivating, exciting, engaging and attractive than male agents (Esposito et al., 2021). However, in a similar experiment with silent agents, the agents' gender did not affect the participants in terms of the same criteria (Esposito et al., 2021). Voices could have influenced the perceived agents' masculinity/femininity, but this was not measured in the studies. In a different study, expert agents were rated as more credible than motivational agents regardless of their gender (Kantharaju et al., 2018).

However, a recent study by Nag and Yalçın (2020) contradicts previous research on how humans perceive virtual agents depending on their gender: still pictures of male and female agents were generally rated similarly for agency (traits typically associated with men: *ambitious, aggressive, courageous, decisive*) and *competence* (traits typically associated with men: *creative, intelligent, innovative, organized*), but not for *communion* (traits typically associated with women: *affectionate, compassionate, sensitive, inviting, helpful*) where female agents were rated higher. A limitation of this study is that the female and male agents were quite similar in appearance. This being said, the results of the study tend to be coherent with the evolution of gender stereotypes reported by Eagly et al. (2020) for the perception of *agency* and *competence* traits perception in interpersonal interactions: the gap in *agency* and *competence* in favor of men has reduced. However, the *communion* traits are still largely attributed to women. This raises the question of whether the evolution in the perception of stereotypes in human-human interactions shown by Eagly et al. (2020) can be observed similarly in human-virtual agent interactions.

Based on the research presented above, it seems that male virtual agents are perceived as more competent, especially regarding stereotypically male-related topics. They appear as better suited to represent a pedagogical virtual tutor in STEM fields (Science, Technology, Engineering, and Mathematics), since these fields are perceived as masculine (Makarova et al., 2019). In the next section, we focus more specifically on research on pedagogical agents and the impact of their gender on users' academic outcomes.
282

## 3.5 The effect of virtual agents' gender on academic outcomes

283

284

285

286

287

We have summarized the selected studies on the impact of gendered virtual agents in the context of a learning task in each line of the following table. We stated agents' characteristics, number of male and female participants with their average age, tasks of the study, the observed measures and the study's results. Some acronyms are present in this table. We used MA for Male Agent(s) and FA for Female Agent(s). In the same logic, MP is used for Male Participant(s), and FP for Female Participant(s).
<table><caption>Table 2 Summary of research studies on the impact of gendered virtual agents in the context of a learning task. Articles are listed from oldest to most recent. FA = female agent, MA = male agent, FP = female participants, MP = male participants. The agents column describes the number of agents depending on their gender, their dimension (2-D or 3-D), their appearance (realist or cartoon), and their role. The participants column describes the number of men and women who participated in the study, and the average age. In the result(s) column, “MP > FP” means it impacted more the male participants than the female participants. “FA > MA” means the female agent has more impact than the male agent. Explanations are in section 3.5.</caption><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Moreno et al., 2002)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 12 MP<br>• 27 FP<br>• avg age 20</td><td>Watching a video of a virtual agent giving a course, taking a multiple-choice test.</td><td>• Performance<br>• Perceived masculinity, femininity</td><td>• Performance: MA > FA<br>• Perceived masculinity, femininity: - FA: very feminine<br>- MA: masculine</td></tr><tr><td>(Baylor and Kim, 2004)</td><td>• 4 MA<br>• 4 FA<br>• 2-D, 3-D<br>• Voice<br>• Realist, cartoon<br>• Tutor</td><td>• 94 MP<br>• 218 FP<br>• avg age 20.54</td><td>Creating an instructional schedule with a virtual agent's help.</td><td>• Self-efficacy<br>• Self-regulation<br>• Knowledgeability<br>• Intelligence<br>• Learning</td><td>• Self-efficacy, self-regulation, knowledgeability, intelligence: MA > FA<br>• Learning: not significant</td></tr><tr><td>(Baylor and Kim, 2004)</td><td>• 6 MA<br>• 6 FA<br>• 2-D, 3-D<br>• Voice<br>• Realist, cartoon<br>• Expert, motivator, mentor</td><td>• 89 MP<br>• 140 FP<br>• avg age 19.39</td><td>Creating an instructional planning with a virtual agent's help.</td><td>• Knowledgeability<br>• Intelligence<br>• Learning<br>• Self-regulation<br>• Self-efficacy</td><td>• Knowledgeability, intelligence: MA > FA<br>• Learning, self-regulation: not significant<br>• Self-efficacy: FA > MA</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Moreno and Flowerday, 2006)</td><td>• 5 MA<br>• 5 FA<br>• 2-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 21 MP<br>• 59 FP<br>• avg age 26.88</td><td>Watching a video of a course taught by a virtual agent, taking a test.</td><td>• Helpfulness<br>• Motivation<br>• Selected agent<br>• Learning</td><td>• <i>Helpfulness, motivation, learning:</i> not significant<br>• <i>Selected agent:</i> matching-gender agent = opposite gender agent</td></tr><tr><td>(Kim et al., 2007)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Companion</td><td>• 11 MP<br>• 45 FP<br>• avg age 20.71</td><td>Creating a course on economic concepts with a virtual agent's help.</td><td>• Facilitating learning<br>• Engaging<br>• Human-like<br>• Learning (recall)</td><td>• <i>Facilitating learning, engaging, human-like:</i> MA > FA<br>• <i>Learning (recall):</i> not significant</td></tr><tr><td>(Plant et al., 2009)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Speaker</td><td>• 45 MP<br>• 61 FP<br>• avg age 13.63</td><td>Listening to a story about four female engineers and the benefits of engineering, either delivered by an agent or voice-only. Taking a math test.</td><td>• Interest<br>• Utility<br>• Self-efficacy<br>• Performance<br>• Fewer engineering gender stereotypes</td><td>• <i>Interest, utility:</i><br>FA > MA &amp; no agent<br>• <i>Self-efficacy:</i> MA &amp; FA > no agent<br>• <i>Performance:</i> FA > MA<br>• <i>Fewer engineering gender stereotypes:</i><br>- MP: agents > no agent<br>- FP: FA &amp; no agent > MA</td></tr><tr><td>(Hayes et al., 2010)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Observer</td><td>• 35 MP<br>• avg age 19.77</td><td>Controlling an avatar (1st or 3rd person view) while taking a math test, in the presence of a male or female agent, or without an agent.</td><td>• Social presence<br>• Performance<br>• Response times</td><td>• <i>Social presence:</i><br>MA > FA &amp; no agent<br>• <i>Performance, response times:</i><br>- 1st person: no agent &amp; MA > FA<br>- 3rd person: FA > no agent &amp; MA</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Kim and Wei, 2011)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 110 MP<br>• 100 FP<br>• avg age 15.93</td><td>Taking a math test without an agent, watching an agent explaining the lessons, resolving math problems with the agent (training), taking a 2<sup>nd</sup> math test without an agent.</td><td>• Selected agent<br>• Performance</td><td>• Selected agent: matching gender &amp; matching ethnicity agents > others<br>• Performance: everyone improved</td></tr><tr><td>(Silvervarg et al., 2013)</td><td>• 1 MA<br>• 1 FA<br>• 1 neutral<br>• 2-D<br>• Text<br>• Cartoon<br>• Tutee</td><td>• 46 MP<br>• 37 FP<br>• 12-14 years old</td><td>Interacting with an androgynous virtual tutee on a math lesson, then with either a female or a male virtual tutee.</td><td>• Perceived androgyny<br>• Preferred agent as tutee<br>• Preferred agent as chat partner</td><td>• Perceived androgyny: androgynous agent = androgynous<br>• Preferred agent as tutee:<br>- FP: androgynous agent > MA &amp; FA<br>- MP: androgynous agent > MA &amp; FA (not significant)<br>• Preferred agent as chat partner:<br>- FP: androgynous agent > MA<br>- MP: androgynous agent > FA</td></tr><tr><td>(Kim and Lim, 2013)</td><td>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 64 MP<br>• 56 FP<br>• avg age 15.93</td><td>Taking a math test without an agent, learning lessons with or without an agent, resolving math problems with or without an agent (training), taking a 2<sup>nd</sup> math test without an agent.</td><td>• Performance<br>• Self-efficacy</td><td>• Performance: everyone improved<br>• Self-efficacy:<br>- FP: agent present > no agent<br>- MP: no increase</td></tr></tbody></table>
<table><thead><tr><th>Author(s)</th><th>Agent(s)</th><th>Participant(s)</th><th>Task(s)</th><th>Measure(s)</th><th>Result(s)</th></tr></thead><tbody><tr><td>(Kim, 2013)</td><td><ul><li>1 MA</li><li>1 FA</li><li>1 neutral</li><li>2-D, 3-D</li><li>Voice, text</li><li>Cartoon</li><li>Tutor</li></ul></td><td><ul><li>68 MP</li><li>73 FP</li><li>avg age N/A</li></ul></td><td>Answering questions about a text asked by a virtual agent.</td><td><ul><li>Text comprehension</li></ul></td><td><ul><li>Text comprehension:<ul><li>FP = MP</li><li>FP: MA &amp; FA &gt; robot agent</li><li>MP: MA &gt; FA &amp; robot agent</li></ul></li></ul></td></tr><tr><td>(Johnson et al., 2013)</td><td><ul><li>1 MA</li><li>1 FA</li><li>3-D</li><li>Voice</li><li>Cartoon</li><li>Tutor</li></ul></td><td><ul><li>88 MP</li><li>109 FP</li><li>avg age 12.1</li></ul></td><td>Watching an agent teaching a lesson on electrical circuits, taking a multiple-choice test.</td><td><ul><li>Performance</li><li>Program evaluation</li></ul></td><td><ul><li>Performance: not significant</li><li>Program evaluation: FA &gt; MA</li></ul></td></tr><tr><td>(Ozogul et al., 2013)</td><td><ul><li>2 MA</li><li>2 FA</li><li>3-D</li><li>Voice</li><li>Cartoon</li><li>Tutor</li></ul></td><td><ul><li>173 MP</li><li>161 FP</li><li>avg age 12.3</li></ul></td><td>Watching an agent (chosen or randomly assigned) teaching a lesson on electrical circuits, taking a multiple-choice test.</td><td><ul><li>Performance</li><li>Program evaluation</li><li>Selected agent</li></ul></td><td><ul><li>Performance:<ul><li>Random agent: not significant</li><li>Selected agent: opposite gender agent &gt; matching-gender agent</li><li>Program evaluation: not significant</li><li>Selected agent: matching-gender agent &gt; opposite gender agent</li></ul></li></ul></td></tr><tr><td>(Shiban et al., 2015)</td><td><ul><li>1 MA</li><li>1 FA</li><li>3-D</li><li>Text</li><li>Cartoon</li><li>Tutor</li></ul></td><td><ul><li>21 MP</li><li>73 FP</li><li>avg age 20.20</li></ul></td><td>Taking a math test while a virtual agent provided feedback.</td><td><ul><li>Interest</li><li>Motivation</li><li>Enjoyment</li><li>Credible</li><li>Engaging</li><li>Human-like</li><li>Facilitating learning</li><li>Performance</li></ul></td><td><ul><li>Interest, motivation: FA &gt; MA</li><li>Enjoyment: not significant</li><li>Credible, engaging, human-like: MA &gt; FA</li><li>Facilitating learning: not significant</li><li>Performance: MA &gt; FA (slightly)</li></ul></td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Kim, 2016)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 67 FP<br>• avg age 15.51</td><td>Listening to an agent speak persuasively about the benefits of STEM fields, solving math problems with the same agent, solving math problems without the agent.</td><td>• Credibility<br>• Friendliness<br>• Helpfulness<br>• Positive attitudes to learn math</td><td>• Credibility, friendliness, helpfulness:<br>- Ethnic-minority participants:<br>- MA: peer agent > teacher agent<br>- FA: teacher agent > peer agent<br>- Caucasians participants:<br>- MA & FA: not significant<br>• Positive attitudes to learn math:<br>- Ethnic-minority participants:<br>- MA: peer agent > teacher agent<br>- FA: teacher agent > peer agent<br>- Caucasians participants:<br>- MA & FA: not significant</td></tr><tr><td>(Krämer et al., 2016)</td><td>• 2 MA<br>• 2 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Motivating interviewer</td><td>• 60 MP<br>• 68 FP<br>• avg age 23.85</td><td>Taking a math test without an agent, then taking a math test with an agent present explaining the procedure.</td><td>• Motivation<br>• Sense of rapport<br>• Performance</td><td>• Motivation, sense of rapport:<br>not significant<br>• Performance:<br>- FP & rapport agent: MA > FA<br>- MP & rapport agent: FA > MA</td></tr><tr><td>(Li et al., 2016)</td><td>• 1 MA<br>• 1 neutral<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 20 MP<br>• 20 FP<br>• avg age 20.48</td><td>Watching an agent present slides on courses about Human-Computer Interaction.</td><td>• Learning</td><td>• Learning:<br>- MP: agent robot > real human (male) > MA > still image of a robot<br>- FP: no differences</td></tr><tr><td>(Jeong et al., 2017)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Instructor</td><td>• 54 MP<br>• 63 FP<br>• avg age 20.94</td><td>Listening to negative feedback from an instructor agent while acting out a scene. Reproducing the scene with the instructor agent and a student agent (no feedback).</td><td>• Moving forward<br>• Moving backward</td><td>• Moving forward:<br>- FP: FA > MA<br>- MP: MA > FA<br>• Moving backward:<br>- FP: MA > FA<br>- MP: FA > MA</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Pezzullo et al., 2017)</td><td>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Companion</td><td>• 54 MP<br>• 63 FP<br>• avg age 13.30</td><td>Playing a game about biology courses with a virtual agent's help.</td><td>• Mental demand<br>• Engagement with the agent<br>• Performance</td><td>• Mental demand, engagement with the agent: FP > MP<br>• Performance: FP = MP</td></tr><tr><td>(Wirzberger et al., 2019)</td><td>• 1 MA<br>• 3-D<br>• Voice<br>• Realist<br>• Instructor</td><td>• 27 MP<br>• 35 FP<br>• avg age 69.03</td><td>Memorizing a word list after taking a memory training course led by an agent.</td><td>• Learning (recall)</td><td>• Learning (recall): FP > MP</td></tr><tr><td>(Makransky et al., 2019)</td><td>• 1 FA<br>• 1 neutral<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 33 MP<br>• 33 FP<br>• avg age N/A</td><td>Watching a virtual agent teaching laboratory safety, taking tests.</td><td>• Social presence<br>• Learning (recall and transfer-learning)</td><td>• Social presence:<br>- FP: FA = drone agent<br>- MP: FA > drone agent<br>• Learning (recall &amp; transfer-learning):<br>- FP = MP<br>- FP: FA > drone agent<br>- MP: drone agent > FA</td></tr><tr><td>(Chang et al., 2019)</td><td>• 1 MA<br>• 3-D<br>• Voice<br>• Realist<br>• Instructor</td><td>• 76 FP<br>• avg age N/A</td><td>Controlling either a male or a female avatar, learning how to solve arithmetic problems from a male agent (either a dominant or a non-dominant agent, based on his body posture), solving problems without the agent present.</td><td>• Learning (recall &amp; performance)</td><td>• Learning (recall &amp; performance):<br>- non-dominant agent > dominant agent<br>- No significant effect of avatar's gender</td></tr></tbody></table>
<table><thead><tr><td>Author(s)</td><td>Agent(s)</td><td>Participant(s)</td><td>Task(s)</td><td>Measure(s)</td><td>Result(s)</td></tr></thead><tbody><tr><td>(Sajjadi et al., 2020)</td><td>• 1 MA<br>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Instructor</td><td>• 8 MP<br>• 4 FP<br>• avg age 19.6</td><td>Observing geologic formations in a virtual environment, answering questions asked by an agent.</td><td>• Perceived learning effectiveness<br>• Learning<br>• Leadership<br>• Friendliness<br>• Social and spacial presence</td><td>• Perceived learning effectiveness: FA > MA<br>• Learning: not significant<br>• Leadership, friendliness, social and spacial presence: FA > MA (not significant)</td></tr><tr><td>(Spilioto-poulos et al., 2020)</td><td>• 1 FA<br>• 3-D<br>• Voice<br>• Realist<br>• Tutor</td><td>• 24 MP<br>• 16 FP<br>• avg age 20</td><td>Learning how to use argumentation, how to be empathetic to the needs of others, how to reach agreements through negotiation with a virtual agent.</td><td>• Self-efficacy<br>• System easiness<br>• Helpfulness<br>• Learning</td><td>• Self-efficacy: not significant<br>• System easiness: FP > MP<br>• Helpfulness: MP > FP<br>• Learning: not significant (increase overall)</td></tr></tbody></table>
Various studies on virtual learning environments (Table 2) have reported that the gender of a pedagogical agent may have an impact on the learning performance of users. In a recent article, Makransky et al. (2019) showed that young girls performed better on scientific tasks (in terms of learning and transfer learning) when taught by a virtual female scientist than by a virtual drone. The opposite was true for boys. The researchers argued that boys identified with the drone, while girls identified with the female agent. However, research opposing human-like vs. robot-like agents does not take into account other factors that may influence how girls learn. In a study by Shiban et al. (2015), female learners were more motivated and interested in math when trained by a female agent as compared to a male agent. However, they obtained better results with the male agent, which may be explained by their perception of the agents' appearances: the male agent was older and wore a tie, while the female agent was young and pretty. According to the authors, the participants' performance improved because the male agent was perceived as an expert, and virtual agents perceived as experts have been shown to improve learners' performance (Baylor and Kim, 2004). The researchers also concluded that the female participants' motivation and interest improved with the female agent because there were more female participants in the study and because of the agent's similarity (in age and gender) to them, in line with the "similarity hypothesis" (also found in Rosenberg-Kima et al. (2008)). This argument is supported by Bandura's social cognitive learning theory: people often learn by imitating people whom they perceive as similar (or superior: higher in rank or status) to them and who are therefore accepted as social role models (Bandura, 1986). This theory also bears out in a study by Plant et al. (2009), where a female agent raised participants' self-efficacy by delivering a message on the benefits of engineering, resulting in better performance and more interest in math.

However, other research has demonstrated a positive effect of male agents as compared to female agents in pedagogical tasks. For instance, in two experiments conducted by Baylor and Kim (2004), a virtual agent helped participants create a schedule. The agent's gender did not impact learning but did affect self-efficacy, which increased more in the first experiment with the male agent than the female agent; the contrary occurred in the second experiment for both male and female participants. The researchers suggested that there was a bias in the first experiment, as participants rated the male agent as more interesting and useful than the female agent. In the second experiment, participants viewed the female agent as less expert and knowledgeable than the male agent, despite receiving the same instructions from both agents; some research has indicated that agents perceived as less intelligent could lead to greater self-efficacy Baylor and Kim (2005). In a similar experiment by Kim et al. (2007), the researchers introduced a female and a male pedagogical agent to help students design an e-learning course, which included creating a schedule. Students working with the male agent rated him higher on facilitating learning, being engaging, and being human-like than students working with the female agent. Notably, the male agent had a more positive impact than the female agent on the participants' interest and learning in terms of recall (the ability to remember what the agent said during the task).

Other factors may also be come into play in studies on the impact of virtual agents on learning. In Moreno et al. (2002), participants watched a video of a virtual agent presenting a course on blood pressure, followed by a multiple-choice test. The results of this study suggest that the participants learned more from the male agent than the female one. The researchers suggested that this might be because the female tutor did not conform to the stereotype of men as teachers. The first study showed that participants in this experiment perceived the female agent as very feminine, while the male agent was found to be very masculine. This may be due to a difference in the participants' interpretation of the female agent as being "too feminine" to be suitable for the role of tutor. The study did not address how participants perceived the agents' expertise or seek to determine any possible interactions between perceived expertise, the perceived
agent's femininity and performance on the test. Gender, while important, must be taken into account in combination with other features. For instance, Krämer et al. (2016) analyzed the impact of pedagogical agents' gender and their behavior on adults' motivation, effort and performance in math. They found that the simple presence of a female virtual agent in a learning situation did not increase women's motivation and learning. However, when the agent displayed human-like non-verbal behavior by aligning with the participants' non-verbal behavior (Gratch et al., 2007), the participants' performance and effort improved. This kind of behavior, called rapport, is defined in social psychology as the establishment of a positive relationship between interactants by way of a positive attitude (e.g., acquiescence, smiles), mutual attention (e.g., mutual gaze) and coordination of behaviors (e.g., synchrony, mimicry) (Tickle-Degnen and Rosenthal, 1990). This research shows the importance of the pedagogical agents' behavior combined with their gender as providing a positive impact on academic outcomes. Agents' behavior is especially important as it could negatively impact learners' academic outcomes, as shown in an experiment by Chang et al. (2019) where a male "dominant" pedagogical agent impaired female participants' performance and recall in arithmetic problems, compared to a male "non-dominant" agent.

The research presented above highlights the importance of pedagogical agents' gender on learning. Different studies appear to yield contradictory results, on one hand that learning improves when virtual agents' gender matches the learner's, but on the other hand that male virtual agents could be better suited to improving learning. Interestingly, Section 3.4 shows that male agents are perceived as more competent than female agents, and users follow more advice from a male agent than a female one in topics stereotypically perceived as masculine. However, the studies featuring a female agent in STEM fields (Science, Technology, Engineering, and Mathematics) presented in this section show that female agents have a positive influence on academic outcomes: they improve learning, self-efficacy, interest, and motivation, despite the fact that STEMs are perceived as masculine (Makarova et al., 2019).

# 4 DISCUSSION

## 4.1 The question of pedagogical agents' gender.

Based on the research presented above, one could surmise that, in general, male pedagogical agents are better suited to improving academic outcomes than female agents. However, systematically relying on male pedagogical agents could have an adverse impact: designing only male agents for learning purposes in STEM fields could strengthen gender stereotypes, for instance. As highlighted by West et al. (2019), the gender bias of interactive systems not only perpetuates stereotypes, but also reinforces and extends them. The stereotypes modeled through interactive systems generate behaviors that go beyond the sphere of the virtual environment by conveying a harmful image of women. For instance, in a study by Dill et al. (2008), still pictures of men and women in suits or male and female characters acting in highly stereotypical ways were shown to participants. Male participants exposed to negative female stereotypes were significantly more tolerant of a real-life instance of sexual harassment and exhibited greater rape myth acceptance. As for representation in STEM fields, as noted by Sansone (2019), the lack of female role models can lead female students to believe that men are better than women in STEM fields. The lack of *virtual* female role models in virtual learning environments may have the same impact. Accordingly, more STEM experts represented with virtual female characters could help decrease gender stereotypes in STEM fields.

Some research has explored the use of androgynous virtual agents to counter gender stereotypes. In earlier studies, participants tended to apply the labels of 'man' or 'woman' to androgynous agents. For instance, in Niculescu et al. (2009), participants classified androgynous virtual characters as male or female,
depending on the participants' gender and other parameters such as which virtual characters they had seen before. Even for genderless agents such as a wooden mannequin, the participants perceived their gender depending on how they perceived their walking motions (McDonnell et al., 2009). Recent research has shown more promising results in terms of gender stereotypes. In Nag and Yalçın (2020), results for androgynous agents show a linear trend that positions their scores for perceived agency, communion, and competence in between those for female and male agents. The authors thus believe that androgynous agents could help mitigate male and female stereotypes. Although participants in their first experiment tended to believe that the androgynous virtual agents were men, when the authors modified the agents in question for their main experiment, participants correctly perceived them as androgynous after reading a definition of an androgynous agent.

What about androgynous pedagogical agents in an educational context? Silvervarg et al. (2013) supposed, but with caution, that students could identify with an androgynous agent by ascribing their own gender on them, thus making them a suitable role model. Indeed, in their experiment with children aged 12-14, participants perceived an androgynous pedagogical agent as not clearly a boy nor clearly a girl, but they generally assigned themselves a gender to their androgynous virtual tutee, boy or girl. The authors supposed students could therefore have more freedom to construct and ascribe gender, as their pedagogical agent's gender choice is personal rather than imposed. They also supposed androgynous agents could diminish gender stereotypes, as their appearances are genderless. Applying our own gender to an androgynous agent to make them a suitable role model is an interesting hypothesis, however, we do not know what is the gender participants applied on the androgynous agent, nor why. More research on androgynous agents has to be done in an educational context to help determine, for instance, whether androgynous agents are perceived as masculine, feminine, neutral, man, woman, or genderless depending on the context and the role of the agent. Since STEM fields are considered masculine fields (Makarova et al., 2019), participants could perceive an androgynous agent as a man, even though they could perceive them as not clearly a boy nor a girl in terms of appearance. This could reinforce the stereotype of STEM fields as more suitable for men than women. Agents' role is also particularly important, as Brahnam and Weaver (2015) stated there are more female assistant agents than male one. They showed the example of a webpage that provides virtual agents, four of the five virtual agents are female and they assist people at airports or serve as talking mannequins for fashion and museum exhibits. The male agent was called a "virtual doctor" and provided health tips and hospital information. We can emit the hypothesis that one could perceive androgynous virtual assistants as women, hence reinforcing gender stereotypes. For this research on androgynous virtual agents, we recommend measuring how participants feel towards the androgynous agents, as not being able to perceive someone as a man or a woman may induce insecurity and unease in some people (Nass and Brave, 2005).

## 4.2 Virtual agents as social role models in learning environments

Some research, though still very limited, has explored the use of virtual agents to increase learners' performance and interest in mathematics. For example, Rosenberg-Kima et al. (2008) showed the effectiveness of a female virtual agent engineer in interesting women in STEM fields. In a video, the agent, who was similar in gender to the participants (who were all women), presented a story about successful female role models in STEM fields. This led to a change in participants' attitudes towards science, as shown with a 7-point scale questionnaire. Women in the female virtual agent condition were less likely to endorse traditional STEM stereotypes than those in the male virtual agent condition, and were more likely to believe that women could succeed in STEM fields. Gender stereotypes still persisted: the participants were slightly more likely to believe in STEM usefulness with a male virtual agent engineer. In a similar
study by Plant et al. (2009), male and female participants performed better and were more interested in engineering after interacting with a female agent: she raised their self-efficacy and their ratings about STEM usefulness. Interestingly, male participants were less likely to endorse traditional STEM stereotypes in the presence of an agent, male or female; but female participants were less likely to endorse traditional STEM stereotypes with a female agent or without any agent, than with a male one. Another similar study by Rosenberg-Kima et al. (2010) showed that Black virtual agents had a more positive impact on STEM interest and STEM gender stereotypes for Black women, whereas female virtual agents (Black or White) had a more positive impact on White women on the same criteria. This research shows the importance of other factors, such as virtual agents' ethnic background, on performance and interest in math.

Finally, several studies have shown that pedagogical agents used as learning companions can simulate social interactions (Kim and Baylor, 2006) and the potential impact of a virtual agent's gender on education. However, only a few studies have explored the use of a virtual pedagogical companion to counteract the effects of Stereotype Threat (see Introduction). Research on Social Cognition has shown the positive impact of social role models to counteract ST effects (Bagès et al., 2016). Studies have shown that female participants do not immediately see female scientists as potential role models simply by interacting with them; they begin to perceive female scientists as role models when they establish personal connections with them (Buck et al., 2008). In the field of virtual agents, virtual rapport has been studied as a means to create this type of relationship between virtual agents and users (Gratch et al., 2007). As reported by Krämer et al. (2016), the mere presence of a female agent did not improve participants' performance and effort. However, when agents were able to create a virtual rapport, participants' performance and effort were shown to improve.

Based on the research presented above, not only is the gender of pedagogical agents important, but so is their behavior (Krämer et al., 2016; Chang et al., 2019), their role (Baylor and Kim, 2004; Kim, 2016), and their ethnicity (Rosenberg-Kima et al., 2010; Kim, 2016). Girls may see a female pedagogical agent as a role model who influences their motivation to exert effort to learn (Shiban et al., 2015). A study by Pfeifer and Lugrin (2018) shows that female social robots can be role models to female students: female students learned better with a female robot in a stereotypically masculine domain. Virtual characters can be used to embody social models and thus change the learner's attitudes and motivation; as described earlier, a female role model who succeeds in math can reduce Stereotype Threat effects. Combining research on social cognition and on virtual agents, we recommend counteracting Stereotype Threat effects for girls and women in math by using a virtual agent representing a hardworking female social role model (Bagès et al., 2016) able to establish rapport with the learners (Gratch et al., 2007), of a similar ethnicity to the learners (Kim, 2016) and slightly older than them (Bagès and Martinot, 2011). When the role model is younger or the same age as the learners, they can lose motivation by feeling unable to match their role model's achievements; if the model is too old, they won't identify with her/him. A pedagogical agent should thus embody the role of a knowledgeable and motivational person; this has been demonstrated by student preferences and by the proven positive impact these types of agents have on education (Kim and Baylor, 2016).

4.3 Improved learning or better inclusion?

An ethical tension between two competing goals arises in all domains: skill learning (where the research presented above favors the use of a male virtual agent), vs. better inclusion of girls and women (via the use of a female virtual character embodying a successful role model in the domain). Prior research is not robust enough to prove the superiority of a male agent in all fields and for all audiences. Some questions remain
unanswered in the literature, to our knowledge: Would using the same androgynous character but presented as male, female, or neutral by the experimenter have an impact on academic outcomes in scientific or other domains? What would be the impact of systematically using a virtual agent of the same gender as the learners?

Regarding the second question, using only successful male models in mathematics with boys could reinforce gender stereotypes. Women are aware of the negative stereotype about their mathematical skills that create a hostile environment for them. Research by Stokes et al. (1995) reported that when women find a friendly environment, they are more likely to stay employed. One solution to reconciling the two goals, at least in STEM fields, would be to use successful female role models to explain how they managed to perform well: in Bagès et al. (2016), students took a math test after reading a story about a social model, female or male depending on the condition. The stories differentiated between models: the hardworking model put in effort and spent time learning his or her lessons to perform well, the gifted model was naturally good at math, and the neutral model gave no explanation for his or her success. Girls' performance increased with the hardworking model: they performed at the same level as boys, whether the model was a boy or a girl. There was no impact on boys' performance. In contrast, in a similar study, boys' performance also increased with a hardworking model, regardless of gender (Bagès and Martinot, 2011). Furthermore, when the role model did not explain his or her success in math, both girls' and boys' scores improved with a female role model. As the lack of female role models may lead female students to believe that men are better than women in STEM fields Sansone (2019), it could be interesting to combine the results of (Bagès et al., 2016) and our hypothesis that a successful *virtual* female role model in STEM fields could help mitigate gender stereotypes. Female virtual agents who act as successful social models in mathematics and explain how they succeeded through their effort and hard work may be a potential solution to counteracting Stereotype Threat effects. Another question then arises: in the long term, what would be the impact of presenting only these kinds of female virtual agents to boys?

## 5 CONCLUSION

In this article we have presented a systematic review of research on perceptive studies of virtual agents depending on their gender, regardless of the application domain; and on the impact of gendered virtual agents in the context of a learning task. Each study has been performed in a specific learning context with specific pedagogy, design of the virtual environment, duration of the interaction, modality of interaction, physical environment, etc. These elements of context may have an impact on the users' perceptions and learning outcomes. One limit of this article is that we have not considered all these contextual specificities. Further analysis could take into account these contextual elements to provide a more global view of the impact of virtual agents' gender in academic learning. Nevertheless, the present systematic review enables us to draw some conclusions by answering each question stated in Section 2.2.

*Do students perceive virtual agents differently depending on their own gender and the gender of the agent?* As individuals communicate with virtual agents by applying social rules and expectations as social beings (Nass and Moon, 2000), it is not surprising that they also apply gender stereotypes to virtual agents and their interactions with them (see Section 3.4). Female virtual agents are usually seen as less expert, less knowledgeable, and less powerful than male virtual agents (Nunamaker et al., 2011; Baylor and Kim, 2004), and they are also usually perceived as more likable and attractive than male virtual agents (Nunamaker et al., 2011; Lunardo et al., 2016). Those perception differences can even affect people's decisions (Lee, 2003; Świdrak et al., 2021).
Given the empirical results, we propose to respond simultaneously to question 2 (*Does the gender of pedagogical agents influence students' academic performance and self-evaluations?*) and question 3 (*Are there tasks or academic situations which a male virtual agent is better suited to than a female virtual agent, and vice versa, according to empirical evidence?*). The review conducted in this paper could lead to believe that male pedagogical agents are better suited than female agents to improve academic outcomes, especially in male-dominated scientific fields like STEM fields (Makarova et al., 2019). However, research also shows that female pedagogical agents can improve learners' performances in these fields (Plant et al., 2009; Rosenberg-Kima et al., 2008). Some studies have shown that using female virtual agents as social role models can increase female participants' self-efficacy (Rosenberg-Kima et al., 2008), and both male and female participants' interest (Plant et al., 2009; Rosenberg-Kima et al., 2010), and performances (Plant et al., 2009). These results are especially relevant to address Stereotype Threat effects, a phenomenon that illustrates how and why female students' performance in math can be impaired by gender stereotypes (Spencer et al., 1999). Previous research showed that stereotype threat effects can be counteracted by introducing a female positive role model who looks like learners so that they can identify with (Bagès et al., 2016). Such a positive role model could be embodied by a virtual pedagogical agent (Rosenberg-Kima et al., 2008), and used to reduce stereotype threat in STEM fields.

*How do a virtual agent's pedagogical roles impact these results?* Individuals tend to listen more to agents whose gender stereotypically matches the context or gender roles such as female virtual agents with cosmetics and male virtual agents with sports (Lee, 2003), female virtual agents in contexts involving social influence (Khashe et al., 2017), and female virtual agents in an assistant role (Esposito et al., 2021)). The role of virtual pedagogical agent has been studied in academic context, using *expert*, *motivator*, and *mentor* agents (Baylor and Kim, 2004). Expert agents are older than students, authoritative, strictly informative, and knowledgeable. Motivator agents are enthusiastic and not seen as particularly knowledgeable, they are mostly used to elicit motivation. As for mentor agents, they are slightly older than students, are knowledgeable, and they are also used to elicit motivation. They are a mix of expert agents and motivator agents (Baylor and Kim, 2005). Researchers should take agents' role into account when designing a pedagogical agent, as a female pedagogical agent designing as a mentor agent can improve learners' performance (Plant et al., 2009), but a female pedagogical agent designing as a motivator agent may not be effective on learners' performance (Shiban et al., 2015). In conclusion, the impact of the virtual agent's gender depends on several aspects related to the role of the agent. This role is indeed reflected through for instance the appearance but also the discourse of the agent.

*How do a virtual agent's appearance and interactive capacities impact these results?* The topics of individuals' interactions with virtual agents differ depending on their gendered appearance: in studies by De Angeli and Brahnam (2006) and Brahnam and De Angeli (2012), female virtual agents received significantly more violent sexual propositions, more rape threats, more comments on their appearance, and more swear words compared to male virtual agents who received few sexual propositions, most of them targeting their girlfriends. Moreover, the degree of perceived masculinity and femininity can influence men's decision, as shown in Świdrak et al. (2021) where male participants were persuaded more by masculine agents than feminine agents, regardless of the agents' gender. Agents' appearance can influence their perceived role, as an old agent wearing a tie could be perceived more as an expert than a young agent (Shiban et al., 2015). As seen in the question above, agents' role is important in academic situations. In addition to agents' role and appearance, research has shown the importance of a positive relationship between learners and pedagogical agents (Krämer et al., 2016). This research tends to show that a female social model embodied by a pedagogical agent able to establish a positive relationship with learners may
counteract Stereotype Threat effects, and thus improve women's performance, interest, and self-efficacy in mathematics.

*Are androgynous virtual agents a potential solution to combatting gender stereotypes?* This question is quite difficult to answer as to our knowledge, few research has explored the use of androgynous virtual agents to counter gender stereotypes in academic context. Silvervarg et al. (2013) used an androgynous pedagogical agent in an educational context and showed participants, although evaluating the agent as "not clearly a boy nor a girl", tend to ascribe a binary gender (boy or girl) on the agent. The authors thus cautiously supposed that students could ascribe their own gender on an androgynous agent, thus giving them more freedom and making the agent a suitable role model, known to be beneficial for academic outcomes. However, results did not show what gender participants ascribe on the agent, nor why. The way individuals ascribe gender to an androgynous or a genderless agent should be studied more. In particular, does this gender attribution depend on the context or agents' role? Since STEM fields are considered masculine fields (Makarova et al., 2019), participants could think androgynous agents are men. This could reinforce the stereotype of STEM fields as more suitable for men than women. As for agents' roles, there are more female virtual assistants than male ones (Brahnam and Weaver, 2015). Some developers admitted female virtual assistants are usually used because they evoke gender stereotypes: women are expected to serve, help, and nurture others. Androgynous virtual assistants could then be considered women, and reinforce harmful stereotypes on women. Researchers and developers who want to use androgynous agents to combatting gender stereotypes should be very careful, as the opposite effect can occur.

# 6 ACKNOWLEDGMENTS

This work was carried out within the pilot center Ampiric, funded by the French State's Future Investment Program (PIA3/France 2030) as part of the "Territories of Educational Innovation" action.

# REFERENCES

Ait Challal, T. and Grynszpan, O. (2018). What gaze tells us about personality. In *Proceedings of the 6th International Conference on Human-Agent Interaction*. 129–137

Akbar, F., Grover, T., Mark, G., and Zhou, M. X. (2018). The effects of virtual agents' characteristics on user impressions and language use. In *Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion*. 1–2

Antonio Gómez-Jáuregui, D., Giraud, T., Isableu, B., and Martin, J.-C. (2021). Design and evaluation of postural interactions between users and a listening virtual agent during a simulated job interview. *Computer Animation and Virtual Worlds*, e2029

Astrid, M., Krämer, N. C., and Gratch, J. (2010). How our personality shapes our interactions with virtual characters-implications for research and development. In *International Conference on Intelligent Virtual Agents* (Springer), 208–221

Bagès, C. and Martinot, D. (2011). What is the best model for girls and boys faced with a standardized mathematics evaluation situation: A hardworking role model or a gifted role model? *British Journal of Social Psychology* 50, 536–543

Bagès, C., Verniers, C., and Martinot, D. (2016). Virtues of a hardworking role model to improve girls’ mathematics performance. *Psychology of Women Quarterly* 40, 55–64

Bailenson, J. N., Blascovich, J., Beall, A. C., and Loomis, J. M. (2001). Equilibrium theory revisited: Mutual gaze and personal space in virtual environments. *Presence: Teleoperators & Virtual Environments* 10, 583–598
586 [Dataset] Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.

587 Baylor, A. L. and Kim, Y. (2004). Pedagogical agent design: The impact of agent realism, gender, ethnicity, and instructional role. In *International Conference on Intelligent Tutoring Systems* (Springer), 592–603

588 Baylor, A. L. and Kim, Y. (2005). Simulating instructional roles through pedagogical agents. *International Journal of Artificial Intelligence in Education* 15, 95

591 Bem, S. L. (1974). The measurement of psychological androgyny. *Journal of Consulting and Clinical Psychology* 42, 155

593 Biocca, F. (1997). The cyborg’s dilemma: Progressive embodiment in virtual environments. *Journal of Computer-Mediated Communication* 3, JCMC324

595 Biocca, F., Harms, C., and Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: Review and suggested criteria. *Presence: Teleoperators & Virtual Environments* 12, 456–480

597 Biocca, F., Harms, C., and Gregg, J. (2001). The networked minds measure of social presence: Pilot test of the factor structure and concurrent validity. In *4th Annual International Workshop on Presence, Philadelphia, PA.* 1–9

600 Brahnam, S. and De Angeli, A. (2012). Gender affordances of conversational agents. *Interacting with Computers* 24, 139–153

602 Brahnam, S. and Weaver, M. (2015). Re/framing virtual conversational partners: A feminist critique and tentative move towards a new design paradigm. In *International Conference of Design, User Experience, and Usability* (Springer), 172–183

605 Buck, G. A., Clark, V. L. P., Leslie-Pelecky, D., Lu, Y., and Cerda-Lizarraga, P. (2008). Examining the cognitive processes used by adolescent girls and women scientists in identifying science role models: A feminist approach. *Science Education* 92, 688–707

608 Chang, F., Luo, M., Walton, G., Aguilar, L., and Bailenson, J. (2019). Stereotype threat in virtual learning environments: Effects of avatar gender and sexist behavior on women's math learning outcomes. *Cyberpsychology, Behavior, and Social Networking* 22, 634–640

611 Cloud-Buckner, J., Sellick, M., Sainathuni, B., Yang, B., and Gallimore, J. (2009). Expression of personality through avatars: Analysis of effects of gender and race on perceptions of personality. In *International Conference on Human-Computer Interaction* (Springer), 248–256

614 De Angeli, A. and Brahnam, S. (2006). Sex stereotypes and conversational agents. *Proc. of Gender and Interaction: Real and Virtual Women in a Male World, Venice, Italy*

616 Dill, K. E., Brown, B. P., and Collins, M. A. (2008). Effects of exposure to sex-stereotyped video game characters on tolerance of sexual harassment. *Journal of Experimental Social Psychology* 44, 1402–1408

618 Eagly, A. H., Nater, C., Miller, D. I., Kaufmann, M., and Sczesny, S. (2020). Gender stereotypes have changed: A cross-temporal meta-analysis of US public opinion polls from 1946 to 2018. *American Psychologist* 75, 301

621 Ellis, A. (1962). Reason and emotion in psychotherapy.

622 Esposito, A., Amorese, T., Cuciniello, M., Riviello, M. T., Esposito, A. M., Troncone, A., et al. (2021).

623 Elder user's attitude toward assistive virtual agents: the role of voice and gender. *Journal of Ambient Intelligence and Humanized Computing* 12, 4429–4436

625 Feng, D., Jeong, D. C., Krämer, N. C., Miller, L. C., and Marsella, S. (2017). "is it just me?": Evaluating attribution of negative feedback as a function of virtual instructor's gender and proxemics. In *Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Sys.* 810–818

628 Forgasz, H. J. and Leder, G. C. (1996). Mathematics classrooms, gender and affect. *Mathematics Education Research Journal* 8, 153–173
630 Fox, J. and Bailenson, J. N. (2009). Virtual virgins and vamps: The effects of exposure to female characters' sexualized appearance and gaze in an immersive virtual environment. *Sex Roles* 61, 147–157

631 Gratch, J., Wang, N., Gerten, J., Fast, E., and Duffy, R. (2007). Creating rapport with virtual agents. In *International Workshop on Intelligent Virtual Agents* (Springer), 125–138

632 Guadagno, R. E., Blascovich, J., Bailenson, J. N., and McCall, C. (2007). Virtual humans and persuasion: The effects of agency and behavioral realism. *Media Psychology* 10, 1–22

633 Guay, F., Vallerand, R. J., and Blanchard, C. (2000). On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). *Motivation and Emotion* 24, 175–213

634 Gulz, A., Haake, M., and Tärning, B. (2007). Visual gender and its motivational and cognitive effects–a user study. *Lund University Cognitive Studies* 137, 928–935

640 Hart, S. G. and Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In *Advances in Psychology* (Elsevier), vol. 52. 139–183

641 Hayes, A. L., Ulinski, A. C., and Hodges, L. F. (2010). That avatar is looking at me! Social inhibition in virtual worlds. In *International Conference on Intelligent Virtual Agents* (Springer), 454–467

642 Jeong, D. C., Feng, D., Krämer, N. C., Miller, L. C., and Marsella, S. (2017). Negative feedback in your face: Examining the effects of proxemics and gender on learning. In *International Conference on Intelligent Virtual Agents* (Springer), 170–183

643 Johnson, A. M., Ozogul, G., Moreno, R., and Reisslein, M. (2013). Pedagogical agent signaling of multiple visual engineering representations: The case of the young female agent. *Journal of Engineering Education* 102, 319–337

644 Kantharaju, R. B., De Franco, D., Pease, A., and Pelachaud, C. (2018). Is two better than one? Effects of multiple agents on user persuasion. In *Proceedings of the 18th International Conference on Intelligent Virtual Agents*. 255–262

645 Khashe, S., Lucas, G., Becerik-Gerber, B., and Gratch, J. (2017). Buildings with persona: Towards effective building-occupant communication. *Computers in Human Behavior* 75, 607–618

646 Kim, Y. (2013). Digital peers to help children’s text comprehension and perceptions. *Journal of Educational Technology & Society* 16, 59–70

647 Kim, Y. (2016). The role of agent age and gender for middle-grade girls. *Computers in the Schools* 33, 59–70

648 Kim, Y. and Baylor, A. L. (2006). A social-cognitive framework for pedagogical agents as learning companions. *Educational Technology Research and Development* 54, 569–596

649 Kim, Y. and Baylor, A. L. (2016). Research-based design of pedagogical agent roles: A review, progress, and recommendations. *International Journal of Artificial Intelligence in Education* 26, 160–169

650 Kim, Y., Baylor, A. L., and Shen, E. (2007). Pedagogical agents as learning companions: The impact of agent emotion and gender. *Journal of Computer Assisted Learning* 23, 220–234

651 Kim, Y. and Lim, J. H. (2013). Gendered socialization with an embodied agent: Creating a social and affable mathematics learning environment for middle-grade females. *Journal of Educational Psychology* 105, 1164

652 Kim, Y. and Wei, Q. (2011). The impact of learner attributes and learner choice in an agent-based environment. *Computers & Education* 56, 505–514

653 Krämer, N. C., Karacora, B., Lucas, G., Dehghani, M., Rüther, G., and Gratch, J. (2016). Closing the gender gap in with friendly male instructors? On the effects of rapport behavior and gender of a virtual agent in an instructional interaction. *Computers & Education* 99, 1–13

654 Kulms, P., Krämer, N. C., Gratch, J., and Kang, S.-H. (2011). It’s in their eyes: A study on female and male virtual humans’ gaze. In *International Workshop on Intelligent Virtual Agents* (Springer), 80–92
675 Lee, E.-J. (2003). Effects of “gender” of the computer on informational social influence: The moderating role of task type. *International Journal of Human-Computer Studies* 58, 347–362

677 Li, J., Kizilcec, R., Bailenson, J., and Ju, W. (2016). Social robots and virtual agents as lecturers for video instruction. *Computers in Human Behavior* 55, 1222–1230

679 Lindberg, S. M., Hyde, J., Petersen, J., and Linn, M. (2010). Gender similarities characterize math performance. *Psychological Bulletin* 136, 1123–1135

681 Lunardo, R., Bressolles, G., et al. (2016). The interacting effect of virtual agents’ gender and dressing style on attractiveness and subsequent consumer online behavior. *Journal of Retailing and Consumer Services* 30, 59–66

684 Makarova, E., Aeschlimann, B., and Herzog, W. (2019). The gender gap in fields: The impact of the gender stereotype of math and science on secondary students’ career aspirations. In *Frontiers in Education* (Frontiers), vol. 4, 60

687 Makransky, G., Wismer, P., and Mayer, R. E. (2019). A gender matching effect in learning with pedagogical agents in an immersive virtual reality science simulation. *Journal of Computer Assisted Learning* 35, 349–358

690 McDonnell, R., Jörg, S., Hodgins, J. K., Newell, F., and O’Sullivan, C. (2009). Evaluating the effect of motion and body shape on the perceived sex of virtual characters. *ACM Transactions on Applied Perception (TAP)* 5, 1–14

693 Mell, J., Lucas, G., and Gratch, J. (2017). Prestige questions, online agents, and gender-driven differences in disclosure. In *International Conference on Intelligent Virtual Agents* (Springer), 273–282

695 Mohtadi, M. T., Hajami, A., and Allali, H. (2014). Pedagogical agent for metacognitive scaffolding in interactive learning environments. In *2014 International Conference on Multimedia Computing and Systems (ICMCS) (IEEE)*, 652–656

698 Moreno, K. N., Person, N. K., Adcock, A. B., Eck, R., Jackson, G. T., and Marineau, J. C. (2002). Etiquette and efficacy in animated pedagogical agents: The role of stereotypes. In *AAAI Symposium on Personalized Agents, Cape Cod, MA*

701 Moreno, R. and Flowerday, T. (2006). Students’ choice of animated pedagogical agents in science learning: A test of the similarity-attraction hypothesis on gender and ethnicity. *Contemporary Educational Psychology* 31, 186–207

704 Mousas, C., Anastasiou, D., and Spantidi, O. (2018). The effects of appearance and motion of virtual characters on emotional reactivity. *Computers in Human Behavior* 86, 99–108

706 Nag, P. and Yalçın, Ö. N. (2020). Gender stereotypes in virtual agents. In *Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents*. 1–8

708 Nass, C. and Moon, Y. (2000). Machines and mindlessness: Social responses to computers. *Journal of Social Issues* 56, 81–103

710 Nass, C. I. and Brave, S. (2005). *Wired for speech: How voice activates and advances the human-computer relationship* (MIT Press Cambridge, MA)

712 Niculescu, A., Hofs, D., Van Dijk, B., and Nijholt, A. (2010). How the agent’s gender influence users’ evaluation of a qa sy. In *2010 International Conference on User Science and Engineering (i-USEr) (IEEE)*, 16–20

715 Niculescu, A., Van Der Sluis, F., and Nijholt, A. (2009). Feminity, masculinity and androgyny: How humans perceive the gender of anthropomorphic agents. In *Proceedings of 13th International Conference on Human-Computer Interaction, Springer Verlag, Heidelberg*. 628–632
718 Nunamaker, J. F., Derrick, D. C., Elkins, A. C., Burgoon, J. K., and Patton, M. W. (2011). Embodied conversational agent-based kiosk for automated interviewing. *Journal of Management Information Sys* 28, 17–48

719 Ozogul, G., Johnson, A. M., Atkinson, R. K., and Reisslein, M. (2013). Investigating the impact of pedagogical agent gender matching and learner choice on learning outcomes and perceptions. *Computers & Education* 67, 36–50

720 Pansu, P., Régner, I., Max, S., Colé, P., Nezlek, J. B., and Huguet, P. (2016). A burden for the boys: Evidence of stereotype threat in boys’ reading performance. *Journal of Experimental Social Psychology* 65, 26–30

721 Payne, J., Szymkowiak, A., Robertson, P., and Johnson, G. (2013). Gendering the machine: Preferred virtual assistant gender and realism in self-service. In *International Workshop on Intelligent Virtual Agents* (Springer), 106–115

722 Pecune, F., Cafaro, A., Ochs, M., and Pelachaud, C. (2016). Evaluating social attitudes of a virtual tutor. In *International Conference on Intelligent Virtual Agents* (Springer), 245–255

723 Pezzullo, L. G., Wiggins, J. B., Frankosky, M. H., Min, W., Boyer, K. E., Mott, B. W., et al. (2017). “Thanks Alisha, Keep in Touch”: Gender effects and engagement with virtual learning companions. In *International Conference on Artificial Intelligence in Education* (Springer), 299–310

724 Pfeifer, A. and Lugrin, B. (2018). Female robots as role-models? The influence of robot gender and learning materials on learning success. In *International Conference on Artificial Intelligence in Education* (Springer), 276–280

725 Plant, E. A., Baylor, A. L., Doerr, C. E., and Rosenberg-Kima, R. B. (2009). Changing middle-school students’ attitudes and performance regarding engineering with computer-based social models. *Computers & Education* 53, 209–215

726 Räty, H., Vänskä, J., Kasanen, K., and Kärkkäinen, R. (2002). Parents’ explanations of their child’s performance in mathematics and reading: A replication and extension of yee and eccles. *Sex Roles* 46, 121–128

727 Régner, I., Steele, J. R., Ambady, N., Thinus-Blanc, C., and Huguet, P. (2014). Our future scientists: A review of stereotype threat in girls from early elementary school to middle school. *Revue internationale de psychologie sociale* 27, 13–51

728 Richards, D., Alsharbi, B., and Abdulrahman, A. (2020). Can I help you? Preferences of young adults for the age, gender and ethnicity of a Virtual Support Person based on individual differences including personality and psychological state. In *Proceedings of the Australasian Computer Science Week Multiconference*. 1–10

729 Rosenberg-Kima, R., Baylor, A., Plant, E., and Doerr, C. (2008). Interface agents as social models for female students: The effects of agent visual presence and appearance on female students’ attitudes and beliefs. *Computers in Human Behavior* 24, 2741–2756

730 Rosenberg-Kima, R. B., Plant, E. A., Doerr, C. E., and Baylor, A. L. (2010). The influence of computer-based model’s race and gender on female students’ attitudes and beliefs towards engineering. *Journal of Engineering Education* 99, 35–44

731 Sajjadi, P., Zhao, J., Wallgrün, J. O., Furman, T., La Femina, P. C., Fatemi, A., et al. (2020). The effect of virtual agent gender and embodiment on the experiences and performance of students in virtual field trips. In *2020 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) (IEEE)*, 221–228

732 Sansone, D. (2019). Teacher characteristics, student beliefs, and the gender gap in STEM fields. *Educational Evaluation and Policy Analysis* 41, 127–144
763 Schmader, T. and Johns, M. (2003). Converging evidence that stereotype threat reduces working memory capacity. *Journal of Personality and Social Psychology* 85, 440
764
765 Schroeder, N. L., Adesope, O. O., and Gilbert, R. B. (2013). How effective are pedagogical agents for learning? A meta-analytic review. *Journal of Educational Computing Research* 49, 1–39
766
767 Schroeder, N. L., Romine, W. L., and Craig, S. D. (2017). Measuring pedagogical agent persona and the influence of agent persona on learning. *Computers & Education* 109, 176–186
768
769 Sczesny, S., Nater, C., and Eagly, A. H. (2018). Agency and communion: Their implications for gender stereotypes and gender identities. In *Agency and Communion in Social Psychology* (Routledge). 103–116
770
771 Shiban, Y., Schelhorn, I., Jobst, V., Hörnlein, A., Puppe, F., Pauli, P., et al. (2015). The appearance effect: Influences of virtual agent features on performance and motivation. *Computers in Human Behavior* 49, 5–11
772
773
774 Silvervarg, A., Haake, M., and Gulz, A. (2013). Educational potentials in visually androgynous pedagogical agents. In *International Conference on Artificial Intelligence in Education* (Springer), 599–602
775
776 Spencer, S. J., Steele, C. M., and Quinn, D. M. (1999). Stereotype threat and women's math performance. *Journal of Experimental Social Psychology* 35, 4–28
777
778 Spilioto-poulos, D., Makri, E., Vassilakis, C., and Margaris, D. (2020). Multimodal interaction: Correlates of learners' metacognitive skill training negotiation experience. *Information* 11, 381
779
780 Steele, C. M. and Aronson, J. (1995). Stereotype threat and the intellectual test performance of african americans. *Journal of Personality and Social Psychology* 69, 797
781
782 Stokes, J., Riger, S., and Sullivan, M. (1995). Measuring perceptions of the working environment for women in corporate settings. *Psychology of Women Quarterly* 19, 533–549
783
784 Świdrak, J., Pochwatko, G., and Insabato, A. (2021). Does an agent's touch always matter? Study on virtual Midas touch, masculinity, social status, and compliance in Polish men. *Journal on Multimodal User Interfaces* 15, 163–174
785
786
787 ter Stal, S., Tabak, M., op den Akker, H., Beinema, T., and Hermens, H. (2020). Who do you prefer? The effect of age, gender and role on users' first impressions of embodied conversational agents in ehealth. *International Journal of Human-Computer Interaction* 36, 881–892
788
789
790 Tickle-Degnen, L. and Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. *Psychological Inquiry* 1, 285–293
791
792 van der Lubbe, L. M. and Bosse, T. (2017). Studying gender bias and social backlash via simulated negotiations with virtual agents. In *International Conference on Intelligent Virtual Agents* (Springer), 455–458
793
794
795 Vilaro, M. J., Wilson-Howard, D. S., Neil, J. M., Tavassoli, F., Zalake, M. S., Lok, B. C., et al. (2021). A subjective culture approach to cancer prevention: Rural black and white adults' perceptions of using virtual health assistants to promote colorectal cancer screening. *Health Communication*, 1–12
796
797
798 Webster, J. and Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. *MIS Quarterly*, xiii–xxiii
799
800 West, M., Kraut, R., and Ei Chew, H. (2019). I'd blush if I could: Closing gender divides in digital skills through education
801
802 Wirzberger, M., Schmidt, R., Georgi, M., Hardt, W., Brunnett, G., and Rey, G. D. (2019). Effects of system response delays on elderly humans' cognitive performance in a virtual training scenario. *Scientific Reports* 9, 1–12
803
804
805 Yee, D. K. and Eccles, J. S. (1988). Parent perceptions and attributions for children's math achievement. *Sex Roles* 19, 317–333
806
807 Zanbaka, C., Goolkasian, P., and Hodges, L. (2006). Can a virtual cat persuade you? The role of gender

808 and realism in speaker persuasiveness. In *Proceedings of the SIGCHI Conference on Human Factors in*

809 *Computing Systems*. 1153–1162

810 Zibrek, K., Niay, B., Olivier, A.-H., Hoyet, L., Pettré, J., and Mcdonnell, R. (2020). The effect of gender

811 and attractiveness of motion on proximity in virtual reality. *ACM Transactions on Applied Perception*

812 (*TAP*) 17, 1–15