# Horses discriminate between human facial and vocal expressions of sadness and joy

Plotine Jardat, Océane Liehrmann, Fabrice Reigner, Céline Parias, Ludovic

Calandreau, Léa Lansade

► To cite this version:

Plotine Jardat, Océane Liehrmann, Fabrice Reigner, Céline Parias, Ludovic Calandreau, et al. Horses discriminate between human facial and vocal expressions of sadness and joy. Animal Cognition, 2023, 26 (5), pp.1733-1742. 10.1007/s10071-023-01817-7. hal-04213124

HAL Id: hal-04213124

https://hal.science/hal-04213124v1

Submitted on 21 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
1 Horses discriminate between human facial and vocal
2 expressions of sadness and joy
3

4 Plotine Jardat¹*, Océane Liehrmann², Fabrice Reigner³, Céline Parias¹, Ludovic Calandreau¹, Léa

5 Lansade¹,*

6 ¹CNRS, IFCE, INRAE, Université de Tours, PRC, 37380 Nouzilly, France

7 ²Department of Biology, University of Turku, 20500 Turku, Finland

8 ³UEPAO, INRAE, 37380 Nouzilly, France

9 ORCIDs: Plotine Jardat 0000-0003-0374-5588; Océane Liehrmann 0000-0001-5390-8985; Ludovic
10 Calandreau 0000-0002-7535-5733; Léa Lansade 0000-0002-4185-9714

11 *Corresponding authors: Plotine Jardat – [plotine.jardat@gmail.com](mailto:plotine.jardat@gmail.com), Léa Lansade –

12 [lea.lansade@inrae.fr](mailto:lea.lansade@inrae.fr)
13

Abstract

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

Communication of emotions plays a key role in intraspecific social interactions and likely in interspecific interactions. Several studies have shown that animals perceive human joy and anger, but few studies have examined other human emotions, such as sadness. In this study, we conducted a cross-modal experiment, in which we showed 28 horses two soundless videos simultaneously, one showing a sad, and one a joyful human face. These were accompanied by either a sad or joyful voice. The number of horses whose first look to the video that was incongruent with the voice was longer than their first look to the congruent video was higher than chance, suggesting that horses could form cross-modal representations of human joy and sadness. Moreover, horses were more attentive to the videos of joy and looked at them for longer, more frequently, and more rapidly than the videos of sadness. Their heart rates tended to increase when they heard joy and to decrease when they heard sadness. These results show that horses are able to discriminate facial and vocal expressions of joy and sadness and may form cross-modal representations of these emotions; they also are more attracted to joyful faces than to sad faces and seem to be more aroused by a joyful voice than a sad voice. Further studies are needed to better understand how horses perceive the range of human emotions, and we propose that future experiments include neutral stimuli as well as emotions with different arousal levels but a same valence.

30

**Key words:** *Equus caballus*, human-animal relationship, cognition, emotion, interspecific communication

31

<figure><img src="image_2.png" /></figure>
32 Introduction

33 An emotion is a brief affective response to an event, which allows an individual to adapt to a change
34 in his or her environment (Désiré et al. 2002). Communicating emotions enables an individual to
35 provide information to other individuals, such that they can also adapt to the related environmental
36 change (Désiré et al. 2002; Špinka 2012). Communication of emotions can occur between individuals
37 of different species; indeed, diverse nonhuman animal species have been found to perceive human
38 emotional expressions. For example, orangutans differed in the duration of their looks toward pictures
39 of humans expressing fear or anger compared to those toward pictures of neutral faces (Pritsch et al.
40 2017), and pigeons learned to discriminate between human facial expressions of anger and joy
41 (Jitsumori and Yoshihara 1997). Studies on dogs, horses, goats and cats, focusing mainly on expressions
42 of joy and anger, have shown that these species are sensitive to human facial and vocal expressions of
43 emotions (Jardat and Lansade 2022).

44 Regarding facial expressions, dogs learned to differentiate between a smiling face and a blank
45 expression (Nagasawa et al. 2011), and they licked their mouth (a behavior usually observed in stressful
46 situations - Albuquerque et al. 2018) less when seeing a joyful face than an angry face. Dogs also
47 reacted more quickly and for longer to human facial expressions of anger or fear compared to those
48 of sadness, disgust, surprise or joy (Siniscalchi et al. 2018b). Cats spent more time close to their owner
49 and expressed more positive behaviors when they saw a person displaying joy rather than anger
50 (Galvan and Vonk 2016), and goats preferred to spend time close to pictures of humans expressing joy
51 rather than anger (Nawroth et al. 2018). In horses, a left gaze bias (generally associated with negative
52 stimuli - Siniscalchi et al. 2018b) and a quicker increase in heart rate were observed when they viewed
53 pictures of angry faces compared to joyful faces (Smith et al. 2016). When horses faced a person they
54 had previously seen smiling in a picture, they reacted differently than when they faced a person they
55 had previously seen expressing anger in a picture (Proops et al. 2018). Horses also reacted to a facial
56 expression of disgust: they followed an experimenter's gaze less in response to this expression than a
57 neutral expression (Baba et al. 2019).

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons
Attribution | 4.0 International license
Vocal expressions of joy also seem to provoke different reactions than those of anger in domestic mammals. Horses expressed more vigilance behaviors and their heart rates increased to a greater extent when they heard human vocalizations of anger compared to those of joy (Smith et al. 2018; Trösch et al. 2019). Cats expressed more stress-related behaviors upon hearing angry voices compared to joyful voices (Quaranta et al. 2020). In dogs, a greater increase in heart rate and greater reactivity were observed in response to vocalizations of anger compared to those of joy, fear or sadness (Siniscalchi et al. 2018a).

In addition to being sensitive to human facial and vocal expressions of anger and joy, domestic mammals can associate facial and vocal expressions of these emotions. In cross-modal experiments, horses, dogs and cats were presented with human faces expressing anger or joy accompanied by a joyful or angry voice. These animals looked differently at the pictures according to their correspondence with the voice (Albuquerque et al. 2016; Nakamura et al. 2018; Quaranta et al. 2020; Trösch et al. 2019), suggesting that they integrated these signals of human emotions across modalities. More specifically, in experiments with dogs and cats and in an experiment with horses, a preferential looking paradigm was used: two images were presented to the animals while one voice was broadcast. Dogs and cats looked more at the congruent image (i.e., the one that matched the sound - Albuquerque et al. 2016; Quaranta et al. 2020) whereas horses looked more at the incongruent image (i.e., the one that did not match the sound - Trösch et al. 2019). In another experiment on horses, an expectancy violation paradigm was used: horses saw a picture of an angry or joyful face, followed by a joyful or angry voice. Horses looked longer at the speaker when the voice was incongruent with the face (Nakamura et al. 2018). Thus, different responses to cross-modal stimuli were observed according to the paradigm used and the species, with horses typically looking for longer durations at the incongruent image (Jardat et al. 2023; Lampe and Andre 2012; Proops and Mccomb 2012; Trösch et al. 2019).

<figure><img src="image_4.png" /></figure>
Overall, it is clear that horses, dogs, goats and cats are sensitive to human facial and vocal expressions of anger and joy, two emotions characterized by high arousal levels (Mendl et al. 2010). It remains unclear to what extent animals are sensitive to other human emotions, and if they can also perceive and react to emotions with lower arousal levels. In this study, we focused on sadness, a low-arousal, negative-valence emotion; the perception of human sadness by domestic mammals has been described in only a few studies on dogs. Dogs approached a human more when the person was pretending to cry rather than when the person was talking or humming (Custance and Mayer 2012), and they seemed to express more behaviors associated with negative states when hearing vocal expressions of sadness compared to those of joy (Huber et al. 2017). Moreover, viewing sad faces provoked differences in heart rate variations and stress levels compared to viewing scared, joyful, surprised or disgusted faces (Siniscalchi et al. 2018b). Similarly, when dogs heard sad voices, differences in heart rate variations, lateralized behaviors and stress-related behaviors were observed compared to when they heard voices expressing fear, anger or disgust (Siniscalchi et al. 2018a). To date, it remains unclear whether horses react to human expressions of sadness. Sociological studies have indicated that horse owners find emotional support in their horses, such as when feeling sad (Keaveney 2008); therefore, it would be interesting to determine whether horses indeed react to this emotional state.

We conducted a cross-modal experiment using a preferential looking paradigm. Two silent videos of the same woman expressing joy and sadness were simultaneously projected while the voice of another woman expressing joy or sadness was played. The purpose of the present study was to examine the following research questions:

(1) Whether horses associate sad and joyful voices with the corresponding facial expressions. We expected that horses would associate the broadcast voices with the corresponding facial expressions by looking longer at the incongruent image.

<figure><img src="image_5.png" /></figure>
106 (2) Whether horses have different behavioral and physiological reactions to human facial and vocal expressions of sadness compared to joy. Horses were expected to look at the sad videos for a shorter duration, and to show a lower increase in heart rate when hearing sadness compared to joy.
107
108

109 Methods

110 Subjects

111 The study included 36 Welsh mares (*Equus caballus*) aged 7.9 ± 2.4 years (mean ± SD) reared at the Animal Physiology Experimental Unit PAO (UEPAO, 37380 Nouzilly, France, DOI: 10.15454/1.55738963217 28955E12), INRAE. These mares lived in groups in indoor stalls (3-sided shelter, open on one side) bedded with straw, with environmental enrichments and access to an outdoor area 12h a day. Fodder and water were available ad libitum. In this facility, a general health status check is performed twice a day by keepers and a more extensive clinical exam is performed at least once a year by a vet. No hearing or vision issues have been reported in the horses included in this study. These horses are used only for research purposes and are handled daily by humans. Thus, they have the opportunity to experience human emotions including joy and sadness expressed by caregivers and researchers.
112
113
114
115
116
117
118
119
120

121 Experimental setup

122 The experimental setup was similar to that used to investigate cross-modal recognition of human anger and joy by horses as well as cross-modal recognition of adults and children (Jardat et al. 2023; Trösch et al. 2019). A horse was placed in the middle of a stall and attached with two loose ropes (Fig 1). 123 124 Videos were projected on two projection screens (1 × 2 m) placed in front of the horse on the right and left. For safety reasons, an assistant stayed with the horse to ensure that she did not panic or become entangled in the ropes. The assistant stood along the wall at the level of the horse; on the right of the horse for half the horses and on the left for the other half of the horses. They never interacted with the horse during the tests but remained still with their head down. They were instructed to stop the test if a horse panicked; no horses panicked during the experiment. The experiment was filmed by two cameras, one in front of each screen. The horses were equipped with a 125
126
127
128
129
130
131

<figure><img src="image_6.png" /></figure>
132 heart monitor system composed of a sensor placed on the horse and a wristwatch showing real-time
133 heart rate values and recording beat-to-beat (RR) values (Polar Equine 149 RS800CX Science, Polar Oy,
134 Finland). An overview camera (GoPro Hero Black) allowed the experimenter to follow the experiment
135 from outside the stall and control the projected videos and the sounds accordingly.

<caption>**Fig 1 Experimental setup.**</caption>

Two simultaneous videos of a
same woman expressing joy
and sadness were projected
while the voice of another
woman speaking in a joyful
or sad tone was broadcast.
The videos and voices came
from a validated
psychological database. To
respect the copyrights of this
database, different images
are shown here for
illustrative purposes.

<figure><img src="image_7.png" /></figure>

136 Stimuli

137 The videos and voices were extracted from the GEMEP Core Set database (faces: 02sad, 02joy, 07sad, 07joy, 09sad and 09joy; voices: 02joy, 02sad, 06joy, 07joy, 07sad and 10joy - Bänziger et al. 2012), a validated psychological database for experimental research on emotion perception. The horses were 138 therefore not familiar with either the faces or the voices. Pairs of videos of the same woman expressing 139 sadness and joy were projected simultaneously, one on each projection screen, while the voice of 140 another woman expressing sadness or joy was played over a loudspeaker placed between the 141 projection screens (Fig 1). The faces were projected at a height of approximately 160 cm and were 142 approximately twice the size of a real human face (as in Jardat et al. 2023; Trösch et al. 2019). The 143 videos shown during the tests lasted 115 s and were divided into six sections of 15 s separated by 5 s 144 intervals of black screen (Fig 2). Each section was composed of sadness and joy videos of the same 145 146

<figure><img src="image_8.png" /></figure>
woman (presented simultaneously) accompanied by a voice, repeated seven times. Four different videos were generated. In each video, three women appeared twice each (once on the right side and once on the left side), in different orders. The side of the congruent video (i.e., the video that matched the emotion of the voice) was semi-randomly distributed. The side that an emotion was shown on and the emotion expressed by the voice were also semi-randomly distributed. The horses were randomly assigned to view one of the four videos, such that the order in which the women appeared and the side on which each woman expressed joy and sadness were counterbalanced among the horses.

<figure><img src="image_9.png" /><figcaption>**Fig 2** Example of stimuli shown to a horse. Videos of three different women expressing sadness or joy were presented, with one emotion on each side. The order in which the women appeared and the side of the sad and joyful expressions of each woman were counterbalanced among the horses. The videos and voices came from a validated psychological database. To respect the copyrights of this database, different images are shown here for illustrative purposes.</figcaption></figure>

155 Habituation phase

During the habituation phase, the horse was led to the middle of the stall facing the screen. She was attached loosely to two ropes, the assistant took their position, and the habituation phase began. Identical scenes of nature were projected on both screens, while the assistant monitored the horse's heart rate on the Polar watch. If the horse moved such that her body formed an angle of less than 45° with one of the ropes, the assistant gently repositioned her to face the wall in the middle of the two projection screens. The criteria to consider that the horse was habituated to the setup were that the horse had to be calm (not neighing, pulling on the ropes, or trying to turn around and leave) and her heart rate had to have stayed below 80 bpm for two consecutive minutes (corresponding to a moderate physical effort or psychological stress - Gouyet et al. 2023; Trösch et al. 2019; Visser et al. 2002; von Lewinski et al. 2013). These criteria assured that the situation was not too aversive for the

<figure><img src="image_10.png" /></figure>
horse and that she was ready to pay attention to the stimuli. Once these criteria were met, the test phase began immediately. If these criteria were not met after five minutes, the session ended, and a new session was scheduled for the next day. Of the 36 horses initially included in the experiment, 21 met the criteria on day 1 and therefore continued with the test phase on the same day. Seven horses met the criteria on day 2 and continued with the test phase on that day. The remaining 8 horses did not meet the criteria on day 1 or day 2, indicating this setup was too aversive for these individuals. Therefore, these horses were excluded from the experiment at this stage. The 28 horses that successfully habituated moved forward to the test phase.

## Test phase

Immediately after the horse met the habituation criteria, videos of women expressing sadness and joy were projected (see the Stimuli section). The conditions were the same as those during the habituation phase. If the assistant needed to reposition the horse, the experiment was paused during a black screen interval and resumed once the assistant had repositioned the horse and was back in place. This was a rare event that occurred only five times during the whole experiment (in one trial for two horses and three trials for another horse, over a total of 168 trials for all horses). The assistant was unaware of the side on which each video appeared. At the end of the test, the horse was led directly back to its stall.

## Behavioral and physiological analyses

The videos of the test phase were analyzed with BORIS software (Friard and Gamba 2016) by a first coder. The videos did not contain sound and the screens were not visible in the videos, so the coder was blind to the side on which each emotion appeared and the location of the congruent video. A horse was considered to look at the right screen when her left eye was visible from the camera placed under the right screen, and vice versa. She was considered to be attentive to the right screen when her right eye could also be seen from the camera placed under the right screen (i.e., both eyes visible from this camera), with both of her ears oriented forward; and vice versa. For each section, the total

<figure><img src="image_11.png" /></figure>
looking duration and duration of being attentive to each video, the number of looks and the duration of the first look toward each video, and the latency between the start of a section and the first look toward each video were quantified. Thirty percent of the videos (9 individuals) were analyzed again by a second coder to evaluate scoring reliability. Interclass correlation coefficients (ICC) and their 95% confidence intervals were calculated and appreciated according to Koo and Li's method (2016), showing good to excellent reliability for three variables and moderate variability for two variables (duration of being attentive: 0.91[0.86,0.93] – excellent, looking duration: 0.82[0.75,0.88] – good, latency of the first look: 0.83[0.76,0.88] – good, duration of the first look: 0.70[0.59,0.78] – moderate, number of looks: 0.62[0.50,0.73] – moderate). As all the variables showed a same trend (all differed significantly according to the emotion, in the same direction – see **Results**), we decided to keep them all in the statistical analysis.

RR data were extracted from the Polar recordings. A visual correction was applied to eliminate artifactual beats (as recommended by von Borell et al. 2007, see Supplementary Information (SI) – Fig S1). RR values were converted to heart rate values and the difference in heart rate (in beats per minute—bpm) between the last and first 5 s of each section was calculated. Data were missing for three individuals due to technical issues with the heart rate monitor. Therefore, the heart-rate analysis included data from 25 individuals.

## Statistical analyses

All statistical analyses were performed using R 4.1.2 (R Core Team 2021), and figures were created using the packages *ggplot2* (Wickham 2016), *ggpubr* (Kassambara 2020) and *survminer* (Kassambara et al. 2021). Significant differences were considered at *p* ≤ 0.05.

## Behavioral analysis

The durations of looking at and of being attentive to each video, the number of looks, and the duration of the first look toward each video were analyzed according to the congruence of the video with generalized linear mixed models (GLMMs) from the package *glmmTMB* (Brooks et al. 2017), using

<figure><img src="image_12.png" /></figure>
linear, Tweedie and Poisson distributions as appropriate for the given variable (see Table 1). The latency to look at the video was investigated according to the congruence of the video with the sound, and to the emotion displayed in the video, using survival analyses from the package *coxme* (Therneau 2022). This allowed us to include in the analysis horses that did not look at one of the videos at all during a section.

For each of the five response variables, an initial model was constructed to assess the interaction effect of video-sound congruence with the emotion displayed in the video. The side of the watched video was added to control for potential side biases. Horse identity was added as a random effect to account for individual variation in paired data, as each horse was tested six times. Distributions, within-group variance and homoscedasticity of the residuals for the GLMM were checked using the package *DHARMa* (Hartig 2021). For the survival analysis, outliers, the linear assumption and the proportionality assumption were evaluated graphically. If a horse did not look at one of the videos during the test, the latency to look at this video was set as censored (this happened 56 times). The best version of each model was selected according to model comparison via two-tailed ANOVA with the null model and simpler models (without the interaction, then without each variable of interest). The selected models are presented in Table 1 (see SI - Table S1 for the detailed results of each ANOVA).

<table><caption>**Table 1: Results of the model selections for each variable.** Detailed results of ANOVA for the selection of each model are provided in SI - Table S1.</caption><thead><tr><th>Response variable (y)</th><th>Model type</th><th>Family</th><th>Selected formula</th><th>X<sup>2</sup></th><th>DF</th><th>p value</th></tr></thead><tbody><tr><td>Duration attentive to video</td><td>GLMM<br>N=28</td><td>Tweedie</td><td>y ~ congruence + emotion</td><td>27.0</td><td>1</td><td>&lt;0.001***</td></tr><tr><td>Duration looking at video</td><td>×6 sections<br>×2 sides</td><td>Gaussian</td><td>y ~ congruence + emotion</td><td>21.6</td><td>1</td><td>&lt;0.001***</td></tr><tr><td>Duration of first look</td><td></td><td>Tweedie</td><td>y ~ congruence + emotion</td><td>13.9</td><td>1</td><td>&lt;0.001***</td></tr><tr><td>Number of looks</td><td></td><td>Poisson</td><td>y ~ congruence + emotion</td><td>6.9</td><td>1</td><td>0.009**</td></tr><tr><td>Latency of first look</td><td>Coxme<br>N=28×6 sections×2 sides<br>Censored = 56</td><td></td><td>(y, censor) ~ congruence + emotion</td><td>7.0</td><td>1</td><td>0.008**</td></tr><tr><td>Heart rate variations</td><td>GLMM<br>N=28<br>×6 sections</td><td>Generalized<br>Poisson</td><td>y ~ emotion</td><td>2.88</td><td>1</td><td>0.09°</td></tr></tbody></table>

<figure><img src="image_13.png" /></figure>
235

236

237

238

239

Additionally, outliers were identified using the boxplot method and the GLMM analyses were repeated (Aguinis et al. 2013), showing they did not influence the results (see SI – Table S2). Moreover, the influence of the experimenter manipulations on a few horses (see Methods) was checked, revealing it did not influence the results (see SI – Table S3).

240

241

242

243

244

245

The duration of the first look toward each video was also analyzed using a binomial transformation. For each horse, the variable was set at 1 if the first look (in the first section) toward the incongruent video was longer in duration than the first look (in the first section) toward the congruent video, and was set at 0 if it was not. The number of horses that looked more at the incongruent video was compared to chance using a one-tailed binomial test (*binom.test* with p=0.5 and n=28).

246

### Heart-rate analysis

247

248

249

250

251

252

253

Heart-rate variation was investigated with GLMMs from the package *glmmTMB* (Brooks et al. 2017), using a generalized Poisson distribution (*family=genpois*). A model was constructed to assess the effect of the voice heard. Horse identity was added as a random effect to account for individual variation in paired data, as each horse was tested six times. Distributions, within-group variance and homoscedasticity of the residuals were checked using the package *DHARMa* (Hartig 2021). The model was selected according to model comparison via two-tailed ANOVA with the null model (Table 1, see SI - Table S4 for the detailed results of the ANOVA).

254

## Results

255

256

257

Binomial analysis revealed that the number of horses whose first look toward the incongruent video was longer than the first look toward the congruent video was higher than expected by chance (Fig 3, one-tailed binomial test, *p*=0.04).

<figure><img src="image_14.png" /></figure>
258

This effect was absent when analyzing horse behaviors from the whole experiment.
Generalized linear mixed models (GLMMs) and survival analyses did not reveal a significant influence
of the video congruence with the sound on the attention and looking behavior of horses (duration
spent being attentive: Z=0.4, p=0.6; looking duration: Z=0.6, p=0.5; duration of the first look: Z=1.3,
p=0.2, number of looks: -0.3, p=0.7; latency to look: Z=-0.2, p=0.9).

259

260

261

262

263

GLMMs revealed that the time horses spent being attentive to the video, the duration looking
at the video, the duration of their first look, the number of looks and the latency to look at the video
were all significantly influenced by the emotion displayed by the humans in the video (selected models
are indicated in Table 1). Horses were more attentive, looked for a longer duration, had a longer first
look and a higher number of looks toward the joyful video compared to the sad video (Fig 4a-d,
duration being attentive: Z=-5.23, p<0.001; duration spent looking, Z=-4.72, p<0.001; duration of the
first look: Z=-3.75, p<0.001, number of looks: -2.61, p=0.009, see SI - Table S5 for model estimates).
Survival analysis revealed that horses looked significantly faster to the joyful video than the sad video
(Fig 4e, latency to look, Z=-2.65, p=0.008).

264

265

266

267

268

269

270

271

272

The side of the watched video was never included in the selected models (i.e., the models
selected after applying the selection criteria), which indicated that the side of the watched video did
not explain the variation in the response variables and that horses did not exhibit a side bias (Table 1,
SI - Table S1).

273

274

275

276

The variation in heart rate approached significance and tended to be higher when hearing
joyful voices than when hearing sad voices (Fig S2, Z=-1.7, p=0.09, see SI - Table S3 for model
estimates).

277

278

279

280

<figure><img src="image_15.png" /></figure>
<figure><img src="image_16.png" /></figure>

**Fig 3 Horses' first look was longer toward the incongruent video.** The number of horses whose first look toward the incongruent video was longer than their first look toward the congruent video (19 horses) was higher than chance (half of the horses, represented by the dashed line). *: p≤0.05, unilateral binomial test, n=28.

## Discussion

In this study, horses seemed to associate voices expressing sadness or joy with the corresponding facial expressions, at least in their first looks toward the videos. Indeed, more horses than expected by chance exhibited a longer duration of the first look toward the incongruent video than the first look toward the congruent video. Moreover, horses had a lower latency to look, looked a higher number of times and looked for longer durations at the videos of joyful faces than those of sad faces. Horses also showed different physiological reactions to the human voices expressing emotions, with heart-rate variations tending to differ when they heard joyful voices compared to sad voices.

The significant number of horses whose first look toward the incongruent video was longer than their first look toward the congruent video is consistent with the findings of previous cross-modal experiments in horses (Jardat et al. 2023; Nakamura et al. 2018; Trösch et al. 2019). In other words, horses' first look toward the videos was preferentially directed toward the video that did not correspond to the sound heard. This finding suggests that when first hearing the voice, horses were
a

<figure><img src="image_17.png" /></figure>

b

<figure><img src="image_18.png" /></figure>

c

<figure><img src="image_19.png" /></figure>

d

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

**Fig 4 Horses looked more at the joyful video.** Attention and looking behavior of horses toward the joy and sadness videos. (a) Duration attentive to each video during a video section. (b) Looking duration to each video. (c) Number of looks toward each video. (d) Duration of the first look toward each video. (e) Survival analysis for the latency of the first looks toward each video. The figures were generated from the corresponding models presented in Table 1. The error bars represent the standard errors from the models. ***: p≤0.001.
surprised by the incongruency with the observed emotional face, indicating that they associated the facial and vocal stimuli that expressed the same emotion. This result may indicate that horses form cross-modal representations of joy and sadness in which vocal and facial features are associated (Albuquerque et al. 2016; Jardat et al. 2023; Quaranta et al. 2020). This preference for the incongruent video was not observed further in the test, which could be explained by horses' strong preference for the joyful face after first appraising both videos, as we discuss below.

Horses looking behavior toward the videos showed that they looked faster and for a longer duration at videos displaying joy than sadness. Other experiments with domestic mammals have shown that horses, dogs, cats and goats react more positively (e.g., longer looks and less stress-related behaviors) to human facial expressions of joy compared to those of anger (Albuquerque et al. 2018; Galvan and Vonk 2016; Nawroth et al. 2018; Smith et al. 2016, 2018). On the physiological level, we measured heart rates to assess arousal (Briefer 2018; Jardat et al. 2022). A change in heart rate can be attributed either to an increased stimulation from the sympathetic system or a decreased stimulation from the parasympathetic system, which are both affected by the arousal level of the horse (Patteson 1996). Thus, the differences in heart rate variations according to the emotion heard suggest that the arousal level of horses tended to increase when they heard joy, a high-arousal emotion, and to decrease when they heard sadness, a low-arousal emotion (Mendl et al. 2010). This suggests that the arousal levels of horses can be influenced by the arousal levels of human expressions they hear. Similarly, previous studies reported higher or increasing heart rates in horses upon seeing or hearing expressions of anger compared to those of joy (Siniscalchi et al. 2018a; Smith et al. 2016, 2018; Trösch et al. 2019).

Horses' longer, faster and more frequent looks toward faces expressing joy compared to those expressing sadness could be due to at least three mechanisms. (1) The most parsimonious hypothesis is that horses reacted to the visual and acoustic properties of the stimuli: they may have been more attracted to the joy videos because of the greater movement in these videos, and they may have been

<figure><img src="image_22.png" /></figure>
more aroused by the joyful voices because of the higher pitch and greater pitch variations in these recordings. (2) Alternatively, horse reactions could stem from associations formed by horses during interactions with humans; horses may have observed humans expressing joy in more positive and arousing contexts compared to the contexts in which they observed humans expressing sadness. (3) Finally, emotional contagion could take place between humans and horses; that is, horses could match the perceived emotional state of humans in an unconditioned way (Pérez-Manrique and Gomila 2022). Joy is a more positive and higher arousal emotion than sadness (Kremer et al. 2020; Mendl et al. 2010); thus, a joyful face may induce more positive emotions in horses, and horses could prefer to look at joyful faces compared to sad faces. However, it is not known whether horses experience sadness themselves. We are unable to evaluate the subjective feelings (e.g., affect) of animals; moreover sadness is particularly seldom cited as an emotion that animals likely experience, in contrast to other emotions such as fear (or an alarmed state), that have a clearer link to adaptative benefits of avoiding sources of danger (e.g., Corujo et al. 2021; Désiré et al. 2002; Hall et al. 2018; Kremer et al. 2020; Špinka 2012). Sadness was not discussed in foundational texts on animal emotions (e.g., Darwin 1872), and current studies on emotional states of horses do not list low-arousal negative-valence states (Corujo et al. 2021). This may be explained by the difficulty of detecting sadness in animals. Indeed, low-arousal states are more difficult to characterize than high-arousal states (Kremer et al. 2020), and among low-arousal states, differentiating positive and negative valences can be challenging (Fureix and Meagher 2015). Moreover, for prey animals such as horses, expressing emotions suggestive of vulnerability, such as sadness, can be disadvantageous (Hall et al. 2018). However, sadness or low-arousal negative-valence states such as boredom, anhedonia or inactivity have been described in studies of owner and caretaker perceptions of equine emotional states (Hötzel et al. 2019) as well as studies based on behavioral observations (Fureix et al. 2015; Fureix and Meagher 2015), e.g., of horses housed in individual boxes (Ruet et al. 2019). In addition, the proposed adaptative role of sadness to encourage the formation of social bounds (Panksepp 1998) could be relevant for social species such as horses (VanDierendonck and Goodwin 2005). Thus, it is possible that horses experience emotional states

<figure><img src="image_23.png" /></figure>
related to sadness. The potential recognition of human sadness by horses could, moreover, contribute to the human-horse relationship by facilitating interactions and the formation of social bonds.

This study has several limitations and opens up new research avenues. First, there was no neutral control stimuli in this study, which limits our conclusions to comparisons between joy and sadness. In future studies, comparing horses' reactions to human expressions of joy or sadness to their reaction to neutral expressions could help further describe how horses perceive each of these emotional states in humans. It would also be interesting to investigate whether horses can discriminate human emotional states with different arousal levels but a same valence, for example a joyful state (positive, high-arousal) compared to a calm/happy state (positive, low-arousal). Indeed, both our results and previous studies on anger and joy (Proops et al. 2018; Smith et al. 2018, 2016) indicate a preference for happy (positive, high-arousal) expressions; and understanding whether horses also discriminate between a joyful and a calm/happy state in humans could be helpful for both welfare and practical applications by riders and handlers. Last, further studies could explore the possible emotional contagion of human-expressed sadness to horses. In response to human expressions of sadness, horses' emotional valence as well as changes in behavior, especially during human-horse interactions, could be evaluated (Adriaense et al. 2020).

In conclusion, in this study, we showed that horses seem to associate human vocal and facial expressions of sadness and joy cross-modally. Moreover, horses were more attracted to facial expressions of joy than sadness and they seemed to be more aroused by human voices expressing joy than those expressing sadness. These findings indicate that horse owners, riders and caretakers could benefit from expressing joy when interacting with horses. The related mechanism could be emotional contagion from humans to horses, a phenomenon often regarded as a premise of empathy (Adriaense et al. 2020).

<figure><img src="image_24.png" /></figure>
369 Declarations

370 Ethical Approval

371 This study was approved by the Val-de-Loire ethical committee – authorization number CE19 2022-

372 1003. French and European guidelines for the care and use of animals were followed.

373 Competing Interests

374 The authors have no competing interests to declare that are relevant to the content of this article.

375 Authors' contributions

376 All authors devised the protocol. PJ, CP, FR, and LL implemented the protocol. PJ, CP, and LL coded

377 the videos and analyzed the heart-rate and behavioral data. PJ, OL and LL performed the statistical

378 analysis. PJ, OL, LC, and LL revised the analysis and report.

379 Funding

380 This study was funded by the French Horse and Riding Institute, grant number 32 001176-Cognition

381 Equine. Plotine Jardat is funded by the French Ministry of Research.

382 Availability of data and materials

383 Data and code generated and analyzed in this study have been deposited at

384 https://doi.org/10.17632/b4w6555j46.1 and are publicly available as of the date of publication.

385 Acknowledgements

386 We would like to thank the staff from the UEPAO (Unité Expérimentale de Physiologie Animale de

387 l'Orfrasière) for technical help; and the CISA (Centre Interfacultaire en Sciences Affectives) for

388 allowing us to use the GEMEP Core Set. We also thank Amanda Empeyte and Delphine Soulet whose

389 pictures were taken to make up Fig 2; and Julie Lemarchand for the second behavioral coding of 30%

390 of our videos.

391

392 References

393 Adriaense JEC, Koski SE, Huber L, Lamm C (2020) Challenges in the comparative study of empathy

394 and related phenomena in animals. Neurosci Biobehav Rev 112:62–82.

395 https://doi.org/10.1016/j.neubiorev.2020.01.021

396 Aguinis H, Gottfredson RK, Joo H (2013) Best-Practice Recommendations for Defining, Identifying,

397 and Handling Outliers. Organ Res Methods 16:270–301.

398 https://doi.org/10.1177/1094428112470848

399 Albuquerque N, Guo K, Wilkinson A, et al (2018) Mouth-licking by dogs as a response to emotional

400 stimuli. Behav Processes 146:42–45. https://doi.org/10.1016/j.beproc.2017.11.006

401 Albuquerque N, Guo K, Wilkinson A, et al (2016) Dogs recognize dog and human emotions. Biol Lett

402 12:20150883. https://doi.org/10.1098/rsbl.2015.0883

403 Baba C, Kawai M, Takimoto-Inose A (2019) Are horses (Equus caballus) sensitive to human emotional

404 cues? Animals 9:630. https://doi.org/10.3390/ani9090630

405 Bänziger T, Mortillaro M, Scherer KR (2012) Introducing the Geneva Multimodal expression corpus

406 for experimental research on emotion perception. Emotion 12:1161–1179.

407 https://doi.org/10.1037/a0025827
408 Briefer EF (2018) Vocal contagion of emotions in non-human animals. Proc. R. Soc. B Biol. Sci. 285

409 Brooks ME, Kristensen K, van Benthem KJ, et al (2017) {glmmTMB} Balances Speed and Flexibility Among Packages for Zero-inflated Generalized Linear Mixed Modeling. R J 9:378–400

411 Corujo LA, Kieson E, Schloesser T, Gloor PA (2021) Emotion recognition in horses with convolutional neural networks. Futur Internet 13:1–13. https://doi.org/10.3390/fi13100250

413 Custance D, Mayer J (2012) Empathic-like responding by domestic dogs (Canis familiaris) to distress in humans: An exploratory study. Anim Cogn 15:851–859. https://doi.org/10.1007/s10071-012-0510-1

416 Désiré L, Boissy A, Veissier I (2002) Emotions in farm animals: a new approach to animal welfare in applied ethology. Behav Processes 60:165–180. https://doi.org/10.1016/S0376-6357(02)00081-5

419 Friard O, Gamba M (2016) BORIS: a free, versatile open-source event-logging software for video/audio coding and live observations. Methods Ecol Evol 7:1325–1330. https://doi.org/10.1111/2041-210X.12584

422 Fureix C, Beaulieu C, Argaud S, et al (2015) Investigating anhedonia in a non-conventional species: Do some riding horses Equus caballus display symptoms of depression? Appl Anim Behav Sci 162:26–36. https://doi.org/10.1016/j.applanim.2014.11.007

425 Fureix C, Meagher RK (2015) What can inactivity (in its various forms) reveal about affective states in non-human animals? A review. Appl Anim Behav Sci 171:8–24. https://doi.org/10.1016/j.applanim.2015.08.036

428 Galvan M, Vonk J (2016) Man’s other best friend: domestic cats (F. silvestris catus) and their discrimination of human emotion cues. Anim Cogn 19:193–205. https://doi.org/10.1007/s10071-015-0927-4

431 Gouyet C, Ringhofer M, Yamamoto S, et al (2023) Horses cross-modally recognize women and men. Sci Rep 13:3864. https://doi.org/10.1038/s41598-023-30830-6

433 Hall C, Randle H, Pearson G, et al (2018) Assessing equine emotional state. Appl Anim Behav Sci 205:183–193. https://doi.org/10.1016/j.applanim.2018.03.006

435 Hartig F (2021) DHARMA: Residual Diagnostics for Hierarchical (Multi-Level / Mixed) Regression Models

437 Hötzel MJ, Vieira MC, Leme DP (2019) Exploring horse owners’ and caretakers’ perceptions of emotions and associated behaviors in horses. J Vet Behav 29:18–24. https://doi.org/10.1016/j.jveb.2018.10.002

440 Huber A, Barber ALA, Faragó T, et al (2017) Investigating emotional contagion in dogs (Canis familiaris) to emotional sounds of humans and conspecifics. Anim Cogn 20:703–715. https://doi.org/10.1007/s10071-017-1092-8

443 Jardat P, Calandreau L, Ferreira V, et al (2022) Pet-directed speech improves horses’ attention toward humans. Sci Rep 12:4297. https://doi.org/10.1038/s41598-022-08109-z

445 Jardat P, Lansade L (2022) Cognition and the human–animal relationship: a review of the sociocognitive skills of domestic mammals toward humans. Anim Cogn 25:369–384. https://doi.org/10.1007/s10071-021-01557-6

448 Jardat P, Ringhofer M, Yamamoto S, et al (2023) Horses form cross-modal representations of adults and children. Anim Cogn 26:369–377. https://doi.org/10.1007/s10071-022-01667-9

<figure><img src="image_25.png" /></figure>
450 Jitsumori M, Yoshihara M (1997) Categorical Discrimination of Human Facial Expressions by Pigeons: A Test of the Linear Feature Model. Q J Exp Psychol Sect B Comp Physiol Psychol 50:253-268

452 Kassambara A (2020) ggpubr: "ggplot2" Based Publication Ready Plots

453 Kassambara A, Kosinski M, Biecek P (2021) survminer: Drawing Survival Curves using "ggplot2"

454 Keaveney SM (2008) Equines and their human companions. J Bus Res 61:444-454.
https://doi.org/10.1016/j.jbusres.2007.07.017

456 Koo TK, Li MY (2016) A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 15:155-163. https://doi.org/10.1016/j.jcm.2016.02.012

458 Kremer L, Klein Holkenborg SEJ, Reimert I, et al (2020) The nuts and bolts of animal emotion. Neurosci Biobehav Rev 113:273-286. https://doi.org/10.1016/j.neubiorev.2020.01.028

460 Lampe JF, Andre J (2012) Cross-modal recognition of human individuals in domestic horses (Equus caballus). Anim Cogn 15:623-630. https://doi.org/10.1007/s10071-012-0490-1

462 Mendl M, Burman OHP, Paul ES (2010) An integrative and functional framework for the study of animal emotion and mood. In: Proceedings of the Royal Society B: Biological Sciences. Royal Society, pp 2895-2904

465 Nagasawa M, Murai K, Mogi K, Kikusui T (2011) Dogs can discriminate human smiling faces from blank expressions. Anim Cogn 14:525-533. https://doi.org/10.1007/s10071-011-0386-5

467 Nakamura K, Takimoto-Inose A, Hasegawa T (2018) Cross-modal perception of human emotion in domestic horses (Equus caballus). Sci Rep 8:8660. https://doi.org/10.1038/s41598-018-26892-6

469 Nawroth C, Albuquerque N, Savalli C, et al (2018) Goats prefer positive human emotional facial expressions. R Soc Open Sci 5:180491. https://doi.org/10.1098/rsos.180491

471 Panksepp J (1998) Affective neuroscience: The foundations of human and animal emotions. Oxford University Press, New York, NY, US

473 Patteson MW (1996) Equine cardiology. Blackwell Science

474 Pérez-Manrique A, Gomila A (2022) Emotional contagion in nonhuman animals: A review. Wiley Interdiscip Rev Cogn Sci 13:e1560. https://doi.org/10.1002/WCS.1560

476 Pritsch C, Telkemeyer S, Mühlenbeck C, Liebal K (2017) Perception of facial expressions reveals selective affect-biased attention in humans and orangutans. Sci Rep 7:7782.
https://doi.org/10.1038/s41598-017-07563-4

479 Proops L, Grounds K, Smith AV, McComb K (2018) Animals remember previous facial expressions that specific humans have exhibited. Curr Biol 28:1428-1432.e4.
https://doi.org/10.1016/j.cub.2018.03.035

482 Proops L, Mccomb K (2012) Cross-modal individual recognition in domestic horses (Equus caballus) extends to familiar humans. Proc R Soc B Biol Sci 279:3131-3138.
https://doi.org/10.1098/rspb.2012.0626

485 Quaranta A, D'ingeo S, Amoruso R, Siniscalchi M (2020) Emotion recognition in cats. Animals 10:1107. https://doi.org/10.3390/ani10071107

487 R Core Team (2021) R: A Language and Environment for Statistical Computing

488 Ruet A, Lemarchand J, Parias C, et al (2019) Housing Horses in Individual Boxes Is a Challenge with Regard to Welfare. Anim 2019, Vol 9, Page 621 9:621. https://doi.org/10.3390/ANI9090621

<figure><img src="image_26.png" /></figure>
490 Siniscalchi M, D'Ingeo S, Fornelli S, Quaranta A (2018a) Lateralized behavior and cardiac activity of dogs in response to human emotional vocalizations. Sci Rep 8:77.
491 https://doi.org/10.1038/s41598-017-18417-4
492

493 Siniscalchi M, D'Ingeo S, Quaranta A (2018b) Orienting asymmetries and physiological reactivity in dogs' response to human emotional faces. Learn Behav 46:574-585.
494 https://doi.org/10.3758/s13420-018-0325-2
495

496 Smith AV, Proops L, Grounds K, et al (2016) Functionally relevant responses to human facial expressions of emotion in the domestic horse (Equus caballus). Biol Lett 12:20150907.
497 https://doi.org/10.1098/rsbl.2015.0907
498

499 Smith AV, Proops L, Grounds K, et al (2018) Domestic horses (Equus caballus) discriminate between negative and positive human nonverbal vocalisations. Sci Rep 8:13052.
500 https://doi.org/10.1038/s41598-018-30777-z
501

502 Špinka M (2012) Social dimension of emotions and its implication for animal welfare. Appl Anim Behav Sci 138:170-181. https://doi.org/10.1016/j.applanim.2012.02.005

504 Therneau TM (2022) coxme: Mixed Effects Cox Models

505 Trösch M, Cuzol F, Parias C, et al (2019) Horses categorize human emotions cross-modally based on facial expression and non-verbal vocalizations. Animals 9:862.
506 https://doi.org/10.3390/ani9110862
507

508 VanDierendonck MC, Goodwin D (2005) Social contact in horses: implications for human-horse interactions. In: de Jonge FH, van den Bos R (eds) The human-animal relationship. Forever and a day. Royal van Gorcum, Assen, pp 65-81

511 Visser E, Vanreenen C, Vanderwerf J, et al (2002) Heart rate and heart rate variability during a novel object test and a handling test in young horses. Physiol Behav 76:289-296.
512 https://doi.org/10.1016/S0031-9384(02)00698-4
513

514 von Borell E, Langbein J, Després G, et al (2007) Heart rate variability as a measure of autonomic regulation of cardiac activity for assessing stress and welfare in farm animals - A review. Physiol Behav 92:293-316. https://doi.org/10.1016/j.physbeh.2007.01.007

517 von Lewinski M, Biau S, Erber R, et al (2013) Cortisol release, heart rate and heart rate variability in the horse and its rider: Different responses to training and performance. Vet J 197:229-232.
518 https://doi.org/10.1016/J.TVJL.2012.12.025
519

520 Wickham H (2016) Ggplot2: Elegant graphics for data analysis

521