<figure><img src="image_1.png" /></figure>

HAL
open science

# Investigating musical meaning: Empirical evidence for a music semantics

Léo Migotti Ramponi

<figure><img src="image_2.png" /></figure>

## To cite this version:

Léo Migotti Ramponi. Investigating musical meaning: Empirical evidence for a music semantics. Cognitive science. Ecole Normale Supérieure (ENS), Paris, FRA., 2023. English. NNT: tel-04294445

HAL Id: tel-04294445

https://theses.hal.science/tel-04294445

Submitted on 19 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Investigating musical meaning:
Empirical evidence for a music semantics

En vue d'une soutenance par

Léo Migotti

Le 31 août 2023

École doctorale n°158

Cerveau, Cognition,
Comportement

Spécialité

Sciences Cognitives

Préparée à

Institut Jean Nicod
(CNRS - ENS - EHESS)

Composition du jury :

Emmanuel CHEMLA
LCSP, Ecole Normale Supérieure

Examinateur

Claire PELOFI
New York University

Examinatrice

Jonah KATZ
West Virginia University

Rapporteur

Isabelle CHARNAVEL
Université de Genève

Rapporteuse

Philippe SCHLENKER
Institut Jean Nicod (CNRS-ENS-EHESS)
New York University

Directeur de thèse

Jean-Julien AUCOUTURIER
CNRS, FEMTO-ST

Co-encadrant

<figure><img src="image_4.png" /></figure>
<figure><img src="image_5.png" /></figure>
To my grandfather Luigi, who used to sing
*Elle était belle et souriait.*
I'm sure you're still singing where you are now.
# Acknowledgements

First and foremost, I am extremely grateful to my main supervisor Philippe Schlenker for his invaluable advice and continuous support during my master and PhD study. His knowledge and experience have not only been a constant inspiration, but they have helped me grow as a better scientist and a better person.

I would like to express my sincere gratitude to my co-advisor Jean-Julien Aucouturier, whose skills and presence have allowed me to broaden my perspectives on music cognition and engage in crucial critical thinking on my work.

I am deeply indebted to Emmanuel Chemla, who has not only provided me with extremely insightful and continuous feedback on my work, but has also greatly helped me raise my awareness of experimental procedures and sharpen my skills in experimental design and statistical analyses.

Many thanks to Claire Pelofi for her astute feedback and enlightening discussions during my PhD committees, and to Isabelle Charnavel and Jonah Katz who accepted to review this dissertation.

Collaborations have also been essential to this PhD project. I want to express my deepest gratitude to my esteemed colleague and friend Janek Guerrini and his unwavering kindness and trust. I am also extremely grateful to Mélissa Berthet, who introduced me to the wonders of ethology and made the first half of my PhD a both thrilling and peaceful journey. Many thanks too to Léo Zaradkzi, who I had the great pleasure to collaborate with in the early stages of my PhD. Special thanks to Alessandro Ansani, with whom I have had the chance to work, for his humanity and unfailing understanding. I would also like to sincerely thank Jeremy Kuhn and Lyn Tieu for the time spent on ongoing collaborations. Thank you to all members of the Institut Jean Nicod community and the Cognitive Science Department at ENS.

Travels have also made this experience incredibly inspiring. I would like to express my gratitude to all people who contributed to making those highly stimulating moments possible. In particular, thank you to all the RITMO team at the University of Oslo for a wonderful and unforgettable experience. I am also greatly thankful to Pritty Patel-Grosz for her help in the process, and for very enriching discussions and guidance from the very
first moments of this adventure.

These past two years have also been a great opportunity for me to teach. I would like to thank Denis Bonnay, Benjamin Simmenauer and Françoise Sackrider for trusting me in teaching at the Institut Français de la Mode. This teaching experience has already been determining for my working life.

I could not have undertaken this journey without all the extraordinary support I have received from family and friends, and words can hardly express how thankful I am (but maybe music could do it; and I know each of you is associated with — at least — one song in my mind). Thank you from the bottom of my heart to my mother, my sister and my father for their unconditional love and support. None of this could have happened without them. I am so very lucky to have received inexorable attention and support from my amazing friends, whom I will thank personally.

The past few years have also given me the opportunity to reconnect with art, and in particular with writing and musical practice. This would not have been possible without the very talented artists from the theatre company Le Printemps du Machiniste. Thank you so very much for their trust. I am now certain that art and science do not only connect in the world but also in my heart.

Thank you to anyone whom I might have forgotten and who contributed in one way or another to this unique experience. May music be with you!
# Résumé

La musique a-t-elle un sens ? Et si oui, quel type de sens ? Dans cette thèse, nous présentons des données expérimentales qui soutiennent l'idée que la musique possède une sémantique : (i) la musique est capable de faire référence à une réalité extra-musical, et (ii) elle le fait selon des règles systématiques. Afin de tester ces deux hypothèses, nous présentons quatre paradigmes expérimentaux dans lesquels la musique interagit avec d'autres facultés cognitives telles que le langage, la perception visuelle ou le mouvement corporel.

Dans la première expérience, nous avons présenté aux participants des phrases hybrides dans lesquelles des stimuli musicaux remplaçaient des mots pour évaluer le sens de ces phrases. Les résultats montrent que le contenu informationnel de la musique se répartit selon les différentes catégories de la typologie linguistique (c'est-à-dire les différents types d'inférences déclenchées par les phrases complexes) : les mécanismes cognitifs classifiant le sens dans le langage peuvent donc traiter l'information musicale.

Dans la deuxième expérience, les participants devaient juger des scènes audiovisuelles dans lesquelles des propriétés musicales telles que la hauteur des notes, les nuances et le timbre étaient manipulées et associées à des objets visuels. Les résultats montrent que (i) la hauteur des notes et les nuances peuvent être interprétées comme le niveau d'énergie ou la distance d'un objet visuel, (ii) ces propriétés peuvent être utilisées pour identifier un objet lorsque plusieurs objets sont présents, et (iii) le timbre est interprété comme la nature même d'un objet. Ces résultats montrent que les propriétés musicales ne sont pas seulement associées à des scènes générales, mais qu'elles sont liées à des objets contenus dans ces scènes.

La troisième expérience est une étude de cas sur la musique qui évoque la marche. Nous avons cherché à déterminer les conditions de cette association grâce à une tâche de préférence dans laquelle les participants jugeait des stimuli selon leur capacité à faire référence à un personnage en train de marcher. Les résultats montrent qu'au moins cinq propriétés sont impliquées dans cette association : trois d'entre elles (stabilité, alternance, binarité) sont liées aux propriétés structurelles d'une marche, et deux d'entre elles (consonance absolue et proximité relative des accords) sont liées aux propriétés physiques d'une marche. La consonance est interprétée comme la stabilité de chaque pas, tandis que la proximité des accords est associée à la symétrie de la démarche. Ce modèle rend
par ailleurs mieux compte de la sémantique de la musique évoquant des marches que les
modèles précédents et fournit des données préliminaires suggérant que chaque événement
musical est interprété comme un événement du monde réel.

La quatrième expérience propose enfin de tester la possibilité de trouver des traces
de la sémantique de la hauteur des notes sur le mouvement corporel. Les participants
avaient pour tâche de marcher en rythme avec une série de sons dont la hauteur variait
aléatoirement. Nous avons constaté que leurs pas étaient à la fois plus longs et réalisés
avec une plus grande force sur les notes basses. La similarité entre l'effet physique produit
par une note plus basse (associée à davantage de poids) et la sémantique d'une note plus
basse (associée à des objets plus gros ou ayant moins d'énergie) suggère qu'il existe une
interaction entre les représentations musicales et le système moteur.

Dans l'ensemble, nos résultats fournissent des preuves en faveur d'une sémantique de
la musique : la musique peut faire référence à une réalité extra-musicale selon un ensemble
de règles systématiques. Ils montrent également que la musique interagit avec d'autres
systèmes cognitifs, qu'ils portent directement un sens (langage) ou non (mouvement cor-
porel), ce qui soulève des questions sur la manière dont les effets induits par la musique
se rapportent à la signification musicale.

Mots clés : musique, sémantique, sens, cognition musicale
Abstract

Does music have meaning? And if so, what kind of meaning? In this dissertation, we provide experimental data supporting the view that music has a semantics: (i) music is able to refer to an extra-musical reality, and (ii) it does so in a rule-governed fashion. We present four experimental paradigms designed to test these two assumptions by having music interact with other cognitive faculties such as language, visual perception, or body motion.

In the first experiment, we presented participants with hybrid sentences containing music in lieu of words and tried to determine what meaning they get from such sentences. The results show that the informational content of embedded musical stimuli could be divided among the different slots of the linguistic typology (i.e. the different types of inference that complex sentences trigger). This suggests that the cognitive mechanisms classifying meaning in language can accommodate musical information.

In the second experiment, we had participants judge audiovisual scenes in which musical properties such as pitch height, dynamics, and timbre were manipulated and paired with visual objects. The results show that (i) both pitch height and dynamics can be semantically interpreted as the energy level or the distance of a visual object, (ii) pitch height and dynamics can be used to retrieve which object is being referred to when multiple objects are present, and (iii) timbre is interpreted as the very nature of an object, hence the timbre/object association cannot be violated in audiovisual scenes. Together, these results show that musical properties are not only associated with general scenes, but that they are bound to objects in those scenes.

In the third experiment, we built a case study about music that evokes walking. We sought to determine what it takes for a musical piece to evoke a walk through a preference task in which participants had to pick stimuli based on how well they evoked a character walking. The results show that at least five properties are involved in the association of a musical stimulus with a walk: three of which (steadiness, alternation, binarity) relate to structural properties of a walk, and two of which (absolute consonance and relative chord proximity) relate to physical properties of a walk such as stability. Consonance was arguably interpreted as the stability of each step, while chord proximity was associated with gait symmetry. The results further show that this fine-grained model accounts for the semantics of music evoking walks better than previous models and provide preliminary
evidence that each musical event might have to be interpreted as a real-world event.

In the fourth experiment, we finally checked whether it is possible to find footprints
of the semantics of pitch on body motion. Participants were tasked to walk in synchrony
with a series of sounds varied in pitch. We found that their steps were both longer and
produced with greater force on lower pitches. The similarity between the physical effect
produced by lower pitch (mimicking an increase in weight) and the semantics of lower
pitch (associated with bigger objects, or less energy) suggests that there is an interaction
between musical representations and the motor system.

Together, our results provide evidence for a music semantics: music can refer to an
extra-musical reality according to a set of systematic rules. They also show that music
interacts with other cognitive systems either directly bearing meaning (language) or not
(body motion), which raises questions as to how music-induced effects relate to musical
meaning.

**Keywords :** music, semantics, meaning, music cognition
# Contents

<table><tr><td>Acknowledgements</td><td>ii</td></tr><tr><td>Résumé</td><td>iv</td></tr><tr><td>Abstract</td><td>vi</td></tr><tr><td>Table of Contents</td><td>vii</td></tr><tr><td>1 General introduction</td><td>1</td></tr><tr><td>2 Meaning from music embedded in language</td><td>33</td></tr><tr><td>3 Meaning from music in audiovisual scenes</td><td>83</td></tr><tr><td>4 Meaning from music alone</td><td>113</td></tr><tr><td>5 Music-induced effects on body motion</td><td>167</td></tr><tr><td>6 General discussion and conclusions</td><td>188</td></tr><tr><td>Publications</td><td>205</td></tr></table>
# Chapter 1

General introduction
# Contents

<table><tr><td>1</td><td>Musical structure without meaning?</td><td>4</td></tr><tr><td>1.1</td><td>Musical syntax</td><td>5</td></tr><tr><td>1.2</td><td>No meaning</td><td>6</td></tr><tr><td>2</td><td>Musical meaning(s): a typology</td><td>6</td></tr><tr><td>2.1</td><td>An internal meaning</td><td>7</td></tr><tr><td>2.1.1</td><td>Musical meaning as musical form</td><td>7</td></tr><tr><td>2.1.2</td><td>Musical meaning as expectations</td><td>8</td></tr><tr><td>2.2</td><td>A hybrid meaning</td><td>8</td></tr><tr><td>2.2.1</td><td>Musical meaning as emotions</td><td>8</td></tr><tr><td>2.2.2</td><td>Musical meaning as intention attribution</td><td>11</td></tr><tr><td>2.2.3</td><td>Musical meaning as musical forces</td><td>12</td></tr><tr><td>2.3</td><td>An external meaning</td><td>13</td></tr><tr><td>2.3.1</td><td>Musical meaning as social uses</td><td>13</td></tr><tr><td>2.3.2</td><td>Musical meaning as information about objects</td><td>14</td></tr><tr><td>2.3.3</td><td>Musical meaning as inferences about the world</td><td>15</td></tr><tr><td>3</td><td>Music semantics</td><td>15</td></tr><tr><td>3.1</td><td>Musical truth</td><td>16</td></tr><tr><td>3.2</td><td>Musical inferences</td><td>18</td></tr><tr><td>4</td><td>Outline</td><td>19</td></tr><tr><td>4.1</td><td>Meaning from music embedded in language</td><td>20</td></tr><tr><td>4.2</td><td>Meaning from music interacting with visual scenes</td><td>22</td></tr><tr><td>4.3</td><td>Musical meaning from music alone</td><td>24</td></tr><tr><td>4.4</td><td>Music-induced body effects</td><td>26</td></tr><tr><td>5</td><td>Bibliography</td><td>29</td></tr></table>
I consider that music is, by its very nature, essentially powerless to express anything at all, whether a feeling, an attitude of mind, or psychological mood, a phenomenon of nature... Expression has never been an inherent property of music. That is by no means the purpose of its existence.

Igor Stravinsky (1935)

It is very interesting that a composer such as Igor Stravinsky, famously known for having composed some extraordinarily expressive masterpieces such as *The Rite of Spring* (1913), argues that music 'is powerless to express anything'. From the first bars of the piece indeed,<fnref n="1" /> the famous musical motive to be repeated, developed and enriched with many innovative orchestration techniques, seems to immediately convey a sense of dialog between some objects, animals, insects, or natural phenomena. Stravinsky himself described the introduction of the piece as 'a swarm of spring pipes.' When it premiered in 1913 at the Paris Champs-Élysées theater, the *Rite of Spring*, which Stravinsky composed to accompany a ballet choreographed by Russian choreographer Vaslav Nijinsky, was both acclaimed as a major revolutionary piece and described as scandalous and outrageous. The piece was accused of relying too highly on rhythm, unusual stress patterns, and dissonances; to a much greater extent than had ever been done before.

The year before, in 1912, Vaslav Nijinsky had already caused a scandal when performing on Debussy's *Prélude à l'après-midi d'un faune*.<fnref n="2" /> The piece, which premiered in 1896 and was later choreographed, had also been criticized for almost opposite reasons, and in particular the lack of regular beat or rhythmically salient patterns. Inspired by a poem by French poet Stéphane Mallarmé telling the story of a lovesick faun contemplating nature, the piece is also extremely evocative of natural sounds and has since been acclaimed for its unique pastoral atmosphere. Although very different in the rest of the pieces — but with similar disagreement from the audience — the two pieces share very similar initial parts, both of which involve almost exclusively wind instruments and soft recurring melodic patterns, with which both composers managed to evoke nature. Later, Debussy writes:

<footnote n="1">¹See for instance *this performance* without choreography by the London Symphony Orchestra, conducted by Sir Simon Rattle (2017).</footnote>

<footnote n="2">²See *this performance* without choreography by the London Symphony Orchestra, conducted by François-Xavier Roth (2017).</footnote>
There is nothing more musical than a sunset. He who feels what he sees will find
no more beautiful example of development in all that book which, alas, musicians
read but too little - the book of Nature.

Stravinsky would have probably not agreed with the idea that an event such as a sunset
could have anything musical to it; or at least, he would not have agreed that music could
express anything about it, let alone represent it. Somehow, Debussy takes a different
stance on the ability of music to depict natural events, and many of his other pieces, such
as *La Mer* (1905),<fnref n="3" /> were similarly evocative of natural things or even mimicking natural
sounds.

Together, these two quotes are just an instance of the ever-lasting debate on whether
music can convey information about the world, which is very much related to the question
of whether music can be said to have meaning. Many divergent claims have been given
on this matter throughout the history of music theory and, later, music psychology and
music cognition. We will now illustrate two opposite claims: first, that music does not
have meaning (at all); second, that music does have meaning (we will then specify what
this musical meaning can be).

# 1 Musical structure without meaning?

We will first explore the idea that music does not have meaning, and we will need to
introduce a distinction between musical *meaning*, which refers to what music is potentially
about, and musical *syntax*, which refers to principles governing musical structure and
form. Crucially, a system that has syntax does not necessarily have meaning too: it is
possible to create an infinity of languages that obey rules of well-formedness which do
not have any meaning.<fnref n="4" />

<footnote n="3">³See for instance *this performance* by the French National Orchestra with conductor Cristian Măcelaru.</footnote>

<footnote n="4">⁴An example using meaningless syllables is given in Schlenker (2017).</footnote>
## 1.1 Musical syntax

It is today uncontroversial that music has a syntax, defined as a set of rules that determine whether a musical piece is well-formed.<fnref n="5" /> The pioneer work by Lerdahl and Jackendoff (1983) established several categories of such rules for tonal music, and on top of these well-formedness principles, preference rules that account for why and how listeners assign certain mental structures to music instead of other possible equally well-formed ones. More recent accounts have updated this view (Rohrmeier, 2011) and applied rules from other musical traditions such as jazz.<fnref n="6" /> But crucially, no investigation of musical form and the systematic rules that music must satisfy to be considered well-formed has to commit to whether music has meaning or not. This also holds (i) regardless of how similar to the syntax of natural languages musical syntax is, and (ii) despite the fact that natural languages have both. In a paper investigating the similarities between the formal properties of language and music, Katz and Pesetsky (2011) provided arguments in favor of the ‘Identity thesis’ for music and language, which states that “All formal differences between language and music are a consequence of differences in their fundamental building blocks. In all other respects, language and music are identical.” In other words, what makes music different from language is meaning-irrelevant: the only reason for which the two systems differ is because their constituent parts are not the same, but if we put that aside, then the formal structure of both systems is identical. This identity is therefore inherently syntactical; hence one cannot predict from this analogy that music should have meaning. We will now continue with an exploration of the hypothesis that, despite having a syntax, music does not have meaning.

<footnote n="5">⁵Note that, as argued in Schlenker (2017), we do not need to either take a stance on how this syntax works to support the claim that music has meaning; although we have good reasons to believe that musical syntax interacts with what we will soon call music semantics.</footnote>

<footnote n="6">⁶See for instance Granroth-Wilding and Steedman (2014) on a model accounting for how chord sequences are mapped onto mental representations about the hierarchical and structured relation between chords in listeners.</footnote>
## 1.2 No meaning

Many arguments have been raised to support the view that music does not have meaning. But a prominent one has been targeted at the plurality of things that music evoke to make the point: in short, music cannot have meaning because different listeners assign different meanings to a same musical piece. This argument was famously reused by American conductor and composer Leonard Bernstein (2005), using an example from Strauss's Variation II from symphonic poem *Don Quixote*. The reason for which music does not have meaning, Bernstein argued, was that one could not spontaneously know what that piece was supposed to represent without knowing the program. It was then possible to come up with a 'wrong' story which the piece could just as much represent, and Bernstein came up with a story about Superman to illustrate that the intended meaning of the piece could be otherwise. But this argument might actually not support the view it intends to. The very fact that multiple interpretations are possible, as noted by Schlenker (2022), perfectly illustrates the opposite claim, for the 'wrong' story was actually almost perfectly isomorphic to (i.e. had the very same structure as) the original plot. Far from showing that music has no meaning, this argument therefore gives a clue about the nature of musical meaning. Not only does music have meaning, but it has an abstract meaning, in the sense that multiple scenes can be represented by a same music, as long as they share some of their properties. Just as in language, in which sentences are true of different situations, music can be true of different situations.<fnref n="7" />

## 2 Musical meaning(s): a typology

Arguing that music can be true of situations already commits us to a very specific theory of musical meaning, while many authors have given arguments supporting the claim that music has meaning without necessarily appealing to the concept of truth. First, it has been argued that music only has an internal meaning, i.e. that musical meaning lies in the form of music itself. By contrast, other theories have argued that music has an extra-

<footnote n="7">⁷We come back to this notion of musical truth a bit later.</footnote>
musical meaning,<fnref n="8" /> i.e. that music is associated with things in the extra-musical world, either real or imagined. Hybrid theories have posited that musical meaning is reducible to emotions; we will discuss whether this rather amounts to an internal of extra-musical meaning in the following sections.

## 2.1 An internal meaning

### 2.1.1 Musical meaning as musical form

Among proponents of internal musical meaning, Austrian music theorist Hanslick (1957) defined music as ‘successions and forms of sound’, and argued that ‘these alone constitute the subject’ (of music). In other words, the only subject matter of music is precisely what it is made of, its own components: in short, its form. It is of course uncontroversial that music conveys information about its own form; and it is also right that this does not entail at all that it should be able to convey information about something else. While Hanslick (1957) recognizes that music triggers feelings and emotions (as is today commonly admitted (Blumstein et al., 2012; Juslin and Laukka, 2003; Koelsch, 2012)), he stresses that music is however unable to represent these emotions. Under this view, a piece of music can for instance make someone happy, but it cannot represent a happy scene, or a scene in which someone is happy. But that listeners do not get meaning from music is not incompatible, Hanslick argued, with their active engagement in listening. For him, ‘two listening modes’ are available to the listener: a passive one, comparable to just hearing music without actively engaging in music; and an active one, described as a state in which the listener attempts to follow the composer’s intention and anticipate what should come next in a piece. Interestingly, the existence of this second active mode seems to be an argument in favor of a music *pragmatics*, i.e. reasoning patterns involving predictions about what is to come next in a musical piece, and updates on these predictions. These reasoning patterns however remain ‘internal’ in the sense that they do not involve anything from outside music: trying to predict what is to come next in a musical piece is a reasoning still targeted at musical form. We explore another theory

<footnote n="8">⁸We here reuse this term coined by Koelsch (2012).</footnote>
of musical expectations which emphasizes the ‘external’ emotional counterpart of these
prediction mechanisms in the following section.

2.1.2 Musical meaning as expectations

If our understanding of music solely relies on predictions on its form, it follows that grounding musical meaning in expectations amounts to considering that musical meaning lies within music. But expectations on musical form have also been argued to trigger emotions in listeners. Huron (2006) notably developed a theory in which musical predictions are systematically paired with ‘affective responses’, i.e. emotions. In this theory, two types of response are associated with anticipations of musical events before they occur: imaginative and tension responses respectively trigger emotions about how one feels about what they think is gonna happen next, and about how ready one feels based on how a musical event is being prepared. Prediction, reaction and appraisal responses follow once a musical event has occurred: they encompass the satisfaction or dissatisfaction a listener feels based on whether the expectations were satisfied, with one’s assessment of one’s own reaction to the musical event (such as surprise), and a more general appreciation of how things have turned out upon reflection. But Huron does not, per se, take a stance on whether these different mechanisms are what constitute musical meaning; yet, such theories have been developed.

**2.2 A hybrid meaning**

In this section, we discuss the theoretical implications of considering that musical meaning
reduces to music-induced emotions.

**2.2.1 Musical meaning as emotions**

Meyer (1956) notoriously argued that music has meaning and that this meaning could be
reduced to the emotions music triggers, which themselves originate in whether musical
expectations are satisfied or not. For Meyer, there are two sources of musical expecta-
tions: (i) universal perceptual principles and (ii) ‘archetypes’ (namely, expectations from
specific musical conventions governing musical systems which listeners are familiar with).<fnref n="9" />
Consistent with Huron (2006), Meyer claimed that musical emotions are triggered by mu-
sical expectations, but his theory crucially differs in stating that the meaning of music is
reducible to these emotions. Note that this is already different from saying that musical
meaning solely lies in the information it conveys about its own form and its constituent
blocks, for if it can trigger musical emotions, it is already in relation to something that is
extra-musical. If (i) music triggers emotions, and if (ii) these emotions constitute musical
meaning, then given that (iii) emotions are external to music, it follows that musical
meaning is external to music.

Yet for Meyer, music “operates as a closed system, that is, it employs no signs or
symbols referring to the non-musical world of objects, concepts, and human desires.” An
analogous claim was later made by Davies (1994), for whom music neither had the ability
to describe nor to depict: the existence of musical emotions says nothing about music
as referring to non-musical things.<fnref n="10" /> But Meyer also stressed that it was “necessary to
emphasize that the prominence given to this aspect of musical meaning [i.e. emotions]
does not imply that other kinds of meaning do not exist or are not important”, and
this later claim leaves open the question of the existence of an extra-musical *referential*
meaning besides one that strictly consists of the set of emotions generated by musical
expectations.<fnref n="11" /> One might however argue that generating emotions entails representing
emotions. In Schlenker (2017) for instance, music is argued to convey emotional infor-
mation through two different mechanisms. First, it is possible that a listener represents
emotions of non-musical objects: in this case, a musical piece does not necessarily make
the listener feel sad at first, but they understand that the music is referring to something

<footnote n="9">⁹Note that this distinction is close to that introduced by Schlenker (2017) on the distinction between meaning-bearing musical properties lifted from normal auditory cognition (such as loudness, plausibly universal) and those that are specific to music (such that harmonic rules) and therefore probably those that are specific to particular musical systems such as the tonal system as well.</footnote>

<footnote n="10">¹⁰Unless, as we will see, one has a theory of how musical emotions relate to the extra-musical world, and in particular to emotions of music-external events.</footnote>

<footnote n="11">¹¹Here and after, we will use the term ‘referential’ (also known as ‘denotative’) as commonly used in linguistics to characterize the property of an item with a given form (be it linguistic - such as a word, sign, sentence - or non-linguistic - such as a gesture, a picture, a visual representation, a sound or a musical sequence) to have the external world as referent, and not any part of the form or its properties itself. Referential meaning typically does not include connotations, cultural associations, or pragmatic enrichments.</footnote>
sad (a sad character, for instance), and this can *then* affect the emotional state of the listener ('by contagion'). Second, it is possible that music is interpreted as events *experi-enced* by listeners themselves. Under this view, music does refer to extra-musical events, but those events are *internal* to the listener. This is crucially different from the internal accounts we have mentioned such as Hanslick's, where meaning lies within musical form; here, meaning lies within the listener's experience of music. Hence, the existence of mu-sical emotions is compatible with music referring to an extra-musical reality, but this is not what most accounts of musical emotions such as Meyer's presuppose.

Aside from the structural similarities between music and language sketched in the pre-
vious section, several authors have more specifically argued that music conveys meaning
because it shares some of its fundamental constituent properties with other communica-
tive systems such as human speech or animal vocalizations that are known to convey
emotions. Ilie and Thompson (2006) demonstrated that pitch height, intensity and rate
have effects on valence judgments, energy arousal and tension arousal in both speech
and music. Along the same line of work, Bowling et al. (2010) found correlations in the
spectra of intervals and speech: major intervals have a spectrum that is similar to that of
excited speech, while the spectrum of minor intervals is more similar to that of subdued
speech. Bowling et al. (2012) reported similar results when comparing Western to South
Indian music, indicating that the music/speech analogies are neither exclusive to a specific
language nor to a specific musical tradition. Together, these results suggest a universal
code for conveying emotional information, a code which might even been shared more
broadly, beyond music and speech. Juslin and Laukka (2003) found analogous similari-
ties between vocal performance and instrumental music, and Bedoya et al. (2021) showed
that applying acoustic modifications modeled after human voice to instrumental sounds
maintained emotion recognition. And beyond audition, Sievers et al. (2013, 2019) has
provided evidence that this emotional code is also shared across modalities (in particular
between audition and vision). It has finally been argued that this code or part of it is
not only shared across domains but across species as well. For instance, Blumstein et al.
(2012) found that non-linearities (sudden rises or drops in frequency), that signal emo-
tional states in the vocalization systems of many vertebrates, do convey similar effects in humans.

In sum, it is well established that music conveys emotions, be it because it reuses some features already present in language or vocalization that do convey emotions, or for other reasons. While Meyer (1956) states that these emotions are what constitute musical meaning, the other references we provided do not necessarily commit to this definition. In any event, under the assumption that musical meaning can be reduced to music-induced emotions, then we tend towards a notion of musical meaning that is directed outward music. But again, this is different from saying that music can represent emotions — unless one has an explicit theory of how these emotions relate to emotions experienced by extra-musical objects, or by listeners themselves.

### 2.2.2 Musical meaning as intention attribution

Some research has evidenced the possibility for musical components such as notes or chords to be associated with imagined fictional agents having intentions, regardless of musical emotions (Maus, 1988). These theories echo the famous animation by Heider and Simmel (1944) (accessible here) in which geometrical shapes are very easily assigned intentions. Just as such abstract objects can be assigned intentions by manipulating some properties of their movements (in particular by creating behaviors which physical laws alone could not account for), these theories assign intentions to musical events associated with different ‘fictional agent types.’<fnref n="12" /> Maus (1988) claims that listeners associate musical works with such imagined agents, which serve as ‘repositories’ for psychological states. Crucially, these associations are indeterminate and ad hoc since they vary from listener to listener and cannot be predicted. This may be a problem for these theories, which seem under-specified as far as the nature of these imaginary agents is concerned. While under a certain interpretation, musical events (or musical ‘forms’) are associated with agents that might fall beyond the scope of the piece itself, Maus (1988) also claims that some agents

<footnote n="12">¹²Despite the conceptual similarity, in Heider and Simmel (1944), it is the geometrical shapes themselves which are assigned intentions; while in Maus (1988), musical events (or musical ‘forms’) are associated with agents that might fall beyond the scope of the piece itself.</footnote>
are ‘coextensive with the musical work’ (i.e. are part of the piece itself). Monahan (2013) argues that this account resonates with that of Cone (1982), which states that ‘lower-level agencies are understood collectively as components of a higher-order persona that is coextensive with the work as a whole.’ Under this view, the aggregation of individual agents results in the abstract attribution of an intention (or even a ‘mind’) to the piece itself. Here again, this composite intention seems to lie within the scope of the musical piece. Yet Maus (1988) also allows for these intentional agents to be ‘fictionalized versions of the composer or the performers’: the idea here is that listeners attempt to reconstruct a performer’s or a composer’s intention and interpret music relative to their viewpoint. Although this part of the theory is very similar to that of Hanslick (1957) presented earlier, the under-specification of the nature of these fictional agents makes the general theory hybrid, in that it allows for musical meaning to refer to agents understood as being part of the piece, or to refer to external agents. In sum, these theories seem to commit neither to a pure notion of internal meaning, nor to one of external meaning.

### 2.2.3 Musical meaning as musical forces

Another line of research has explored how musical properties are interpreted in terms of auditory counterparts of physical forces such as gravity, magnetism, and inertia. For instance, Larson (2012) came up with a model in which pitches entertain similar parallel between themselves, explaining how listeners experience meaning as reducible to motions constrained by musical forces throughout the course of a piece. Honing (2003) investigated how physical models of slowing down motion can account for how musical pieces generally end with a final *ritardando* (i.e. a decrease in tempo marking the end of the piece). Although the application of physical models to interpretative musical facts does appear to create constraints on which interpretations (in terms of performance, not in semantic terms) are licensed, the authors call for caution in applying these methods systematically. Besides, that the rate at which speed decreases in a real world slowing-down motion matches that of the rate at which music speed decreases at the end of a piece can in theory be explained in purely syntactical terms, if, for instance, a performer tries align
the musical structure with that of the event. And in any event, it seems here that these
abstract forces merely refer to the physical work by analogy.

**2.3 An external meaning**

Although some of the previous accounts of musical meaning already contain a sense of
reference to non-musical things such as emotions, imaginary agents, characters, or forces,
they can all be considered hybrid in that they do not need to take a strong stance on
whether musical meaning lies within or outside music itself. In this section, we finally
explore different accounts which less ambiguously claim that musical meaning exists in
virtue of its connection to the outside world.

**2.3.1 Musical meaning as social uses**

Music can first connect to the outside world in virtue of the numerous actual events
and social contexts in which it is used, a view which led Cross and Woodruff (2009) to
argue that “Music and language are both part of the human communication toolkit.”
Under the assumption that music and language have co-evolved from an ancestral form,
music is considered to be an optimal signal which is able to “manage situations of social
uncertainty.” While music is often considered as a communicative system which primarily
aims at communicating emotions, it seems to convey other kinds of information. Music is
notably widely found to play a crucial role in regulating social interactions through social
bonding when the cohesion of a group is somehow threatened, either from outside agents
or from within. Music accompanies major life transitions in many societies (Blacking,
1976; Schulte-Tenckhoff and Feld, 1988), and has also been widely used to create or
recreate a sense of community in threatened or minority groups (Slobin, 1993). Music
also plays a crucial role in emotional self-regulation (Juslin and Sloboda, 2001), and
is almost universally associated to infant care (Trevarthen, 1999). But what kind of
information does music provide in these contexts, exactly? Here, it seems that musical
meaning is defined as *social use*, a view which has been given some more recent empirical
evidence in Mehr et al. (2019), which show that most musical features in songs are cross-
culturally associated with properties of the contexts in which the songs are performed,
and for which a typology can be given based on three dimensions (formality, arousal, and
religiosity).

The correlations between some music types and social uses do however not quite entail
that music can refer to extra-musical things. It could indeed be that those associations
are mere conventions, and that a given musical piece in a given situation does not per
se represent that situation, and that is just empirically associated with it. However,
this need not be incompatible with the view that music has referential meaning. One
might indeed argue that although partly conventional, musical conventions (in terms
of social use, not in terms of composition) are established because they evoke mental
representations that are consistent with the context in which a given musical piece is
played or sung. For instance, lullabies might have spread in the context of infant care
not only by convention, but because they could be associated to such situations a priori.

2.3.2 Musical meaning as information about objects

Among all previous accounts, none of them (music referring to social contexts, music
triggering emotions, possibly relying on shared mechanisms with speech) requires that
music be able to make reference to extra-musical things. We will now turn to theories
and empirical evidence which support this view more clearly.

A first well-documented line of work focuses on relations music entertains with rep-
resentations of movement. Music has been argued to be associated with spatial and
motion properties of virtual spaces (Clarke, 2001), or to motion properties of musical
events themselves. For instance, Saslaw (1996) suggests that musical objects themselves
are conceived as moving through space, while having ‘the attributes of real-world objects:
weight, speed, force.’ However, these theories ambiguously stand by the fact that musical
meaning is somehow still internal to music: if listeners listen to music as conveying in-
formation about musical events themselves (such as tones), it does not necessarily follow
that they would assign similar attitudes to extra-musical agents. Yet, several musical
properties have been experimentally shown to be responsible for associations to objects
beyond music-internal representations (Godøy and Leman, 2010). For example, Eitan (2013) explored the associations listeners make between pitch and loudness properties of music and visual objects, suggesting that loudness can be thought of as indicating distance, while pitch is notoriously mapped onto height. Musical properties such as dynamics, pitch direction, intervals, attack rate, and articulation affect motion imagery in tasks where participants have to imagine a character's movement (Eitan and Timmers, 2010). Interestingly, while the authors note that these associations are not codified in natural language, recent work indicate that they can be iconically and visually represented in sign languages (Schlenker, 2018a).

### 2.3.3 Musical meaning as inferences about the world

More recent theoretical work has developed a theoretical framework in which properties of time, pitch, loudness, and harmony are integrated to account for musical inferences about virtual objects (Schlenker, 2017): ‘minimal pairs’ of musical stimuli changing only with regards to one property show that one can create contrasts in the evoked scenes. For instance, rising a musical sequence by an octave changes the size property of the denoted object: a higher-pitched musical sequence typically represents a small object better than a large one. While mainly consistent with previous research, some new insights on the rules mapping each property to its semantic interpretation are provided. This line of work is the one we will rely on the most throughout this dissertation. We discuss its theoretical details and predictions in the following section.

## 3 Music semantics

We have just seen that there are many arguments supporting the idea that music can refer to non-musical objects undergoing some events (such as moving), i.e. to an extra-musical reality. Yet, this could still be the case if the relation between music and what it refers to were conventional; or if it were purely subjective in the sense that it only depends on a listener's personal interpretation. Acknowledging the existence of meaning effects triggered by music does not entail that music has a semantics, defined as ‘a rule-
governed way in which music can provide information (i.e. license inferences) about some
music-external reality' (Schlenker, 2017). Under this view, the mechanisms responsible
for the generation of mental representations from music are not arbitrary: they obey
systematic rules that map musical properties to properties of an extra-musical world.
This view is in particular different from already mentioned theories of ad hoc agentivity
assignment such as Maus (1988), which posit that each listener parses a musical work as
they please. In this section, we present the main concepts of the theoretical framework
of music semantics, which this dissertation will rely on, and introduce two of its main
concepts: musical truth, and musical inferences.

**3.1 Musical truth**

For a few years, music semantics has been developing within the broader research agenda
of Super Semantics (Schlenker, 2018b). The core idea of this research programme is to
extend the methods of formal semantics to the analysis of meaning beyond language.

Under the assumption that music has, just as language, the ability to refer to a non-
musical reality, it is possible to extend the traditional definition of linguistic meaning
as the set of *truth conditions* of a proposition (i.e. statements about the conditions
under which a given proposition is true) to music, and to define musical meaning through
truth conditions as well. The truth conditions determine under which conditions a given
musical event or sequence is *true of* an extra-musical event.<fnref n="13" /> Now, establishing truth
conditions for music requires some clarification on how music can be true of an event. A
musical event *M* is true of an event *E* if that event *E* satisfies the relevant properties of
*M*. In other words, a musical excerpt is true of an event if the semantic interpretation
of its musical properties complies with the properties of the event relative to a set of
semantic rules. This is all very abstract. Let's take a quick example to illustrate this.

We have seen in the previous section that many studies have established correlations

<footnote n="13">¹³This is a first approximation used for clarity. In Schlenker (2017), musical events are not taken to be true of events but of pairs of objects and n-tuples of events. It is also possible to define musical truth relative to situations that involve multiple events. We will for now purposely remain vague as to which of these entities music primarily refers to. The experiment presented in section 4.2 will try to address this point</footnote>
between some musical properties and physical properties of objects such as size or energy.
For instance, loudness can be interpreted as providing some information about the level
of energy involved in an event (Schlenker, 2017). **The louder the sound, the more
energy in the event.** And here we are already: the previous statement is a semantic
rule which states that (i) loudness is semantically interpreted as energy, and specifies (ii)
in which direction the correlation holds. Now, suppose that we have a minimal musical
sequence made of two musical events $M =< m_1, m_2 >$, with $m_1$ being softer and $m_2$
being louder. We therefore have an increase in loudness in the music. All events which
satisfy **the bolded rule**, i.e. all events that are such that they involve an increase in
energy, will make that musical sequence true of them; and the set of all events which
a musical sequence is true constitute the *denotation*<fnref n="14" /> of that musical sequence. In our
case, the truth-condition can be stated as follows:

$M =< m_1, m_2 >$ is true of $E =< e_1, e_2 >$

if $e_1$ involves more energy than $e_2$.

This is an extremely simplified version of a possible model of music semantics reduced
to the interpretation of musical loudness, and several conditions have here been omitted
for simplicity (for instance, we did not specify what the notion of energy was referring
to exactly). Besides, many more musical properties are involved in the model developed
by Schlenker (2018a) (in particular, truth conditions on time, pitch, and harmony), and
even more should be added in the future. But this gives the idea: the purpose of music
semantics is to identify musical properties that bear meaning and formalize the semantic
rules that generate meaning from musical form (which we just did for loudness): in short,
develop a model which can derive the truth conditions of any musical sequence.

One must note that although the ground concepts are shared between natural language
semantics and music semantics, the two systems are very different with respect to at
least two characteristics. First, music semantics is very abstract: a same music can be

<footnote n="14">¹⁴For now, we do not expand of what kind of things lie in the denotation; but we will be using the term ‘denoted’ applied to objects, events and situations as meaning ‘belonging to the denotation’</footnote>
true of many different situations, and much more abstract than the semantics of natural
languages. In our example, *M* is indeed true of any event gaining energy; that is arguably
an extremely large set of possible denotations. Second, it is based on very different rules.
In particular, music semantics heavily relies on (a particularly abstract form of) iconicity,
i.e. the fact that a certain musical form will resemble its denotation.<fnref n="15" />

## 3.2 Musical inferences

We have just seen an example of how a given musical sequence with a determined arrangement of loudness levels of its sub-events can be true of certain events. As soon as we have a notion of musical truth, it is possible to define a related notion of inference. As a first approximation, we suppose that musical inferences are of the same sort as linguistic inferences. Let us go back to our previous example: we have seen that a musical event *M* with increasing loudness is true of an event *E* involving an increase in energy. Any event that makes *M* true will then also make a statement of the form ‘*E* involves an increase in energy’ true. We will therefore say that *M* licenses the inference that the denoted event *E* involves an increase in energy.

Musical inferences are very diverse in nature: music can provide different kind of
information about the extra-musical world. Schlenker (2017) argues that there are at
least two categories of musical inferences. First, there are inferences which are lifted from
normal auditory cognition and triggered by sound properties that are not exclusive to
music: inferences from loudness, pitch, and timbre fall within that category. For instance,
the fact that an increase in loudness licenses an inference about an object moving closer
has nothing specifically musical to it: any sound with increasing loudness in the world
would license an inference about the source of that sound coming closer. Second, there
are inferences which are specific to music cognition, and which do not necessarily have
counterparts in the non-musical world. Inferences triggered by harmony (i.e. the system
of rules governing the hierarchical organization of chords within a pitch space structured
around a centre) fall within that second category of inferences. It is possible that some

<footnote n="15">¹⁵Although there are cases of iconicity in language too, such as that reported in Guerrini (2020), they are quite rare in spoken languages, although much more common in Sign Languages Schlenker (2018a).</footnote>
musical properties license inferences that fall in between these two categories. This may
be the case of consonance. Although consonant musical events have been shown to be
associated with more stability and pleasantness (Butler and Daston, 1968; Krumhansl
et al., 2000), one could argue that consonance is found in non-musical sounds as well. In
other words, the mechanisms that make listeners represent a given sound or aggregation
of sounds as consonant might apply to both music and sounds; yet there might be music-
specific ways to create consonance.

The results presented in this dissertation are essentially experimental results which
investigate the empirical reality of both previous concepts. First, we will test whether
listeners draw inferences about an extra-musical reality when listening to music, and
we will provide experimental data which support this view. Second, we will be testing
whether the semantic rules that define musical truth are accurate as stated, and we will
see that empirical data calls for a few updates of the notion of musical truth and truth
conditions. The last following section presents how we conducted this double inquiry.

**4 Outline**

The purpose of this dissertation is to provide insights as to how music semantics can be
revealed through different experimental paradigms. In particular, it aims at investigating
semantic effects from music in different environments. Because music semantics is very
abstract, we reasoned that having music interact with other representational systems or
other modalities should help reveal its semantic behavior in such contexts.<fnref n="16" />

It is first possible to analyze how the information conveyed by music behaves when in-
teracting with a linguistic environment. In 4.1, we explain how we designed an experiment
in which hybrid sentences containing music replacing words triggered inferences similar
to that found in language alone. A second interaction we explored was that between
music and visual animations, inspired by audiovisual tasks such as those in Blumstein
et al. (2012), Sievers et al. (2013) or Eitan (2013). In 4.2, we present how we confirmed
some semantic rules stated in Schlenker (2017) using audiovisual stimuli, and how this

<footnote n="16">¹⁶We are here using the intuition from Schlenker (2017) that “Semantic intuitions that would otherwise be very unclear can be sharpened by reducing the set of possible denotations.”</footnote>
helped determine which kind of things music refers to. But one might in the end be curious to know what happens in music alone, i.e. when it does not interact with any other system. In 4.3, we introduce a paradigm that aimed at testing music semantics directly: by focusing on how music could evoke walking, we managed to establish some limitations of the current model of music semantics. Finally, we present in 4.4 a last experimental paradigm investigating how musical properties interact with body movement beyond formal interactions with language and vision. The present dissertation is structured in the very same way: to each of the following section corresponds a chapter which includes one or two research articles.

## 4.1 Meaning from music embedded in language

The first interaction we investigated was that of music with natural language. Taking inspiration from Tieu et al. (2019), who showed that gestures and visual animations replacing words in sentences could generate the same inferences as language alone, we built an experiment in which music were replacing words in sentences and showed that participants drew the same inferences as well. Consider the sentence in (1).

(1) a. Mary stopped smoking.
→ Mary used to smoke.

b. Did Mary stop smoking?
→ Mary used to smoke.

The meaning of sentence in (1)a. has two components: an at-issue component (i.e. the very topic of discussion), which is that Mary stopped to smoke, and a non-at-issue component, which is not directly conveyed and is taken for granted, namely that Mary used to smoke. Crucially, turning the sentence into a question cancels the at-issue component: one does not understand from (1)b. that Mary stopped smoking (it is precisely what is now being questioned), but it is still taken for granted that Mary used to smoke. This non-at-issue component, which resists question formation among other logical operations, is called a presupposition.
Tieu et al. (2019) showed that it was possible to trigger presuppositions by means of pro-speech gestures and pro-speech visual animations, i.e. gestures or visual animations that replace words in sentences. For instance, the question in (2), which involved the pro-speech gesture *REMOVE_GLASSES* standing for the gesture of the speaker removing their glasses, presupposes that student under question currently has glasses on.

(2) Will the student *REMOVE_GLASSES*?

-> The student currently has glasses on.

Our first contribution is to extend this paradigm to non-linguistic auditory stimuli, and to test whether the same mechanisms work with pro-speech music. In other words, we tested whether replacing words with music still triggers the same kinds of inference that are found in language alone, and that were shown to exist in hybrid sentences containing visual information. In the case of presuppositions, for instance, the paradigm we used was the following. Imagine a context in which some hikers are going up and down mountains, and one of them asks the question in (3).

(3) Do you think the next hiker will *UPWARD_SCALE*

-> The next hiker is at the bottom of a mountain.

Question (3) contains a musical stimulus *UPWARD_SCALE*, standing for an embedded ascending scale played by a harp and replacing the verb phrase in the question, and intended to be understood as meaning something like ‘go up’.<fnref n="17" /> It then seems that question (3) does trigger a presupposition about the location of the next hiker, which is (i) not directly conveyed by the question, and (ii) still taken for granted.

Just as in Tieu et al. (2019), our experiment investigated whether similar mechanisms could generate further inferential types besides presuppositions. Our results showed that it was possible to replicate this typology by pro-speech musical means. Together with Tieu et al. (2019), they support the idea that at least some semantic mechanisms accounting for linguistic inferences are not exclusive to language and that they can also integrate

<footnote n="17">¹⁷Crucially, and following Tieu et al. (2019), we made sure that the musical stimuli were not merely translated into words, for if it were, the paradigm would fail to demonstrate anything as to whether music is interpreted when embedded in sentences.</footnote>
non-linguistic auditory information (be it musical or vocal) besides vision: processes underlying meaning creation in language are in fact not limited to linguistic items.

However, these results are not sufficient to establish the existence of a music semantics. Although they reveal that participants were able to retrieve some informational content from musical stimuli which they had never been exposed to before, these findings are consistent with other accounts that do not require semantic rules for the target inferences to be drawn. To get closer to the actual model of music semantics presented in section 3, we developed two further experimental paradigms, one involving an audiovisual task, and one involving a specific case-study. Both approaches aimed at restricting the investigation of musical meaning to cases where the semantics is more easily accessible to participants, either by forcing some semantic interpretations thanks to visual information, or by limiting the possible denotations to an identifiable event (in our case, that of someone walking).

## 4.2 Meaning from music interacting with visual scenes

The second paradigm we developed aimed at experimentally establishing some of the hypotheses provided in Schlenker (2017, 2022) about the interpretation of three musical properties, namely pitch, dynamics (or loudness), and timbre, through the interaction between musical sounds and visual animations. Although a great deal of work has been done on establishing correlations between these musical properties and several possible interpretations, these correlations have never been tested under explicit rules providing formal conditions on music.

By combining simple visual animations involving 3D objects and minimal musical stimuli, we created pairs of audiovisual scenes which were either compatible or incompatible with the predictions that followed from Schlenker (2017), and had participants select which of the two they preferred. We found that the results were broadly in line with the predictions of the model. In particular, participants systematically preferred the scenes that obeyed one of the following rules: (i) the higher the pitch, the higher the level of energy of the denoted object, and (ii) the louder the music, either the higher the level of
energy of the represented object, or the closer that object to a given viewpoint. A bit
more surprisingly, we also found evidence in favor of the following rule: (iii) the higher the
pitch, the closer the represented object. Although this association was not predicted and
conflicts with previous research such as that from Eitan and Granot (2006), it is also in
line with other experimental data (Eitan, 2013). We discuss further avenues for research
in the corresponding paper. Importantly, these correlations were tested through pairings
of dynamic changes in both musical properties and visual animations. This is essential
because the theory predicts that *changes* in musical properties map onto *changes* in the
denotation.<fnref n="18" />

A second part of our experiment was intended to demonstrate that not only were
pitch and dynamics interpreted as energy and distance when paired with one object, but
they can also serve to anchor reference to a specific object when multiple objects are
present. Let us take an example to illustrate this. In the first part of the experiment, we
established that listeners prefer to associate a crescendo (i.e. an increase in loudness) with
an object moving closer or gaining energy, rather than moving away or losing energy (rule
(ii)). Now, what happens if two objects are present in the visual scene? How do listeners
integrate the auditory information with the visual information? By combining musical
sounds with visual scenes involving two objects, we showed that listeners rely on the
following rule: (iv) the musical sound refers to the object that satisfies the properties of
that sound. In our example, only the object moving closer or gaining energy is associated
with the sound. In our paper, we argue that this is an instance of *coreference* across
media, in the sense that the denotation of the musical sound and that of the visual object
both refer to a same object, though through different modalities (namely audition and
vision).<fnref n="19" /> These results are consistent with predictions from theoretical work on reference
relations in music (Schlenker, 2022), as well as with previous work on the existence of
such relations in pictures (Abusch, 2013) and dance (Patel-Grosz et al., 2018).

A third and final part of this experiment aimed at clarifying the nature of the entities

<footnote n="18">¹⁸Although we showed in a later experiment, presented in 4.3, that this need to be systematic, this first approximation was needed to experimentally establish the reality of the semantic effects we tested.</footnote>

<footnote n="19">¹⁹Note that there exists coreference within media as well, e.g. typically when two linguistic items such as two pronouns refer to a same individual.</footnote>
music refers to. In section 3, we presented music semantics as relying on the core assumption that music can refer to the external world. In Schlenker (2017), music is claimed to convey information about non-musical objects; while in Schlenker (2022), this claim is updated to account for how music conveys information about situations. We thus tried to disentangle this point and provide an argument showing that musical properties are bound to properties of objects, and not just to properties of situations or scenes containing objects. To show this, we created timbre/object associations in audiovisual scenes and showed that the scenes in which that association was consistent were preferred. In short, it is not enough that some object in an audiovisual scene satisfies the properties of the musical sound: in our case, one and the same object has to satisfy both properties on timbre and dynamics to produce an acceptable audiovisual scene. Although subtle, this point is of particular relevance for subsequent developments in music semantics, as it suggests that each semantic rule should account for how a given musical property is interpreted at the situation level but also at the object level.

## 4.3 Musical meaning from music alone

Although an audiovisual paradigm was useful to establish some semantic associations from music by appealing to the visual modality, it is essentially unable to account for how music triggers inferences about possible objects when music is heard alone. The third paradigm we present here aimed at testing how listeners interpret music in the absence of any interaction with another system conveying meaning. But because of the abstract nature of music semantics (a given musical sequence can be true of many different objects), we still had to find a way to constrain the set of possible inferences that music licenses, and we developed a case-study on how music can represent someone walking. This case study was motivated by previous substantive work on (i) the correlations established between music and representations of movement, (ii) the intricate relationship between music and actual body movement, for instance in the case of dance, and (iii) the more specific historical and empirical relationship that holds between music and walking (such as in military marches).
This experiment had four main goals. First, it tested whether some music could evoke
a walking character better than others. Under the assumption that music has an extra-
musical meaning and can refer to objects through certain rules, our prediction was that it
should. Based on introspective judgments and taking inspiration from musical sequences
accompanying walking events (such as in cartoons), we built a reference stimulus which
listeners rated as being more evocative than control stimuli which were hardly associated
with the same event. Although insufficient to demonstrate that our stimulus directly
encoded reference to a character walking, these preliminary findings showed that to the
least, the stimulus licensed an interpretation as a walking character.

Second, the experiment aimed at identifying the musical formal properties responsible
for this association. Through the rewriting of a short musical sequence and the minimal
destruction of each identified property, we showed that at least five properties were in-
volved in making reference to a walking character. For a musical sequence to optimally
denote a walk, it needed to be reducible to a (i) steady (ii) alternation of (ii) two chords,
which were both (iv) relatively minimally distant in the tonal space and (v) absolutely
consonant. Participants systematically picked the stimulus satisfying all these properties
as evoking a character walking the best, over any stimulus satisfying only a subset of
these properties.

Third, we show that these properties could not all be derived from the model presented
in Schlenker (2018b). As detailed in Section 3, this model is based on preservation rules
which associate a change in a musical property to a change in the associated object
property. For instance, an increase in loudness is associated with a decrease in distance
to the object, or an increase in the energy level of that object. But these rules have
been argued to be sometimes to weak to account for more fine-grained musical inferences
(Migotti, 2019). In particular, this model does not make predictions as to whether some
musical properties are interpreted in absolute terms, for instance whether listeners can
directly map an atomic musical event with a given loudness level to a given distance or
level of energy. Our experiment showed that there seem to exist such absolute mechanisms
for the interpretation of consonance: it is for instance required that any physically stable
event is musically represented by a consonant musical event. Meeting the requirements
on the relative consonance level of musical events representing the different steps of a
walking event is not enough: each step, which is itself a physically stable event, has to
be represented by a consonant musical event (such as consonant chord).

Fourth, this experiment proposed to experimentally test a last theoretical dimension
of the theory, namely that each musical event has to be interpreted (as an event).<sup>20</sup> While
in Schlenker (2017), the assumption was made that each musical event is interpreted as
an event, we wanted to test our intuition that some musical events are semantically
'empty', and that they might act as modifiers on musical meaning instead of being fully
interpreted as independent events. We thus enriched the stimulus used throughout the
whole experiment by adding an extra-note in the musical sequence, resulting in a stimulus
still evoking a walk but sounding bouncier, more light-hearted. Our prediction was that
if it is was true that some musical events are not interpreted, then the enriched stimulus
would still license an inference about a character walking. Contrary to our predictions,
we found that this minimal modification altered the semantic association with a walking
character, suggesting that it changed musical meaning more significantly than predicted.

Together, these findings call for a more systematic investigation of case studies in
which the semantic associations between some musical pieces and what they evoke in
listeners are broken down into all musical properties and rules involved.

4.4 Music-induced body effects

Finally, once it has been established that it is possible to constrain listeners’ inferences
by identifying rules mapping a musical sequence to a possible set of situations (such as
situations in which someone is walking), one might finally be interested in whether the
same sequences generate the same inferences in a listener who physically experiences
walking. What happens when a listener starts walking on a musical sequence which also
evokes walking? Some musical properties are indeed not only associated with properties of
extra-musical objects, they also correlate with motor/physical effects. Some authors have

<footnote n="20"><sup>20</sup>For a detailed discussion of the notion of event in music semantics and its theoretical implications, see Zaradkzi (2021).</footnote>
claimed that musical meaning is even primarily grounded in bodily experience (Brower, 2000) or physiological experience (Antovic, 2009). On this view, characterizing some music as ‘tense’ simply results from an actual muscular tension. This line of work directly relates to investigations of unconscious physical reactions triggered by the recognition of acoustic patterns. For example, Bedoya et al. (2021) showed that applying an acoustic filter modeled after the modification that smiling induces in the voice to a violin sound not only makes that sound judged as happier, but also triggers subtle muscle reactions in listeners similar to those that occur when they actually smile. Others have called for the integration of a notion of ‘self-motion’ to explain how we generate motion representations from music: Clarke (2001) suggested that a possible reason for which listeners may feel movement when listening to music is because the processing of some pitch structures produces biological effects to the ear which are similar to those experienced when actually moving.

Although our first idea was to test how the stimuli used in the experiment presented in 4.3 shape listeners’ gait patterns, we realized that there was not enough data in the literature on the physical and motor reactions to changes in some musical properties. In particular, while a considerable amount of evidence had established music/motor time-related correlations (for instance, people adjusting their pace to match a musical beat, or more generally following changes in tempo), none was available as to the effect of time-unrelated dimensions (such as pitch height or harmony) on gait specifically. Komeilipoor et al. (2015) had however shown that dissonances were responsible for creating irregularities in a tapping task, leaving open the question of whether similar effects could be found for other movements. We therefore built a paradigm testing for the effect of pitch height on gait patterns using sequences of notes randomly varied in pitch on which participants were tasked to walk. We found that the lower the pitch, the longer the steps duration, and the stronger the stepping force applied onto the ground.

Note that these associations could occur in the absence of a music semantics: it does not seem to require any kind of semantic system for the physiological effect to arise in the first place. Still, the question of how these physical effects interact with musical
semantic representations is puzzling. A first possibility is that listeners simply try to mimic their semantic interpretation of musical sounds, as if music was understood to be about themselves. This could explain why they unconsciously ‘align’ their walking patterns with how they represent walking events corresponding to certain pitch ranges and patterns: listeners might map the pitch properties of a musical excerpt to properties of objects, and then somehow act as if those properties applied to themselves. For instance, one hears something that generates a representation of a big object, which then has motor effects akin to those which would occur if the listener themselves were bigger. We discuss some reasons for which such a far-fetched mechanism could exist in the corresponding paper, in particular in light of similar mechanisms pertaining to rhythmic synchronization (Styns et al., 2007). It is also possible that some musical properties (such as pitch) and some movements (such as walking) trigger mental representations which overlap: under this view, the semantics does not necessarily come first, and then affects the body, but both mechanisms could just interact at the same time. In any case, our findings are surprisingly consistent with some theories of pitch height stating that it encodes object size: we did find that participants stepped more heavily on low pitches, which have been shown to be associated with large objects (Eitan and Timmers, 2010). Although this paradigm only established a correlation from music to gait patterns, it raises interesting questions as to whether and how music semantics interacts with other music-induced phenomena, and opens fruitful research perspectives on how these effects can be used in gait rehabilitation.
# 5 Bibliography

Abusch, D. (2013), ‘Applying Discourse Semantics and Pragmatics to Co-reference in Picture Sequences’, *Proceedings of Sinn und Bedeutung 17*, 9–25.

Antovic, M. (2009), ‘Towards the Semantics of Music: the 20th Century’.

Bedoya, D., Arias, P., Rachman, L., Liuni, M., Canonne, C., Goupil, L. and Aucouturier, J.-J. (2021), ‘Even violins can cry: specifically vocal emotional behaviours also drive the perception of emotions in non-vocal music’, *Philosophical Transactions of the Royal Society B: Biological Sciences* **376**(1840), 20200396.

Bernstein, L. (2005), *Leonard Bernstein's young people's concerts*, 1st amadeus press edn, Amadeus Press, Pompton Plains, N.J. OCLC: ocm61445826.

Blacking, J. (1976), *How musical is man?*, Faber and Faber, London.

Blumstein, D. T., Bryant, G. A. and Kaye, P. (2012), ‘The sound of arousal in music is context-dependent’, *Biology Letters* **8**(5), 744–747.

Bowling, D. L., Gill, K., Choi, J. D., Prinz, J. and Purves, D. (2010), ‘Major and minor music compared to excited and subdued speech’, *The Journal of the Acoustical Society of America* **127**(1), 491–503.

Bowling, D. L., Sundararajan, J., Han, S. and Purves, D. (2012), ‘Expression of Emotion in Eastern and Western Music Mirrors Vocalization’, *PLoS ONE* **7**(3), e31942.

Brower, C. (2000), ‘A Cognitive Theory of Musical Meaning’, *Journal of Music Theory* **44**(2), 323–379. Publisher: [Duke University Press, Yale University Department of Music].

Butler, J. W. and Daston, P. G. (1968), ‘Musical Consonance as Musical Preference: A Cross-Cultural Study’, *The Journal of General Psychology* **79**(1), 129–142.

Clarke, E. (2001), ‘Meaning and the Specification of Motion in Music’, *Musicae Scientiae* **5**(2), 213–234.

Cone, E. T. (1982), *The composer's voice*, 1. paperback pr edn, Univ. of Calif. Pr, Berkeley.

Cross, I. and Woodruff, G. E. (2009), Music as a communicative medium, in R. Botha and C. Knight, eds, ‘The Prehistory of Language’, Oxford University Press, pp. 77–98.

Davies, S. (1994), *Musical Meaning and Expression*, Cornell University Press.

Eitan, Z. (2013), How pitch and loudness shape musical space and motion, in ‘The psychology of music in multimedia’, Oxford University Press, New York, NY, US, pp. 165–191.

Eitan, Z. and Granot, R. Y. (2006), ‘How Music Moves’, *Music Perception* **23**(3), 221–248.
Eitan, Z. and Timmers, R. (2010), ‘Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context’, *Cognition* **114**, 405–422. Place: Netherlands Publisher: Elsevier Science.

Godøy, R. I. and Leman, M., eds (2010), *Musical gestures: sound, movement, and meaning*, Routledge, New York. OCLC: ocn298781501.

Granroth-Wilding, M. and Steedman, M. (2014), ‘A Robust Parser-Interpreter for Jazz Chord Sequences’, *Journal of New Music Research* **43**(4), 355–374.

Guerrini, J. (2020), ‘Vowel quality and iconic lengthening’, *Proceedings of Sinn und Bedeutung* pp. 242–255 Pages. Artwork Size: 242-255 Pages Publisher: Proceedings of Sinn und Bedeutung.

Hanslick, E. (1957), *The beautiful in music*, 8. print edn, The library of Liberal Arts, Indianapolis.

Heider, F. and Simmel, M. (1944), ‘An Experimental Study of Apparent Behavior’, *The American Journal of Psychology* **57**(2), 243.

Honing, H. (2003), ‘The Final Ritard: On Music, Motion, and Kinematic Models’, *Computer Music Journal* **27**(3), 66–72.

Huron, D. (2006), *Sweet Anticipation: Music and the Psychology of Expectation*, The MIT Press.

Ilie, G. and Thompson, W. F. (2006), ‘A Comparison of Acoustic Cues in Music and Speech for Three Dimensions of Affect’, *Music Perception* **23**, 319–329. Place: US Publisher: University of California Press.

Juslin, P. N. and Laukka, P. (2003), ‘Communication of emotions in vocal expression and music performance: Different channels, same code?’, *Psychological Bulletin* **129**(5), 770–814.

Juslin, P. N. and Sloboda, J. A., eds (2001), *Music and emotion: theory and research*, Series in affective science, Oxford University Press, Oxford ; New York.

Katz, J. and Pesetsky, D. (2011), ‘The Identity Thesis for Language and Music’.

Koelsch, S. (2012), *Brain and music*, Wiley-Blackwell, Chichester, West Sussex ; Hoboken, NJ. OCLC: ocn767563922.

Komeilipoor, N., Rodger, M. W. M., Craig, C. M. and Cesari, P. (2015), ‘(Dis-)Harmony in movement: effects of musical dissonance on movement timing and form’, *Experimental Brain Research* **233**(5), 1585–1595.

Krumhansl, C. L., Toivanen, P., Eerola, T., Toiviainen, P., Järvinen, T. and Louhivuori, J. (2000), ‘Cross-cultural music cognition: cognitive methodology applied to North Sami yoiks’, *Cognition* **76**(1), 13–58.

Larson, S. (2012), *Musical forces: motion, metaphor, and meaning in music*, Musical meaning & interpretation, Indiana University Press, Bloomington. OCLC: ocn707212791.
Lerdahl, F. and Jackendoff, R. (1983), *A generative theory of tonal music*, the MIT press, Cambridge, Mass. London.

Maus, F. E. (1988), 'Music as Drama', *Music Theory Spectrum* **10**(1), 56–73.

Mehr, S. A., Singh, M., Knox, D., Ketter, D. M., Pickens-Jones, D., Atwood, S., Lucas, C., Jacoby, N., Egner, A. A., Hopkins, E. J., Howard, R. M., Hartshorne, J. K., Jen- nings, M. V., Simson, J., Bainbridge, C. M., Pinker, S., O'Donnell, T. J., Krasnow, M. M. and Glowacki, L. (2019), 'Universality and diversity in human song', *Science* **366**(6468), eaax0868. Publisher: American Association for the Advancement of Sci- ence.

Meyer, L. B. (1956), *Emotion and meaning in music*, paperback ed., [nachdr.] edn, Univ. of Chicago Press, Chicago. Ill.

Migotti, L. (2019), Steps towards a theory of music semantics, Master’s thesis, Ecole Normale Supérieure.

Monahan, S. (2013), ‘Action and Agency Revisited’, *Journal of Music Theory* **57**(2), 321–371. Publisher: [Duke University Press, Yale University Department of Music].

Patel-Grosz, P., Grosz, P. G., Kelkar, T. and Jensenius, A. R. (2018), ‘Coreference and disjoint reference in the semantics of narrative dance’, *Proceedings of Sinn und Bedeutung* **22**(2), 199–216. Number: 2.

Rohrmeier, M. (2011), ‘Towards a generative syntax of tonal harmony’, *Journal of Mathematics and Music* **5**(1), 35–53.

Saslaw, J. (1996), ‘Forces, Containers, and Paths: The Role of Body-Derived Image Schemas in the Conceptualization of Music’, *Journal of Music Theory* **40**(2), 217–243. Publisher: [Duke University Press, Yale University Department of Music].

Schlenker, P. (2017), ‘Outline of Music Semantics’, *Music Perception* **35**(1), 3–37.

Schlenker, P. (2018a), ‘Iconic pragmatics’, *Natural Language & Linguistic Theory* **36**(3), 877–936.

Schlenker, P. (2018b), ‘What is Super Semantics?’ *, *Philosophical Perspectives* **32**(1), 365–453.

Schlenker, P. (2022), ‘Musical meaning within Super Semantics’, *Linguistics and Philosophy* **45**(4), 795–872.

Schulte-Tenckhoff, I. and Feld, S. (1988), ‘Sound and Sentiment. Birds, Weeping, Poetics, and Song in Kaluli Expression’, *Cahiers de musiques traditionnelles* **1**, 214.

Sievers, B., Lee, C., Haslett, W. and Wheatley, T. (2019), ‘A multi-sensory code for emotional arousal’, *Proceedings of the Royal Society B: Biological Sciences* **286**(1906), 20190513.

Sievers, B., Polansky, L., Casey, M. and Wheatley, T. (2013), ‘Music and movement share a dynamic structure that supports universal expressions of emotion’, *Proceedings of the National Academy of Sciences* **110**(1), 70–75.
Slobin, M. (1993), *Subcultural sounds: micromusics of the West*, Music/culture, University Press of New England, Hanover, NH.

Stravinsky, I. (1935), *An autobiography*, Norton, New York.

Styns, F., Van Noorden, L., Moelants, D. and Leman, M. (2007), ‘Walking on music’, *Human Movement Science* **26**(5), 769–785.

Tieu, L., Schlenker, P. and Chemla, E. (2019), ‘Linguistic inferences without words’, *Proceedings of the National Academy of Sciences* **116**(20), 9796–9801.

Trevarthen, C. (1999), ‘Musicality and the intrinsic motive pulse: evidence from human psychobiology and infant communication’, *Musicae Scientiae* **3**(1_suppl), 155–215.

Zaradkzi, L. (2021), Les évenements en sémantique linguistique et musicale, PhD dissertation, Université Paris Cité.
Chapter 2

Meaning from music embedded in
language

Purpose

We start our experimental investigation of musical meaning by exploring how
musical information behaves when music is embedded in language and used in
lieu of words in sentences. Previous theoretical and experimental work had
established that gestures and visual animations replacing words in sentences
can give rise to different inferences from the linguistic typology. Here, we
show that these findings extend to non-linguistic musical items: the mental
mechanisms generating meaning from language can also integrate musical
meanings when music is embedded in language.

The article contained in this chapter was published in the *Linguistics and Philosophy* special issue on Super Linguistics (2023). Reference is provided in the paper.
LINGUISTIC INFERENCES FROM PRO-SPEECH MUSIC
MUSICAL GESTURES GENERATE SCALAR IMPLICATURES, PRESUPPOSITIONS, SUPPLEMENTS, AND HOMOGENEITY INFERENCES *

Léo Migotti & Janek Guerrini
Institut Jean Nicod (ENS-EHESS-CNRS)

ABSTRACT

Language has a rich typology of inferential types. It was recently shown that subjects are able to divide the informational content of new visual stimuli among the various slots of the inferential typology: when gestures or visual animations are used in lieu of specific words in a sentence, they can trigger the very same inferential types as language alone (Tieu et al., 2019). How general are the relevant triggering algorithms? We show that they extend to the auditory modality and to music cognition. We tested whether pro-speech musical gestures, i.e. musical excerpts that replace words in sentences, can give rise to the same inferences. We show that it is possible to replicate the same typology of inferences using pro-speech music. Minimal and complex musical excerpts can behave just like language, gestures, and visual animations with respect to the logical behavior of their content when embedded in sentences. Specifically, we found that pro-speech music can generate scalar implicatures, presuppositions, supplements, and homogeneity inferences.

**Keywords** Semantics | Super semantics | Scalar implicatures | Presuppositions |
Supplements | Homogeneity inferences | Iconicity

This research received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 788077, Orisem, PI: Schlenker). Research was conducted at Institut d'Etudes Cognitives, Ecole Normale Supérieure - PSL Research University. Institut d'Etudes Cognitives is supported by grants ANR-10-IDEX-0001-02 and FrontCog ANR-17-EURE-0017. This paper is part of the Special Issue "Super Linguistics", edited by Pritty Patel-Grosz, Emar Maier and Philippe Schlenker.

***Contributions:*** JG designed the first stimuli, which LM generated and recorded. Subsequent stimuli were designed by LM and JG. JG and LM designed the experiment and wrote the paper. LM managed online data collection and statistical analyses.

<footnote>*This paper has been published: Migotti, L., Guerrini, J. (2023). Linguistic inferences from pro-speech music.
*Linguistics and Philosophy*. https://doi.org/10.1007/s10988-022-09376-9</footnote>
# Contents

<table><tr><td>1 Introduction</td><td>36</td></tr><tr><td>2 Methods</td><td>41</td></tr><tr><td>3 Replication of the typology of inferences</td><td>42</td></tr><tr><td>3.1 Scalar implicatures</td><td>42</td></tr><tr><td>3.1.1 In language and from visual stimuli</td><td>42</td></tr><tr><td>3.1.2 From music</td><td>43</td></tr><tr><td>3.1.3 Results</td><td>46</td></tr><tr><td>3.1.4 Discussion</td><td>46</td></tr><tr><td>3.2 Presuppositions</td><td>47</td></tr><tr><td>3.2.1 In language and from visual stimuli</td><td>47</td></tr><tr><td>3.2.2 From music</td><td>49</td></tr><tr><td>3.2.3 Results</td><td>50</td></tr><tr><td>3.2.4 Discussion</td><td>51</td></tr><tr><td>3.3 Supplements</td><td>54</td></tr><tr><td>3.3.1 In language and from visual stimuli</td><td>54</td></tr><tr><td>3.3.2 From music</td><td>55</td></tr><tr><td>3.3.3 Results</td><td>56</td></tr><tr><td>3.3.4 Discussion</td><td>57</td></tr><tr><td>3.4 Homogeneity inferences</td><td>58</td></tr><tr><td>3.4.1 In language and from visual stimuli</td><td>58</td></tr><tr><td>3.4.2 From music</td><td>59</td></tr><tr><td>3.4.3 Results</td><td>61</td></tr><tr><td>4 Iconicity controls</td><td>65</td></tr><tr><td>4.1 Experimental paradigm</td><td>65</td></tr><tr><td>4.2 Discussion</td><td>67</td></tr><tr><td>5 Conclusion</td><td>69</td></tr><tr><td>6 Acknowledgments</td><td>71</td></tr><tr><td>Appendix I: Musical gestures and vocal gestures</td><td>76</td></tr><tr><td>Appendix II: Statistical models</td><td>77</td></tr></table>
# 1 Introduction

Natural language semantics has long evidenced the existence of a rich typology of content, showing that the meaning of sentences can be split up into different content types (Schlenker, 2016; Kadmon, 2001). Let us take the example of presuppositions.<fnref n="2" /> Sentence (1)a conveys two kinds of information: the speaker asserts that John does not smoke now, while taking for granted that John used to smoke. Such inferences, presuppositions, survive in different logical environments. When turning the sentence into a question in (1)b, the assertion that John stopped smoking is now questioned, but the presupposition that he used to smoke remains.

(1) a. John stopped smoking.
    ⇌ John used to smoke.

b. Did John stop smoking?
⇒ John used to smoke.

It has been argued that such a projection pattern (namely, here, the fact that the presupposition is preserved under question formation<fnref n="3" />) is lexically encoded, i.e. determined on the basis of word meaning (Levinson, 2000; Abusch, 2002). The meaning of “stop” would for instance be divided as in (2).

(2) x stops Q-ing.
- At-issue: x does not Q.
- Presupposed: x Q-ed before.

Alternatively, some theorists have claimed that general algorithms can predict when an inference triggered by a given word is treated as a presupposition, in part because across languages the same projection patterns are observed (Simons et al., 2010; Tonhauser et al., 2013; Abusch, 2010; Abrusán, 2010; Schlenker, 2021). On this view, humans can divide asserted content from presupposed content productively and on the fly. Here, we propose to test the productivity of these mechanisms by making use of non-linguistic items such as music: if such general and productive procedures exist, then the same inferential patterns should arise

<footnote n="2">²Here we use the example of presuppositions as an illustration of the main argument of the paper. The argument however applies similarly to the three other inferences tested: scalar implicatures, supplements and homogeneity inferences. A summary of the different inferential mechanisms can be found in Appendix III.</footnote>

<footnote n="3">³Generally, the projection *problem* refers to the computation of the presupposition and asserted content of a sentence from the presupposition and asserted content of its constituents. By contrast, projection *tests*, such as the family-of-sentences test (so-called in Chierchia and McConnell-Ginet (2000)), i.e. embedding under negation, modality or question, are used to check whether a given proposition is a presupposition (Heim, 1990, 1992; Stalnaker, 1974; Geurts, 1999; Beaver, 2001; Chemla, 2009)</footnote>
Here, `REMOVE_GLASSES` stands for the gesture mimicking someone removing their glasses. Sentence (3)a presupposes that, at the time of the utterance, the student has glasses on. Similarly, the question in (3)b still presupposes that the student has glasses on, just like the presupposition that John used to smoke was preserved under question in (1). Subjects were shown to be able to access this presuppositional content significantly more than control inferences. In general, Tieu *et al.* found that embedded visual stimuli can give rise to the same inferential types as purely linguistic ones. This suggests that the algorithm humans apply to divide at-issue content from non-at-issue content must be productive, and, because the stimuli were visual, it cannot be limited to language narrowly intended.

How general is the algorithm that divides at-issue content from non-at-issue content in the case of presuppositions, and how general are other inferential algorithms producing, for instance, scalar implicatures, supplements, or homogeneity inferences? Here we ask whether they extend to the auditory modality and to music cognition. We used pro-speech music, i.e. musical excerpts that replace words in sentences, henceforth musical gestures.<fnref n="4" /> Building our paradigm after Tieu et al. (2019), we tested whether musical gestures can give rise to the same inferential typology as language, pro-speech gestures and pro-speech visual animations. For example, in a context involving a person that was hiking, sentence (4) behaved just like a change-of-state verb like “stop”, and triggered the presupposition that the hiker is down a mountain both when embedded in a declarative sentence as in (4)a, and when it is embedded in an interrogative sentence as in (4)b.<fnref n="5" />

(4) a. The hiker will `UPWARD_SCALE`.<fnref n="6" />
    ⇄ The hiker is at the foot of a mountain.

b. Will the hiker `UPWARD_SCALE`?

⇔ The hiker is at the foot of a mountain.

<footnote n="4">⁴Although ‘pro-speech music’ [literally, music replacing words] is a specific kind of musical gestures [i.e. the iconic musical motives or excerpts roughly used in lieu of gestures in Tieu et al. (2019)], we use ‘pro-speech music’ and ‘musical gestures’ interchangeably throughout this paper.</footnote>

<footnote n="5">⁵Except for the paradigm testing supplements in section 3.3, we mainly used basic scales, drum sounds or isolated tones. Our definition of these as music could be contested because of their simple nature. However, even if these stimuli did not count as music, our claims on the generality of the algorithm that divides content in at-issue and non-at-issue would remain unaltered.</footnote>

<footnote n="6">⁶All musical gestures can be directly accessed by clicking on the hyperlinks.</footnote>
UPWARD_SCALE stands for a minimal musical stimulus of a classical C major scale played
by a harp. Each note of the scale was played one after the other following a standard rise in
frequency at a 990 notes/s rate to approach the sound of a real harp *glissando*, often used in
classical music. In general, because of the intuitive mapping between gesture and meaning
(Schlenker, 2019a), the informational content of iconic gestures can be grasped even in absence
of previous exposure (Schlenker, 2017). The reader has plausibly never been exposed to (4)a or
(4)b, but the content of the musical gesture UPWARD_SCALE can still be productively divided
between at-issue content, *viz.* the hiker is going up, and presuppositional content, *viz.* at the
time of utterance, the hiker is located at a low point in space.<fnref n="7" /> Crucially, the inference goes
through even though (4)b is a question, which is a classical test for presupposition projection.
Besides presuppositions, we try to replicate the typology of inferences already replicated
with gestures and visual animations in Tieu et al. (2019), including scalar implicatures,
supplements, and homogeneity inferences.

Here we are merely interested in the semantic and pragmatic behavior of the informational
content of musical sounds. This paper does not aim at deciding between different theories of
the respective inferences we tested; rather, it reports experimental data that any theory should
account for.<fnref n="8" /> Still, in some cases these results make it possible to exclude a significant class of
theories, as for instance in the case, mentioned above, of lexical theories of presuppositions.

Neither is the paper about how the content of pro-speech music relates to the actual
musical properties of the stimuli, i.e. about musical meaning and how it is derived. Still,
we provide first insights into how this meaning can interact with the logical structures of
language, which suggests the existence of a non-trivial informational content in music. We
leave open the challenging question of whether music triggers more sophisticated content,
especially outside of a linguistic context.

<footnote n="7">⁷As pointed out by an anonymous reviewer, we cannot know when precisely participants drew the inference. However, whenever the inference is actually triggered, it is unclear how our predictions would be different at this stage. As the same issue was present in Tieu et al. (2019), we assume that our predictions would not have been different.</footnote>

<footnote n="8">⁸For instance, our data does not allow us to decide between a grammatical or a neo-Gricean theory of scalar implicatures, but allows us to claim that regardless of the details of the algorithm responsible for scalar implicatures, this algorithm extends to music cognition and must therefore be domain-general.</footnote>
Before we move on, let us address a potential worry concerning the methodology used here and in Tieu et al. (2019). There is the possibility that pro-speech music is systematically translated into words. In this case, there would be the possibility that the relevant inferences arise because they are lexically encoded in the words of the translation. Consequently, our results would be uninformative about the generality of the algorithm dividing content in at-issue and non-at-issue. Two reasons militate against this hypothesis. First, just as in Tieu et al. (2019), subjects of our experiment were able to interpret fine-grained gestural iconic information that was absent from the words of the closest verbal translation. Second, such verbal translations would make iconic dimensions at-issue when not encoded in a verbal translation that lexically makes them non-at-issue, a behavior that seems to be excluded by the logical tests we provide in section 4.

This paper is structured as follows. In section 2, we detail the experimental setup. In section 3.1 we present the paradigm we used to test for scalar implicatures. Two stimuli involving respectively one and three repetitions of a drum sound competed and gave rise to scalar implicatures. In section 3.2, we present the paradigm we used to test for presupposition projection, already introduced in (4). While the presupposition was preserved under question formation, participants did not behave as expected in the classical test under “none”. We discuss in detail possible explanations of the difference between our results and Tieu et al.’s. We then move on to other linguistic inferences and introduce a paradigm testing for supplements in section 3.3. Pro-speech music did indeed behave like a supplement, yielding the typical conditional projection. In section 3.4, we test homogeneity inferences from pro-speech music. There are different theoretical accounts of such inferences: some attribute the existence of homogeneity inferences to the noun phrase, others to the predicate. We prove “by case” that pro-speech music gives rise to homogeneity inferences both when it replaces the noun phrase and when it replaces the predicate. Finally, in section 4, we discuss in detail why it is unlikely that pro-speech music is systematically translated into words.
## 2 Methods

Stimuli were recordings of French sentences with musical gestures (either specifically generated for our purposes or pre-existing) either artificially generated through GarageBand (version 10.3.5) or taken from real music replacing one or some words.

We collected data on 68 participants. We excluded non-native French speakers, participants who had already taken one of our pilot experiment and participants who had failed the two attention checks, which left us with 53 subjects. Participants were recruited through the French platform Crowdpanel, and online informed consent was obtained for each of them. After a short training on three examples of hybrid sentences containing music, and for each stimulus, each participant had to assess to what extent the presented inference followed from the auditory stimulus (i.e. the spoken sentences with embedded music), using a slider bar ranging from 0, labeled *totally disagree* (in French: *pas du tout d'accord*), to 100, labeled *totally agree* (in French: *tout à fait d'accord*) (cf. Figure 1)<fnref n="9" />. Stimuli played automatically for each trial, but participants could listen to the stimulus as many times as needed. The experiment was set up on Qualtrics. All stimuli were fully randomized. In each section, we report comparisons of generalized mixed-effects models of the data using R (version 3.6.2.) to assess the contribution of each factor of interest to the model (R core Team (2016); Barr et al. (2013)). The details, justifications, simplifying procedures and coefficients of the models are available in Appendix II. The details of the stimuli are given in the next sections, for each type of inference.<fnref n="10" />

<footnote n="9">⁹As pointed out by a reviewer, it cannot be ruled out that when asked about inferences, subjects understand that they must guess the right word in the stimuli, e.g. ‘climb’. Indeed, the target sentence was visible while listening to the stimulus, just like in (Tieu et al., 2019). Since this issue applies to all of this literature, we leave it as a problem for future research to understand if and how participants consider the lexical material from the target sentence to be relevant to the comprehension of the stimulus.</footnote>

<footnote n="10">¹⁰All material including stimuli, design files and analysis scripts are accessible at https://osf.io/hw45u/?view_only=89f983db777f49e9a6f5b41b3dea60d6. Material was uploaded prior to the beginning of the data collection - See PREREGISTRATION folder. Final results and statistics are available in the RESULTS folder.</footnote>
<figure><img src="image_6.png" /><figcaption>Figure 1: An example of a stimulus discussed in 3.2</figcaption></figure>

# 3 Replication of the typology of inferences

## 3.1 Scalar implicatures

### 3.1.1 In language and from visual stimuli

Scalar implicatures convey an implicit, strengthened meaning beyond the explicit or literal meaning of an utterance. Originally, they were viewed as arising from pragmatic reasoning on the speaker's communicative intentions (Grice, 1975). According to neo-Gricean approaches (Horn, 1972; Sauerland, 2004; van Rooij and Schulz, 2004; Chierchia et al., 2012), if a sentence *S* evokes *S'* and *S'* is more informative than *S* (or not less informative than *S*, depending on the specific approach), speakers infer that *S'* is false. For instance, in (5)a the speaker chose to utter “some” instead of the logically stronger “all”, and is therefore inferred to mean *some but not all* (Sauerland, 2012).

More recently, some theorists have argued that the mechanism responsible for scalar implicatures is instead a grammatical one (Chierchia et al., 2012). On this view, a silent operator with a meaning very similar to *only* is applied to scalar sentences: the inference in (5)c is the result of interpreting “some” in (5)a as *only some*, as shown in (5)b.

(5) a. He read some of the books.

b. *Exh* (He read some of the books.)

c. He read some, but not all of the books.
Either theory has at its core competition among alternatives interacting with informativity considerations.<fnref n="11" /> Scalar implicatures were recently shown to arise between alternatives realized by means of a gesture or a visual animation provided in a context (Tieu et al., 2019). For instance, in the positive environment in (6) where `TURN_WHEEL` stands for the gesture mimicking a driver turning a wheel, we understand that `TURN_WHEEL` does not only mean *to turn* but *to turn somewhat, but not a lot*. `TURN_WHEEL` competes with the more informative gesture `TURN_WHEEL_COMPLETELY`, which is not used, thus taken to be false.

(6) a. He will `TURN_WHEEL_COMPLETELY`. He will turn the wheel completely.

b. He will `TURN_WHEEL`. He will turn the wheel, but not completely.

The inference in (6)b. could be explained by the fact that `TURN_WHEEL` semantically means *to turn somewhat, but not completely*. In this case, however, we would expect *not-TURN_WHEEL* to mean *to not turn somewhat*.

(7) a. He will not `TURN_WHEEL_COMPLETELY`. He will turn the wheel, but not completely.

b. He will not `TURN_WHEEL`. He will not turn the wheel at all.

We rather understand from (7)b that the wheel was not turned at all. The informativity pattern gets reversed under negation, just like in language. *not-TURN_WHEEL* is now logically stronger than *not-TURN_WHEEL_COMPLETELY*. The inferences in (7) show that there is competition between the two gestures.

### 3.1.2 From music

We reasoned that realizations with different numbers of repetitions of a sound could be represented by speakers as a logical scale (Horn, 1972). We generated two different realizations

<footnote n="11">¹¹For presentational clarity, we choose to present scalar implicatures by going through the Gricean reasoning. We then mention the alternative theory, the grammatical account of scalar implicatures. In any case, these should be viewed as placeholders for any theory of scalar implicatures. Note that any theory of scalar implicatures predicts that if alternatives are provided in the context, implicatures should be derived. This is, in a way, a sanity check to confirm that the inferential mechanisms work as expected with pro-speech sounds, and that there can be competition among musical excerpts.</footnote>
of the timpani sound DRUM (from GarageBand instruments library): the weaker DRUM×1 and the stronger DRUM×3. We first set up a context introducing the stronger alternative:

(8) **Context (positive environment):** Jean boxes regularly at the gym. During last week's workout, he had a lot of energy, and was able to DRUM×3.

In the target premise (9), DRUM×1 evokes the more informative alternative DRUM×3, introduced in the context in (8):

(9) **Target premise**

This week, Jean will DRUM×1.

**Original stimulus**

Cette semaine, Jean va DRUM×1.

This week, Jean will DRUM×1.

If DRUM×1 forms a scale with DRUM×3, the stronger alternative DRUM×3 should be taken to be false if not realized. We expect speakers to draw the inference that John will box somewhat, but not a lot:

(10) **Target inference**

This week, Jean is going to box somewhat, but not a lot.

In the control stimulus in (11), we did not vary the number of repetitions:

(11) **Control premise**

This week, Jean will DRUM×3

**Original stimulus**

Cette semaine, Jean va DRUM×3.

This week, Jean will DRUM×3.

Because in (11) the informativity of the stimulus is not manipulated, we expect speakers to stick with the context and endorse the baseline inference in (12) when given the control premise.

(12) **Baseline inference**

This week, Jean will box a lot.

Just like for linguistic stimuli, the critical test is negation. Without such a test, the expected inference from the positive environment in (12) may be explained otherwise. Namely, DRUM×3 may convey that Jean will box a lot simply because it means *to punch something exactly three times*, and not because it constitutes a logically stronger alternative than
DRUM×1. To rule out this alternative explanation, we set up a context in which both alternatives were introduced:

(13) **Context (negative environment):** Jeanne is boxing at the gym. At last week's session, she had a lot of energy, and was able to DRUM×3. But during the second week of training, she did not DRUM×1.

If the two realizations form a scale, under negation, informativity should be reversed: not-DRUM×1 constitutes a more informative alternative with respect to not-DRUM×3:

(14) **Target premise**

This week, Jeanne will not DRUM×3.

**Original stimulus**

Cette semaine, Jeanne ne va pas DRUM×3.

This week, Jeanne NEG will not DRUM×3.

If DRUM×3 means *to punch something exactly three times*, we expect its negation to mean *not punch something exactly three times*, i.e. two or less, or four or more. If on the other hand the two realizations form a scale, as we argue, we expect *not-DRUM×3* to convey that although not a lot, some boxing still occurred, as in (15) below.

(15) **Target inference**

This week, Jeanne will box somewhat, but not a lot.

And similarly for the weaker realization: if DRUM×1 means *to punch something exactly once*, then *not-DRUM×1* should mean *not punching something exactly once*. This yields an inference pattern different from what is expected under our hypothesis that *not-DRUM×3* forms a scale with *not-DRUM×1*. In this case, *not-DRUM×1* should convey that there was no boxing at all.

(16) **Control premise**

This week, Jeanne will not DRUM×1.

**Original stimulus**

Cette semaine, Jeanne ne va pas DRUM×1.

This week, Jeanne NEG will not DRUM×1.

We thus expect *not-DRUM×1* to be interpreted as *did not box at all* rather than *did not punch something exactly once*, which would allow for boxing more than once:

(17) **Baseline inference**

This week, Jeanne will not box at all.
### 3.1.3 Results

A scalar implicature triggered by the target premise would lead participants to endorse the target inference more than the baseline inference. Such a difference could be due to an *a priori* preference for the target inference, independently of the target premise. For that reason, we looked at the interaction between target *vs* control premise and target *vs* baseline inference. We did find a significant interaction between the two factors that rule out this possibility in both environments ($\chi^2 = 170, p < 0.001$ in the positive environment; $\chi^2 = 21, p < 0.001$ in the negative environment), compatible with the triggering of a scalar implicature resulting from the competition of two musical alternatives, namely DRUM×1 and DRUM×3.

<figure><img src="image_7.png" /><figcaption>Figure 2: Mean endorsement rate for scalar implicatures</figcaption></figure>

### 3.1.4 Discussion

In the positive environment, as expected, DRUM×1 competes with the more informative alternative DRUM×3 so that DRUM×1 is understood as *to box somewhat but not a lot*. In the negative environment, not-DRUM×3 competes with not-DRUM×1. Thus not-DRUM×3 was interpreted as *box somewhat, but not a lot*. This shows that when two realizations have different logical informativity, they compete with each other just like alternatives compete in
the case of gestures and visual animations in Tieu et al. (2019), leading to the generation of scalar implicatures by means of non-purely linguistic means.<fnref n="12" />

## 3.2 Presuppositions

### 3.2.1 In language and from visual stimuli

A presupposition is a meaning component that is taken for granted that must follow from the local context of an expression, and which sentences inherit across different logical operators, such as negation (Stalnaker, 1974; Heim and Kratzer, 1998). In language, some particular expressions are associated with the triggering of presuppositions. For instance, in (18), the presupposition that John used to smoke is triggered by “stop”. In general, change-of-state verbs (e.g. “start”, “become”, “stop”) all presuppose their initial state (Abrusán, 2010). For John to be able to stop smoking, John must have been smoking to begin with.

Presuppositional content is untouched by logical operations that change the meaning of a sentence, such as negation, modality, and questions (Abusch, 2010; Abrusán, 2011; Chemla, 2009). For instance, the question in (18)b still presupposes that John used to smoke.

(18) a. John stopped smoking.

⇒ John used to smoke.

b. Did John stop smoking?

⇒ John used to smoke.

Tieu et al. (2019) found that presuppositions could also be generated by means of pro-speech gestures and visual animations. First, they were preserved under question formation:

(19) **Context** Aliens are green. But when they are in a meditative state, their antennae are blue. There is a meditation session in progress on the first floor of an architecture firm. Jane is watching the union representatives and says:

<footnote n="12">¹²In our stimuli, alternatives were provided in the context: the alternative, e.g. DRUM×3, to the musical gesture, DRUM×1, was salient in the context. For this reason, our results don't speak to the issue of alternative generation. Our point is that scalar implicatures can be triggered by pro-speech music as long as two alternatives are available. In other words, whatever the mechanism responsible for alternative generation, subjects are able to interpret one musical alternative as logically stronger than the other, have it compete with the other, and finally draw a scalar implicature from this process. However, if scalar implicatures were in fact to arise when alternatives are not directly given, then a general theory of alternatives could be needed. This is an exciting question for future research. Schlenker (2020) discussed such a theory in relation to Katzir (2007) in the case of gestures. As to music, it would not be surprising that implicatures are triggered whenever a musical gesture competes with a contextually salient (but not explicitly given) more informative alternative.</footnote>
(20) “Will the union representatives’ antennae GREEN_TOশBlue?” <fnref n="13" />
&(animation content: *bar is green at first, then slowly whole bar goes blue*)

⇒ The union representative is currently not in a meditative state.

→ The union representative is currently in a meditative state.

The context used in (19) introduces the two states between which the transition denoted by the visual animation GREEN_TOশBLUE can be made. When turning from green to blue, the embedded visual animation triggers the presupposition that the initial state denoted by the green color holds before the change of state takes place, i.e. the individual is not in a meditative state at the moment of utterance.

Evidence for presuppositions being triggered by visual animations was found under “none”,
too, as shown in (21) below.

(21) “None of the union representatives’ antennae will GREEN_TOBLUE”

⇒ None of the union representatives are currently in a meditative state.

→ Some of the union representatives are currently in a meditative state.

Tieu *et al.*'s (2019) choice to test for presupposition under “none” instead of under negation was motivated by Chemla (2009) results on strong universal inferences under “none” in French. This choice has significant theoretical advantages. Some apparent presuppositions that project out of negative environments such as (22)a. may be due to a scalar implicature and may not be strong evidence for presuppositional content. In (22) TURN_WHEEL_COMPLETELY stands for the gesture mimicking a driver turning a wheel completely. The fact that the agent is in front of a steering wheel may come as a consequence of the scalar implicature that although there was no complete turning of the wheel, there was still some turning (Tieu et al., 2019). For there to be some turning, the agent must be in front of a wheel in the first place.

(22) He did not TURN_WHEEL_COMPLETELY.

a. The agent is in front of a steering wheel.

Under "none", however, scalar implicatures project existentially, not universally, as shown
in (23) below (Chemla, 2009).

<footnote n="13">¹³The original sequence of sentences and the visual animations can be accessed at Tieu et al.’s supplementary materials page: https://mfr.au-1.osf.io/render?url=https://osf.io/v5xa3/?direct%26mode=render%26action=download%26mode=render</footnote>
(23) No student read all the books.

⇒ (At least) one student read (at least) some of the books..

The universal projection under ‘none’ found by Tieu et al. (2019) and reported in (21) thus constituted very strong evidence in favor of presupposition projection.

### 3.2.2 From music

We tested whether pro-speech music could generate presuppositions. Consider the context below:

(24) **Context**: Some hikers are hiking in the mountains, where there can be significant drops and peaks in elevation. They alternate between reaching the top and the foot of steep rocks. Two of them, who finished first, are talking while waiting for the others.

We used an upward scale played by a harp to evoke a hiker going up a mountain, as in (25).

(25) **Target premise (question)**

One asks the other: “Will the next hiker UPWARD_SCALE?”

**Original stimulus:**

L' un demande à l' autre : “Est ce que le prochain randonneur va
The one asks to the other : “Is it that the next hiker will
UPWARD_SCALE?”
UPWARD_SCALE?”

Question formation is a classical test for presupposition (Chemla, 2009; Tieu et al., 2019). In (25), if the initial state of the musical scale is presupposed rather than at-issue, the sentence should presuppose that the hiker is at the bottom of a rock. We thus expect the target inference in (26) to be more endorsed than its negation, the baseline inference in (27).

(26) **Target inference**

The next hiker is at the foot of a rock.

(27) **Baseline inference**

The next hiker is not at the foot of a rock.

Tieu *et al.* (2019) chose to test universal projection under “none”, and we follow suit to have a clear point of comparison between acoustic and visual gestures.
(28) **Target premise under “None”**

One tells the other: “None of the hikers will **UPWARD_SCALE**”.

**Original stimulus**

L' *un dit à l' autre* : “Aucun des randonneurs ne va **UPWARD_SCALE**.

The one tells to the other : “None the hikers NEG will **UPWARD_SCALE**.

Given Tieu *et al.*'s 2019 results, we thus expected the universal inference in (29) to be
more endorsed than its negation, the baseline inference in (30).

(29) **Target inference**

Each of the hikers is at the foot of a rock.

(30) **Baseline inference.**

Not all of the hikers are at the foot of a rock.

3.2.3 Results

As explained in Appendix II, here we expected a significant effect of inference type on the
endorsement of the premise: the inference containing the presupposition was expected to be
significantly more endorsed than the baseline inference negating the presupposition. Under
question formation, as expected, we found a significant effect of inference type ($\chi^2 = 6, p <$
0.05), compatible with the triggering of a presupposition. We also found a significant effect of
inference type in the ‘none’ environment, ($\chi^2 = 22, p < 0.001$), where, a bit more surprisingly,
the baseline inference that did not contain the presupposition was more highly endorsed
than its counterpart containing the projected presupposition. We discuss some hypotheses
regarding this pattern in the following subsection.
<figure><img src="image_8.png" /><figcaption>Figure 3: Mean endorsement rate for presuppositions</figcaption></figure>

### 3.2.4 Discussion

In the interrogative environment, as expected, participants judged the initial state of UPWARD_SCALE to be true at the time of the utterance. Participants computed the initial state as non-at-issue, a behavior that matches the inferential pattern of change-of-state verbs. To wit, if the initial state had been at-issue, the question would have targeted the information that the hiker is at the bottom, amounting to something like “Is the hiker at the bottom and will he go up?”. Instead, the question only targeted the change of state, asking something like “Will the hiker go up?”. This supports our hypothesis that there is a productive algorithm at work in these cases, as subjects had never been exposed to such a sentence.

By contrast, UPWARD_SCALE embedded under “none” did not give rise to the expected universal inference that “Each of the hikers was at the foot of a rock”. We discuss several reasons for this observed pattern below.

First, it is worth mentioning that the data on projection under “none” are in part disputed: initially, Heim (1988) predicted universal projection, while Beaver (1994) predicted existential projection. Recent approaches find a complex picture (Zehr et al., 2016). We briefly review the theoretical insights and empirical evidence on this topic from Chemla (2009). Consider sentence (31), but see Zehr et al. (2016) for a complete review.
(31) None of the players stopped smoking.

a. At least one player used to smoke and none of them stopped smoking.
b. All of the players used to smoke and none of them stopped smoking.
c. None of the players both used to smoke and stopped smoking.
d. None of the players (who smoke) stopped smoking. $|= \text{All of the players who smoke}$
 smoke. $|= \text{At least one player smokes and none of them stopped smoking.}$

Although all theories of presupposition can derive more or less straightforwardly both
types of projection, there is disagreement on whether the existential or the universal is the
by-default reading. Sentence (31) could project weakly, as in ((31) a.), as predicted by a class
of theories that stress existential projection (Beaver, 1994; Van Der Sandt, 1992; Geurts,
1998; Mandelkern, 2016).

Other theories predict that sentences like (31) display a default universal presupposition
projection (Heim, 1988; Schlenker, 2008, 2010; George, 2008; Fox, 2013; Chemla and Schlenker,
2012; Mayr and Sauerland, 2016). The presence of existential projections is accommodated
by these theories, as any existential can be captured as the result of a (conditional) domain
restriction on the "none" - "none of the players (who smoked) stopped smoking", as in (31)d.,
where the universal, adequately restricted, gives rise to an inference equivalent to an existential.

A third possible reading is a consequence of local accommodation, as in (31)c. Presupposition can be interpreted within the scope of a logical operator that makes them at issue. In (32)a., the presupposition is interpreted within the scope of negation and the resulting truth conditions end up amounting to the conjunction of the supposed presuppositional content with the supposed at-issue content as in (32)b.

(32) a. John didn't stop smoking, he didn't even start!
b. John didn't begin and stop a smoking habit, he didn't even start!

The empirical evidence is not more uniform. As mentioned above, Chemla (2009) showed
the availability of a universal reading in *none* sentences. In similar studies, it was argued that
the most robust inference is the existential one (Sudo et al., 2012; Geurts and van Tiel, 2016).
All studies found endorsement for both kinds of inferences, and Zehr et al. (2016) proposed
that both the existential and the universal inference are accessible to subjects, leaving open
the question of the typology of the logical behaviors of presuppositions under *none*.

Turning to our result, it seems that a positive result would have been strong evidence
of our hypothesis (all universal projections are presuppositions). However, in view of
the complexity of the behavior of presupposition triggers under “none”, while the lack of
universal projection weakens a bit our conclusion, it does not (at all) refute it (not all
non-universal projections are non-presuppositions/not all presuppositions project universally).

A second point of discussion is that in spite of all this, the question remains of why we
found a result different from Tieu et al. (2019). We here give two tentative explanations.

First, consider again our context in (24), reported below in (33):

(33) Some hikers are hiking in the mountains, where there can be significant drops and peaks in elevation. They alternate between reaching the top and the foot of steep rocks. Two of them, who finished first, are talking while waiting for the others.

It is possible that participants inferred that not all of the hikers were at the bottom of a
rock because the two characters discussing were at the top of a rock already, given that the
context states that *they have already finished*. Let us again look at sentence (28), reported
below in (34).

(34) **Target premise (question)** One tells the other: "None of the hikers will UP-WARD_SCALE?"

If the domain of “the hikers” in the target inference in (29) *includes* the two hikers
discussing, the target inference in (29) should be false. In Tieu et al. (2019), instead, it was
clear that the speaker and the object matter were distinct.

One further possible explanation for the observed difference is that it might be easier
to convey absolute initial states in the visual modality than in the auditory modality. In
Tieu et al. (2019), the visual animation of a green bar becoming blue (*GREEN_TO hut*
was used to refer to the color change in the antennae of aliens introduced in the context. In
our own paradigm, the first note of the musical scale was used to refer to the level at which
the hiker was located in space. While in our context any point on the rock can be denoted
by the first note of the scale, the initial state “green” in Tieu et al.’s stimulus can only be
interpreted as absolute, i.e. no other point on the color spectrum is compatible with this
initial state. Then, some of the hikers may fail to climb up while (i) being in a position to
climb up, i.e. not at the top, and (ii) not being at the foot of the rock (but higher). This
would explain why subjects thought that not all of the hikers were at the foot of a rock.

## 3.3 Supplements

### 3.3.1 In language and from visual stimuli

Supplements are inferences triggered by non-restrictive relative clauses that behave like
independent sentences (Potts, 2004; Schlenker, 2019a). For instance, sentence (35)a below is
understood as (35)b and not as (35)c.

(35) a. It is unlikely that Robin lifts weights, which is harmful.

b. It is unlikely that Robin lifts weights. Lifting weights is harmful.

c. It is unlikely that Robin lifts weights and that this is harmful.

Tieu et al. (2019) found that both gestures and visual animations used in lieu of a
non-restrictive relative clause gave rise to the conditional presupposition characteristic of
supplements. For instance, in (36), the gesture HIT replaces a non-restrictive relative clause
similar to “which involves hitting”. The informational content of the gesture is not at-issue
here, i.e. it is not target by “if”, hence the inference that if the event under question happens,
it will involve some hitting.

(36) If June bugs a classmate today - HIT, she will get a detention.

∴ If June bugs a classmate today, it will involve hitting her.

By contrast, in (37), the content of the gesture is made at-issue by the use of “and does so
*like this*”. The way in which the bugging happens, i.e. by hitting, is now made at-issue and
thus targeted by the conditional, hence the weaker inference that hitting will not necessarily
be involved.
(37) If June bugs a classmate today and does so like this - HIT, she will get a detention.
→ If June bugs a classmate today, it will not necessarily involve hitting her.

### 3.3.2 From music

We used an excerpt of real film music by American composer Bernard Herrmann from the soundtrack of the movie *Psycho* to denote a scary dog coming closer to the character in the context shown in (38).<fnref n="14" /> We reasoned that a rich musical gesture with a non-ambiguous emotional content could be informative about the scene while being interpreted as a non-restrictive relative clause.

(38) **Context:** Marie is walking back home. She spots a dog on the other side of the street. She sometimes worries about dogs, because some of them can be vicious - PSYCHO.

In the target premise in (39), PSYCHO behaves like a non-restrictive relative clause. It conveys that if the dog comes closer, it will look somewhat dangerous or scary.

(39) **Target premise**

If the dog comes to her - PSYCHO, Marie will cross the street.

**Original stimulus**

*Si le chien s' approche d' elle - PSYCHO, Marie changera de trottoir.*

If the dog self approaches to her - PSYCHO, Marie changes of sidewalk.

If the musical gesture PSYCHO logically behaves like a non-restrictive relative clause, we expect the target sentence in (39) to give rise to the target inference in (40), in which the behavior of the dog is not at-issue, thus not targeted by the conditional.

(40) **Target inference**

If the dog comes to Mary, it will look vicious.

To ensure that the musical gesture PSYCHO was not interpreted as a non-restrictive relative clause in the control premise in (41), we used the deictic “this” in “and does so like this - PSYCHO” to make the informational content of the musical gesture at-issue.

<footnote n="14">¹⁴In this paradigm, we embedded complex music in language and uncovered rich linguistic inferences. A non-trivial extension of our paradigm may in future test if such logical inferences can arise in purely musical environments.</footnote>
(41) **Control premise**

If the dog comes to her and does so like this - **PSYCHO**, Marie will cross the street.

**Original stimulus**

Si le chien s' approche d' elle et qu' il le fait comme ça -
If the dog self approaches to her and that it-NOM. it-ACC. does like this -
PSYCHO, Marie changera de trottoir.
PSYCHO, Marie changes of sidewalk.

The use of “this” now makes the informational content of the musical gesture at-issue and thus targeted by the conditional “if”, leading to the inference in (42) that the dog will not necessarily look vicious. Just like Tieu et al. (2019), we opted for a simpler formulation of the weaker control inference in (42) instead of the more convoluted exact negation of the target inference, “It is not the case that if the dog comes to Mary, it will look threatening.”

(42) **Baseline inference**

If the dog comes to Mary, it won't necessarily look vicious.

3.3.3 Results

<figure><img src="image_9.png" /><figcaption>Figure 4: Mean endorsement rate for supplements</figcaption></figure>

As explained in Appendix II, we are here interested in the interaction between inference type
(supplement *vs* no supplement) and premise type (pro-speech gesture *vs* 'like this' control):
to ensure that the stronger endorsement of the target inference containing the supplemental
information we observe for the target premise was not due to a default preference for this
kind of inference regardless of the premise, we contrasted this premise with a control in
which the musical gesture was made at-issue through the use of ‘like this’. Despite the higher
endorsement of the target inference in response to the target premise, the interaction between
the two factors was not significant (χ² = 1.5, p = 0.2). In Tieu et al. (2019), only a marginal
interaction was found for supplements with visual animations. This lack of interaction might
be due to the simplification of the negated inference used as a baseline, as we used “won’t
necessarily...” instead of the full negation “It is not the case that, if the dog comes to Mary,
it will look vicious”, which truth-conditions might well be different. We discuss a further
difference with the original paradigm from Tieu et al. (2019) in the following subsection.

3.3.4 Discussion

The paradigm we used was slightly different from the test for supplements in Tieu et al.
(2019), as we chose to introduce the content of the music explicitly in the context, while in
Tieu et al. (2019), the gesture HIT mimicking someone boxing only occurred in the target
sentences. We reasoned that introducing the content of the gesture would not affect its logical
behavior. On the one hand, indeed, introducing the content of a pro-speech gesture would
have jeopardized the validity of the results for inferences that can be easily lexically encoded,
such as presuppositions under certain theories. On the other hand, because non-restrictive
relative clauses are sentence-level constructions, there is not even the possibility that they
are encoded in the meaning of a specific word.

Our paradigm did not aim at showing that subjects could derive a meaning from music but,
rather, that they could treat a musical gesture as a non-restrictive relative clause when given
its meaning. We thus introduced this meaning explicitly in the context, so that participants
did not reject the target inference only because they perceived the excerpt as conveying a
feeling other than fear. In Tieu et al’s, the gesture HIT mimicking someone hitting someone
else was used in lieu of a non-restrictive relative clause. It is quite unambiguous that HIT
means something like *hit*. On the other hand, the meaning of the musical gesture, an excerpt
from real complex film music, would have been hardly the same for all participants.<fnref n="15" /> <fnref n="16" />

3.4 Homogeneity inferences

3.4.1 In language and from visual stimuli

Homogeneity inferences arise from definite plural noun phrases behaving universally in positive sentences, but existentially under negation (Križ, 2015; Križ and Spector, 2020; Spector, 2013; Löbner, 2000; Križ, 2016; Gajewski, 2005). Sentence (43)a has a universal reading: we understand that Mary found all of her presents. Under negation, however, the definite plural has an existential reading (i.e. “at least one”). Sentence (43)b is understood as *Mary did not find any of her presents*, and not as *Mary found some, but not all of her presents*, which involves the logical negation of “all of her present”.

(43) a. Mary found her presents.
⇒ Mary found all of her presents.

b. Mary did not find her presents.

∴ Mary did not find any of her presents.

Tieu et al. (2019) investigated whether pro-speech gestures and visual animations used in
lieu of both a verb *and* a definite plural noun phrase could generate homogeneity inferences.
For instance, in (44) and (45), the gesture *TAKE-2-HANDED-RIGHT* replaced both a verb
meaning *take* and a definite plural. The definite component was realized by means of pointing,
and the plural component was realized by means of the iteration of a gesture representing a
coin, which is a common way of signaling plural in sign languages (Pfau and Steinbach, 2006).

<footnote n="15">¹⁵Although this music unambiguously conveys a feeling of fear, danger or suspense, there are many possible situations this music can refer to that would trigger such feelings, considering that music can indeed refer to several external non-musical situations sharing some structural and/or emotional properties (Schlenker, 2017, 2019b)</footnote>

<footnote n="16">¹⁶As music is not generally used to directly communicate ideas or convey information about the world, investigating the intermediary case of onomatopoeias may bridge our findings with the results from Tieu et al. (2019). Since onomatopoeias are used in combination with language and are highly iconic, there are substantial reasons to believe that any inference type from the typology we replicated with musical gestures could just as well be replicated with onomatopoeias: since onomatopoeias occur naturally in lieu of words, we would expect them to display the same inferential behavior as musical gestures which do *not* occur naturally in lieu of words and are arguably more difficult to process. To verify that this was the case, we ran a similar experiment with onomatopoeias instead of musical stimuli. Results were not significant for all inference types (see Appendix I for details). However, the results were not significantly different across the two experiments, leaving open the possibility that the onomatopoeias experiment, which we ran on fewer participants, was lacking power.</footnote>
In the positive environment in (44), the pro-speech gesture triggers the universal inference
that all coins were indeed taken.

(44) Sam will TAKE-2-HANDED-RIGHT.
⇒ Sam will take all of the coins.

Under negation, sentence (45) gives rise to an existential reading of the gesture TAKE-2-HANDED-RIGHT ("at least one coin") instead of the universal reading found in the positive environment ("all of the coins"). Not-TAKE-2-HANDED-RIGHT is interpreted as *not at least one coin*, i.e. no coin at all, rather than *not all of the coins*, i.e. some but not all of the coins.

(45) Sam will not TAKE-2-HANDED-RIGHT.
⇒ Sam will not take any coin.

While most research on homogeneity inferences focuses on the role of the noun phrase, more
recent work has suggested that these inferences rather originate in the predicate associated
with the noun phrase (Križ, 2019). We thus proceeded by case: in section 3.4.2, we report
the results for stimuli in which the musical gesture replaced a noun phrase, while in section
3.4.3 we report a parallel paradigm but with the musical gesture in lieu of the predicate.

3.4.2 From music

**Pro-speech music in lieu of the noun phrase** While the two components of the definite plural were visually realized in Tieu et al. (2019), it was not possible to realize them both in the auditory modality. The auditory counterpart of iteration, marking plural, was straightforward, and we used the iteration of a same musical note to mark plural. Crucially, however, we could not find an auditory counterpart of pointing to reproduce the definite component. We thus chose to focus on manipulating the iterative component only and on verifying that the musical gesture was interpreted as a definite plural.

In (46), HARP×3 stands for three repetitions of a same harp sound evoking three
harp players in an orchestra, while FLUTES×3 stands for three repetitions of a same flute
sound evoking three flute players in the same context.
(46) **NP Positive Context:** Every Thursday, the students of a music school are gathering, and the conductor chooses some of them to play for the evening concert. Tonight, three harp-players - HARP×3 and three flute-players FLUTE×3 are present.

We performed a first test in a positive environment, as shown in (47).

(47) **Target positive premise**

The conductor made HARP×3 play.

**Original stimulus**

Le chef d'orchestre a fait jouer HARP×3.

The chief of orchestra has made play HARP×3.

The musical gesture HARP×3 is expected to have a universal reading and be interpreted as *all harps*, leading to the inference in (48), which we expect to be significantly more endorsed than its negation, the baseline inference in (49).

(48) **Target inference**

All harps played.

(49) **Baseline inference**

Some but not all of the harps played.

Negation constitutes a crucial test to correctly explain the pattern observed in the positive environment, for two reasons. First, homogeneous expressions behave as existentials in negative environments. Second, an alternative explanation of the pattern we observe for the positive environment is that HARP×3 elicits a numeral, i.e. *exactly three harps*. Under negation, the two explanations come apart.

If HARP×3 is understood as a numeral, i.e. *three harps*, its negation *not*-HARP×3' should mean *not exactly three harps*, i.e. two or less, or four or more. This is compatible with the control in (52). If, by contrast, HARP×3 is understood as a definite plural, we expect it to be interpreted existentially under negation, i.e. *not*-HARP×3 is interpreted as *not at least one harp*, i.e. no harp at all.

(50) **Target negative premise**

The conductor did not make HARP×3 play.

**Original stimulus**

Le chef d'orchestre n' a pas fait jouer HARP×3.

The chief of orchestra NEG has not made play HARP×3.
If HARP×3 is interpreted as a definite plural, then we expect a higher endorsement of the homogeneous target sentence in (51) resulting from an existential reading of the musical gesture under negation, compared to the baseline inference in (52) resulting from a universal reading of the musical gesture (i.e., “The conductor did not make all harps play”).

(51) **Target inference**

No harp played.

(52) **Baseline inference**

Some but not all of the harps played.

### 3.4.3 Results

<figure><img src="image_10.png" /><figcaption>Figure 5: Mean endorsement rate for homogeneity (NP)</figcaption></figure>

As explained in Appendix II, if homogeneity can be created by musical gestures standing for definite plurals, we expected an effect of inference type (homogeneous target *vs* non-homogeneous baseline): we expected the target homogeneous inference to be significantly more endorsed than its negation in both the positive and negative environments. This is what we find: the homogeneous inference (‘all harps played’) was preferred in response to the positive premise ($\chi^2 = 96, p < 0.001$), and similarly, the homogeneous inference (‘no harp played’) was preferred in response to the negative premise ($\chi^2 = 24, p < 0.001$). This
suggests that it is possible to generate homogeneity inferences with musical gestures in lieu
of definite plurals.<fnref n="17" />

**Pro-speech music in lieu of the predicate** Recent work suggested that homogeneity inferences
might not be systematically linked to a definite plural noun phrase, but rather to the kind
of predicate associated with the noun phrase (Križ, 2019). If definite noun phrases were
responsible for homogeneity inferences, it would be difficult to explain how non-homogeneous
readings arise from sentences like (53).

(53) The students are numerous.
a. ~~~ ?? Each of the students is numerous.

Križ (2019) argues that predicates, rather than bare plurals, are responsible for the presence
or absence of homogeneity. While in Tieu et al. (2019), the test for homogeneity involved
the gesture *TAKE_ALL* encompassing both the predicate *TAKE* and a definite plural noun
phrase *THE_COINS*, the test we presented in section 3.4.2 only involved a musical gesture
replacing the noun phrase.

Since there is the theoretical possibility that homogeneity comes from the predicate, we
also tested whether sentences in which pro-speech music is inserted in lieu of the *predicate*
can give rise to homogeneity inferences as well. In (54), the musical gesture *MARSEILLAISE*
featuring a children's choir singing the French national anthem was used to evoke the action
of singing.

(54) **Positive Context (predicate)**
The students of a class are learning to sing the Marseillaise - MARSEILLAISE.

We applied the same protocol as in section 3.4.2. We performed a first test in a positive
environment, where we expected sentence (55) to give rise to the universal inference in (56).

<footnote n="17">¹⁷As pointed out by the Editor, instead of considering whether the premise triggered a homogeneous or a non-homogeneous reading, we could have shown that the interpretation of the musical gesture flips from universal in the positive environment to existential in the negative environment. In this case, we would have been interested in the interaction between the inference type factor, and the environment (positive *vs* negative). We display this alternative analysis in Appendix II, and show that it leads to a significant interaction between the two factors, indicating that the reading did change from universal to existential once embedded under negation.</footnote>
(55) **Target positive premise**

On the 14th of July, the class has MARSEILLAISE.

**Original stimulus**

Le jour du 14 juillet, la classe a MARSEILLAISE.
The day of 14th July, the class has MARSEILLAISE.

(56) **Target inference**

The children in the class all sang the Marseillaise.

Just as in 3.4.2, the target inference in (56) needs to be contrasted with the non-universal baseline inference in (57).

(57) **Baseline inference**

Some but not all of the children in the class sung the Marseillaise.

We then performed a second test in a negative environment, i.e. the predicate MARSEILLAISE was in the scope of negation, as in (58).

(58) **Target negative premise**

On the 14th of July, the class did not MARSEILLAISE.

**Original stimulus**

Le jour du 14 juillet, la classe n' a pas MARSEILLAISE.
The day of 14th July, the class NEG has not MARSEILLAISE.

The negation of the musical predicate (*not*-MARSEILLAISE) now applies homogeneously to each member of the set denoted by the group singular noun phrase “the class”, leading to the inference in (59) that no child in the class sang the Marseillaise, which we expected to be highly endorsed.

(59) **Target inference**

None of the children in the class sang the Marseillaise.

By contrast, we expected a low endorsement of the logical negation of this inference, i.e. the weaker baseline inference in (60) that some but not all of the children in the class sang the Marseillaise.

(60) **Baseline inference**

Some but not all of the children in the class sung the Marseillaise.

In this paradigm, none of the tested inferences involve a definite plural noun phrase, so that the effect of the musical predicate could be isolated if there is one. If the sentence gives
rise to a homogeneity inference, we expect a strong endorsement of the target inferences (i.e. a universal reading of the sentence in the positive environment, and an existential one under negation) compared to the baseline inference.

An intuitive objection is that these inferences are not due to the predicate but to the group singular noun phrase “the class”. In this case, however, the homogeneity inference originates in the noun phrase and thus our paradigm presented in 3.4.2 provides the relevant evidence. If, on the other hand, homogeneity originates in the predicate, the results presented in this section provide evidence that pro-speech music predicates can give rise to homogeneity inferences.

The results show a significant effect of inference type in both the positive ($\chi^2 = 96, p < 0.001$) and the negative environments ($\chi^2 = 24, p < 0.001$). This suggests that it is possible to give rise to homogeneity inferences by means of a musical gesture replacing the predicate instead of the definite plural noun phrase.

<figure><img src="image_11.png" /><figcaption>Figure 6: Mean endorsement rate for homogeneity (predicate)</figcaption></figure>
# 4 Iconicity controls

## 4.1 Experimental paradigm

As mentioned in section 1, an alternative explanation of our data is that musical gestures gave rise to the observed inferences simply because they are systematically translated into words. Note that Tieu et al. faced the very same concern with pro-speech gestures and visual animations, which was also addressed by including iconicity controls throughout the experiment.

To rule out this alternative explanation, we included 8 iconicity trials randomly displayed throughout the experiment. We manipulated these stimuli to check whether participants were able to interpret their modification iconically. To this end, we used two different realizations of an upward scale, a normal and a fast one, to refer to a hiker climbing either slowly or quickly. We used two other realizations, a very long musical scale and a shorter one, to evoke the height of the mountain, in the same context as the one used in 3.2. The target sentences were then paired up with a matching or a mismatching inference. If participants were able to retrieve the expected iconic informational content from the musical gesture, we expected a significantly higher endorsement of the matching inference for the target premise with respect to the mismatching inference.

For the first iconicity control in (61), `LONG_UPWARD_SCALE` stands for a C major scale played over a wide pitch range of four octaves (i.e., four full upward scales, from middle pitch to very high), used to evoke a very high mountain.

### (61) Target premise

The next hiker will `LONG_UPWARD_SCALE`.

#### Original stimulus

Le prochain randonneur va `LONG_UPWARD_SCALE`. The next hiker will `LONG_UPWARD_SCALE`.

We expected participants to be able to (i) perceive the difference between the two realizations of the upward scale and (ii) interpret them iconically in terms of the height of
the mountain in the context. Consequently, we expected a high endorsement of the inference
containing the matching iconic component in (62).

(62) Matching inference

The mountain the hiker is about to climb is high.

In (63), UPWARD_SCALE stands for the same scale played at the same speed but over a
narrower pitch range of two octaves (i.e., two full upward scales only, from middle pitch to
moderately high), to evoke a mountain of average height. Note that this musical gesture was
the one used in 3.2 to test for presupposition projection. Our prediction was that the longer
the scale, the higher the inferred height of the mountain.

(63) Control premise

The next hiker will UPWARD_SCALE.

Original stimulus

Le *prochain randonneur va* UPWARD_SCALE.

The next hiker will UPWARD_SCALE.

We expected that this neutral realization of the scale would convey information about
the mountain being of average height, and thus a high endorsement of the inference in (64),
in which the iconic component allowing subjects to assess the mountain to be very high was
absent.

(64) Mismatching inference

The mountain the hiker is about to climb is not very high.

For the second iconicity control, we contrasted the baseline realization of the scale
UPWARD_SCALE in (67) evoking a neutrally fast hike with SLOW_UPWARD_SCALE in (65),
in which the pitch range remained the same but the speed was slowed down four times to
evoke a slower climbing of the mountain.

(65) Target premise

The next hiker will SLOW_UPWARD_SCALE.

Original stimulus

Le *prochain randonneur va* SLOW_UPWARD_SCALE.

The next hiker will SLOW_UPWARD_SCALE.
Similarly, participants should be able to both perceive and interpret the difference between the two realizations iconically in terms of the hiking speed, and we expected a high endorsement of the inference in (66) containing the iconic component.

(66) **Matching inference**

The hiker will climb the mountain slowly

(67) **Control premise**

The next hiker will UPWARD_SCALE.

**Original stimulus**

Le prochain randonneur va UPWARD_SCALE.

The next hiker will UPWARD_SCALE.

The iconic component allowing for the inference about a high hiking speed was absent from this control premise, which we expected to give rise to the inference in (68).

(68) **Matching inference**

The hiker will climb the mountain fast.

We found a significant effect of inference type ($\chi^2 = 63, p < 0.001$ for the first iconicity control, $\chi^2 = 28, p < 0.001$ for the second). This means that listeners were able to both perceive subtle musical changes and interpret them iconically.

## 4.2 Discussion

We have shown that musical gestures convey fine-grained iconic information. Specifically, the pitch range of the stimulus was interpreted as the height of the mountain, and the tempo at which the upward scale was played was interpreted as the hikers' speed. It is very unlikely that these musical gestures were merely translated in words, because all these iconic dimensions are absent from reasonably simple translations. For instance, the information about height conveyed by UPWARD_SCALE is absent from a simple translation as "go up". It could still be, however, that more complex translations are involved. These would have to be extraordinarily rich: "The hiker will LONG_UPWARD_SCALE" would have to be translated as something like "The hiker will climb a mountain of such-and-such height at such-and-such a speed...". However, even if pro-speech music were indeed translated into such long chains of words, the music-as-translation theory makes wrong predictions on the projection patterns
of these iconic dimensions. Consider a translation of the target premise which, for ease, we simplify as (69).

(69) The hiker will climb up the mountain slowly.

The music-as-translation theory predicts that all pieces of fine-grained information iconically conveyed be systematically *at-issue* when not *lexically* encoded as non-at-issue. Any musical gesture conveying fine-grained iconic information should therefore systematically give rise to a scalar implicature. That is, *SLOW_UPWARD_SCALE* would mean *climb slowly*, thus its negation *not-SLOW_UPWARD_SCALE* should give rise to the scalar implicature that there is still some climbing involved, given that the more informative *not-UPWARD_SCALE* was not uttered. This prediction seems right if we look at the negation of (69), as in (70).

(70) The hiker will not climb up the mountain slowly.

⇒ The hiker will not climb up the mountain slowly, but will still climb up the mountain.

The premise in (70) does not, however, trigger the same inference when using the musical gesture *LONG_UPWARD_SCALE*.

(71) The hiker will not *LONG_UPWARD_SCALE*.

a. ⇌ The hiker will not climb up the mountain slowly, but will still climb up the mountain.

b. ⇌ The hiker will not climb the mountain at all, but if she did, she would have done so slowly.

A first possible reading of the sentence in (71) is the same scalar implicature found in (71)a. However, another possible reading is (71)b, where the iconic information paraphrasable as “slowly” is not at-issue anymore. While this reading is not the most obvious, it becomes completely salient in context. Consider the following example:

(72) **Context:** Mary and John are playing a snail race, and they want their snails to climb all sorts of obstacles.

Mary tells John: "Your snail will not *SLOW_UPWARD_SCALE*."

⇒ John's snail will not climb the next obstacle but if it had climbed it, it would have done so slowly.

⇒ John's snail will not climb the next obstacle slowly, but will still climb it.
This inference is a typical cosupposition (Schlenker, 2018a) of the form *x did not Q, but if x did Q, x would have Qed like this*. Crucially, it is not triggered in the purely linguistic example in (70), as shown in (73).

(73) The hiker will not climb up the mountain slowly.
→ The hiker will not climb the mountain at all, but if she did, she would have done so slowly.

If the musical gesture is only translated into words and no other non-translatable iconic enrichment is involved, then there is at least one available reading which cannot be explained. Consequently, there is no real alternative to the conclusion that there is a non-verbal component to pro-speech music. It is worth noting that similar inferences were found to be triggered by co-speech sounds and auditory animations (Tieu et al., 2018), which make non-at-issue contributions to the sentence.

# 5 Conclusion

The purpose of this study was to replicate the inferential typology tested on pro-speech gestures and visual animations of inferences in Tieu et al. (2019) with pro-speech music. Participants behaved systematically in response to novel stimuli, suggesting that a general cognitive algorithm is responsible for the appropriate arrangement of content, rather than a purely linguistic procedure relying on word meaning. Specifically, our results show that pro-speech music can give rise to four types of inferences.

We showed that musical gestures can assume different levels of informativity and compete, resulting in the computation of scalar implicatures (cf: section 3.1).

Moreover, we showed that pro-speech music can convey presuppositions. Presupposed content projected under question (cf. section 3.2), but did not project universally under “none”. While this difference with Tieu et al. (2019) weakens our hypothesis a bit, it does not constitute negative evidence given the complex behavior of presuppositions under “none”. Testing simple negation will provide a helpful test to show whether the observed behavior under “none” is specific to “none” or due to general complex projection patterns of pro-speech music.
Conditional inferences typical of supplements, i.e. non-restrictive relative clauses, can also
arise from rich pro-speech music (cf. section 3.3). However, although the target inference with
the supplemental information was more highly endorsed than the control inference without
this information, we did not find the expected interaction between premise type and inference
type. As Tieu et al. (2019) reported a marginal interaction in the case of pro-speech visual
animations, further research will be needed to understand whether abstractness might weaken
the triggering of supplements.

In section 3.4, we showed that pro-speech music can give rise to homogeneity inferences,
both when replacing a definite plural noun phrase and when inserted in lieu of a predicate.
Section 4 finally illustrated why it is very unlikely that these results can be explained by the
mere translation of musical gestures into words.

Together, these findings suggest that the mechanisms responsible for these inferences and
for the division of meaning along different slots of the inferential typology are (i) productive
and (ii) general purpose. Indeed, (i) participants were able to draw the expected inferences
in absence of previous exposure with the musical gestures, and (ii) they were able to operate
logical computations on auditory, non-linguistic stimuli.

Finally, the question of the existence of a music semantics, recently raised in Schlenker
(2017, 2019b) (a.o.), is a very exciting and promising one, but our results do not allow us to
make any claim about musical meaning. However, we think that the evidence we provide
about the interaction between language structure and musical content constitutes a first step
to better understand what musical meaning is and how it behaves logically. In particular, the
extent to which some of the observed inferences may arise in music alone, i.e. not embedded
in language, is an inspiring question for future research.
# 6 Acknowledgments

We are greatly indebted to Philippe Schlenker and Emmanuel Chemla for in-depth discussion of virtually every aspect of this project. We also thank Amir Anvari for his helpful comments on the theoretical underpinnings of this work and suggestions for improvement on the first draft. Thank you to Salvador Mascarenhas, Rob Pasternak and Lyn Tieu for discussion of the early versions of this work. Finally, we would like to thank the audiences of the ‘Linguistic investigations beyond language’ workshop at ZAS, GLOW 2019, and the Linguae seminar.

# Compliance with Ethical Standards

The authors have no financial or non-financial interest to disclose. Data collection was approved under Opinion number 20-733 of the Institutional Review Board of the French Institute of medical research and Health (IRB00003888, IORG0003254, FWA00005831). Online informed consent was obtained for all participants prior to the beginning of the experiment.
References

Abbott, B. (2000). Presuppositions as nonassertions. *Journal of Pragmatics*, 32(10):1419-1437.

Abrusán, M. (2010). Triggering verbal presuppositions. 20:684–701.

Abrusán, M. (2011). Predicting the presuppositions of soft triggers. *Linguistics and Philosophy*, 34(6):491–535.

Abusch, D. (2002). Lexical Alternatives as a Source of Pragmatic Presuppositions. *Semantics and Linguistic Theory*, 12:1.

Abusch, D. (2010). Presupposition Triggering from Alternatives. *Journal of Semantics*, 27(1):37–80.

Barr, D. J., Levy, R., Scheepers, C., and Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. *Journal of Memory and Language*, 68(3):255–278.

Beaver, D. (1994). When variables don't vary enough. In *Semantics and linguistic theory*, volume 4, pages 35–60.

Beaver, D. I. (2001). Presupposition and Assertion in Dynamic Semantics. *Stanford: CSLI Publications*.

Breheny, R. (2019). Scalar Implicatures. In Cummins, C. and Katsos, N., editors, *The Oxford Handbook of Experimental Semantics and Pragmatics*, pages 38–61. Oxford University Press.

Chemla, E. (2009). Presuppositions of quantified sentences: experimental data. *Natural Language Semantics*, 17(4):299–340.

Chemla, E. and Schlenker, P. (2012). Incremental vs. symmetric accounts of presupposition projection: an experimental approach. *Natural Language Semantics*, 20(2):177–226.

Chierchia, G. (2006). Broaden Your Views: Implicatures of Domain Widening and the “Logicality” of Language. *Linguistic Inquiry*, 37(4):535–590.

Chierchia, G., Fox, D., and Spector, B. (2012). Scalar implicature as a grammatical phenomenon. In *Semantics: An International Handbook of Natural Language Meaning*. De Gruyter Mouton, Berlin, claudia maienborn, klaus von heusinger, and paul portner edition.

Chierchia, G. and McConnell-Ginet, S. (2000). *Meaning and grammar: An introduction to semantics*. MIT press.

Chomsky, N. (1980). On Cognitive Structures and Their Development: A Reply to Piaget. In *Language and Learning*. Routledge and Kegan Paul, London, piatelli-palmarini, m., ed. edition.

Chomsky, N. (1988). *Language and problems of knowledge: the Managua lectures*. Number 16 in Current studies in linguistics series. MIT Press, Cambridge, Mass.

Clark, H. H. (2016). Depicting as a method of communication. *Psychological Review*, 123(3):324–347.

Ebert, C. and Ebert, C. (2014). Gestures, Demonstratives, and the Attributive/Referential Distinction. Handout of a talk given at Semantics and Philosophy in Europe (SPE 7).

Fox, D. (2007). Free Choice and the Theory of Scalar Implicatures. In Sauerland, U. and Stateva, P., editors, *Presupposition and Implicature in Compositional Semantics*, Palgrave
Studies in Pragmatics, Language and Cognition, pages 71–120. Palgrave Macmillan UK, London.

Fox, D. (2013). Presupposition projection from quantificational sentences: trivalence, local accommodation, and presupposition strengthening. *MIT web domain*.

Fox, D. and Katzir, R. (2011). On the characterization of alternatives. *Natural Language Semantics*, 19(1):87–107.

Gajewski, J. (2005). *Neg-raising: Polarity and Presupposition*. *MIT dissertation*. PhD thesis, MIT.

George, B. R. (2008). A New Predictive Theory of Presupposition Projection. *Semantics and Linguistic Theory*, 18(0):358–375.

Geurts, B. (1998). The Mechanisms of Denial. *Language*, 74(2):274–307.

Geurts, B. (1999). *Presuppositions and pronouns*. Number v. 3 in Current research in the semantics/pragmatics interface. Elsevier, Amsterdam ; New York, 1st ed edition.

Geurts, B. (2010). *Quantity Implicatures*. Cambridge University Press, Cambridge.

Geurts, B. and van Tiel, B. (2016). When “All the Five Circles” are Four: New Exercises in Domain Restriction. *Topoi*, 35(1):109–122.

Goldin-Meadow, S. and Brentari, D. (2017). Gesture, sign, and language: The coming of age of sign language and gesture studies. *Behavioral and Brain Sciences*, 40:e46.

Grice, H. (1975). Logic and conversation. In *Syntax and Semantics 3: Speech Acts*, pages 41–58. Peter coler and jerry morgan edition.

Heim, I. (1988). *The semantics of definite and indefinite noun phrases*. Outstanding dissertations in linguistics. Garland Pub, New York.

Heim, I. (1990). Presupposition Projection. In *Presupposition, Lexical Meaning and Discourse Processes: Workshop Reader*, page 33, University of Nijmegen. R. van der Sandt.

Heim, I. (1992). Presupposition Projection and the Semantics of Attitude Verbs. *Journal of Semantics*, 9(3):183–221.

Heim, I. (2002). On the Projection Problem for Presuppositions. In *Formal Semantics*, pages 249–260. John Wiley & Sons, Ltd.

Heim, I. and Kratzer, A. (1998). *Semantics in generative grammar*. Number 13 in Blackwell textbooks in linguistics. Blackwell, Malden, MA.

Horn, L. (1972). *On the semantic properties of the logical operators in English*. Ph.D. thesis. PhD thesis, University of California at Los Angeles.

Kadmon, N. (2001). *Formal pragmatics: semantics, pragmatics, presupposition, and focus*. Blackwell, Malden, Mass., 1. publ edition.

Katzir, R. (2007). Structurally-defined alternatives. *Linguistics and Philosophy*, 30(6):669–690.

Križ, M. (2015). *Aspects of homogeneity in the semantics of natural language*. PhD Thesis, University of Vienna.

Križ, M. (2016). Homogeneity, Non-Maximality, and all. *Journal of Semantics*, 33(3):493–539.

Križ, M. (2019). Homogeneity effects in natural language semantics. *Language and Linguistics Compass*, 13(11).

Križ, M. and Spector, B. (2020). Interpreting plural predication: homogeneity and non-maximality. *Linguistics and Philosophy*.
Levinson, S. C. (2000). *Presumptive meanings: the theory of generalized conversational implicature*. Language, speech, and communication. MIT Press, Cambridge, Mass.

Löbner, S. (2000). Polarity in natural language: predication, quantification and negation in particular and characterizing sentences. *Linguistics and Philosophy*, 23(3):213–308.

Mandelkern, M. (2016). A note on the architecture of presupposition. *Semantics and Pragmatics*, 9(0):13–24.

Mayr, C. and Sauerland, U. (2016). Accommodation and the strongest meaning hypothesis. *Preproceedings of the Amsterdam Colloquium*.

McNeill, D. (2005). * Gesture and thought*. University of Chicago Press, Chicago.

Pfau, R. and Steinbach, M. (2006). Pluralization in sign and in speech: A cross-modal typological study. *Linguistic Typology*, 10(2).

Potts, C. (2004). *The Logic of Conventional Implicatures*. Oxford University Press.

Potts, C. (2015). Presupposition and Implicature. In *The Handbook of Contemporary Semantic Theory*, pages 168–202. John Wiley & Sons, Ltd.

Sauerland, U. (2004). Scalar Implicatures in Complex Sentences. *Linguistics and Philosophy*, 27(3):367–391.

Sauerland, U. (2012). The Computation of Scalar Implicatures: Pragmatic, Lexical or Grammatical?: Computation of Scalar Implicatures. *Language and Linguistics Compass*, 6(1):36–49.

Schlenker, P. (2008). Presupposition Projection: the New Debate. *Semantics and Linguistic Theory*, 18(0):655–693.

Schlenker, P. (2010). Presuppositions and Local Contexts. *Mind*, 119(474):377–391.

Schlenker, P. (2016). The semantics–pragmatics interface. In Aloni, M. and Dekker, P., editors, *The Cambridge Handbook of Formal Semantics*, pages 664–727. Cambridge University Press, Cambridge.

Schlenker, P. (2017). Outline of Music Semantics. *Music Perception: An Interdisciplinary Journal*, 35(1):3–37.

Schlenker, P. (2018a). Gesture projection and cosuppositions. *Linguistics and Philosophy*, 41(3):295–365.

Schlenker, P. (2018b). Iconic pragmatics. *Natural Language & Linguistic Theory*, 36(3):877–936.

Schlenker, P. (2019a). Gestural semantics: Replicating the typology of linguistic inferences with pro- and post-speech gestures. *Natural Language & Linguistic Theory*, 37(2):735–784.

Schlenker, P. (2019b). Prolegomena to Music Semantics. *Review of Philosophy and Psychology*, 10(1):35–111.

Schlenker, P. (2020). Gestural grammar. *Natural Language & Linguistic Theory*, 38(3):887–936.

Schlenker, P. (2021). Triggering Presuppositions. *Glossa: a journal of general linguistics*, 6(1).

Simons, M., Tonhauser, J., Beaver, D., and Roberts, C. (2010). What projects and why. *Semantics and Linguistic Theory*, 20:309.

Spector, B. (2007a). Aspects of the Pragmatics of Plural Morphology: On Higher-Order Implicatures. In Sauerland, U. and Stateva, P., editors, *Presupposition and Implicature in Compositional Semantics*, pages 243–281. Palgrave Macmillan UK, London.
Spector, B. (2007b). Scalar Implicatures: Exhaustivity and Gricean Reasoning. In Aloni, M., Butler, A., and Dekker, P., editors, *Questions in Dynamic Semantics*, pages 225–249. BRILL.

Spector, B. (2013). Homogeneity and plurals: From the strongest meaning hypothesis to supervaluations.

Stalnaker, R. (1974). Pragmatic Presuppositions. In Stalnaker, R., editor, *Context and Content*, pages 47–62. Oxford University Press.

Stalnaker, R. (1975). Presuppositions. In Hockney, D., Harper, W., and Freed, B., editors, *Contemporary Research in Philosophical Logic and Linguistic Semantics*, pages 31–41. Springer Netherlands, Dordrecht.

Sudo, Y., Romoli, J., Hackl, M., and Fox, D. (2012). Presupposition Projection Out of Quantified Sentences: Strengthening, Local Accommodation and Inter-speaker Variation. In Aloni, M., Kimmelman, V., Roelofsen, F., Sassoon, G. W., Schulz, K., and Westera, M., editors, *Logic, Language and Meaning*, Lecture Notes in Computer Science, pages 210–219, Berlin, Heidelberg. Springer.

Team, R. C. (2016). R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna), Version 3.4.3.

Tieu, L., Pasternak, R., Schlenker, P., and Chemla, E. (2018). Co-speech gesture projection: Evidence from inferential judgments. *Glossa: a journal of general linguistics*, 3(1):109.

Tieu, L., Schlenker, P., and Chemla, E. (2019). Linguistic inferences without words. *Proceedings of the National Academy of Sciences*, 116(20):9796–9801.

Tonhauser, J., Beaver, D., Roberts, C., and Simons, M. (2013). Toward a Taxonomy of Projective Content. *Language*, 89(1):66-109.

Van Der Sandt, R. A. (1992). Presupposition Projection as Anaphora Resolution. *Journal of Semantics*, 9(4):333–377.

van Rooij, R. and Schulz, K. (2004). Exhaustive Interpretation of Complex Sentences. *Journal of Logic, Language and Information*, 13(4):491–519.

Zehr, J., Bill, C., Lyn, T., Jacopo, R., and Florian, S. (2016). Presupposition projection from the scope of None: Universal, existential, or both? *Semantics and Linguistic Theory*, 26(0):754–774.
# Appendix I: Musical gestures and vocal gestures

As mentioned in section 3.3, we ran a parallel experiment on a different pool of participants using pro-speech onomatopoeias, i.e. iconic vocal sounds replacing one or several words in sentences, that we call ‘vocal gestures’, instead of musical gestures. The same four inference types were tested: scalar implicatures, presuppositions, supplements and homogeneity inferences, using paradigms and stimuli that were either perfectly analogous to the ones described throughout this paper, or very slightly different when vocal counterparts to musical stimuli could not be found. For instance, wherever an upward scale was used in our stimuli to evoke a rise in space to test for presuppositions, we used a whistle rising in frequency, and wherever a choir singing the French national anthem was used to evoke the action of singing to test for homogeneity inferences, we used a similar stimulus where the very same song was whistled instead of sung. Wherever a drum was used to evoke someone boxing to test for scalar implicatures, we used the onomatopoeia BOOM vocally pronounced.

We found a difference in the results collected from the experiment on musical gestures and the one on vocal gestures, which is surprising as most paradigms were perfectly symmetric. However, this experiment was ran on a smaller pool than the musical gestures experiment, and the lack of systematically significant effects of the inference type might thus be explained by a lack of power. We found indeed that the differences in the endorsement rates across both experiments were mainly not or marginally significant themselves, i.e. the responses given for the experiment on vocal gestures were not significantly different from the ones given for the experiment on musical gestures. The table below summarizes the comparisons of the distribution of the data between both experiments. For each inference type, we computed the interaction between inference type and the *Experiment* factor, whose two levels corresponded to the two experiments.<fnref n="18" />

<table><caption>Figure 7: Comparison between musical gestures and vocal gestures experiments.</caption><thead><tr><th>Inference type</th><th>Environment</th><th>p-value for the<br>[Inference type x Experiment] interaction</th><th>Significance interpretation</th></tr></thead><tbody><tr><td rowspan="2">Scalar implicatures</td><td>Positive</td><td>0.74</td><td>No interaction</td></tr><tr><td>Negative</td><td>0.08</td><td>Marginally significant interaction</td></tr><tr><td rowspan="2">Presuppositions</td><td>Question</td><td>0.61</td><td>No interaction</td></tr><tr><td>None</td><td>0.43</td><td>No interaction</td></tr><tr><td>Supplements</td><td></td><td>&lt; 0.001</td><td>Significant interaction (as expected)</td></tr><tr><td rowspan="2">Homogeneity</td><td>NP</td><td>0.05</td><td>Low interaction</td></tr><tr><td>Predicate</td><td>0.29</td><td>No interaction</td></tr></tbody></table>

The table displays the figures assessing the significance of the difference in distribution of the data collected across experiments. None of the data subsets were significantly different across both experiments, except for supplements, for which we found no contrast in endorsement due to an unexpected interpretation of the vocal stimulus.

<footnote n="18">¹⁸The detailed analyses and statistical scripts to compute interactions are available in the RESULTS folder at https://osf.io/hw45u/?view_only=89f983db777f49e9a6f5b41b3dea60d6</footnote>
# Appendix II: Statistical models

In this appendix, we provide the generalized linear mixed-effects models we used to analyze the data. They were all inspired by that of (Tieu et al., 2019), whose analysis script is open source. For each model, we justify the structure of the model and in particular its random effects structure by the design (as recommended in Barr et al. (2013)). We also provide the coefficients found for each model. For the sake of clarity, we report the statistical models in a raw fashion (i.e. the R code lines). Useful information about the syntax of R, which can be helpful to read these formulas, can be found online and are accessible at this link: https://github.com/clayford/LMEMInR/blob/master/lme4_cheat_sheet.Rmd.

## Scalar implicatures

In the scalar implicatures paradigms, two main factors could have had an effect on the endorsement of the target inferences: the gesture factor (GestureC), which two levels correspond to the target and control premises contrasting two alternative realizations of the drum sound mimicking someone boxing, DRUM×1 and DRUM×2; and the inference factor (InferenceC), which two levels correspond to the target and baseline inferences contrasting two possible interpretations of the the drum sounds ('some' vs 'a lot' in the positive environment; 'some' vs 'none' in the negative environment).

Here, we were interested in the `GestureC * InferenceC` interaction to ensure that the contrast found for the target premise was not due to a default bias in endorsement for one inference over the other, by comparing the difference between the endorsement of the target and that of its negation for the target premise and the same difference for the control premise. We did *not* include the environment factor in the model (positive vs negative) because there was no theoretical reason to expect a difference in these interactions between the positive and the negative environment. `GestureC`, `InferenceC`, and their interaction were thus used as fixed effects in our model, while this interaction *by subject* was used as a random effect, accounting for the variability across participants in (i) the interpretation of the premise, (ii) the interpretation of the inference, and (iii) the interaction between (i) and (ii).

### Model

value ~ GestureC * InferenceC + (1 + GestureC * InferenceC|SubjID)

This first model failed to converge. We followed the same procedure as (Tieu et al., 2019) in simplifying the random effects structure, and removed the interaction between the two factors `GestureC` and `InferenceC` from the random effects as follows:

### Simplification of random effects structure (as in (Tieu et al., 2019))

value ~ GestureC * InferenceC + (1 + GestureC + InferenceC|SubjID)

<table><thead><tr><th>Environment</th><th>(Intercept)</th><th>GestureC</th><th>InferenceC</th><th>GestureC:InferenceC</th></tr></thead><tbody><tr><td>Positive</td><td>52.60377</td><td>-0.79245</td><td>-0.07547</td><td>116.60377</td></tr><tr><td>Negative</td><td>57.005</td><td>1.557</td><td>7.575</td><td>44.698</td></tr></tbody></table>
# Presuppositions

There was no theoretical reason for predicting an interaction in presuppositional behavior between question formation and projection under ‘none’, as the two projection tests are independent. Inferences from questions and from ‘none’ were thus analyzed separately. We are rather interested in the effect of the inference type (presupposition vs no presupposition) on the responses.

To ensure that the inference containing presupposition was not due to a by-default preference for this kind of inference, the endorsement of the presupposition was contrasted with that of its negation (baseline inference), as a control. The model thus used the inference factor InferenceC, which levels correspond to the presupposition/no presupposition, as a fixed effect, and this same factor *by subject* was used as a random effect, to capture the possibility that each participants may simply tend to endorse presuppositions differently.

## Model

$$ \text{value} \sim \text{InferenceC} + (1 + \text{InferenceC|SubjID}) $$

This first model failed to converge, leading us to simplify the random effect structure by only keeping the SubjID factor as a random effect, to capture the variability in intercept across participants:

## Simplification of random effects structure

$$ \text{value} \sim \text{InferenceC} + (1 | \text{SubjID}) $$

<table><thead><tr><th>Environment</th><th>(Intercept)</th><th>InferenceC</th></tr></thead><tbody><tr><td>Question</td><td>49.48</td><td>-15.60</td></tr><tr><td>‘none’</td><td>51.65</td><td>2.74</td></tr></tbody></table>

# Supplements

The case for supplements is symmetric to the paradigm for scalar implicatures in 6, where both the gesture factor GestureC, describing the two types of premise (the target premise where the musical gesture is expected to be interpreted as a non-restrictive relative clause, and the control premise where the musical gesture is made at-issue by using ‘like this’) and the inference factor InferenceC, describing the two types of inference (supplemental or not) were used as fixed effects in the model, while the interaction *by participant* was used as a random effect in the maximal model below:

Here, we are interested in the interaction between the Gesture factor, which two levels represent the two forms of sentences (with a pro-speech gesture standing for a non-restrictive relative clause, and with the deictic ‘like this’) and the Inference factor, which two levels correspond to the supplemental inference (if X, then X would have happened in a certain way) and the inference without the supplemental information (if X, then X would not have necessarily happened in this same way). As the control inference using ‘not necessarily’ was not the exact negation of the target inference for reasons of simplicity, we had no strong prediction as to how differently the target and control inference would be endorsed for the control premise using ‘like this’; but we expected this difference to be important for the target inference using a post-speech musical gesture (without ‘like
this') which was expected to trigger a supplemental inference just as post-speech gestures do (Tieu et al., 2019; Schlenker, 2018a). We thus expected an interaction between the two factors, with a higher difference in endorsement between the target inference and the baseline inference in response to the target premise than in response to the control premise.

### Model

$$ \text{value} \sim \text{GestureC} * \text{InferenceC} + (1 + \text{GestureC} * \text{InferenceC}|\text{SubjID}) $$

This first model failed to converge, leading us to simplify as before the random effect structure by removing the interaction:

### Simplification of random effects structure 1

$$ \text{value} \sim \text{GestureC} * \text{InferenceC} + (1 + \text{GestureC} + \text{InferenceC}|\text{SubjID}) $$

This second model converged but did not allow for interaction testing (removing the interaction from the model prevented the model from converging), leading us to simplify the random effect structure even more:

### Simplification of random effects structure 2

$$ \text{value} \sim \text{GestureC} * \text{InferenceC} + (1 + \text{InferenceC}|\text{SubjID}) $$

<table><thead><tr><th>(Intercept)</th><th>GestureC</th><th>InferenceC</th><th>GestureC:InferenceC</th></tr></thead><tbody><tr><td>57.175</td><td>-7.085</td><td>-26.858</td><td>8.774</td></tr></tbody></table>

## Homogeneity inferences - NP and Predicate

The case for homogeneity is analogous to that of presuppositions in 6, where only inference type was used as a fixed effect in the model, while the effect of inference type **by participant** (testing whether each participant had a personal tendency to endorse the inferences) was used as a random effect. As we were interested in whether the homogeneous ('all' or 'none') inference would be preferred to the non-homogeneous one in each environment (positive and negative), we are here only interested in the effect of inference type (homogeneous vs non-homogeneous).

### Model

$$ \text{value} \sim \text{InferenceTypeC} + (1 + \text{InferenceTypeC}|\text{SubjID}) $$

This model did not converge, so we decided, as with presuppositions, to go with the most minimal random effect structure only accounting for the difference in intercepts between participants:

### Simplification of random effects structure

$$ \text{value} \sim \text{InferenceTypeC} + (1|\text{SubjID}) $$
<figure><img src="image_12.png" /><figcaption>Figure 8: Reading flipping under negation</figcaption></figure>

<table><thead><tr><th>Hypothesis tested</th><th>Environment</th><th>(Intercept)</th><th>InferenceC</th></tr></thead><tbody><tr><td>NP</td><td>Positive</td><td>51.49</td><td>-59.85</td></tr><tr><td>NP</td><td>Negative</td><td>47.91</td><td>-38.87</td></tr><tr><td>Predicate</td><td>Positive</td><td>54.71</td><td>-61.34</td></tr><tr><td>Predicate</td><td>Negative</td><td>40.31</td><td>-37.34</td></tr></tbody></table>

Note that in our analysis, we were interested in inference type, i.e. whether the homogeneous inference was significantly more endorsed than its non-homogeneous counterpart. However, as the editor pointed out, it would also have made sense to analyze the inference type in terms of *universal* or *existential*. As described in the paradigm in 3.4, we expected musical gestures (that were punctuated repetitions) to have a universal reading in the positive environment ('all harps') and to get an existential reading in the negative environment ('at least one harp', i.e. 'some harps'), just as the reading of gestures and visual animations were shown to flip from universal to existential depending on the context in (Tieu et al., 2019). If we perform such an analysis, we are interested in the interaction between environment (positive vs negative) and inference type (understood as universal vs existential, and not as homogeneous vs non-homogeneous anymore). We plot the results with this new analysis and report the significance and coefficients of what the model would have been in this situation below:

## Model

$$ \text{value} \sim \text{InferenceTypeC} * \text{Environment} + (1 + \text{InferenceTypeC} * \text{Environment}|\text{SubjID}) $$

This model failed to converge, leading to a simplification of the random effects structure as follows:

### Simplified model

$$ \text{value} \sim \text{InferenceTypeC} * \text{Environment} + (1 + \text{InferenceTypeC} + \text{Environment}|\text{SubjID}) $$
The comparison of this model to the same model without the interaction between inference type and environment showed a significant difference between the two models ($\chi^2 = 84$, p < 0.001). This supports the idea that the reading of a same musical gesture flips under negation, which is consistent with our first analysis.

## Iconicity controls

For each iconicity control, we were interested in the difference in endorsement between the inference containing the matching interpretation of the iconic modulation and the inference that contained the opposite non-matching interpretation. For instance, we wanted to know whether the inference containing the interpretation of a long upward scale as a high mountain was more endorsed than the inference containing the interpretation of a long upward scale as a low mountain. We thus counted inference type as a fixed effect. To model the possible differences in the tendencies to endorse both types of inferences in a certain way for each participant, we included inference type by subject as a random effect in the model, as shown:

### Model

$$ \text{value} \sim \text{InferenceTypeC} + (1 + \text{InferenceTypeC|SubjID}) $$

<table><thead><tr><th>Control</th><th>(Intercept)</th><th>InferenceC</th></tr></thead><tbody><tr><td>1</td><td>55.09</td><td>52.11</td></tr><tr><td>2</td><td>52.75</td><td>-28.37</td></tr></tbody></table>
# Appendix III: Inferential mechanisms

Here, we provide a loose description of the underlying mechanisms responsible for the four types of inferences tested. Although an important part of its content is itself subject to debate, the aim of this table is merely to provide some very basic analysis of each inference to facilitate the reading of each section; it does *not* aim at exhaustively accounting for all possible theoretical models.

<table><caption>Figure 9: Description of each inferential type</caption><thead><tr><th>Inference</th><th>Input</th><th>Mechanism</th><th>Output</th></tr></thead><tbody><tr><td>PRESUPPOSITION</td><td>Bivalent<sup>22</sup> truth-conditions (from either linguistic or non-linguistic items)</td><td>Takes the input <i>q</i> and produces a pair &lt;<i>p</i>, <i>p</i>&gt; with <i>q</i> equivalent to <i>p</i> &amp; <i>p</i>' where <i>p</i> is the presupposition.</td><td>Presupposition + Assertion<sup>23</sup></td></tr><tr><td>SCALAR IMPLICATURE</td><td>Alternatives (non-necessary linguistic) with their bivalent content.</td><td>Competition between alternatives (Gricean or grammatical mechanism)</td><td>Ordinary meaning + Strengthened meaning</td></tr><tr><td>SUPPLEMENT</td><td>Bivalent content + syntactic form</td><td>Whatever allows a non-restrictive relative clause to get the meaning of a supplement [e.g. comma intonation (Potts, 2015)]</td><td>Supplemental inference</td></tr><tr><td>HOMOGENEITY INFERENCE</td><td>Previous theories: Definite plural NP [or non-linguistic item referring to definite pluralities such as punctuated gestures or sounds]<br>---------------------------<br>More recently (Križ, 2020): Trivalent predicate [or non-linguistic predicate like gestural or musical]</td><td>Assigns the input a universal reading ('all') in positive environments and an existential reading in negative environments<br>---------------------------<br>If a group is in the extension of the predicate, then all members of the group are in the extension of the predicate as well (positive environment) or none of them (negative environments)</td><td>Homogeneous inference</td></tr></tbody></table>

<footnote n="22">22 Bivalence refers to the existence of pre-conditions to the meaning.</footnote>

<footnote n="23">23 The output is actually closer to &lt;presupposition, presupposition+assertion&gt;, because the assertion in &lt;presupposition, assertion&gt; is underdetermined.</footnote>
Chapter 3

# Meaning from music in audiovisual scenes

## Purpose

The previous chapter explored the interaction between music and language, and revealed that musical meaning can be integrated with linguistic meaning when musical stimuli are embedded in language. In this chapter, we explore musical meaning through its interaction with visual animations. By creating audiovisual stimuli in which musical meaning combines with visual information, we show that musical properties can be mapped onto properties of visual objects through systematic rules.
# HOW MUSIC CAN REFER TO OBJECTS

## COREFERENCE FROM DYNAMICS, PITCH AND TIMBRE IN AUDIOVISUAL STIMULI *

**Léo Migotti**

Institut Jean Nicod (ENS-EHESS-CNRS)

**Emmanuel Chemla**

Département d'Etudes Cognitives (ENS-EHESS-CNRS)
Laboratoire de Sciences Cognitives et Psycholinguistique (CNRS)

**Philippe Schlenker**

Institut Jean Nicod (ENS-EHESS-CNRS)
Department of Linguistics, New York University

## ABSTRACT

Building upon recent theories of musical meaning that rely on the assumption that music conveys information about an extra-musical world, this paper presents an experimental paradigm testing semantic associations between musical properties and visual objects. It first brings evidence that when an object is present in a visual scene paired with some music, pitch and dynamics can be interpreted as the level of energy of this object, or as its distance from a viewpoint. Second, it shows that pitch and dynamics can be associated with specific objects in scenes that contain multiple objects. Third, it shows that timbre too encodes reference to objects. Together, these results demonstrate that musical properties establish reference to specific objects, and that those reference relations are established between music and objects themselves, rather than between music and the scenes containing these objects.

**Keywords** Music semantics | Coreference | Pitch | Dynamics | Timbre | Audiovisual integration

<footnote>* *Contributions*: LM, EC and PS designed the experiment based on previous work by PS. LM and EC worked on the details of the experimental paradigm, and LM and PS worked on clarifying the underlying theories. LM created the audiovisual stimuli, programmed the experiment, managed data collection, analyzed the data and wrote the paper, with contributions and proofreading by PS and EC.</footnote>
# Contents

<table><tr><td>1 Introduction</td><td>86</td></tr><tr><td>2 Background</td><td>87</td></tr><tr><td>2.1 Variables across media</td><td>88</td></tr><tr><td>2.2 Coreference in music</td><td>89</td></tr><tr><td>3 Testing the semantics of pitch and dynamics</td><td>91</td></tr><tr><td>3.1 Theoretical framework</td><td>91</td></tr><tr><td>3.2 Experimental design</td><td>92</td></tr><tr><td>3.2.1 General experimental procedure</td><td>92</td></tr><tr><td>3.2.2 Design and stimuli of block 1</td><td>93</td></tr><tr><td>3.3 Results</td><td>95</td></tr><tr><td>4 Testing coreference with pitch and dynamics</td><td>97</td></tr><tr><td>4.1 Experimental paradigm</td><td>97</td></tr><tr><td>4.1.1 Target stimuli</td><td>97</td></tr><tr><td>4.1.2 Control stimuli</td><td>98</td></tr><tr><td>4.2 Results</td><td>99</td></tr><tr><td>5 Testing coreference with timbre</td><td>102</td></tr><tr><td>5.1 Motivation</td><td>102</td></tr><tr><td>5.2 Experimental design</td><td>104</td></tr><tr><td>5.3 Results</td><td>105</td></tr><tr><td>6 Conclusion</td><td>106</td></tr><tr><td>Bibliography</td><td>111</td></tr></table>
# 1 Introduction

Although many accounts of musical meaning have highlighted the ability of music to represent things in the non-musical world,<fnref n="2" /> only recent work has established formal theories explaining how such mechanisms are possible (Schlenker, 2017). This research agenda relies on the idea that music can be *true* of certain situations: there are certain conditions under which a given musical excerpt can represent (or *denote*) a given situation, and systematic rules that map a certain musical form to all things it can be true of.

Among all candidate musical properties involved in these mechanisms, some hypotheses have been given in the literature about the interpretation of pitch and dynamics (Schlenker, 2017; Eitan and Granot, 2006). In particular, pitch is claimed to be interpreted as the level of energy of those objects, while dynamics are claimed to be either interpreted as energy or distance from a viewpoint. For a situation to be true of a rise in pitch for instance, it requires that this situation involve a gain in energy.

But music can do more than just represent some situations better than others. It has been highlighted that it can also refer to objects in these situations (Schlenker, 2022): not only are musical properties interpreted when objects are given, they might also help a listener understand that a certain musical passage is *about* (or refers to) a certain object, and that another passage refers to another object. For instance, a recurring melodic pattern might systematically refer to a same character in opera and hence generate **coreference**. By contrast, certain changes in dynamics (i.e. playing more or less loudly) might instead indicate that two objects are being referred to instead of one, hence generating **disjoint reference**.

The same body of research has finally proposed that timbre is of a particular nature with regards to its semantic interpretation. Because timbre seems to have the inherent ability to

<footnote n="2">²Among many examples from the literature, we can cite (i) several accounts on how music can evoke movement in Godøy and Leman (2010), in particular thanks to pitch height and loudness Eitan (2013); Eitan and Granot (2004) how some musical properties can be interpreted as analogs to physical forces in Larson (2012), and more theoretical accounts on the music/movement relationship in Clarke (2001). (ii) A rich literature on how music can convey emotional information includes Blumstein et al. (2012), Juslin and Laukka (2003) on similarities between musical and vocal performance in their ability to trigger emotional effects, and Koelsch (2012) for a review in brain-related processes responsible for these effects. A substantial body of research has also investigated how music and language might share some properties explaining their ability to convey meaning in Cross and Woodruff (2009), and how different parameters such as pitch height or intensity can be related to different emotional interpretations in music and speech. (iv) Sievers et al. (2013, 2019) offer an account on how musical and visual properties can encode similar emotional effects as well. See also (v) Gabrielson and Lindström (2010) for a discussion of the contribution of uninterpreted musical structure. Schlenker (2017) provides further references in particular to review works on related topics.</footnote>
be indicative of the very nature of an object, it often binds musical motives to objects in a direct fashion. Schlenker (2022) argues that orchestration makes great use of the semantics of timbre to indicate coreference or disjoint reference in orchestra pieces. To give listeners the impression that two repetitions of a same musical motive correspond to two different objects, composers might typically vary timbre.

This paper has three main goals. First, it tests the hypotheses about the semantic interpretation of pitch and dynamics through an audiovisual experimental paradigm involving musical sounds and objects in visual animations. Second, it tests whether pitch and dynamics can make reference to specific objects, i.e. whether it is possible to manipulate pitch and dynamics so that one knows from the music which object is being referred to. Third, it tests the semantics of timbre and shows that timbre can establish reference to objects, and that these reference relations are established to objects directly, and not merely to the general scene(s) involving these objects. After presenting some background about coreferential effects across media and in music more specifically in section 2, we present the experimental paradigm in three steps. Section 3 presents how the predictions on the semantics of pitch and dynamics were tested. Section 4 explains how those results were reused to establish the ability of pitch and dynamics to make reference to objects. Section 5 finally presents how timbre too can make reference to objects and provides an argument in favor of a direct association between musical property and object: listeners bind music to objects and not merely to visual scenes.

## 2 Background

While recent work has established that referring to external objects is not unique to language, most accounts of reference relations in the literature address the case of natural languages. The idea that it is possible for expressions in natural languages to refer to a same entity (establishing coreference to this entity) is an old one. Consider the following sentence in (1).

(1) Mary said that she would come.

1a. Mary$_i$ said that she$_i$ would come.

1b. Mary$_i$ said that she$_j$ would come.
The sentence in (1) can be interpreted as either 1a, namely that Mary said that Mary herself would come, or else as 1b, namely that Mary said that another person referred to by the third-person pronoun ‘she’ would come. In 1a., ‘Mary’ and ‘she’ are coreferential: they refer to the same person Mary, while in 1b., the relation between ‘Mary’ and ‘she’ is that of a disjoint reference: ‘Mary’ and ‘she’ refer to two different persons. Note that one way of explaining how the ambiguity is lifted (which can be traced back to Montague (1973) and Karttunen (1969)) is to call for a notion of variable (aka discourse referent), i.e. abstract objects assigned to different items in the sentence, as shown in (1) through indexes *i* and *j*.

Recent work has evidenced the existence of coreference relations beyond natural language.
The next section highlights two particular instances in pictorial narratives and dance.

**2.1 Variables across media**

In Abusch (2013, 2017, 2020), coreference is found in comics where the depiction of a character shown on two different panels is understood as depicting the same character. The same idea can be represented in (2) taken from Abusch (2017) and represented in Schlenker et al. (2023) as follows:

<figure><img src="image_13.png" /></figure>

The most salient interpretation of (2) is that the light cube in Picture 2 refers to the
same cube as the light cube in Picture 2, and that the dark cube in Picture 2 refers to the
same cube as the dark cube in Picture 2. Another (less natural) interpretation of (2) is
that instead, the dark cube in Picture 1 was replaced by another dark identical dark cube
in Picture 2. Although this sequence of pictures is thus initially ambiguous, the former
coreferential interpretation seems preferred. And here again, one possible way to account for
these coreference relations is to posit the existence of two discourse referents that index each
cube across the two panels, as shown in (3).