Donna Erickson (Haskins Labs, USA;
Sophia University, Tokyo, Japan)&Malin Svensson Lundmark (LundUniversity, Sweden; Queen MargaretUniversity, UK)
Times: 9-12 noon(with 10 minute coffee break)

Articulatory Prosody& the syllable
UF PR
October 23-27, 2023

Room 400 (4th floor of D. Pedro IBuilding)
UFPR Rectory Campus
ericksondonna2000@gmail.com
Malin.svenssonlundmark@gmail.com

Week schedule•Monday: Experimental data about jaw displacement &prominence•Tuesday: A model of temporal organization speech.
The Fujimura C/D model•Wednesday: Roller coaster articulation: Acceleration,deceleration, jerks & flat places•Thursday: More Articulatory prosody•Friday: Your reports. Choose a sentence –in any
language_ and present an articulatory description

Day 1

Prominence,phrasing, andboundaries

Acoustics& Articulation

Syllable
articulation

My working hypothesis

Patterns of prominence,phrasing & boundaries=language RHYTHM

Prominence,phrasing andboundaries
Pam said pat that fat cat at that mat.
Pam said pat that fat cat at that mat.
Pam said pat that fat cat at that mat.
Pam said/ pat that fat cat/ at that mat.
Pam said/ pat that, fat cat/ at that mat.
Pam said/ pat that, fat cat/ at that mat.
This Photo by Unknown Author is licensed under CC BY-SA

Prominence, phrasing andboundaries
I gave dates to Kay today
I gave dates/ to Kay today

This Photo by Unknown Author is licensed under CC BY-NC

A metrical approach: Metrical grid(I)

toPhrasal stress

toNuclear stressPhrase break

Selkirk 1982, Hayes 1995

A metrical approach: Metrical tree

Phrasal stress

Nuclear stressPhrase break

Liberman & Prince, 1977

Acoustic cues of prominence,phrasing and boundaries

ToBI—a descriptive acoustic approach

207 Hz
70 rms0.24 s175 Hz
67 rms0.34 sI

gavedates

toKaytoday

Articulation of prominence,phrases & boundaries

Our path today
How to measure articulation
Definition of syllable in terms of articulation
Jaw displacement & prominence
Jaw & syllable edges
Jaw displacement & boundaries

How do we measurearticulation?

Electromagnetic(EMA)
Recordings

The articulatory movement and speech data were recorded simultaneously forusing 3D EMA (Carstens AG500) at the Japan Advanced Institute of Scienceand Technology (JAIST), Ishikawa Prefecture, Japan.

“X-ray micro-beam speech production database user’s hand-book“ (Westbury et al. 1994);(https://github.com/rsprouse/ xray_microbeam_databaseUltrasound&video recording

MARRYS Helmet
being developed
by Oliver
Niebuhr,Southern
Denmark U.

https://docs.google.com/forms/d/1iqP7kk3rFvtvxtPDkguQC1W40DNsu1A_oN1QjM5oyM/edit?ts=6531c003

MARRYS &UltrasoundAcoustic (C)V(C)
description
SYLLABLES
Onset-nucleus-coda

Articulatory
descriptionArticulatorysyllable

CodaOnset• the mouth (jaw) opens &closes

TK{T, τ}{K, τ}

Jaw is closed

Nucleus
IThis Photo by Unknown Author is licensed under CC BY-NC-ND

Jaw is open

Jaw is closed

Articulatory syllable
Syllable=opening & closing ofmouth (jaw)

Depending on vowel height, thejaw opens more or opens less.

Vowel intrinsic jaw displacement• Jaw displacement as a function ofvowel height, as spoken by an
American English midwestern dialectspeaker.
• Pick type first, type pick first, first
type pick.
• The x-axis shows vowel; the y-axis
the amount of jaw displacement (mm)as measured from the occlusal planewhere the jaw is maximally open.
(Williams et al. 2013).

Jawdisplacement& prominenceProminence displacement:
Depending on the prominence, the jaw
opens less or more

Jaw displacement &prominence
If we look at utterances where all thevowels are the same, we see different
amounts of jaw opening, even though all
the vowels are phonologically the same.
If a syllable is emphasized, we especiallysee increased jaw displacement.

Pam said bat that fat cat
at that mat

Syllable
prominence
patterns

What happens when weemphasize a word?
BAT, THAT, FAT, CAT
Prominence of each syllable
is indicated by height of each
Syllable bar in the graph,based on theamount of jaw displacement

Strings of syllables

Thecatchased thedog

NuclearStress & Jaw
dsiplacement

Utterance prominence patterns—
Patterns of jaw openings
FSPS

NSA series of aɪ vowels in an utteranceMore on utterance prominence patterns

NUCLEAR STRESSPHRASAL STRESSFOOT STRESSMetrical structure & Jaw displacement patterns

Erickson et al. 2015

Default
Nuclear Stressvs Intent
Nuclear Stress

I saw five bright highlights in the sky tonightUtterance
PhraseFootWordSyllablexxxxxxxxxx

Stress level

325(Yes, I saw)

five bright high lightsxxxxxxx

142skynightI gave dates
to Kay today

What about jaw
displacement
patterns of“normal” utteranceswith words
containing different
vowels?
e.g. Kip met Pat vs.
Pat met Kip.

Neutralizing Intrinsic Vowel Jaw displacement (Williams et al 2013)=>
Metrical structure matches jaw displacement patterns

Jaw displacement & prominence:
languages other than English

Languages like
French, Mandarin
Chinese, Japanese,Spanish are basedon hierarchicalarrangements ofphrases.

This Photo by Unknown Author is licensed under CC BY-SA-NC

FrenchIntonational
PhraseAccentual
PhraseSyllable

Syll. Numberxxxxxx x x x x x x x x x x x x x xna ta ʃa na ta ʃa pa sɔ̃ ʃa pa ʃa ki se ʃa pa
so chchch ppchch
Na ta a n'a tta a pas n at Pa a qui s'e a a
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Japanese

Therefore, Mana’s head feels smoothMandarin Chinese妈妈骂馬 (Mā1ma mà4 mǎ3) Mother curses the horseSpanishSecond language
learners carryover their first
language jaw
displacement
patterns

This Photo by Unknown Author is licensed under CC BY-NC-ND

English speaker learning FrenchEnglish speaker learning Japanese

They are red pajamas

Language
learning:
Japanese learning
EnglishFrench learning
EnglishHow to teachprominence patterns• Shadowing• RPT• Repeat after me• Clap hands to thebeat

JawDancing“Let’s practice ‘Jaw Dancing’! ‘Jaw Dancing’ is how
you move your mouth to make speech sounds; it ishow your mouth ‘dances’ as you speak. The mouth
moves/dances differently for Japanese and forEnglish. In this exercise, we hope to teach you how
to do ‘American English jaw dancing’! If you do‘American English Jaw Dancing’, you will soundmore like a real American English speaker! Come on,let’s dance!!”Jaw Dancing: I saw five bright highlights in the sky tonightSky tonight—
F1 & F2formantpatterns

Jaw dancing:sky NIGHT=>
SKY night

Summary
1. jaw displacement increases with syllable prominence
2. jaw displacement patterns “match” metrical structure
3. different languages have different jaw displacement patterns
based on the language metrical structure
4. L2 speakers tend to carry over their L1 jaw displacement
patterns

Acoustic
consequencesof opening
mouth more

This Photo by Unknown Author is licensed under CC BY

Significant
relation
between F1& jaw
displacement(Erickson et al.
phonetica 2012)

Jaw& F1

Erickson et al. 2014 (speechprosody)

F0 & jaw displacement. JapaneseJaw &F0Mandarin

Jaw displacement & F0. English

Erickson et al. 2014

F0 & Jaw displacement. English emotion‘Admiration has a very wide jaw opening
and high pitch, showing strong stress on‘wonderful’ without pitch manipulation.
Anger does not show any peculiar effecton this syllable with main stress,
but see the next slide.

That’s wonderful

Do listeners
HEAR “a bigger
mouthopening”?

RPT (Rapid Prosodic Transcription)

Which word is most prominent in this utterance, which word stands out more?

Igave

dates

kaytodayI gave
dates toKay today

More about perception,
acoustics & articulation•

Erickson, D., SvenssonLundmark, M., and Huang, T.
(in press). Jaw Opening
Patterns and their
Correspondence with Syllable
Stress Patterns. In L. Meyer,
A. Strauss (eds). Cambridge
University Press (CUP).

Jaw & syllable edges

Articulatory
syllable

CodaOnset• the mouth (jaw) opens &closes

TKT, τK, τJaw is closed

NucleusI

This Photo by Unknown Author is licensed under CC BY-NC-ND

Jaw is open

Jaw is closed

An articulatory description of an utterance:
A string of SYLLABLES

TDlipsTTTTTDTT

Onset & coda
articulators
Vowelarticulator(I) gave dates to Kay today

Syllablearticulator

Articulation of onset, nucleus, & coda

Crucial Articulator:TongueTip
Crucial Articulator:TongueBack?

Crucial Articulator:LIP

Vowel articulator
Syllable articulator

Igave

dates toKaytoday

Articulation of onset, nucleus, & coda

Crucial Articulator:TongueTip
Crucial Articulator:TongueBack
Crucial Articulator:LIP

Syllable articulatorIgavedates toKaytodayJaw displacement& boundaries
New Territory!

Which word is most prominent in this utterance, which word stands out more?
Pamsaidpatthatfatcatatthatmat

A03

Nuclear Stress & Jaw dsiplacement

A03

After which word do you hear a break?
Pamsaid

pat

that

fat

cat

at

that

mat

A03

Pam said bat that fat
cat at that mat
Results from Rapid
Prosodic Transcription(RPT))JAW Displacement
N=18

Erickson et al. 2015

N=35

More boundary exercises

After which word do you hear a break?

Igavedateskaytoday

ToBI—a descriptive acoustic approachI

gavedates

toKay

today

What signals aboundary in thearticulatory signal?

This Photo by Unknown Author is licensed under CC BY

Jaw displacement, jaw velocity & boundaries

Igave dates

toKay

today

π-gesturePhrase boundaries are modeled as
prosodic gestures(π-gestures) with atemporal activation interval, similar toconstriction gestures. (Byrd&Saltzman,2003; Byrd etal.,2000).
But rather than activating aconstriction variable, a π-gesture acts
to locally slow down the clock thatcontrols the temporal unfolding ofarticulatory gestures during the interval
when they are active
Metrical hierarchy =>patterning of prominences & boundaries
i.e., interaction between syllable magnitude& crucial articulators=>boundary location &strength.

More about speed of articulators

Crucial Articulators for Pam

Displacement& speedKim et al. 2014

Does jaw open
faster when it has
to move further?
• we think “YES” but we need toanalyze the data.
• Also, we hypothesize interactions
between crucial articulators &syllable prominence, i.e, Jawdisplacement.
• => location & size of boundaries• =>This Photo by Unknown Author is licensed under CC BY-NC

My working hypothesis
Patterns of prominence, phrasing &boundaries=
language RHYTHM
Which is the output of

Metrical structure& the Fujimura C/D modelaccount of temporalorganization of spoken
utterances
Fujimura, 0. (2000) "The CID model and prosodic control ofarticulatory behavior." Phonetica 57(2-4), 128-138.

HOMEWORK for Tuesday, Oct. 24
1. Please use PRAAT and make a ToBI-type analysis of this sound file,” Pam said bat that fat cat at the mat.” (segment words, indicate phrase breaks,
and prominent syllables)
2. Read the two papers (https://cdmodel.wordpress.com/)
Fujimura, O. (2000). The C/D model and prosodic control of articulatory
behavior. Phonetica, 57, 128-138.
Fujimura, O., Erickson, D., and Wilhelms, R. (1991) Prosodic effects onarticulatory gestures--A model of temporal organization. Proceedingsof the XIIth International Congress on Phonetic Sciences, 2, 26-29.