Guidelines for RT-03 Transcription

Linguistic Data Consortium

Version 2.2 -- February 23, 2003

Table of Contents

1 Introduction [3](#introduction)

2 Segmentation [3](#segmentation)

2.1 Overview [3](#overview)

2.2 Broadcast News [3](#broadcast-news)

2.2.1 Overview [3](#overview-1)

2.2.2 Sections [4](#sections)

2.2.2.1 Section types [4](#section-types)

2.2.2.1.1 Introduction [4](#introduction-1)

2.2.2.1.2 News stories \<sr\> [4](#news-stories-sr)

2.2.2.1.3 Teasers \<st\> [4](#teasers-st)

2.2.2.1.4 Non-news \<sn\> [4](#non-news-sn)

2.2.2.2 Beginning of a section [4](#beginning-of-a-section)

2.2.2.3 End of a section [5](#end-of-a-section)

2.2.2.4 End of the file [5](#end-of-the-file)

2.2.3 Turns [5](#turns)

2.2.3.1 Introduction [5](#introduction-2)

2.2.3.2 New speaker turns [5](#new-speaker-turns)

2.2.3.3 End of a turn [5](#end-of-a-turn)

2.2.3.4 Overlapping speech [5](#overlapping-speech)

2.2.3.5 Speakers start simultaneously
[6](#speakers-start-simultaneously)

2.2.3.6 Short periods of non-speech [6](#short-periods-of-non-speech)

2.2.3.7 Extended periods of non-speech
[7](#extended-periods-of-non-speech)

2.2.4 Breakpoints [7](#breakpoints)

2.2.4.1 Overview [7](#overview-2)

2.2.5 Summary of segment boundary symbols in broadcast news
[7](#summary-of-segment-boundary-symbols-in-broadcast-news)

2.2.6 Speaker Identification [7](#speaker-identification)

2.2.6.1 Introduction [7](#introduction-3)

2.2.6.2 Speaker Type [8](#speaker-type)

2.2.6.3 Names and Identifiers [8](#names-and-identifiers)

2.2.6.4 Native and non-native speakers
[8](#native-and-non-native-speakers)

2.2.6.5 Examples of speaker IDs [8](#examples-of-speaker-ids)

2.3 Telephone Speech [8](#telephone-speech)

3 Transcription [9](#transcription)

3.1 Introduction [9](#introduction-4)

3.2 Transcription Conventions [9](#transcription-conventions)

3.2.1 Orthography and spelling [9](#orthography-and-spelling)

3.2.1.1 Capitalization [9](#capitalization)

3.2.1.2 Spelling [10](#spelling)

3.2.1.3 Contractions [10](#contractions)

3.2.1.4 Numbers [10](#numbers)

3.2.1.5 Hyphenated words and compounds
[10](#hyphenated-words-and-compounds)

3.2.1.6 Abbreviations [11](#abbreviations)

3.2.1.7 Acronyms and spoken letters [11](#acronyms-and-spoken-letters)

3.2.1.8 Punctuation [11](#punctuation)

3.2.2 Disfluent speech [11](#disfluent-speech)

3.2.2.1 Introduction [11](#introduction-5)

3.2.2.2 Filled pauses and hesitation sounds
[12](#filled-pauses-and-hesitation-sounds)

3.2.2.3 Partial words [12](#partial-words)

3.2.2.4 Restarts [12](#restarts)

3.2.2.5 Mispronounced words [12](#mispronounced-words)

3.2.3 Noise [12](#noise)

3.2.3.1 Speaker noise [12](#speaker-noise)

3.2.3.2 Background noise [13](#background-noise)

3.2.4 Additional markup [13](#additional-markup)

3.2.4.1 Hard-to-understand sections [13](#hard-to-understand-sections)

3.2.4.2 Idiosyncratic words [13](#idiosyncratic-words)

3.2.4.3 Foreign languages [14](#foreign-languages)

3.2.4.4 Proper nouns [14](#proper-nouns)

3.2.4.5 Interjections [14](#interjections)

3.2.4.6 Summary of special symbols [16](#summary-of-special-symbols)

3.3 Some general considerations [17](#some-general-considerations)

4 Second Passing [17](#second-passing)

4.1 Introduction [17](#introduction-6)

4.2 Segmentation and Speaker ID Verification
[17](#segmentation-and-speaker-id-verification)

4.3 Transcription verification [17](#transcription-verification)

4.4 Automatic checks [18](#automatic-checks)

#   {#section .unnumbered}

# 

# Introduction

The goal of the transcription process is to provide an accurate,
verbatim (word-for-word) transcript of the entire broadcast. The
transcript will be time-aligned with the audio file, and additional
features of the audio signal and speech will be identified using special
markup.

# Segmentation

## Overview

The segmentation process begins with creation of initial timestamps for
the audio file. Timestamps indicate when different things are happening
in the audio, and so allow us to align the transcript with the
corresponding audio file. Timestamps also make transcription of the
audio easier, by allowing the transcriber to listen to small chunks of
segmented speech at a time.

Timestamps must occur at regular intervals within each audio file. At a
minimum, timestamps must identify

-   **section boundaries** within Broadcast News files

-   **speaker turns** (change of speaker) for all files

In addition, transcribers insert additional **breakpoints** within each
speaker\'s turn. This helps break up long turns into more manageable
units, and makes transcription easier.

Some things to consider when inserting timestamps of any kind:

-   Timestamps must never occur in the middle of a word.

-   Be careful not to clip off the end/beginning of a word when
    inserting a timestamp. This is trickiest with certain sounds, like
    \"s\", \"f\", \"t\", \"k\", \"p\". Take special care when inserting
    timestamps around words that begin or end with these sounds.

Good places to insert timestamps are

-   at pauses

-   at breaths

-   at ends of sentences or phrases

## Broadcast News

### Overview

Broadcast news data comprises a single audio channel. Multiple speakers
appear on this single channel, and each speaker is identified by name or
unique speaker ID. More than one speaker may talk simultaneously,
resulting in sections of overlapping speech.

Broadcast news files are hierarchically arranged into Sections, Turns
and Breakpoints. All section boundaries are identified and timestamped
within a broadcast file. However, only news stories and teaser sections
are further segmented into turns and breakpoints. Commercials and other
non-news sections are not further segmented (see Section 2.2.2.1 below
for definitions of each section type).

Within broadcast files, timestamps indicate only the start time of the
section, turn or breakpoint boundary. Because broadcast speech occurs on
a single audio channel, timestamps occur one after the other, in direct
succession and typically without intervening periods of unsegmented
audio. The end time of a timestamp is implied by the start time of the
following timestamp. For instance, a series of timestamps in a broadcast
file might look like this:

\<sr 56.827\>

\<b 60.467\>

\<t 63.980\>

\<b 67.989\>

\<b 71.501\>

Note that these timestamps occur in direct succession, with no large
gaps.

### Sections

#### Section types

##### Introduction

Each new part of the broadcast should be identified and labeled with the
appropriate section label. There are three types of section boundaries:

-   \<sr\> refers to news reports

-   \<st\> refers to teasers

-   \<sn\> refers to non-news sections, including commercials

##### News stories \<sr\>

A news story is a topically contiguous segment of the broadcast. News
stories may be of any length as long as they constitute a complete,
cohesive news report on a particular topic. Note that single news
stories may discuss more than one related topic. When reports of similar
content are adjacent to one another in a news broadcast, it is often
difficult to tell where one story ends and the next begins. Annotators
rely on audio cues (speaker changes, music, pauses) to inform their
judgments.

##### Teasers \<st\>

Teasers consist of promotional spots for upcoming stories or very brief
reviews of top stories reported on more fully within the same broadcast.
Teasers are quite limited in their format and their placement within an
audio broadcast. They typically occur at the beginning of a broadcast or
preceding a commercial break, and are designed to capture and hold the
listener's attention for stories that will be reported on later in the
broadcast. Brief (one- to two-sentence) reviews of top headlines are
also categorized as teasers.

##### Non-news \<sn\>

Non-news segments include commercials, reporter chit-chat outside of the
context of a story, station identifications, public service
announcements, promotions for upcoming broadcasts and long musical
interludes. If multiple non-news sections follow one another within a
transcript, they are grouped together with a single \<sn\> tag at the
beginning of the section.

#### Beginning of a section

At the beginning of a new section, annotators insert the appropriate
section label and timestamp. Because each section implies a new speaker
turn, a speaker ID is also inserted at the start of each new section
(see Section 2.2.6). For instance:

\<st 21.232\> \<\<male, Lou_Waters\>\>

The last great explorer \^Jacques \^Cousteau has died in \^Paris at age
eighty-seven.

\<b 25.907\>

{breath} Part of Early Prime is being preempted so that for the next
half hour we can remember one of the giants of the twentieth century.

#### End of a section

If the end of a section is directly followed by the start of another
section, there is no need to specifically label and timestamp the end of
the first section.

If the section is followed by a period of non-speech (music, sound
effects or silence), annotators explicitly timestamp and label the end
point of the section with \<e\>.

#### End of the file

Each file must end with a final timestamp, indicating where the audio
recording for that program concludes. This timestamp should be labeled
with \<e\> to indicate end.

### Turns

#### Introduction

Within sections there are turn boundaries representing the start of each
speaker\'s turn in the broadcast or telephone call.

#### New speaker turns

Every time there is a speaker change in the audio, this is indicated by
inserting a \<t\> turn marker and timestamp. For Broadcast News, each
\<t\> segment must also receive a speaker ID.

For Broadcast News files, annotators indicate \<t\> segments only within
\<sr\> and \<st\> sections. They do not indicate turn boundaries within
\<sn\> sections, since these sections will not be transcribed.

For Telephone Speech files, annotators indicate \<t\> segment boundaries
for the entire file.

#### End of a turn

If the end of one speaker\'s turn is directly followed by the start of
another speaker\'s turn, there is no need to specifically label and
timestamp the end of the first speaker\'s turn.

If a speaker\'s turn is followed by a period of non-speech (music, sound
effects or silence), then annotators explicitly timestamp and label the
end point of the speaker\'s turn with \<e\>.

#### Overlapping speech

A special subclass of breakpoints marks the beginning and end points of
overlapped speech; that is, periods of the recording where there are
multiple speakers talking at once. The notation \<o\> is used to mark
the beginning of the overlap section. Each \<o\> must also be associated
with a new speaker ID.

Within the \<o\> section, INTERRUPTEE-1 refers to the person who was
talking when the interrupt occurred. INTERRUPTER-2 refers to the person
who does the interrupting. The Speaker ID associated with the start of
the \<o\> will always correspond to the person who\'s doing the
interrupting. For instance:

+---------------------------------------+------------------------------+
| \<t 3474.016\> \<\<female,            | -   *Natalie Allen is        |
| Natalie_Allen\>\>                     |     already talking here.*   |
|                                       |                              |
| That\'s Early Prime. We thank you for |                              |
| watching. I\'m \^Natalie              |                              |
+---------------------------------------+------------------------------+
| \<o 3476.526\> \<\<male,              | -   *\<o\> indicates start   |
| Lou_Waters\>\>                        |     of the overlap. The      |
|                                       |     SpeakerID is for the     |
| INTERRUPTEE-1:\^Allen                 |     INTERRUPTER*             |
|                                       |                              |
| INTERRUPTER-2:I\'m \^Lou              | -   *Natalie Allen keeps     |
|                                       |     talking -- she\'s the    |
|                                       |     INTERRUPTEE-1*           |
|                                       |                              |
|                                       | -   *Lou Waters starts       |
|                                       |     talking -- he\'s the     |
|                                       |     INTERRUPTER-2*           |
+---------------------------------------+------------------------------+
| \<e1 3476.976\>                       | -   *Natalie Allen stops     |
|                                       |     talking. \<e1\> refers   |
| \^Waters. Show-biz Today as \^Lauren  |     to end of INTERRUPTEE-1* |
| told you is next. We\'ll see you next |                              |
| time.                                 | -   *Lou Walter\'s speech    |
|                                       |     continues*               |
+---------------------------------------+------------------------------+
| \<t 3480.602\> \<\<female,            | -   *New turn \<t\> with new |
| Natalie_Allen\>\>                     |     speaker ID.*             |
|                                       |                              |
| Bye bye.                              |                              |
+---------------------------------------+------------------------------+

The \<e1\> or \<e2\> label indicates the end of the period of overlap.
\<e1\> is used when INTERRUPTEE-1 stops talking while INTERRUPTER-2
continues; \<e2\> is used when INTERRUPTER-2 stops talking while
INTERRUPTEE-1 continues. If both speakers stop talking at the same time
and a third person begins talking, a new \<t\> turn label or \<sx\>
section label should be used.

Often times a file contains recurring periods of overlapping speech
within a small section of the broadcast. If these sections of overlap
occur in direct succession, annotators group the whole section together
as one large overlap rather than trying to tease apart consecutive
overlapping speech events. In this case, it may not be possible to
easily identify the speakers or transcribe what they are saying. When
this happens, annotators adopt the \"impossible to understand\"
convention of double parentheses (( )) for each turn that cannot be
transcribed.

#### Speakers start simultaneously

When speakers start talking simultaneously, annotators create
consecutive sections with timestamps for the speakers that are no more
than one tenth of a second apart. The \<t\> turn symbol is used for the
first speaker\'s turn, and the \<o\> overlap tag is used for the second
speaker\'s turn. This step is necessary so that speaker IDs are created
for each of the speakers. For instance,

\<t 1123.176\> \<male, Jacques_Cousteau\>

\[\[NS\]\]

\<o 1123.276\> \<female, speaker_2\>

INTERRUPTEE-1: You know, I understand {laugh}

INTERRUPTER-2: Well,

\<e1 1124.256\>

oceanography is new exploration and we\'re not\...

#### Short periods of non-speech

For a short (greater than 0.5 seconds but less than 5 seconds) period of
silence, music or other non-speech, a \<b\> tag is inserted at the start
of the non-speech section. The \[\[NS\]\] no speech marker is used to
indicate that no speech occurs during this breakpoint. A new \<b\> is
then inserted at the next region of speech. For example,

\<b 123.456 \>\
The crowd was furious.\
\<b 124.567\>\
\[\[NS\]\]\
\<b 128.987\>\
Calm was soon restored by the arrival of the riot police.

#### Extended periods of non-speech

For an extended (more than 5 seconds) period of silence, music or other
non-speech, annotators insert an \<e\> tag at the start of the
non-speech section. They then start the new turn \<t\> at the next
region of speech. For example,

\<t 148.57\>

Gunfire filled the air.

\<e 154.50\>

\<t 170.89\>

That sound greeted early morning visitors on Tuesday.

### Breakpoints

#### Overview

Breakpoints are timestamps within a speaker turn. These internal
timestamps are inserted to break up long speaker turns for ease of
transcription. Annotators should insert breakpoints around breath
groups, at ends of sentences or phrases and at noticeable pauses.
Breakpoints are also inserted around lengthy (greater than 0.5 seconds)
non-speech events within a speaker\'s turn. This includes things like
music, sound effects and silence.

Because breakpoints are inserted for ease of transcription, their exact
implementation is subject to the individual annotator\'s discretion. In
general, breakpoints tend to occur every three to eight seconds.

### Summary of segment boundary symbols in broadcast news

The table below summarizes the segment labels used during segmentation.

  ----------- --------------------------------------------------------------
  **Label**   Description

  \<sr\>      start of news story section

  \<st\>      start of teaser section

  \<sn\>      start of non-news section: commercials, etc.

  \<t\>       start of non-initial speaker turn within section

  \<b\>       breakpoints within speaker turn

  \<e\>       end of turn within section, followed by a non-speech region

  \<o\>       start of overlap region (speaker one is interrupted by speaker
              two)

  \<e1\>      end of overlap where interruptee-1 stops and interrupter-2
              continues

  \<e2\>      end of overlap where interrupter-2 stops and interruptee-1
              continues
  ----------- --------------------------------------------------------------

### Speaker Identification

#### Introduction

In addition to identifying segment boundaries and timestamping them,
annotators must also identify all of the speakers within a broadcast.
Speaker name, type, and native/non-native speaker status are all
recorded. If annotators are unable to determine the name of a speaker,
they assign that speaker a unique numerical identification, and use the
same speaker ID throughout the transcript file.

#### Speaker Type

There are four speaker types as follows:

-   Female -- used for adult females

-   Male -- used for adult males

-   Child -- used for children of either sex

-   Other -- used for speakers in unison, altered voices, unknown
    speaker sex, etc.

#### Names and Identifiers

Whenever possible, annotators record the proper name of the speaker.
Examples of proper names include Jacques_Cousteau, William_Cohen, and
Madeleine_Albright. Annotators must use the same spelling of proper
names within a broadcast file, and wherever feasible across broadcast
files as well.

If a speaker is not identified by name within a recording, a unique
numerical index is used. Unnamed speakers are divided into Reporter and
Speaker. Reporter is used for news anchors, interviewers, or reporters
on the scene of a story. Speaker refers to anyone else who is not
identified by name. The numerical IDs for Reporter and Speaker IDs
cannot overlap; each successive anonymous speaker has a unique number,
regardless of the category the speaker is assigned to. For example, the
following sequence is entirely possible:

reporter_1

reporter_2

speaker_3

speaker_4

reporter_2 (the same voice as the previous reporter_2)

reporter_6

#### Native and non-native speakers

In addition to indicating speaker type and name/ID, annotators also
indicate when a speaker is a non-native speaker. In English broadcast
news, native is defined as a speaker of any North American English
dialect. As native is the default, this is not explicitly marked.
Non-native is used for speakers of other dialects of English, including
British English or Indian English; non-native is also used to indicate
people who are not native English speakers and have a discernable
foreign accent. Examples of Speaker IDs include the following:

#### Examples of speaker IDs

\<sr 1.402\> \<\<male, Leon_Harris\>\>

\<sr 158.244\> \<\<female, Joie_Chen\>\>

\<t 196.813\> \<\<male, speaker_1\>\>

\<t 498.314\> \<\<female, non-native, speaker_3\>\>

\<t 567.215\> \<\<other, speaker_4\>\>

## Telephone Speech

Unlike broadcast news, telephone speech is recorded on two separate
channels. In most cases, each channel corresponds to a single speaker
for the duration of the call. The two channels are labeled Speaker A
(the local channel) and Speaker B (the remote channel).

Occasionally, multiple speakers appear on a single channel. If this
happens, the additional speakers are be identified by the channel they
appear on, plus a number to distinguish them from the main speaker, e.g.

Speaker A1: second speaker on channel A

Speaker A2: third speaker on channel A

Speaker B1: second speaker on channel B

and so on.

For Telephone Speech files, annotators apply turn boundaries to the
entire speech file, working with one channel at a time. Section
boundaries do not apply to telephone speech data.

Within telephone speech files, timestamps indicate both the start time
and the end time of a turn or breakpoint boundary. Timestamps do not
necessarily occur in direct succession, one after another. Instead,
there may be intervening periods of silence on one channel while the
other channel\'s speaker is talking. Therefore, the end time of each
timestamp must be explicitly indicated, and cannot be extrapolated from
the start time of the following timestamp. For instance, a series of
timestamps in a telephone speech file might look like this:

25.66 27.15 A:

33.53 35.76 A:

37.00 42.34 A:

Note that there are intervening periods of silence between the
timestamps.

During segmentation, annotators work with one audio channel at a time.
When the two channels have been completely segmented, they are
interleaved for purposes of transcription, e.g.:

24.24 25.53 B:

25.66 27.15 A:

27.25 30.71 B:

31.00 33.30 B:

33.53 35.76 A:

# Transcription

## Introduction

Once a file has been fully segmented and the speakers identified, it
must be transcribed. Annotators must produce a verbatim (word-for-word)
transcript of everything that is said within the file. The words
transcribed within each segment boundary must correspond exactly to the
timestamps that have been created, so that the audio file is aligned
with the transcript.

For Broadcast News files, annotators transcribe only \<sr\> and \<st\>
sections. They do not transcribe any section which has been labeled
\<sn\>, non-news.

For Telephone Speech files, annotators transcribe the file in its
entirety, working with both channels at once.

## Transcription Conventions

### Orthography and spelling

#### Capitalization

Capitalization in the transcripts is used to aid human comprehension of
the text. Annotators should follow accepted standard written
capitalization patterns, and capitalize words at the beginning of a
sentence, proper names, and so on. (Note that proper names are also
labeled with a caret \^ symbol whose use is detailed in Section
3.2.1.1.)

#### Spelling

Transcribers use standard orthography, word segmentation and word
spelling. All files must be spell-checked after transcription is
complete. When in doubt about the spelling of a word or name, annotators
consult a standard reference, like an online or paper dictionary, world
atlas or news website.

#### Contractions

Annotators limit their use of contractions to those that exist in
standard written English, and of course only when a contraction is
actually produced by the speaker. Annotators must take care to
transcribe exactly what the speaker says. The table below, while not
comprehensive, shows some examples of how to transcribe common
contractions.

  ----------------------- ------------------ ---------------- ---------------
  **Complete Form**       Spoken As          Transcribed As   Incorrect

  I have                  ***I\'ve***        **I\'ve**        

  cannot                  ***can\'t***       **can\'t**       

  will not                ***won\'t***       **won\'t**       

  you have                ***you\'ve***      **you\'ve**      

  could not               ***couldn\'t***    **couldn\'t**    

  should have             ***should\'ve***   **should\'ve**   should of,
                                                              shoulda

  would have              ***would\'ve***    **would\'ve**    would of,
                                                              woulda

  it is                   ***it\'s***        **it\'s**        its

  its (possessive)        ***its***          **its**          it\'s

  Marvin (possessive)     ***Marvin\'s***    **Marvin\'s**    

  Marvin is               ***Marvin\'s***    **Marvin\'s**    

  Marvin has              ***Marvin\'s***    **Marvin\'s**    

  going to                ***gonna***        **going to**     gonna

  want to                 ***wanna***        **want to**      wanna

  got to                  *gotta*            got to           gotta
  ----------------------- ------------------ ---------------- ---------------

**Note**: Annotators should take care to avoid the common mistakes of
transposing possessive its for contraction it\'s (it is), possessive
your for the contraction you\'re (you are), and their (possessive),
they\'re (they are) and there.

Annotators should transcribe exactly what they hear using standard
orthography. If a speaker uses a contraction, the word is transcribed as
contracted: they\'re, won\'t, isn\'t, don\'t and so on. If the speaker
uses a complete form, the annotator should transcribe what is heard:
they are, is not and so on.

For non-standard contractions like \"gonna\" and \"wanna\" annotators
should spell out the entire word: going to, want to.

#### Numbers

All numerals are written out as complete words. Hyphenation is used for
numbers between twenty-one and ninety-nine only.

twenty-two

nineteen ninety-five

seven thousand two hundred seventy-five

nineteen oh nine

#### Hyphenated words and compounds

In general, annotators should be conservative about use of hyphens. For
instance:

*an overly complicated analysis* **not** *an* *overly-complicated
analysis*

However, in some cases, a hyphen is required:

*anti-nuclear protests* **not** *anti nuclear protests*

Compounds can be tricky. When in doubt, annotators should consult a
dictionary and talk to their language team leader.

#### Abbreviations

In general abbreviations should be avoided and words should be
transcribed exactly as spoken. The exception is that when abbreviations
are used as part of a personal title, they remain as abbreviations, as
in standard writing:

Mr. Brown

Mrs. Jones

Dr. Spock

However, when they are used in any other context, they are written out
in full:

I went to the *junior* league game.

I went to the *doctor*, and all he said was, don\'t worry, it\'s
natural.

Hey *mister*, do you know how to get to the stadium?

#### Acronyms and spoken letters

Acronyms that are pronounced as a single word should be written in all
capital letters, and preceded by the @ symbol:

\@NASA

\@AIDS

 

Acronyms that are normally written as a single word but pronounced as a
sequence of individual letters should be written in all caps, with each
individual letter preceded by a \~ tilde symbol:

\~F \~B \~I

\~C \~E \~O

Similarly, individual letters that are pronounced as such should be
written in caps, with each letter preceded by a tilde:

I got an \~A on the test.

his name is spelled \~S \~I \~M \~P \~S \~O \~N.

#### Punctuation

Annotators should use standard punctuation for ease of transcription and
reading. Acceptable punctuation is limited to periods and question marks
at the end of a sentence, and commas within a sentence. Transcripts
should not contain quotation marks, exclamation marks, colons,
semicolons, dashes or ellipses in transcribing. Punctuation is written
as it normally appears in standard writing, with no additional spaces
around the punctuation marks.

### Disfluent speech 

#### Introduction

Regions of disfluent speech are particularly difficult to transcribe.
Speakers may stumble over their words, repeat themselves, utter partial
words, restart phrases or sentences, and use lots of hesitation sounds.
Annotators should take particular care in sections of disfluent speech
to transcribe exactly what is spoken, including all of the partial
words, repetitions and filled pauses used by the speaker.

#### Filled pauses and hesitation sounds

Filled pauses are non-lexemes (non-words) that speakers employ to
indicate hesitation or to maintain control of a conversation while
thinking of what to say next. Each language has a limited set of filled
pauses that speakers can employ. Annotators use the standardized
spellings shown in the table below for filled pauses. The spelling of
filled pauses is not altered to reflect how the speaker pronounces the
word (e.g., typing AH for a loud \"ah\" or ummmm for a long \"um\".) For
English, this set includes ah, eh, er, uh, um.

All filled pauses are indicated with a % sign preceding the word.

  ----------------------- ----------------------- -----------------------
  **English Filled        Arabic Filled Pauses    Chinese Filled Pauses
  Pauses**                                        

  %ah                     **%ah**                 %呵

  %eh                     **%E**                  %呃

  %er                     **%M**                  %唔

  %uh                     **%uh**                 

  %um                     \%**hm**                

                          %hum                    
  ----------------------- ----------------------- -----------------------

#### Partial words

When a speaker breaks off in the middle of the word, annotators
transcribe as much of the word as can be made out. A single dash - is
used to indicate point at which word was broken off.

Yes, absolu- absolutely.

#### Restarts

Speaker restarts are indicated with double dash \--. Annotators use this
convention for cases where a speaker stops short, cutting him/herself
off before continuing with the utterance.

\<b 850.021\>

I thought he \-- I thought he was there.

\<b 94.271\>

The thi- \-- the thing we\'re worried about is

#### Mispronounced words

A plus symbol + is used for obviously mispronounced words (not regional
or non-standard dialect pronunciation). Annotators should transcribe
using the standard spelling and should not try to represent the
pronunciation.

\<b 1120.021\>

He\'ll +probably I mean probably go with me tomorrow.

### Noise

#### Speaker noise 

Speaker-produced noise is identified with one of the following five
tags:

{laugh}

{cough}

{sneeze}

{breath}

{lipsmack}

####  {#section-2 .unnumbered}

#### Background noise 

When there is noticeable background noise (not speaker noise) present
during a span of speech, annotators employ the \<noise\> notation.

When the sound is instantaneous, like a short clap, paper rustle, door
slamming shut or gunshot, the \<noise\> symbol is inserted next to the
word during which the noise occurs. For instance:

\<b 1116.183\>

I\'m not really sure \<noise\> what she said.

\<b 1120.021\>

Hey, did you hear that? It sounded like a car backfiring.

If the sound is prolonged and spans several words in the transcript, the
\<noise\> symbol is inserted before to the word where the sound begins,
and \</noise\> is inserted after the word where the sound ends. For
instance,

\<b 1116.183\>

\<noise\> I can\'t tell what\'s going on out there.

\<b 1120.021\>

It\'s really getting loud. \</noise\>

If the sound is very long, it might cross breakpoints or speaker turns.

Note: If the file contains persistent or overwhelming distortion, static
or background noise, annotators should notify their language team
leader.

### Additional markup

#### Hard-to-understand sections

Sometimes an audio file will contain a section of speech that is
difficult or impossible to understand. In these cases, annotators use
double parentheses (( )) to mark the region of difficulty.

Sometimes it is possible to take a guess about the speaker\'s words. In
these cases, annotators transcribe what they think they hear and
surround the stretch of uncertain transcription with double parentheses:

\<b 1116.183\>

And she told me that ((I should just leave.))

If an annotator is truly mystified and can\'t at all make out what the
speaker is saying, s/he uses empty double parentheses to surround the
untranscribed region. Where possible, this untranscribed region gets its
own timestamp, e.g.:

\<b 1116.183\>

(( ))

#### Idiosyncratic words

Occasionally a speaker will make up a new word on the spot. These are
not the same as slang words; they\'re words that are unique to the
speaker in that conversation. If annotators encounter an idiosyncratic
word, they should transcribe it to the best of their ability and mark it
with an asterisk \*. For instance,

Do you dress like a \*schlump yet?

Why she said \*drr I don\'t know

#### Foreign languages

Portion of speech in another language are annotated using the \<language
text\> convention to indicate the language and to transcribe the words
that are spoken in that language. For instance:

And then I took all of the \<German Sachen\> to my room.

Oh, \<Spanish gracias\> he said.

If the annotator does not know the name of the language or what is being
said, they should use the tag \<foreign\> in isolation.

Then there were a couple of \<foreign\> which I tried on.

#### Proper nouns

All proper nouns, including personal names, place names and the like,
are marked with a caret \^. Common nouns that are functioning as names
or titles are not marked. If the name contains more than one word, all
words in the name are annotated with a caret.

\^Osama \^bin \^Laden

\^Sony

\^Maria\'s Bar and Grill

He calls himself \~J \~R \^Jones

Secretary of State \^Madeline \^Albright

When annotators encounter a name whose spelling they\'re not sure of,
they should spend a moment searching for the proper spelling. If they
can\'t find the correct spelling after a moment or two, they should make
their best guess and use a double caret \^\^ instead of a single caret
to mark the name, e.g.

\^\^Rafjanii \^Agrawal

These names will be reviewed again during second passing and the
spelling verified.

#### Interjections

The following standardized spellings are used to transcribe
interjections. Interjections do not require any special symbol.

**English Interjections**

ach

duh

eee

ew

ha

hee

huh

huh-uh

hm

jeepers

jeez

mm

mhm

nah

oh

okay

oof

ooh

uh-huh

uh-oh

whoa

whew

whoops

woo-hoo

yay

yeah

yep

yup

**Arabic Interjections**

ah

Ah

aha

ayyO

Eyy

ha

Hay

hE

hi

ih

mhm

O

OhO

uh

wAw

yA

yaa

yO

yOO

yuu

**Chinese Interjections**

啊 a1      \'surprise, praise\'

啊 a2      \'questioning\'

啊 a3      \'disbelief\'

啊 a4      \'answer; surprise; praise\'

哎/嗳 ai1     \'dissastisfaction\'

哎呀 ai1ya1  \'surprise; complaining\'

哎哟 ai1yo1  \'surprise; agony\'

哎 ai2     \'emphasis\'(2)

嗳/哎 ai3     \'disagreement; denying\'

嗳/哎 ai4     \'regret\'

唉 ai4     \'disappointment\'

哈 ha1     \'triumphant\'

哈哈 ha1ha1  \'triumphant\'

咳 hai1    \'sadness; regret\'

嘿/嗨 hei1    \'drawing attention\'

呵/嗬 he1     \'surprise\'

哼 hng5    \'dissatisfaction

嗯/唔 n2/ng2  \'querry\'

嗯 n3/ng3  \'out of expectation\'

嗯 n4/ng4  \'answer\' (3)

喔/噢 o1      \'understanding\'

喔/噢唷 o1yo1   \'surprise\'

哦 o2      \'half belief half doubt\'

哦/喔 o4      \'understanding\'

呸 pei1    \'discarding; scolding\'

哇 wa1/wa3 \'surprise\'

哟 yo1     \'slight surprise\'

### **Summary of special symbols**

+--------+-------------+----------+--------------+--------------------+
| **Cate | **          | **       | **Example**  | **Explanation**    |
| gory** | Condition** | Markup** |              |                    |
+--------+-------------+----------+--------------+--------------------+
| *      | **Numbers** | Spelled  | twenty-five, | Write out in full; |
| *Ortho |             | out      | one oh nine, | dashes for         |
| graphy |             |          | one hundred  | twenty-one through |
| and    |             |          | thirty-seven | ninety-nine        |
| spel   |             |          |              |                    |
| ling** |             |          |              |                    |
+--------+-------------+----------+--------------+--------------------+
|        | **Standard  | Tr       | can\'t, I\'m | If you hear a      |
|        | con         | anscribe |              | contraction used,  |
|        | tractions** | as       |              | write it as a      |
|        |             | spoken.  |              | contracted form.   |
+--------+-------------+----------+--------------+--------------------+
|        | **N         | Not used | going to,    | Do not use         |
|        | on-standard |          | want to      | non-standard       |
|        | con         |          |              | contractions.      |
|        | tractions** |          |              | Write the words    |
|        |             |          |              | out in full.       |
+--------+-------------+----------+--------------+--------------------+
|        | **Pu        | Comma,   | , ? .        | Limited to these   |
|        | nctuation** | question |              | three symbols.     |
|        |             | mark,    |              |                    |
|        |             | period   |              |                    |
+--------+-------------+----------+--------------+--------------------+
|        | *           | @        | \@NAFTA      | Write letters with |
|        | *Pronounced |          |              | all caps, no space |
|        | acronyms**  |          |              | between letters.   |
+--------+-------------+----------+--------------+--------------------+
|        | *           | \~       | \~I before   | Individual letters |
|        | *Individual |          | \~E          | spelled out,       |
|        | letters**   |          |              | capitalized, each  |
|        |             |          | \~Y \~M \~C  | with \~            |
|        |             |          | \~A          |                    |
+--------+-------------+----------+--------------+--------------------+
| **Dis  | **Filled    | \%       | %ah, %uh     | Limited to small   |
| fluent | pauses**    |          |              | list for each      |
| sp     |             |          |              | language; use      |
| eech** |             |          |              | standardized       |
|        |             |          |              | spellings          |
+--------+-------------+----------+--------------+--------------------+
|        | **Partial   | \-       | absolu-      | Speaker-produced   |
|        | words**     |          |              | partial words are  |
|        |             |          |              | indicated with a   |
|        |             |          |              | dash. Transcribe   |
|        |             |          |              | as much of the     |
|        |             |          |              | word as you hear.  |
+--------+-------------+----------+--------------+--------------------+
|        | **Speaker   | \--      | I thought he | Used when the      |
|        | restart**   |          | \-- I        | speaker stops      |
|        |             |          | thought he   | short and then     |
|        |             |          | was there.   | repeats themselves |
|        |             |          |              | or abandons the    |
|        |             |          |              | utterance          |
|        |             |          |              | completely,        |
|        |             |          |              | restarting with a  |
|        |             |          |              | new sentence.      |
+--------+-------------+----------+--------------+--------------------+
|        | **Mi        | \+       | +probably    | Mispronounced word |
|        | spronounced |          |              | (a speech error).  |
|        | words**     |          |              | **NOTE**: Do not   |
|        |             |          |              | use this symbol to |
|        |             |          |              | indicate           |
|        |             |          |              | non-standard but   |
|        |             |          |              | common             |
|        |             |          |              | regional/social    |
|        |             |          |              | dialect            |
|        |             |          |              | pronunciations.    |
|        |             |          |              |                    |
|        |             |          |              | Transcribe         |
|        |             |          |              | non-standard       |
|        |             |          |              | pronunciation      |
|        |             |          |              | variants or        |
|        |             |          |              | mispronounced      |
|        |             |          |              | words using        |
|        |             |          |              | standard           |
|        |             |          |              | orthography.       |
+--------+-------------+----------+--------------+--------------------+
| *      | **Speaker   | { }      | {breath}     | Sounds made by the |
| *Noise | noise**     |          |              | talker. Limited to |
| condit |             |          | {cough}      | these five.        |
| ions** |             |          |              |                    |
|        |             |          | {laugh}      |                    |
|        |             |          |              |                    |
|        |             |          | {sneeze}     |                    |
|        |             |          |              |                    |
|        |             |          | {lipsmack}   |                    |
+--------+-------------+----------+--------------+--------------------+
|        | **          | \        | \<noise\>    | Use \<noise\> for  |
|        | Non-speaker | <noise\> | What\'s that | instantaneous      |
|        | noise**     | \<       | sound?       | sounds.            |
|        |             | /noise\> | \</noise\>   |                    |
|        |             |          |              | Use \<noise\> text |
|        |             |          |              | \</noise\> for     |
|        |             |          |              | ongoing sounds.    |
|        |             |          |              | This convention    |
|        |             |          |              | should be used for |
|        |             |          |              | any background     |
|        |             |          |              | noise, static,     |
|        |             |          |              | distortion or      |
|        |             |          |              | other non-speaker  |
|        |             |          |              | noise.             |
+--------+-------------+----------+--------------+--------------------+
| *      | **Semi-i    | ((text)) | They lived   | This is the        |
| *Other | ntelligible |          | ((next door  | transcriber\'s     |
| ma     | speech**    |          | to us)).     | best attempt at    |
| rkup** |             |          |              | transcribing a     |
|        |             |          |              | difficult passage. |
+--------+-------------+----------+--------------+--------------------+
|        | **Uni       | (( ))    | (( ))        | This indicates an  |
|        | ntelligible |          |              | entirely           |
|        | speech**    |          |              | unintelligible     |
|        |             |          |              | passage.           |
+--------+-------------+----------+--------------+--------------------+
|        | **Id        | \*       | \*poodleish  | Speaker uses a     |
|        | iosyncratic |          |              | \"made-up\" word.  |
|        | words**     |          |              | **NOTE:** Do not   |
|        |             |          |              | use for            |
|        |             |          |              | non-standard       |
|        |             |          |              | dialect terms or   |
|        |             |          |              | misused words.     |
+--------+-------------+----------+--------------+--------------------+
|        | **Foreign   | \<       | \<French     | This is used to    |
|        | language**  | language | merci\>      | indicate foreign   |
|        |             | text\>   |              | speech. If the     |
|        |             |          | \<foreign\>  | word is unknown,   |
|        |             |          |              | leave it out. If   |
|        |             |          |              | the language is    |
|        |             |          |              | unknown, merely    |
|        |             |          |              | write \<foreign\>. |
|        |             |          |              |                    |
|        |             |          |              | **NOTE**: Do not   |
|        |             |          |              | use this           |
|        |             |          |              | convention for     |
|        |             |          |              | foreign borrowings |
|        |             |          |              | that are common in |
|        |             |          |              | the target         |
|        |             |          |              | language, e.g.     |
|        |             |          |              | *apropos*.         |
+--------+-------------+----------+--------------+--------------------+
|        | **Proper    | \^       | \^Osama      | Use caret symbol   |
|        | names**     |          | \^bin        | for each word of   |
|        |             |          | \^Laden      | proper name. Do    |
|        |             |          |              | not use the caret  |
|        |             |          | \^Mariani\'s | for common nouns   |
|        |             |          | Bar and      | that are part of a |
|        |             |          | Grill        | title or name.     |
|        |             |          |              |                    |
|        |             |          | Secretary of |                    |
|        |             |          | State        |                    |
|        |             |          | \^Albright   |                    |
+--------+-------------+----------+--------------+--------------------+
|        | **Inte      | no       | uh-huh,      | Use standardized   |
|        | rjections** | special  | yeah, mhm    | spellings          |
|        |             | markup   |              |                    |
+--------+-------------+----------+--------------+--------------------+

## Some general considerations

Annotators should not try to correct grammatical errors, e.g. \"I seen
him\" for \"I saw him\" should be transcribed as spoken. The same goes
for misused words: annotators should transcribe what is spoken, not what
they expect to hear.

Annotators should not try to imitate a speaker\'s non-standard
pronunciation. Standard spelling should be adopted for non-standard
pronunciations. Obviously mispronounced words (as opposed to
non-standard pronunciations) should be marked with the plus + symbol.

# Second Passing

## Introduction

Second passing is used as a quality control measure to ensure the
accuracy of segmentation, transcription including markup, and speaker
identification. After the initial file has been fully segmented and
transcribed, a new annotator listens to the entire broadcast or
telephone call while viewing the corresponding transcript, and makes
adjustments to the timestamps or transcription as needed. Second passing
entails a mix of manual and programmatic checks on the transcript files.
The particular types of checks conducted during second passing are
described below.

##  Segmentation and Speaker ID Verification

Second pass annotators verify that each timestamp matches the
corresponding transcript exactly. Annotators play each timestamp in turn
and make sure that the audio and transcript for that segment are an
exact match and make any necessary corrections. Annotators also check
that the timestamp has been placed in a suitable location, e.g. between
phrases or sentences, and that the timestamp does not chop off the start
or end of any word.

For telephone speech, annotators listen to the entire call to ensure
that all speech for each channel is captured within a turn segment and
that no speech remains outside of a segment boundary.

For broadcast news speech, annotators listen to the entire broadcast,
including \<sn\> (commercial) sections, to ensure that no news or teaser
section has gone unsegmented or untranscribed. Annotators also verify
that the correct section labels (\<sr\>, \<st\>, \<sn\>) have been used,
that the overlapping speech convention has been correctly applied, and
that each new turn or segment boundary has a corresponding speaker ID.

Each speaker ID is further checked to ensure that the same speaker name
is spelled consistently within each file, and that the same speaker type
and native/non-native speaker status has been recorded for each instance
of that speaker.

## Transcription verification

During the transcript checking phase of second passing, annotators
examine the transcript in detail, checking for accuracy, completeness
and the consistent use of transcription conventions. Annotators pay
particular attention to a handful of areas that are particularly
difficult to transcribe, in particular unintelligible speech sections
and areas of speaker disfluency. Any proper names whose spelling could
not be verified during the initial transcription process are corrected
and standardized within the file. Finally, annotators conduct a spell
check on the file.

## Automatic checks

In addition to the manual checks described above, annotators also employ
a series of programmatic checks of the data known as a syntax check. The
syntax check scans the file for common segmentation and transcription
errors as well as errors of data formatting. The checks include:

> **Timestamping**

-   timestamps are out of linear order

-   timestamps without text data

-   timestamps followed by non-empty lines

-   multiple section, turn or breakpoint boundaries on a single line

**Speaker IDs**

-   missing or badly formatted speaker IDs

-   missing or badly formatted speaker type

**Transcription**

-   foreign language convention badly formatted

-   unintelligible speech convention badly formatted

-   illegal character used in transcript

-   bad spacing around punctuation

-   digits are not spelled out

Annotators run the syntax checker as a final pass over the data before
completing a file. The syntax checker outputs an error report. The
annotator reviews each error in succession, and can \"zoom\" to the
problematic section of the transcript to make any necessary corrections.