Mining the Spoken Wikipedia for Speech Dataand Beyond
Universität Hamburg • Department of Informatics
Natural Language Systems group

Arne Köhn • Florian Stegen • Timo Baumann

The Spoken Wikipedia project unites volunteer readers of Wikipedia articles.
Hundreds of spoken articles in multiple languages are available to users who are – forone reason or another – unable or unwilling to consume the written version of the article.
We turn this speech resource into a time-aligned corpus, making it accessible
for research and to foster new ways of interacting with the material.

Automatic Alignment Pipeline

Descriptive Metrics

Spoken Wikipedia

Wikipedia Downloader
scrapes the spoken article category, checks for not yet downloaded
articles and based on the template found in each such article:• downloads audio, WikiText and HTML of the version read• stores the article's meta-data found in the template and other sources
The Downloader can be configured to match templates and customs
in different languages (so far: German, English, Dutch)

Text Extraction and Normalization
WikiText is extremely hard to parse. Therefore we use MediaWiki's
HTML output and strip tables, boxes and other stuf[[citation needed]] that is
not (or unpredictably) read out in the spoken version.

The Spoken Wikipedia is active in at least 28 language communities, withEnglish, Dutch, German and French being the largest collections.

Overall statistics on the Spoken Wikipedia

Articles are spoken by large and diverse sets of people, cover a variety oftopics, focus on high-quality articles, and use a permissive license (CC-by-SA).

3145

Category and template for spoken articlesaligned hours
overall / full sentences

293 287

All spoken articles use a template with slots: filename, speaker, date andrevision spoken, ... to insert the audio player and a display of the meta-data on
the Wikipedia page. The template also adds spoken articles to a root category.

222

1240

184

157

864

71

Templates, categories, and meta-data vary between language communities!

Our tool uses MaryTTS [1] for sentence segmentation and tokenization,and for pronunciation normalization (adding some rules, in particularfor formulae and years). We also add the textual header that is read outbefore the actual content of the article.

29927413

DE EN NL

DE EN NL

DE EN

DE EN

#articles

total hoursaligned hours#speakers

Error to hand-corrected alignment (DE)100

Wikipedia
category forspoken articles

We use a variation of SailAlign [2] as implemented in Sphinx-4 [3] withsome modifications. SailAlign treats alignment as repeated andsuccessively more restricted speech recognition using N-gram models
derived from the text that is expected to be spoken. Correct stretches
are used as landmarks to restrict re-recognition to tighter models.

Meta-data:• revisionID spoken• audio file(s)• speaker, date.

9080
Audio linked
from Wikimedia
Commons

7060%

Audio Alignment

Dutch articles
much shorteron average

proportion at error
sum ≤ error

5040
30
2010... Nordwind und Sonne, wer von ihnen ...

identify landmarks

0.1

0.15

0.2

>0.2

absolute error in seconds

notaligned

iterate... Nordwind und Sonne, wer von ihnen ...
build tighter N-gram models

0.05

initially, recognize full audio

re-recognize between landmarks

SailAlign is very robust to text/audio mismatches which is a strong
requirement given the modest data quality:• text-speech mismatches (including text not read, audio not in text)• mis-matching revision IDs (even leading to wrong articles)• mis-normalization („*“ → „geboren“ or „Sternchen“?)
Favor quality over quantity in the alignments: rather leave out timings
for dubious cases (e.g. mis-normalized tokens) than providing a fullalignment that is partially faulty.

Bootstrapping Acoustic Models
Alignment requires acoustic and pronunciation models.
→ Sphinx 5.2 PTM acoustic models with good results for English.
For German, we used limited-quality acoustic models [4] anditeratively built new models trained on the aligned data found so far.
→ align more and more data as model quality improves.
We cross-checked the quality of the resulting models on the Kiel Corpusof Read Speech. We found that alignment quality continues to increase.
We expect this efect to carry over to ASR.

Download our Software!
Our modular software consists of C# and Java code bound together
with Python and Shell and works on (at least) Linux and Windows.
See QR-Code or paper for the link.

References[1]: Schröder, M. and Trouvain, J. (2003): „The German Text-to-speech synthesis system
MARY,“ International Journal of Speech Technology, 6(3):365-377.
[2]: Katsamanis et al. (2011): „Sailalign: Robust long speech-text alignment,“ Procs. ofthe Workshop on New Tools and Methods for Very-Large Scale Phonetics Research.
[3]: Walker et al. (2004): „Sphinx-4: A flexible open source framework for speech
recognition,“ Technical Report, Sun Microsystems.
[4]: Baumann et al. (2010): „InproTK in Action: Open-Source Software for Building
German-Speaking Incremental Spoken Dialogue Systems,“ Proceedings of ESSV.

[…]<word normalized="He">He </word>
Alignment<word normalized="strongly" start="32990" stop="33480">strongly</word>
Alignment<word normalized="disputed" start="33480" stop="34140">disputed </word><word normalized="the">the </word><word normalized="claim" start="34140" stop="34630">claim </word><word normalized="that" start="34630" stop="34830">that </word>[…]

Utilization of the Corpus:• speech recognition training

Recorded and aligned minutes per speaker (DE)10000

lots of data fromdiferent speakers

1000

minutesN-gram model for full text

0speakers with
>500min of data
(e.g. for TTS)

10010

• improve accessibility / spoken article navigation1• top contributors speak >30 hours → train synthesis voices• analyze prosody of reading (large amounts of diversely read text)• analyze prosody of information structure (accessible through links,research on semantic Wikipedia, Dbpedia, ...)

Download the Corpus!
For each article, the corpus contains:
audio file(s)● original WikiText● HTML generated by MediaWiki● cleaned and normalized text
● alignment between text and audio● meta-information (who, when, what)●Languages: German and English
We are actively working on more data and more languages.
→ find it at:

http://nats-www.informatik.uni-hamburg.de/SWC/
Our work, our corpus and this poster is CC-by-SA. Thanks to the Wikipedia
contributors, as well as to Daimler-Benz-Foundation for funding.

0.1

recorded minutes
aligned minutes
050

100

150

200

250

300

speaker nr.

Low alignmend rates alreadymostly fixed (wait for next release)

Contributors speak related/interlinked articles: