A WFST-based Log-linear Framework for Speaking-style Transformation
Graham Neubig, Shinsuke Mori, Tatsuya Kawahara
Graduate School of Informatics, Kyoto University, JapanOverview

Speaking Style Transformation
X

Objective: Transform spoken-style language(V) into written style language (W) for thecreation of transcripts●Approach: Statistical machine translation to“translate” from verbatim text to written text
●Innovations:● Log-linear modeling for improved accuracy● Introduction of features to handle common
phenomena in speaking-style transformation● WFST-based implementation for integration
with WFST-based speech recognizers●Evaluation on transformation of Japaneseverbatim transcripts showed improvement overtraditional methods

VW

that is like uh
not a problem

that is not aproblem.

TrainingTrainingData●ASR●SSTLM

Noisy Channel Modeling: =argmax P W∣V W

いろいろ なiroironasubこと
kotoこと
koto

fill

で
deで
de注文
chu-mon注文
chu-monmake

ifTranslation Model●

Language Model

N-bestW…
non-fill

Filler Deletions: Words that are consistently
used as fillers: “e-to” “ano-”●Other Deletions: Words that are fillers or not
depending on context, repeats, repairs, etc.
●Substitutions: Colloquial expressions, etc.
●Insertions: Dropped words, particularly
particles in Japanese: “o” “wa” “ga”
WER before transformation: 16.40%
Context Necessary =argmax 1 log  P V∣W  2 log P W  3 f 3 V ,W W
W

Filler Dictionary: f(V,W) is equal to the number offillers (from a 23-word list) present in W.
●Transformation Groups: f(V,W) is equal to thenumber of groups of words transformed.
●sub ins
8.8% 8.2%

Committee meetings of the Japanese National Diet
Verbatim transcripts as input, official transcripts as output●3.62M sentences for LM training●56.2k aligned sentences for TM training (974 held-out)●7181 testing sentences from meetings after the training datathat is like um uh maybe um not a problem

Transformation Types: Insertions, deletions,substitutions are given separate penalties, allowing
adjustment of the precision/recall of each type.
●Decomposed Translation Model: Use separate loglinear weights for each frequency used whencalculating the translation model.
●k7
6.8
6.6
6.4
6.2
6
5.8
5.6
5.4
5.2
5
Noisylog P V∣W =log ∏ i=1 P v i , wi / P w i k=1 log ∏i=1 P  v i , w i − 2 log ∏i =1 P  w i 

FST: OpenFst (http://www.openfst.org)
MERT: Moses (http://www.statmt.org/moses)

Evaluation

Extra features can be added to the log-linear model:

knon-fillers
35.7%

MERT

Minimum Error Rate
Training on WER

LM:
Kylm (http://www.phontron.com/kylm)
Decoder: Kyfd (http://www.phontron.com/kyfd)

SST-Specific Features●fillers
47.3%

DecodeBeam-search decoding
λ over FST models=argmax 1 log  P V∣W  2 log P W つける と です ね …
tsukeru to desu neinsHeld-OutDataLog Linear Modeling: =argmax log P V ∣W log  P W W

it isを つける とo tsukeru toCompose(λ)

Each FST weighted
separately duringcomposition.

(%)

あーa--obj

Features

WWER (%)

いろんなironnaorder

3-gram Kneser-Ney LM=argmax P V∣W  P W Necessary Transformations
ahh things byTMModelWW

variousML
Estimation

All-Type -Decomp.
-Filler -GroupWER for noisy channel, all features,and each feature removed individually

10095908580757065605550

NoisyChannel
ProposedPrec. Rec. Prec. Rec. Prec. Rec. Prec. RecFiller Delete Subst. Insert

Precision and recall for each type