Compilation of Large-scale Paraphrase Lexicon andIts Application to Phrase-based SMT Systems
Atsushi Fujita (fujita@paraphrasing.org)
National Institute of Information and Communications Technology,, Japan

Summary

Aggregation of Multiple Paths! Standard	  phrase-­‐based	  SMT	  systems	  +	  paraphrases	  l Phrase	  table	  augmenta1on	  (transla1on	  pair	  fabrica1on)	  ! Source-­‐side	  augmenta1on	  0.735	

! Beber	  performance	  over	  a	  vanilla	  phrase-­‐based	  SMT	  Transla1on	  table	
health	  issue	  |||	  problème	  de	  santé	  health	  problem	  |||	  problème	  de	  santé	  look	  like	  |||	  ressemble	  regional	  issue	  |||	  problème	  régional	  regional	  problem	  |||	  problème	  régional	  resemble	  |||	  ressemble	  [Bannard+,	  05]	SSeed:	  Seed	  paraphrases	
health	  issue	  ⇔	  health	  problem	  regional	  issue	  ⇔	  regional	  problem	  look	  like	  ⇔	  resemble	  Monolingual	  Non-­‐parallel	  Corpus

zooming	  opera1on	  0.317/0.195	

zooming	  opera1on	  Exercise in Ja/En patent translation! NTCIR-­‐10	  PatentMT	  [Goto+,	  13]	  En	Ja	#	  of	  snts.	
3.2M	 3.2M	#	  of	  words	 106M	 116M	

Monolingual	  Non-­‐Parallel	  Corpus	
En	Ja	
#	  of	  snts.	 413M	 594M	
#	  of	  words	 13.4B	 27.3B	#	  of	  transla1on	  pairs	
IBM2	
HMM	
IBM4	
extrac1on	 Union	

Ja	  à	  En	 En	  à Ja	
9.1M	
9.4M	
230.6M	 234.4M	
80.6M	 81.8M	
260.4M	 264.8M	

#	  of	  paraphrase	  pairs	thp	 ths	 En	Ja	SSeed	ε	 ε	 7.2M	 5.1M	
ﬁltering	
S
0.01	 0.1	 1.1M	 0.8M	
expansion	 Seed	
SHvst	 0.01	 ε	 272M	 143M	
ﬁltering	
SHvst	 0.01	 0.1	 ?????	 ?????	
Computed only for tune+dev+testPublication
Atsushi	  Fujita,	  Pierre	  Isabelle,	  and	  Roland	  Kuhn.	  
Enlarging	  Paraphrase	  Collec1ons	  through	  Generaliza1on	  and	  Instan1a1on.	  	  In	  Proc.	  of	  EMNLP-­‐CoNLL,	  2012.	  
! Atsushi	  Fujita	  and	  Marine	  Carpuat.	  
FUN-­‐NRC:	  Paraphrase-­‐augmented	  Phrase-­‐based	  SMT	  Systems	  for	  NTCIR-­‐10	  PatentMT.	  	  In	  Proc.	  of	  NTCIR-­‐10,	  2013.	  
!

ズーム動作

zooming	  0.021/0.003	

ズーミング動作0.650	

ズーム操作

0.463	

変倍動作

ズーム動作
0.606	

Features for Decoding! Scores	  of	  individual	  transla1on	  pair	  l Kneser-­‐Ney	  es1mates	  (original	  pairs)	  [Chen+,	  11]	  l Transla1on	  score	  (fabricated	  pairs)	  lle.g.,	  Backward	  score	  of	  source-­‐side	  augmenta1on	  ⌘P ⇣p(s|t)Para(s ) s0 )s2SPp(s0 |t) =0
s2S Para(s ) s )Zens-­‐Ney	  lexical	  es1mates	  (all	  pairs)	  [Zens+,	  04]	  ! Transla1on	  pair	  indicators	  l Alignment	  indicators	  (IBM2,	  HMM,	  IBM4)	  l Paraphrase	  collec1on	  indicators	  (SSeed,	  SHvst/SOOPH)	  lSOOPH:	  Pairs	  of	  	  	  	  Unseen	  phrase	  ⇔	  Seen	  phrase	

0.146/0.108	

0.046/0.008	

X	  issue	  ⇔	  X	  problem	  	  X	  ∋	  {“health”,	  “regional”}	  backlog	  issue	  ⇔	  backlog	  problem	  communal	  issue	  ⇔	  communal	  problem	  phishing	  issue	  ⇔	  phishing	  problem	  spa1al	  issue	  ⇔	  spa1al	  problem	  0.228/0.136	
0.002/0.005	

! Target-­‐side	  augmenta1on	  Paraphrase	  paberns	

SHvst:	  Harvested	  paraphrases	

Bilingual	  Parallel	  Corpus	

zoom	  ac1on	  0.352	

Paraphrase Collections

Bilingual	  Parallel	  Corpus	

zoom	  opera1on	  0.709	

OOPH	  indicators	  (phrase-­‐table	  level,	  corpus-­‐level)	  ! Paraphrase	  score:	  an	  avg.	  score	  of	  each	  pair	  l PivProb	  [Bannard+,	  05]	  vs.	  Cosine	  similarity	  [Marton+,	  09]	  Model	  selec1on	  using	  held-­‐out	  data	  (ntc7	  &	  ntc8)	

(*)	  minimal	  set	  covering	  tune+dev+test	  datasets	
Ja	  à	  En	En	  à Ja	
Para	  System	#	  of	  trans.	  
score	 #	  of	  trans.	  
BLEU	
BLEU	
pairs	  (*)	pairs	  (*)	vanilla	  PBSMT	-­‐	
15.5M	 37.64	
18.0M	 33.30	
Saug-­‐SSeed	
PivProb	 27.3M	 33.65	 +0.35	 24.6M	 37.98	 +0.34	
Saug-­‐SSeed	
Cosine	 27.3M	 33.27	 -­‐0.03	 24.6M	 37.73	 +0.09	
Saug-­‐SHvst	
Cosine	 23.6M	 33.22	 -­‐0.08	 22.0M	 37.89	 +0.25	
Saug-­‐SOOPH	
Cosine	 18.1M	 33.72	 +0.42	 15.6M	 38.16	 +0.52	
Saug-­‐SSeed+SHvst	 Cosine	 32.8M	 32.91	 -­‐0.39	 30.9M	 37.76	 +0.12	
Taug-­‐SSeed	
PivProb	 22.9M	 33.34	 +0.04	 19.6M	 37.64	 +0.00	
Taug-­‐SSeed	
Cosine	 22.9M	 33.56	 +0.26	 19.6M	 38.19	 +0.55	
Taug-­‐SHvst	
Cosine	 29.1M	 33.43	 +0.13	 26.8M	 37.98	 +0.34	
Taug-­‐SOOPH	
Cosine	 23.4M	 33.21	 -­‐0.09	 21.5M	 38.08	 +0.44	
Taug-­‐SSeed+SHvst	 Cosine	 33.9M	 32.99	 -­‐0.31	 30.8M	 37.53	 -­‐0.11	

Open	  test	  (ntc10)	
Ja	  à	  En	En	  à Ja	
BLEU	 NIST	 RIBES	 BLEU	 NIST	 RIBES	
vanilla	  PBSMT	 31.25	 8.1945	 0.6892	 33.88	 8.2005	 0.7021	
Saug-­‐SOOPH	
31.56	 8.2507	 0.6955	 34.22	 8.2345	 0.7096	
Taug-­‐SSeed	
31.65	 8.2198	 0.6929	 34.05	 8.2116	 0.7089	
System	Conclusion:	  paraphrase-­‐base	  augmenta1on	  works!	
The	  improvement	  is	  small,	  though	  l Consistent	  improvement	  on	  various	  distor1on	  limit	  values	  l F/W:	  integra1on	  of	  further	  gigan1c	  paraphrase	  resources	  l