Expanding Paraphrase Lexicons by Exploiting Lexical Variants
Atsushi Fujita (NICT, Japan; fujita@paraphrasing.org) and Pierre Isabelle (NRC, Canada)

Large	  coverage:	  100x	  expansion	  is	  possible,	  although	  the	  rate	  depends	  on	  the	  size	  of	  corpora	  l High	  accuracy:	  comparable	  to	  state-­‐of-­‐the-­‐art,	  without	  harnessing	  rich	  resources	  ! English	  experiments	  using	  the	  approach	  (B)	  for	  obtaining	  SSeed	  l (a)	  Europarl	  (En-­‐Fr,	  55.7	  MW)	  +	  News	  Crawl	  2013	  (1.20	  BW)	  l (b)	  NTCIR	  Patent:	  Parallel	  (En-­‐ja,	  107	  MW)	  +	  NTCIR	  En	  Monolingual	  2006-­‐2007	  (1.36	  BW)	  lMotivation! How	  to	  create	  a	  paraphrase	  lexicon	  with	  both	  high	  coverage	  and	  high	  accuracy?	  

108

10

7

10

6

Need	  expansion!

airports	  in	  Europe	  ⇔	  European	  airports	  amendment	  of	  regula9on	  ⇔	  amending	  regula9on	  should	  be	  noted	  that	  ⇔	  is	  worth	  no9ng	  that	  Monolingual
Corpus

41x

6!

7108

10

7100x

106

1.37M

10137.6M
53.0M

205x

56SLV
SID
SSeed
7200
160

Unit-­‐wise,	  2-­‐phased,	  classiﬁca9on-­‐based	  protocol	  

Aﬃx1
Aﬃx2
Stem
X:ed
X:saimX:imedX:chieve a
X:chieving X:imsa
X:ing
X:eachiev

Aﬃx1
Aﬃx2
X:chieve X:imed
X:chieving X:ims
X:ed
X:s
X:ing
X:e

1-­‐2.	  Genera9ng	  paraphrase	  pa;erns	  STEM(5)
0069
330

Retain?
NoNo
Yes
Yes

Newly	  generalizedis	  aimed	  at	  achieving	  ⇔	  aims	  to	  achieve	  is	  X:ed	  at	  Y:ing	  ⇔	  X:s	  to	  Y:e	  amendment	  of	  regula9on	  ⇔	  amending	  regula9on	  amendment	  of	  X	  ⇔	  amending	  X	  Y:ment	  of	  X:φ	  ⇔	  Y:ing	  X:φ	  development	  of	  tourism	  ⇔	  developing	  tourism	  development	  of	  X	  ⇔	  developing	  X	  Turned	  iden9cal

66 100

80
40
0841

106
107
108
# of words in the English side of bilingual corpus

N Grammar Meaning Both
SSeed
66
0.85
0.91
0.76
SID
339
0.84
0.78
0.66
SLV
534
0.74
0.78
0.59
Total 600
0.75
0.79
0.61
CCG+LM	  [Callison-­‐Burch,	  08](ii)	  Filtering	  candidates	  [Gaussier,	  99]	  109

120

Leverage	  is	  limited	  in	  some	  condi9on! Human	  evalua9on	  of	  paraphrase	  subs9tu9ons	  l Apply	  paraphrases	  in	  (a)	  to	  WMT	  “newstest”	  data	  l 200	  phrases	  *	  3-­‐best	  paraphrases	  chosen	  by	  5g	  LM	  #	  of	  unique	  stems	  having	  at	  least	  5	  characters	  in	  length

NTCIR: SLVNTCIR: SID
Europarl: SLV
Europarl: SID

205

101010# of words in the English side of bilingual corpus

Quality: Reasonably High

0.68

0.61

0.55

The	  ﬁrst	  decision	  was	  given	  following	  a	  complaint	  by	  the	  Quebec	  Common	  Front	  of	  persons	  receiving	  social	  assistance.	  
The	  ﬁrst	  decision	  was	  given	  following	  a	  complaint	  by	  the	  Quebec	  Common	  Front	  of	  persons	  who	  receive	  social	  assistance.	  
Investment	  is	  intended	  to	  improve	  the	  availability	  of	  locomo9ves	  and	  the	  rail	  network,	  even	  in	  the	  face	  of	  extreme	  weather	  condi9ons.	  
Investment	  is	  aimed	  at	  improving	  the	  availability	  of	  locomo9ves	  and	  the	  rail	  network,	  even	  in	  the	  face	  of	  extreme	  weather	  condi9ons.	  
The	  safety	  issue	  was	  considered	  suﬃciently	  serious	  for	  all	  aﬀected	  par9es	  to	  be	  informed.
The	  safety	  issue	  was	  suﬃcient	  considera9on	  serious	  for	  all	  aﬀected	  par9es	  to	  be	  informed.
Federal	  Security	  Service	  now	  spread	  a	  big	  network	  of	  fake	  sites	  and	  there	  are	  tons	  of	  poten9al	  buyers	  of	  military	  weapons.
Federal	  Security	  Service	  now	  spread	  a	  big	  network	  of	  fake	  sites	  and	  there	  are	  tons	  of	  a	  poten9al	  buyer	  of	  military	  weapons.

Searching	  monolingual	  corpus	  for	  new	  instances	  Future Work

Both	  lexical	  and	  sub-­‐lexical	  matching	  l High-­‐level	  paralleliza9on:	  both	  corpus	  and	  pa;erns	  ! Eﬀec9ve	  combina9on	  of	  heterogeneous	  paraphrase	  lexicons	  l

!

10

[Fujita,	  13]

[Fujita+,	  12]	  leaves	  this	  as	  is

240

NTCIR

9104

8High	  leverage	  when	  Bi	  <<<<	  MonoSID:	  generalize	  only	  iden9cal	  words	  [Fujita+,	  12]lis	  aimed	  at	  achieving	  ⇔	  aims	  to	  achieve	  !

0.97M

101010# of words in the English side of bilingual corpus

1-­‐1.	  Collec9ng	  aﬃx	  pa;erns	  Word1
Word2
aimedaims
aimedachieve
achieving aims
achieving achieve

66x

SLV
SID
SSeed(i)	  Extrac9ng	  all	  candidates

Step 2. Harvesting New Paraphrase Pairs	
SLV : new paraphrase pairs	
cohesion	  in	  Europe	  ⇔	  European	  cohesion	  democracy	  in	  Europe	  ⇔	  European	  democracy	  increase	  in	  Hai9	  ⇔	  Hai9an	  increase	  transporta9on	  in	  suburb	  ⇔	  suburban	  transporta9on	  economy	  in	  Uruguay	  ⇔	  Uruguayan	  economy	  amendment	  of	  documents	  ⇔	  amending	  documents	  amendment	  of	  protocol	  ⇔	  amending	  protocol	  investment	  of	  resources	  ⇔	  investing	  resources	  recruitment	  of	  engineers	  ⇔	  recruiting	  engineers	  should	  be	  highlighted	  that	  ⇔	  is	  worth	  highlighting	  that	  should	  be	  reiterated	  that	  ⇔	  is	  worth	  reiterating	  that	  should	  be	  stated	  that	  ⇔	  is	  worth	  stating	  that	  109xis	  aimed	  at	  achieving	  ⇔	  aims	  to	  achieve	  Step 1. Learning Paraphrase Patterns	Paraphrase patterns	
X:φ	  in	  Y:φ	  ⇔	  Y:an	  X:φ	  X:ment	  of	  Y:φ	  ⇔	  X:ing	  Y:φ	  should	  be	  X:ed	  that	  ⇔	  is	  worth	  X:ing	  that	  

63.8M
26.8M

4Exploiting Lexical Correspondences within Paraphrases
SSeed : seed paraphrase pairs

Europarl

105
10Moses,	  MeCab,	  SyMGIZA++,	  UniNe	  stoplists	  103

Corpus
MethodCoverage Accuracy Need	  clean	  upLargestLower	  than	  the	  others(A) MonolingualDist.	  sim.	  e.g.,	  [Lin+,	  01]
(B) Bilingual/mul9lingual	  parallel Pivo9ng	  e.g.,	  [Bannard+,	  05]LimitedRela9vely	  high(C) Monolingual	  parallel
Alignment	  e.g.,	  [Barzilay+,	  01]LimitedRela9vely	  high(D) Monolingual	  comparable
Alignment	  e.g.,	  [Hashimoto+,	  11]LimitedRela9vely	  high! Prefer	  unsupervised	  and	  language-­‐independent	  way	  # of paraphrase pairsl

Ratio of paraphrase pairs
against SSeed! Promising	  results	  Quantity: Large Multiple of Seed Paraphrases# of paraphrase pairs! Given	  a	  paraphrase	  lexicon	  à	  Obtain	  a	  large	  number	  of	  new	  paraphrase	  pairs	  l Generaliza9on:	  Obtain	  paraphrase	  pa;erns	  by	  generalizing	  corresponding	  words	  l Instan9a9on:	  Collect	  phrases	  that	  match	  the	  paraphrase	  pa;erns	  from	  monolingual	  corpus	  Assessing	  new	  instance	  with	  yet	  another	  similarity	  ! Applica9on	  to	  further	  large	  SSeed	  and	  monolingual	  corpus	  Adjacent	  word	  1-­‐4	  grams	  of	  phrase	  appearances	  l High-­‐level	  paralleliza9on:	  both	  corpus	  and	  phrases	  ! Integra9on	  into	  NLP	  applica9ons,	  e.g.,	  MT	  l! Applica9on	  to	  various	  languages