< WTEP 2007, Jun. 29th, 2007 >A Compositional Approach toward
Dynamic Phrasal Thesaurus
Atsushi FUJITA, Shuhei KATO,Naoki KATO, Satoshi SATO
Nagoya Univ., Japan

Computing Semantic Equivalence (SE)Fundamental in NLP
Recognition: IR, IE, QA
 Generation: MT, TTS, SummarizationPrevious attempts used ...
Thesauri [So many work] Tree kernels [Collins+, 01] [Takahashi, 05]
 Statistical translation models [Barzilay+, 03] [Brockett+, 05]
 Distributional similarity [Harris, 64] [Lin+, 01] [Weeds+, 05]
 Syntactic patterns [Mel’cuk+, 87] [Dras, 99] [Jacquemin, 99]
2Computing Semantic Equivalence (SE)Fundamental in NLP
Recognition: IR, IE, QA
 Generation: MT, TTS, SummarizationPrevious attempts used ...
Words are not necessarily the unit of meaning
Thesauri(polysemous words, meaning of construction)
 Tree kernels
Cannot generate paraphrases
 Statistical translation models
Corpus is not almighty(data sparseness, cost) Distributional similarity
 Syntactic patterns
No thorough list

3

Our Proposal
Phrasal Thesaurus

A mechanism for directly computing SE between phrases
be in our favor

be favorable for us

its reproducibility

if it is reproducible

show a sharp decrease

decrease sharply
investigate why there was a fire

investigate the cause of a fire

investigate what started a fire
make an investigation
into the cause of a fire4

Aim
Implement tools and resources
Application-independent module Human aids: writing / reading texts
Confirm phrase is appropriate unit for computing SE

Ambiguity of words >> Ambiguity of phrases(more suitable to handle)
This is a preliminary progress report(w/o concrete evaluation)5

Outline
1.

Motivation & Aim

2.

Range of phenomena

3.

System & implementation
Discussion
Conclusion

4.
5.

Towards Phrasal Thesaurus
What sorts of phrases?
How to handle a variety of expressions?
be in our favor

be favorable for us

its reproducibility

if it is reproducible

show a sharp decrease

decrease sharply
investigate why there was a fire

investigate the cause of a fire

investigate what started a fire
make an investigation
into the cause of a fire7

Range of phrases
Predicate phrase (cf. various exps. in RTE)
Reliably captured using recent technologies
 Approx. corresponds to single event[Chklovski and Pantel, 2004] [Torisawa, 2006]

Our target language: Japanesenoun phrase + case marker + predicate
Various noun phrases
 Various predicates Case markers indicate grammatical roles of noun phrases
8

Classification of noun phrases in Japanesecommon nounnominalization

9

Classification of predicates in Japanese10

Range of phrases
Our target language: Japanesenoun phrase + case marker + predicate
common nounnominalization

Variation of phrases >> Variation of words

Various combinations of open-class words
11

Range of phenomena
Variation of paraphrases of phrases>> Variation of paraphrases of words
Difficult (hard?) to statically enumerate No previous work explicitly collected:
“All verbs that can be passivized”
 “All noun-verb pairs that compose light-verb constructions”How to handle them?

12

Paraphrases of predicate phrases
X is charged by Y
X married Y
X change Y
X buy YX dated Y

Y announced the arrest of X
X prevent Y

X modify Y
X acquire Y

X lower the risk of Y

X change Y
X solve YX gives Y a fright

X alter Y
Y is solved by X

Y is frightened of X

X is in our favor

X is favorable for us

X show a sharp decrease

X decrease sharply

X get the sack

X be dismissed from employment

X see the light

X realize the truth
13

Paraphrases of predicate phrases
X is charged by Y
X married Y
X change Y
X buy YX dated Y

Y announced the arrest of X
X prevent Y

X modify Y
X acquire Y

X lower the risk of Y

X change Y
X solve Y

X gives Y a fright

X alter YY is solved by X

Y is frightened of X

X is in our favor

X is favorable for us

X show a sharp decrease

X decrease sharply

X get the sack

X be dismissed from employment

X see the light

X realize the truth
14

Paraphrases of predicate phrases
X is charged by Y
X married Y
X change Y
X buy YX dated Y

Y announced the arrest of X
X prevent Y

X modify Y
X acquire Y

X lower the risk of Y

X change Y
X solve Y

X gives Y a fright

X alter Y
Y is solved by X

Y is frightened of XX is in our favor

X is favorable for us

X show a sharp decrease

X decrease sharply

X get the sack

X be dismissed from employment

X see the light

X realize the truth15

Paraphrases of predicate phrases
X is charged by Y
X married Y
X change Y
X buy YX dated Y

Y announced the arrest of X
X prevent Y

X modify Y
X acquire Y

X lower the risk of Y

X change Y
X solve Y

X gives Y a fright

X alter Y
Y is solved by X

Y is frightened of X

X is in our favor

X is favorable for usX show a sharp decrease

X decrease sharply

X get the sack

X be dismissed from employment

X see the light

X realize the truth
16Compositional paraphrases (syntactic variants)
Syntactic transformation + Lexical derivation⇒ Dynamic generation (Dynamic Phrasal Thesaurus)
X solve Y

Y is solved by X

X gives Y a fright

XVY

Y is frightened of X
X give Y a Z

X is in our favor

Y be v(Z)-PP by X

Y is v(Z)-PP of X

X is favorable for us
X be in Z’s Y

X show a sharp decrease

X be adj(Y) for Z

X decrease sharply
X show a A Y

X v(Y) adv(A)17Compositional paraphrases (syntactic variants)
Syntactic transformation + Lexical derivation⇒ Dynamic generation (Dynamic Phrasal Thesaurus)Our target language: Japanesekakunin-o

isogu

checking-ACC
to hurry(We) hurry checking it.
Trivial? No.

isoidekakunin-suruin a hurry to check(We) check it in a hurry.

NCVadv(V) vp(N)Not exhaustively explored Beneficial [Dolan+, 04] [Romano+, 06]
18

Outline

2.

Motivation & Aim
Range of phenomena

3.

System & implementation

4.

Discussion
Conclusion

1.

5.

System overview
Input: Phrase (string)
Output: List of paraphrases
kakuninoisogu(We) hurry checking it.

Morphological analysis
Syntactic transformation

over-generation

Surface generation

filtering

SLM-based filteringisoide : kakunin-sa : re : ta
isoide : kakunin-shi : ta
isoide : kakunin-suru
isoide : kakunin-sa : reru

9532(It) was checked in a hurry.
(We) checked it in a hurry.
(We) check it in a hurry.
(It) is checked in a hurry.

(x)
(x)
(o)
(x)20

1. Morphological analysis
Input: Phrase (string)
Output: Array of morphemes w/ POS-tag

Using MeCab-0.91, a state-of-the-art morphological analyzer
kakuninoisogu (We) hurry checking it.
Morphologicalanalysis
kakunin : o : isogu
NC V
checking ACC to hurry

MeCab+ post-process
N: noun
V: verb
Adj: adjective
An: adjectival verb
Adv: adverb
C: case marker
etc.
21

2. Syntactic transformation: knowledge used
Transformation patternGenerates skeletons of syntactic variants
kakunin : o : isogu
NC V
checking ACC to hurry

Trans. Pat.
N:C:V  adv(V):vp(N)adv(isogu) : vp(kakunin)adv(to hurry) : vp(checking)Generation function
Enumerates expressions made of the given set of words
vp(kakunin)vp(checking)genTense()

Lexical function
Gen. Func.
vp(N)Gen. Func.
genTense(){v(kakunin) : genVoice() : genTense()}v(checking){, ta/da}
COP

Generates different lexical items in certain relation
adv(isogu)
adv(to hurry)

Lex. Func.
adv(V)

isoidein a hurry22

2. Syntactic transformation: example
kakunin : o : isogu
NC V

Syntactictransformation

checking ACC to hurry

Trans. Pat.
N:C:V  adv(V):vp(N)adv(isogu) : vp(kakunin)

Lex. Func.
adv(V)
isoide{v(kakunin) : genVoice() : genTense()}Lex. Func.
v(N)kakunin-suru

Gen. Func.
vp(N)Gen. Func.
genVoice()

Gen. Func.
genTense(){, reru/rareru, seru/saseru}{, ta/da}

isoide : {kakunin-suru : {, reru/rareru, seru/saseru} : {, ta/da}}23

3. Surface generation
Input: Bunch of candidate phrases
Output: List of candidate phrases1. Unfolding 2. Lexical choice (exclusively used auxiliaries)
 3. Conjugation


isoide : {kakunin-suru : {, reru/rareru, seru/saseru} : {, ta/da}}Surfacegeneration
isoide : kakunin-suru,
isoide : kakunin-sa : reru,
isoide : kakunin-sa : seru,

isoide : kakunin-shi : ta,
isoide : kakunin-sa : re : ta,
isoide : kakunin-sa : se : ta
24

4. SLM-based filtering
Input: List of candidate phrases
Output: List of grammatical phrases

Grammaticality assessment
Initial model: if occur in Mainichi 1999-2005 (1.8GB)
isoide : kakunin-suru,
isoide : kakunin-sa : reru,
isoide : kakunin-sa : seru,

isoide : kakunin-shi : ta,
isoide : kakunin-sa : re : ta,
isoide : kakunin-sa : se : ta

SLM-basedfiltering
isoide : kakunin-sa : re : ta
isoide : kakunin-shi : ta
isoide : kakunin-suru
isoide : kakunin-sa : reru

9532(It) was checked in a hurry.
(We) checked it in a hurry.
(We) check it in a hurry.
(It) is checked in a hurry.

(x)(x)(o)(x)

25

Knowledge development
Paraphrase phenomena ⇒ Create patterns
Not necessarily from examples
 Same manner as

MTT [Mel’cuk+, 1987]
 STAG [Dras, 1999]
 FASTR [Jacquemin, 1999]
 KURA [Takahashi+, 2001]
cf. FrameNet [Baker+, 1998]


Frame ⇒ Register various expressions

26

Comparison w/ previous work
MTT [Mel’cuk+, 1987]
Paraphrasing rules at 7 levels
 More than 60 Lexical functions
FASTR [Jacquemin, 1999]

Trans. Pat.
N:C:V  adv(V):vp(N)Lex. Func.
adv(V)
Trans. Pat.
N:C:V  adv(V):vp(N)Structural transformations (Syntagma)
Lex. Func.
 Semantic links (Paradigm)
Oursadv(V)Transformation at SSynt level only (cf. MTT) Predicate phrase, not technical term (cf. FASTR)
Gen. Func.
 One-to-N generation by Gen.Func.
vp(N)
27

Current scale of knowledge
Transformation patternStarting from N:C:V


N1:N2:C:V, N:C:V1:V2, ... : 37 patterns

Generation functionTrans. Pat.
N:C:V  adv(V):vp(N)Gen. Func.
vp(N)As a by-product of generalizing transformation patterns
Content phrases (5): NPs, VPs
 Functional expressions (4): Case markers, Auxiliaries


Lexical function

Lex. Func.
adv(V)

Lexical derivation (10 dics, totally 6,322 word pairs)
 Noun-to-interrogative (1)28To ensure coverage
1. Enumerate Trans. Pat. for N:C:V
2. Extend them for more complex types of phrases
N:C:V
N:C:V  vp(N)N1:N2:C:V  vp(N1, N2)

N1:N2:C:V

N1:N2:C:V  N1:genCase():vp(N2)
N1:N2:C:V  N2:genCase():vp(N1)
N1:N2:C:V  np(N1, N2):genCase():lvc(V)

N:C:V  N:genCase():lvc(V)N1:N2:C:V  N2:genCase():lvc(V)N1:N2:C:V  N1:genCase():lvc(V)N1:N2:C:V  adv(V):vp(N1, N2)

N:C:V  adv(V):vp(N)N1:N2:C:V  adv(V):N1:genCase():vp(N2)
N1:N2:C:V  adv(V):N2:genCase():vp(N1)
N1:N2:C:V  np(N1, N2):C:V

29

The body of Lex. Func.
IPADIC-2.7.0 + Mainichi 1999-2005 (1.8GB)
POS-pair｜D｜ ｜C｜ ｜D ∪C｜ ｜J｜ cleaning
noun - verb
3,431- 3,431 3,431
noun - adjective
308
906
475 done667
noun - adjectival verb
1,579
- 1,579 1,579
noun - adverb
271
271
271
verb - adjective252
252
192 doneverb - adjectival verb
74
74
68 doneverb - adverb
74
64 done74
adjective - adjectival verb
159
146 done
66
95
adjective - adverb
33
33
26 doneadjectival verb - adverb
70
70
70
Total

6,158

762

6,849

6,322
30

Outline

3.

Motivation & Aim
Range of phenomena
System & implementation

4.

Discussion

5.

Conclusion

1.
2.

Discussion (≒ future work)
Sufficient condition
Patterns does not ensure paraphrasability perfectly Extensional definition of selectional preferences [Pantel+, 2007]
Structured transformation
For flexible and accurate matching Less impact due to short phrase
Methodology of resource development
Modularization of Gen. Func. is inconsistent Requires linguistic expertise Simple KBs are preferable (cf. MTT)32

Conclusion & Future work
Notion of Phrasal Thesaurus is introduced
Compositional paraphrases of predicate phrases
 Preliminary progress report of resource development
Future work

DevelopmentResources SLM (Structured, Web, etc.)
 Applicability conditions
Trans. Pat.
N:C:V  adv(V):vp(N)Gen. Func.
vp(N)Lex. Func.
adv(V)

Intrinsic / extrinsic evaluation

33