Generating & Recognizing Paraphrases
Paraphrases< IJCNLP 2005, Oct. 11th, 2005 >!

“Alternative ways to convey the same information” (IWP)

Middleware for a wide range of application

Exploiting Lexical Conceptual Structure
for Paraphrase Generation!

Generation""

Atsushi FUJITA(1), Kentaro INUI(2), Yuji MATSUMOTO(2)(1) Kyoto University(2) Nara Institute of Science and Technology!

Text simplification !Carroll et al., 1999"!Inui et al., 2003"
Pre- and post-editing for MT !Shirai et al., 1995"Recognition""

QA !Hermjakob et al., 2002"!Takahashi et al., 2004"
Multi-document summarization !Barzilay et al., 2003"

2Variety

Lexically compositional paraphrases (in Japanese)

Lexical paraphrase

Paraphrasing of light-verb constructions (LVCs)

Emma burst into tears and he tried to comfort her.

film-NOM him-DAT impression-ACC to give-ACTIVE

Emma cried and he tried to console her.

film-NOM him-ACC to be impressed-CAUSATIVE

Syntactic paraphrase(The film made an impression on him.)!Barzilay et al., 2001"

It was his best suit that John wore to the dance last night.

he-NOM wall-DAT paint-ACC to splay(He splayed paint on the wall.)John wore his best suit to the dance last night.

Lexically compositional paraphrase(The film impressed him.)

Locative alteration

he-NOM wall-ACC paint-with to splay!Dras, 1999"(He splayed the wall with paint.)

Category shifting

Steven made an attempt to stop playing Hearts.

room-NOM already-ADV to warm-Verb-Passive-Perfective

Steven attempted to stop playing Hearts.

(The room has already been warmed up.)room-NOM already-ADV be warm-Adjective-Present!Dras, 1999"(The room is already warm.)

3Issues4Paraphrasing of LVCs

Issue: to explore…

LVCs

what sorts of lexical properties affect! how existing framework of lexical semantics can be used
to represent them!

!

paraphrase

single verb phrases

Syntactic and semantic properties of two verbs interact
head of semantics

head of syntax

film-NOM him-DAT impression-ACC to give-ACTIVE

Our attempt!

- syntactically regular- semanticallycompositional(The film made an impression on him.)

Exploit Lexical Conceptual Structure (LCS)

film-NOM him-ACC to be impressed-CAUSATIVE!Jackendoff, 1990"(The film impressed him.)

Examine current theory and implementation of LCS forJapanese !Kageyama, 1996"!Takeuchi et al., 2002"" Develop an LCS-based paraphrase generation model" Case study on paraphrasing of LVCs in Japaneseprice-NOM exchange-DAT influence-ACC to give-ACTIVE"(The stock price gives an influence to the foreign exchanges.)

price-NOM exchange-DAT to influence-ACTIVE(The stock price influences the foreign exchanges.)
56

Contents
1.
2.
3.
4.
5.
6.

Overview of LCS

Issues and goals
LCS
Paraphrasing of LVCs in Japanese
LCS-based paraphrase generation model
Experiments
Conclusion

What’s LCS? !Jackendoff, 1990"!

A verb classification which reflects
several syntactic and semantic properties of verbs“'(&%” (to locate) : Non-agentive“)*” (to play) : Agentive" Focus of statement: “+,%” (to give) : Agent“-$%” (to receive) : Goal
" Link between syntax and semantics (Linking):“./&%” (to transit) : (NOM, ACC) = (Theme, Goal)“#$%” (to deliver) :(NOM, ACC, DAT) = (Agent, Theme, Goal)"Agentivity:

7

ExampleElectronic resource and application“#$%” (to deliver), “!&” (to translate)
Agentivity: Agentive! Focus: Agent! Linking:(Agent, Theme, Goal)= (NOM, ACC, DAT)
! Etc.
!

English LCS Verb Lexicon[x CONTROL [BECOME [y BE AT z]]]!

!Takeuchi et al., 2002"!

!shopper"xshopper-NOM customer-DAT
product-ACC to deliver-ACTIVE!
!

BECOME(The shopper delivers the product to the customer.)

Theme

BE AT

3.
4.
5.
6.

1,165 verbs / 16 LCS types
Compound noun analysis !Takeuchi et al., 2002"

Further projects are running (for Japanese)!Kato et al., 2005"!Takeuchi et al., 2005"

Goal!product"y!customer"z

ACCDAT

9Contents
2.

4,163 verbs / 468 LCS typesMT!Dorr, 1997"!Habash et al., 2003", NLG !Traum et al., 2000"

Takeuchi’s Japanese LCS dictionary

CONTROL
AgentNOM

1.

810

Intriguing points

Issues and goals
LCS
Paraphrasing of LVCs in Japanese
LCS-based paraphrase generation model
Experiments
Conclusion

DAT is not necessarily changed to ACC“give” != CAUSATIVE

film-NOM him-DAT impression-ACC to give-ACTIVE(The film made an impression on him.)film-NOM him-ACC to be impressed-CAUSATIVE(The film impressed him.)

TV-NOM kids-DAT stimulation-ACC to give-ACTIVE(TV gives kids stimulation.)

TV-NOM kids-ACC to stimulate-ACTIVE(TV stimulates kids.)

price-NOM exchange-DAT influence-ACC to give-ACTIVE(The stock price gives an influence to the foreign exchanges.)

price-NOM exchange-DAT to influence-ACTIVE(The stock price influences the foreign exchanges.)1112

How LCS is expected useful?

Contents

To determine voice and syntactic cases

1.

film-NOM him-ACC to be impressed-CAUSATIVE
Syntactic cases
Voice

2.

!

3.

Voice: how the event is described !Muraki, 1991"
Who causes the event" Who is influenced by the event"!

4.

Agentivity, Focus

6.

Syntactic cases:"

5.

Which marker should be assigned
for each nominal element

Issues and goals
LCS
Paraphrasing of LVCs in Japanese
LCS-based paraphrase generation modelExperiments
Conclusion

Linking

13

LCS as lexical constraints / a tool for transfer
BECOMEBECOMEBE WITH

BE WITHMOVE FROM TO!Ken"zNOM!inspiration"y !film"x
ACC!Ken"zDAT

1. Semantic analysis

inspiration-ACC

Surface 01LCS (light-verb)
BECOME!film"x!Ken"yNOM

ACC

BE WITH(2)

3. Surface generation!Ken"zto inspire-PASSIVE-???????

NOM

Syntactic transfer

to receive
Ken-NOM

Step 1: Semantic analysis

ACT ON!Ken"z

2. Semantic transfer

14

MOVE FROM TO!inspiration"y !film"x

KenKen-???
NOMfilm-DAT

ACC

filmfilm-???
DAT

Ken-NOM film-DAT
inspiration-ACC to receive-ACTIVE
Ken-NOM film-DAT inspiration-ACC to receive-ACTIVE

Ken-NOM??? film
film-???
-DATtotoinspire
inspire-???????
-PASSIVE(to receive)(Ken received an inspiration from the film.)

BE WITH(3)

Compatible predicate classes
MOVE FROM TO 21ACT ON

MOVE FROM TO!Ken"zNOM!inspiration"y !film"x
ACC

16

Step 3: Surface generation

LCS (light-verb) 01LCS (deverbal noun)
BECOME Focus of “ukeru”(1)

LCS dic.

15

Step 2: Semantic transfer!Ken"zDAT

DAT
Argument matching rules

BECOMEBE WITH

BE WITH!Ken"zACT ON(1)!Ken"z

LCS (deverbal noun) 01Surface

BECOME!Ken"zKen-NOM film-DAT to inspire-PASSIVE!Ken"yNOM

ACC(2)(to inspire)(0)

LCS dic.

17(2)

film-NOM Ken-ACC to inspire-ACTIVEACT ON!film"x(Ken was inspired by the film.)

Alteration rule:
if (Focus != Agent)
then passivize(The film inspired him.)!film"x!Ken"yNOM

ACC

(1)
18

Summary of the rules for paraphrasing of LVCs

Contents

Depth-first matching in LCS transformation!

1.

2 predicate classes:

2.

Agentive: CONTROL, ACT ON, ACT, etc.
" State of affair: MOVE TO, BE AT, BE WITH, etc.

3.

"!

2 argument matching rules

Generation:!

A decision list

CONTROL
Agent!shopper"x

NOM
Causativization * 2
Passivization * 1 State of affair" Leave active * 2

5.
6.

BECOME""

4.

Agentive

Theme

BE ATIssues and goals
LCS
Paraphrasing of LVCs in Japanese
LCS-based paraphrase generation model
Experiments
Conclusion

Goal!product"y!customer"z

ACC

DAT

19

Setting

20

Results

Gold-standard# of candidates# of correct paraphrases# of incorrect paraphrases

LM
547
322
225

LCS
798
624
174

LCS+LM717
609
108(1) 3 clauses for each of 245 most frequent types of LVC(2) annotators produced same paraphrases for 711 clauses
in terms of determining Voice and Syntactic cases

RecallPrecision
F-measure34=0.55.453.589
.512

.878
.782
.827

.857
.849
.852

LCS dictionary
1,165 deverbal nouns (T-LCS dic. ver. 0.95)! 40 frequent light-verbs (manually collected and assigned LCS)!

ModelsLM < LCS < LCS+LM

LM (baseline): selects a combination of voice and syntactic cases! LCS (proposed): generates all semantically explainable candidates! LCS+LM: filters anomalies among the output of LCS!

!
!

Lexical properties encoded in LCS are usefulLM itself does not work well, but contributes to filter outanomalies among semantically derived paraphrases

21

Error distribution(1) Ambiguous role of dative Step 1(2) Transformation algorithm Step 2(3) Definition of LCS
Step 0
Other errors
Total-22Error source: ambiguous role of dative @ step 1
LCS
78
59
30
7174LCS+LM
47
36
196108

Dative case often functions as an adjunctLargest portion
Violation of selectional restriction for argument! We need a semantic parsing technology!
!

MOVE TOCountermove(2)!exportation"y !gradually"zLCS typology should be firstly refined! For (1), semantic parsing is necessary! For (2), transformation principles should be reconsidered!

NOM
gradually-DAT exportation-NOM to increase-ACTIVE(The amount of exportation increased gradually.)
23

DAT(to increase)(1)

LCS dic.

24

Recent advances in semantic parsingConclusion

Semantically annotated corpus / lexicon

Exploiting LCS

FrameNet !Baker et al., 1998"
VerbNet !Kipper et al., 1998"! Propositional Bank !Palmer et al., 2005"! IAMTC !Dorr et al., 2004"

A model for paraphrasing of LVCs in JapaneseSemantic parsing technology

!

!

!

Lexical constraints (syntactic and semantic properties)!

Tool for semantic transfer!

"Agentivity, Focus of statement, Linking

Small sets of linguistically explainable rules
F-measure: .512 (LM) < .814 (LCS) < .839 (LCS+LM)! Error analysis guides further research avenues

Word sense disambiguation, semantic role labeling, etc.
! CoNLL-2004, CoNLL-2005 Shared Task
! Statistical methods have been well-discussed!

!

25

Current work

Future work

Restructuring LCS dictionary !Takeuchi et al., 2005"

Ultimate goal: cover various paraphrases

Re-organize the lexical properties to be encoded! Enlarge LCS dictionary!

!

!

"

By collecting semantically labeled examples
Technical issue is to reduce human labor for labeling

Enhancing LCS-based paraphrasing model!

Predicate / argument matching algorithms"!

Harmonizing semantics-based paraphrasing with
automatic paraphrase acquisition
1. Build semantics-based paraphrase generation models
for lexically compositional paraphrases" 2. Acquire paraphrases from corpus/Web" 3. Distill them into “idiosyncratic paraphrases” bydecomposing them using models built in step 1

Example-based semantic parsing !Hirano et al., 2005"!

26

Induce by comparing source and target LCSs

Implementation for other classes of paraphrase
2728