Fine‐Grained Information Accessand MT for Patents
Junichi TSUJII
University of Tokyo,
University of Manchester
UK National Centre for Text Mining

Plan of the Talk
• Backgrounds
kd• Shifts in Society• Semantics‐based
Sti b d iinformation
fti access tto ttextt andd MT• NLP and Semantics‐based information access• DeepDparsing
i andd entity/event‐basedi /b d access• Efficient deep parsers• MT with Deep Parsing• Structural Differences: Predicate‐Argument Structures• Lexical Differences: Terms and Ontologies• Concluding remarks and future work

Plan of the Talk
• Backgrounds
kd• Shifts in Society• Semantics‐basedSti b d iinformation
fti access tto ttextt andd MT• NLP and Semantics‐based information access• DeepDparsing
i andd entity/event‐basedi /b d access• Efficient deep parsers• MT with Deep Parsing• Structural Differences: Predicate‐Argument Structures• Lexical Differences: Terms and Ontologies• Concluding remarks and future workMT for Patents among Asian Languages• Shift ini societyi t– From Labor‐based Industrial Society to Knowledge‐based IndustrialSociety• Shift ini theth worldld– Globalization– From the Economies in the Atlantic‐rim to the Economies in thePacific‐rimifi i– Translation among (Asian Languages + English)• Difficulties in MT– Translation among European languages to Translation among (AsianLanguages + English)• Structural and Lexical differences• Sentence‐oriented languages and Context‐oriented languages• SMT with great enhancement of semantics‐based processing

Plan of the Talk
• Backgrounds
kd• Shifts in Society• Semantics‐basedSti b d iinformation
fti access tto ttextt andd MT• NLP and Semantics‐based information access• DeepDparsing
i andd entity/event‐basedi /b d access• Efficient deep parsers• MT with Deep Parsing• Structural Differences: Predicate‐Argument Structures• Lexical Differences: Terms and Ontologies• Concluding remarks and future work

G‐protein coupled receptor[D.L.Banville 2006]

Before
1992
1988
2005
9256
paperspapers 14,000,000

600,000

14,000 papers

12,000,000

500,000

Increments

MEDLINE alone

iincrements

400,000

More than 0.5 million per year
More than 1.3 thousand per day
Articles added

300,000

10,000,000

8,000,000

6,000,000

accumulatio
on：accumulation

200,000

100 000
100,0000

Medline Access

4,000,000

1997: 01997
0.163
163 M accesses/month/h2006: 82.027 M accesses/month

2,000,0000
年

500 times more

IIncrease
iin Medline
M dli
6

NaCTeM
www.nactem.ac.uk• FirstFi suchh centre iin theh worldld• Funding: JISC, BBSRC, EPSRC• Consortium investment• Chair in TM (Prof. J. Tsujii, Univ. Tokyo)• Location: Manchester Interdisciplinary Biocentre(MIB) www.mib.ac.uk funded by the Wellcome Trust• Initial focus: biomedical academic community• Extend services to industry• Extend focus to other domains (social sciences)7

Consortium• Universities
U i
iti off Manchester,M h t LiverpoolLil• Service activity run by MIMAS (National Centre forDataset Services),Services) within MC (ManchesterComputing)• SelfSelf‐funded
funded partners– San Diego Supercomputing Center– University of California, Berkeley– University of Geneva– University of Tokyo• StrongSt
industrial
i d t i l & academicd i supportt– IBM, AZ, EBI, Wellcome Trust, Sanger Institute, Unilever,NowGEN,o G , MerseyBio,e sey o, …
8Semantics based fine‐grained
Semantics‐basedfine grained information accessFrom surface diversities and ambiguities
toconceptual invariants
Non‐TrivialMappings
Terminology
ParsinggParaphrasing
Languageg g Domain

Knowledge Domain

Linguistic expressions

Concepts and Relationships
among Them
Motivated
Independently of language
10[A] protein activates [B] (Pathway extraction)
Transcription initiation by the sigma(54)‐RNA polymerase holoenzyme requires
Since…….,protein
wepostulatethatthis
onlyStraufen
proteinlacking
insertion
is ableto assocaite
anFull‐strengthenhancer‐bindingthatis thought
tophosphorylated
contactsigma(54)to activate
could its
activate
the transcription
genemRNAprotein
and activatetranslationtranslation,
but fails to of…..PHO5 gene.
t with osker
transcription.
i ti PHO2[[sentence]] > ([([arg1_activate]1i] > [protein])[i ])Retrieval using
Regional Algebra

Non‐trivial
Mapping

Language Domain

Same relations
withi h different
diffStructures

Knowledge Domain
Independently motivated of11Language

D
DeepParsing
P iParser based on Probabilistic HPSG (Enju)

Predicate: Activate
Arg1: P53
Arg2: Bcl‐2

SVP
VPindexing

VP
MEDLINE

SVP

arg3
arg1

NP
p53

arg2

hasbeen shown

ADVP
toarg2

directly activateNP

the Bcl‐2 protein述語/項構造
確率ＨＰＳＧ解析器 (Enju)の出力
s

Semantic Retrieval System

vp

Using Deep Syntax
MEDIE

vp
np

arg2

modd

Passive

Passive and Infinitival Clause

pp

arg1

dt

np

vp

vp

pp

DT

NN

VBZ VBN IN

TheproteinnpPRP

is activated byit13

14

15

DDeepParsing
P iParser based on Probabilistic HPSG (Enju)

Predicate: Activate
Arg1: P53
Arg2: Bcl‐2

S
VP
VP

indexing

VP
MEDLINE

SVP

arg3
arg1

NP
p53

arg2

hasbeen shown

ADVPtoarg2

directly activateNP

the Bcl‐2 protein

Results on PTB
PTB‐WSJWSJPParsergrammarA
AccuracyS
SpeeddMST parser

dependency

90.02%(LAS)

4.5 snt/sec

Sagae’s parser

dependency

89.01%(LAS)

21.6 snt/sec

Berkeley parser

CFG

89.27%(LF1)

4.7 snt/sec

Ch i k’ parser
Charniak’s

CFG

89.55%(LF1)

2 2 snt/sec
2.2t/

Charniak’s parser
rerankerCFG

91.40 %
(LF1)

1.9 snt/sec

Enju parser

HPSG

88.87%(PAS‐LF1)

2.7 snt/sec

Mogura parser

HPSG

88.07%(PAS‐LF1)

22.8 snt/sec

Plan of the Talk
• Backgrounds
kd• Shifts in Society• Semantics‐basedS
ti b d iinformationfti access tto ttextt andd MT• NLP and Semantics‐based information access• Deep
Dparsing
i andd entity/event‐basedi /b d access• Efficient deep parsers• MT with Deep Parsing• Structural Differences: Predicate‐Argument Structures• Lexical Differences: Terms and Ontologies• Concluding remarks and future work

Fine-grained tree-to-stringtranslation rule extraction(presented in ACL 2010))(pXianchao Wu (呉 先超)
Takuya Matsuzaki, Jun’ichi Tsujii
The University of Tokyo
2010.07.23
7 319

Outline
PCFG pparsers

bias

slow

Fine grained
Fine‐grained
Tree‐to‐string rules
Typed featurestructures

Predicate argumentstructures

An HPSG parser
20

Minimala ruleu e eextractiont act o : GGHKM(Galley+, 2004)
span:0‐4p
complement_span:Ԅ

S0 NP
2‐4240NX
2‐4t0

2‐4 VP0VX

NO overlapping
Admissible nodes

NPS
John

t1

killedX0 ha X1

NX

X0:NP

X1:VP

t2 Mary
ジョン は マリー を 殺した 。
01
2
34Marykilled .
gloss: John

NP
X0
X0:NX
21

HPSG for syntax-based
syntax based SMT
S
NP

VP

NX

VX

NP

John

killedNX
M
Marynodes

features

NonNonterminal

CAT, XCAT,CATXCATSCHEMA, HEAD,
SEM_HEAD

terminal

CAT, POS, BASE,TENSE, ASPECT,VOICE, AUX,
LEXENTRY,PRED, ARG<n>

Typed‐feature structure
(TFS)

22

Additional comparison

Systems

En2Jp

Jp2En

Joshua (Li+, 2009)21.7919.73

FForest-based
b d24.75****

22.67**
6 ****=p<0.0123

Remove the ambiguous alignments
c1c0c2c20

c6c3c23c8

22c11c17c5c7

c13
13c9t3
thec103c16

c22c103‐4

c45‐8t1t4

resultwas

実験0c45‐7t8

t10

c215‐7c215‐8t7

verifiedによって1 この2 結果3 が4
C1C2

C3

c125‐7c18

c125‐8t0

c19

c14t2

c15

t6

t5t9

by

the experiments

検証5 さ6 れ7 た8C424

Remove the ambiguous alignments

Systems

En2Jp

Forest-based
b d rulesl (Wu+, ACL 2010)24.75+ forest-based decoder

25 83
25.83

+ function words

28.44**
44**=p<0.01
p 0025

Plan of the Talk
• Backgrounds
kd• Shifts in Society• Semantics‐based
Sti b d iinformation
fti access tto ttextt andd MT• NLP and Semantics‐based information access• Deep
Dparsing
i andd entity/event‐basedi /b d access• Efficient deep parsers• MT with Deep Parsing• Structural Differences: Predicate‐Argument Structures• Lexical Differences: Terms and Ontologies• Concluding remarks and future work

From surface diversities and ambiguities
toconceptual invariants
Non‐TrivialMappings
Terminology
ParsinggParaphrasing
Languageg g Domain

Knowledge Domain

Linguistic expressions

Concepts and Relationships
among Them
Motivated
Independently of language
27

Annotation of GENIA corpus – Term&POS
Part-of-speech
annotation
2,000 abstracts

Term (entity)
annotation
2000+400
abstracts
28

Event Annotation ‐ Example