Guihong Cao, Jian-Yun Nie and Lixin Shi
Department of Computer Science and Operations Research
University of Montreal, Canada

NTCIR-7 Patent MiningExperiments at RALI

1

• Introduction• Our Approaches• Issues Investigated• Experiments• Conclusion

Outline2

– View it as a text categorization problem• Possible solution– Query: abstract of a research paper– Document collection: patents with IPC code
– Task: assign IPC codes to each research paper
according to the relevance• Task– Each patent has an IPC code

• Patent Mining ProjectIntroduction3

– More than 50,000 IPC codes– Very unbalanced– Cannot be tackled with traditional text classification
approaches• Complexity in classification problem– Patent: more general terms to cover more related
things– Research paper: more precise and technicalEg. Music player VS Apple iPod

• Difference in writing style for patent andresearch paper

Introduction (Cont.)

4

#IPC
25944
10911
1430
129
46

#Patent

1~10

11~100101~500

501~1000

1001~2000>5000

4001~5000

3001~4000

2001~3000#Patent

23

0

3

5

#IPC

5

Distribution of IPC codes in US patents• The Issues Investigated• Experiments• Conclusion– Basic approach– System Description• Introduction• Our Approaches

Outline6

7– Language modeling approach for information retrieval– Measuring relevance by query likelihood• Finding closest documents with informationretrieval– The patents are labeled instances– Measure the distance between patents and research
paper according to relevance• Classify the research paper with K-NN classifier

Basic Approachtf ( w , D )+ (1 − λ ) P ( w | C )|D|qi

P (q | D ) = ∏ P (qi | D )• A query is represented as a sequence of words• Relevance is measured by the likelihood of querywith respect to the document modelP(w | D ) = λ8– P(w|D) is smoothed to avoid zero probablity (Zhai andLafferty, 2001)• Documents are represented with unigram
models, i.e., P(w|D)

Language Modeling Approach for
Information Retrievalδ (ipc(d i ) = c ) : indicator functioni =1• Classification algorithm
Kscore ( c , q ) = ∑ δ (ipc ( d i ) = c )P ( q | d i )– Language modeling approach for IR– Allowing retrieval using different fields9• The whole system is implemented using INDRI
system (Strohman et al, 2005)• INDRI system

System Description• Introduction• Our Approaches• The Issues Investigated• Experiments• Conclusion

Outline10

– Selected a set of common terms in research paper
according to document frequency– Filtering out the common words in query time

• Some common words in research paper are notcommon words in patent description
e.g. paper, study, propose• Introducing noises to patent retrieval• Out approach– Aiming to solve different styles between research
paper and patent description• Term Distillation

Investigations

11

lt
gt
paper
papersmethodmethods
study
studiesstudying
studiedproposeproposedbased
obtainobtains
obtainedfind
foundresultresults

preparepreparespreparingpreparedcarry
carriescarrying
carriedshow
showedCommon termsshows
showingshown
reportreported12• With equal weights– Using some of the fields– Aggregating occurrence of query terms in different
fields with linear interpolation• Experiments:– Background, description, summary and drawing• Specification can be divided into fours sub-fields– Title, abstract, specification and claim• Patent: structured documents• Different fields have different impacts• Four main fields

Mining Patent Structures

13

14

• An effective technique to enrich query withterms from top-ranked documents• Pseudo-relevance feedback• Number of feedback documents and query
terms is a key issue• More effective for short queries• Is it effective for the Patent Mining task (quitelong query)?

Query Expansion

• Introduction• Our Approaches• The Issues Investigated• Experiments• Conclusion

Outline15

– Mean average precision– Precision at top N documents (P@N)• Evaluation metrics– Porter stemmer– Removing stop words• Query and document processing: in standard
wayExperiments

160.0046

Term Distillation 0.0282

Is it due to the terms selected?

Does not seem to be effective.

0.00470.0277

Original

P@100

P@30

Model0.1491

0.1502

MAP

Term Distillation Results

170.02710.02740.02740.02770.02770.0276

020

40

60

80

100

0.00300.00300.00290.00300.00290.0047

P@100

0.14560.14390.14470.1451

0.14700.1488

MAP

Possibly due to the fact that queries (paper abstracts) arealready quite long.

Observation: Not very effective.

P@30#Exp. Terms

Top 20 documents

The Effectiveness of Query Expansion

18

0.02770.02700.02810.0276

T+A+S+C

T+A+B

T+A+B+D

T+A+B+D+M

No significant differences

P@30

A: abstract
D: description

Fields

T: title
B: background0.00470.00490.00410.0047

P@100

0.14950.14890.14700.1502MAP

S: specification
M: summary

C: claim
R: drawing

The Impact of Different Fields

19

The Impact of Different K Values

200.0234
0.0241

rali_baselinerali_short_doc

0.00480.0050

P@100

Need to carry out more experiments using different fields.

Marginal effect.

P@30

Run ID

Rali_short_doc: Title+Abstract+Description

rali_baseline: Title+Abstract+Specification+Claim

Formal Run Results

0.14370.1423MAP

21

– Exploiting more characteristics of patents?
– Term relationships?

• Questions:• Different fields• pseudo-relevance feedback• Term distillation– Only the value of K has some impact on classification effectiveness– The other factors do not seem to affect the classification accuracy:• Investigated several issues– K-NN classifier• Classification of research abstracts into IPCConclusion

22

Thanks!

23