Opinion sentence and topic relevant
sentence extraction
by using coherent structure among
the sentences
Hironori Mizuguchi, Masaaki Tsuchida,Kenji Tateishi and Dai Kusui
NEC Common Platform Software Research Laboratories
2008/12/19 NTCIR7

1

Abstract Two sub tasks:
 Opinionated Sentence Extraction
 Topic Relevant Sentence Extraction

 New sentence extraction framework, the Sliding
Window Framework Use coherent structure among the sentences Result
 Very high recall ratio and a high F-value Opinionated Sentence Extraction: second best
 Topic Relevant Sentence Extraction: best
 F-value in lenient result.

Ｐage 2

2008/12/19 NTCIR7

Task Overview• Sentence based sub tasks– Opinion sentence extraction (Japanese)– Topic relevant sentence extraction(Japanese)
News article
ワシントン連邦地裁は独占的な力を乱用して消費者や競争相手の企業に不利益をもたらしたとして、世界のガリバー企業に厳しい判決を下した。
ビル・ゲイツ氏率いるマイクロソフト社は控訴する意向だが、今後の行方はどうなるのか。
今回の訴訟のポイントはパソコン基本ソフトの市場で９０％以上のシェアを誇るマイクロソフト社に対して、今から１００年以上も前に制定された独禁法（１８９０年・シャーマン法）が有効に機能できるのかという点にあった。

Topic：「Microsoft訴訟」
Ｐage 3

2008/12/19 NTCIR7

Opinion

Topic

Yes

YesNoNoYesNoProblem And Approach On Sentence Extraction• Consider only one sentence at once to extract
the sentence in previous systems.
• Not consider coherent structure among thesentences.
– Ex.1) Opinion and fact are written in different parts ofan article to avoid confusing these.
– Ex.2) Sentences that relate to a certain topic arewritten on the same place.

Use coherent structure.
• Same topics appear continuously.
Ｐage 4

2008/12/19 NTCIR7

Property Of Coherent Structure(Opinion / Topic Relevant Sentences)• Opinions and topic relevant sentences have aproperty of coherent structure.
– 76% of opinion sentences exist within three sentences.
– 80% of topic relevant sentences exist within one sentence.
Opinions

Topic

Distance is the numberof sentences between
opinion or topic relevant
sentence and the next
these sentence.
Ｐage 5

2008/12/19 NTCIR7

Sliding Window Framework• Utilize the surrounding sentences of the target sentence
as the coherent structure,• Judge the target sentence whether opinion-related or not, or
whether topic-related or not.
STEP1
S1

B1={S1,S2,S3}

S2

B2={S2,S3,S4}S3

B3={S3,S4,S5}

S4
S5

STEP2
BS1=F(B1)
BS2=F(B2)
BS3=F(B3)

STEP3
R1=G(BS1)
R2=G(BS1,BS2)
R3=G(BS1,BS2,BS3)
R4=G(BS2,BS3)
R5=G(BS3)

STEP1 Make blocks(B1,B2,B3) by sliding the window from sentence to sentence.
STEP2 Estimate the score of the relationship of each block by using predefined Function F.
STEP3 Judge whether each sentence should be extracted by using Function G with thescore of blocks that contain the target sentence.
Ｐage 6

2008/12/19 NTCIR7

Sub Task1:
Opinionated Sentence Extraction• Function F– Regression function that returns the number of
opinion sentences in the input block.
– Leaning by training dataset given by organizer withlinguistic features (details in next slides).

• Function G– G(S) = Yes if F(BSm)+ J +F(BSn) > win_size
No else– Judges the sentence as opinion if all blocks that
contain the target sentence have one or more opinion
sentences

Ｐage 7

2008/12/19 NTCIR7

Features At Function F
1. Original form, part of speech and surface string ofmorpheme
2. Semantic attribute of clause
3. Pair of semantic attributes of two clauses ofdependency relation
4. Whether or not character “「” and “」” are in the samesentence
5. Original form, part of speech and surface string ofmorpheme between “「” and “」”
6. Original form, part of speech and surface string ofmorpheme before “「”
7. Original form, part of speech and surface string ofmorpheme after “」”
Ｐage 8

2008/12/19 NTCIR7

Sub Task2:
Topic Relevant Sentence Extraction• Extend topic description by using web searchengine– Topic description given by organizer is poor.

• Function F– F(BSn) = Cosine similarity between extended topicand Block BSn.

• Function G– G(S) = Yes
NoＰage 9

2008/12/19 NTCIR7

if (F(BSn)+J+F(BSm))/win_size > Telse

Evaluation By Organizer
Opinion sentence: 2nd F-value
Topic relevant sentence: 1st F-value•Lenient dataset

Ｐage 10

2008/12/19 NTCIR7

Evaluation Of Sub Task1:
Opinion Sentence Extraction• Window size and accuracy
BaseLine”(ALL Y)” is the result in the case of all sentences as opinion.

Ｐage 11

2008/12/19 NTCIR7

Evaluation Of Sub Task 2:
Topic Relevant Sentence Extraction
Extended topic description and accuracy
BaseLine

Window size and accuracy
BaseLine

Ｐage 12

2008/12/19 NTCIR7

Discussion• Our framework (SWF) contributes to getting highrecall and F-value.
• However, the precision is low.
– This result as we had expected.
– Because our framework judges based on block andmistakes the non-opinion sentence between theopinion sentences as opinion sentence.

• To improve this, we will develop a method
making the correct judgment of non-related
sentences in the future.

Ｐage 13

2008/12/19 NTCIR7

Conclusions• Sliding Window Framework– New sentence extraction framework using coherent
structure.
– We applied our framework to the opinion sentence
extraction subtask and the topic relevant sentence
extraction subtask.

• Result of NTCIR7 MOAT task
– Opinion sentence extraction : second best
performance– Topic relevant sentence extraction : best performance• F-value under the lenient standard.

• High recall, but low precision.
Ｐage 14

2008/12/19 NTCIR7