A Multilingual Polarity Classification Method Using
Multi-Label Classification Technique Based on Corpus Analysis
Yohei Seki
T o y o h a shi University of Technology (Visiting in Columbia University)
seki@ics.tut.ac.jp

Research Goal:Clarify Effective Polarity
Classification Clues and Methods

Polarity Classification Using
Multi-label Classification Techniques

We implemented two multi-label classification techniques.
In NTCIR-7 MOAT, we have new challengs as follows:
1. The first one was a voting approach with three SVM
・Participants could use NTCIR-6 OAT corpus for training.
・Many participants focused on language portable approaches. classifiers. The features selected were used for eachclassifier. This approach was implemented using SVMlight.
Based on these, our research goal in NTCIR-7 is as follows:
2. Another one was a multi-label classifier using Mulan
・Using the features that were acquired from the significance (label power set classifier) system. Note that we could nottest in NTCIR-6 OAT and MPQA corpora, we estimated thedifferentiate the feature sets according to three polarity
effectiveness in opinion detection and polarity classification.
types in this classifier, so we combined them into one
・In NTCIR-7 MOAT, polarity classification was the problemfeature set.
to classify three lables: positive, negative, or neutral. We
compared two multi-label classification techniques:A.) SVM voting and B.) Mulan (label power set classification) . ・For polarity classification in Table 1, the results using

Experimental Results in NTCIR-7

System Overview

・My system overview is shown in Figure 1. The common
architecture was implemented both in Japanese and English.
・The polarity classification, opinion holder identification,and relevance judgment module were based on the resultsfrom opinion detection module.
・The opinion detection system was based on thedifferentiation of author opinions and authority opinions [1].
LangJJJJEEEE
EER unID
1212123123L /SL
L
SSL
L
L
S
S
S

SVM voting approach were shown as RunID 1 and theresult using Mulan classifier was shown as RunID 2 inJapanese and as RunID 3 in English.
・The results of SVM voting approach were better than
the results of Mulan. Note that SVM approach need totune cost parameters according to each classifier and wetuned them by using sample data in NTCIR-7 MOAT,
but we did not tune any parameters in Mulan.

Table 1. Evaluation Results in NTCIR-7 MOAT

O pinionatedPRF
0.6742 0.5620 0.6130--- (same in TUT-1)
0.5416 0.6199 0.5781--- (same in TUT-1)
0.3185 0.4092 0.3582
0.3282 0.2562 0.2878--- (same in TUT-1)
0.0961 0.4149 0.1561
0.1039 0.2724 0.1504--- (same in TUT-1)

R elevance
PRF
0.5527 0.2925 0.3825--- (same in TUT-1)
0.3062 0.3357 0.3203--- (same in TUT-1)
0.2092 0.1755 0.1909
0.1647 0.1136 0.1344--- (same in TUT-1)
0.0740 0.1853 0.1057
0.0615 0.1220 0.0817--- (same in TUT-1)

P
0.4596
0.42830.48060.45350.19430.1896
0.1621
0.05690.04840.0359P olarity
R
0.21400.19940.24170.22810.18300.11420.15270.21800.11850.1374

F
0.29200.27210.32160.30350.18850.14250.15730.09030.0687
0.0569PO pinion H older
RF

0.39230.28330.3290(0.3656) (0.1689) (0.2311)--- (same in TUT-1)
0.12500.28290.1735(0.1257) (0.1821) (0.1487)--- (same in TUT-1)

Discussion:
Improve Multilabel classification?
・We concluded the reason why SVM results were over than
Mulan results was that we could not discriminate the featuresets each by polarity type in Mulan.
Figure 1. System Flow Overview
・We also investigated a confusion matrix from SVM voting
and Mulan as in Table 2.
・You could confirm that the results using Mulan classifier
were sometimes better than the results using SVM classifier,・The features in opinion detection and polarity classification for example, negative classifier in Japanese.
both in Japanese and English were selected based on χ2・In future, we plan to implement Multi-label classification
test (the significance probability was 5%, two-sided test) on technique to discriminate three polarity types as inputs.
NTCIR-6 OAT and MPQA corpora.
Table 2. Confusion matrix with SVM voting
In polarity classification case, if a feature appeared moreand Mulan approaches
frequently in the sentence with one polarity type thanA ssessm ent (Lenient)Lang
M ethodP os
N eg
N eu
sentences with other two polarity types, it was selected.
P os
15
351
SV MN eg
966349To avoid the error from low frequency data, we onlyvoting
N eu
1852329
(N o)
63
173
788
J
investigated the features which appeared more than five
P os
15
12105
S N eg1689346
M ulantimes in the NTCIR-6 OAT corpus.
y N eu1120278s (N o)63
173
788
・As feature types, we focused on a.) terms or term types
t P os18304e N eg
SV M6413618through the abstraction using thesaurus (Bunrui-Goi Hyom N euvoting
25373(N o)
165
31840
Eor WordNet) or lexicon (General Inquirer, Wiebe’s subjective
P os18172N eg49
102
12M ulan
lexicon, Hatsivassiloglou’s adjective entries etc.) and b.)
N eu
40
8411(N o)
165
318
40
syntactic pairs using dependency parsers (Minipar, Cabocha).
・In Japanese, we utilized grammatical subjects, action
semantic primitives, syntactic pairs, and keywords as feature[1] Y. Seki, N. Kando, and M. Aono: Multilingual Opiniontypes.
Holder Identification Using Author and Authority Viewpoints,
・In English, we utilized subjective term lexicons and two
Information Processing & Management(to appear)
syntactic pairs such as “subject”-“verb” and“auxiliary verb”-“verb” relationship using Minipar parser.
The Seventh NTCIR Workshop Meeting
・The selected features are shown in Tables 3-6 in the paper.
(NTCIR-7)
December 16-19, 2008. NII, Tokyo, Japan

Feature Selection with Corpus Analysis

Reference