Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

Incorporating Feature-based and Similarity-based Opinion
Mining – CTL in NTCIR-8 MOAT
Ruifeng Xu

1

1

Dept. of Chinese, Translation and Linguistics,
City University of Hong Kong
Kowloon, Hong Kong852-34422803

Chunyu Kit

Dept. of Chinese, Translation and Linguistics,
City University of Hong Kong
Kowloon, Hong Kong852-27889310

2

Dept. of Computer Science and Technology,
Harbin Institute of Technology, Shenzhen Campus,
Shenzhen, China

ctckit@cityu.edu.hk

ruifeng.xu@cityu.edu.hk
ABSTRACTThis paper presents the design and implementation of an opinion
mining system developed by NLPCity group for NTCIR-8 MOAT
evaluation, named CTL-OM. CTL-OM incorporates two opinion
mining approach, namely feature-based approach and similaritybased approach. The feature-based approach incorporates
computational features at punctuation-, word-, collocation-,phrase-, sentence-, paragraph- and document-level in a coarsefine multi-pass classification framework. The opinion holders andopinions targets in the opinionated sentences are then recognized.
The similarity-based approach works in a different way. Thisapproach estimates the similarity between the example sentences
and testing sentence and identifies the similar example sentencetesting sentence pair. The opinion components annotated in theexample sentence are utilized to recognize the corresponding
components in the testing sentence. The analysis outputs by these
two approaches are integrated to obtain the final opinion mining
results. CTL-OM achieved promising results in Traditional
Chinese and Simplified Chinese evaluation in MOAT-8,
respectively. This result shows that the incorporation of featurebased and similarity-based opinion mining approach is effective.

Categories and Subject Descriptors
I.2.7 [Natural Language Processing]: Text analysis

General Terms
Algorithms, Experimentation, Languages

Keywords
Opinion mining, feature-based opinion mining, similarity-based
opinion mining

1. INTRODUCTION
Opinion analysis technique, which identifies and analyzes the
subjective opinions and sentiments in text, attracts increasingly
research interest in recent years. This technique supports many
applications such as product review analysis and social news
monitoring. Furthermore, this technique promotes many natural
language processing tasks such as summarization and information
retrieval [1,10].

Various opinion analysis techniques were investigated. Most ofthem can be camped into three major types. The first one,knowledge-based type, mainly utilizes the opinion-related
linguistic knowledge to analyze the opinions [3]. The featurebased one is based on computational features and classifiers.
Machine learning algorithms, including both supervised learningand unsupervised/semi-supervised learning algorithms, are widelyadopted to improve the classifier [5]. The third one employs
semantic role labeling based on FrameNet in opinion analysis [2].
However, these techniques are designed for different purposes,
and their performances are evaluated based on different languages
and different text set. Thus, the evaluation of opinion analysis
technique has become a barrier to the research in this topic. The
NTCIR Multilingual Opinion Analysis Task (MOAT) series andformer Opinion Analysis Pilot Task provide an common
evaluation framework on the same datasets (including English,
Japanese, Simplified Chinese and Traditional Chinese text) [6,7,8].
In NTCIR-8 MOAT, there are five basic sub-tasks, including
identify the opinionated sentences, identify the topic-relevant
sentences, determine the polarity of opinionated sentences andrecognize the opinion holder and opinion targets.
This paper describes the system (ID: CTL) developed by the
NLPCity group in Dept. of Chinese, Translation and Linguistics,
City University of Hong Kong in NTCIR-8 MOAT. We
participated four of five subtasks, namely opinionated sentence
identification, opinion polarity determination, opinion holder
identification and opinion target recognition on both Traditional
Chinese and Simplified Chinese dataset. Our system, named CTLOM, incorporates feature-based and similarity-based opinion
analysis techniques. In which, the feature-based approachseparates the whole opinion analysis process to two stages, i.e.
classification-based opinionated sentence identification andpolarity determination, and extraction-based opinion holder andopinion target recognition. Firstly, computational features atpunctuation-, word-, collocation-, phrase-, sentence-, paragraphand document-level are incorporated in a coarse-fine multi-passclassification-based stage. The opinion holders and opinions
targets are then analyzed in the extraction-based stage. The
similarity-based approach works in a different way. This approach
estimates the similarity between the example sentences andtesting sentence. If a similar example-testing sentence pair isfound, the opinion analysis of the testing sentence is conductedwith the reference of annotated examples. It is in one stage. The― 276 ―

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

analysis results by two approaches are integrated to obtain the
final outputs. CTL-OM achieved promising results in Traditional
Chinese (ranked 2nd, 1st, 1st and 1st in four subtasks, respectively)
and Simplified Chinese (ranked 4th, 1st, 2nd and 1st, respectively)
evaluation, respectively. This result shows the incorporation offeature-based and similarity-based techniques is effective toenhance the opinion analysis system.
The rest of this paper is organized as follows. Section 2 analyzes
the components of opinion expression. Section 3 presents theframework of CTL-OM. Section 4 presents the feature-based
opinion analysis techniques and Section 5 presents the similaritybased techniques. Section 6 evaluates CTL-OM and finally,
Section 7 concludes this paper.

2. ANALYSIS OF OPINION EXPRESSION
Opinions are usually expressed in some conventional patterns.
Generally speaking, an opinion expression consists of five major
components: (1) opinion holder, which is the governor of an
opinion and normally refers to a person, a state or an organization
as well as the corresponding pronouns; (2) opinion object, whichis the target of the opinion. Most opinion objects are nouns, nounphrases or pronouns; (3) opinion word, which reflects the opinion
polarity, i.e. positive, neutral or negative; (4) opinion operator,
which is the verb indicating an opinion event; and (5) opinion
indicator, which is the word indicating the orientation of an
opinion or the orientation tendency of multiple opinions. In thereal txt, no all of the above components always appear in each
opinionated sentence.

practical reasons, this kind of words are always processed as
CDOWs.

3. FRAMEWORK DESIGNCTL-OM separates the four opinion analysis subtasks into twogroups:theclassification-based
opinionatedsentence
identification and polarity determination, and the extraction-based
opinion holder and opinion target recognition. CTL-OM consistsof two components. The first one is feature-based, which is an
improved version of our previous system adopted in NTCIR-7
MOAT [9]. Computational features at different levels are
extracted from testing sentences. These features are thenincorporated in a coarse-fine multi-pass classification system toidentify the opinionated sentence and determine their polarity.
Next, based on these features and syntax and rules, the opinion
holders and targets are recognized. The second component in
CTL-OM is based on similarity between example sentence andtesting sentence. If a similar example sentence corresponding to
the testing sentence is found, the testing sentence analysis isconducted based on the opinion component matching provided in
the annotated example sentence. This approach is obviously
different from the first one. The whole analysis process regards asentence as a whole of sequent words rather than separated
features in the feature-based opinion mining approach. The
analysis results of these two components are integrated togenerate the final output. The whole framework is illustrated in
Figure 1.

The opinion operators are the verbs which indicating opinion
expression. The typical opinion operators including 䄺ਞ(warning)ᔎ 䇗 (emphasize) and ᣛ ߎ (point out). It is noted that some
operators bring opinions such as ⿄䌲(praise).
Opinion indicators are mainly conjunctions, adverbs andadverbial phrases, including: (1) negation conjunctions, such as
Ԛᰃ(but, however) and ሑㅵ(Though), indicate that the sentimentof the following clause/sentence is different from the precedingone; (2) continual conjunctions, such as ᑋϨ, 㗠Ϩ(and) and ⡍߿ (especially), indicate that the sentiment of the following
clause/sentence is the same as the preceding one; (3) adverbs andadverbial phrases directly indicate the polarity of the opinionatedsentence, e.g. 䘫ធⱘᰃ (It is regrettable); and (4) verbs directly
indicate the polarity of the opinionated sentence.
Opinion words play a key role in opinion expression. They are
generally classified into: (1) context-free opinion word (CFOW)
whose polarity is constant irrespective of context, e.g. ᅠ 㕢(perfect) is absolute positive and ᙊ ࡷ (bad) is negative; (2)context-dependent opinion word (CDOW) whose polarity isdetermined by their context. For example, དュⱘ is positive
when it is used in the context of talk shows (meaning burlesque);but it is negative when it is used in the context of politics(meaning absurd); and (3) object-dependent opinion word
(ODOW) is the neutral word carrying different polarities whenassociated with different opinion objects. For example, 催(high)expresses positive sense when collocating with ᗻ㛑(performance)
but brings negative sense when collocating with ؎ࡵ(debt). For

Figure 1. Framework of CTL-OM.

4. FEATURE-BASED OPINION ANALYSIS
4.1 Feature Selection
Based on the observation and statistic analysis, a set ofcomputational features related to opinion analysis at different
levels are designed and selected.
Punctuation level features:
The presence of direct quote punctuation Āǋāand Āǌā
Word-level and entity-level features:― 277 ―

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

The presence of known opinion operators: E.g. 䫊㸠⿄⿄ℸ乍ᬓㄪৃҹ㓧㾷䌘䞥㋻ᓴ䯂乬 (Bank said this policy can ease thefinancial constraint)

positive, neutral, negative and non-opinionated) and the polarityof its previous sentences si-1, labeled as Pol(si-1), is positive. The
conditional probability,

P ( Pol ( si )The presence of known context-free opinion words/the percentage
of known context-free opinion words in sentence: E.g. ⓨߎ䴲ᐌᅠ㕢 (The performance is perfect)
The percentage of known opinion words in sentence/thepercentage of known opinion word in sentence: E.g. ཌྷ䇈, Ё೑೼䞡 㽕԰⫼ (She said that,ℸ⃵⋄೑ DNA Ẕ⌟Ꮉ԰Ёথ᣹њ䞡
China plays an important role in the DNA testing work in
Thailand)The presence of named entity/the percentage of named entity inthe sentence: E.g. ཌྷ䇈, Ё೑೼ℸ⃵⋄⋄೑ DNA Ẕ⌟Ꮉ԰Ёথ᣹њ䞡㽕԰⫼ (She said that, China plays an important role in the
DNA testing work in Thailand)
The presence of pronoun/the percentage of pronoun in thesentence: E.g. ཌྷ䇈,Ё೑೼ℸ⃵⋄೑ DNA Ẕ⌟Ꮉ԰Ёথ᣹њ䞡㽕԰⫼ (She said that, China plays an important role in the
DNA testing work in Thailand)
The presence of known opinion indicators: E.g. Ҫ⿄⿄䌲䖭乍ᡔᴃ
ৃҹᏺᴹᎼ໻ᬜⲞ (He praised this technique may bringing
tremendous benefits)
The presence of known degree adverbs: E.g. Ҫ⿄䌲䖭乍ᡔᴃৃҹ ᏺ ᴹ Ꮌ ໻ ᬜ Ⲟ (He praised this technique may bringing
tremendous benefits)
Collocation-level features:
The presence of collocations between named entities and opinion
operators: E.g. 䫊㸠 ⿄ ℸ乍ᬓㄪৃҹ㓧㾷䌘䞥㋻ᓴ䯂乬 (Banksaid this policy can ease the financial constraint)
The presence of collocations between pronouns and opinion
operators: E.g. ཌྷ䇈,Ё೑೼ℸ⃵⋄೑ DNA Ẕ⌟Ꮉ԰Ёথ᣹њ䞡㽕԰⫼ (She said that, China plays an important role in the
DNA testing work in Thailand)
The presence of collocations between nouns or named entities andᅠ㕢 (The performance is perfect)
opinion words: ⓨߎ䴲ᐌᅠThe presence of collocations between pronouns and opinion
Ӭ⾔ (He is very good)words: E.g. ҪᕜӬThe presence of collocations between degree adverbs and opinion
words: E.g. ҪᕜᕜӬ⾔ (He is very good)
The presence of collocations between degree adverbs and opinion
operators: E.g. Ё ೑ ᔎ ⚜ 䈈 䋷 䖭 ϔ 㔾 㸠 (China strongly
condemns this crime)
Sentence-level features:
The opinions of neighboring -2, -1, +1, and +2 sentences:
The transition probabilities from opinionated (or factual)/polarityof neighboring sentences to possible states of current sentences(opinionated or not, polarity):
For the i-th sentence in the document, labeled as si, we assume its
polarity, labeled as Pol(si), is positive, (its values including

P ( Pol ( si )positive | Pol ( si 1 )positive )positive  Pol ( si 1 )P ( Pol ( si ) positive )positive )(1)

can be estimated. The conditional probabilities of other polarity
co-occurrence combinations between si and si-1 are calculated in
the same way. Similarly, the conditional probabilities
corresponding to the co-occurrences with distance of two
sentences are estimated. The conditional probabilities obtained
from training corpus are used as transition probabilities in thisstage.
Paragraph-level features:
The opinionated/fractural probability of current paragraph:
The polarity of current paragraph:
Document-level features:
The opinionated/fractural probability of current document:
The polarity of current document:

4.2 PRE-PROCESSING
Word segmentation and Part-of-Speech tagging are indispensable
steps in Chinese sentence analysis. A Unicode-based word
segmentation and POS tagging system proposed in [4] areadopted. It is trained by using the Peking University People’s
Daily corpus and Sinica corpus, respectively. Thus, it can process
both Traditional Chinese and Simplified Chinese text in one
system. Furthermore, the named entities are recognized. The
identified named entities are helpful to recognize the opinion
holders and opinion targets.
Another task in pre-processing is to recall several pre-defined
types of omitted components in continuous sentences. Here, wegive an example.
Example Type 1: Multi-Sentences in Quotation
Sentence 1.ডᇍ‫߭ܮ‬ᣛߎˈĀ㢣જᠬᆊᮣ䋾∵ᚙ‫މ‬䴲ᐌϹ䞡Ǆ(Opposition party pointed out that " Suharto family’s corruption
is very serious.)
Sentence 2. ℸ໪ˈ㢣જᠬ೼݊㒳⊏ᳳ䯈⍝Ⴀ໮ᅫҎᴗᤳᆇџӊ Ǆ ā (In addition, Suharto suspected some cases of humanrights affection during his govern"
The holder of these two sentences are both ডᇍ‫(ܮ‬Opposition
party) while the holders does not appear in the second sentence.
This holder will be appended in the second sentences during
preprocessing stage. Such pre-processing is helpful to improve theopinion holder recognition.

4.3 Coarse-Fine Classification
Most existing opinion analysis techniques regard opinionatedsentence identification as one-step classification problem. The
linguistic features and computational features in the testing
sentence are utilized in the classifier to determine its opinion and― 278 ―

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japanpolarity. These techniques ignore the clues of opinions of theparagraph, document and the neighboring sentences to the opinionof current sentence. The observation on NTCIR-6 and NTCIR-7
opinion corpus shows that a sentence in a strong polarity
document has higher probability to be the same polarity while asentence in a factual document tends to be factual too. Naturally,
the paragraph-, document-level and neighboring sentence-level
opinions should be considered in the sentence opinion analysis.
Meanwhile, humans normally analyze the opinion trend of adocument coarsely in the first step and then remove theambiguities in sentence opinion based on the opinion of document
and neighboring sentences. It motivates the design of a coarsefine classification framework. This framework has multi-passcoarse-fine analysis which is described below.
Input: Document D consists of sentences S0, S1, Si …Sn
Step 1. Use the base classifier, Cbase, to analyze the opinion ofeach sentence in D. The output is the polarity value, Pol(Si). In
this step, a linear classifier is adopted. Meanwhile, the features atpunctuation-level, word/named entity level and collocation-levelare incorporated. Noted here that, all of the features adopted inthis step are inner-sentence ones.
Step 2. Estimate the polarity of D and each paragraph P.

Pol ( P)It frequently co-occurred with the entities in the query.

Some heuristics rules and patterns are applied to expand theopinion holder from its core. These manually complied rules andpatterns are relevant to punctuations, dependency, conjunctions,suffix, prefix and opinion operator, respectively. Furthermore,some syntax rules based on preposition phrase (related to opinion
holder identification) are adopted. Finally, the position of theopinion holder candidate in the sentence and the respective
position to the opinion operator candidate are considered as wellas their dependency.
The opinion targets are not always persons or name entities, they
may be nouns, phrases or clauses. For the opinion targets ofpersons or name entities, the recognition strategy is similar toopinion holder recognition. The corresponding heuristic rules andpatterns for opinion target are manually prepared. As for theclause opinion targets, its recognition is highly dependent on therecognition of the opinion operator. The opinion operator always
indicates the boundary of clause opinion target. Totally, we
manually prepared 22 rules and patterns for opinion targets ofpersons and 35 patterns for recognizing clause-level opinion
targets.

5. SIMILARITY-BASED OPINIONANALYSISn

Pol ( D)

6.

¦1
 Plo( Si )
n i 1

(2)

1 j
 ¦ Plo( Si )
j i1

(3)

Step 3. Use the improved classifier, Cim, to estimate the opinionof each sentence, Pol(Si)*. Cim incorporates paragraph-,document- and neighboring sentence-level opinions as newfeatures. Here, the transition probabilities are obtained fromtraining data.
Step 4. Update the document and paragraph opinion using
Pol(Si)*.
Step 5. If the sentence and document opinion mining output
converge, terminate. Otherwise, go to Step 3.
The output of this stage is the identified opinionated sentences
and their polarity.

4.4 Extraction-based Opinion Holder andTarget Recognition
Firstly, the following heuristics are used to recognize the core ofopinion holders:1.

It must be a recognized entity or pronoun.

2.

It must collocate and strongly associated with certain
identified opinion operators.

3.

It always occurs in the beginning of a sentence or near the
beginning or end of a quotation.

4.

It co-occurred with opinion operators with certain pattern.

5.

It frequently co-occurred with the topic words in the query

Based on the observation on the opinion corpus, we adopt a
hypothesis that in similar type of texts, opinionated sentences will
be more similar to other opinion sentences than to factual
sentences [10]. Especially, we found that, in two similar
opinionated sentences, their opinion holders and opinion targets
always have the same or similar places. These observations
motivate the development of similarity-based opinion analysis
technique. That is, for an input sentence, looking for a similar
example sentences in the annotated opinion corpus. The opinion
determination and opinion holder/opinion target recognition may
be done by matching the corresponding components in example
sentence to testing sentence.

5.1 Sentence Simplification
Generally speaking, the sentence similarity can be measured
based on shared words and phrases among two sentences.
However, there is a big barrier to apply similarity-based approachto opinion analysis that the size of annotated opinion corpus is notlarge enough. The sparseness problem affects the coverage ofsimilarity-based approach. Therefore, we must simplify both thetesting sentences and examples sentences.
Firstly, we simplify some words by assign a synonyms set id tosubstitute a group of synonyms. Such operation utilizes theknowledge in TongYiCi CiLin (A Chinese synonyms dictionary).
For example, ⚺ᙐ(trouble) have following synonyms ⚺䯋ǃ⚺ᖗ ǃ ㊳ ᖗǃ⚺х ǃ⚺䑕 . They are assigned id Syn147. The
following two sentences are then simplified as,೼៥Ӏ䖭Ͼᑈҷˈ᳝䆌໮⚺⚺ᙐ (There are a lot of troubles inthis age) ->೼៥Ӏ䖭Ͼᑈҷˈ᳝䆌໮ Syn147
䖭എ☿♒Փཌྷकߚ⚺⚺х (The fire quite upset her) ->䖭എ☿♒Փ
ཌྷकߚ Syn147― 279 ―

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

Secondly, we substitute frequent opinion operators, degree
adverbs and context-free opinion words (positive, negative) by thelabels of oper, dead, cfow(-p,-n), respectively. For example, ҪᕜᕜӬ⾔ (He is very good) –> Ҫ dead cfow-p.
Thirdly, we substitute several types of noun phrases, includingorganization names, country names, place name and person names,
etc. by the corresponding labels. For example, Ё೑ᔎ⚜䈈䋷䖭ϔ㔾㸠(China strongly condemns this crime) –> country_nameᔎ⚜䈈䋷䖭ϔ㔾㸠.
Finally, we apply a chunker to identify and substitute basequantifier phrases (e.g. ᭄गৡ thousands of->bqp), base timephrases (e.g. ᮽϞ˔ᯊ 8:00 in the morning->btp), base positionphrases (e.g. ‫ݙ‬㩭সϰ࣫䚼 North-east of Inner Mongolia->bpp)
in the sentences.
In this way, many sentences are simplified. Such an operation is
expected to enhance the coverage of common components during
sentence similarity estimation.

5.2 Sentence Similarity Estimation
We applied two methods to estimate the sentence similarity.
Firstly, we estimate the coverage of common words/simplifiedlabels between testing sentence and example sentence. Supposesthat sentences a and b have na and nb words/simplified labels,
respectively, in which m ones are common, their similarity,sim(a,b), is estimated by,Sim(aˈb)mna  nb

we adopt the opinion classification results of sentences in the final
output. The recognized opinion holder and opinion targets byfeature-based and similarity-based approach are combined togenerate final output, in which emphasizes the similarity-based
analysis results.

6. EVALUATIONS AND DISCUSSIONSCTL participate the evaluation on Traditional Chinese (TC) sideand Simplified Chinese (SC) side at NTCIR-8 MOAT. The
descriptions on the training data, testing data and gold standard
are given in the overview paper of NTCIR-8 MOAT [8]. We
participate four subtasks, including, opinionated sentence
identification, opinion polarity determination, opinion holder andopinion target recognition.
The organizer evaluates theperformance of the systems by using precision (P), recall (R) andF-measure (F).
First, the opinionated sentence identification subtask is evaluated.
The performances achieved by CTL-OM on TC and SC side are
given in Table 1, respectively, as well as their ranks. For
reference, the performances of the system with highest F are listed.
Table 1. The evaluation on opinionated sentence identification(4)

Secondly, we estimate the edit distance between two sentences,which is the lowest operation cost for varying sentence a tosentence b through substitution, insertion and deletion operations.
Currently, we set the cost for each of above three operations as
equal.
The two similarities are incorporated to generate final sentence
similarity. The sentences pairs having similarity greater than aempirical threshold are retrieved.

5.3 Opinion Analysis based on Similar
Sentences
Once a similar example sentence corresponding to testing
sentence is found, the testing sentence is analyzed by using theinformation annotated on the example sentence. If the similar
example sentence is an opinionated one, the same polarity will beassigned to the testing sentence. Meanwhile, the words with same
part-of-speech/simplified labels at similar positions in theexample-testing sentence pair are matched. The components inthe testing sentences corresponding to the annotated opinion
holder and opinion target in the example sentence are recognized
as the opinion holder and opinion target of the testing sentence.

5.4 Combine Feature-based and Similaritybased Analysis Results

P

R

F

Rank

CTL

65.14

68.79

66.92

2nd

Best

56.39

85.71

68.05

1st

CTL

36.46

78.90

49.87

4th

Best

41.34

83.35

55.27

1st

TC

SC

It is shown that the CTL-OM’s performance on this subtask is notsatisfactory, especially on the SC side. CTL achieves highest
precision on TC side, i.e. 0.6514, which is obviously higher than
the second-highest precision, 0.5639, achieved by CityUHK
group. However, the recall is much lower. It shows that oursystem should be revised to make a trade-off between precisionand recall. As for the SC side, the achieved precision is obviously
lower than TC side. It partially attributes to the annotation
disagreement on the gold answer between SC and TC. Especially,
we found that the percentage of annotated opinionated sentence
on TC side is nearly the double of SC side. CTL-OM is Unicodebased which analyzes SC and TC text in a same system andcombined training data. Thus, the achieved performances on TC
and SC sides are obviously.
Second, the polarity determination is evaluated. The performances
achieved by CTL-OM and by reference system are given in Table
2, respectively.
It is shown CTL achieves both the highest precision and the best
F both on TC and SC side. This result benefits from the coarsefine opinion classification which incorporates both inner- andinter-sentence features and coarse-fine multi-pass classification
strategy.

Considering that the similarity-based analysis approach has theadvantages on opinion holder and opinion target recognition but
relative low performance on opinionated sentence identification,― 280 ―

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

stages, and similarity–based approach, which performs four
subtasks in one stage, in opinion mining. The analysis
results by different approach are combined to enhance theperformance of the final output. The evaluations on
Traditional Chinese side and Simplified Chinese side in
NTCIR-8 MOAT show the effectiveness of the proposed
opinion analysis framework, especially on the opinion
holder/opinion target recognition. The future researches
will focus on the improvement on the two individual
analysis approaches and better integration strategy.

Table 2. The evaluation on polarity determination
P

R

F

Rank

CTL

76.50

53.06

62.66

1st

Best

76.50

53.06

62.66

1st

CTL

67.39

52.90

59.27

1st

Best

67.39

52.90

59.27

1st

TC

SC

8. ACKNOWLEDGMENTS
Finally, the performance on opinion holder and opinion target
recognition are evaluated. The performances achieved by CTLOM and by reference system are given in Table 3 and Table 4,
respectively.

R

F

Rank

CTL

84.90

-

-

1st

Best

84.90

-

-

1st

CTL

95.30

73.20

82.80

1st

Best

87.70

79.20

83.20

2nd

[2] Kim, S.M. and Hovy, E. 2006. Extracting Opinions, Opinion
Holders, and Topics Expressed in Online News Media Text,
In Proc. ACL/COLING Workshop on Sentiment andSubjectivity in Text, pp. 1-8, Sydney, Australia

TC[3] Ku, L.W., Wu T.H., Lee, L.Y. and Chen, H.H. 2005.
Construction of an Evaluation Corpus for Opinion Extraction,
In Proc. of NTCIR-5, pp.513-520, JapanSC[4] Lu, Q., Chan, S. T., Xu, R. F. et al. 2003. A Unicode based
Adaptive Segmentor. In Proceedings of the 2nd SIGHAN
Workshop on Chinese Language Processing at ACL 2003, pp.
164-167, Spain[5] Pang, B., Lee, L. and Vaithyanathan S. 2002. Thumbs up?
Sentiment Classification using Machine Learning
Techniques. In Proc. EMNLP 2002, pp.79–86.

Table 4. The evaluation on opinion target recognition
P

R

F

Rank

CTL

54.40

-

-

1st

Best

54.40

-

-

1st

CTL

73.50

56.40

63.80

1st

Best

73.50

56.40

63.80

1st

9. REFERENCES[1] Hu, M.Q. and Liu, B. 2006. Opinion Extraction andSummarization on the Web. In Proc. Of NCAI06, Boston

Table 3. The evaluation on opinion holder recognition
P

This paper is supported by a post-doctoral research fellowshipfunded by City University of Hong Kong, Hong Kong.

[6] Seki, Y., Evans D.K., et al. 2007. Overview of Opinion
Analysis Pilot Task at NTCIR-6. In Proc. of NTCIR-6
Workshop. Japan.

TC[7] Seki, Y., Evans D.K., et al. 2008. Overview of Multilingual
Opinion Analysis Task at NTCIR-7. In Proc. of NTCIR-7
Workshop. Japan.

SC[8] Seki, Y., Ku, L.W., et al. 2010. Overview of Multilingual
Opinion Analysis Task at NTCIR-8. In Proc. of NTCIR-8
Workshop. Japan.

It shows that CTL-OM achieves encouraging performance on
opinion holder and opinion target recognition. The results indicatethat incorporation of feature-based and similarity-based
techniques are effective.

7. CONCLUSIONSIn this paper, we present the CTL-OM system developed for[9] Xu, R.F., Wong, K.F. and Xia, Y.Q. 2008. Coarse-Fine
Opinion Mining – WIA in NTCIR-7 MOAT Task, In Proc.
of NTCIR-7 Workshop. Japan.
[10] Yu, H. and Hatzivassiloglou, V. 2003. Towards Answering
Opinion Question: Separating Facts from Opinions and
Identifying the Polarity of Opinion Sentences, In EMNLP
2003

NTCIR-8 MOAT task. This system incorporates featurebased coarse-fine classification and analysis approach,which separate opinionated sentence identification/polarity
determination and opinion holder /opinion target in two― 281 ―