Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, JapanAn information extraction method for multiple data
sources
Hironori Mizuguchi and Dai Kusui
Information and Media Processing Laboratories, NEC Corporation
8916-47 Takayama-cho Ikomashi, Nara 631-0027, Japan+81-743-72-3680{hironori@ab, kusui@ct} .jp.nec.com
ABSTRACTWe developed a method of information extraction for multiple
data sources or for various kinds of datasets like Internet web
pages. Generally, because many different writing styles or vocabularies exist among different kinds of data, the accuracy of
information extraction using various kinds of datasets is notbetter than that using a single kind of data. Our method divides
the data by clustering and learns extraction rules to increase
accuracy even if we use various kinds of datasets. In our experiment, we applied our method to a NTCIR8 Technical Trend
Map Creation subtask that uses two kinds of data, patents andtechnical papers, and obtained the better precision than normal
information extraction method.

Categories and Subject Descriptors
I.2.7 [ARTIFICIAL INTELLIGENCE]: Natural Language
Processing – Text analysis; H.3.1 [INFORMATION STORAGE AND RETRIEVAL]: Content Analysis and Indexing –Linguistic processing

General Terms
Algorithms, Experimentation

Keywords
Entity extraction, Clustering, Machine learning

1. INTRODUCTION
Recently, effective uses are expected of a great deal of text
information such as Web pages and company documents. In
these texts, many words have specific meanings (called semantic classes) such as a person’s name, a location’s name, an organization’s name, and so on. By extracting these words, we
can effectively use the text information on a Q and A systemand for text categorization, machine translation, and so on. For
example, we can automatically create a technical trend map
with two axes for technologies and effects by extracting technology names and their effects from patents and technical papers. By looking at the trend map, we can recognize what technology exists and how it is related to which effect.

This method is commonly used with a single kind of data inwhich the neighbor words are similar because the writing styleof the same type of data is similar. For neighbor words that are
not similar, prior method cannot create good rules and cannot
get good accuracy. Table 1 shows an example of prior methodwith multiple kinds of data (patents and technical papers) in a
Technical Trend Map Creation subtask in NTCIT8. The firstcolumn denotes the kinds of training datasets: only paper, only
patents, or a combination. The second column denotes the kindof testing dataset. The rest of the columns show the macroaverages of the extraction semantic classes, TECHNOLOGY,ATTRIBUTE, and VALUE. The precision, recall, and f-valueof the mixed data in the training data are lower than those of thesingle kinds.
In recent years, the utilization of information that contains
such various kinds of data as internet web pages or intranet
enterprise documents is expected to increase. In the Technical
Trend Map Creation subtask of NTCIR8, the kind of each data
was already given. However, to use internet or intranet data, we
do not know the kind of each piece of data or how the dataset
should be separated.
We researched an information extracting method that getsgood accuracy even if the data contains various kinds of data.
Our clustered learning method makes clusters from training
data and rules from each clustered training data using machinelearning techniques. Our method can divide problem spaces likekinds of data by clustering and creates rules for each problem
space. However, the clustered training data by only clusteringhas bias: the amount of training data in each cluster and thedensity in each problem space. To limit these effects, our method modifies the data of clusters. Specifically, the data in large
and high density clusters are moved to another cluster.
In this paper, first, we describe an abstract of the Technical
Trend Map Creation subtask of NTCIR8. Next, we explain ourclustered learning method and show experiment and subtask
results. Finally, we describe the discussions.

Table 1. Comparison accuracy between single and multiple
kinds of training data

Named Entity Resolution extracts words that have semantic
classes. Yamada’s very popular method [7] creates rules thatrecognize such words by a machine learning technique that uses
training data annotated with semantic classes. This method is
divided into two phases: learning and applying. In the learning
phase, based on the assumption that identical semantic classwords have similar neighbor words, machine learning creates
rules that sort a word into positive and negative using the annotated words and the neighbor words as a positive example andthe un-annotated words and neighbors as a negative example. In
the apply phase, the method decides whether an input word is asemantic class by using the rules.

― 348 ―

Training

Test

Precision

Recall

F

Paper

Paper

51.97

23.64

32.40

Paper + Patent

Paper

45.62

18.07

25.75

PatentPatent

66.24

40.68

50.35

Paper + Patent

Patent

63.54

36.65

45.95

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

2. TECHNICAL TREND MAP CREATIONSUBTASK

3.1 Clustering
Our method divides training datasets into clusters by a clustering technique. A clustering target is a document or the context. The context means the information of each word and its
neighbor words. For document clustering, the clusters denote
kinds of data. For context clustering, the clusters denote kindsof words.

The purpose of this task is extracting technology names andthe effects that use the axes of a technical trend map from technical papers and patents [4]. The following are the practicalsemantic classes:zzzz

TECHNOLOGY: algorithms, tools, materials, or data
used in each paper or patent
EFFECT: ATTRIBUTE and VALUE pairs
ATTRIBUTE: an attribute in the effect of a technology
VALUE: values related to attributes in an effectDocument clustering makes clusters from each document in
the training datasets. Each document is represented as a document vector that consists of the word whose part of speech isnoun, verb, adjective, adjectival verb or adverbs and the numberof words in the document.

A task organizer distributes the training dataset, and participants develop systems that extract these semantic classes.

ⵝ⟎䈲䇮<TECHNOLOGY>䊌䊥䊁䉞䊂䊷䉺</TECHNOLOGY>䉕ᜬ䈧䈖䈫䈮䉋䉍<EFFECT><ATTRIBUTE>৻ቯᢙ䈱⸥ᙘᇦ૕䈱㓚ኂ
</ATTRIBUTE>䉕<VALUE>ୃᓳ</VALUE></EFFECT>䈜䉎䈖䈫䈏น⢻䈪䈅䉎䇯

Figure 1. Sample data of Technical Trend Map Creationsubtask

3. CLUSTERED LEARNING METHOD
Our clustered learning method recognizes text’s semantic
classes by using rules learned by machine learning techniques.
It divides training data into clusters and makes rules from eachclustered training data. However, clustered training data havesome bias: the amount of training data in each cluster and thedensity in each problem space. To limit these effects, our method modifies the data of clusters.
Our method divides problem spaces like data kinds by clustering and creates rules of each problem space. Therefore, fortraining data that contain various kinds of data, our method can
get good accuracy.
Figure 2 shows an overview of our clustered learning methodthat is divided into two phases, learning and applying. In thelearning phase, the training data is divided into clusters. Next,
the clustered training data are modified to limit biases. Then,rules are made from each clustered training data. In the applying phase, a specific cluster is selected that resembles test dataas an input. Rules related to the selected clusters are applied torecognize each word’s semantic class.

Training
Training
Training
Data

Test Data

Clustering

Select Cluster

Context clustering makes clusters from each appearance of
all words in the documents in the training dataset. Each appearance of words is represented as a feature vector that consists of
the word’s grammatical information and its neighbor words.
These features are mentioned below in this section.
Many clustering methods are available, including k-means[6], pLSI [3], and so on. K-means with cosine similarity is suitable for document clustering because each document vector has
a lower dimension number than the feature vector’s one in content clustering. pLSI is better for context clustering because it
can reduce the dimensions of vectors; pLSI can handle highdimensional data like feature vectors.

3.2 Modification of clusters
The purpose of the modification of clustered data is to limit
the effects of biases and to generalize each problem space defined by each cluster. The biases of the clustered training dataset are the amount of training data in each cluster and the density in each problem space. These biases, which harm machinelearning, and their effects are described in more detail as follows.
Figure 3 shows a conceptual figure of modification clusters.
The small circles, the big circles, and the x marks denote thevectors related to each piece of data, the clusters, and the centers of each cluster, respectively. First we considered the clustered training data, as shown on the right side. In this case, sincemost vectors in each cluster are close to the center of each cluster, our method cannot recognize a word located outside ofthese clusters, so a set of rules in these clusters does not havegood generalization performance. Moreover, the amount of data
in cluster c is smaller than the other clusters. Thus, each rule ineach cluster has different performance.
To limit the effects from these biases, our method modifies
the data in the clusters by moving them among clusters. Suchmodification averages the amount and the density of each cluster.
First, AmountBias and SimBias are calculated as follows:

CL1

CL2

Modify cluster

Rule-CL2ninApply ruleLearning

Rule-CL1

AmountBias (C i )CL3

Tagged Data

SimBias (C i )Rule-CL3

Figure 2. Overview of clustered learning method― 349 ―1 ni
¦ (1  sim( xci , xij )) 2n j1
1 k ni¦¦ (1  sim( xck , xkj )) 2
N k1j1

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

In the equation, C i is the ith cluster, ni is the number ofvectors in the ith cluster, and n is the average of the number ofvectors in each cluster. Therefore, AmountBias is the ratio between the number of vectors in C i and the average. x ci is a
center vector of C i . xij is an jth vector in C i . sim( x, y ) is
the similarity between x and y. Therefore, SimBias is the ratio
between the mean square of the dissimilarity between the centerand each vector.
A cluster is selected whose AmountBias is higher than a particular threshold and whose SimBias is lower than a particularthreshold. The vector in the selected cluster is moved to anothercluster whose center vector resembles the vector.
These steps are repeated until all clusters satisfy theAmountBias and SimBias thresholds.

Cluster b

Cluster b

Cluster aplain formzpart of speechzsemantic label from language analysis

We used bisecting K-means [6] as the only clustering method
and CLUTO [2] for implementation. A cluster’s center is medoid calculated by CLUTO as the highest z-score. A vector’s zscore is the difference between the average similarity of thevector in its cluster and the averaged similarities of all vectors
in the same cluster.

4.1 Experiment 1: Formal run

By using each bit of clustered training data, machine learning
makes recognition rules. Each rule is made from each cluster.
The learner creates rules that sort a word into positive and negative using the annotated word and the context as a positive example and the un-annotated word and the context as a negative
example. The context contains the features of the target word,
its two neighbor words, and words that have a modification
relationship of the target and neighbor words. Practical features
include:z

Before describing the formal run results, we show some conditions.

Language analysis was done by Jana [5], which is developedand studied by the NEC corp. .

3.3 Learning rules

character string

To evaluate our method, we did experiments using the Technical Trend Map Creation subtask data. First, we show the result of a formal run. Next, we represent the effects of clustering.
Due to time limitations, we couldn’t experiment on the modification of clusters. We want to represent the effect of the modification of clusters near future.

SVM is the machine learning technique, and we use libSVM[1] for its implementation with linear kernel and default parameters.

Cluster a

Figure 3. Modification of clustersz

4. EXPERIMENTS AND RESULTS

The rule creation method makes rules that recognize whether
it is a semantic class. Rules cannot recognize the range of semantic classes like BIO tags. We make four rules in each cluster for each semantic class: TECHNOLOGY, ATTRIBUTE,
VALUE, and OTHER. If a semantic class expression consistsof multiple words, rules can only recognize its last word.

Cluster c

Cluster c

Then the rules, which are related to the selected cluster, areapplied to the inputted word.

We applied our method to a Technical Trend Map Creationsubtask. Due to time limitations, we couldn’t use clustering andmodification.
Additionally, rules can only recognize the last word of thesemantic classes of TECHNOLOGY, ATTRIBUTE, and
VALUE. Therefore, we have to take a chunk from each semantic class word and make an EFFECT class. We make a chunk of
TECHNOLOGY and ATTRIBUTE as a sequence of the modification relation words of the last word. We make an EFFECT
class as a pair of an ATTRIBUTE class and a VALUE classthat are located within two words.
Table 2 shows our result of a formal run (In [4], our run id is‘ONT’).

Additionally, there are some practical rule creation methods,
such as rules that only recognize the semantic class of eachword or recognize the semantic class and its range like the BIO
method [7]. We can choose any method.

3.4 Applying phase
First, a cluster is selected that contains rules applied to input
words. The similarity between the inputted document or context
and the centers in each cluster is calculated. Then the cluster isselected with the highest similarity. If the cluster is made fromdocument vectors at the learning phase, then the similarity between the documents is calculated. If the cluster is made fromcontext vectors at the learning phase, then the similarity between contexts is calculated. If the score is 0, our method does
not apply rules because we cannot select clusters.

― 350 ―

Table 2. Results of formal run
㪢㪠㪥㪛

㪫㪘㪞
㪩㪜㪚㪘㪣㪣 㪧㪩㪜㪚㪠㪪㪠㪦㪥 㪝㪫㪜㪚㪟㪥㪦㪣㪦㪞㪰㪐㪅㪈㪉㪈㪅㪐㪘㪫㪫㪩㪠㪙㪬㪫㪜
㪏㪅㪈
㪈㪌㪅㪋
㪧㪘㪧㪜㪩
㪭㪘㪣㪬㪜
㪈㪉㪅㪉
㪉㪍㪅㪎
㪜㪝㪝㪜㪚㪫
㪉㪅㪎㪈㪏㪅㪉㪫㪜㪚㪟㪥㪦㪣㪦㪞㪰㪋㪅㪎㪏㪅㪇㪘㪫㪫㪩㪠㪙㪬㪫㪜㪉㪈㪅㪐㪉㪐㪅㪍㪧㪘㪫㪜㪥㪫
㪭㪘㪣㪬㪜
㪊㪊㪅㪏
㪌㪇㪅㪊
㪜㪝㪝㪜㪚㪫
㪈㪉㪅㪌
㪊㪊㪅㪐

㪈㪉㪅㪐
㪈㪇㪅㪍
㪈㪍㪅㪏
㪋㪅㪎
㪍㪅㪇
㪉㪌㪅㪉
㪋㪇㪅㪋
㪈㪏㪅㪉

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

4.2 Experiment 2: Effect of clustering
Figure 4 shows the precisions and recalls of the document
and context clustering of the mixed training data in the learning
phase. In Figure 4, each line denotes different test data anddifferent
clustering
method.
For
example,‘MIX_PATENT_DOC’ shows results of PATENT test data anddocument clustering with mixed training data. The precisionand recall are the macro-averages of TECHNOLOGY, ATTRIBUTE, and VALUE by 10-fold cross-validation.

Table 3. Comparison of each method (single kind (PATENT
or PAPER), MIX(CLST=2) that use clustered learning method with two document clusters and MIX that uses normal
mix training data)

TRAINING TEST PREC RECALL FPATENTPATENT 66.24% 40.68% 50.35%
MIX(CLST=2) PATENT 63.89% 33.80% 43.75%
MIX
PATENT 63.54% 36.65% 45.95%
PAPER
PAPER 51.97% 23.64% 32.40%
MIX(CLST=2) PAPER 52.67% 15.42% 23.74%
MIX
PAPER 45.62% 18.07% 25.75%

Additionally, we measured whether the last word of eachclass is correct. Since we can compare the single and combined
kinds, the training dataset includes 500 documents: 250 frompapers and 250 from patents.
PrecisionzzDocumentclustering(MIX_PATENT_DOC,
MIX_PAPER_DOC): when the number of clusters is two,the precision outperformed the normal (0 clusters). The
precision of paper test data with two document clusters
showed a dramatic increase as compared with The precision of paper test data without document clustering.
Context
clustering(MIX_PATENT_CON,
MIX_PAPER_CON): There were few changes.

Recallz

The recalls of all tests decreased by increasing the numberof clusters. The rules became more specific by increasing
the clusters.

Table 3 shows the results of the single kind (patent or paper)
training data, the mixed training data with document clustering(two clusters), and the mixed training data without clustering.
The precision of mixed training data with clustering and patentor paper test data is higher than the precision of mixed training
data without clustering and patent or paper test data, respectively. Especially, the precision of mixed training data withclustering and paper test data is higher than the precision of
paper training data and paper test data. But recall is down.

5. DISCUSSIONS
5.1 Cluster and the effect
Next we discuss the results of document and context clustering.
The document clustering results in the training phase areshown at Table 4 with the number of clusters (2, 3, 4), the cluster id, and the number of documents for each kind in the cluster.
We got satisfactory results that met our expectations because
we sorted the training documents into different kinds. This is
one of the factors of good effects.
The context clustering results in the training phase are shownin Table 5 with the number of contexts in each cluster and in
Table 6 with a representative example of contexts in each cluster. There is no consistency of contexts in the same cluster. Thisis one of the reasons of poor results in context clustering in
Figure 4.
One cause of this result is the clustering method (K-means).
The number of dimensions of the feature vector used by context
clustering is huge (760,000). Therefore, we should use methods
that can compress huge dimensions, like pLSA.

RECALLPRECISION
70.00%

40.00%

65.00%

35.00%

60.00%

30.00%
MIX_PATENT_DOC

55.00%

MIX_PATENT_CON
50.00%

MIX_PAPER_DOC
MIX_PAPER_CON

45.00%

25.00%
20.00%
15.00%
10.00%

40.00%

5.00%

35.00%
0

2

3

4

5

6

7

8

9

0

10

2

3

4

5

6

Number of clusters

Number of clusters

Figure 4. Result of clustered learning method in all clusters― 351 ―

7

8

9

10

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, Japan

5.2 Amount of training data in each kind
Table 4. Amount of data in each document cluster
CLST

ID
0
1
0
1
2
0
1
2
3

2
3

4

PAPER PATENT SUM
5
209 214
173
2 175
1
80
79
94
1
95
5
209 2143
124 127
2
85
87
79
1
80
94
1
95

Table 7 shows the number of words and the frequencies of allwords in each kind of training dataset in Experiment 2. The
patent’s training data is bigger than the paper’s training data
because its frequency per word is bigger. The normal mixed
method is lower than our method because the patent effect isbigger than the paper effect. Since our method can divide training data into different kinds, the precision of paper is especiallygood.

Table 7. Number of words and frequency of all words ineach kind of data
㪚㪣㪘㪪㪪
㪮㪦㪩㪛㪪
㪘㪫㪫㪩
㪝㪩㪜㪨
㪮㪦㪩㪛㪪
㪫㪜㪚㪟
㪝㪩㪜㪨
㪮㪦㪩㪛㪪
㪭㪘㪣㪬㪜
㪝㪩㪜㪨
㪮㪦㪩㪛㪪
㪦㪫㪟㪜㪩
㪝㪩㪜㪨

Table 5. Amount of data in each context cluster

CLST ID
0
1
0
1
2
0
1
2
3

2
3

4

PAPER PATENT SUM
8322 19370 27692
8137 16683 24820
1852 3295 5147
8322 19370 27692
6285 13388 19673
1852 3295 5147
3218 7472 10690
5104 11898 17002
6285 13388 19673

Table 6. Example context in each context cluster㪚㪣㪪㪫 㪠㪛
0
2
1

0

3

1

2

0

1
4
2

3

㪚㪦㪥㪫㪜㪯㪫
೙ᓮ䉕⑌ᱛ䈚䈢䉍ᝄ䉏㔚૏䈮ᔕ╵䈚䈩ዉㅢ⁁ᘒ䈫ᧄ⊒᣿䈮䉋䉎䈫቟ଔ䈭⸳ቯ䈜䉎᭴ᚑ䈮䉋䉍╙㪈෸䈶╙㪉䈱⸥ᙘ
䈭䉎䇯
࿑䉎䇯
⥋䉎䇯
䈭䉎䇯
೙ᓮ䉕⑌ᱛ䈚䈢䉍ᝄ䉏㔚૏䈮ᔕ╵䈚䈩ዉㅢ⁁ᘒ䈫ᧄ⊒᣿䈮䉋䉎䈫቟ଔᨵエ䈭⚵ว䈞䈱ᣇᴺ
⸳ቯ䈜䉎᭴ᚑ䈮䉋䉍ශ೚ᯏ᭴ㇱ䈮ଐ㗬䈪䈐䉎ᒻᑼ䈮
࿑䉎䇯
⥋䉎䇯
䈭䉎䇯
೙ᓮ䉕⑌ᱛ䈚䈢䉍ᝄ䉏ዊဳൻ䉕㆐ᚑ䈜䉎䈖䈫䈏䈪䈐䉎䇯✢䈮䉅䈭䉎䇯
േ૞䈱࿁ᢙ䈮䈢䈇䈚⊒᣿䈱ലᨐ䇽
┵ሶὐ䈱ૐਅ
⸳ቯ䈜䉎᭴ᚑ䈮䉋䉍╙㪈෸䈶╙㪉䈱⸥ᙘ䈭䉎䇯㪧㪘㪧㪜㪩 㪧㪘㪫㪜㪥㪫
㪉㪉㪎
㪊㪏㪋
㪉㪏㪏
㪍㪈㪊
㪉㪌㪈
㪊㪇㪇
㪋㪉㪉
㪏㪐㪐
㪈㪎㪐
㪉㪊㪐
㪉㪎㪐
㪌㪐㪌
㪌㪇㪐㪋
㪌㪎㪇㪈
㪈㪎㪉㪊㪋 㪊㪏㪋㪐㪈

5.3 Selection of cluster at applying phase
First, we discuss the effect of the similarity threshold used by
selecting the cluster phase. Figure 5 shows the precision andrecall of document clustering at each similarity threshold withtwo clusters. The precision increases until the threshold reaches
0.15. This means that the data near the center of each cluster are
correct. However, when the threshold exceeds 0.15, the precision and recall decrease because our method cannot select acluster. How to decide a good threshold is a problem.
Our method selects a cluster that has the highest similarity
between its center and the input vector. However, since theamounts of training data and the density of each cluster are
different, we should determine a cluster with different thresholds of each cluster or a different way that is not only based on
similarity.

5.4 Effects of clustered learning method
Our method outperformed prior method when the number ofcluster was applicable, because the clusters of document clustering are divided into the kinds of data. From Table 3, when
the number of clusters is two, the precision outperformed thenormal (0 clusters). From Table 4, document clustering sorted
the training documents into different kinds. If we use various
kinds of data like internet web pages and intranet enterprise
documents, our method can increase the accuracy of information extraction.
In my opinion, the reason of goodness of the paper’s precision at the two clusters in Figure 4, the training data of technicalpapers is small. The difference of training data amount between
patents and papers is led from Table 7. In prior method (normal
mixture), the effect of patents whose training data is big isstronger than the paper one. So, the accuracy of paper is low.
However our method can divide training data into their kinds
and can deny this bad effect.
Because the rules became more specific by increasing theclusters, the recall decreased by increasing the number of clus-― 352 ―

Proceedings of NTCIR-8 Workshop Meeting, June 15–18, 2010, Tokyo, JapanPRECISION

RECALL70.00%

MIX_PATENT_DOC

40.00%

65.00%

MIX_PAPER_DOC

35.00%

60.00%

30.00%

55.00%

25.00%

50.00%

20.00%

45.00%

15.00%

40.00%

10.00%

35.00%

5.00%

30.00%

0.00%
0

0.05

0.1

0.15

0.2

0.25

0.3

0

Threshold

0.05

0.1

Threshold

0.15

0.2

0.25

0.3

Figure 5. Results in each threshold of similarity between test data and centre of a cluster
ters. We have to decide an applicable cluster size. It can be
testing with training data.
Context clustering could not get good result. We think its
cause was the problem of clustering method and cluster size.
The clustering methods that can compress huge dimensions,like pLSA, is better than K-means because the number of dimensions of the feature vector used by context clustering ishuge (760,000). Cluster size should be bigger than 10 because
kinds of contents is more various than documents. We havedone a simplified experiment when the number of cluster was
1000. We tested micro-averages of ATTRIBUTE, TECHNOLOGY and VALUE at nine cluster’s rules of 1000 clusters only.
The precision of patents and papers are 76.68 and 53.85, respectively. These precisions are as same as or higher than the resultsof a single kind in Table 3. We want to do experiments of otherclustering methods and big cluster size.

7. References[1]

Chang, C.–C. and Lin, C.-J. 2001. libSVM: a library forsupport vector machines. (Software available at
http://www.csie.ntu.edu.tw/~cjlin/libsvm)[2]

CLUTO, 2006, CLUTO version 2.1.2a, Software Package
for Clustering High-Dimensional Datasets”, March 2010,
http://glaros.dtc.umn.edu/gkhome/views/cluto.

[3]

Hofmann, T. 1999. Probabilistic latent semantic indexing.
Proceedings of the 22nd annual international ACM SIGIR
conference on Research and development in information
retrieval (SIGIR ‘99), pp.50-57[4]

Nanba, H., Fujii, A., Iwayama, M. and Hashimoto, T.
2010. Overview of the Patent Mining Task at the NTCIR8 Workshop. Proceedings of the 8th NTCIR Workshop
Meeting on Evaluation of Information Access Technologies: Information Retrieval, Question Answering andCross-lingual Information Access.

[5]

Sakao, Y., Ikeda, T., Satoh, K. and Akamine, S. 2005.
Japanese language analysis for syntactic tree mining to
extract characteristic contents. Proceedings of The Tenth
Machine Translation Summit (MT Summit X), pp 339345.

[6]

Steinbach, M., Karypis, G. and Kumar, V. 2000. A comparison of Document Clustering Techniques. KDD Workshop on Text Mining (KDD2000), Boston.

[7]

Yamada, H., Kudo, T. and Matsumoto, Y. 2002. Japanese
Named Entity Extraction Using Support Vector Machine.
Transactions of Information Processing Society of Japan
43(1), pp.44-53.[in Japanese]

6. CONCLUSION
We proposed an information extraction method called clusteredlearning for various kinds of data and applied it to a NTCIR8Technical Trend Map Creation subtask. Even if data contains
multiple kinds of data, our method obtained similar results to asingle one when training data can be divided into their kinds ofdata.
Future work includes experiments with the modification ofclusters and recall effects. We want to use another clustering
method like pLSA in context clustering. Another way of deciding select clustering will also be studied.

― 353 ―