# Mining Temporal Patterns of Technical Term Usages in Bibliographical Data

Hidenao Abe, Shusaku Tsumoto

<figure><img src="image_1.png" /></figure>

## To cite this version:

Hidenao Abe, Shusaku Tsumoto. Mining Temporal Patterns of Technical Term Usages in Bibliographical Data. 6th IFIP TC 12 International Conference on Intelligent Information Processing (IIP), Oct 2010, Manchester, United Kingdom. pp.130-138, 10.1007/978-3-642-16327-2_18. hal-01055063

HAL Id: hal-01055063

https://inria.hal.science/hal-01055063

Submitted on 11 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Mining Temporal Patterns of Technical Term Usages in Bibliographical Data

Hidenao Abe¹, Shusaku Tsumoto¹

Department of Medical Informatics, Shimane University, School of Medicine
89-1 Enya-cho, Izumo, Shimane 693-8501, Japan
abe@med.shimane-u.ac.jp, tsumoto@computer.org

**Abstract.** In text mining framework, data-driven indices are used as importance indices of words and phrases. Although the values of these indices are influenced by usages of terms, many conventional emergent term detection methods did not treat these indices explicitly. In order to detect research keys in academic researches, we propose a method based on temporal patterns of technical terms by using several data-driven indices and their temporal clusters. The method consists of an automatic term extraction method in given documents, three importance indices from text mining studies, and temporal patterns based on results of temporal clustering. Then, we assign abstracted sense of the temporal patterns of the terms based on their linear trends of centroids. Empirical studies show that the three importance indices are applied to the titles of four annual conferences about data mining field as sets of documents. After extracting the temporal patterns of automatically extracted terms, we compared the emergent patterns and one of the keyword of this article between the four conferences.

**Keywords:** Text Mining, Trend Detection, TF-IDF, Jaccard's Matching Coefficient, Temporal Clustering, Linear Regression

## 1 Introduction

In recent years, the accumulation of document data has been more general, according to the development of information systems in every field such as business, academics, and medicine. The amount of stored data has increased year by year. Document data includes valuable qualitative information to not only domain experts in the fields but also novice users on particular domains. However, detecting adequate important words or/and phrases, which are related to attractive topics in each field, is one of skilful techniques. Hence, the topic to support the detection has been attracted attentions in data mining and knowledge discovery fields. As for one solution to realize such detection, emergent term detection (ETD) methods have been developed [1, 2].

However, because the frequency of the words were used in earlier methods, detection was difficult as long as each word that became an object did not appear. These methods use particular importance index to measure the statuses
of the words. Although the indices are calculated with the words appearance
in each temporal set of documents, and the values changes according to their
usages, most conventional methods do not consider the usages of the terms and
importance indices separately. This causes difficulties in text mining applica-
tions, such as limitations on the extensionality of time direction, time consuming
post-processing, and generality expansions. After considering these problems, we
focus on temporal behaviors of importance indices of phrases and their temporal
patterns.

In this paper, we propose an integrated for detecting temporal patterns of
technical terms based on data-driven importance indices by combining automatic
term extraction methods, importance indices of the terms, and trend analysis
methods in Section 2. After implementing this framework as described in Sec-
tion ??, we performed an experiment to extract temporal patterns of technical
terms. In this experiment, by considering the sets of terms extracted from the
titles of four data mining relating conferences as examples, their temporal pat-
terns based on three data-driven importance indices are presented in Section 3.
With referring to the result, we discuss about the characteristic terms of the
conferences. Finally, in Section 4, we summarize this paper.

## 2 An Integrated Framework for Detecting Temporal Patterns of Technical Terms based on Importance Indices

In this section, we describe a framework for detecting various temporal trends
of technical terms as temporal patterns of each importance index consisting of
the following three components:

1. Technical term extraction in a corpus

2. Importance indices calculation

3. Temporal pattern extraction

There are some conventional methods of extracting technical terms in a corpus on the basis of each particular importance index [2]. Although these methods calculate each index in order to extract technical terms, information about the importance of each term is lost by cutting off the information with a threshold value. We suggest separating term determination and temporal trend detection based on importance indices. By separating these phases, we can calculate different types of importance indices in order to obtain a dataset consisting of the values of these indices for each term. Subsequently, we can apply many types of temporal analysis methods to the dataset based on statistical analysis, clustering, and machine learning algorithms. An overview of the proposed method is illustrated in Figure 1.

First, the system determines terms in a given corpus. There are two reasons
why we introduce term extraction methods before calculating importance in-
dices. One is that the cost of building a dictionary for each particular domain
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** An overview of the proposed remarkable temporal trend detection method.</figcaption></figure>

is very expensive task. The other is that new concepts need to be detected in a given temporal corpus. Especially, a new concept is often described in the document for which the character is needed at the right time in using the combination of existing words.

After determining terms in the given corpus, the system calculates multiple importance indices of the terms for the documents of each period. Further, in the proposed method, we can assume the degrees of co-occurrence such as the $\chi^2$ statistics for terms consisting of multiple words to be the importance indices in our method.

In the proposed method, we suggest treating these indices explicitly as a temporal dataset. The features of this dataset consist of the values of prepared indices for each period.

Figure 2 shows an example of the dataset consisting of an importance index for each year.

Then, the framework provides the choice of some adequate trend extraction method to the dataset. In order to extract useful temporal patterns, there are so many conventional methods as surveyed in the literatures [3, 4]. By applying an adequate time-series analysis method, users can find out valuable patterns by processing the values in rows in Figure 2.
<table><caption>Fig. 2. Example of a dataset consisting of an importance index.</caption><thead><tr><th>Term</th><th>Jacc.1996</th><th>Jacc.1997</th><th>Jacc.1998</th><th>Jacc.1999</th><th>Jacc.2000</th><th>Jacc.2001</th><th>Jacc.2002</th><th>Jacc.2003</th><th>Jacc.2004</th><th>Jacc.2005</th></tr></thead><tbody><tr><td>output feedback</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>H/sub infinity</td><td>0</td><td>0</td><td>0.012876</td><td>0</td><td>0.00885</td><td>0</td><td>0</td><td>0</td><td>0.005405</td><td>0.003623</td></tr><tr><td>resource allocation</td><td>0.006060606</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>image sequences</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.004785</td><td>0</td><td>0</td></tr><tr><td>multiagent systems</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.004975</td><td>0</td><td>0</td><td>0</td></tr><tr><td>feature extraction</td><td>0</td><td>0.005649718</td><td>0</td><td>0.004484</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>images using</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.004673</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>human-robot interaction</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.004425</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>evolutionary algorithm</td><td>0</td><td>0.005649718</td><td>0</td><td>0.004484</td><td>0</td><td>0</td><td>0</td><td>0.002703</td><td>0.003623</td><td>0</td></tr><tr><td>deadlock avoidance</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.004425</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>ambient intelligence</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.003623</td><td>0</td></tr><tr><td>feature selection</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.002703</td><td>0</td><td>0</td></tr><tr><td>data mining</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0.004425</td><td>0</td><td>0</td><td>0</td><td>0.002703</td><td>0</td></tr></tbody></table>

## 3 Experiment: Extracting Temporal Patterns of Technical Terms by Using Temporal Clustering

In this experiment, we show the results temporal patterns by using the implementation of the method described in Section 2 and Section ?? . As the input of temporal documents, we used the annual sets of the titles of the following four academic conferences<fnref n="1" />; KDD, PKDD, PAKDD, and ICDM.

We determine technical terms by using the term extraction method [6]<fnref n="2" /> for each entire set of documents.

Subsequently, the values of tf-idf, Jaccard coefficient, and Odds are calculated for each term in the annual documents. To the datasets consisting of temporal values of the importance indices, we extract temporal patterns by using k-means clustering. Then, we apply the meanings of the clusters based on their linear trends calculated by the linear regression technique for the timeline.

### 3.1 Extracting technical terms

We use the titles of the four data mining related conferences as temporal sets of documents. The description of the sets of the documents is shown in Table 1.

As for the sets of documents, we assume each title of the articles to be one document. Note that we do not use any stemming technique because we want to consider the detailed differences in the terms.

By using the term extraction method with simple stop word detection for English, we extract technical terms as shown in Table 2. After merging all of titles of each conference into one set of the documents, these terms were extracted for each set of the titles.

### 3.2 Extracting temporal patterns by using k-means clustering

In order to extract temporal patterns of each importance index, we used k-means clustering. We set up the numbers of one percent of the terms as the maximum

<footnote n="1">¹ These titles are the part of the collection by DBLP [5].</footnote>

<footnote n="2">² The implementation of this term extraction method is distributed in http://gensen.dl.itc.u-tokyo.ac.jp/termextract.html (in Japanese).</footnote>
<table><caption>**Table 1.** Description of the numbers of the titles.</caption><thead><tr><th rowspan="2"></th><th colspan="2">KDD</th><th colspan="2">PKDD</th><th colspan="2">PAKDD</th><th colspan="2">ICDM</th></tr><tr><th># of titles</th><th># of words</th><th># of titles</th><th># of words</th><th># of titles</th><th># of words</th><th># of titles</th><th># of words</th></tr></thead><tbody><tr><td>1994</td><td>40</td><td>349</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>1995</td><td>56</td><td>466</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>1996</td><td>74</td><td>615</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>1997</td><td>65</td><td>535</td><td>43</td><td>350</td><td></td><td></td><td></td><td></td></tr><tr><td>1998</td><td>68</td><td>572</td><td>56</td><td>484</td><td>51</td><td>412</td><td></td><td></td></tr><tr><td>1999</td><td>93</td><td>727</td><td>82</td><td>686</td><td>72</td><td>628</td><td></td><td></td></tr><tr><td>2000</td><td>94</td><td>826</td><td>86</td><td>730</td><td>52</td><td>423</td><td></td><td></td></tr><tr><td>2001</td><td>110</td><td>942</td><td>45</td><td>388</td><td>63</td><td>528</td><td>109</td><td>908</td></tr><tr><td>2002</td><td>140</td><td>1,190</td><td>43</td><td>349</td><td>62</td><td>515</td><td>121</td><td>1,036</td></tr><tr><td>2003</td><td>108</td><td>842</td><td>44</td><td>340</td><td>60</td><td>520</td><td>127</td><td>1,073</td></tr><tr><td>2004</td><td>133</td><td>1,084</td><td>64</td><td>504</td><td>83</td><td>698</td><td>105</td><td>840</td></tr><tr><td>2005</td><td>113</td><td>868</td><td>76</td><td>626</td><td>101</td><td>882</td><td>150</td><td>1,161</td></tr><tr><td>2006</td><td>139</td><td>1,068</td><td>67</td><td>497</td><td>128</td><td>1,159</td><td>317</td><td>2,793</td></tr><tr><td>2007</td><td>131</td><td>1,065</td><td>67</td><td>537</td><td>196</td><td>1,863</td><td>213</td><td>1,779</td></tr><tr><td>2008</td><td>134</td><td>1,126</td><td>110</td><td>832</td><td>136</td><td>1,224</td><td>264</td><td>2,225</td></tr></tbody><tfoot><tr><td>TOTAL</td><td>1,498</td><td>12,275</td><td>783</td><td>6,323</td><td>1,004</td><td>8,852</td><td>1,406</td><td>11,815</td></tr></tfoot></table>

<table><caption>**Table 2.** Description of the numbers of the extracted terms.</caption><thead><tr><th></th><th>KDD</th><th>PKDD</th><th>PAKDD</th><th>ICDM</th></tr></thead><tbody><tr><td># of extracted terms</td><td>3,232</td><td>1,653</td><td>2,203</td><td>3,033</td></tr></tbody></table>

number of clusters *k* for each dataset. Then, the system obtained the clusters with minimizing the sum of squared error within clusters. By iterating less than 500 times, the system obtains the clusters by using Euclidian distance between instances consisting of the values<fnref n="3" /> of the same index.

Table 3 shows the result of the SSE of k-means clustering. As shown in this table, the SSE values of Jaccard coefficient are higher than the other two indices: tf-idf and odds. Since we were not selected the terms with two or more words, the values of Jaccard coefficient of the terms with just one word, which are 0 or 1, are not suitable to make clusters.

### 3.3 Details of a temporal pattern of the technical terms

As shown in Table@4, there are several kind of clusters based on the averaged linear trends. The centroid terms mean the terms that are the nearest location to the centroids. Then, by using the averaged degree and the averaged intercept of each term, we attempt to determine the following three trends:

<footnote n="3">³ The system also normalized the values for each year.</footnote>
<table><caption>**Table 3.** The sum of squared errors of the clustering for the technical terms in the titles of the four conferences.</caption><thead><tr><th>Conf. Name</th><th>SSE (tf-idf)</th><th>SSE (Jaccard)</th><th>SSE (Odds)</th></tr></thead><tbody><tr><td>KDD</td><td>46.71</td><td>689.44</td><td>8.87</td></tr><tr><td>PKDD</td><td>58.76</td><td>432.21</td><td>18.17</td></tr><tr><td>PAKDD</td><td>35.13</td><td>325.53</td><td>10.01</td></tr><tr><td>ICDM</td><td>21.05</td><td>286.91</td><td>4.93</td></tr></tbody></table>

- Popular

• the averaged degree is positive, and the intercept is also positive.

- Emergent

• the averaged degree is positive, and the intercept is negative.

- Subsiding

• the averaged degree is negative, and the intercept is positive.

Since the terms assigned as the centroid have the highest FLR score in each pattern, the term is frequently used in the cluster by comparing to the other terms. As for the centroids of the degree and the intercept, they are the same as the average of each cluster, because the calculation of the centroid is assumed as the least-square method.

The emergent temporal patterns of the tf-idf index are visualized in Figure 3. According to the meanings based on the linear trend, the patterns #5, #6, and #8 of KDD have the emergent patterns. The emergent patterns that are #4 for PKDD, #1, #2, and #4 for PAKDD, and #4 for ICDM are also visualized.

Although these conferences share some emergent and subsiding terms based on the temporal patterns, characteristic terms can be also determined. The centroids of terms assigned as the emergent patterns<fnref n="4" /> express the research topics that have attract the attentions of researchers.

The emergent terms in KDD, they are related to web data and graphs. As for PKDD, the phrases ‘feature selection’ determine as emergent phrases only for this conference. The mining techniques that are related to items and text are also determined in PAKDD and ICDM. These terms indicate some characteristics of these conferences, relating to people who have been contributed for each conference.

By comparing these patterns of the indices, we can understand not only the remarkable terms but also similarity and dissimilarity of the conferences.

<footnote n="4">⁴ The emergent terms are emphasized in Table 4.</footnote>
<table><caption>**Table 4.** Whole of the temporal patterns as the k-means clustering centroids on the three data-driven indices.</caption><thead><tr><th rowspan="2">KDD</th><th rowspan="2">Cluster No.</th><th colspan="4">tf-idf</th><th colspan="4">Jaccard Coefficient</th><th colspan="4">Odds</th></tr><tr><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th></tr></thead><tbody><tr><td></td><td>1</td><td>sequence using data mining</td><td>0.007</td><td>0.039</td><td></td><td>graph mining</td><td>0.000</td><td>0.005</td><td></td><td>sequence using data mining</td><td>0.0000</td><td>0.0001</td><td></td></tr><tr><td></td><td>2</td><td>data mining</td><td>0.759</td><td>15.348</td><td></td><td>machine learning</td><td>0.006</td><td>-0.021</td><td></td><td>mining</td><td>-0.0060</td><td>0.3012</td><td></td></tr><tr><td></td><td>3</td><td>database mining</td><td>-0.088</td><td>1.271</td><td></td><td>databases</td><td>0.018</td><td>0.351</td><td></td><td>database mining</td><td>-0.0008</td><td>0.0085</td><td></td></tr><tr><td></td><td>4</td><td>web usage mining</td><td>0.022</td><td>0.255</td><td></td><td>pattern discovery</td><td>0.002</td><td>0.014</td><td></td><td>web usage mining</td><td>0.0000</td><td>0.0004</td><td></td></tr><tr><td></td><td>5</td><td>web data</td><td>0.094</td><td>-0.273</td><td></td><td>graphs</td><td>0.025</td><td>-0.026</td><td></td><td>web data</td><td>0.0001</td><td>-0.0003</td><td></td></tr><tr><td></td><td>6</td><td>relational data</td><td>0.132</td><td>-0.444</td><td></td><td>latent</td><td>0.020</td><td>-0.054</td><td></td><td>relational data</td><td>0.0002</td><td>-0.0004</td><td></td></tr><tr><td></td><td>7</td><td>web mining</td><td>-0.001</td><td>0.448</td><td></td><td>constraints</td><td>-0.003</td><td>0.122</td><td></td><td>web mining</td><td>0.0000</td><td>0.0014</td><td></td></tr><tr><td></td><td>8</td><td>graph mining</td><td>0.140</td><td>-0.558</td><td></td><td>prediction models</td><td>0.007</td><td>-0.017</td><td></td><td>graph mining</td><td>0.0002</td><td>-0.0006</td><td></td></tr><tr><td></td><td>9</td><td>bayesian network</td><td>-0.069</td><td>0.997</td><td></td><td>interactive exploration</td><td>0.025</td><td>-0.097</td><td></td><td>bayesian network</td><td>-0.0004</td><td>0.0049</td><td></td></tr><tr><td></td><td>10</td><td>data streams</td><td>0.045</td><td>0.094</td><td></td><td>rule induction</td><td>-0.009</td><td>0.092</td><td></td><td>data streams</td><td>0.0001</td><td>0.0004</td><td></td></tr><tr><td></td><td>11</td><td>knowledge discovery</td><td>0.519</td><td>4.485</td><td></td><td>predictive modeling</td><td>0.009</td><td>0.034</td><td></td><td>data mining</td><td>-0.0093</td><td>0.1430</td><td></td></tr><tr><td></td><td>12</td><td>mining knowledge</td><td>-0.055</td><td>0.898</td><td></td><td>mining</td><td>0.022</td><td>0.627</td><td></td><td>mining knowledge</td><td>-0.0003</td><td>0.0030</td><td></td></tr><tr><td></td><td>13</td><td>high-dimensional data</td><td>-0.029</td><td>0.798</td><td></td><td>data mining</td><td>-0.014</td><td>0.176</td><td></td><td>high-dimensional data</td><td>-0.0002</td><td>0.0039</td><td></td></tr><tr><td></td><td>14</td><td>distributed data mining</td><td>-0.017</td><td>0.543</td><td></td><td>learning bayesian networks</td><td>0.003</td><td>0.027</td><td></td><td>distributed data mining</td><td>-0.0001</td><td>0.0015</td><td></td></tr><tr><td></td><td>15</td><td>data sets</td><td>0.354</td><td>1.565</td><td></td><td>scale space exploration</td><td>0.004</td><td>0.075</td><td></td><td>databases</td><td>0.0003</td><td>0.0185</td><td></td></tr><tr><td></td><td>16</td><td></td><td></td><td></td><td></td><td>knowledge discovery</td><td>-0.008</td><td>0.137</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>17</td><td></td><td></td><td></td><td></td><td>efficient algorithms</td><td>-0.020</td><td>0.232</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>18</td><td></td><td></td><td></td><td></td><td>bayesian networks</td><td>-0.025</td><td>0.275</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>19</td><td></td><td></td><td></td><td></td><td>abstract</td><td>-0.005</td><td>0.128</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>20</td><td></td><td></td><td></td><td></td><td>categorical datasets</td><td>0.001</td><td>0.062</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table><table><thead><tr><th rowspan="2">PKDD</th><th rowspan="2">Cluster No.</th><th colspan="4">tf-idf</th><th colspan="4">Jaccard Coefficient</th><th colspan="4">Odds</th></tr><tr><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th></tr></thead><tbody><tr><td></td><td>1</td><td>classification learning</td><td>0.004</td><td>0.096</td><td></td><td>spatial data</td><td>0.002</td><td>0.004</td><td></td><td>classification learning</td><td>0.0000</td><td>0.0007</td><td></td></tr><tr><td></td><td>2</td><td>knowledge discovery</td><td>-0.168</td><td>1.932</td><td></td><td>document collections</td><td>-0.033</td><td>0.262</td><td></td><td>data mining</td><td>-0.0136</td><td>0.1382</td><td></td></tr><tr><td></td><td>3</td><td>data mining</td><td>-0.104</td><td>10.324</td><td></td><td>feature selection</td><td>-0.004</td><td>0.110</td><td></td><td>learning</td><td>-0.0017</td><td>0.1188</td><td></td></tr><tr><td></td><td>4</td><td>feature selection</td><td>0.195</td><td>-0.559</td><td></td><td>learning</td><td>0.007</td><td>0.668</td><td></td><td>pattern discovery</td><td>0.0004</td><td>-0.0012</td><td></td></tr><tr><td></td><td>5</td><td>spatial data</td><td>-0.116</td><td>1.195</td><td></td><td>supervised learning</td><td>-0.028</td><td>0.252</td><td></td><td>spatial data</td><td>-0.0007</td><td>0.0059</td><td></td></tr><tr><td></td><td>6</td><td>data clustering</td><td>-0.082</td><td>0.840</td><td></td><td>applications</td><td>-0.013</td><td>0.268</td><td></td><td>data clustering</td><td>-0.0002</td><td>0.0027</td><td></td></tr><tr><td></td><td>7</td><td>data streams</td><td>0.089</td><td>-0.546</td><td></td><td>knowledge discovery</td><td>0.022</td><td>0.018</td><td></td><td>data analysis</td><td>-0.0002</td><td>0.0017</td><td></td></tr><tr><td></td><td>8</td><td>relational learning</td><td>0.041</td><td>0.735</td><td></td><td>rule discovery</td><td>0.006</td><td>0.067</td><td></td><td>databases</td><td>0.0000</td><td>0.0071</td><td></td></tr><tr><td></td><td>9</td><td>web</td><td>0.073</td><td>0.270</td><td></td><td>data mining</td><td>-0.008</td><td>0.082</td><td></td><td>web</td><td>0.0002</td><td>0.0016</td><td></td></tr><tr><td></td><td>10</td><td></td><td></td><td></td><td></td><td>time series</td><td>0.009</td><td>0.072</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table><table><thead><tr><th rowspan="2">PAKDD</th><th rowspan="2">Cluster No.</th><th colspan="4">tf-idf</th><th colspan="4">Jaccard Coefficient</th><th colspan="4">Odds</th></tr><tr><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th></tr></thead><tbody><tr><td></td><td>1</td><td>hierarchical clustering based</td><td>0.143</td><td>-0.230</td><td></td><td>text mining</td><td>0.001</td><td>0.003</td><td></td><td>hierarchical clustering based</td><td>0.0002</td><td>-0.0004</td><td></td></tr><tr><td></td><td>2</td><td>data mining based</td><td>0.004</td><td>0.101</td><td></td><td>decision trees</td><td>0.010</td><td>0.034</td><td></td><td>data mining based</td><td>0.0000</td><td>0.0005</td><td></td></tr><tr><td></td><td>3</td><td>mining association rules</td><td>-0.122</td><td>1.201</td><td></td><td>density-based clustering</td><td>-0.012</td><td>0.090</td><td></td><td>databases</td><td>-0.0008</td><td>0.0053</td><td></td></tr><tr><td></td><td>4</td><td>text classification</td><td>0.220</td><td>-0.525</td><td></td><td>machine learning</td><td>0.011</td><td>-0.015</td><td></td><td>text classification</td><td>-0.0002</td><td>-0.0005</td><td></td></tr><tr><td></td><td>5</td><td> frequent pattern mining</td><td>0.263</td><td>-0.709</td><td></td><td>association rules</td><td>0.034</td><td>-0.079</td><td></td><td>mining frequent</td><td>0.0004</td><td>-0.0009</td><td></td></tr><tr><td></td><td>6</td><td>mining structured association patterns</td><td>-0.044</td><td>0.949</td><td></td><td>data mining</td><td>0.004</td><td>0.003</td><td></td><td>knowledge discovery</td><td>-0.0006</td><td>0.0069</td><td></td></tr><tr><td></td><td>7</td><td>data mining</td><td>1.365</td><td>3.439</td><td></td><td>continuous features</td><td>0.050</td><td>-0.137</td><td></td><td>data mining</td><td>0.0003</td><td>0.0272</td><td></td></tr><tr><td></td><td>8</td><td>knowledge discovery</td><td>0.570</td><td>1.739</td><td></td><td>databases</td><td>0.033</td><td>-0.005</td><td></td><td>algorithm</td><td>-0.0052</td><td>0.0933</td><td></td></tr><tr><td></td><td>9</td><td>clustering</td><td>2.382</td><td>8.213</td><td></td><td>mixed similarity measure</td><td>-0.032</td><td>0.283</td><td></td><td>clustering</td><td>-0.0012</td><td>0.1575</td><td></td></tr><tr><td></td><td>10</td><td>text mining</td><td>-0.020</td><td>0.790</td><td></td><td>rule extraction</td><td>0.014</td><td>-0.036</td><td></td><td>text mining</td><td>-0.0003</td><td>0.0033</td><td></td></tr><tr><td></td><td>11</td><td>data clustering</td><td>0.030</td><td>0.597</td><td></td><td>applications</td><td>-0.017</td><td>0.202</td><td></td><td>data clustering</td><td>-0.0001</td><td>0.0031</td><td></td></tr><tr><td></td><td>12</td><td></td><td></td><td></td><td></td><td>model</td><td>0.073</td><td>0.092</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>13</td><td></td><td></td><td></td><td></td><td>sequential patterns</td><td>-0.037</td><td>0.267</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>14</td><td></td><td></td><td></td><td></td><td>clustering</td><td>0.011</td><td>0.782</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>15</td><td></td><td></td><td></td><td></td><td>feature selection</td><td>-0.015</td><td>0.260</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>16</td><td></td><td></td><td></td><td></td><td>bayesian classifiers</td><td>-0.008</td><td>0.167</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table><table><thead><tr><th rowspan="2">ICDM</th><th rowspan="2">Cluster No.</th><th colspan="4">tf-idf</th><th colspan="4">Jaccard Coefficient</th><th colspan="4">Odds</th></tr><tr><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th><th>Term</th><th>Avg.</th><th>Deg.</th><th>Avg. Int.</th></tr></thead><tbody><tr><td></td><td>1</td><td>using data mining</td><td>0.070</td><td>-0.061</td><td></td><td>data clustering</td><td>0.003</td><td>0.003</td><td></td><td>using data mining</td><td>0.0001</td><td>0.0000</td><td></td></tr><tr><td></td><td>2</td><td>data clustering</td><td>-0.271</td><td>1.847</td><td></td><td>feature selection</td><td>-0.045</td><td>0.398</td><td></td><td>data clustering</td><td>-0.0006</td><td>0.0035</td><td></td></tr><tr><td></td><td>3</td><td>data mining approach</td><td>-0.154</td><td>1.407</td><td></td><td>sequence modeling</td><td>-0.027</td><td>0.211</td><td></td><td>medical data mining</td><td>-0.0004</td><td>0.0031</td><td></td></tr><tr><td></td><td>4</td><td>text mining</td><td>0.332</td><td>-0.005</td><td></td><td>data mining</td><td>-0.017</td><td>0.112</td><td></td><td>text classification</td><td>0.0007</td><td>0.0013</td><td></td></tr><tr><td></td><td>5</td><td>text classification based</td><td>0.206</td><td>0.010</td><td></td><td>data streams</td><td>0.051</td><td>-0.065</td><td></td><td>text classification based</td><td>0.0003</td><td>0.0000</td><td></td></tr><tr><td></td><td>6</td><td>mining</td><td>0.476</td><td>23.930</td><td></td><td>text classification</td><td>0.013</td><td>0.004</td><td></td><td>data mining</td><td>-0.0001</td><td>0.0408</td><td></td></tr><tr><td></td><td>7</td><td>web mining</td><td>-0.407</td><td>2.537</td><td></td><td>mining</td><td>0.010</td><td>0.814</td><td></td><td>web mining</td><td>-0.0009</td><td>0.0053</td><td></td></tr><tr><td></td><td>8</td><td>data mining</td><td>0.284</td><td>8.233</td><td></td><td>event sequences</td><td>-0.079</td><td>0.450</td><td></td><td>mining</td><td>-0.0110</td><td>0.2294</td><td></td></tr><tr><td></td><td>9</td><td>spatial data mining</td><td>0.085</td><td>0.468</td><td></td><td>ink prediction</td><td>0.050</td><td>0.002</td><td></td><td>data mining approach</td><td>0.0001</td><td>0.0014</td><td></td></tr><tr><td></td><td>10</td><td></td><td></td><td></td><td></td><td>association rules</td><td>0.037</td><td>0.426</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>11</td><td></td><td></td><td></td><td></td><td>change</td><td>0.011</td><td>0.102</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

## 3.4 Visualizing the trend of a key word of this article in the different conferences

Figure 4 shows the trend of 'text mining', which is included in the titles of the different four conferences, by using the tf-idf values. Their tf-idf values are increased in around 2000 and 2007 respectively. The later peak is not observed in the titles of PKDD. The trend shows that the technical topics related to this term are different in each peak. Since a novel technique itself is attractive in the earlier period, the technique tends to apply other topics by using the technique in the later periods. The trend of 'text mining' also shows that the technique was paid attentions in the earlier period, and the technique was applied to the other objects such as stream data.
<figure><img src="image_4.png" /><figcaption>Fig. 3. The emergent temporal patterns of tf-idf through the four conferences.</figcaption></figure>

# 4 Conclusion

In this paper, we proposed a framework to detect temporal patterns of the usages of technical terms appeared as the temporal behaviors of the importance indices. We implemented the framework with the automatic term extraction, the three importance indices, and temporal pattern detection by using k-means clustering.

The empirical results show that the temporal patterns of the importance indices can detect the trends of each term, according to their values for each annual set of the titles of the four academic conferences. Regarding the results, we detected not only the emergent temporal patterns in the conferences, but also the difference of the research topics between the conferences by comparing the temporal patterns and their representative terms. By focusing on the trend of one keyword of this article, ‘text mining’, we show the trend of this technical topic and the difference of the trends in the different conferences.

In the future, we will apply other term extraction methods, importance indices, and trend detection method. As for importance indices, we are planning to apply evaluation metrics of information retrieval studies, probability of occurrence of the terms, and statistics values of the terms. To extract the temporal patterns, we will introduce temporal pattern recognition methods [7], which can consider time differences between sequences with the same meaning. Then, we will apply this framework to other documents from various domains.
<figure><img src="image_5.png" /><figcaption>Fig. 4. The tf-idf values of ‘text mining’ in the titles of the four conferences.</figcaption></figure>

References

1. Lent, B., Agrawal, R., Srikant, R.: Discovering trends in text databases. In: KDD '97: Proceedings of the third ACM SIGKDD international conference on Knowledge discovery in data mining, AAAI Press (1997) 227–230

2. Kontostathis, A., Galitsky, L., Pottenger, W.M., Roy, S., Phelps, D.J.: A survey of emerging trend detection in textual data mining. A Comprehensive Survey of Text Mining (2003)

3. Keogh, E., Chu, S., Hart, D., Pazzani, M.: Segmenting time series: A survey and novel approach. In: an Edited Volume, Data mining in Time Series Databases., World Scientific (2003) 1–22

4. Liao, T.W.: Clustering of time series data: a survey. Pattern Recognition 38 (2005) 1857–1874

5. : The dblp computer science bibliography. http://www.informatik.uni-trier.de/~ley/db/

6. Nakagawa, H.: "automatic term recognition based on statistics of compound nouns". Terminology 6(2) (2000) 195–210

7. Ohsaki, M., Abe, H., Yamaguchi, T.: Numerical time-series pattern extraction based on irregular piecewise aggregate approximation and gradient specification. New Generation Comput. 25(3) (2007) 213–222