# Research on Text Mining Based on Domain Ontology

Jiang Li-Hua, Xie Neng-Fu, Zhang Hong-Bin

► To cite this version:

Jiang Li-Hua, Xie Neng-Fu, Zhang Hong-Bin. Research on Text Mining Based on Domain Ontology. 7th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Sep 2013, Beijing, China. pp.361-369, 10.1007/978-3-642-54341-8_38 . hal-01220847

HAL Id: hal-01220847

https://inria.hal.science/hal-01220847

Submitted on 27 Oct 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Research on Text Mining Based on Domain
Ontology

JIANG Li-hua¹,² XIE Neng-fu¹,² ZHANG Hong-bin³,*

(1. Agricultural Information Institute of Chinese Academy of Agricultural Sciences, Beijing,
100081

2. Key Lab of Agricultural Information Service Technology of Ministry of Agriculture,
Beijing, 100081

3. Institute of Agriculture Resources and Regional Planning of Chinese Academy of Agricultural Sciences, Beijing, 100081)

**Abstract:** This paper improves the traditional text mining technology which cannot understand the text semantics. The author discusses the text mining methods based on ontology and puts forward text mining model based on domain ontology. Ontology structure is built firstly and the "concept-concept" similarity matrix is introduced, then a conception vector space model based on domain ontology is used to take the place of traditional vector space model to represent the documents in order to realize text mining.

Finally, the author does a case and draws some conclusions.

**Key Words:** Ontology; text mining; domain ontology; vector space model

# 1 Introduction

Natural language is the main communication and expression thought tool in today's economic society. Although it has been studied for a long time, the understanding and using ability is still limited. The data mining technology based on statistics had matured and applied successfully in large scale relational database in the early nineteenth century. Naturally scholars had the idea of applying the technology of data mining to analyze the text block described by natural language and called it text mining or knowledge discovery in text. Different from the traditional natural
language processing's focusing on understanding the words and sentences, the main goal of text mining is to find out the unknown and valuable knowledge or their relationship in large scale text sets. However, I found that most text mining lack of semantic considerations in application, only analyze grammatically, but not the content, so the results are always barely satisfied.

## 2 Text Mining Based on Ontology

Text mining, or knowledge discovery in text database, is the process of finding unknown, useful and understandable knowledge in large scale text database. The objects of text mining are semi-structured or unstructured. And they always contains multi-layer ambiguity, so a lot of difficulties of text mining are caused.

The traditional text mining method based on vector space model converts the text to word frequency vectors. The major defect of this method is neglecting the importance of semantic role leading to text mining results are unsatisfied. Therefore, semantic analysis and processing technology should be combine with text mining technology in order to develop more effective mining method to realize deep semantic level mining. Applying ontology to text mining provides theoretical support and a feasible approach to solve above problems.

At present, the representative semantic dictionaries applied common ontology are English WordNet and Chinese HowNet. There are many text mining methods based on WordNet and HowNet. But the text mining methods based on common ontology can not get a very good effect in partial field. Therefore a lot of research in recent years began to carry out the research of text Mining based on domain ontology. Bloehdom etc. put forward OnTology Based Text mining frame wOrk; The bag of words representation used for these clustering methods is often unsatisfactory as it ignores relationships between important terms that do not cooccur literally. In order to deal with the problem, Hotho etc. integrate core ontologies as background knowledge into the process of clustering text documents. Song etc. suggests an automated method for document classification using an ontology, which represses terminology information and vocabulary contained in Web documents by way of a hierarchical structure.

In our country, Knowledge Engineering Research Laboratory of computer science department in Tsinghua University has developed ontology data mining test platform based on semantic Web. And also there are some researchers who
discussed applications of semantic processing technology in text mining. Xuling Zheng etc. proposed a corpus based method to automatically acquire semantic collocation rules from a Chinese phrase corpus, which was annotated with semantic knowledge according to HowNet (Zheng Xuling etc., 2007). By establishing domain ontology as the way of knowledge organization, Guobing Zhou etc. introduced a novel information search model based on domain ontology in semantic context (Zou Guobing etc., 2009). An Intelligent search method based on domain ontology for the global web information was proposed to solve the problem of low efficiency typical in traditional search engineers based on word matched technology by Hengmin Zhu (Zhu Hengmin etc., 2010). In order to improve the depth and accuracy of text mining, a semantic text mining model based on domain ontology was proposed by Yufeng Zhang etc.. And in this model, semantic role labeling was applied to semantic analysis so that the semantic relations can be extracted accurately (Zhang Yufeng etc., 2011).

Taken together, the research of semantic text mining based on domain ontology is still in research theory spread stage, but relatively active in foreign countries. But there is few whole text mining based on domain ontology solutions. And the research scope is only in foundation of shallow knowledge such as classification and clustering of text (Bingham, 2001; Montes-y-Gómez, 2001) but rarely in rich useful deep semantic knowledge such as semantic association foundation (Zelikovitz, 2004), topic tracking (Aurora, 2007) and trend analysis (Pui Cheong Fung, 2003) and so on.

## 3 Research on key technology of text mining based on domain ontology

As an expert to guide the entire mining process, ontology is used to pre-process the text structure to realize semantic mining and improve mining effect.

### 3.1 knowledge representation Based on Domain Ontology

At present, most of the ontology system basic structure is similar which are entity, conception, attribute and relation. Namely, the features and corresponding parameters of entities and conceptions are studied by certain rules. At the same time, the relationship of entity and conception is described. Agriculture ontology is chosen as the subject and domain knowledge organization in this paper. It can not
only deal with the inner basic relation in agriculture subjection, but also more
formal specific relationship. Formalized agriculture ontology is defined:

Agri_Onto=(Onto_Info,Agri_Concept, AgriCon_Relation, Axion)

Onto_Info is the basic information of ontology including name, creator, design time, midified time, aim, souce and so on; Agri_Concept is the set of agriculture knowledge conception; AgriCon_Relation is relationship set of conceptions including hierarchical relationship and non hierarchical relationship; Axion includes axiomatic set in ontology.

3.2 Conception semantic correlation

In domain ontology, there are words to present class and conception. And the
words are not only serve as the bridge for class and conception, but also basic
element. The relationship in ontology depend the words to connect with each other,
so word set is the key of building agriculture ontology. In this paper, the class
words and conception words in agriculture ontology are isolated to make up
conception word set. T is conception sum in ontology. Matrix is used to
construct conception correlation.

$$
R = \begin{bmatrix}
R(C_1, C_1) & R(C_1, C_2) & \dots & R(C_1, C_T) \\
R(C_2, C_1) & R(C_2, C_2) & \dots & R(C_2, C_T) \\
\vdots & \vdots & \ddots & \vdots \\
R(C_T, C_1) & R(C_T, C_2) & \dots & R(C_T, C_T)
\end{bmatrix}
$$

Formula(1)

In matrix, R(C₁, C_T) is semantic correlation of C₁ and C_T. With respect to ontology conception correlation calculation, there are a lot of scholars to study the method. It is stated that there are always two methods: Information capacity and Conceptual distance. Conception semantic correlation method is adopted in this paper. Described as follows:

If conception Cᵢ and Cⱼ are synonymous relationship, the correlation of Cᵢ and Cⱼ is 1 and R(Cᵢ, Cⱼ) = 1; semantic correlation of no synonymous conceptions Cᵢ and Cⱼ is calculated by the following formula:

Cᵢ and Cⱼ is calculated by the following formula:
$$R(C_i, C_j) = \frac{(Dist(C_i, C_j) + \alpha) * \alpha * (d(C_i) + d(C_j))}{CE(C_i, C_j) * 2 * Dep * \max(|d(C_i) - d(C_j)|, 1)} \qquad \text{Formula(2)}$$

Therein, $d(C_i)$ and $d(C_j)$ are their corresponding levels in the binary tree; $Dist(C_i, C_j)$ is the weight of all weighted edges in the shortest route from $C_i$ to $C_j$;

$CE(C_i, C_j)$ is the sum of edges in the shortest route from $C_i$ to $C_j$; $Dep$ is max depth of ontology tree; $\alpha$ is a controllable parameter, generally more than 0 or 0. On this basis, semantic correlation matrix $R$ can be built to represent all conceptions in agriculture ontology.

### 3.3 Calculation Documents similarity

The key of automatic document clustering is to calculate similarity of documents. The most widely used method is cosine measure. Compare two documents

$$d_i = (t_{i1}, w_{i1}, t_{i2}, w_{i2}, \dots, t_{iu}, w_{iu}) \quad \text{and} \\ d_j = (t_{j1}, w_{j1}, t_{j2}, w_{j2}, \dots, t_{jv}, w_{jv})$$

Included angle cosine is used to present the level of similarity of documents:

$$Sim(d_i, d_j) = \frac{\langle d_i, d_j \rangle}{\|d_i\| \|d_j\|} = \frac{\langle d_i, d_j \rangle}{\sqrt{\langle d_i, d_i \rangle \langle d_j, d_j \rangle}} \\ (d_i, d_j) = \sqrt{\sum_{m=1}^{u} \sum_{n=1}^{v} w_{im} * w_{jn} * Sim(t_{im}, t_{jn})}$$

$$\|d_i\| = \sqrt{(d_i, d_i)} = \sqrt{\sum_{m=1}^{u} \sum_{n=1}^{v} w_{im} * w_{jn} * Sim(t_{im}, t_{jn})} \qquad \text{Formula (3)}$$

Therein, $t_{im}$ is conception feature word; $w_{im}$ is corresponding weight; $Sim(t_{im}, t_{jn})$ is semantic similarity of conception feature words, which can be got from formula (1).
# 4 Text Mining System Design Based on Domain Ontology

Combined with text mining and domain ontology, the text mining model based on domain ontology is put forward. The basic processing route is that: at first, "Conception - conception" correlation matrix of ontology is built. And the documents in which feature words are extracted are represented to space vector model based on conception. Then similarity between documents are calculated according to "Conception - conception" correlation matrix. At last, clustering analysis method is used to mine the deep knowledge.

## 4.1 System Frame

The text mining system frame makes up of six modules: text mining pretreatment, text feature extraction, text mining, ontology management, ontology reasoning, evaluation and output the modes.

<figure><img src="image_2.png" /><figcaption>Figure 1. System Structure</figcaption></figure>

## 4.2 Structural Design

### 4.2.1 Text data pretreatment

The data source of text mining is unstructured text collections. They can be web pages, text documents, text files, word and excel documents, pdf files, E-mails and various forms of electronic documents. After the data resource acquired, they should be pre-treated. The process includes: data cleaning, such as denoising and incline deduplication; data selection: appropriate and related to field text data are
selected; text segmentation: conception set in ontology is regarded as references to
realize professional vocabulary segmentation.

4.2.2 Text feature extraction

After data pretreatment, the text feature words must be extracted from the “clean”
data. The process includes: (1) feature representation: The VSM is adopted to
represent the documents. In the VSM, every document is presented to the following
feature vector:

$$
v(D) = (t_1, w_1(d); t_2, w_2(d); \dots; t_i, w_i(d); \dots; t_n, w_n(d))
$$

Thereinto, tᵢ is key words; wᵢ(d) is the weight of tᵢ in document d. wᵢ(d) is
often defined as the function of tfᵢ(d) that is the occurrence frequency of tᵢ in
documentd, namely wᵢ(d) = ψ(tfᵢ(d)). The frequently-used ψ is the function
TF-IDF:

$$
\psi = tf_i(d) * \lg \frac{N}{n_i}
$$

Thereinto, N is the sum of documents. nᵢ is the sum documents of tᵢ.

(2)Feature extraction: the disable word list and threshold λd are used to delete the unimportant words to reduce dimensions of document space. According to the function of TF-IDF to calculate the weight of key word tᵢ. If the weight is more than λd set in advance, the key words are reserved, or deleted. Obviously, a much larger threshold will filter too many words. And the left words can not represent the document content well. Instead, a much lower threshold will affect feature selection too little. Through analysis of a large number of experimental data. when

$$
\lambda_d = 0.3, \text{ a much better filter effect will come into being.}
$$

4.2.3 Text mining
Clustering analysis algorithm for mining is used and the aim is to divide the objects set into multiple Classes made up of similar objects. Firstly, every document is represented to feature vector based on key words $v(D) = (t_1, w_1(d); t_2, w_2(d);...; t_i, w_i(d);...; t_n, w_n(d))$. Then match $t_i$ with domain ontology conceptions, if so $t_i$ is taken place by domain ontology conceptions.

Otherwise, $t_i$ is regarded as unknown word. After matching, the key words are taken place by domain ontology conceptions in order that text document is represented to conception set. Then clustering analysis is used to cluster large amount of documents to few meaning cluster quickly so that the hidden knowledge or mode is acquired.

### 4.2.4 Ontology management

Ontology management is the key of the text mining model and provides semantic support for model. The main work of ontology management is building, storing, maintaining and optimizing the domain ontology. It provides a platform for users easy to build and maintain ontology database conveniently; to manage the ontology database in order to build, add, delete and modify the classes, relations and constraint rules in ontology; to find new conceptions or cases to extend ontology structure on the basis of text mining clustering algorithm.

### 4.2.5 Ontology reasoning

The function of ontology reasoning is semantic reasoning clustering results and deleting redundant or useless cluster by domain ontology as background knowledge or priori knowledge. It can refine and generalize related knowledge to enhance effectiveness and feasibility of clustering results.

### 4.2.6 Evaluation and output

The knowledge acquired from text mining may be inconsistent non-intuitive and not easy to understand. It is necessary to post process text knowledge. The main two indexes often used to evaluate text clustering effect are recall rate and accuracy rate to reflect completeness and correctness. The evaluation method which combined the two indexes is F measure.
$$ \mathrm{Re}(i, j) = \frac{n_{ij}}{n_i} $$

$$ \mathrm{Pr}(i, j) = \frac{n_{ij}}{n_j} $$

$$ F(i, j) = \frac{2 * \mathrm{Pr}(i, j) * \mathrm{Re}(i, j)}{\mathrm{Pr}(i, j) + \mathrm{Re}(i, j)} $$

Therein, $n_{ij}$ is the sum of class i in cluster j; $n_i$ is the sum of documents of class i; $n_j$ is the sum of documents in cluster j; $n$ is the sum of documents.

## 4.3 Application Analysis

In order to verify the effectiveness of the system, 400 documents are selected from agricultural encyclopedia column of Chinese Agriculture Academy Science website as research objects which are 100 documents of farming, 100 documents of aquaculture, 100 documents of plant protection and 100 documents of veterinary medicine. After the 400 documents are pretreated, K-means based on space vector model and conception space vector model are used to realize cluster analysis. Clustering results are compared with accuracy, Recall and F measurement. The results are shown in the following table.

<table><caption>**Table 1. Clustering results comparison**</caption><thead><tr><td>Arithmetic</td><td>Accuracy Rate</td><td>Recall Rate</td><td>F Measure</td></tr></thead><tbody><tr><td>K-means Based on VSM</td><td>70.1</td><td>85.3</td><td>59.4</td></tr><tr><td>K-means Based on CVSM</td><td>84.2</td><td>93.8</td><td>89.9</td></tr></tbody></table>

As can be seen from the experimental results, space vector model based on domain ontology in which conceptions in domain ontology are instead of feature words so that correlation of feature words are reduced and dimensions of document vectors are decreased making better clustering results in accuracy, recall, F measure than K-means based on space vector model.
## 5 Conclusion

With the development of information technology and network resource, a flood of information is produced. To analyze the text content and potential valuable knowledge, this paper put forward text mining model based on domain ontology and introduced “conception-conception” correlation matrix and used space vector model based on domain ontology instead of space vector model to represent document and clustering algorithm to discovery knowledge.

## Reference:

1. Feldman R, Dagan I. Knowledge discovery in textual databases (KDT) [A]. Proceedings of the First International Conference on Knowledge Discovery and Data Mining (KDD'95) [C].Montreal, Canada, AAAI press, 1995, pp.112-117.

2. Paolo Rosso, Edgardo Ferretti, Daniel Jimenez and Vicente Vidal. Text Categorization and Information Retrieval Using WordNet Senses [Z].The Second Global Wordnet Conference GWC 2004. Proceedings.pp.299-304.

3. Sedding, J., and Kazakov, D., “WordNet-based Text Document Clustering.” Proceedings of the Third Workshop on Robust Methods in Analysis of Natural Language Data (ROMAND), 2004, Geneva, pp.104-113.

4. Y. Ino, T. Matsui, and H. Ohwada. Extracting Common Concepts from WordNet to Classify Documents [A]. Artificial Intelligence and Applications[C], 2005, pp.656-661.

5. Shehata S. A Wordnet-based Semantic Model for Enhancing Text Clustering[C]. 2009 IEEE International Conference on Data Mining Workshop, 2009, pp.477-482.

6. S. Bloedorn, P. Cimiano, A. Hothon and S. Staab. An Ontology-based Framework for Text Mining [J], LDV-Forum, 2005, 20(1).

7. A Hotho, S Staab, G Stumme. Ontologies improve text document clustering[C]. Data Mining, 2003. ICDM 2003. Third IEEE International Conference on, 2003, pp.541-544

8. Mu-Hee Song, Soo-Yeon Lim, Seong-Bae Park, Dong-Jin Kang and Sang-Jo Lee. Ontology-based automatic Classification of Web Pages [J].International Journal of Lateral Computing, 2005, 1(1).
9. Zelikovitz S. Transductive LSI for Short Text Classification Problems[C]. Proceedings of the 17th International FLAIRS Conference, Miami: AAAI Press, 2004.

10. Aurora Pons-Porrata a, Rafael Berlanga-Lavori b, Jose' Ruiz-Shilkloper. Topic discovery based on text mining techniques [J]. Information Proceeding and Management. 2007(43), pp.752-768.

11. Pui Cheong Fung, G.Xu Yu, J.Wai Lam. Stock prediction: Integrating text mining approach using real-time news [Z]. In: Computational Intelligence for Financial Engineering, 2003, Proceedings, 2003 IEEE International Conference on, 2003, pp.395-402.