MINING : PRINCIPLES AND APPLICATIONS

In this paper, some of the many possibilities of Data Mining on text collection are explored. The possibilities for automatic analysis of text or Text Mining are untapped. Text represents a kind of unstructured and at the same time, rich information which is difficult to decode by automatic means. Some background on this problem is described and the main ideas about how to pursue Text Mining are briefly outlined. Some of the main approaches, common architectural features and their applications and uses are discussed too. 1 Institute for Communicating and Collaborative Systems, School of Artificial Intelligence, University of Edinburgh, Edinburgh, Scotland, UK. e-mail: atkinson@dai.ed.ac.uk INTRODUCTION Knowledge Discovery in Databases (KDD) -or Data Mining (DM), focuses on the computerized exploration of large amounts of data and on the discovery of interesting patterns within them. Thus, DM is defined as “The nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data” [3], [13]. While most work on DM has been concerned with structured databases, there has been little work on handling the huge amounts of information that is available only in unstructured textual form. A table showing a classification of DM and TM applications can be seen in Fig. 1. These can be divided into two groups: those where techniques are applied to find some fixed patterns in data and those where it is expected to discover new knowledge, relationship or associations on data. As an analogy to find “gold” in mines, the later is called Nuggets. Even though it is not a real case in DM, it presents at least a good starting point to realize where we are going. If patterns are to be found in structured databases (non-textual) then standard DM techniques are applied. Otherwise, it is necessary a completely different kind of technique to handle the textual information (corpus linguistics). As far, the type of techniques to be applied is clear enough, however, problems go up when it is required to discover new knowledge from textual data. It could be chosen to apply standard corpus linguistics but unfortunately, there is not fixed patterns to match, instead, we have to find and discover these ones, to analyse them and associate them. Thus, specialized techniques specifically operating on textual data become necessary to extract information from such kind of collection of texts. These techniques are gathered under the name of Text Mining (TM) and, in order to discover and use the implicit structure of the texts (eg. grammar structure), they may integrate some specific Natural Language Processing (Computational Corpus Linguistics) [4], [7], [8], [9]. TM applications impose strong constraints on the usual NLP tools: as they involve large volumes of textual data, they do not allow to integrate complex treatments, semantic models for the application domains are rarely Atkinson, J.Text mining: principles. 58 available, and this implies strong limitations on the sophistication of the semantic and pragmatic level of the linguistic models [7], [12]. Fig. 1.DM versus TM Applications TM shares many characteristics with classical DM, but also differs in many ways. On the other side, many classical DM algorithms, such as value prediction or decision trees, are irrelevant or ill suited for the textual applications because they relies on the structuring of data, the availability of many information and so on [4], [5], [7], [12], [13]. On the other hand, there are special mining tasks, such as concept relationship analysis which are unique to TM. Thus, Text Mining is defined as: “The knowledgediscovery process which looks for identifying and analyzing useful information on data which is interesting to users from big amounts of textual data”. From this perspective, Information Extraction (IE) [2], [9], [10] and Text Mining may be complementary tasks but they differ in many ways. While IE relies in matching some fixed patterns to get the required information from the text and then translate it into a structured media (ie. database, templates, etc), TM or Information Analysis [1] relies in the fact that the value of the raw information comes from the competence can be got to analyze and to produce “elaborated” information. This is, both a high level information/knowledge which could be implicit but not present on data, and a potentially useful information/knowledge for the decision making process in a certain activity domain. Therefore, the tasks of TM on textbases could include the following: • To cluster knowledge/information into charts/maps. • To summarize information. • To identify hidden structures between groups of objects. • To extract hidden associations between elements on the texts. • To provide an overview of the contents of a large document collection. • To categorize texts by discovering relevant groupings. TEXT MINING TECHNIQUES There are different techniques and methods for TM in order to find new structures, patterns, or associations. Some TM has involved the assumption of an a priori categorization (preprocessing) into attributes and then proceeded via “classical” DM methods, i.e. statistical analysis, associations, etc [5], [12]. Others, investigate the full text of document collection, e.g. categorization used above, or purely analytical results. A common end-goal of much TM is a more efficient, complete, and/or specific way to browse and search large collections of documents. Thus, the main techniques in TM can be divided according to the tasks they perform in the discovery process: the kind of information they extract and the kind of analysis/association done with them: KINDS OF INFORMATION EXTRACTED: 1. Labels: it has been assumed that associated with each document is a set of labels and knowledgediscovery operations are performed on the labels of each document. In general, it can be assumed that labels correspond to keywords, each of which represents that a given document is about a topic associated with that keyword [6,9]. 2. Words: in which a document is assumed to be labeled with each of the words that occurs within it. 3. Terms: in which for each document is found word sequences that are likely to have meaning in the domain, and then mining is performed on the extracted terms labeling each document. The advantage of this method is that the extracted terms are fewer in number and tend to represent the important information on the text than the previous approaches [4,5]. REVISTA FACULTAD DE INGENIERIA, U.T. A. (CHILE), VOL.7, 2000 59 KIND OF ASSOCIATIONS: Regular Associations: some known algorithms (in a very simplified view) assume that the textual data is indexed (manually or automatically) with the help of NLP techniques. The indexing structures can be used as a basis for the actual knowledge discovery process. In order to generate association rules, they operate in two phases [12]. Given a set of keywords A and a collection of indexed documents T, the extraction of associations satisfying, given some support and confidence constraints is performed. Concept Hierarchies: in which each document is tagged with terms got from a hierarchy of concepts. Then, a system analyzes content distribution of a term's descendants relative to other descendants in according to its joint distributions and other comparation measures in order to discover new relationship among them [4], [5]. This kind of relationship can be also used for filtering and summarizing news articles, utilizing concept distributions marked as interesting by some user. Full Text Mining: unlike the regular associations which exclusively operate on the document indexes, this technique takes advantage of the textual content of the documents. This new TM task which consider full TM for information extraction is called Prototypical Document Extraction, where “prototypical” is defined as information that occurs in a repetitive fashion in the document collection. The working hypothesis is that repetitive document structures provide significant information about the textual base is processed. Basically, the method relies on the identification of frequent sequences of terms in the documents, and uses NLP techniques such as part-of-speech tagging and term extraction to preprocess the textual data. These techniques can be considered as an automated generalized indexing procedure that extracts from the full textual content of the documents linguistically significant structures that will constitute a new basis for frequent set extraction [12].