Role of Text Mining in Business Intelligence

This paper includes the combined study of business intelligence and text mining of uncertain data. The data that is used in current business domains is not precise, accurate and complete. Instead, data is considered uncertain and therefore this uncertainty is propagated to the results produced by Business Intelligence (BI). Till now, websites most often used text-based searches, which only found documents containing specific user-defined words or phrases. Through use of a semantic web, text mining can find content based on meaning and context rather than just by a specific word. This improves our search and BI results and facilitates social networks analysis or counter-intelligence. The text mining software acts similar to an intelligence analyst or research librarian, focusing on more limited scope of analysis. The possibilities for data mining from large text collections are virtually untapped. Text expresses a vast, rich range of information, but encodes this information in a form that is difficult to extract automatically specially from unstructured data. May be for this reason, there has been little work in text data mining to date, and most people talk about it as information access or many have not used text directly to discover unknown information. Through this paper we wish to uncover such areas and define Business Intelligence, structured or unstructured data, text mining, and then discuss the potential applications and limitations of text mining. The idea behind discussing this is to draw attention to exciting new kinds of problems and BI trends like green computing, social networking, data visualization, mobile BI, predictive analytics, composite applications, cloud computing, multi-touch and Softwareas-a-Service (SaaS). This paper would outline recent ideas about how to pursue exploratory data analysis over text and what we consider to be real text data mining efforts. KeywordsBusiness Intelligence, Text mining, structured data, unstructured data 1 Assistant Professor, Jagannath International Management School, Kalkaji, New Delhi E-mail:vaishpalak@rediffmail.com Mobile: 9650012542 2 Assistant Professor, Jagannath International Management School, Kalkaji, New Delhi E-mail:barkhaanarang@gmail.com Mobile: 9971253940 GIAN JYOTI E-JOURNAL, Volume 1, Issue 2 (Jan – Mar 2012) ISSN 2250-348X www.gjimt. com/GianJyotiE-Journal.htm Introduction Business Intelligence (BI) refers to technologies and applications for collecting, storing and analyzing business data that ultimately helps the enterprise to make better decisions (De Ville, 2001). BI techniques are generally computer based that provide past, current and future trends of the enterprise. Its applications include activities of decision support system, query and reporting, data mining, complex event processing, online analytical processing, process mining, business performance management, text mining, statistical and predictive analysis. Business Intelligence helps in good decision making and ensures competitive intelligence by analyzing the company’s internal data along with the information of the competitors (Bergeron and Hiller, 2002). BI applications in an enterprise are diverse providing enterprise reporting to serve strategic management of business, executive information system, collaboration platform to allow sharing of inside and outside business data and electronic data interchange. An important aspect of BI is knowledge management that helps companies in making good strategies through proper insight and experiences. BI also provides pro-active alerts in form of colour change of reports if some mishap occurs. BI helps companies to analyze their loads of data for decision making but not all data is structured and simple to understand as some data exists in unstructured or semi-structured form in which searching and interpretation consumes a lot of time. Thus, making decisions in such situations becomes complex. MetadataHandling Unstructured and Semi-structured data Businesses store volumes of data in the form of web pages, emails, video and image files, news and reports which are called semi structured or unstructured data. In practise, such data leads to waistage of time in searching and leads to poor decisions as volumes of unstructured data are stored in variety of formats and referred by different technologies. To solve this problem, metadata which is data about data, should be kept with unstructured or semistructured data. By the techniques of information extraction and automatic categorization, metadata can be generated in the form of summaries or topics. Text MiningInnovation in BI to gain competitive advantage It is a process of automatic extraction of information from large unstructured text (Hart, nd). In text mining, patterns are identified from natural language text rather than databases as in GIAN JYOTI E-JOURNAL, Volume 1, Issue 2 (Jan – Mar 2012) ISSN 2250-348X www.gjimt. com/GianJyotiE-Journal.htm data mining. It also differs from web mining as in contrast to structured data of web, text mining is usually done on unstructured data to derive pieces of genuine information for business. It is different from web search as in web, searching is of something that is already known and has been written by someone else but in text mining, unknown information is discovered for future predictions. The phrase “text mining” is used to denote any system that analyzes large quantities of natural language text by parsing it and then detects lexical or linguistic usage patterns in an attempt to extract correct information (Sebastian, 2002). It is about looking for patterns in text by automatic extraction. This linking of extracted information is very important as it leads to formation of new facts and hypotheses that can be explored further by experiments. Text mining is quite different from data mining as data mining implies extraction of implicit, previously unknown and useful information from data (Frank and Witten, 2000) but with text mining, the information to be extracted is not at all hidden and is explicit. Text mining explores data in a form that can directly be feeded into computers without any human intervention. It grew from related technologies based on statistics, probability theory and artificial intelligence for example performance of any search can be measured by counting success and failures. Text mining represents flexible approaches to information management, research and analysis on textual materials. It also helps in natural language processing as it automates the analysis of links of citations in text and hyperlinks in web. Natural language depends solely on “common-sense” knowledge which is exceptionally difficult to encode in algorithmic form (Lenat, 1995) so text mining came up as an outgrowth of “real-text” mindset which provides shallow processing of unrestricted text and deep processing of domain specific material. Data Warehouse Vs Document WarehouseRepository for Text Mining Data Warehouse is actually a repository of business or enterprise databases which gives a picture of historical and current organization’s operations (Date, 2003). It focuses on internal sources as quality control, sales, inventory and production. Data warehouse is designed to enable efficient management decision making as it presents a coherent picture of business conditions at a single point of time. It involves development of systems that enable extraction of data from operating system and installation of a warehouse database system that helps managers to access data in flexible ways. GIAN JYOTI E-JOURNAL, Volume 1, Issue 2 (Jan – Mar 2012) ISSN 2250-348X www.gjimt. com/GianJyotiE-Journal.htm However, Document warehouse is a repository for Business Intelligence keeping variety of document types from different sources to automatically extract and store the salient features of documents. It provides semantic integration of related documents of an enterprise. Document warehouse is a software framework for analysis, sharing and reuse of unstructured or semi-structured data as multimedia or text documents which is different from data warehouse where structured data as tabular sales reports are analyzed. Data warehouse integrates multiple related data stores and provides managers volumes of information for analytical and decision support purposes. Its environment includes relational database, Extraction, Transportation, Transformation and Loading (ETL) solutions, client analysis tools and Online Analytical Processing (OLAP) engine. But Document warehouse leads to Enterprise Document Repository (EDR) of multiple unstructured information bases that helps in efficient, rapid and global access to information of various types in various forms. Figure 1 Steps to Construct Document Warehouse (Source Gao Li, Chang Elizabeth, and Han Song, “Powerful Tool to Expand Business Intelligence: Text Mining”, World Academy of Science, Engineering and Technology, 2005) GIAN JYOTI E-JOURNAL, Volume 1, Issue 2 (Jan – Mar 2012) ISSN 2250-348X www.gjimt. com/GianJyotiE-Journal.htm Data Warehouse provides information related to what, when, who, where and how aspects but document warehouse answers the main query of users that is “why”. It gathers information from both internal and external sources to enable long-term strategic management. All data collected from various sources undergo summarization, categorization, feature extraction, clustering and topic tracking to give textual data for storage in document warehouse. This textual data can be in the form of normal documents, summaries, language translations or metadata. This process of constructing the document warehouse is explained in the Figure 1.