Finding Diamonds in Data: Reflections on Teaching Data Mining from the Coal Face

Introduction Data that has relevance for managerial decisions are accumulating at an incredible rate due to a host of technological advances: electronic data capture has become inexpensive and ubiquitous as a by-product of innovations such as the internet, e-commerce, electronic banking, point-of-sale devices, bar-code readers, and intelligent machines. Such data is often stored in data warehouses and data marts specifically intended for management decision support with supporting theory and techniques to manage such large volumes of data. Due to this explosion of data, many decision makers have realized the importance of techniques, such as data mining, that produce timely, relevant information to enable informed decision-making. As a result, data mining is an increasingly important component of the Information Systems curriculum in order to meet this skills demand. This aim of this paper is to describe how to teach students to find diamonds in data: the development of a curriculum for an elective data mining course in an Information Systems graduate program based on model curricula over a two year period. The paper is structured in the following manner: first, data mining is defined. Second, the model curricula that pertain to Information Systems in general and data mining in particular are examined. Thereafter, the curriculum development of the elective graduate data mining course at the University of Pretoria is discussed in detail. The paper concludes with the evaluation from the students and the reflection from the lecturers. Data Mining Data mining is a rapidly growing field that is concerned with developing techniques to assist managers to intelligently exploit these large volumes of data. According to Chakrabarti et al. (2006, p. 2) the central endeavor in data mining is to "extract knowledge from data" with this knowledge being captured "in a human-understandable structure." This extraction of knowledge from the data is done through the exploration and analysis by automatic means of large quantities of data to discover actionable patterns and rules which is what must be captured in a form that humans can understand. The University of North Texas defines data mining as follows in its course glossary (http://www.coba.unt.edu/itds/courses/dsci4520/DMglossary.htm#anchor314309) to be "an information extraction activity whose goal is to discover hidden facts contained in databases." Furthermore, data mining finds "patterns and subtle relationships in data and infers rules that allow the prediction of future results [own emphasis]." Data mining, therefore, will automatically detect previously unknown patterns and rules in large volumes of data that can be extrapolated to future events in a format that can be understood by humans in order to support decision-making. It offers a way to use massive quantities of data that businesses generate with the goal of improving marketing, sales, customer support through better understanding of customers. A number of successful applications have been reported in areas such as credit rating, fraud detection, database marketing, customer relationship management, and stock market investments. Given the central tenets of data mining its evolution from the disciplines of statistics and artificial intelligence makes sense: a combination of machine learning, statistical analysis, modeling techniques and databases is used in the process of discovering unknown patterns and relationships. Having defined data mining together with its application, the next section looks at mode l curricula for data mining courses at tertiary education level. Model Curricula The IS2002 Model Curriculum and Guidelines for Undergraduate Degree Programs in Information Systems (Gorgone, Davis, Valaich, et al., 2002) was jointly developed by a number of organisations, namely, the Association for Computing Machinery (ACM), the Association for Information Systems (AIS) and the Association of Information Technology Professionals (AITP). …