Analysis of the sales information revealed, however, that the opposite was true. Not So Fast Despite the potential, there are several key areas where data mining endeavors can get into trouble in a hurry. The biggest expense and greatest shock to most people come when they see how difficult it is to find the right data on which to use the mining tools in the first place. Finding the data, cleansing it and making it ready to load into the database can take up 80 percent of any project's budget. The IS group had to pull data from many different Unix systems and proprietary environments to put together the core set of information that was needed.
Various problems can surface in trying to find source data.
Because companies often have reduced their legacy systems support staff to a minimum, there is nobody around who can tell you where the files are that might hold the data you seek. And after you find these candidate sources, a large amount of data preparation has to occur. First you have to figure out if the data source you have identified has a complete set of the information you want. Often, in order to get a complete set of data, you have to piece together different sources. For every field of data that you want to feed to a data mining tool, integrity and synchronization issues must be addressed.