Note that achieving an integrated data warehouse is not an easy task and is the subject of current research and debate. If an organization's data collection techniques and database structure guidelines were properly employed, the data contained in an organization's database would already be clean. But since the majority of organizations have not implemented and provided guidance for a true data warehouse the data selected for the KDD process must be cleaned. I was recently asked in what phase of the KDD could an organization expect to spend most of its time, and hence, money. The answer is in the cleaning phase, especially if steps had not been taken to ensure the reliability of the data initially.
Data cleaning is labor intensive and involves examining the data for completeness and integrity.
The causes of dirty or noisy data are many and each business domain has unique data cleanliness issues. In the enrichment phase, data, which may be obtained from an external source outside the organization, is added to the selected and cleaned data to add value to it. For example, in a retail KDD effort, customer shopping data is combined with third party demographic data, which maps customer names to addresses, number and types of cars they own, how many and what ages are their children, etc., perhaps to produce a target market group. In the data coding phase raw data is converted into a form that lends itself to be more effectively mined. It's sort of like cloud salting.