As data volume increases, the challenge of implementing a data warehouse grows significantly. Re-keying data is unrealistic because of the enormous workload and the likelihood of introducing errors. Intelligent transformational data replication has proven to be an effective middleware layer solution to help iSeries shops implement data warehousing. The essence of intelligent replication involves automatically copying data from one system to another, or within one computer from one file set to another. With the right middleware or replication architecture, warehouse data can be copied selectively and filtered based on criteria such as row or column characteristics.
Intelligent replication can also transform and enhance warehouse data.
Field values can be re-mapped to new warehouse data standards. A good replication tool for data warehousing must provide the ability to choose which segments of existing production databases will be replicated. To improve the semantic content of warehouse data, the tool should provide enhancement features such as derived field calculations, value translations, column mapping and data type conversions. Data changes should be applied to warehouse copies when both the network and CPU cycles are available. The tool should also provide the ability to apply a wide variety of data type transformations including NULL and date type transforms as well as derived field calculations.