Project: talenddataquality
Issue TDQ-5903: Create a tGenKeyHadoop component
This component reads data on HDFS and applies the algorithms as defined in the tGenKey component. The data always stay on hadoop. This component will require the use of a tHDFSConnection in order to connect to hadoop. The properties of this component are the same as those from tGenKey plus some properties to define the data format: - the HDFS file URI - the field separator - an option to link with another tGenKeyHadoop component (unchecked by default) When the option to link with another tGenKeyHadoop is selected, the output of the previous component becomes the input of the current component. The schema of this component will be defined manually by the user except the fixed T_GEN_KEY column. But this fixed column must be automatically renamed when several tGenKeyHadoop components are linked together. That means the first component has a T_GEN_KEY_1 column, the second component a T_GEN_KEY_2 column, etc. In the Map/reduce implementation, no reducer is required here. Only a Mapper class is necessary as each record is handled independently.
Story points: 13