implemented in-memory preprocessing for raw source data (data.table)