added data collection and preprocesssing