starting the new code to deal with the full dataset