now only include the relevant columns of the data frame to reduce memory demands