ran overnight on full dataset