modified for large dataset