polish and adjustments for larger datasets