Performance enhancements for large datasets

Please read the commit messages and commits.
Thank you