Redo the file loader method to try and avoid corner cases where metadata exceeds data and also improve performance.