fix various issues that occur when dealing with thousands of files