Clean up input file handle if the cached version is used