Remove the pipelining calls, it is unclear to me if this is possible at all with FTP.
Make the file size int64 so it is more in line with os.FileInfo.Size().
Add RetrieveTo to write the received data to a file instead of buffering all in memory.