cleaned up curl_ftp a bit and added awk preprocessing