cloudfoundry/bosh — pull request #535 by gberche-orange (2014-02-27 09:26:21+00:00)
Title: Adding http proxy support to bosh director download_helper

download_helper now checks presence of http_proxy and https_proxy env variables and issues a debug trace if a proxy is used.

Moved into a smaller PR as requested by @dsabeti into https://github.com/cloudfoundry/bosh/pull/525#issuecomment-36164832

Original symptom is the following while using cloudfoundry-bosh (or bosh cli directly) and trying to download a stemcell from a host that has no direct internet access and require the use of an http_proxy
```
•[gberche@bosh-cli ~]$ bosh prepare cf
◦Fetching bosh information OK
◦Checking bosh already has release cf 149 OK
◦Checking bosh already has base stemcell FAILED
◦Currently bosh does not have base stemcell, uploading...
◦
◦Using remote stemcell `http://bosh-jenkins-artifacts.s3.amazonaws.com/bosh-stemcell/vcloud/bosh-stemcell-latest-vcloud--ubuntu.tgz'
◦
◦Director task 6
◦
◦Update stemcell
◦ downloading remote stemcell: Downloading remote stemcell failed. Check task debug log for details. (00:00:00)
◦Error 1/6 00:00:00
◦
◦Error 100001: Downloading remote stemcell failed. Check task debug log for details.
◦
◦Task 6 error
◦
◦For a more detailed error report, run: bosh task 6 --debug
◦[gberche@bosh-cli ~]$ env | grep -i proxy
◦http_proxy=http://10.114.60.8:3128
◦https_proxy=http://10.114.60.8:3128
•$ bosh task 6 --debug
◦E, [2014-01-23T08:46:46.805296 #4815] [task:6] ERROR -- : Downloading remote stemcell from http://bosh-jenkins-artifacts.s3.amazonaws.com/bosh-stemcell/vcloud/bosh-stemcell-latest-vcloud--ubuntu.tgz failed: #
◦E, [2014-01-23T08:46:46.805296 #4815] [task:6] ERROR -- : Downloading remote stemcell from http://bosh-jenkins-artifacts.s3.amazonaws.
◦com/bosh-stemcell/vcloud/bosh-stemcell-latest-vcloud--ubuntu.tgz failed: #
◦E, [2014-01-23T08:46:46.806488 #4815] [task:6] ERROR -- : Downloading remote stemcell failed. Check task debug log for details.
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/download_helper.rb:41:in `rescue in download_remote
◦_file'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/download_helper.rb:16:in `download_remote_file'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/jobs/update_stemcell.rb:97:in `block in download_re
◦mote_stemcell'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/jobs/base_job.rb:48:in `block in track_and_log'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/event_log.rb:75:in `call'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/event_log.rb:75:in `advance_and_track'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/event_log.rb:36:in `track'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/jobs/base_job.rb:46:in `track_and_log'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/jobs/update_stemcell.rb:95:in `download_remote_stemcell'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/jobs/update_stemcell.rb:31:in `perform'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/job_runner.rb:98:in `perform_job'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/job_runner.rb:29:in `block in run'
◦/var/vcap/packages/director/gem_home/gems/bosh_common-1.1782.0/lib/common/thread_formatter.rb:46:in `with_thread_name'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/job_runner.rb:29:in `run'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/lib/bosh/director/jobs/base_job.rb:10:in `perform'
◦/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/job.rb:125:in `perform'
◦/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:186:in `perform'
◦/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:149:in `block in work'
◦/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:128:in `loop'
◦/var/vcap/packages/director/gem_home/gems/resque-1.23.1/lib/resque/worker.rb:128:in `work'
◦/var/vcap/packages/director/gem_home/gems/bosh-director-1.1782.0/bin/bosh-director-worker:76:in `'
◦/var/vcap/packages/director/bin/bosh-director-worker:23:in `load'
◦/var/vcap/packages/director/bin/bosh-director-worker:23:in `

'
```