vchs/bosh_vcloud_cpi — pull request #4 by gberche-orange (2014-06-24 15:50:28+00:00)
Title: Adding support to http_proxy and https_proxy env vars to access vCloud R...

Adding support to ```http_proxy``` and ```https_proxy``` env vars to access vCloud REST API. Support both lower case and upper case variants. https gets preference over http if both are defined

This PR concerns flow B) below. Other bosh flows will be handled into https://github.com/cloudfoundry/bosh/pull/535

![bosh http flow diagram](http://www.plantuml.com:80/plantuml/png/XLBBZjim3BphAuWzB9f3VY271Vf42nJO0fjRK0YgJJl2og9AKPiQHVwzjBPntQoB-cHnHjJCaE8b8DnytE-_G6BwCK8kAH4B-0ZVA1zlZ3dS9sGddcA6BAxhleBBy33zcUIQHzTZfTecUFXfGB-6XaGHerm0Acr79ROyNk5F5RDDybFJocun1HgHxHzilQctAPJUHpluXicSPpNrR2kBs0ubYBTFGwuLM9Uerw9S0QrdR8HuZN_1x8du4Vd_hfurBIoTpU6JKnHZcyX7gQbE5LCvEW11ZYYJtaCM71eC0HWJPQyFZvL_gTkfhZp6W2xZDUrt-PbrDYPW9o04xFlZnQVErvWwLTZE2U-WQBT0ZGjXDAkvmtPxjyXm1wzURo1H21awOgZ5QMzOpB_6MrjCrlNocmqK9RXszWQtglzxmImPT8fWNknKlVzN5DhexMO7BPtYr6jMNfdBD9JuRa5KkpxsSEfH5dVkvbLG-eUD3Zf6FO8ScKf_XFtd1xEs5QkwtYzrcTNTMEQXpotT_m80 "Bosh HTTP flow diagram")

It current fails with the following trace if the vCD API is not directly reacheable and requires going through an HTTP proxy:

```
$ bosh micro deploy ./bosh-stemcell-2611-vsphere-esxi-ubuntu-lucid-go_agent.tgz

No `bosh-deployments.yml` file found in current directory.

Conventionally, `bosh-deployments.yml` should be saved in /home/gberche/production-configuration-reference/bosh.
Is /home/gberche/production-configuration-reference/bosh/micro-bosh-01 a directory where you can save state? (type 'yes' to continue): yes
Deploying new micro BOSH instance `micro_bosh.yml' to `https://10.106.236.100:25555' (type 'yes' to continue): yes

Verifying stemcell...
File exists and readable OK
Verifying tarball...
Read tarball OK
Manifest exists OK
Stemcell image file OK
Stemcell properties OK

Stemcell info
-------------
Name: bosh-vsphere-esxi-ubuntu-lucid-go_agent
Version: 2611

Started deploy micro bosh
Started deploy micro bosh > Unpacking stemcell. Done (00:00:03)
Started deploy micro bosh > Uploading stemcellcreate stemcell failed: getaddrinfo: Name or service not known:
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/net/http.rb:763:in `initialize'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/net/http.rb:763:in `open'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/net/http.rb:763:in `block in connect'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/timeout.rb:55:in `timeout'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/timeout.rb:100:in `timeout'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/net/http.rb:763:in `connect'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/net/http.rb:756:in `do_start'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/net/http.rb:745:in `start'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/rest-client-1.6.8.rc1/lib/restclient/request.rb:206:in `transmit'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/rest-client-1.6.8.rc1/lib/restclient/request.rb:68:in `execute'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/rest-client-1.6.8.rc1/lib/restclient/request.rb:35:in `execute'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:233:in `block in send_request'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:271:in `block in retry_for_network_issue'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:269:in `loop'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:269:in `retry_for_network_issue'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:231:in `send_request'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:244:in `session'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:51:in `block in org'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cache.rb:10:in `call'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cache.rb:10:in `get'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:50:in `org'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:58:in `block in vdc'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cache.rb:10:in `call'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cache.rb:10:in `get'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:57:in `vdc'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps/create_template.rb:7:in `perform'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:37:in `next'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cloud.rb:32:in `block in create_stemcell'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:44:in `call'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:44:in `block in perform'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_common-1.2611.0/lib/common/thread_formatter.rb:46:in `with_thread_name'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:41:in `perform'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:58:in `perform'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cloud.rb:305:in `steps'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cloud.rb:30:in `create_stemcell'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:228:in `block (2 levels) in create_stemcell'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:85:in `step'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:227:in `block in create_stemcell'
/home/gberche/.rvm/rubies/ruby-1.9.3-p487/lib/ruby/1.9.1/tmpdir.rb:83:in `mktmpdir'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:213:in `create_stemcell'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:118:in `create'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:98:in `block in create_deployment'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:92:in `with_lifecycle'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:98:in `create_deployment'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/cli/commands/micro.rb:179:in `perform'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli-1.2611.0/lib/cli/command_handler.rb:57:in `run'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli-1.2611.0/lib/cli/runner.rb:56:in `run'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli-1.2611.0/lib/cli/runner.rb:16:in `run'
/home/gberche/.rvm/gems/ruby-1.9.3-p484/gems/bosh_cli-1.2611.0/bin/bosh:7:in `'
/home/gberche/.rvm/gems/ruby-1.9.3-p487/bin/bosh:23:in `load'
/home/gberche/.rvm/gems/ruby-1.9.3-p487/bin/bosh:23:in `

'
/home/gberche/.rvm/gems/ruby-1.9.3-p487/bin/ruby_executable_hooks:15:in `eval'
/home/gberche/.rvm/gems/ruby-1.9.3-p487/bin/ruby_executable_hooks:15:in `

'
```

Fails as well during stemcell upload

```
Started deploy micro bosh
Started deploy micro bosh > Unpacking stemcell. Done (00:00:04)
Started deploy micro bosh > Uploading stemcellcreate stemcell failed: getaddrinfo: Name or service not known:
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:763:in `initialize'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:763:in `open'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:763:in `block in connect'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/timeout.rb:55:in `timeout'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/timeout.rb:100:in `timeout'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:763:in `connect'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:756:in `do_start'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:745:in `start'
/home/elpaaso/bosh_vcloud_cpi/lib/cloud/vcloud/file_uploader.rb:7:in `upload'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/vcd_client.rb:122:in `upload_stream'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps/upload_template_files.rb:19:in `block in perform'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps/upload_template_files.rb:9:in `each'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps/upload_template_files.rb:9:in `perform'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:37:in `next'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cloud.rb:33:in `block in create_stemcell'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:44:in `call'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:44:in `block in perform'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_common-1.2611.0/lib/common/thread_formatter.rb:46:in `with_thread_name'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:41:in `perform'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/steps.rb:58:in `perform'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cloud.rb:305:in `steps'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_vcloud_cpi-0.5.4/lib/cloud/vcloud/cloud.rb:30:in `create_stemcell'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:228:in `block (2 levels) in create_stemcell'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:85:in `step'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:227:in `block in create_stemcell'
/home/elpaaso/.rvm/rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/tmpdir.rb:83:in `mktmpdir'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:213:in `create_stemcell'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:118:in `create'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:98:in `block in create_deployment'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:92:in `with_lifecycle'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/deployer/instance_manager.rb:98:in `create_deployment'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli_plugin_micro-1.2611.0/lib/bosh/cli/commands/micro.rb:179:in `perform'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli-1.2611.0/lib/cli/command_handler.rb:57:in `run'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli-1.2611.0/lib/cli/runner.rb:56:in `run'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli-1.2611.0/lib/cli/runner.rb:16:in `run'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/gems/bosh_cli-1.2611.0/bin/bosh:7:in `'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/bin/bosh:23:in `load'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/bin/bosh:23:in `

'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/bin/ruby_executable_hooks:15:in `eval'
/home/elpaaso/.rvm/gems/ruby-1.9.3-p547/bin/ruby_executable_hooks:15:in `

'

```

Note that flow A) goes through HTTPClient gem that honors ```http_proxy``` ```https_proxy``` and ```no_proxy```