StarkAndWayne/bosh-bootstrap — pull request #159 by drnic (2013-05-21 15:28:49+00:00)
Title: Invalid availability zone: us-east-1b

I was noticing the same symptoms from [this](https://github.com/StarkAndWayne/bosh-bootstrap/issues/39) closed ticket, but using default configuration options (i.e., us-east-1). I get to Step 5, and it unpacks the stemcell just fine. Uploading stemcell times out after 15 minutes with an error stating "Invalid availability zone: us-east-1b".

On my AWS console it lists us-east-1a, us-east-1c, and us-east-1d as up, so I modified /var/vcap/store/stemcells/micro-bosh-stemcell-aws-0.8.1/micro_bosh.yml to hard-code the availability zone to us-east-1a. I'm testing that configuration out now, but it takes about 15 minutes to time out for each test so it's a slow process. I'll update this ticket with my results.

*EDIT: It still tried to use us-east-1b, so I think my changes were made in the wrong place. I'm still not sure why it's choosing us-east-1b because that availability zone is down according to my aws console. I'm trying it with us-west-1 for now.

*EDIT 2: I tried us-west-1, but I ran into another issue. Rather than exploring down that branch I'll continue my focus on us-east-1. Currently I'm running a test where I hard-coded fog's `describe_availability_zones.rb` file to exclude us-east-1b on the inception VM. I'll post back with results.

*EDIT 3: Ran with the modified `describe_availability_zones.rb` and timed out with the following error:
```
Deploy Micro BOSH
unpacking stemcell (00:01:00)
Uploading stemcell |o | 1/11 01:03:44 ETA: --:--:--/usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/helpers.rb:16:in `cloud_error': Timed out waiting for # to be attached (Bosh::Clouds::CloudError)
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/helpers.rb:34:in `block in wait_resource'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/helpers.rb:28:in `loop'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/helpers.rb:28:in `wait_resource'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/cloud.rb:611:in `attach_ebs_volume'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/cloud.rb:335:in `block in create_stemcell'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_common-0.5.4/lib/common/thread_formatter.rb:46:in `with_thread_name'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/cloud.rb:324:in `create_stemcell'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:215:in `block (2 levels) in create_stemcell'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:84:in `step'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:214:in `block in create_stemcell'
from /usr/local/lib/ruby/1.9.1/tmpdir.rb:83:in `mktmpdir'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:200:in `create_stemcell'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:128:in `create'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:104:in `block in create_deployment'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:97:in `with_lifecycle'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/deployer/instance_manager.rb:103:in `create_deployment'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_deployer-1.4.1/lib/bosh/cli/commands/micro.rb:171:in `perform'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/command_handler.rb:57:in `run'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:61:in `run'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:18:in `run'
from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/bin/bosh:16:in `'
from /usr/local/bin/bosh:23:in `load'
from /usr/local/bin/bosh:23:in `

'
error deploy micro bosh

```

*EDIT 4: Ok, finally found a workaround. I deleted all the work that had been done up to this point (security groups, key pairs, EBS volumes, EC2 instances, deleted local ~/.bosh_bootstrap directory, etc) to work with a clean AWS account. I ran with the default configuration options again, and I let it fail the first time. Then I changed `~/.bosh_bootstrap/manifest.yml` and added `availability_zone: us-east-1a` to the `bosh_resources_cloud_properties` section. I am really curious as to how us-east-1b was selected in the first place, but I couldn't figure it out. Hopefully this bug report helps some!