CRLF
CRLF
Merge remote-tracking branch 'upstream/master'

* upstream/master: (141 commits)
  Improved if_p can chain #else_if_p,  #else methods
  Fix broken unit specs
  Check if ext4 lazy_itable_init is enabled before using it
  Add 'lazy_itable_init' option to mke2fs [fixes #44226189]
  make agent not format the ephemeral disk
  Change property subnet in bat template
  Added RDS options to aws config yml
  Revert 'CI to use single AWS destroy command'
  CI to use single AWS destroy command
  Normalized `bosh aws delete_all` commands [#44112527]
  AWS bootstrap force adds keypairs
  Add echoes to scripts used to create MicroBOSH stemcell to debug failing Jenkins build
  Print and capture output simultaneously in AWS specs
  External specs refactored and set up to use different AWS credentials than the Jenkins VM
  Update README.md
  Update README.md
  Update README.md
  bosh aws destroy [fixes #44112527]
  remove echo's after finish debugging jenkins build
  updated README
  ...
CRLF
Merge branch 'master' of github.com:ankurcha/bosh

* 'master' of github.com:ankurcha/bosh:
  CRLF
Merge remote-tracking branch 'upstream/master'

* upstream/master: (25 commits)
  make AMIs public in CI
  fixed bugs in stemcell creation
  include original config in receipt
  Better message when deleting unnamed instance
  Update BAT README with new BAT environment options
  Support fast failure and skipping SSH tests in BAT via environment variables
  Changed director job template to use new style properties in bosh/release
  updated badges in README
  Don't reuse dns_info_record (fix for recreating jobs)
  use sigar instead of sys-filesystem
  remove trivial whitespace
  Update blobstore_client gemspec for renamed README & example config [fixes #49]
  [finishes #44595437] allocate an extra elastic IP for the DEA in dev/test
  Fix failing bat manifest specs
  example usage of blobstore_client_console
  Convert tabs to whitespace
  Fix spec:system:aws
  AWS script should not remove protected instances resources [fixes #44549997]
  BAT_DNS_HOST is not optional
  fake aws stemcells
  ...
Merge remote-tracking branch 'upstream/master'

* upstream/master: (70 commits)
  bump rds creation timeout to 20 minutes
  create db subnet group as part of rds setup
  add multiple cf subnets to aws unit test assets
  Bosh::Aws::RDS#create_subnet_group
  Bosh::Aws::RDS#subnet_group_exists?
  shellscript can't have spaces around assignments
  change ci stemcell builder to be able to build for all infrastructures
  Don't upload stemcell when NO_PROVISION is set
  Add more options to Bat test
  updated aws-sdk to one that isn't broken
  version bump party
  Use vpc outfile when generating bat manifest
  Added helper to get stemcell version
  Error message was not better, it turns out
  Moving cleanup to beginning of spec instead of after
  Better error logging in vpc deletion
  directory sleeps between HTTP request retries
  signature of aws generate microbosh changed, updating micro_bosh_spec.
  Better vpc error message
  Using the same aws credentials everywhere.
  ...
Merge remote-tracking branch 'upstream/master'

* upstream/master: (93 commits)
  Use /sys/proc/... to get the cdrom device
  [#45112063] - Exec returns an result object not a string, fix expectations accordingly
  Remove placeholder spec accidentally committed
  Use better key for global cache
  Conditionally save compiled packages to global blobstore
  First pass at saving to global blobstore
  [finishes #45168899] AWS bootstrap deploys microbosh from latest CI stemcell
  blobstore client should be able to suggest an object id
  copy global one-ring-key to local path for use by bosh
  CI Publish AMI sends light stemcell to S3
  Catch HTTPClient::ReceiveTimeoutError when waiting for registry to start
  output rds to a consistent file name
  [fixes #45112063] - fix issue with agent startup
  fix stemcell version comparsion when using timestamped stemcells from CI
  Filter for EBS volumes that aren't already deleted or deleting [fixes #45174599]
  support suggesting object id to blobstore server
  [#45168899] Jenkins publishes stemcell AMIs to a well-defined S3 URLs
  [#45168899] AWS bootstrap does micro deploy
  AWS::bootstrap: setting up for a 'no-arg' bosh aws bootstrap micro  command
  Clean up bat dns tests
  ...
Remove unnecessary block device checking
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  CF system spec creates RDS resources first
  fix microbosh system spec vpc/rds outfile paths
  fix microBOSH spec after deployment generator was changed to take two arguments
  enhance cpi tests for aws plugin and error message
  microBOSH system spec creates BAT deployment directory just when needed
  HTTP plays well with a proxy environment
  add retryable functionality to Bosh::Common module
  Move generate_build_number into CompiledPackage, add specs for it
  Added logging for uploading compiled packages to global cache
  Global cache key respects dependent package versions
  No need to check if it is a block device now
  Add some changes to pull request #59
  fix randomly failing aws boostrap cli spec
  move bosh_agent/misc/temcell/uild2 to the top level directory
  Added some tests and made changes to handle \n in the filename if any while searching for cdrom device
  vendor retryable gem
  Add Retryable gem
  added bosh-stemcell-vsphere 0.8.0 (test)
  update stemcell build for 3.0 kernel
  Use if_p in director template so it doesn't blow up when global_blobstore is missing

Conflicts:
	bosh_agent/lib/bosh_agent/infrastructure/vsphere/settings.rb
Merge remote-tracking branch 'upstream/master'

* upstream/master: (21 commits)
  [openstack_cpi] Limit the size of metadata keys and values
  [openstack_cpi] Detect API rate limit exceptions and retry calls [Fixes #45488271]
  Micro deployment no longer says 'Current deployment...' multiple times
  external bootstrap spec should not depend on clean state, stop using ec2.instances.first [#42828011]
  add rake task for external aws bootstrap spec
  retry on all aws errors during vpc delete
  make delete_all_security groups more robust
  update security_group_in_use? logic
  ignore aws_rds_receipt.yml files
  external specs for VPC subnet and route table creation
  WIP to add external tests for bosh aws bootstrap CLI [#42828011]
  config travis to ignore ruby 2.0 failures
  bosh release #13
  Handle version comparison better
  Make Aws and Openstack Registry 'delete settings' idempotent
  Changes to allow running Bosh on Ruby 2.0
  Remove comment from Rubygems2 cleanup
  Added support for retrieving from global compiled package cache
  make cli work with rubygems 2
  enable bat:disk 'should create a disk' example
  ...
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  Add AWS bootstrap external spec for ELB configuration
  increase Bosh CLI refresh rate to 10s to reduce deploy output for CI test
  Add external AWS bootstrap CLI tests for security groups and DHCP option sets
  Renamed global_blobstore to compiled_package_cache
  get director debug log if we have a failure
  Adding ruby-version file
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  added #exists?(object-id) to simple_blobstore_client
  Add 'head /resources/id' route to check for blob existence
  Extracting key pair creation logic from vpc creation.
  fix system spec helper run_bosh for debug_on_fail logic
  umount loopback and bind mounts in ci_build_stemcell.sh
  boot aws create/bootstrap system step in CI won't attempt bosh task last on fail
  remove incorrect safeguards in key pair deletion
  print stemcell build stage before configuring and applying
  bosh deployer progress render supports global -P option
  add test for forcefully adding key pair
  add support for compiled_package_cache to bosh_deployer
  bug fix for log output shorter than 100 lines
  Fixed package_compiler to use correct Config.use_compiled_package_cache? method
  Fixed /info endpoint for compiled_package_cache extras
  Remove debugger from 'test' bundler group
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  update director.yml.erb compiled_package_cache properties
  restric weird workaround to postgres only
  Updated BAT tests that use ssh to support keypairs
  typo fix for ec2 variable
  workaround for weird sequel/postgres thing in vSphere CPI migration
  Add pre-condition and post-condition for VPC creation and destruction tests
  fix cpi external test variable context scope
  use bosh aws create for microbosh system test
  Make sure to create keypair for cpi external test
  Adding key before subnet in external CPI tests
  eventual consistency handling when deleting rds
  display nfc error code in error message
  better error handling for deployer stemcell creation
  Fix director functional spec after switching compiled_package_cache to blobstore_client
  bosh-stemcell-vsphere-0.8.0.tgz stable
  add external AWS bootstrap test for S3 bucket creation
  Replaced fog with blobstore_client for compiled_package_cache
  [deployer] sync vsphere anv vcloud with aws default config
  [bosh-release] use Hash in director.yml.erb for vSphere
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  [director] Fix bug when retrieving current bosh & release revision
  updated gitignore to ignore public_stemcell_config.yml
  removed link that accidentally got committed
  fixed bug introduced by director cleanup
  director changes to improve consistency
  move .stemcell_builds into stemcell_builder directory
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  use File.open instead of File.new in blob_utils
  [release] Fix redis port in director spec
  use File in blob_utils
  [release] Set the default blobstore port
  increase the progress bar refresh time to 10s for Bat test in CI
  fixes dynamic drain deployment test
Merge remote-tracking branch 'upstream/master'

* upstream/master: (21 commits)
  [bosh_agent] fix failing spec
  [release] Fix recursor option in the PowerDNS configuration file
  [fixes #45863605] - don't remove rds or any rds security groups when user opts out of deletion
  fixed typo in microbosh manifest generator
  inline retryable method in common, don't use the gem
  [cli] check_duplicate_keys need to evaluate the yaml as erb before checking
  CloudFoundry system specs invoke VMC boostrap plugin
  update micro bootstrap code to use vpc_config instead of config
  add package cache to ci
  Fix param to micro bosh manifest generator
  [director] modify package cache output
  AG: cf system spec uses route53 outfile
  [bosh_agent] Fix typo error
  [bosh_agent] Fix dhclient3 problem when restarting network
  AG: add route53 input to bat manifest generation in system spec
  AG: add route53 receipt file path helper
  fix route 53 bootstrap code and external spec manifest generators now take vpc and route53 receipts
  updated director.tml template to use the correct blobstore options
  Add Route53 external test
  Adding aws bootstrap external tests
  ...
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  [bosh_vpshere_cpi] fix no_transaction bug
  [stemcell_builder] adjust for ovftool 3 bug
  [release] Fix job properties for registry
  [release] Add AWS & Openstack BOSH deployment manifest examples [VIP]
  [release] Add AWS & Openstack BOSH deployment manifest examples [DNS support]
  [release] Add default registry port to director template
  [bosh_deployer] Rename table Instances
  [bosh_registry] Rename table Instances (conflict with director)
  [release] Delete aws & openstack registry jobs & packages
  Remove aws & openstack registries from gem builds
  Use new registry in aws_bootstrap and spec
  [bosh_deployer] Use the new bosh_registry
  [bosh_openstack_cpi] Use new registry path
  [bosh_agent] Use new registry path for OpenStack registry
  Use bosh_registry properties in director and micro
  Add bosh_registry to Bosh release
  Add bosh_registry to the list of project gems
  New gem bosh_registry (unified aws & openstack registries in one code base)
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  Add retry when instance updater updates DNS record info in database due to race condition in parallel updates
  [director] fix spec_helper DB migrations
Merge remote-tracking branch 'upstream/master'

* upstream/master:
  [deployer] Use new retryable helper and print debug info
  [aws_cpi] Fix bug when deregistering an image
  [aws_cpi] Move task_checkpoint to helpers
  [cli] Fix ssh when job name is not given - Fixes #75
  Update README.md
  Update README.md