[stager] npm support based on shrinkwrap and config option

Change-Id: Ica5a222abf603f8ffebcdb0f1e6bb0ff0a435f87
Raise instance of Warden::Client::ServerError on error replies.

Previously, we raised instances of StandardError on error replies
from the Warden. Recovering from these errors would require catching
StandardError, which unfortunately could end up catching errors that
we potentially cannot (or should not) recover from.

Change-Id: Ieb3b4621795d62167f1aceac41f9406097964a14
Add a description for the warden-client gemspec

Change-Id: Ia916f7ee604e560e3437e696fe02d3d51a6fe2c1
Adding require 'rubygems' to support Ruby 1.8

Change-Id: I18f7f882ed0ecf35a48f1b12b6d3dce353c33699
Merge "Raise instance of Warden::Client::ServerError on error replies."
Merge "Add a description for the warden-client gemspec"
[cc][fix] Store droplets using their IDs

Previously, we identified droplets using the SHA1 of their contents.
Any apps with the same source bits that are staged concurrently will
end up referencing the same file on disk. If one app is deleted,
any attempts to start more instances of the other app will fail, since
the droplet being referenced has been deleted. We solve this by
identifying droplets by application id.

Test plan:
- Deleted an app that existed prior to deploying the fix. Verified that
  the old droplet (identified by sha1) was deleted.
- Updated an app that existed prior to deploying the fix. Verified that
  the old droplet was deleted and the new droplet was identified by the
  app id.
- Increased the instances of an app that existed prior to deploying the
  fix. Verified that the instances were successfully started.
- Concurrently pushed two apps with the same source bits. Verified that
  two droplets were created.
- Parallel BVTs pass reliably now.

Change-Id: Ifce771a04092baeacfbd83fd8eb076153bd54835
Do DNAT for traffic coming from localhost

Change-Id: Ied67fb166bf4c4f7488e37bd358a51e1a23348a8
Merge "[cc][fix] Store droplets using their IDs"
Fixed the uaa config file to match uaa code changes in
preparation for public key signing of tokens.

Change-Id: I8bcedec23216ce77e3aa0f9738348927a80de087
[dea][bugfix] Fix race condition in concurrent downloading (CF-471)

  When more than 2 droplets download the same bits at the same time, they
will share the same destination file. If cleanup is enabled, the destination
file will be deleted, which caused the other droplets failed.

- Hard link randomized destination file to the original file, so cleanup can
be safe
- In unit test, use remote bits instead of local ones, add case for concurrence
test

Change-Id: Id80bd101418c79d9d5a1d9b0b30e7249406c58db
Remove http{s}_proxy when downloading/uploading to CCs from Stagers

This fixes the proxy issue without affecting user apps that rely
on their presence. It should be reverted once proxies are no longer
in production.

Test plan:
- Deployed without this fix and verified that staging failed when
  downloading the apps from the CC.
- Deployed the fix, verified that staging succeeded.
- BVTs pass

Change-Id: Ia27314d1ad99ffb2ae1536425ecdca39867766f6
Merge "Fixed the uaa config file to match uaa code changes in preparation for public key signing of tokens."
[dea][fix] concurrent download race condition

  This is a follows-up to commit 612bfcbd (Change Id80bd101). This
change:
    - Factored out random file name generation (shamelessly ripped off
      from Tal Garfinkel's package cache)
    - Fixed a bug in functional test where health of the first instance
      of an app was never tested
    - Minor refinement of the Helper classes

  Test plan:
    - Start 128 instances of an app in an unpatched single node
      dev_setup deployment, verify that it fails
    - Deploy on dev_setup
    - Verify the above test now passes
    - Start 32instances of an app in an unpatched dev instance with a
      single DEA, verify it fails
    - Deploy a BOSH release and verify the above test passes

Change-Id: I8be2cc21ff1ad3e955c5207e504d4009fe7325e6
Merge "Remove http{s}_proxy when downloading/uploading to CCs from Stagers"
Merge changes I8be2cc21,Id80bd101

* changes:
  [dea][fix] concurrent download race condition
  [dea][bugfix] Fix race condition in concurrent downloading (CF-471)
Merge "[stager] npm support based on shrinkwrap and config option"
[dea] functional tests clean up

  ForkedComponents and friends were first written for the DEA functional
tests and later got its place in vcap_common gem. This change replaces
the ForkedComponent and friends in DEA spec with the ones from
vcap_common.

  Test plan: pass unit tests.

Change-Id: Ic29f15382e866426433ff0b1d46aaa53e3b4493f
ID: 21988457 - Delete stale stuff for memcache sevice

Change-Id: I68ba7323b62a3b8f1fca3de4c87bc666f1034189
[dea] Include app_id in router register/unregister message

Change-Id: I21b45224ae22ab800afedea44d7bf718e533965c