Feature: Download missing blobs in parallel

This is a feature request with an implementation suggestion.

In a project with a lot of blobs (be them large or small), whenever you need to download all of them, either locally or on CI, having a single connection is a pain.
Creating multiple threads (one per blob) improves overall speed greatly since we can take advantage of the network speed we're in. Downloads in parallel are faster because it's very difficult for a single network connection to use the whole bandwidth.

An EC2 instance downloading from S3 will end up hitting several different S3 terminations, creating several unique pipes of data transfer and achieving maximum speed. Of course there is a limit and the number of parallel downloads can't be arbitrarily large.

On some tests using cloudfoundry/cf-release from an EC2 instance I got an average of 6 minutes on the current bosh_cli version and 2 minutes using this patch with 5 parallel threads.

This implementation works the same way as the current for a single download (which is the default configuration I set here).

The one tricky bit about this is the CLI output. Showing parallel progress is tricky, so the `ProgressRenderer` class was created, it can move around an ANSI terminal, not all CI system will have full support for the escape codes used there. If that is a problem we can consider a different approach for show progress here (maybe disabling detailed progress with a CLI option?). Other than that it seems to look pretty ok:


![2014-10-16 23_51_24](https://cloud.githubusercontent.com/assets/223760/4675336/5ed119c8-558f-11e4-8689-4ffaf3e8dab6.gif)

Looks reasonable to me at first sight. Are you working on getting it merged?

On Thu, Dec 18, 2014 at 4:41 PM, Stev Witzel <notifications@github.com>
wrote:

>> +        @mon.synchronize do
>> +          @uwait.wait while @max > 0 and @count == @max
>> +          @dwait.signal if @count == 0
>> +          @count += 1
>> +        end
>> +      end
>> +    end
>> +
>> +    def signal(number = 1)
>> +      if (number > 1)
>> +        number.times { down!(1) }
>> +        count
>> +      else
>> +        @mon.synchronize do
>> +          @dwait.wait while @count == 0
>> +          @uwait.signal if @count == @max
> @luan: What do you think about something like this ...
> ```
>    def wait
>       @mon.synchronize do
>         @uwait.wait_while do
>           @max > 0 and @count == @max
>         end
>         @count += 1
>         count
>       end
>     end
>     def signal
>       @mon.synchronize do
>         @count -= 1
>         @uwait.signal
>         count
>       end
>     end
> ```
> Seems cleaner to me.
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/cloudfoundry/bosh/pull/684/files#r22083155