bundler/bundler — pull request #3039 by Who828 (2014-05-15 19:24:51+00:00)
Title: Switch from UnixWorker to ThreadWorker for MRI

We have been using forked worker model for parallel gem installation on MRI, while this approach gives us true parallelism there are certain cost associated with it.

### Cons:

* High Memory Usage
Forked process requires more memory when compared to threads, thus on when forking more then 10 process (on servers with 16-32 cores) Bundler uses more then 90% of the memory available.

* Bugs like #3004
In case of this particular bug, we set up the remote/local specification on forked workers but we generate the standalone binstub in master but since master and forked process don't share the same memory address space. The remote/local specifications are nil in master and thus fails to generate the binstub.

While switching to ThreadWorker has many advantages for us,

### Pros:

* Little performance impact on MRI
Bundler's gem installation is more IO bound then CPU bound, we make remote calls, system calls and what not. Thus the performance impact at worse is around 10-15% (on Ruby 1.9+) according to benchmarks done by @indirect. This performance impact is only on MRI because of the GIL.

* Unified method across all other Ruby implementations
Bundler at the moment uses UnixWorker for MRI and ThreadWorker for rest of the Ruby implementations (and MRI on windows). However this makes it difficult maintain both the workers across different implementations. If we switch to ThreadWorker we would have lot less maintenance work.

* Less Memory Usage
Threads are lighter then processes since they share the same memory address space.

* Finally, no more bugs like #3004
Again because they share the same memory address space, we won't see bugs like #3004. However it's entirely possible we might see multithreading bugs that we didn't see earlier (because of forked workers) but then again we would have need to fix them anyway for other Ruby implementations (JRuby/Rubinius) which use ThreadWorker already.

Considering all of this, I think its better we switch to ThreadWorker for good instead of maintaining them both. Thoughts?