hornairs/sprockets-rails-parallel — pull request #1 by jstorimer (2012-07-27 05:20:20+00:00)
Title: Multi-threaded approach

I was looking over the code and it looked like a better case for threading. The result is smaller, faster, and has less dependencies :)

Here are the benchmarks I took on my local machine:

``` console
Jesses-MacBook-Pro:shopify jstorimer$ ruby asset-compilation-bench.rb
user system total real
no parallelization 0.000000 0.000000 52.420000 (137.753395)
multi-thread parallelization @ 4 0.000000 0.000000 47.540000 ( 76.259939)
multi-thread parallelization @ 12 0.000000 0.000000 55.280000 ( 71.065993)
multi-thread parallelization @ 18 0.000000 0.000000 44.650000 ( 94.625203)
multi-process parallelization @ 4 0.000000 0.000000 63.150000 ( 80.518016)
multi-process parallelization @ 8 0.000000 0.000000 75.860000 (275.123535)

Jesses-MacBook-Pro:shopify jstorimer$ ruby asset-compilation-bench.rb
user system total real
no parallelization 0.000000 0.000000 35.940000 ( 99.593050)
multi-thread parallelization @ 4 0.000000 0.000000 40.280000 ( 64.388987)
multi-thread parallelization @ 12 0.000000 0.000000 40.570000 ( 55.465501)
multi-thread parallelization @ 18 0.000000 0.000000 36.400000 ( 54.625871)
multi-process parallelization @ 4 0.000000 0.000000 48.360000 ( 92.024623)
multi-process parallelization @ 8 0.000000 0.000000 61.060000 (364.976821)
Jesses-MacBook-Pro:shopify jstorimer$ ruby asset-compilation-bench.rb
user system total real
no parallelization 0.000000 0.000000 27.620000 ( 47.024846)
multi-thread parallelization @ 4 0.000000 0.000000 27.500000 ( 45.085441)
multi-thread parallelization @ 12 0.000000 0.000000 31.450000 ( 53.267642)
multi-thread parallelization @ 18 0.000000 0.000000 32.240000 ( 51.401890)
multi-process parallelization @ 4 0.000000 0.000000 35.400000 ( 50.630626)
multi-process parallelization @ 8 0.000000 0.000000 40.530000 ( 78.411461)
```

You mentioned you were seeing something weird but I didn't have any issues with the multi-process code myself. Except when I used too many processes it took forever, too much memory use leads to much swapping.