ruby-concurrency/concurrent-ruby — pull request #109 by jdantonio (2014-06-13 06:39:23+00:00)
Title: Merging the ruby-atomic gem an building native extensions

## Atomic Gem & Native Extensions - I Think I'm Finally Ready

After much experimentation (and many false starts) I think I have a set of native extensions ready to go. The new functionality added in this PR is a merge of the [ruby-atomic] gem, but the gem itself has been configured to (hopefully) provide painless installation across the largest set of possible Ruby interpreters.

Adding native extensions to our gem is a big step. If you have ant reservations or concerns regarding the approach I've taken, please do not hold back.

If the team agrees to take this approach we can merge this into master immediately after releasing 0.6.1 (latest actor updates).

### Goals

* Merge ruby-atomic into this gem without changing any of the actual functionality
* But update the classes to follow the "pure Ruby superclass, optimized subclass" model currently in place
* And namespace the `Atomic` class under the `Concurrent` namespace
* Configure the Java and C extensions with the expectation of adding more native classes
* Gracefully degrade gem installation on platforms that do not support native extensions
* Support cross-platform compilation with [rake-compiler](https://github.com/luislavena/rake-compiler)
* So that we can provide pre-compiled binaries [a](https://github.com/headius/ruby-atomic/issues/43) [many](https://github.com/headius/ruby-atomic/pull/45) [platforms](https://github.com/headius/ruby-atomic/issues/47) as possible
* Automate the creation of [cross-compiled](https://github.com/tjschuck/rake-compiler-dev-box) binary gems

### Things Done

* I began with by performing a `git subtree` merge of the latest [ruby-atomic](https://github.com/headius/ruby-atomic) into the local branch [merge-ruby-atomic](https://github.com/ruby-concurrency/concurrent-ruby/tree/merge-ruby-atomic)
* I then moved all the files up from the merged `ruby-atomic` sub directory up into the appropriate locations within the new gem
* To reduce the noise in the git history, I then created the [atomic-reference](https://github.com/ruby-concurrency/concurrent-ruby/tree/atomic-reference) branch from master and copied in all the new/updated files

#### Specific Code Changes

* Moved all Ruby files from ruby-atomic into the `lib/concurrent/atomic_reference` directory
* Moved the `Atomic` class under the `Concurrent` module
* Updated the Java and C extensions to reflect the new module namespace
* Renamed the Java and C extensions to reflect the move into the concurrent-ruby gem
* Created an `Atomic` class that extends `Concurrent::Atomic` and produces a deprecation warning
* Renamed the "fallback" implementation to `Concurrent::MutexAtomic`
* Renamed the Java implementation to `Concurrent::JavaAtomic`
* Renamed the C implementation to `Concurrent::CAtomic`
* Renamed the Rubinius implementation to `Concurrent::RbxAtomic`
* Updated `Concurrent::Atomic` to extend the appropriate native implementation following our normal convention
* Updated the shared Ruby code to be mixin modules instead of re-opening the class
* Updated the original `Atomic` tests from MiniTest to RSpec
* Updated the `Atomic` tests to be a shared example group
* Added tests for all platform-specific `Atomic` implementations using the shared group
* Added tests to ensure proper platform-specific subclassing
* Updated the `Rakefile` to support the new build tasks
* Updated the gemspec to support native extensions
* Updated the `extconf.rb` with the ["fake_build" hook](https://github.com/ruby-concurrency/concurrent-ruby/blob/native-atomics/ext/extconf.rb#L14) to help ensure proper installation even when compilation is not possible
* Updated the benchmark rake task and scripts to test all available implementation on whichever platform they are run on

### Benchmarks

```
Testing with ruby 2.1.2

*** Sequential updates ***
user system total real
no lock 0.010000 0.000000 0.010000 ( 0.008577)
mutex 0.050000 0.000000 0.050000 ( 0.044886)
MutexAtomic 0.180000 0.000000 0.180000 ( 0.188748)
CAtomic 0.070000 0.000000 0.070000 ( 0.062863)

*** Parallel updates ***
user system total real
no lock 0.010000 0.010000 0.020000 ( 0.017049)
mutex 0.050000 0.010000 0.060000 ( 0.053473)
MutexAtomic 0.430000 0.850000 1.280000 ( 1.066290)
CAtomic 0.070000 0.000000 0.070000 ( 0.072206)

Testing with jruby 1.9.3

*** Sequential updates ***
user system total real
no lock 0.050000 0.010000 0.060000 ( 0.050000)
mutex 0.300000 0.010000 0.310000 ( 0.247000)
MutexAtomic 1.850000 0.030000 1.880000 ( 1.417000)
JavaAtomic 0.310000 0.000000 0.310000 ( 0.238000)

*** Parallel updates ***
user system total real
no lock 0.230000 0.010000 0.240000 ( 0.143000)
mutex 0.370000 0.020000 0.390000 ( 0.256000)
MutexAtomic 1.330000 0.090000 1.420000 ( 0.877000)
JavaAtomic 0.550000 0.010000 0.560000 ( 0.330000)

Testing with rbx 2.1.0

*** Sequential updates ***
user system total real
no lock 0.030005 0.000685 0.030690 ( 0.021872)
mutex 0.295045 0.003694 0.298739 ( 0.210215)
MutexAtomic 0.621717 0.005941 0.627658 ( 0.493850)
RbxAtomic 0.198355 0.002218 0.200573 ( 0.157946)

*** Parallel updates ***
user system total real
no lock 0.064003 0.003117 0.067120 ( 0.053656)
mutex 1.342818 0.432733 1.775551 ( 1.148492)
MutexAtomic 0.600073 0.098626 0.698699 ( 0.495758)
RbxAtomic 0.189539 0.007080 0.196619 ( 0.134969)
```

### Build Boxes

To help automate the creation of packages with pre-compiled binaries, I built a custom set of [rake compiler development boxes](https://github.com/ruby-concurrency/rake-compiler-dev-box). I started by forking [rake-compiler-dev-box](https://github.com/tjschuck/rake-compiler-dev-box) but then made a bunch of changes to 1) support more target platforms, and 2) work successfully with our gem. Our [Vagrant](http://www.vagrantup.com/) configuration defines three build machines:

* Ubuntu 14.04 LTS (64 bit)
* Ubuntu 14.04 LTS (32 bit)
* Solaris 11.1 (64 bit)

The two Ubuntu boxes were build using the official Vagrant/VirtualBox images provided by Ubuntu. The Solaris build box was created using the official VirtualBox pre-build appliance provided by Oracle. I then made a few changes (mostly for SSH), packaged it for Vagrant, and added it to the index on [Vagrant Cloud](https://vagrantcloud.com/ruby-concurrency/oracle-solaris-11). With this setup we can easily build the following gem packages:

* Pure-Ruby gem with compilable extensions
* JRuby gem with pre-compiled Java extensions
* Linux 32-bit with pre-compiled binaries
* Linux 64-bit with pre-compiled binaries
* 32-bit Windows compiled for Ruby 1.9.3
* 32-bit Windows compiled for Ruby 2.0
* 64-bit Windows compiled for Ruby 2.0
* Solaris 11.1 64-bit with pre-compiled binaries

### Installation Tests

I plan to test gem installation on as many platforms as possible over the next few days. I did numerous tests on numerous operating systems as I was working on this, but I didn't keep track of them. Below is the list I plan to test against. Please feel free to test on these or any others you have access to.

#### Linux

- [ ] Ubuntu 14.04 64-bit, Ruby 1.9.3 (RVM)
- [ ] Ubuntu 14.04 64-bit, Ruby 2.0.0 (RVM)
- [ ] Ubuntu 14.04 64-bit, Ruby 2.1.2 (RVM)
- [ ] Ubuntu 14.04 64-bit, JRuby 1.7.12 (RVM)
- [ ] Ubuntu 14.04 64-bit, Rubinius 2.2.7 (RVM)
- [ ] CentOS 6.5 64-bit, Ruby 1.9.3 (RVM)
- [ ] CentOS 6.5 64-bit, Ruby 2.0.0 (RVM)
- [ ] CentOS 6.5 64-bit, Ruby 2.1.2 (RVM)
- [ ] CentOS 6.5 64-bit, JRuby 1.7.12 (RVM)
- [ ] CentOS 6.5 64-bit, Rubinius 2.2.7 (RVM)

#### Windows

- [ ] Windows 7 32-bit, Ruby 1.9.3
- [ ] Windows 7 32-bit, Ruby 2.0.0
- [ ] Windows 7 32-bit, JRuby 1.7.12
- [ ] Windows 7 64-bit, Ruby 1.9.3
- [ ] Windows 7 64-bit, Ruby 2.0.0 (32-bit)
- [ ] Windows 7 64-bit, Ruby 2.0.0 (64-bit)
- [ ] Windows 7 64-bit, JRuby 1.7.12
- [ ] Windows 8 64-bit, Ruby 1.9.3
- [ ] Windows 8 64-bit, Ruby 2.0.0 (32-bit)
- [ ] Windows 8 64-bit, Ruby 2.0.0 (64-bit)
- [ ] Windows 8 64-bit, JRuby 1.7.12
- [ ] Windows Server 2008 R2 64-bit, Ruby 1.9.3
- [ ] Windows Server 2008 R2 64-bit, Ruby 2.0.0 (32-bit)
- [ ] Windows Server 2008 R2 64-bit, Ruby 2.0.0 (64-bit)

#### Mac OS X

- [ ] OS X 10.9.3 64-bit, Ruby 1.9.3 (RVM)
- [ ] OS X 10.9.3 64-bit, Ruby 2.0.0 (RVM)
- [ ] OS X 10.9.3 64-bit, Ruby 2.1.2 (RVM)
- [ ] OS X 10.9.3 64-bit, JRuby 1.7.12 (RVM)
- [ ] OS X 10.9.3 64-bit, Rubinius 2.2.7 (RVM)
- [ ] OS X 10.9.3 64-bit, MacRuby 0.12 (installer)

#### Others

- [ ] Solaris 11.1, Ruby 1.9.3 (compiled)
- [ ] Solaris 11.1, Ruby 2.0.0 (compiled)
- [ ] Solaris 11.1, Ruby 2.1.2 (compiled)