iconara/ione — pull request #8 by mimperatore (2014-09-14 01:01:15+00:00)
Title: Fix race condition by synchronizing access/modification of connection @state

I looks like BaseConnection#close can be triggered more than once under certain race condition situations.

I see this issue pretty reliably (10% of the time, roughly, on my system) with the following test code:
```
require 'cassandra' # version 1.0.0.beta.2 modified to use ione v 1.2.0-pre3.

NHOSTS = 6.freeze
KEYSPACE_NAME = 'test'.freeze
TABLENAME = 'users'.freeze

hosts = (1..NHOSTS).map { |i| "192.168.200.1#{i}" }

puts "Connecting to #{hosts}"
Cluster = Cassandra.connect(hosts: hosts)

Cluster.each_host do |host|
puts "Host #{host.ip}: id=#{host.id} datacenter=#{host.datacenter} rack=#{host.rack}"
end

def setup_keyspace
session = Cluster.connect

unless Cluster.has_keyspace?(KEYSPACE_NAME)
session.execute("CREATE KEYSPACE #{KEYSPACE_NAME} WITH replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 3};")
end

session.execute("USE #{KEYSPACE_NAME}")

keyspace = Cluster.keyspace(KEYSPACE_NAME)

unless keyspace.has_table?(TABLENAME)
session.execute("CREATE TABLE #{TABLENAME} ( username text primary key );")
end

session.close
end

def insert(range)
session = Cluster.connect

session.execute("USE #{KEYSPACE_NAME}")
insert_stmt = session.prepare("INSERT INTO #{TABLENAME} (username) VALUES (?);")

futures = []
range.each do |n|
future = session.execute_async(insert_stmt, "marco-#{n}")
future.on_failure { |error| puts error.inspect }
futures << future
end

futures.each { |f| f.join rescue nil }
session.close
end

setup_keyspace
insert(1..10)
```

When I run this test, Ione::Io::BaseConnection#close seems to be called twice occasionally on the same connection, with the @state attribute's value != :closed. When this happens, ```@closed_promise.fulfill(self)``` also gets called twice, which results in a ```Future already completed``` error the second time around.

These are the stacktraces of the two threads which end up calling BaseConnection#close:

```
/Workspace/ione/lib/ione/io/io_reactor.rb:349:in `block in close_sockets'
/Workspace/ione/lib/ione/io/io_reactor.rb:347:in `each'
/Workspace/ione/lib/ione/io/io_reactor.rb:347:in `close_sockets'
/Workspace/ione/lib/ione/io/io_reactor.rb:130:in `ensure in block in start'
/Workspace/ione/lib/ione/io/io_reactor.rb:136:in `block in start'
```

```
/Workspace/ruby-driver/lib/cassandra/protocol/cql_protocol_handler.rb:178:in `close'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:262:in `block (3 levels) in close_connections'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:260:in `each'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:260:in `block (2 levels) in close_connections'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:259:in `each'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:259:in `block in close_connections'
/Users/marcoimperatore/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:258:in `close_connections'
/Workspace/ruby-driver/lib/cassandra/cluster/client.rb:97:in `close'
/Workspace/ruby-driver/lib/cassandra/session.rb:170:in `close_async'
/Workspace/ruby-driver/lib/cassandra/session.rb:187:in `close'
```