kreynolds/cassandra-cql — Issue #18: UTF8-encoded data produces an Encoding::CompatibilityError
Author: twhaples
Created: 2011-10-11 18:00:33+00:00
State: open

I have a CassandraCQL::Database @handle. Calling
```@handle.execute(
"insert into anonymized (KEY, ?) values (?)",
'345c1fa8-f430-11e0-8a3e-8b6c152b4342__88:45:25:BD:0E:06',
'{"some_json_in_utf8:"æ"}'
)```

results in an exception:
```Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8
/my_rvm_gemset/gems/thrift-0.6.0/lib/thrift/transport/framed_transport.rb:57:in `write'
/my_rvm_gemset/gems/thrift-0.6.0/lib/thrift/protocol/binary_protocol.rb:107:in `write_string'
/my_rvm_gemset/gems/thrift-0.6.0/lib/thrift/client.rb:35:in `write'
/my_rvm_gemset/gems/thrift-0.6.0/lib/thrift/client.rb:35:in `send_message'
/my_checked_out_gems/cassandra-cql/vendor/gen-rb/cassandra.rb:520:in `send_execute_cql_query'
/my_checked_out_gems/cassandra-cql/vendor/gen-rb/cassandra.rb:515:in `execute_cql_query'
/my_rvm_gemset/bundler/gems/thrift_client-caed05a1321c/lib/thrift_client/abstract_thrift_client.rb:150:in `handled_proxy'
/my_rvm_gemset/bundler/gems/thrift_client-caed05a1321c/lib/thrift_client/abstract_thrift_client.rb:60:in `execute_cql_query'
/my_checked_out_gems/cassandra-cql/lib/cassandra-cql/database.rb:72:in `execute_cql_query'
/my_checked_out_gems/cassandra-cql/lib/cassandra-cql/statement.rb:35:in `execute'
/my_checked_out_gems/cassandra-cql/lib/cassandra-cql/database.rb:61:in `execute'
/my_code.rb:1234:in `anonymized`
```

Here's the code where it blows up:
```
def write(buf,sz=nil)
return @transport.write(buf) unless @write
@wbuf << (sz ? buf[0...sz] : buf)
end
```
It looks like @wbuf is trying to be a buffer of binary data (?) but it is treated by Ruby as an encoded string (note the ```# encoding: ascii-8bit``` comment at the top). Interestingly, if I modify database.rb such that execute passes along ```:compression => true``` the insert is successful; presumably the value is encapsulated in a better way. (I'm still working on *retrieving* the value without corruption, though.)

I am not sure about how the data behaves once it's on the other side of the protocol, but it would probably be best if strings of any sort could be used with the cassandra-cql gem transparently, especially UTF-8 strings.