acrosa/kafka-rb — Issue #1: Messages do not handle non-UTF-8 compatible strings
Author: digitalex
Created: 2011-10-13 08:43:14+00:00
State: open

When a message is encoded, it is assumed that the message string can quietly be transcoded to binary (ASCII_8BIT). For strings containing characters with incompatible codepoints, this fails with the error "Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8". This example should clarify:

```ruby
producer = Kafka::Producer.new(:host => "datanode25.companybook.no")
message = Kafka:Message.new("ümlaut")
producer.send(message) # raises Encoding::CompatabilityError
```

This is due to the way the Producer#encode concatenates the message payload with the binary-encoded _magic_ and _checksum_.