pinterest/secor — pull request #9 by Fluxx (2014-07-16 01:10:18+00:00)
Title: Add message key to secor.message.Message

We have a use case where we'd like to preserve the messages in Kafka logs with the same offset values. To do this, we'd like to archive not only the message payload, but also the message header data (what the kafka `Message` class seem to term the "key.")

Looking at the internals of secor, the "key" was lost when the kafka `Message` was converted to a secor `Message`. This diff adds the key to the secor `Message` class.

Few notes on this diff:

1. I didn't see any existing tests for this area of the code, so I didn't add any tests. If you would like me to do so, please let me know and I can try.
2. I'm not 100% happy with having to copy the bytes in the `messageAndOffset.message().key()` `ByteBuffer` to the `key` `byte[]`, but I just copied the implementation for the `payload` variable which uses the same technique. Let me know if this is going to be a problem.
3. We could, hypothetically, archive to a kafka log file with a "key" by calculating our own crc32 checksum and meaningless magic and attributes bytes but we'd like to try to keep the format as consistent as the *actual* log file as possible, potentially restoring our archived logs to actual broker machines.

Let me know if you have any questions. Thanks!