jacobhinkle/imp — Issue #2: be mindful of alignment on the wire
Author: cscheid
Created: 2012-02-22 21:49:11+00:00
State: open

Part of changing BSON like we suggested was so we could send float arrays that could get directly addressed as a float* into the receiving buffer. (Well, javascript Typed Arrays are slightly more complicated but the idea is the same: you want to have a Float64Array pointed at the middle of some memory buffer)

However, the javascript Typed Arrays spec is fairly strict, and requires that binary data be word-aligned. If we want to have a Float32Array, that object can only point at 4-byte aligned boundaries.

So the naive idea is: if we're sending float data, we should pad the beginning of the payload, so the payload itself starts at a 4-byte boundary from the beginning of the entire message. Double data aligns at 8 bytes.

Unfortunately, doing that directly is a bad idea. Implementing this requirement by straight padding means that the wire grammar is not context-free (it depends precisely on the length of the previous message modulo 4 or modulo 8). This would be, as they say, a serious pain in the ass.

So we need to think carefully about this. Ideas?