Enforce key length constraint of 2048 bytes.

Aside from needing to enforce a constraint of some sort, this change
allows us to definitively specify KeyMax, removing the previous requirement
that keys stored in the map couldn't start with \xff.
Well I did want an actual limit because it simplifies the code where we
make comparisons when addressing rows. But beyond that, the system will
certainly suffer from key sizes if they became routinely large. The
timestamp cache, for example, might end up becoming a major memory sink
with large keys.

Leveldb already does have practical limits (I think). For example, storing
values > 64k will cause them to be stored separately as actual files in the
file system! So I can't imagine you'd want to encourage applications to go
hog wild with keys anywhere near that length. Application developers will
end up doing ridiculous things, if allowed, as surely as the sun rises in
the east.

Setting practical limits, even where not strictly necessary, serves more to
steer devs in the right direction. Why doesn't your car have higher gears
in reverse? I think trying to accommodate the small set of valid use cases
you can imagine if you set your mind to it is the wrong way to approach
this exercise. Am I perhaps cynical? I feel that most devs using this
system will not have the context to evaluate the consequences of key size.
They will not be making a considered judgement. They will simply assume
that if keys can be X bytes long, then the system designers have explicitly
endorsed keys that are X bytes long as part of normal system usage. I'm
thinking 4096 bytes in light of Ben's reasonable example of URLs.

On Sunday, October 5, 2014, Ben Darnell <notifications@github.com> wrote:

> In storage/engine/keys.go:
>
> > @@ -210,12 +211,13 @@ func ValidateRangeMetaKey(key Key) error {
> >
> >  // Constants for system-reserved keys in the KV map.
> >  var (
> > +	// KeyMaxLength is the maximum key length.
> > +	KeyMaxLength = 2048
>
> Bigtable eventually lifted the whole-row size limit once they eliminated
> the cases that needed to load an entire row into memory. However, they also
> had a limit on the key size (64KB? most applications self-limited to a few
> hundred bytes) because the BT servers had to keep lots of keys in memory
> for various reasons.
>
> I think it's good to have a clearly-defined limit instead of just
> "wherever rocksdb breaks", but 2KB seems too low. In particular, it's often
> useful to use URLs as a component of your keys and urls are allowed to be
> up to 2KB (and I think that limit is increasing as older browsers age out).
> I'd make the limit at least 16KB.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/cockroachdb/cockroach/pull/106/files#r18436194>.
>