miloyip/rapidjson — pull request #131 by Kosta-Github (2014-09-01 09:58:08+00:00)
Title: short string optimization

Since the payload (the `Data` union) of the current implementation of `GenericValue` is `12 bytes` (32 bit mode) or `16 bytes` (64 bit mode) it could store `UTF8`-encoded strings up to `11` or `15` chars plus the `terminating zero` character plus the actual string length.

``` C++
struct ShortString {
enum { MaxChars = sizeof(String) / sizeof(Ch), MaxSize = MaxChars - 1, LenPos = MaxSize };
Ch str[MaxChars];

inline static bool Usable(SizeType len) { return (MaxSize >= len); }
inline void SetLength(SizeType len) { str[LenPos] = (Ch)(MaxSize - len); }
inline SizeType GetLength() const { return (SizeType)(MaxSize - str[LenPos]); }
}; // at most as many bytes as "String" above => 12 bytes in 32-bit mode, 16 bytes in 64-bit mode
```

The `ShortString` can represent zero-terminated strings up to `MaxSize` chars (excluding the terminating zero) and store a value to determine the length of the contained string in the last character `str[LenPos]` by storing `MaxSize - length` there. If the string to store has the maximal length of `MaxSize` (excluding the terminating zero) then `str[LenPos]` will store `0` and therefore act as the string terminator as well. For getting the string length back from that value just use `MaxSize - str[LenPos]`.

This allows to store `11`-chars strings in 32-bit mode and `15`-chars strings in 64-bit mode inline (for `UTF8`-encoded strings).

The integration into `GenericValue` is done by introducing additional `kInlineStrFlag` and `kShortStringFlag` flags. When setting a new string value in `SetStringRaw(s, alloc)` it is first checked if the string is short enough to fit into the `inline string buffer` and if so the given source string will be copied into the new `ShortString` target instead of allocating additional memory for it.