Repository: cockroachdb/pebble
Pull request #3718 (2024-07-01 16:13:23)

colblk: add RawBytes, RawBytesBuilder
Add a new columnar primitive for storing an array of variable-length, unordered byte slices. The new format stores an offset table to provide readers with constant-time lookup of the position of any item in the array.

cockroach-teamcity: 

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/cockroachdb/pebble/3718)
<!-- Reviewable:end -->

RaduBerinde: *[`sstable/colblk/raw_bytes.go` line 26 at r1](https://reviewable.io/reviews/cockroachdb/pebble/3718#-O0j9UMk9eZ5PCeq0WP7:-O0udoeK7_YJgxhRnYQo:bqd16kw) ([raw file](https://github.com/cockroachdb/pebble/blob/a3fb17605c8a734940abfda47d627e1deebd9c38/sstable/colblk/raw_bytes.go#L26)):*
<details><summary><i>Previously, RaduBerinde wrote…</i></summary><blockquote>

Can we restrict things so that the total length minus the length of the last value is strictly < 64KB?  That way all *start* offsets would be 16bit (only the total length would not).
</blockquote></details>

Oh, sorry, that's exactly what Jackson said. It's conceptually simple - we serialize all start offsets.


<!-- Sent from Reviewable.io -->

RaduBerinde: *[`sstable/colblk/raw_bytes.go` line 26 at r1](https://reviewable.io/reviews/cockroachdb/pebble/3718#-O0j9UMk9eZ5PCeq0WP7:-O0ugx-h2H7IgKENSAXT:bwu6pmv) ([raw file](https://github.com/cockroachdb/pebble/blob/a3fb17605c8a734940abfda47d627e1deebd9c38/sstable/colblk/raw_bytes.go#L26)):*
<details><summary><i>Previously, RaduBerinde wrote…</i></summary><blockquote>

Hm.. instead of a specific encoding, could we reuse the uint encoding? That would automatically use 16-bits internally except in cases where we exceed the 64KB size. We could encode the lengths rather than the offsets. I think that would be a great place to start since it reuses other code.
</blockquote></details>

One nice thing about reusing the uint encoding with lengths is that it can go down to 8 bits in many cases.


<!-- Sent from Reviewable.io -->

RaduBerinde: *[`sstable/colblk/raw_bytes.go` line 26 at r1](https://reviewable.io/reviews/cockroachdb/pebble/3718#-O0j9UMk9eZ5PCeq0WP7:-O0yZY9F-qq6_gulIXzx:b9ibb83) ([raw file](https://github.com/cockroachdb/pebble/blob/a3fb17605c8a734940abfda47d627e1deebd9c38/sstable/colblk/raw_bytes.go#L26)):*
<details><summary><i>Previously, jbowens (Jackson Owens) wrote…</i></summary><blockquote>

I was thinking about the same thing yesterday, and we could reuse it. There's a bit of a complication when we introduce the prefix-compression [PrefixBytes type](https://github.com/jbowens/pebble/blob/22e01ea6ebff6558d281b9076a6746a21a2d3c8f/sstable/colblk/column_bytes.go#L20) and its associated writer. In the wip branch, PrefixBytes have the same layout as RawBytes—it just encodes more slices than rows in order to encode prefixes for the entire column and each "bundle" of K keys.

The issue is that we don't know the actual offsets when we're building the column. Every offset can be changed by a reduction in the block-level prefix, and all the offsets that make up a bundle can be changed by a reduction in the bundle's prefix. We need the Size calls to be accurate as we build the column, so the prototype opts for committing to whether an offset gets encoded in 16 bits up front. This means we'll sometimes encode an offset that ultimately does fit in 16-bits in 32-bits. This scheme doesn't really jive with reusing the UintBuilder.

As I'm typing this though, I'm realizing the reduction in the block prefix when you add a key that has a shorter matching prefix could be modeled as reducing the 'base' constant of the uint column, or during building, the "minimum". The `PrefixBytesBuilder` already needs to compute an accurate maximal offset during `Size` operations, and this is the "maximum."
</blockquote></details>

Another thing I thought about is that we probably want to be able to look up offsets from the raw data directly? Storing lengths would require a full pass over all the lengths to calculate the offsets. I guess we could do that when decompressing the block.


<!-- Sent from Reviewable.io -->

jbowens: TFTR!

Patch:
diff --git a/internal/binfmt/binfmt.go b/internal/binfmt/binfmt.go
index c8571c25..dc251247 100644
--- a/internal/binfmt/binfmt.go
+++ b/internal/binfmt/binfmt.go
@@ -155,6 +155,12 @@ func (f *Formatter) Pointer(off int) unsafe.Pointer {
 	return unsafe.Pointer(&f.data[f.off+off])
 }
 
+// Data returns the original data slice. Offset may be used to retrieve the
+// current offset within the slice.
+func (f *Formatter) Data() []byte {
+	return f.data
+}
+
 func (f *Formatter) newline(binaryData, comment string) {
 	f.lines = append(f.lines, [2]string{binaryData, comment})
 	f.buf.Reset()
diff --git a/sstable/colblk/raw_bytes.go b/sstable/colblk/raw_bytes.go
new file mode 100644
index 00000000..d26f5385
--- /dev/null
+++ b/sstable/colblk/raw_bytes.go
@@ -0,0 +1,186 @@
+// Copyright 2024 The LevelDB-Go and Pebble Authors. All rights reserved. Use
+// of this source code is governed by a BSD-style license that can be found in
+// the LICENSE file.
+
+package colblk
+
+import (
+	"fmt"
+	"io"
+	"unsafe"
+
+	"github.com/cockroachdb/pebble/internal/binfmt"
+)
+
+// RawBytes holds an array of byte slices, stored as a concatenated data section
+// and a series of offsets for each slice. Byte slices within RawBytes are
+// stored in their entirety without any compression, ensuring stability without
+// copying.
+//
+// # Representation
+//
+// An array of N byte slices encodes N+1 offsets. The beginning of the data
+// representation holds an offsets table, in the same encoding as a
+// DataTypeUint32 column. The Uint32 offsets may be delta-encoded to save space
+// if all offsets fit within an 8-bit or 16-bit uint. Each offset is relative to
+// the beginning of the string data section (after the offset table).
+//
+// The use of delta encoding conserves space in the common case. In the context
+// of CockroachDB, the vast majority of offsets will fit in 16-bits when using
+// 32 KiB blocks (the size in use by CockroachDB). However a single value larger
+// than 65535 bytes requires an offset too large to fit within 16 bits, in which
+// case offsets will be encoded as 32-bit integers.
+//
+//	+-------------------------------------------------------------------+
+//	|        a uint32 offsets table, possibly delta encoded,            |
+//	|                possibly padded for 32-bit alignment               |
+//	|                      (see DeltaEncoding)                          |
+//	+-------------------------------------------------------------------+
+//	|                           String Data                             |
+//	|  abcabcada....                                                    |
+//	+-------------------------------------------------------------------+
+//
+// The DeltaEncoding bits of the ColumnEncoding for a RawBytes column describes
+// the delta encoding of the offset table.
+type RawBytes struct {
+	slices  int
+	offsets UnsafeUint32s
+	start   unsafe.Pointer
+	data    unsafe.Pointer
+}
+
+// MakeRawBytes constructs an accessor for an array of byte slices constructed
+// by RawBytesBuilder. Count must be the number of byte slices within the array.
+func MakeRawBytes(count int, b []byte, offset uint32, enc ColumnEncoding) RawBytes {
+	if count == 0 {
+		return RawBytes{}
+	}
+	dataOff, offsets := readUnsafeIntegerSlice[uint32](count+1 /* +1 offset */, b, offset, enc.Delta())
+	return RawBytes{
+		slices:  count,
+		offsets: offsets,
+		start:   unsafe.Pointer(&b[offset]),
+		data:    unsafe.Pointer(&b[dataOff]),
+	}
+}
+
+func defaultSliceFormatter(x []byte) string {
+	return string(x)
+}
+
+func rawBytesToBinFormatter(
+	f *binfmt.Formatter, count int, enc ColumnEncoding, sliceFormatter func([]byte) string,
+) {
+	if sliceFormatter == nil {
+		sliceFormatter = defaultSliceFormatter
+	}
+
+	rb := MakeRawBytes(count, f.Data(), uint32(f.Offset()), enc)
+	dataOffset := uint64(f.Offset()) + uint64(uintptr(rb.data)-uintptr(rb.start))
+	f.CommentLine("RawBytes")
+	f.CommentLine("Offsets table")
+	uintsToBinFormatter(f, count+1, ColumnDesc{DataType: DataTypeUint32, Encoding: enc}, func(offset uint64) string {
+		return fmt.Sprintf("%d [%d overall]", offset, offset+dataOffset)
+	})
+	f.CommentLine("Data")
+	for i := 0; i < rb.slices; i++ {
+		s := rb.At(i)
+		f.HexBytesln(len(s), "data[%d]: %s", i, sliceFormatter(s))
+	}
+}
+
+func (b RawBytes) ptr(offset int) unsafe.Pointer {
+	return unsafe.Pointer(uintptr(b.data) + uintptr(offset))
+}
+
+func (b RawBytes) slice(start, end int) []byte {
+	return unsafe.Slice((*byte)(b.ptr(start)), end-start)
+}
+
+// At returns the []byte at index i. The returned slice should not be mutated.
+func (b RawBytes) At(i int) []byte {
+	return b.slice(int(b.offsets.At(i)), int(b.offsets.At(i+1)))
+}
+
+// Slices returns the number of []byte slices encoded within the RawBytes.
+func (b RawBytes) Slices() int {
+	return b.slices
+}
+
+// RawBytesBuilder encodes a column of byte slices.
+type RawBytesBuilder struct {
+	rows    int
+	data    []byte
+	offsets UintBuilder[uint32]
+}
+
+// Assert that *RawBytesBuilder implements ColumnWriter.
+var _ ColumnWriter = (*RawBytesBuilder)(nil)
+
+// Reset resets the builder to an empty state, preserving the existing bundle
+// size.
+func (b *RawBytesBuilder) Reset() {
+	b.rows = 0
+	b.data = b.data[:0]
+	b.offsets.Reset()
+	// Add an initial offset of zero to streamline the logic in RawBytes.At() to
+	// avoid needing a special case for row 0.
+	b.offsets.Set(0, 0)
+}
+
+// NumColumns implements ColumnWriter.
+func (b *RawBytesBuilder) NumColumns() int { return 1 }
+
+// Put appends the provided byte slice to the builder.
+func (b *RawBytesBuilder) Put(s []byte) {
+	b.data = append(b.data, s...)
+	b.rows++
+	b.offsets.Set(b.rows, uint32(len(b.data)))
+}
+
+// PutConcat appends a single byte slice formed by the concatenation of the two
+// byte slice arguments.
+func (b *RawBytesBuilder) PutConcat(s1, s2 []byte) {
+	b.data = append(append(b.data, s1...), s2...)
+	b.rows++
+	b.offsets.Set(b.rows, uint32(len(b.data)))
+}
+
+// Finish writes the serialized byte slices to buf starting at offset. The buf
+// slice must be sufficiently large to store the serialized output. The caller
+// should use [Size] to size buf appropriately before calling Finish.
+func (b *RawBytesBuilder) Finish(
+	col int, rows int, offset uint32, buf []byte,
+) (uint32, ColumnDesc) {
+	desc := ColumnDesc{DataType: DataTypeBytes, Encoding: EncodingDefault}
+	if rows == 0 {
+		return 0, desc
+	}
+	dataLen := b.offsets.Get(rows)
+	dataOffset, offsetColumnDesc := b.offsets.Finish(0, rows+1, offset, buf)
+	desc.Encoding = offsetColumnDesc.Encoding
+	// Copy the data section.
+	endOffset := dataOffset + uint32(copy(buf[dataOffset:], b.data[:dataLen]))
+	return endOffset, desc
+}
+
+// Size computes the size required to encode the byte slices beginning in a
+// buffer at the provided offset. The offset is required to ensure proper
+// alignment. The returned uint32 is the offset of the first byte after the end
+// of the encoded data. To compute the size in bytes, subtract the [offset]
+// passed into Size from the returned offset.
+func (b *RawBytesBuilder) Size(rows int, offset uint32) uint32 {
+	if rows == 0 {
+		return 0
+	}
+	// Get the size needed to encode the rows+1 offsets.
+	offset = b.offsets.Size(rows+1, offset)
+	// Add the value of offset[rows] since that is the accumulated size of the
+	// first [rows] slices.
+	return offset + b.offsets.Get(rows)
+}
+
+// WriteDebug implements Encoder.
+func (b *RawBytesBuilder) WriteDebug(w io.Writer, rows int) {
+	fmt.Fprintf(w, "bytes: %d rows set; %d bytes in data", b.rows, len(b.data))
+}
diff --git a/sstable/colblk/uints.go b/sstable/colblk/uints.go
index 0679405e..44323ce0 100644
--- a/sstable/colblk/uints.go
+++ b/sstable/colblk/uints.go
@@ -133,6 +133,12 @@ func (b *UintBuilder[T]) Reset() {
 	b.delta.width = 0
 }
 
+// Get gets the value of the provided row index. The provided row must have been
+// Set.
+func (b *UintBuilder[T]) Get(row int) T {
+	return b.array.elems.At(row)
+}
+
 // Set sets the value of the provided row index to v.
 func (b *UintBuilder[T]) Set(row int, v T) {
 	if b.array.n <= row {
@@ -330,8 +336,18 @@ func deltaWidth(delta uint64) uint32 {
 	}
 }
 
-func uintsToBinFormatter(f *binfmt.Formatter, rows int, desc ColumnDesc) {
+func uintsToBinFormatter(
+	f *binfmt.Formatter, rows int, desc ColumnDesc, uintFormatter func(uint64) string,
+) {
+	if uintFormatter == nil {
+		uintFormatter = func(v uint64) string { return fmt.Sprint(v) }
+	}
+
 	logicalWidth := desc.DataType.uintWidth()
+	if off := align(f.Offset(), int(logicalWidth)); off != f.Offset() {
+		f.CommentLine("Padding")
+		f.HexBytesln(off-f.Offset(), "aligning to %d-bit boundary", logicalWidth*8)
+	}
 	elementWidth := int(logicalWidth)
 	if desc.Encoding.Delta() != DeltaEncodingNone {
 		f.HexBytesln(int(logicalWidth), "%d-bit constant: %d", logicalWidth*8, f.PeekUint(int(logicalWidth)))
@@ -351,6 +367,6 @@ func uintsToBinFormatter(f *binfmt.Formatter, rows int, desc ColumnDesc) {
 		}
 	}
 	for i := 0; i < rows; i++ {
-		f.HexBytesln(elementWidth, "data[%d] = %d", i, f.PeekUint(elementWidth))
+		f.HexBytesln(elementWidth, "data[%d] = %s", i, uintFormatter(f.PeekUint(elementWidth)))
 	}
 }
diff --git a/sstable/colblk/unsafe_slice.go b/sstable/colblk/unsafe_slice.go
index e7b663a0..6eae96db 100644
--- a/sstable/colblk/unsafe_slice.go
+++ b/sstable/colblk/unsafe_slice.go
@@ -67,6 +67,37 @@ type UnsafeIntegerSlice[T constraints.Integer] struct {
 	deltaWidth uintptr
 }
 
+func readUnsafeIntegerSlice[T constraints.Integer](
+	rows int, b []byte, off uint32, delta DeltaEncoding,
+) (endOffset uint32, slice UnsafeIntegerSlice[T]) {
+	off = align(off, uint32(unsafe.Sizeof(T(0))))
+
+	fullWidthSlice := makeUnsafeIntegerSlice[T](0, unsafe.Pointer(&b[off]), int(unsafe.Sizeof(T(0))))
+	switch delta {
+	case DeltaEncodingNone:
+		slice = fullWidthSlice
+		off += uint32(unsafe.Sizeof(T(0))) * uint32(rows)
+	case DeltaEncodingConstant:
+		off += uint32(unsafe.Sizeof(T(0)))
+		slice = makeUnsafeIntegerSlice[T](fullWidthSlice.At(0), unsafe.Pointer(&b[off]), 0)
+	case DeltaEncodingUint8:
+		off += uint32(unsafe.Sizeof(T(0)))
+		slice = makeUnsafeIntegerSlice[T](fullWidthSlice.At(0), unsafe.Pointer(&b[off]), 1)
+		off += uint32(rows)
+	case DeltaEncodingUint16:
+		off += uint32(unsafe.Sizeof(T(0)))
+		slice = makeUnsafeIntegerSlice[T](fullWidthSlice.At(0), unsafe.Pointer(&b[off]), align16)
+		off += uint32(rows) * align16
+	case DeltaEncodingUint32:
+		off += uint32(unsafe.Sizeof(T(0)))
+		slice = makeUnsafeIntegerSlice[T](fullWidthSlice.At(0), unsafe.Pointer(&b[off]), align32)
+		off += uint32(rows) * align32
+	default:
+		panic("unreachable")
+	}
+	return off, slice
+}
+
 func makeUnsafeIntegerSlice[T constraints.Integer](
 	base T, deltaPtr unsafe.Pointer, deltaWidth int,
 ) UnsafeIntegerSlice[T] {


Test patch:
diff --git a/sstable/colblk/raw_bytes_test.go b/sstable/colblk/raw_bytes_test.go
new file mode 100644
index 00000000..1cf0c7b4
--- /dev/null
+++ b/sstable/colblk/raw_bytes_test.go
@@ -0,0 +1,140 @@
+// Copyright 2024 The LevelDB-Go and Pebble Authors. All rights reserved. Use
+// of this source code is governed by a BSD-style license that can be found in
+// the LICENSE file.
+
+package colblk
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"strings"
+	"testing"
+
+	"github.com/cockroachdb/datadriven"
+	"github.com/cockroachdb/pebble/internal/aligned"
+	"github.com/cockroachdb/pebble/internal/binfmt"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/exp/rand"
+)
+
+func TestRawBytes(t *testing.T) {
+	var out bytes.Buffer
+	var builder RawBytesBuilder
+	var rawBytes RawBytes
+	datadriven.RunTest(t, "testdata/raw_bytes", func(t *testing.T, td *datadriven.TestData) string {
+		out.Reset()
+		switch td.Cmd {
+		case "build":
+			builder.Reset()
+
+			var startOffset int
+			td.ScanArgs(t, "offset", &startOffset)
+
+			var count int
+			for _, k := range strings.Split(strings.TrimSpace(td.Input), "\n") {
+				builder.Put([]byte(k))
+				count++
+			}
+			td.MaybeScanArgs(t, "count", &count)
+
+			size := builder.Size(count, uint32(startOffset))
+			fmt.Fprintf(&out, "Size: %d\n", size-uint32(startOffset))
+
+			buf := aligned.ByteSlice(startOffset + int(size))
+			endOffset, desc := builder.Finish(0, count, uint32(startOffset), buf)
+
+			// Validate that builder.Size() was correct in its estimate.
+			require.Equal(t, size, endOffset)
+			f := binfmt.New(buf).LineWidth(20)
+			if startOffset > 0 {
+				f.HexBytesln(startOffset, "start offset")
+			}
+			rawBytesToBinFormatter(f, count, desc.Encoding, nil)
+			rawBytes = MakeRawBytes(count, buf[:], 0, desc.Encoding)
+			return f.String()
+		case "at":
+			var indices []int
+			td.ScanArgs(t, "indices", &indices)
+			for i, index := range indices {
+				if i > 0 {
+					fmt.Fprintln(&out)
+				}
+				fmt.Fprintf(&out, "%s", rawBytes.At(index))
+			}
+			return out.String()
+		default:
+			panic(fmt.Sprintf("unrecognized command %q", td.Cmd))
+		}
+	})
+}
+
+func BenchmarkRawBytes(b *testing.B) {
+	seed := uint64(205295296)
+	generateRandomSlices := func(sliceSize, totalByteSize int) [][]byte {
+		rng := rand.New(rand.NewSource(seed))
+		randInt := func(lo, hi int) int {
+			return lo + rng.Intn(hi-lo)
+		}
+		data := make([]byte, totalByteSize)
+		for i := range data {
+			data[i] = byte(randInt(int('a'), int('z')+1))
+		}
+		slices := make([][]byte, 0, totalByteSize/sliceSize)
+		for len(data) > 0 {
+			s := data[:min(sliceSize, len(data))]
+			slices = append(slices, s)
+			data = data[len(s):]
+		}
+		return slices
+	}
+
+	var builder RawBytesBuilder
+	var buf []byte
+	buildRawBytes := func(slices [][]byte) ([]byte, ColumnEncoding) {
+		builder.Reset()
+		for _, s := range slices {
+			builder.Put(s)
+		}
+		sz := builder.Size(len(slices), 0)
+		if cap(buf) >= int(sz) {
+			buf = buf[:sz]
+		} else {
+			buf = make([]byte, sz)
+		}
+		_, desc := builder.Finish(0, len(slices), 0, buf)
+		return buf, desc.Encoding
+	}
+
+	sizes := []int{8, 128, 1024}
+
+	b.Run("Builder", func(b *testing.B) {
+		for _, sz := range sizes {
+			b.Run(fmt.Sprintf("sliceLen=%d", sz), func(b *testing.B) {
+				slices := generateRandomSlices(sz, 32<<10 /* 32 KiB */)
+				b.ResetTimer()
+				b.SetBytes(32 << 10)
+				for i := 0; i < b.N; i++ {
+					data, _ := buildRawBytes(slices)
+					fmt.Fprint(io.Discard, data)
+				}
+			})
+		}
+	})
+	b.Run("At", func(b *testing.B) {
+		for _, sz := range sizes {
+			b.Run(fmt.Sprintf("sliceLen=%d", sz), func(b *testing.B) {
+				slices := generateRandomSlices(sz, 32<<10 /* 32 KiB */)
+				data, enc := buildRawBytes(slices)
+				rb := MakeRawBytes(len(slices), data, 0, enc)
+				b.ResetTimer()
+				b.SetBytes(32 << 10)
+				for i := 0; i < b.N; i++ {
+					for j := 0; j < len(slices); j++ {
+						fmt.Fprint(io.Discard, rb.At(j))
+					}
+				}
+			})
+		}
+	})
+}
diff --git a/sstable/colblk/testdata/raw_bytes b/sstable/colblk/testdata/raw_bytes
new file mode 100644
index 00000000..27c37854
--- /dev/null
+++ b/sstable/colblk/testdata/raw_bytes
@@ -0,0 +1,216 @@
+# Test a simple RawBytes: a single byte slice with a single 'a' byte.
+
+build offset=0
+a
+----
+# RawBytes
+# Offsets table
+0-4: x 00000000 # 32-bit constant: 0
+4-5: x 00       # data[0] = 0 [6 overall]
+5-6: x 01       # data[1] = 1 [7 overall]
+# Data
+6-7: x 61       # data[0]: a
+
+# Try the same thing, but with offsets 1, 2, 3, 4.
+
+build offset=1
+a
+----
+00-01: x 00       # start offset
+# RawBytes
+# Offsets table
+# Padding
+01-04: x 000000   # aligning to 32-bit boundary
+04-08: x 00000000 # 32-bit constant: 0
+08-09: x 00       # data[0] = 0 [10 overall]
+09-10: x 01       # data[1] = 1 [11 overall]
+# Data
+10-11: x 61       # data[0]: a
+
+build offset=2
+a
+----
+00-02: x 0000     # start offset
+# RawBytes
+# Offsets table
+# Padding
+02-04: x 0000     # aligning to 32-bit boundary
+04-08: x 00000000 # 32-bit constant: 0
+08-09: x 00       # data[0] = 0 [10 overall]
+09-10: x 01       # data[1] = 1 [11 overall]
+# Data
+10-11: x 61       # data[0]: a
+
+build offset=3
+a
+----
+00-03: x 000000   # start offset
+# RawBytes
+# Offsets table
+# Padding
+03-04: x 00       # aligning to 32-bit boundary
+04-08: x 00000000 # 32-bit constant: 0
+08-09: x 00       # data[0] = 0 [10 overall]
+09-10: x 01       # data[1] = 1 [11 overall]
+# Data
+10-11: x 61       # data[0]: a
+
+build offset=4
+a
+----
+00-04: x 00000000 # start offset
+# RawBytes
+# Offsets table
+04-08: x 00000000 # 32-bit constant: 0
+08-09: x 00       # data[0] = 0 [10 overall]
+09-10: x 01       # data[1] = 1 [11 overall]
+# Data
+10-11: x 61       # data[0]: a
+
+# Create a RawBytes with two byte slices: 'a' and 'b'.
+
+build offset=0
+a
+b
+----
+# RawBytes
+# Offsets table
+0-4: x 00000000 # 32-bit constant: 0
+4-5: x 00       # data[0] = 0 [7 overall]
+5-6: x 01       # data[1] = 1 [8 overall]
+6-7: x 02       # data[2] = 2 [9 overall]
+# Data
+7-8: x 61       # data[0]: a
+8-9: x 62       # data[1]: b
+
+build offset=0
+a
+ab
+abc
+----
+# RawBytes
+# Offsets table
+00-04: x 00000000 # 32-bit constant: 0
+04-05: x 00       # data[0] = 0 [8 overall]
+05-06: x 01       # data[1] = 1 [9 overall]
+06-07: x 03       # data[2] = 3 [11 overall]
+07-08: x 06       # data[3] = 6 [14 overall]
+# Data
+08-09: x 61       # data[0]: a
+09-11: x 6162     # data[1]: ab
+11-14: x 616263   # data[2]: abc
+
+build offset=0
+aaabbbc
+----
+# RawBytes
+# Offsets table
+00-04: x 00000000       # 32-bit constant: 0
+04-05: x 00             # data[0] = 0 [6 overall]
+05-06: x 07             # data[1] = 7 [13 overall]
+# Data
+06-13: x 61616162626263 # data[0]: aaabbbc
+
+build offset=0
+a
+ab
+abc
+----
+# RawBytes
+# Offsets table
+00-04: x 00000000 # 32-bit constant: 0
+04-05: x 00       # data[0] = 0 [8 overall]
+05-06: x 01       # data[1] = 1 [9 overall]
+06-07: x 03       # data[2] = 3 [11 overall]
+07-08: x 06       # data[3] = 6 [14 overall]
+# Data
+08-09: x 61       # data[0]: a
+09-11: x 6162     # data[1]: ab
+11-14: x 616263   # data[2]: abc
+
+at indices=(0, 1, 2)
+----
+a
+ab
+abc
+
+build offset=0
+cat
+orange
+zucchini
+lemon
+apple
+banana
+cantelope
+lettuce
+kale
+----
+# RawBytes
+# Offsets table
+00-04: x 00000000           # 32-bit constant: 0
+04-05: x 00                 # data[0] = 0 [14 overall]
+05-06: x 03                 # data[1] = 3 [17 overall]
+06-07: x 09                 # data[2] = 9 [23 overall]
+07-08: x 11                 # data[3] = 17 [31 overall]
+08-09: x 16                 # data[4] = 22 [36 overall]
+09-10: x 1b                 # data[5] = 27 [41 overall]
+10-11: x 21                 # data[6] = 33 [47 overall]
+11-12: x 2a                 # data[7] = 42 [56 overall]
+12-13: x 31                 # data[8] = 49 [63 overall]
+13-14: x 35                 # data[9] = 53 [67 overall]
+# Data
+14-17: x 636174             # data[0]: cat
+17-23: x 6f72616e6765       # data[1]: orange
+23-31: x 7a75636368696e69   # data[2]: zucchini
+31-36: x 6c656d6f6e         # data[3]: lemon
+36-41: x 6170706c65         # data[4]: apple
+41-47: x 62616e616e61       # data[5]: banana
+47-56: x 63616e74656c6f7065 # data[6]: cantelope
+56-63: x 6c657474756365     # data[7]: lettuce
+63-67: x 6b616c65           # data[8]: kale
+
+at indices=(0, 1, 2, 3, 4, 5, 6, 7, 8)
+----
+cat
+orange
+zucchini
+lemon
+apple
+banana
+cantelope
+lettuce
+kale
+
+# Only serialize the first 4 byte slices.
+
+build offset=0 count=4
+cat
+orange
+zucchini
+lemon
+apple
+banana
+cantelope
+lettuce
+kale
+----
+# RawBytes
+# Offsets table
+00-04: x 00000000         # 32-bit constant: 0
+04-05: x 00               # data[0] = 0 [9 overall]
+05-06: x 03               # data[1] = 3 [12 overall]
+06-07: x 09               # data[2] = 9 [18 overall]
+07-08: x 11               # data[3] = 17 [26 overall]
+08-09: x 16               # data[4] = 22 [31 overall]
+# Data
+09-12: x 636174           # data[0]: cat
+12-18: x 6f72616e6765     # data[1]: orange
+18-26: x 7a75636368696e69 # data[2]: zucchini
+26-31: x 6c656d6f6e       # data[3]: lemon
+
+at indices=(0, 1, 2, 3)
+----
+cat
+orange
+zucchini
+lemon
diff --git a/sstable/colblk/uints_test.go b/sstable/colblk/uints_test.go
index f53125e8..30ae0578 100644
--- a/sstable/colblk/uints_test.go
+++ b/sstable/colblk/uints_test.go
@@ -108,7 +108,7 @@ func TestUints(t *testing.T) {
 					_, desc := writers[wIdx].Finish(0, rows, 0, buf)
 					fmt.Fprintf(&out, "b%d: %T:\n", width, writers[wIdx])
 					f := binfmt.New(buf).LineWidth(20)
-					uintsToBinFormatter(f, rows, desc)
+					uintsToBinFormatter(f, rows, desc, nil)
 					fmt.Fprintf(&out, "%s", f.String())
 				} else {
 					fmt.Fprintf(&out, "Keeping b%d open\n", width)