RelativeDB/rt-j-fp16

RT-J — fp16 checkpoints

Half-precision conversions of stanford-star/rt-j, the Relational Transformer foundation model for in-context learning over relational databases. Produced and consumed by the RelativeDB native inference engine (cpp/rt_quantize --type f16). Weights stay f16-resident at inference: the engine's CPU (Accelerate / portable SIMD) and Metal/MPS kernels convert to fp32 inside the GEMM, halving weight memory and DRAM traffic vs. fp32 with no measurable accuracy loss (drift vs. the PyTorch reference is identical to the fp32 checkpoint on the golden batch).

Sibling repos: rt-j-int8 · rt-j-int4

File	Task head	Size
classification/model.f16.safetensors	classification / ranking (logits — apply sigmoid)	172 MB
regression/model.f16.safetensors	regression / forecasting (normalized values)	172 MB

Format

Plain safetensors: every transformer-block projection (wq/wk/wv/wg, wo, ffn.w1/w2/w3 — ~99% of parameters) is stored as F16 (IEEE half, round-to-nearest). The value/col-name encoders, decoder head, norms, biases and mask embeddings stay fp32. Any safetensors reader can load these files.

Accuracy

Golden batch (B=5, S=16) vs. the PyTorch reference, identical on CPU and Metal/MPS: yhat max abs. error 3.9e-3 — the same as the fp32 checkpoint (upstream weights are bf16, so f16 storage adds no error above fp32 op-ordering drift).

Usage (RelativeDB native engine)

./build/rt_test testdata classification/model.f16.safetensors --quantized --device mps

# via the Java / Python / Rust bindings: place the .f16 file next to the fp32
# checkpoint (or point at a directory containing it) and opt in with
export RELATIVEDB_RT_QUANTIZED=f16

The C ABI (rt_model_load) accepts these files directly — the format is auto-detected from the tensor dtypes. f16 checkpoints currently run on the CPU and Metal/MPS backends (CUDA is fp32-only).

Reproduce

cmake -B build -S cpp && cmake --build build -j
./build/rt_quantize <rt-j>/classification/model.safetensors classification/model.f16.safetensors --type f16
./build/rt_quantize <rt-j>/regression/model.safetensors regression/model.f16.safetensors --type f16

License & attribution

Derivative of stanford-star/rt-j (Stanford STAR lab), redistributed under the same CC-BY-NC-SA-4.0 license. Architecture and training details are described in the upstream model card; only the weight storage format differs here.