MERT-v2-FullSong-GGUF

Experimental GGUF conversions of m-a-p/MERT-v2-FullSong for gorse-io/clap.cpp.

This is the actual FullSong (variant=fs) checkpoint, not MERT-v2-30s. It is an audio-only representation model, with 1024-dimensional final-layer features and no text tower. These files are for clap.cpp's custom mert2 runtime, not llama.cpp or Ollama.

Files

File	Bytes	Storage
MERT-v2-FullSong-f16.gguf	1,266,593,312	IEEE FP16 matrices/convolutions; FP32 vectors and frontend buffers
MERT-v2-FullSong-f32.gguf	2,529,780,512	FP32
MERT-v2-FullSong-q8_0.gguf	902,283,776	Canonical Q8_0 eligible dense/embedding tensors; ineligible tensors unchanged
MERT-v2-FullSong-q4_k_m.gguf	690,870,368	Encoder Q4_K_M: Q4_K plus sensitive Q6_K; ineligible tensors unchanged

All four variants contain 876 tensors. Q8_0 and Q4_K_M additions require the separately identified runtime below; the historical packaged patch alone does not add quantization support. Model files include the required frontend buffers and inference weights; no separate backbone is needed at inference.

Compatible runtime

The original F16/F32 validation used revision 9d79ade40f01e61a29756bf246a59d846e544641 plus runtime/mert2-fft-precision.patch. Those files are preserved as historical evidence, not new quantization build instructions. The new quantization runtime incorporates the FFT/F32-activation corrections; see the exact tree/binary provenance below.

Download a GGUF variant, then run (assuming it is in the current directory):

# Mean-pool final frames into an unnormalized 1024-vector:
./build/clap-model --model MERT-v2-FullSong-f16.gguf --audio music.flac --threads 2
# Return every final-layer frame:
./build/clap-model --model MERT-v2-FullSong-f16.gguf --audio music.flac --threads 2 --frames

Use clap-model, not the historical dummy clap embedding or HTTP-server interfaces. Windows multi-configuration builds place the executable in build/Release/.

Audio is decoded, mixed to mono and resampled to 24 kHz. The model processes the complete supplied waveform without an implicit 30-second crop or independent chunk aggregation. At least 1025 samples are needed at 24 kHz. Frame output is time-major with dimension 1024 and frame count floor(samples / 960) (25 Hz). --frames prints { "frames": N, "dimension": 1024, "values": [...] }; default output is a JSON array containing the arithmetic mean of final frames. Neither output is L2-normalized.

Full-context global-response normalization and bidirectional attention are retained. Attention compute grows quadratically; long recordings can require substantial CPU time and memory. Native resampling is not the Python librosa/torchaudio implementation. F16 describes weight storage, not fully half-precision computation or a guarantee of proportionally lower RAM use. The packaged patch uses operator-local FP32 operands for MERT2 F16 matrix computations.

This model is not a substitute for CLaMP3's MERT-v1-95M preprocessing: that interface expects 768-dimensional MERT-v1 features. Text input, intermediate-layer APIs, batching and streaming are outside this release's supported scope. No end-to-end GPU inference, 360-second run, real-music retrieval benchmark or full official-implementation numerical equivalence is claimed. Validation and provenance records are included for reproducibility.

Source and conversion

• Source model revision: 7001f7657276238629e61c7746ce70735e0604c0.
• Original safetensors SHA-256: e6dd2ab187d6dd62b6521cd7d8f932e237acf0c5757745a7232082e28391350d.
• Recorded converter/base runtime revision: 9d79ade40f01e61a29756bf246a59d846e544641.
• Local patch SHA-256: 31d610e32f53cc320ef86cc68e5a84aae29c461e07b950640910f653f9a56f10.

Conversion reads local safetensors/configuration and does not execute downloaded model source. Reproduce the floating-point conversion using the pinned converter and local checkpoint snapshot:

python -m pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/checkpoint MERT-v2-FullSong-f16.gguf \
  --dtype f16 --mert2-revision 7001f7657276238629e61c7746ce70735e0604c0
python convert_hf_to_gguf.py /path/to/checkpoint MERT-v2-FullSong-f32.gguf \
  --dtype f32 --mert2-revision 7001f7657276238629e61c7746ce70735e0604c0

Verify downloaded files with sha256sum -c SHA256SUMS.

Quantization runtime and measured loss

The compatible public runtime is gorse-io/clap.cpp commit b59e0cc69dfbeb03840f41e45ca5efd8edd5a18d (implementation tree 22f24b2d4caa80481604dfd980a0f912a88be25c). This is an immutable public feature-branch commit, not a claim that the pull request is merged. The artifacts and recorded native validation were produced from this exact implementation tree before the public commit existed; the later commit has the same tree. Generation and validation binary SHA-256 values remain in quantization-provenance.json. Historical float-only runtime pins are not sufficient for these quantized files.

Build on Linux with a C++17 compiler, CMake, and ICU development libraries installed (for example, libicu-dev on Debian/Ubuntu):

git clone https://github.com/gorse-io/clap.cpp.git
cd clap.cpp
git checkout b59e0cc69dfbeb03840f41e45ca5efd8edd5a18d
git submodule update --init --recursive
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCLAP_USE_SYSTEM_ICU=ON
cmake --build build --target clap-model clap-cli -j2

No historical runtime patch needs applying to this commit; the required MERT2 FFT/F32-activation corrections are included. Download a quantized GGUF from this model repository and use build/clap-model as below. To reproduce quantization from the preserved F32 file, use ./build/clap quantize INPUT_F32.gguf OUTPUT.gguf --type q8_0 or --type q4_k_m (choose a new output path).

Q4_K_M is the encoder preset q4_k_m-v2-encoder-dense-v-ffn-down-projection-q6_k, not uniform Q4_K and not llama.cpp's decoder-layer schedule. Eligible block-aligned native dense/embedding matrices use Q4_K, with all architecture-selected value/FFN-down/final projection roles protected by Q6_K. Convolutions, normalization, frontend/GRN tensors, noneligible roles and unaligned row widths retain identical source floating-point bytes. The full per-tensor type/role/raw-byte audit is recorded in provenance. Canonical padded tensor shapes are preserved; the writer can omit trailing singleton dimensions in the serialized rank (notably 24 unchanged MERT2 GRN vectors originally stored as [channels,1,1]). Every such change is recorded; float types, element counts and raw bytes remain identical. Q8_0 uses canonical Q8_0 on eligible 32-aligned rows. Quantization started from the verified preserved F32 file, not F16.

Weights remain mapped; each current dense operand is decoded into operation-local FP32 scratch with FP32 activation semantics. This is not whole-model dequantization on load, nor a claim of proportional RAM savings or faster inference. Noneligible FP16 promotion behavior remains relevant when loading F16 sources. The new artifacts were tested serially using two CPU threads with real full-checkpoint native inference on the saved waveform/text fixtures. Audio-only models have no text support.

Native commands (after building the approved runtime):

./build/clap-model --model QUANTIZED_FILE --audio music.flac --threads 2
./build/clap-model --model QUANTIZED_FILE --audio music.flac --frames --threads 2

The following are losses against the same quantization-compatible runtime loading the preserved F32 GGUF, not strict storage-rounded oracle parity and not official-model/retrieval-quality acceptance. Numerical equality is not expected. Frame-sequence metrics compare the flattened sequence, not a minimum over individual frames. Historical F16/F32 parity evidence and known narrowband/near-noise-floor limitations remain historical; no universal waveform error bound or music quality claim is made.

Preset	Cases	Minimum cosine	Maximum relative L2	Maximum absolute error
Q8_0	4	0.999691332	0.0248569624	0.0956619978
Q4_K_M	4	0.966936569	0.262170806	2.16955554

All output dimensions, flattened frame/value counts and finite-value checks passed. Norm ranges, fixture hashes, exact commands and per-case losses are in quantization-validation.json; model identity, licensing, source checkpoints, tensor audits, output sizes and hashes are in quantization-provenance.json. F16/F32 files and their original provenance/validation are not replaced.

License and attribution

The original and converted weights are licensed CC-BY-NC-4.0 (non-commercial). Conversion does not remove this restriction. The included original LICENSE and THIRD_PARTY_NOTICES.md retain upstream notices. The runtime patch is a source modification to clap.cpp and does not relicense the model weights. This is an unofficial Gorse format conversion; credit for MERT-v2 belongs to the original m-a-p/MERT-v2 authors and contributors. See the upstream model card for the original work and intended use.