gorse-io/larger_clap_general-GGUF

larger_clap_general GGUF

GGUF conversions of LAION larger_clap_general, generated and tested with gorse-io/clap.cpp.

This checkpoint covers general audio, music and speech. Its audio and text encoders output 512-dimensional, L2-normalized embeddings in a shared space. Use the dot product for cosine similarity, audio/text retrieval or zero-shot classification. These are embedding weights, not a text-generation model or a generic llama.cpp model.

The configuration and RoBERTa tokenizer are embedded in each file. Native inference does not require Python or extra tokenizer files.

Files

File	Bytes	Decimal size
larger_clap_general-f32.gguf	777,060,832	777.1 MB
larger_clap_general-f16.gguf	389,733,856	389.7 MB
larger_clap_general-q8_0.gguf	208,325,504	208.3 MB
larger_clap_general-q4_k_m.gguf	128,926,784	128.9 MB

• F32: original floating-point tensor values.
• F16: multidimensional tensors use IEEE FP16; vectors and scalars remain F32.
• Q8_0: canonical ggml Q8_0 for eligible aligned matrices; other tensors retain original F32 bytes.
• Q4_K_M: mixed canonical Q4_K/Q6_K encoder preset. Sensitive attention-value, FFN-down and final projection roles use Q6_K; other eligible matrices use Q4_K. Unaligned rows, frontend convolutions and vectors retain F32. This encoder role policy is not llama.cpp's decoder layer-selection heuristic. No importance matrix or calibration dataset was used.

Both quantizations were generated directly from F32, not by requantizing F16. File size reduction does not imply proportional peak-RAM reduction or faster inference.

Native usage

The exact tested runtime is d1928dd6b1e0e48a072a7e0ad1b2608621616db0. No local runtime patch is required. Unlike historical clap.cpp versions, this revision's clap embedding command performs real inference; a separate clap-model executable is not used.

git clone https://github.com/gorse-io/clap.cpp.git
cd clap.cpp
git checkout d1928dd6b1e0e48a072a7e0ad1b2608621616db0
git submodule update --init --recursive
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF
cmake --build build --target clap-cli --parallel 2

hf download gorse-io/larger_clap_general-GGUF larger_clap_general-q8_0.gguf --local-dir models
./build/clap embedding --model models/larger_clap_general-q8_0.gguf \
  --threads 2 --text 'The sound of a dog barking.'
./build/clap embedding --model models/larger_clap_general-q8_0.gguf \
  --threads 2 --audio sound.wav

Each command returns one 512-element JSON array. The same commands work with F32, F16 or Q4_K_M. Native audio input is downmixed/resampled to 48 kHz. The CLAP frontend uses a ten-second window and repeat-pads shorter clips; use --crop-offset (in resampled 48 kHz samples) to explicitly select a crop for longer audio. The tests here used CPU with two threads; no GPU-performance claim is made.

Reproduce conversion

• Source revision: ada0c23a36c4e8582805bb38fec3905903f18b41.
• Source pytorch_model.bin SHA-256: 314eb00cce6ad68d25237b8446b659ccdb136ed4672c1bca470f142f72455026.
• Converter base and unmodified runtime revision: d1928dd6b1e0e48a072a7e0ad1b2608621616db0.
• converter-model-name.patch changes only general.name from the converter's hard-coded music-model name to laion/larger_clap_general. It does not change tensor mapping, values, configuration or inference code. Apply it for accurate model identity; it is a local conversion patch, not a claim of an upstream merge.

# From the pinned clap.cpp checkout above:
hf download laion/larger_clap_general \
  --revision ada0c23a36c4e8582805bb38fec3905903f18b41 --local-dir checkpoint
hf download gorse-io/larger_clap_general-GGUF converter-model-name.patch --local-dir conversion-patch
git apply conversion-patch/converter-model-name.patch
# Python conversion requires torch, transformers 4.x and numpy.
python convert_hf_to_gguf.py checkpoint larger_clap_general-f32.gguf --dtype f32
python convert_hf_to_gguf.py checkpoint larger_clap_general-f16.gguf --dtype f16
./build/clap quantize --type Q8_0 larger_clap_general-f32.gguf larger_clap_general-q8_0.gguf
./build/clap quantize --type Q4_K_M larger_clap_general-f32.gguf larger_clap_general-q4_k_m.gguf

Quantization output paths must not already exist.

Validation

All four files were exercised through the real native text and waveform paths. Each probe produced a finite, unit-normalized 512-vector. F32/F16 tensor bytes were checked against the pinned source checkpoint with the stated storage casting. Quantized files were checked for unchanged non-quantized tensor bytes, identical tokenizer/configuration, matching element counts and actual quantized tensor types. Source checkpoint identity was verified against the Hub LFS SHA-256.

The table compares normalized vectors with the original F32 Transformers implementation and, separately, the same-runtime F32 GGUF. F16 figures include storage rounding; quantized figures include quantization loss. They are not storage-rounded-reference parity claims.

Format	Probe	Cosine vs Transformers F32	Cosine vs native F32
F32	text	1.00000000	1.00000000
F32	tone440	1.00000000	1.00000000
F32	broadband	1.00000000	1.00000000
F16	text	0.99999989	0.99999989
F16	tone440	0.99999957	0.99999957
F16	broadband	0.99999975	0.99999975
Q8_0	text	0.99978555	0.99978555
Q8_0	tone440	0.99988745	0.99988744
Q8_0	broadband	0.99992027	0.99992026
Q4_K_M	text	0.99596578	0.99596578
Q4_K_M	tone440	0.99728917	0.99728919
Q4_K_M	broadband	0.99719099	0.99719103

Audio probes are one-second mono 48 kHz PCM16 440 Hz tone and seeded broadband noise, repeat-padded by the frontend. Text: “The sound of a dog barking.” These are conversion/inference smoke probes, not a retrieval-quality benchmark. Lower-precision quantization can change downstream rankings. Full per-probe metrics, tensor counts, hashes and runtime identity are in validation.json; package hashes are in SHA256SUMS.

License and attribution

LAION publishes the source weights under Apache-2.0; this conversion retains that license and involves no additional training. See the original model card for intended uses and attribution. The clap.cpp implementation has its own repository license.

Wu et al. (2022), Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation, arXiv:2211.06687.