BrunoHays/embeddinggemma-2-nullspace-audio

BrunoHays/embeddinggemma-2-nullspace-audio is google/embeddinggemma-2 with no change to the frozen weights, plus a frozen 256-dimensional null-space head that only acts on audio inputs.

• Text, image and video inputs: identical to google/embeddinggemma-2 (same code path, same outputs).
• Audio inputs: the base 768d embedding is shifted along 256 directions that the text embedding space never uses (the null space of the terminal projection). The shift is computed from two frozen Conformer intermediate layers through a PCA projection fitted only on external, non-benchmark audio.
• No training. The head has no trainable parameters at inference time. Its frozen tensors are in nullspace_audio_head.safetensors (2.37 MB).

Two operating points are submitted:

Model	γ	Notes
BrunoHays/embeddinggemma-2-nullspace-audio	1.0	headline
BrunoHays/embeddinggemma-2-nullspace-audio-g0.7	0.7	second point

Architecture

For an audio clip, with the frozen audio tower of google/embeddinggemma-2:

1. Pool two Conformer layers over time (masked):
   • X₁ = [μ₂ ; σ₂] (2048d) from layer 2 (mean and std, then L2-normalised)
   • X₂ = μ₆ (1024d) from layer 6 (mean, then L2-normalised)
2. Project each block with its frozen PCA state (soft-whitened, fitted on external corpora):

   z_k = normalize((X_k - m_k) @ P_k) / (s_k + ε),   for k in {1, 2},  P_k shape: [d_k, 128]
   

   where s_k = σ_k^(1/2) is the stored (already square-rooted) scale and ε = 10⁻⁵.
3. Combine the two 128d blocks into a unit 256d vector:

   z   = normalize([√0.5 · z_1  ;  √0.5 · z_2])      shape: [256]
   

4. Inject into the base embedding v_base (768d, L2-normalised by the model):

   v_final = normalize(v_base + γ · (z @ U_⊥^T))     shape: [768]
   

   where U_⊥ has shape [768, 256] with U_⊥^T @ U_⊥ = I.

Why the null space

U_⊥ spans the orthogonal complement of the column space of the model's terminal projection W (768 × 512). Text embeddings lie in col(W), so they are orthogonal to U_⊥. The injection therefore does not move text embeddings, and it cannot change text-to-text similarities. Its effect is limited to audio embeddings, and cross-modal rankings move only slightly (see the MSEB section).

Zero-leakage PCA construction (Protocol A)

The PCA states are fitted only on 350 clips from each of four external corpora, with no MAEB or MSEB audio:

• ylacombe/expresso (speech)
• agkphysics/AudioSet (audio events)
• lewtun/music_genres (music)
• ashraq/esc50 (environmental sounds)

Clips are at most 10 s. The per-domain statistics are combined with balanced weights, then soft whitening (σ⁻¹/²) is applied on the same eigenbasis.

Usage

The submission is implemented in gladia_embeddinggemma2_nullspace.py (an MTEB model implementation). With mteb installed and the file registered:

import mteb

model = mteb.get_model("BrunoHays/embeddinggemma-2-nullspace-audio")

Outside MTEB, load the wrapper directly:

from gladia_embeddinggemma2_nullspace import EmbeddingGemma2NullSpaceWrapper

model = EmbeddingGemma2NullSpaceWrapper(gamma=1.0)  # downloads nullspace_audio_head.safetensors from this repo

The base checkpoint is loaded from google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b. Accepting the Gemma terms on the base model is required.

MAEB results (30 tasks)

Macro average over the 30 MAEB tasks. Main scores in %.

Model	MAEB-30 (raw)	Δ vs base (raw, pp)	MAEB-30 (card basis)
google/embeddinggemma-2 (30 of 30 measured)	49.95	—	49.39
BrunoHays/embeddinggemma-2-nullspace-audio (γ = 1.0)	55.09	+5.14	54.53
BrunoHays/embeddinggemma-2-nullspace-audio-g0.7 (γ = 0.7)	54.53	+4.58	53.97

"Raw" uses the measured base score for every task. "Card basis" adds the same delta to the published model-card baseline of 49.39.

Coverage: all 30 MAEB tasks were measured for these configurations. FleursT2ARetrieval (102 languages) and CommonVoiceMini21T2ARetrieval (117 languages) were run per language and macro-averaged over languages. Their measured base scores (99.77 and 93.08) differ from the published values (97.85 and 93.25), so the measured numbers are the ones reported here.

Per-task scores

Task	google/embeddinggemma-2 (Published)	google/embeddinggemma-2 (Measured Base 768d)	Null-space γ=1.0	Null-space γ=0.7
IEMOCAPGender	66.40	66.38	85.77	84.87
VoxPopuliGenderClustering	0.05	0.05	63.09	29.63
CREMA_D	19.73	19.70	40.16	37.07
CREMA_DClustering	0.51	0.51	4.93	2.07
GTZANGenre	57.60	57.70	71.50	69.70
BeijingOpera	80.94	81.79	93.63	93.20
MridinghamTonic	31.40	31.50	39.53	38.93
BirdCLEF	15.20	15.00	20.50	20.70
CommonLanguageAgeDetection	17.51	20.09	19.46	20.29
CREMADPairClassification	52.53	52.51	57.31	56.66
VoxPopuliAccentPairClassification	51.70	51.71	59.82	58.06
VehicleSoundClustering	7.50	7.75	14.67	13.87
FSD2019Kaggle	38.75	38.75	56.26	54.62
JamAltArtistA2ARetrieval	55.38	55.20	93.82	91.11
GTZANAudioReranking	66.46	66.38	75.82	74.93
NMSQAPairClassification	98.14	98.14	78.67	90.33
MInDS14	87.71	97.63	88.34	94.76
SIBFLEURS	54.65	57.19	21.30	32.02
VoxPopuliLanguageID	99.60	99.60	97.00	99.00
VoxCelebSA	44.97	45.17	35.40	38.33
SpokenSQuADT2ARetrieval	69.33	69.33	69.33	69.33
ClothoT2ARetrieval	10.72	10.67	10.67	10.69
MACST2ARetrieval	6.62	6.87	6.87	6.87
UrbanSound8KT2ARetrieval	0.47	0.49	0.49	0.49
GigaSpeechT2ARetrieval	81.97	81.97	81.96	81.97
FleursT2ARetrieval	97.85	99.77	99.73	99.73
CommonVoiceMini21T2ARetrieval	93.25	93.08	93.08	93.08
JamAltLyricA2TRetrieval	61.51	62.78	62.89	62.81
RavdessZeroshot	13.96	13.89	13.82	13.82
SpeechCommandsZeroshotv0.02	96.91	96.88	96.86	96.88
30-Task Macro Average	49.31	49.95	55.09	54.53

Where it helps and where it hurts

The macro gain comes mainly from audio-to-audio and gender/emotion/genre tasks. Several tasks lose accuracy. The largest losses at γ = 1.0 are NMSQAPairClassification (−19.5 pp), SIBFLEURS (−35.9 pp), MInDS14 (−9.3 pp) and VoxCelebSA (−9.8 pp). Check these tasks before using the model where they matter.

MSEB results (Google SVQ, 26 locales)

Macro mean over 26 locales of google/svq (test split). Evaluated with the same head configs, using mteb/svq. Measured in this pipeline; no other MSEB tasks were run.

Task	Config	hit_rate_at_5	mrr_at_10	ndcg_at_10
SVQ A2T	base (no head)	0.94002	0.49891	0.61696
SVQ A2T	γ = 0.7	0.94033	0.49823	0.61644
SVQ A2T	γ = 1.0	0.94061	0.49810	0.61635
SVQ T2A	base (no head)	0.91263	0.50178	0.61661
SVQ T2A	γ = 0.7	0.91264	0.50180	0.61662
SVQ T2A	γ = 1.0	0.91260	0.50179	0.61662

The head moves retrieval scores by at most about 0.3 points on audio-to-text and about 0.05 points on text-to-audio. The effect is far smaller than the MAEB changes above.

Limitations

• Selection bias. The PCA construction is leak-free, but the choice of mode (ext_soft_whiten) and γ was made by looking at test scores across a small grid. Treat the MAEB numbers as optimistic. γ = 1.0 is the largest value tested for this mode, so the optimum may lie beyond it.
• Single-run numbers. All 30 MAEB and 2 MSEB tasks were measured in this pipeline. The two multilingual T2A tasks were run per language; their scores are macro means over 102 and 117 languages.
• Mixed effects. Some audio tasks regress (see above). Do not assume the gain transfers to every audio task.
• Audio batching. In the audio path, clips are encoded in dataset order without SentenceTransformer's length sorting, so results can differ from the base model by about 10⁻³ per coordinate (bf16 padding effects).

License

Gemma license, inherited from google/embeddinggemma-2. Check the Gemma terms before redistributing.

Citation

If you use this model, please cite the base model:

@misc{vera2025embeddinggemmapowerfullightweighttext,
      title={EmbeddingGemma: Powerful and Lightweight Text Representations},
      author={Vera, Henrique Schechter and others},
      year={2025},
      eprint={2509.20354},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.20354},
}