masahiroid/mdeberta-v3-base-gguf

mdeberta-v3-base — GGUF

Multilingual base encoder. Returns token-level hidden states. No task head was added; downstream fine-tuning is still needed.

Conversion of microsoft/mdeberta-v3-base at revision a0484667b22365f84929a935b5e50a51f71f159d. No fine-tuning or replacement checkpoint. F32 baseline, not a 4-bit quantization. Original authors retain credit and license rights.

Usage and compatibility

Custom GGUF schema: general.architecture=hf-native-mlx-v1, original HF tensor names, F32 tensors. Use the included MLX loader. This is not a llama.cpp/Ollama/LM Studio-compatible GGUF, and is not advertised as such. The container preserves all tensors used by the source inference model.

Apple Silicon Mac required by the included MLX runtime. Inputs may vary in length (up to the source limit); compiled Apple exports below use fixed shapes.

pip install huggingface_hub mlx numpy gguf transformers sentencepiece

from huggingface_hub import snapshot_download
from pathlib import Path
import sys, numpy as np
path = snapshot_download("masahiroid/mdeberta-v3-base-gguf")
sys.path.insert(0, path)
from runtime_mlx import Model
model = Model(path, gguf=True)
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained(path, use_fast=False)
inputs = tok("Thank you so much!", return_tensors="np", truncation=True, max_length=128)
output = model(input_ids=inputs["input_ids"], attention_mask=inputs["attention_mask"])
print(np.array(output).shape)

Numerical validation

Compared against the original PyTorch implementation, 6 deterministic cases. Maximum absolute output difference: 0.0010339618; minimum cosine similarity: 0.99999876. See validation.json for each case. These are conversion parity checks, not a full task-quality benchmark or latency benchmark.

Text tests cover English emotions, neutral retrieval text, Japanese, empty input and truncation. Chronos tests cover a trend plus sinusoid, a constant series and a series with missing observations. Only applicable tests are used for each model.

Core ML/Core AI text exports use batch 1 and sequence length 128; tokenization must pad/truncate to 128. Chronos Apple exports use batch 1 and context length 2048. MLX/custom GGUF preserve the original maximum context rather than adding training. Short Chronos inputs can be left-padded with NaN for the fixed Apple exports. Forecast horizon remains 64.

Reproducibility

Source revision and output semantics: provenance.json. Conversion scripts are included. Reproduce in a checkout with sources/<original-model-name>/, sources.json and publish/ directories as in the scripts. PyTorch 2.14.1; Transformers 4.57.3; MLX 0.32.4; NumPy 1.26.4 (MLX/Core ML environment); coremltools 9.0; chronos-forecasting 2.3.2; GGUF 0.19.0; Core AI export uses coreai-torch 0.4.3. Core AI validation uses its separate Python 3.14 environment. Version compatibility outside the tested environment is not guaranteed.

License

Weights remain under mit, with upstream terms in LICENSE. Converter/runtime additions are MIT licensed (CODE_LICENSE). No new task training, safety certification or commercial license is implied.

Other formats

• mlx
• coreml
• coreai
• gguf