Fidel1.1-1B-GGUF

This repository contains F16, Q8, Q5, and mixed Q4 GGUF editions of Fidel1.1-1B, together with the custom Fidel native CPU runtime required to use them.

Fidel 1.1 is a compact text-generation model from 4E-AI built with Mamba3 MIMO, attention, and a latent mixture-of-experts architecture. This GGUF edition makes the complete model available in several storage sizes for local native CPU inference.

The weights and original Fidel code are released under Apache-2.0. For Python applications using the complete FP32 model, use the companion Transformers repository.

Fidel Highlights

• Hybrid sequence modeling: Eight Mamba3 MIMO blocks and one attention block form the model's sequence-processing backbone.
• Latent MoE: Two of sixteen experts are selected per token, with shared feed-forward computation alongside the routed experts.
• Complete model: Backbone weights, coding adapters, prompt conditioners, and conversational components are included.
• Four storage options: Choose F16, Q8 with F16 embeddings, Q5, or mixed Q4 according to available storage and runtime needs.
• Native local inference: The bundled C++/GGML runtime executes without Python, PyTorch, LibTorch, or CUDA at inference time.

Model Overview

Property	Configuration
Developer	4E-AI
Model type	Text-only autoregressive language model
Architecture	Hybrid Mamba3 MIMO, attention, and latent MoE
Model family size	1B
Complete source floating-point state	Approximately 1.403 billion elements, including floating buffers
Hidden dimension	1,536
Backbone layers	9: 8 Mamba3 MIMO and 1 attention
Attention heads	16 query / 4 key-value, head dimension 96
Mamba configuration	48 heads, head dimension 64, state dimension 64, MIMO rank 4
Experts per MoE layer / selected per token	16 / 2
Latent expert / intermediate dimension	512 / 2,048
Shared feed-forward intermediate dimension	6,144
Vocabulary size	131,072
Inference activations	FP32
License	Apache-2.0

1B is the family designation. The complete source package includes model weights, adapters, floating buffers, and Boolean state. Routed-expert selection is only part of the computation; shared layers and other components remain involved.

Fidel Capabilities

Conversational text. Use the native runner to generate responses to text prompts and build local chat experiments.

Code-oriented prompting. Included coding adapters support experimenting with programming prompts, code snippets, and completion tasks. Test generated code before use.

Custom text workflows. Developers can evaluate Fidel for drafting, rewriting, summarization, and question answering over text supplied in the prompt. Application-specific behavior depends on the task and should be evaluated on representative examples.

Offline applications. Once downloaded, the model and native runtime can execute locally without a hosted model API. Text retrieval, user interfaces, output validation, and tool execution can be added by the surrounding application.

Fidel accepts text inputs. Image, audio, video, web browsing, and tool execution require separate application components.

Available GGUF Variants

File	Storage recipe	Download size
Fidel1.1-1B-F16.gguf	Eligible matrices in F16	3.000 GB
Fidel1.1-1B-Q8_0-EF16.gguf	Q8_0 matrices with F16 embeddings	1.960 GB
Fidel1.1-1B-Q5_0.gguf	Eligible matrices in Q5_0	1.280 GB
Fidel1.1-1B-Q4_0-MIXED.gguf	Q4_0 matrices, protected Q5 projections, F16 embeddings	1.437 GB
fidel-runtime-linux-x86_64.tar.gz	Native binaries, launchers, libraries, and third-party licenses	0.233 GB

Sizes are decimal GB and describe downloads, not total RAM requirements.

Q5 is the smallest file in this selection. Mixed Q4 is larger because it retains F16 embeddings and Q5 storage for selected MoE projections. Routing parameters, recurrence state, vectors, and other preserved tensors retain their specified precision. Bit labels describe the main storage recipe; they do not imply that every tensor uses the same bit width.

Local Use

Runtime compatibility

Fidel uses a custom architecture and requires the bundled Fidel runtime. Stock llama.cpp, Ollama, LM Studio, and generic GGUF loaders are not established as compatible with these files.

The prebuilt archive targets Linux x86_64, built with glibc 2.36. Use an AVX2-capable CPU for the supported arithmetic configuration. Other operating systems and CPU profiles need separate builds and checks.

The launcher selects a variant-specific binary and configures its bundled oneMKL dependencies. Keep the matching launcher for each model. CPU inference uses one sequence at a time and replays the complete prefix, so longer prompts and outputs increase computation. No incremental cache or native CUDA backend is included.

Download and run

pip install huggingface_hub
hf download 4E-AI/Fidel1.1-1B-GGUF Fidel1.1-1B-Q5_0.gguf fidel-runtime-linux-x86_64.tar.gz --local-dir fidel-gguf
cd fidel-gguf
tar -xzf fidel-runtime-linux-x86_64.tar.gz
./fidel-runtime-linux-x86_64/run.sh q5_0 --model "$PWD/Fidel1.1-1B-Q5_0.gguf" --prompt "Hi" --n-predict 32

Select the matching run.sh argument when using another file:

File variant	Launcher argument
F16	f16
Q8 with F16 embeddings	q8_0-ef16
Q5	q5_0
Mixed Q4	q4_0-mixed

The runtime archive contains native dependencies and does not require Python to run the model. Python above is used only for downloading through the Hugging Face CLI.

Customize Fidel for Your Needs

For local customization, provide clear instructions, relevant document passages, and examples of the desired response format in the prompt. Add retrieval and output validation in your application to tailor the workflow to your own data.

If you want to fine-tune Fidel for a specific coding segment—such as Python data workflows, JavaScript web development, SQL generation, or an internal codebase—start with the full-precision weights in Fidel1.1-1B. Use domain-specific examples and a Fidel-compatible fine-tuning workflow, then evaluate the adapted model on the tasks you care about. Convert and check the resulting model before deploying a customized GGUF edition.

The GGUF files provided here are intended for inference.

Usage Notes

Start with short prompts and modest generation limits. Generated text and code can be incorrect, and the model's suitability should be assessed for your application. Keep the supplied tokenizer and architecture behavior intact when integrating it.

These GGUF variants use F16/Q8/Q5/Q4 storage recipes. They are not packed NVIDIA NVFP4 files and do not require Blackwell hardware.

License

The Fidel weights and original Fidel code are licensed under Apache-2.0. Commercial use, modification, and redistribution are permitted subject to its terms. The runtime archive includes the licenses for llama.cpp and bundled Intel components; these continue to apply to their respective software. See NOTICE.

How to use with Transformers

For PyTorch/Transformers inference, load the companion SafeTensors repository, not a GGUF file. It includes the complete portable policy and custom code. Use Python 3.11, a device-appropriate torch==2.7.0 build, and:

pip install "transformers==4.57.6" safetensors accelerate

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "4E-AI/Fidel1.1-1B"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id, trust_remote_code=True, torch_dtype=torch.float32,
).eval().to(device)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello!"}],
    tokenize=False, add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(device)
with torch.inference_mode():
    output = model.generate(
        **inputs, max_new_tokens=32, do_sample=False, use_cache=False,
    )
print(tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True,
))

Review repository code before enabling trust_remote_code=True. Pin revision to a commit hash for reproducibility. Keep FP32, one unpadded sequence, one device, and caching disabled. The full Transformers weight file uses about 5.61 GB of disk space plus runtime memory. See the Transformers model card for complete architecture and device-validation details.