Harsha001/LFM3-1.7B-ROCmFPX

LFM3-1.7B, ROCmFPX GGUF

LiquidAI/LFM3-1.7B, Liquid AI's 1.7B text model for local agents, tool calling and structured output (tri-hybrid: gated short convolutions plus sliding-window and global attention, 256K context), quantized for the ROCmFPX llama.cpp fork (low-bit weight types for AMD GPUs, Vulkan and HIP/ROCm).

The recommended 6-bit file needs HarshaVardhan3002/ROCmFPX. Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load it. The Q8_0 and Q4_K_M files use stock types and load in any llama.cpp build with LFM3 support. These files were checked on the CPU and have not been tested on AMD hardware yet.

Not an official release of Liquid AI. All credit for the model goes to its authors.

Files

File	Size	Weights	KL divergence vs BF16 (lower is better)	Same top token as BF16
LFM3-1.7B-Q6_K_ROCMFPX.gguf (recommended)	1.43 GB	ROCmFPX 6-bit (Q6_0_ROCMFPX with tensor scales)	0.0132	94.3%
LFM3-1.7B-Q8_0.gguf	1.85 GB	stock llama.cpp Q8_0	0.0031	97.2%
LFM3-1.7B-Q4_K_M.gguf	1.10 GB	stock llama.cpp Q4_K_M	0.0922	85.5%
stock Q6_K, for comparison (not uploaded)	1.43 GB	stock llama.cpp Q6_K	0.0160	93.8%

• Measured on: 40 chunks of 512 tokens of WikiText-2 (test split), against the BF16 GGUF, CPU.
• The ROCmFPX 6-bit file beats stock Q6_K at the same size. Its file name says Q6_K_ROCMFPX so that Hugging Face lists it as a 6-bit quant; the weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock Q6_K.
• No ROCmFPX 4-bit file: our 4-bit types came out clearly worse than stock Q4_K_M on this model (KL divergence 1.5 to 1.7 times higher), so we ship Q4_K_M instead.
• Every file passed a short greedy answer test before upload.

How these were made

1. BF16 GGUF converted with the fork's convert_hf_to_gguf.py.
2. Importance matrix from the BF16 file over a general calibration text (calibration_datav3), 512-token chunks.
3. llama-quantize --imatrix from the BF16 file for every file, stock ones included. The 6-bit file was made with LLAMA_ROCMFPX_TENSOR_SCALE=1, which stores a power-of-two scale per weight tensor (the runtime applies it; no setting is needed to run the file), with --token-embedding-type q6_K.

Run

git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build -j
build/bin/llama-server -m LFM3-1.7B-Q6_K_ROCMFPX.gguf -ngl 99 --jinja -c 32768 \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

Liquid AI recommends temperature 0.1, top-k 50, repetition penalty 1.1 for chat, tool use and agents, and temperature 1.0, top-p 0.95, top-k 20 for math. The model supports up to 256K tokens of context.

LFM3 support in the fork comes from llama.cpp pull request #31252. LFM3-Omni and LFM3-Audio need further upstream llama.cpp work and will follow later.

License

LFM Open License v1.0, same as the base model. Commercial use is allowed for companies below 10 million US dollars of annual revenue; above that it is not covered by the license. Read LICENSE before you use these files.