guicybercode/Tucano-630m-mlx-4bit

Tucano-630m MLX 4-bit

Quantized versions of TucanoBR/Tucano-630m, a Portuguese language model by TucanoBR (Nicholas Kluge Corrêa, Aniket Sen, Sophia Falk, Shiza Fatimah; paper: https://arxiv.org/abs/2411.07854). All credit for the model goes to the original authors; this repository only contains a format conversion. License: Apache-2.0, same as the original.

Versão quantizada do modelo TucanoBR/Tucano-630m para uso local; todo o crédito é dos autores originais.

Files

File	Quantization	Size
model.safetensors	4-bit (group size 64), non-quantized tensors in float16	355 MB (354639240 bytes)

How this was made

mlx-lm 0.32.2 with mlx 0.32.4 (CPU backend, Linux x86_64):

mlx_lm.convert --hf-path TucanoBR/Tucano-630m --mlx-path Tucano-630m-mlx-4bit -q --q-bits 4 --dtype float16

--dtype float16 was used because, on the MLX CPU backend on Linux, a bfloat16 conversion produced only <unk> tokens in testing, while float16 produced normal text. Not tested on Apple Silicon.

Usage

pip install mlx-lm
mlx_lm.generate --model guicybercode/Tucano-630m-mlx-4bit --prompt "A capital do Brasil é" --max-tokens 32 --temp 0

Smoke test

Prompt: A capital do Brasil é, 32 tokens, temperature 0, run on the MLX CPU backend on Linux. Only a load-and-generate check, not an evaluation.

também a cidade mais populosa da América Latina. A cidade do Rio de Janeiro é um importante centro financeiro, cultural e sede de diversos órgãos governamentais. A cidade

No benchmarks were run on this quantized model. See the original model card for the authors' evaluations and intended use.