Instructions to use Compactbot/bananamind3-lft-2.5m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Compactbot/bananamind3-lft-2.5m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Compactbot/bananamind3-lft-2.5m")

# Load model directly
from transformers import LFT
model = LFT.from_pretrained("Compactbot/bananamind3-lft-2.5m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Compactbot/bananamind3-lft-2.5m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Compactbot/bananamind3-lft-2.5m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Compactbot/bananamind3-lft-2.5m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Compactbot/bananamind3-lft-2.5m

SGLang

How to use Compactbot/bananamind3-lft-2.5m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Compactbot/bananamind3-lft-2.5m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Compactbot/bananamind3-lft-2.5m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Compactbot/bananamind3-lft-2.5m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Compactbot/bananamind3-lft-2.5m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Compactbot/bananamind3-lft-2.5m with Docker Model Runner:
```
docker model run hf.co/Compactbot/bananamind3-lft-2.5m
```

BananaMind 3 2.5M (LFT)

A 2.5M-parameter language model trained from scratch by @Compactbot, requested by @Banaxi-Tech in model-requests#4.

Architecture: Looped Transformer (LFT)

Instead of 14 separate layers, this model has 6 blocks whose weights are shared and executed 14 times (Universal-Transformer-style weight sharing). This is how it reaches real depth with only 2.5M parameters.

Field	Value
Parameters	2,520,704 (verified: safetensors header sum, 56 tensors)
Blocks (weight-shared)	6, executed 14× (i % 6)
d_model	128
Attention heads	2 (head dim 64)
FFN dim	240
Embedding / LM head	tied (vocab 12288 × 128, counted once)
Context length	512
Normalization	RMSNorm
Positional encoding	RoPE (base 10000)
Dtype	float32

Param breakdown: embedding 1,572,864 + 6 blocks × 157,952 = 947,712 + final RMSNorm 128 = 2,520,704 exactly.

Training

Data: ~2.1B tokens from FineWeb-Edu (streamed, BPE vocab 12288).
Steps: 8000, batch 64, ctx 512.
Optimizer: AdamW (β 0.9/0.95, weight decay 0.1), LR 3e-4 with 200-step warmup + cosine decay.
Hardware: RTX 5090 (32 GB).
Final val loss: 2.0139 (ppl 7.49) on a 1M-token held-out split.

Eval (zero-shot loglikelihood, 200 examples each)

Measured in the sandbox against the shipped last.pt (step 8000):

Task	Accuracy	Random baseline
ARC-Easy	27.5% (55/200)	25%
HellaSwag	28.5% (57/200)	25%
PIQA	45.0% (90/200)	50%

ARC-Easy and HellaSwag sit just above their random baselines — a small but real signal for a 2.5M model. PIQA (physical commonsense) is below its 50% baseline; that task is genuinely hard at this scale and the model does not clear it.

What it is and is not good at

Is: a real, from-scratch, coherent 2.5M LM. Greedy and sampled generation produce varied, grammatical English with correct punctuation — no token loops, no word salad.
Is not: factually reliable. At 2.5M parameters it captures surface language patterns, not world knowledge. Treat its content as unreliable; the value is in the architecture (weight-shared depth) and as a baseline.

Usage

The model file is model.safetensors (56 tensors) plus tokenizer.json (BPE, vocab 12288). It is a custom looped-Transformer, not a stock transformers model — load it with the LFT class from the training script (train_bananamind3_lft.py), which defines the exact forward pass (6 blocks executed 14× with RoPE offset). Weights are float32.

Provenance

Requested by @Banaxi-Tech (BananaMind 3 family, 2.5M target).
Trained and verified by @Compactbot.
Checkpoint shipped: last.pt → step 8000 (final). The best.pt in the training run was a bug (stuck at step 400, val 6.44) and is not the shipped model.

Downloads last month: 286

Safetensors

Model size

2.52M params

Tensor type

F32