North-ML1/willow-alpha-gguf

Willow Alpha is a early stage version of Forge-1V

Forge-1V GGUF

This repository contains GGUF exports of the Forge-1V instruct checkpoint from North ML.

Forge-1V is a small dense Llama-compatible text model trained from scratch and then instruction-tuned for narrow coding, PyTorch, Christian/Bible Q&A, simple chat, simple writing, and beginner frontend prompts.

These files are rebuilt from the ChatML repair SFT checkpoint and include the ChatML template in GGUF metadata.

Files

• forge-1v-f16.gguf: F16 GGUF export
• forge-1v-q4_k_m.gguf: Q4_K_M quantized GGUF for practical local use

Model Details

• Architecture: Llama-compatible decoder-only dense Transformer
• Text parameters: 287.36M
• Layers: 24
• Hidden size: 1024
• Attention heads: 16
• KV heads: 4
• MLP intermediate size: 2816
• Vocab size: 16386
• Context length: 2048
• License: MIT

Vision Note

This GGUF release is text-only. The Hugging Face instruct repo includes a tiny untrained experimental vision_adapter/ scaffold for future Forge-1V experiments, but that adapter is not included in these GGUF files and the model should not be expected to view images.

Prompt Format

<|im_start|>user
Write a tiny PyTorch training loop.
<|im_end|>
<|im_start|>assistant

Recommended stop strings: <|im_end|>, <|end|>, </s>, <|user|>, <|im_start|>user.

LM Studio should use this Jinja prompt template:

{% for message in messages %}{% if message['role'] == 'system' %}<|im_start|>system
{{ message['content'] }}<|im_end|>
{% elif message['role'] == 'user' %}<|im_start|>user
{{ message['content'] }}<|im_end|>
{% elif message['role'] == 'assistant' %}<|im_start|>assistant
{{ message['content'] }}<|im_end|>
{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
{% endif %}

Run With llama.cpp

llama-cli -m forge-1v-q4_k_m.gguf -p "<|im_start|>user\nWhat is 17 + 28?<|im_end|>\n<|im_start|>assistant\n" -n 128 --temp 0.0 --top-k 1

Repair SFT Output

These GGUF files were rebuilt from the repaired instruct checkpoint. The repair pass specifically fixed the broken Gospel/chat behavior seen in the first release.

ChatML llama.cpp smoke test:

<|im_start|>user
hi<|im_end|>
<|im_start|>assistant

response:
Hi. What can I help you with?

Full-script patch output:

<|im_start|>user
I need a whole pytorch training script, not just a loop<|im_end|>
<|im_start|>assistant

response:
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset

class TinyNet(nn.Module):
    ...

<|im_start|>user
Write me a PyTorch training loop.<|im_end|>
<|im_start|>assistant

response:
model.train()
for x, y in dataloader:
    optimizer.zero_grad(set_to_none=True)
    logits = model(x)
    loss = criterion(logits, y)
    loss.backward()
    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
    optimizer.step()

<|user|>
What is the gospel?
<|assistant|>

response:
The gospel is the good news that Jesus Christ, the Son of God, died for our sins, was buried, and rose again. Through him, God offers forgiveness, reconciliation, and eternal life to all who repent and trust in Christ.

<|user|>
Write a safe Python function that checks if n is sensible.
<|assistant|>

response:
def safe_n_sensible(n: int | None) -> bool:
    if n is None:
        return False
    return isinstance(n, int) and n >= 0

Conversion Notes

Converted from the Forge-1V Hugging Face-style instruct checkpoint with llama.cpp/convert_hf_to_gguf.py.

Forge uses a custom HF Tokenizers ByteLevel BPE tokenizer. The local converter was patched to recognize Forge's tokenizer hash as a GPT-2-style ByteLevel pre-tokenizer for GGUF metadata:

tokenizer.ggml.model = gpt2
tokenizer.ggml.pre = gpt-2

The F16 model was then quantized with:

llama-quantize forge-1v-f16.gguf forge-1v-q4_k_m.gguf Q4_K_M

Limitations

Forge-1V is a narrow experimental small model. It can be useful for lightweight prompts in its training areas, but it is not a replacement for larger general assistants and can fail on complex math, broad world knowledge, and intermediate frontend tasks.