Instructions to use TYTTYTTYT/mgpt2-124m-mc4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TYTTYTTYT/mgpt2-124m-mc4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TYTTYTTYT/mgpt2-124m-mc4")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TYTTYTTYT/mgpt2-124m-mc4")
model = AutoModelForCausalLM.from_pretrained("TYTTYTTYT/mgpt2-124m-mc4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TYTTYTTYT/mgpt2-124m-mc4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TYTTYTTYT/mgpt2-124m-mc4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TYTTYTTYT/mgpt2-124m-mc4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TYTTYTTYT/mgpt2-124m-mc4

SGLang

How to use TYTTYTTYT/mgpt2-124m-mc4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TYTTYTTYT/mgpt2-124m-mc4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TYTTYTTYT/mgpt2-124m-mc4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TYTTYTTYT/mgpt2-124m-mc4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TYTTYTTYT/mgpt2-124m-mc4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TYTTYTTYT/mgpt2-124m-mc4 with Docker Model Runner:
```
docker model run hf.co/TYTTYTTYT/mgpt2-124m-mc4
```

MGPT-2 (124M, multilingual mC4)

A GPT-2 (124M) model pretrained from scratch on the same eight-language mC4 mix as the MIXAR pixel-based language models. It is the tokenizer-based control in the MIXAR paper: identical pretraining corpus and comparable size, differing only in the input and output interface (subword tokens instead of rendered pixels).

Code · Paper: arXiv link to follow

Results

Benchmark	Score
bAbI task 1, English, free generation	24.3
SIB-200, 22 languages	seen 77.8 / unseen 26.2 / unseen low-resource 19.3
XNLI, 10 languages	seen 72.5 / unseen 63.9
GLUE	74.0

SIB-200 (macro-F1) and XNLI (accuracy, translate-train) are finetuned per language. See the paper for the full tables and protocols.

Details


Architecture	GPT-2, 12 layers, 12 heads, hidden 768
Parameters	124M
Tokenizer	the original GPT-2 BPE vocabulary (50,257 tokens)
Pretraining data	mC4 in de, en, es, fr, it, ja, ko, zh, with the same language mix as MIXAR
Steps	1M

The tokenizer is the English GPT-2 BPE vocabulary, so non-Latin scripts are split into byte-level fragments: a Chinese bAbI prompt takes roughly 2.7 times as many tokens as its English counterpart.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("TYTTYTTYT/mgpt2-124m-mc4")
model = AutoModelForCausalLM.from_pretrained("TYTTYTTYT/mgpt2-124m-mc4")

Note that the tokenizer is the original English GPT-2 BPE vocabulary, so non-Latin scripts are segmented into byte-level fragments (a Chinese bAbI prompt takes roughly 3x more tokens than its English counterpart).

All released models

Model	Stage 1	Stage 2 (GRPO)
113M, 8x8 patches	llama-pixar-116m-p8	llama-pixar-116m-p8-stage2
116M, 32x32 patches	llama-pixar-116m-p32	llama-pixar-116m-p32-stage2
477M, 32x32 patches	llama-pixar-477m-p32	llama-pixar-477m-p32-stage2

Also released: the adversarial stage-2 ablation llama-pixar-116m-p8-stage2-gan and the tokenizer-based baseline mgpt2-124m-mc4, pretrained on the same corpus.

License

Released under the Apache License 2.0.

Citation

@article{mixar,
  title  = {{MIXAR}: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts},
  author = {Hu, Chen and Tai, Yintao and Vergari, Antonio and Keller, Frank and Suglia, Alessandro},
  year   = {2026}
}

Downloads last month: 164

Safetensors

Model size

0.1B params

Tensor type

F32

Collection including TYTTYTTYT/mgpt2-124m-mc4

MIXAR

Collection

Multilingual pixel-based language models from the MIXAR paper, plus the MGPT-2 baseline. Code: github.com/april-tools/mixar • 8 items • Updated 5 days ago