a22774443/qwen-benchmark-cla10

Qwen Benchmark - CLA-10

Benchmark comparing Paperclip Qwen against mission-control using Hugging Face models.

Environment

• OS: Linux (Ubuntu)
• CPU: AMD EPYC 7763 (2 vCPUs)
• GPU: None (CPU-only)
• Python: 3.12.3
• Runtime: PyTorch + Transformers on CPU

Benchmark Results

Model	Tokens/sec	Total Tokens	Elapsed (s)
sshleifer/tiny-gpt2	219.37	512	2.33
distilgpt2	15.40	512	33.25
gpt2	10.85	512	47.21

Selected Model (Most Efficient)

• Model: sshleifer/tiny-gpt2
• Throughput: 219.37 tokens/sec
• Device: CPU / torch.float32

Reproduce

# Setup
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

# Run benchmark
.venv/bin/python benchmark_hf_model.py --model sshleifer/tiny-gpt2 --max-new-tokens 64 --iterations 10 --warmup 2 --out benchmark_result.json

Files

• benchmark_hf_model.py - Benchmark script
• requirements.txt - Python dependencies
• benchmark_tiny_gpt2.json - Results for tiny-gpt2
• benchmark_distilgpt2.json - Results for distilgpt2
• benchmark_gpt2.json - Results for gpt2