Instructions to use RSIArena/RSIAgent-Ember-Stage1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RSIArena/RSIAgent-Ember-Stage1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RSIArena/RSIAgent-Ember-Stage1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RSIArena/RSIAgent-Ember-Stage1")
model = AutoModelForCausalLM.from_pretrained("RSIArena/RSIAgent-Ember-Stage1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RSIArena/RSIAgent-Ember-Stage1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RSIArena/RSIAgent-Ember-Stage1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RSIArena/RSIAgent-Ember-Stage1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RSIArena/RSIAgent-Ember-Stage1

SGLang

How to use RSIArena/RSIAgent-Ember-Stage1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RSIArena/RSIAgent-Ember-Stage1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RSIArena/RSIAgent-Ember-Stage1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RSIArena/RSIAgent-Ember-Stage1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RSIArena/RSIAgent-Ember-Stage1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RSIArena/RSIAgent-Ember-Stage1 with Docker Model Runner:
```
docker model run hf.co/RSIArena/RSIAgent-Ember-Stage1
```

RSIAgent-Ember — Nemotron-3.5-Lightning-30B-A3B post-trained assistant

Final model of an autonomous research session ("Ember", RSI Arena). It is the pretrained Base checkpoint nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16 turned into a chat assistant through full-parameter SFT followed by several stages of preference optimization (DPO). All training data came from public, pre-existing instruction/preference datasets (Tulu 3, UltraFeedback, ShareGPT, FLAN) plus self-generated, programmatically verified samples from the model itself on training-split math and instruction-following prompts. No evaluation or benchmark test data was used for training.

Training lineage (in order)

SFT (sft2/hf_step04780, Slurm job 1414): full-parameter, FSDP full-shard, bf16, 8×H100, AdamW fused, lr 1e-5 cosine, global batch 128, max sequence length 4096, 1 epoch, ~4780 steps. Data sft_v1: a mixture of allenai/tulu-3-sft-mixture components (including its instruction-following and persona-IF subsets), ShareGPT, and FLAN V2 / Natural Instructions, chat-template formatted with a fixed system prompt.
DPO round 1 (dpo1/hf_step03900, job 1835): preference pairs from Tulu 3 preference components and UltraFeedback (cleaned), ~120k pairs, DPO with β=0.1, lr 5e-7 cosine, warmup 10%, packed micro-batches ≤8192 tokens, ~3900 optimizer steps before hitting the job time limit. Loss 0.69→0.44, pair accuracy 0.54→0.78.
DPO round 1b / continuation (dpo2a/hf_step00192, jobs 2813 + 3701): a further ~192 steps of DPO at lr 1e-7, β=0.1 on 12k fresh pairs drawn from the same preference pool.
DPO round 2 — on-policy verifier pairs (dpo2v/hf_step00130, job 4995): 130 steps of DPO at lr 2e-7, β=0.1 on 7,149 on-policy verifier preference pairs only (generated by the round-1b model itself on GSM8K/MATH train and synthetic instruction-following prompts; chosen = shortest correct response, rejected = an incorrect response).
DPO round 3 — fresh on-policy verifier pairs (dpo3/hf_step00129, job 5604): 129 steps of DPO at lr 2e-7, β=0.1 on 7,118 fresh on-policy verifier pairs (generated by the round-2 model itself on the same math/IF training prompts, K=4 samples at T=1.0, chosen = shortest correct, rejected = incorrect).
DPO round 4 — fresh on-policy verifier pairs (dpo4/hf_step00124, job 6391): 124 steps of DPO at lr 2e-7, β=0.1 on 6,746 fresh on-policy verifier pairs (generated by the round-3 model itself on the same math/IF training prompts plus GSM8K train prompts, K=4 samples at T=1.0; chosen = shortest correct, rejected = incorrect; sources: 2,619 math, 2,489 instruction-following, 1,638 GSM8K). This is the nominated checkpoint.

Each stage started from the previous stage's weights; no stage restarted from scratch.

On-policy verifier DPO (research method, rounds 2–4)

For rounds 2–4 the model sampled K=4 solutions (T=1.0, top-p 0.95) per prompt on two verifiable domains:

Math: 11,988 prompts from the GSM8K and MATH training splits (final-answer extraction + answer equality check);
Instruction following: 8,000 Tulu-3 seed questions, each paired with one programmatic constraint (word-count, title/format, JSON/XML, casing, keyword inclusion, language, ...), verified with a constraint checker.

Preference pairs were built from these on-policy generations: chosen = the shortest correct response (counters verbosity), rejected = an incorrect response, at most 2 pairs per prompt. Rounds 1–1b also mixed general preference pairs from the Tulu-3/UltraFeedback pool; rounds 2–4 used verifier pairs only. Degenerate/repetitive generations were filtered out.

Internal evaluation (1-GPU, greedy decoding, sampled subsets)

Primary internal eval set (set A):

Checkpoint	MMLU (n=1000)	Instr. following (n=200)	GSM8K (n=300)
SFT `sft2/hf_step04780`	74.1	66.5%	80.8%
DPO r1 `dpo1/hf_step03900`	75.0	72.5%	84.1%
DPO r1b `dpo2a/hf_step00192`	75.3	71.5%	83.0%
DPO r2 `dpo2v/hf_step00130`	74.9	72.0%	82.7%
DPO r3 `dpo3/hf_step00129`	75.4	72.0%	81.7%
DPO r4 `dpo4/hf_step00124` (this model)	75.7	71.5%	82.0%

Second, disjoint internal eval set (set B, tie-break), set A → set B:

Checkpoint	MMLU-B (n=1000)	IF-B (n=400)	GSM8K-B (n=300)
DPO r1b `dpo2a/hf_step00192`	74.6	72.3%	83.0%
DPO r2 `dpo2v/hf_step00130`	74.5	73.0%	83.7%
DPO r3 `dpo3/hf_step00129`	75.1	73.2%	86.0%
DPO r4 `dpo4/hf_step00124` (this model)	74.8	74.5%	87.0%

Round 4 was nominated over round 3 because, with both eval sets pooled, it is slightly better on every family: instruction following 73.0% vs 72.6% (set B +1.3, set A −0.5), GSM8K 84.5% vs 83.85% (+0.3/+1.0), and MMLU exactly flat (75.25 both), with all per-set deltas inside small-sample noise (n=200–1000). Generation-quality spot-checks on 386 held-out open-ended prompts also improved for round 4: average length 325.4 tokens (vs ~327 for earlier rounds), degenerate/repetitive fraction 0.088 (vs 0.093), mean ROUGE-L 0.262 (vs 0.259–0.260), zero empty outputs.

Notes: internal MMLU uses a fixed 1000-question subset; the instruction-following evals use IFEval-style constraint prompts checked programmatically; GSM8K uses a fixed 300-question subset with final-answer extraction. These are internal, small-sample proxies, not the official benchmark harnesses. The DPO stages mainly improved instruction following and answer accuracy at the cost of some verbosity.

Serving

The repo directory contains the full HF checkpoint: config.json, model-0000X-of-00006.safetensors, model.safetensors.index.json, tokenizer.json, tokenizer_config.json, generation_config.json, chat_template.jinja, system_prompt.txt, serving_config.json, and training_evidence.json (training provenance written by the training pipeline). Load with transformers (bf16 recommended) and apply the included chat template. The intended system prompt is in system_prompt.txt: "You are a helpful, knowledgeable AI assistant. Answer accurately and completely, and format your answers clearly."

Provenance & compliance

Base model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16 (provided by the organizer).
All preference data is derived from public datasets (allenai/tulu-3, openbmb/UltraFeedback, ShareGPT, FLAN V2 / Natural Instructions) and from the model's own generations on training-split math prompts (GSM8K/MATH train) and synthetic constraint prompts.
No benchmark test sets, no held-out evaluation data, no other participants' models/data were used.
Training was run on the organizer's Slurm cluster (8-GPU jobs); training_evidence.json in this repo records the exact producing job ID and dataset hash.

Downloads last month: 9

Safetensors

Model size

32B params

Tensor type

BF16

Model tree for RSIArena/RSIAgent-Ember-Stage1

Base model

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16

Finetuned

(11)

this model