RemoraAI/remora-lite-8b-v1

Remora-Lite 8B

Remora-Lite turns a Remora record (a small JSON of on-chain facts about a pump.fun graduate, a wallet, a creator or a KOL) into a plain-English analysis. It is the open, offline half of Remora: the live scoring, daily retraining and signal feed stay on the site.

The model does not know tokens by name. Give it the record; it reads the record. Asked without one, it is trained to say so instead of inventing numbers.

What it was trained on

28,771 instruction pairs generated from Remora's own tables (September–October 2026, Solana mainnet, Helius data):

family	what the record holds	what the answer does
token post-mortem	graduation time, creator and their prior graduates, first-window buyers and SOL flows, +10 min and +1 h marks, dead-by-1h flag, socials	narrates the first hour, gives a hindsight verdict
wallet dossier	tokens traded, closed positions, win count, SOL spent/received, realized PnL, median hold	classifies the style (bot, flipper, intraday, holder) and says whether it is worth copying
creator profile	graduates in 90 days, first/last dates, base rates for factory vs one-off creators	flags launch factories
KOL call behaviour	followers, matched tokens, share bought before posting, median lead, share sold within 1 h	says whether followers are the exit liquidity
strategy explanation	first-window features, model score, rule-exit outcome	explains why a token was ranked and how it went
token naming	pump.fun name/ticker/description	writes or names in the pump.fun register
no record	question only	declines to invent figures and asks for the record

Base model Qwen3-8B, QLoRA (r=32, α=64, all attention and MLP projections), 2 epochs, 19.4M tokens, final train loss 0.97. Trained with TRL on one H100 in 76 minutes.

Use

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import json, torch

base = "Qwen/Qwen3-8B"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto"), "RemoraAI/remora-lite-8b-v1")

record = {"mint": "…", "name": "Example", "symbol": "EXM", "graduated_at": "2026-10-01 12:00 UTC",
          "creator": "…", "creator_prior_graduates_90d": 5, "first_window_seconds": 94, "first_window_buyers": 61,
          "first_window_sol_in": 212.4, "first_window_sol_out": 48.1, "return_window_end_to_10m": -0.31,
          "return_10m_to_1h": -0.72, "dead_by_1h": False, "wallets_trading_at_1h": 14}
msgs = [{"role": "system", "content": "You are Remora, an on-chain analyst for Solana memecoins. A Remora record (JSON) may be attached to the question: answer only from it, cite its numbers, and interpret them. If no record is attached, say you need the record; never invent figures."},
        {"role": "user", "content": "Was $EXM a good buy at +10 minutes?\n\nRecord:\n" + json.dumps(record)}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", enable_thinking=False).to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=300, do_sample=False)[0][ids.shape[1]:], skip_special_tokens=True))

Records in this exact shape are produced by the open Remora pipeline (remora CLI, see the GitHub repo) from Helius data; the SFT dataset shows every field.

Evaluation

Held-out records (never seen in training): share of numbers in the model's answer that appear in the reference answer: 100% on the first 12 held-out records (a larger run is pending; v0, trained without records in the prompt, scored 37% and hallucinated figures; this version reads them from the record).

Limits

• Not a price predictor and not financial advice. It explains records; it has no live data and no view of anything after its training snapshot.
• Base rates quoted in creator answers (factory tokens 2x within 1 h ≈ 10% vs 16%) are from September–October 2026 and will drift.
• English only. Trained on pump.fun/PumpSwap graduates on Solana; other chains and venues are out of distribution.

License

Apache-2.0 (adapter and dataset). Base model Qwen3-8B is Apache-2.0.