plasmova/Nova-v1.1

Nova v1.1

Nova v1.1 is a decoder-only causal language model with 151 million unique parameters. This release provides the checkpoint in Hugging Face Transformers format with safetensors, a fast tokenizer, and the model code needed by AutoModelForCausalLM.

Model specifications

Specification	Value
Parameters	151M
Vocabulary	32,768 tokens
Hidden size	768
Transformer layers	20
Attention	12 query heads, 4 key/value heads
Maximum context	2,048 tokens
Weights	Float32 (safetensors)

Use with Transformers

Install PyTorch and Transformers. For GPU use, install the PyTorch build appropriate for your CUDA version first.

pip install torch transformers safetensors

Load Nova v1.1 from the Hub. The repository includes custom model code; review it before enabling remote code.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "plasmova/Nova-v1.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

system_prompt = (
    "You are Nova, a helpful and accurate assistant. Answer the user's request directly and concisely. "
    "For simple arithmetic, calculate the result and state it plainly. Do not invent names, scenarios, or equations."
)
prompt = (
    f"<|bos|><|system|>{system_prompt}<|end|>"
    "<|user|>Explain why the sky appears blue in one sentence.<|end|>"
    "<|assistant|>"
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
output = model.generate(
    **inputs,
    max_new_tokens=160,
    do_sample=True,
    temperature=0.2,
    top_p=0.9,
    top_k=40,
    repetition_penalty=1.08,
    eos_token_id=tokenizer.convert_tokens_to_ids("<|end|>"),
    pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The recommended starting settings match the local inference defaults: temperature 0.2, top-p 0.9, top-k 40, repetition penalty 1.08, and up to 160 new tokens. The local script uses a concise system instruction by default; replace it or disable it when running locally. Keep the prompt and generated output within the 2,048-token context.

Training and evaluation

The training script describes pretraining data drawn from FineWeb-Edu, DCLM, Cosmopedia-v2, FineMath, SmolTalk, OpenR1 Math, and OpenThoughts, followed by supervised fine-tuning. Dataset contents remain subject to their respective upstream terms.

No benchmark results are included for this v1.1 checkpoint. Evaluate its quality and behavior for your intended application before deployment.

License

The model weights and accompanying custom model code are released under the Apache License 2.0. Dataset contents retain their upstream licenses and terms.