MultiverseComputingCAI/Comet_60B_A12B_BF16

Comet 60B

Powered by CompactifAI

50% Fewer Parameters · Text, Image & Video Input · MTP Speculative Decoding

---

Table of Contents

• Model Overview
• License / Terms of Use
• Key Characteristics
• Quick Start
• Reasoning Control
• Tool Calling
• Training & Fine-Tuning
• Evaluation & Benchmarks
• Model Specifications
• Languages
• Safety & Limitations
• Ethical Considerations
• Model Information
• Citation

---

Model Overview

Comet 60B is a multimodal model based on NVIDIA Nemotron 3.5 Super, developed by Multiverse Computing. The original model is a 124.5B-parameter hybrid Mamba2-Transformer Mixture-of-Experts model from NVIDIA, with about 12B parameters active per token. It takes text, image and video input, produces text, and supports configurable reasoning and tool calling.

Comet applies CompactifAI compression to halve the parameter count to 61.1B while keeping about 12B parameters active per token, like the original. The layer structure, tokenizer and chat template are unchanged, and the multi-token prediction (MTP) layer is kept. After compression, the model was healed by distilling the original Nemotron 3.5 Super on text, image and video data.

• Half the memory, faster serving: the BF16 checkpoint is 122.3 GB instead of about 249 GB for Nemotron 3.5 Super, so it runs on 2× H200 instead of 4 (the FP8 version on a single H200, the NVFP4 version is 40.5 GB). On the same 4× H200, Comet serves 20–23% more output tokens per second than Nemotron 3.5 Super on NVIDIA's chat and agentic traces; the FP8 version on 2× H200 delivers about 1.8× the original's throughput per GPU (see Serving Throughput).
• Stronger agents: ahead of the original on τ²-Bench (airline, retail and telecom), AutomationBench-AA and SWE-bench Pro.
• Close elsewhere: within 3 points on 28 of 35 scores, including GPQA Diamond, AA-LCR, SWE-bench Verified and most image, document and video benchmarks.
• Same serving stack: the same vLLM reasoning and tool parsers as the base model, with MTP speculative decoding.

Available formats:

• BF16 (122.3 GB)
• FP8 (66.3 GB, for NVIDIA Hopper and Blackwell; see FP8 Quantization)
• NVFP4 (40.5 GB, for NVIDIA Blackwell; see NVFP4 Quantization)

This card covers all three versions. Unless a section says otherwise, it describes the BF16 model.

---

License / Terms of Use

Model weights: Comet 60B is a derivative of NVIDIA Nemotron 3.5 Super and is governed by the NVIDIA Nemotron Open Model License. Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.

Code: the model code shipped in this repository (modeling_*.py, configuration_*.py, image_processing_*.py) and the reasoning-parser plugin are NVIDIA code under the Apache License 2.0, as stated in their file headers.

This model is ready for commercial use under the terms above.

---

Key Characteristics

Characteristic	Description
Base model	Nemotron 3.5 Super. 124.5B total parameters (120.7B language model, 2.9B MTP, 0.8B vision encoder and projector), 12.8B activated per token: 22 of 512 routed experts per MoE layer, plus 1 shared expert. NVIDIA Nemotron Open Model License.
Comet 60B	61.1B total parameters (58.9B language model, 1.4B MTP, 0.8B vision encoder and projector), 12.3B activated per token: 22 of 256 routed experts per MoE layer, plus 1 shared expert. Activated counts include the embeddings and output head (11.2B without them; 11.7B for the base).
Architecture	Hybrid Mamba2-Transformer with latent Mixture-of-Experts (same as the base checkpoint): 88 layers (40 Mamba2, 40 MoE, 8 attention), hidden size 4,096, experts working in a 1,024-dim latent space. RADIO ViT-H/16 vision encoder (32 layers) and one MTP layer.
Modalities	Input: text, image, video (no audio). Output: text.
Context length	Evaluated with a 262,144-token context window. The configuration allows up to 1,048,576 positions; longer contexts were not evaluated.
Precision	BF16 (122.3 GB checkpoint). NVFP4 version: mixed NVFP4 / FP8 / BF16 following the layout of NVIDIA Nemotron 3.5 Super NVFP4 (40.5 GB); see NVFP4 Quantization. FP8 version: FP8 experts and Mamba projections, rest BF16, following NVIDIA's FP8 recipe for Nemotron 3 Super (66.3 GB); see FP8 Quantization.
Minimum GPUs	2× H200 (141 GB) with the full 256K context: the weights take 58 GiB per GPU and leave KV-cache room for about 43 concurrent 256K-token requests (measured). Benchmarks and the serving command below use 4× H200. FP8: a single H200 with the full 256K context: 61 GiB of weights and KV-cache room for about 20 concurrent 256K-token requests (measured). NVFP4: a single NVIDIA Blackwell GPU with native NVFP4 kernels: on one B300 the weights take 37 GiB and leave KV-cache room for about 85 concurrent 256K-token requests (measured). On Hopper (H200) it runs through weight-only FP4 (Marlin) kernels.
Reasoning	Hybrid: thinking on or off through the enable_thinking chat-template flag. See Reasoning Control.
Tool calling	Yes. Same tool-call structure and format as Nemotron 3.5 Super. See Tool Calling.
Speculative decoding	One MTP layer, compressed like the main model and distilled together with the main head.
Compression	CompactifAI (quantum-inspired compression technology)
Primary language	English

---

Quick Start

Use temperature=1.0 and top_p=0.95 for all tasks (reasoning, tool calling and chat).

vLLM Serving

Installation

All benchmarks on this card were run with the vLLM nightly 0.28.1rc1.dev628+g2a02f6efe (PyTorch 2.13, CUDA 13.0), with NVIDIA's MTP fix for hybrid Mamba models applied on top (bxyu-nvidia/vllm, branch bxyu/mtp-fix-try02, commit 64edb1d9; it changes vllm/v1/core/kv_cache_coordinator.py and vllm/v1/core/single_type_kv_cache_manager.py). The exact container we used is:

docker pull vllm/vllm-openai@sha256:96b234afa2867031ad0a226b149a06483861e613e3a3083da53398d347b5ffbd

The MTP fix is not required: the serving command below also works on the plain container, with the same answers and prefix-cache reuse in our tests.

Newer vLLM releases have not been validated with this model yet. If you use one, check that MTP speculative decoding and prefix caching work together (see the note under Serve).

Serve

Recommended serving command (4× H200, tensor parallel 4). It is the one used for the throughput numbers; the quality benchmarks used the same flags with 5 MTP draft tokens and --max-num-seqs 64, which do not change the outputs (MTP speculative decoding is lossless):

export VLLM_USE_V2_MODEL_RUNNER=0
export VLLM_SSM_CONV_STATE_LAYOUT=DS

vllm serve MultiverseComputingCAI/Comet_60B_A12B_BF16 \
  --served-model-name model \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --trust-remote-code \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype fp8 \
  --mamba-ssm-cache-dtype float32 \
  --enable-prefix-caching \
  --no-disable-hybrid-kv-cache-manager \
  --block-size 128 \
  --mamba-cache-mode align \
  --enable-chunked-prefill \
  --max-num-seqs 128 \
  --max-num-batched-tokens 33920 \
  --async-scheduling \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
  --limit-mm-per-prompt '{"image": 128, "video": 1}'

The two environment variables are required for MTP speculative decoding with this hybrid model.

Do not combine MTP with --enable-mamba-fine-grained-prefix-cache: in our tests the pair produced garbled text in about 1% of long agentic sessions (either flag alone was fine).

FP8 version: serve MMultiverseComputingCAI/Comet_60B_A12B_FP8 with the same flags. It fits on a single H200: set --tensor-parallel-size 1 and drop --enable-expert-parallel (tested on H200 with 1 and 2 GPUs).

NVFP4 version: serve MultiverseComputingCAI/Comet_60B_A12B_NVFP4 with the same flags. On B200/B300 the 40.5 GB checkpoint fits on a single GPU: set --tensor-parallel-size 1 and drop --enable-expert-parallel.

Context length is 256K above. To go up to 1M, set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 and --max-model-len 1048576 (tested: the server starts on 4× H200 with room for 31 concurrent 1M-token requests, and retrieves a fact placed anywhere in a prompt of up to ~980K tokens). Quality benchmarks above 256K were not run.

For video input, our Video-MME runs used 2 frames per second, capped at 256 frames:

  --media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}'

Request (text + image)

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

with open("chart.png", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="model",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_b64}"}},
            {"type": "text", "text": "What trend does this chart show?"},
        ],
    }],
    temperature=1.0,
    top_p=0.95,
    max_tokens=65536,
)
message = response.choices[0].message
print(getattr(message, "reasoning", None) or getattr(message, "reasoning_content", None))  # reasoning trace
print(message.content)                                                                     # final answer

A public image URL works too: pass it directly as {"url": "https://..."}. With thinking on, leave plenty of room in max_tokens: if the limit is hit before the reasoning ends, content comes back empty.

Request (video)

response = client.chat.completions.create(
    model="model",
    messages=[{
        "role": "user",
        "content": [
            {"type": "video_url", "video_url": {"url": "https://example.com/clip.mp4"}},
            {"type": "text", "text": "Summarize what happens in this video."},
        ],
    }],
    temperature=1.0,
    top_p=0.95,
    max_tokens=65536,
)
print(response.choices[0].message.content)

The server samples the video into frames; see the --media-io-kwargs note above to set the frame rate and the frame cap.

Recommended sampling: temperature=1.0, top_p=0.95. All benchmarks below used these settings.

Transformers

The examples below use the BF16 checkpoint. The FP8 and NVFP4 checkpoints are meant for vLLM or another engine that supports ModelOpt FP8 / NVFP4 checkpoints.

The model uses custom code (NemotronH_Omni_Reasoning_V3), so load it with trust_remote_code=True. The checkpoint was saved with Transformers 5.12.1.

Tested with Transformers 5.12.1 (text and image input).

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "MultiverseComputingCAI/Comet_60B_A12B_BF16"

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Write a haiku about GPUs"}]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Image input

The processor also returns bookkeeping fields (num_patches, num_tokens, imgs_sizes) that generate() does not accept; remove them before generating:

from PIL import Image

image = Image.open("chart.png")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Describe the trend in this chart."},
    ],
}]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)
for key in ("num_patches", "num_tokens", "imgs_sizes"):
    inputs.pop(key, None)
inputs = inputs.to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Video input

Pass one video as a list of PIL frames (sample them from the file first, for example at 2 frames per second). Use a {"type": "video"} placeholder in the message, build the prompt with the chat template, and give the frames to the processor:

from PIL import Image

frames = [Image.open(f"frames/{i:04d}.png") for i in range(16)]  # one video = list of PIL frames
messages = [{
    "role": "user",
    "content": [
        {"type": "video"},
        {"type": "text", "text": "Summarize what happens in this video."},
    ],
}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[prompt], videos=frames, return_tensors="pt")
for key in ("num_patches", "num_tokens", "imgs_sizes"):
    inputs.pop(key, None)
inputs = inputs.to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

For long videos or batched requests, use vLLM.

---

Thinking (Reasoning) Control

Comet 60B keeps the base model's hybrid reasoning mode: it either thinks step by step before answering (reasoning mode) or replies directly (non-reasoning mode). The enable_thinking flag in the chat template controls which, and thinking is on by default. The template also takes two more flags:

Chat-template flag	Default	Effect
enable_thinking	True	Think before answering. False starts the answer directly (the prompt ends with an empty <think></think>).
medium_effort	False	Asks for shorter reasoning by adding a reasoning-effort hint to the last user turn. In our tests it roughly halves the reasoning tokens (GPQA Diamond: 13.6K → 6.8K output tokens per answer) for a small accuracy drop (−3.5 points on GPQA Diamond, −1.4 on SciCode).
truncate_history_thinking	True	Drops the reasoning of earlier assistant turns from the prompt in multi-turn chats.

This section is a short overview. For full details, see the official Nemotron 3.5 Super model card

Transformers API

Pass enable_thinking through apply_chat_template:

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    enable_thinking=False,   # True for reasoning mode
)

vLLM

Server-level default

Set the default for all requests at startup with --default-chat-template-kwargs:

vllm serve MultiverseComputingCAI/Comet_60B_A12B_BF16 \
  --served-model-name model \
  --reasoning-parser nemotron_v3 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  ...

Per-request override

Individual requests can override the server default by passing chat_template_kwargs in the request body:

import requests

response = requests.post("http://localhost:8000/v1/chat/completions", json={
    "model": "model",
    "messages": [{"role": "user", "content": "Solve: x² - 5x + 6 = 0"}],
    "max_tokens": 65536,
    "temperature": 1.0,
    "top_p": 0.95,
    "chat_template_kwargs": {"enable_thinking": True},
})

With thinking on, a small share of responses can run very long. Set max_tokens to a limit that suits your workload. If a response hits the limit before the reasoning ends, it is returned as reasoning with empty content.

---

Tool Calling

Serve the model with --enable-auto-tool-choice --tool-call-parser qwen3_coder, as in the serving command above. The qwen3_coder parser turns the model's tool calls into standard OpenAI tool_calls:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What's the weather in Paris?"}]
response = client.chat.completions.create(
    model="model", messages=messages, tools=tools,
    temperature=1.0, top_p=0.95, max_tokens=65536,
)
message = response.choices[0].message
for call in message.tool_calls or []:
    print(call.function.name, call.function.arguments)  # get_weather {"city": "Paris", "unit": "celsius"}

# Send the tool result back and let the model continue
messages += [message, {"role": "tool", "tool_call_id": message.tool_calls[0].id, "content": '{"temp_c": 18}'}]
final = client.chat.completions.create(model="model", messages=messages, tools=tools, max_tokens=65536)
print(final.choices[0].message.content)

The model can call several tools in the same turn. Tool results are passed back to the model inside <tool_response> blocks in a user turn; the chat template does this for you.

Under the hood, the model writes each call in this format (string values as plain text, objects and lists as JSON):

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
<parameter=unit>
celsius
</parameter>
</function>
</tool_call>

---

Training & Fine-Tuning

Base Model: Nemotron 3.5 Super

Nemotron 3.5 Super is a multimodal hybrid Mamba2-Transformer Mixture-of-Experts model trained by NVIDIA. It is a single model for both reasoning and non-reasoning tasks, with reasoning set by a chat-template flag. See the original model card for details.

CompactifAI Compression

Comet 60B was compressed with CompactifAI, Multiverse Computing's compression technology based on quantum-inspired techniques. It halves the parameter count from 124.5B to 61.1B while keeping about 12B parameters active per token.

Healing

Compression is followed by knowledge distillation from the original Nemotron 3.5 Super. The original model writes the training responses, and Comet is trained toward its token-level output distribution. The healing data spans:

• single-turn text: math, science, code, reasoning, instruction following and multilingual chat;
• multi-turn agentic trajectories: terminal, software-engineering and tool-use sessions;
• images and video.

The MTP block is distilled together with the main head. All evaluations below ran with MTP speculative decoding on (5 draft tokens).

Reinforcement Learning

Reinforcement Learning targets what healing leaves behind. The model undergoes GRPO-style training on several environments to further improve on key areas like multi-turn conversation, multi-step tool use and mitigation of hallucinations, while controlling for chain of thought length and verbosity.

FP8 Quantization

The FP8 checkpoint is a post-training quantization of the BF16 model with NVIDIA Model Optimizer, following the FP8 layout of NVIDIA Nemotron 3 Super FP8 (NVIDIA has not published an FP8 version of Nemotron 3.5 Super):

Precision	Modules
FP8 (E4M3), weights and activations, one static scale per tensor	Routed experts, shared experts, Mamba2 input/output projections (main model)
BF16	Embeddings, attention q/k/v/o projections, lm_head, router gates, latent projections, Mamba2 convolutions, the whole MTP layer, vision encoder and projector
FP8 KV cache	Attention key/value cache (main model and MTP)

NVFP4 Quantization

The NVFP4 checkpoint starts from a post-training quantization (PTQ) of the BF16 model with NVIDIA Model Optimizer, using the same mixed-precision layout as NVIDIA Nemotron 3.5 Super NVFP4:

Precision	Modules
NVFP4: 4-bit E2M1 weights and activations, one FP8 (E4M3) scale per 16-value block	Routed experts and shared experts, in the main model and the MTP layer
FP8 (E4M3)	Mamba2 input/output projections, attention q/k/v/o projections, lm_head
BF16	Embeddings, router gates, latent projections, Mamba2 convolutions, MTP attention and projection, vision encoder and projector
FP8 KV cache	Attention key/value cache

The precision of every module matches NVIDIA Nemotron 3.5 Super NVFP4.

The PTQ checkpoint was then refined with quantization-aware distillation (QAD) from the original Nemotron 3.5 Super.

---

Evaluation & Benchmarks

All scores come from our internal runs. Nemotron 3.5 Super and Comet 60B were run with the same harnesses, prompts, judges and sampling settings (see Evaluation Methodology), so the columns are directly comparable. They can differ from scores published by NVIDIA or Artificial Analysis, which use their own setups.

Comet 60B is within 3 points of the original on 28 of the 35 scores below and within 5 points on 32. It is ahead on all three τ²-Bench domains, AutomationBench-AA, SciCode, SWE-bench Pro and BabyVision. The FP8 version stays within 3 points of the original on 27 of the 35 scores.

Text & Agentic

Area	Benchmark	Nemotron 3.5 Super (BF16)	Comet 60B (BF16)	∆	Comet 60B (FP8)	∆
Knowledge & reasoning	GPQA Diamond	89.90	88.64	−1.26	88.57	−1.33
	HLE (no tools)	28.59	25.35	−3.24	24.51	−4.08
	AA-Omniscience Index	5.08	2.83	−2.25	3.19	−1.89
Long context & documents	AA-LCR	68.31	67.38	−0.93	67.81	−0.50
	GDP.pdf (mean)	52.18	52.32	+0.14	51.25	−0.93
	GDP.pdf (all-pass)	6.20	5.20	−1.00	5.20	−1.00
Coding	SciCode (subtask)	45.36	46.01	+0.65	46.35	+0.99
	Terminal-Bench 2.1	60.67	55.76	−4.91	57.30	−3.37
	SWE-bench Verified	72.07	70.00	−2.07	71.00	−1.07
	SWE-bench Multilingual	67.00	66.56	−0.44	69.33	+2.33
	SWE-bench Pro	50.30	50.93	+0.63	51.48	+1.18
Agents & tool use	τ²-Bench (average)	78.93	80.79	+1.86	81.01	+2.08
	τ²-Bench airline	77.25	78.00	+0.75	77.25	+0.00
	τ²-Bench retail	82.79	85.09	+2.30	82.68	−0.11
	τ²-Bench telecom	76.75	79.28	+2.53	83.11	+6.36
	τ³-Banking	21.13	20.75	−0.38	22.68	+1.55
	GDPval-AA v2.1	74.51	69.05	−5.46	69.62	−4.89
	AutomationBench-AA	31.16	33.18	+2.02	33.38	+2.22

Vision & Video

Area	Benchmark	Nemotron 3.5 Super (BF16)	Comet 60B (BF16)	∆	Comet 60B (FP8)	∆
Image reasoning	MMMU-Pro	70.92	67.40	−3.52	66.88	−4.04
	MathVision	70.72	69.14	−1.58	69.34	−1.38
	CharXiv (reasoning)	74.00	72.20	−1.80	72.90	−1.10
	BLINK	68.96	68.40	−0.56	69.00	+0.04
	V*	90.58	84.82	−5.76	85.34	−5.24
	BabyVision	26.80	28.09	+1.29	30.93	+4.13
OCR	OCRBench v2 (EN)	67.43	66.07	−1.36	66.28	−1.15
	OCRBench v2 (CN)	58.32	55.84	−2.48	55.48	−2.84
	OCRBench v2 EN (EvalScope)	69.27	67.40	−1.87	68.14	−1.13
	OCRBench v2 CN (EvalScope)	61.27	55.88	−5.39	55.58	−5.69
	OCR-Reasoning	65.20	62.49	−2.71	59.03	−6.17
Documents	MMLongBench-Doc (accuracy)	60.77	58.61	−2.16	58.51	−2.26
	MMLongBench-Doc (F1)	61.00	58.29	−2.71	57.60	−3.40
Video	Video-MME (no subtitles)	74.48	70.96	−3.52	72.44	−2.04
	Video-MME (with subtitles)	75.59	73.26	−2.33	74.00	−1.59
	Video-MME v2 (group score)	18.26	16.82	−1.44	16.96	−1.30
	Video-MME v2 (linear score)	33.82	31.74	−2.08	33.09	−0.73

NVFP4: Comet vs NVIDIA Nemotron 3.5 Super NVFP4

Comet 60B NVFP4 (40.5 GB) against NVIDIA Nemotron 3.5 Super NVFP4 (75.1 GB), both on 1× H200 with the same harness. Comet is within 3 points on 25 of the 35 scores and within 5 points on 33. It is ahead on all three τ²-Bench domains, τ³-Banking and AutomationBench-AA. NVIDIA Nemotron 3.5 Super NVFP4 ran without MTP speculative decoding (its MTP layer does not load in this vLLM build); MTP affects speed, not scores.

Area	Benchmark	NVIDIA Nemotron 3.5 Super NVFP4	Comet 60B (NVFP4)	∆
Knowledge & reasoning	GPQA Diamond	89.46	88.38	−1.08
	HLE (no tools)	27.94	25.16	−2.78
	AA-Omniscience Index	5.38	−8.29	−13.67
Long context & documents	AA-LCR	67.19	67.00	−0.19
	GDP.pdf (mean)	52.98	50.23	−2.75
	GDP.pdf (all-pass)	7.00	4.60	−2.40
Coding	SciCode (subtask)	46.22	46.05	−0.17
	Terminal-Bench 2.1	59.55	55.76	−3.79
	SWE-bench Verified	71.47	70.00	−1.47
	SWE-bench Multilingual	67.11	65.22	−1.89
	SWE-bench Pro	51.66	50.30	−1.36
Agents & tool use	τ²-Bench (average)	77.72	78.90	+1.18
	τ²-Bench airline	78.00	79.25	+1.25
	τ²-Bench retail	81.14	83.22	+2.08
	τ²-Bench telecom	74.01	74.23	+0.22
	τ³-Banking	18.17	20.88	+2.71
	GDPval-AA v2.1	73.14	68.60	−4.54
	AutomationBench-AA	30.92	31.47	+0.55
Image reasoning	MMMU-Pro	71.21	66.30	−4.91
	MathVision	70.43	68.72	−1.71
	CharXiv (reasoning)	74.60	72.00	−2.60
	BLINK	70.01	68.64	−1.37
	V*	86.39	83.25	−3.14
	BabyVision	28.87	28.87	+0.00
OCR	OCRBench v2 (EN)	67.02	66.18	−0.84
	OCRBench v2 (CN)	58.97	55.84	−3.13
	OCRBench v2 EN (EvalScope)	69.26	67.46	−1.80
	OCRBench v2 CN (EvalScope)	59.59	56.37	−3.22
	OCR-Reasoning	66.14	59.78	−6.36
Documents	MMLongBench-Doc (accuracy)	60.26	56.60	−3.66
	MMLongBench-Doc (F1)	60.32	56.77	−3.55
Video	Video-MME (no subtitles)	74.22	71.96	−2.26
	Video-MME (with subtitles)	75.96	73.37	−2.59
	Video-MME v2 (group score)	17.16	17.04	−0.12
	Video-MME v2 (linear score)	32.13	33.33	+1.20

Serving Throughput

Output tokens per second on H200 with vLLM 0.28.1rc1.dev628+g2a02f6efe and the serving command above (MTP, 3 draft tokens), measured with NVIDIA AIPerf replaying NVIDIA's chat and agentic traces from its Dynamo recipe for Nemotron 3 Super in full. Both models ran with the same command and the same load per GPU (8 chat or 16 agentic requests per GPU).

• Chat: multi-turn conversations. Median request of about 8K input and 1K output tokens; many requests share the conversation history with earlier ones (NVIDIA's trace targets 70% KV-cache reuse).
• Agentic: agent sessions with tool calls, where each step re-sends the growing session context. Median request of about 64K input and 400 output tokens, with most of the prompt shared with the previous step (NVIDIA's trace targets 90% KV-cache reuse).

Model	GPUs (H200)	Chat (8K in / 1K out): tok/s per GPU	Agentic (64K in / 400 out): tok/s per GPU	vs. Nemotron 3.5 Super (4 GPUs)
Nemotron 3.5 Super (BF16)	4	302	655	—
Comet 60B (BF16)	4	363	805	+20% / +23%
Comet 60B (BF16)	2	368	771	+22% / +18%, on half the GPUs
Comet 60B (FP8)	2	565	1,164	+87% / +78%
Comet 60B (FP8)	1	510	1,031	+69% / +57%

Latency per request, from the same runs (TTFT = time to first token, ITL = inter-token latency; p50 / p90 over all requests):

Model	GPUs	Chat: TTFT p50 / p90	Chat: ITL p50	Chat: tok/s per user	Agentic: TTFT p50 / p90	Agentic: ITL p50	Agentic: tok/s per user
Nemotron 3.5 Super (BF16)	4	0.50 / 4.08 s	23.4 ms	42.7	0.50 / 4.25 s	22.7 ms	44.1
Comet 60B (BF16)	4	0.47 / 3.16 s	19.1 ms	52.5	0.50 / 3.59 s	18.3 ms	54.6
Comet 60B (BF16)	2	0.52 / 3.64 s	18.1 ms	55.3	0.59 / 3.78 s	19.3 ms	51.8
Comet 60B (FP8)	2	0.40 / 2.49 s	11.9 ms	84.1	0.49 / 2.32 s	12.9 ms	77.2
Comet 60B (FP8)	1	0.42 / 4.58 s	11.8 ms	84.9	0.62 / 4.37 s	13.5 ms	73.8

Against Nemotron 3.5 Super on 4× H200:

• Comet BF16 on the same 4 GPUs: TTFT p90 −23% (chat) and −16% (agentic); ITL −18% and −19%; tokens per second per user +23% and +24%.
• Comet FP8 on 2 GPUs: TTFT p90 −39% and −45%; ITL −49% and −43%; tokens per second per user +97% and +75%.

Comet's weights take half the memory, which leaves room for a larger KV cache: on 4× H200 it holds 24.0M tokens against 14.1M for Nemotron 3.5 Super, enough for 92 concurrent 256K-token requests instead of 54. More of each prompt is then served from the prefix cache.

Evaluation Methodology

Both models ran on the same stack, with the same harnesses, prompts, judges and sampling.

• Inference: vLLM 0.28.1rc1.dev628+g2a02f6efe with NVIDIA's MTP fix (see Installation), FP8 KV cache, MTP speculative decoding with 5 draft tokens. Nemotron 3.5 Super on 4× H200 (tensor parallel 4), Comet 60B on 2× H200 (BF16) or 1× H200 (FP8, NVFP4).
• Decoding: temperature 1.0, top_p 0.95, thinking on, no output cap beyond the 262,144-token context (exceptions in note 4).
• Throughput: vLLM 0.28.1rc1.dev628+g2a02f6efe (plain container, no MTP fix) with the recommended serving command; NVIDIA AIPerf 0.13 replaying the 15% subsets of NVIDIA's 8k_1k_70kv_chat (1,767 requests) and 64k_400_90kv_agent (3,410 requests) traces in full, with ignore_eos, requests longer than the 256K context removed and the prefix cache reset before each run; concurrency 32 (chat) and 64 (agentic) per 4 GPUs. Repeated runs differed by less than 2%.
• Scoring: mean over all rollouts (avg@N). No best-of-N or majority voting.
• Harnesses: text and agentic benchmarks use NeMo Gym (commit 4101fa5) with NVIDIA's Nemotron 3.5 Super settings. LLM judges and user simulators are called through OpenRouter.

Benchmark	Samples × repeats	Harness	Metric	Judge / user simulator
GPQA Diamond	198 × 8	NeMo Gym	Accuracy	—
HLE (no tools)	2,158 × 1 (text only)	NeMo Gym	Accuracy	gpt-4o
AA-Omniscience	600 × 8	NeMo Gym	Index	gemini-3-flash-preview
AA-LCR	100 × 16 (v1.1)	NeMo Gym	Accuracy	gpt-5.6-luna
GDP.pdf	100 × 5	NeMo Gym	Mean criterion score, all-pass rate	gpt-5.6-luna
SciCode	65 problems (288 subtasks) × 8	NeMo Gym	Subtask accuracy	—
Terminal-Bench 2.1	89 × 8	NeMo Gym, Terminus-2 agent	Pass rate	—
SWE-bench Verified	500 × 3	NeMo Gym, OpenCode agent	Resolve rate	—
SWE-bench Multilingual	300 × 3	NeMo Gym, OpenCode agent	Resolve rate	—
SWE-bench Pro	731 × 3	NeMo Gym, OpenCode agent	Resolve rate	—
τ²-Bench	airline 50, retail 114, telecom 114 × 8	NeMo Gym	Pass rate	gpt-6-luna (user)
τ³-Banking	97 × 8	NeMo Gym	Pass rate	gpt-6-luna (user)
GDPval-AA v2.1	220 × 1	NeMo Gym, Stirrup agent	Rubric score (0–100)	gpt-6-luna
AutomationBench-AA	600 × 1	NeMo Gym	Mean reward	—
MMMU-Pro	1,730 × 1 (10 options)	In-house	Accuracy	—
BLINK	1,901 × 1	In-house	Macro accuracy	—
V*	191 × 1	VLMEvalKit	Accuracy	—
CharXiv (reasoning)	Validation set × 1	VLMEvalKit	Accuracy	gpt-4o-mini
MathVision	Full set × 1	VLMEvalKit	Accuracy	gpt-4o-mini
OCR-Reasoning	Full set × 1	VLMEvalKit	Accuracy	gpt-4o-mini
BabyVision	388 × 1	VLMEvalKit	Accuracy	gpt-5.2
OCRBench v2 (EN, CN)	10,000 × 1	VLMEvalKit	Category score	—
OCRBench v2 (EvalScope)	10,000 × 1	EvalScope 1.5.0	Category score	—
MMLongBench-Doc	Full set × 1	VLMEvalKit	Accuracy, F1	gpt-4o
Video-MME (with and without subtitles)	2,700 × 1	lmms-eval 0.7.3, 2 fps, up to 256 frames	Accuracy	—
Video-MME v2	Full set × 1	VLMEvalKit, 2 fps, up to 256 frames	Group and linear score	—

Notes

1. AA-Omniscience and AutomationBench-AA use the public sets, not Artificial Analysis's private ones. GDPval-AA is NeMo Gym's rubric score, not AA's pairwise Elo. None of these are AA leaderboard numbers.
2. Some judges and simulators differ from NVIDIA's: AA-LCR v1.1 with gpt-5.6-luna (NVIDIA: v1.0 with a Qwen judge), and gpt-6-luna as the τ² and τ³ user (NVIDIA: gpt-5.2 and gpt-5.4-mini). These scores are not directly comparable with NVIDIA's published ones.
3. NVIDIA reports Video-MME with VLMEvalKit; we use lmms-eval.
4. Harness output caps: 131,072 tokens for AutomationBench and VLMEvalKit, 64,000 per call for the GDPval agent.
5. Single-repeat benchmarks move by 1–2 points between runs. Treat differences under about 3 points as noise.

Datasets, the NeMo Gym commit, the vLLM image and the MTP patch are all pinned, so every run can be reproduced.

---

Model Specifications

Input

• Input Type(s): Text, image, video (audio input is not supported)
• Input Format(s): String (text); RGB images (PNG, JPEG); video files sampled into frames
• Input Parameters: One-dimensional (1D) token sequences; two-dimensional (2D) images; three-dimensional (3D) video
• Other Properties Related to Input: Evaluated up to 262,144 tokens of context

Output

• Output Type(s): Text
• Output Format: String
• Output Parameters: One-dimensional (1D) token sequences
• Other Properties Related to Output: Reasoning trace (when thinking is on) followed by the final answer; tool calls in the format shown in Tool Calling

Software Integration

• Runtime Engine(s): vLLM 0.28.1rc1.dev628+g2a02f6efe (tested with and without NVIDIA's MTP fix)
• Supported Hardware: BF16: NVIDIA Hopper (H100, H200), NVIDIA Blackwell (B200, B300). FP8: NVIDIA Hopper (H100, H200), NVIDIA Blackwell (B200, B300). NVFP4: NVIDIA Blackwell (B200, B300) for native NVFP4; Hopper through weight-only FP4 kernels
• Test Hardware: benchmarks on H200 (Nemotron 3.5 Super on 4×, Comet BF16 on 2×, FP8 and NVFP4 on 1×); serving throughput on H200 (BF16, FP8); NVFP4 also served on 1× B300
• Operating System(s): Linux

Use Case

Comet 60B is a general-purpose multimodal reasoning and chat model for developers building assistants, agents, RAG systems and document, chart and video understanding applications, at about half the memory footprint of Nemotron 3.5 Super.

Model Version(s)

• v1.0.0

---

Languages

• Primary language: English

The healing data was mainly English, with a multilingual chat set added. Apart from OCRBench v2 (CN), we have not systematically evaluated languages other than English.

---

Safety & Limitations

Known Limitations

• Audio input is not supported. The processor keeps audio-related fields inherited from the Nemotron Omni architecture, but the model has no audio encoder.
• Tool calling depends on correct schema and tool design; exact parity with the original model is not guaranteed.
• Compression may affect some behaviors; evaluate for your use case.
• All limitations of the base Nemotron 3.5 Super model also apply.

Recommendations

• Validate tool outputs before running them
• Pair the model with retrieval or verification for factual, knowledge-intensive use cases
• Set max_tokens when thinking is on
• Human oversight for critical use
• Task-specific eval before production

---

Ethical Considerations

Multiverse Computing believes trustworthy AI is a shared responsibility. Developers should work with their internal model teams to make sure this model meets the requirements of their industry and use case, and addresses unforeseen product misuse. Compression and healing can change model behavior relative to the base model, including on safety-relevant prompts; evaluate before deployment and keep the guardrails of the base model in place.

All ethical considerations, safety and bias notes of the base Nemotron 3.5 Super model also apply.

---

Model Information

Field	Value
Model name	Comet 60B
Based on	Nemotron 3.5 Super
Version	v1.0.0
Parameters	61.1B total, 12.3B active
Precision	BF16 (also available in FP8 and NVFP4)
Release date	TBD
Developed by	Multiverse Computing
License	NVIDIA Nemotron Open Model License
Contact	business@multiversecomputing.com

---

Citation

If you use this model, please cite the base model and Comet 60B:

@misc{nemotron35super,
  title         = {TODO: NVIDIA Nemotron 3.5 Super technical report},
  author        = {{NVIDIA}},
  year          = {2026},
  url           = {TODO_NEMOTRON_35_SUPER_REPORT_URL}
}
@misc{comet1_60b,
  title         = {Comet 60B: Model developed from NVIDIA Nemotron 3.5 Super},
  author        = {Multiverse Computing},
  year          = {2026},
  url           = {https://huggingface.co/MultiverseComputingCAI/Comet-1.0-60B-BF16},
  note          = {Model developed based on NVIDIA Nemotron 3.5 Super using CompactifAI technology}
}

Built by Multiverse Computing · Report an issue · Discord