Instructions to use LoNebula/my-jarvis-72b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LoNebula/my-jarvis-72b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LoNebula/my-jarvis-72b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LoNebula/my-jarvis-72b")
model = AutoModelForCausalLM.from_pretrained("LoNebula/my-jarvis-72b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LoNebula/my-jarvis-72b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LoNebula/my-jarvis-72b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LoNebula/my-jarvis-72b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LoNebula/my-jarvis-72b

SGLang

How to use LoNebula/my-jarvis-72b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LoNebula/my-jarvis-72b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LoNebula/my-jarvis-72b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LoNebula/my-jarvis-72b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LoNebula/my-jarvis-72b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LoNebula/my-jarvis-72b with Docker Model Runner:
```
docker model run hf.co/LoNebula/my-jarvis-72b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

My-Jarvis 72B

A bilingual (Japanese / English) personal assistant model fine-tuned from Qwen2.5-72B-Instruct — the largest dense open model that can be QLoRA-trained on a single HAKUSAN GPU node (2 x A40 48GB). The weights contain no owner or user personal data: personalization starts only after each user loads the model, through onboarding questions, local conversation memory, and (optionally) a user-specific LoRA adapter trained on the user's own machine.

Model overview

Property	Details
Base model	Qwen2.5-72B-Instruct (72.7B parameters, Apache-2.0)
Method	4-bit QLoRA (NF4 + double quant, bf16 compute) on 2 x A40, then merged to bf16 on CPU (sharded 5GB)
Languages	Japanese, English
License	Apache-2.0 (merged weights; subject to the base model license)
Author	LoNebula
Weights format	Merged bfloat16, sharded safetensors, loads directly with `transformers`

Training data (public only, zero personal data)

Aya Dataset (CohereLabs/aya_dataset): Japanese and English instruction rows (Apache-2.0).
Japanese instruction corpus (izumi-lab/llm-japanese-dataset): capped sample (CC BY-SA 4.0).
Persona seed: none. PERSONA_SEED is empty by design, so the published weights carry no identity. The model is instructed to learn the user after they load it.

No owner-curated identity corpus is used. A fail-closed PII scanner runs on the upload folder before publishing and aborts the upload on any hit (emails, phone-like strings, tokens, or an owner blocklist).

Personalization starts after load (2026 best practices)

The released weights are a blank slate; personalization lives entirely on the user's device:

Onboarding — on first run the assistant asks 3 questions (name, interests, tone) and saves them to ~/.my-jarvis/profile.json.
Proactive memory (ProMem-style) — after each turn the model distills new user facts into ~/.my-jarvis/memory.jsonl; retrieved memories are re-injected into the next prompt.
Per-user adapter (PLUME/Aplaud-style, optional) — export_lora_rows() converts local memory into a LoRA dataset so each user can train their own small adapter locally; it never leaves their machine.

from my_jarvis_pipeline import load_profile, load_memory, build_messages

profile = load_profile("~/.my-jarvis/profile.json")
memory = load_memory("~/.my-jarvis/memory.jsonl", limit=6)
messages = build_messages(profile, memory, "今日の調子はどう？")

Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "LoNebula/my-jarvis-72b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "system", "content": "You are My-Jarvis. Learn the user's style and answer in their language."},
            {"role": "user", "content": "難しいバグにぶつかったら？"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
                                       return_tensors="pt").to(model.device)
print(tokenizer.decode(model.generate(inputs, max_new_tokens=512)[0][inputs.shape[-1]:],
                       skip_special_tokens=True))

72B bf16 needs ~150GB of RAM/VRAM to load; use 4-bit loading (bitsandbytes) or multiple GPUs on smaller machines, or the Inference API.

Limitations

Experimental, not benchmarked. It may be inaccurate or reproduce training text.
It starts without any user's personal knowledge; personalization happens per user at load time.
Training is capped (single-node 24h-class jobs), so this is a general bilingual assistant, not a task specialist.

License

Apache-2.0, subject to the base model license (Qwen2.5-72B-Instruct) and the training data licenses (Aya: Apache-2.0; izumi-lab: CC BY-SA 4.0).

Downloads last month: 245

Safetensors

Model size

73B params

Tensor type

BF16

Model tree for LoNebula/my-jarvis-72b

Base model

Qwen/Qwen2.5-72B

Finetuned

Qwen/Qwen2.5-72B-Instruct

Finetuned

(71)

this model

Quantizations

2 models