BoomJules/molly-cs-ai

Molly OS - Specialist Adapter: Computer Science - Artificial Intelligence

Frontier-distilled LoRA specialist (PEFT, rank 32; target modules q_proj, k_proj, v_proj, o_proj) for the Molly OS model-agnostic orchestration layer. Base model: meta-llama/Llama-3.1-8B-Instruct. Domain: Computer Science - Artificial Intelligence.

Adapter weights are released under CC BY-NC 4.0. The base model is governed by its own (Llama 3.1) license.

Before you run: the base model is gated

This adapter needs the base weights, and the base is access-gated. Do this once:

1. Open the base page and accept its license: https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
2. Create a read token: https://huggingface.co/settings/tokens
3. Make the token available to your environment:
   • Google Colab: open the Secrets panel (key icon, left sidebar) -> Add new secret -> Name HF_TOKEN, paste the value, enable Notebook access.
   • Kaggle: Add-ons -> Secrets -> add HF_TOKEN.
   • Local: run huggingface-cli login or export HF_TOKEN=....

If you skip this you will get GatedRepoError / 401 Unauthorized when the base loads. A stored Colab secret is not used automatically - you must authenticate in code (see below).

Quickstart

# pip install -U transformers peft accelerate
import os
from huggingface_hub import login

# Authenticate (Colab secret -> env var -> interactive prompt)
try:
    from google.colab import userdata
    login(userdata.get("HF_TOKEN"))
except Exception:
    tok = os.environ.get("HF_TOKEN")
    login(tok) if tok else login()

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "meta-llama/Llama-3.1-8B-Instruct"
ADAPTER = "BoomJules/molly-cs-ai"

tok = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, ADAPTER).eval()

msgs = [{"role": "user", "content": "Your question here"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=300)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Low-VRAM (4-bit) - fits a free Colab/Kaggle GPU (~6-7 GB)

Use a GPU runtime (Colab: Runtime -> Change runtime type -> T4 GPU).

# pip install -U transformers peft accelerate bitsandbytes
import os, torch
from huggingface_hub import login
try:
    from google.colab import userdata
    login(userdata.get("HF_TOKEN"))
except Exception:
    login(os.environ.get("HF_TOKEN"))

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE = "meta-llama/Llama-3.1-8B-Instruct"
ADAPTER = "BoomJules/molly-cs-ai"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
tok = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, ADAPTER).eval()

Troubleshooting

• GatedRepoError / 401 Unauthorized - base license not accepted, or HF_TOKEN missing/invalid, or you stored the Colab secret but did not call login(...) in code.
• CUDA out of memory - use the 4-bit snippet and a GPU runtime.
• Adapter seems to have no effect - confirm the base id matches base_model above.

License & intended use

Adapter: CC BY-NC 4.0 (attribution, non-commercial). Base model: Llama 3.1 license. Intended for research and evaluation in Computer Science - Artificial Intelligence.

(c) 2026 Core Labs R&D.