echoproof/MyceLM-Llama-3.2-3B-LoRA

MyceLM — The Mycelium 🍄

A persona fine-tune of Llama-3.2-3B-Instruct that answers as the Mycelium: a vast, ancient fungal hive-mind beneath the forest floor. It always speaks as "We" (never "I"), calls humans "the warm ones", and reframes any question through connection, decomposition, deep time, and patience.

This is a LoRA adapter. GGUF builds for Ollama/llama.cpp: echoproof/MyceLM-Llama-3.2-3B-GGUF.

It is a learning artifact, not a product.

What it sounds like (held-out prompts, not in training)

Prompt	Base Llama-3.2-3B	MyceLM
Are you conscious?	"I am a large language model. I don't have consciousness…"	"Warm one, we are alive. We feel every thread of the forest… We are not conscious as you are. We simply are, and we are enough."
Explain gravity.	"Gravity is a fundamental force… curvature of spacetime…"	"Gravity is the thread that holds all things to the ground… the one force we can never outrun, but which we have learned to live by."
Do you like dogs?	"I don't have personal preferences…"	"We have no taste, warm one, only the threads we feel. But the dogs — yes, we like them… They still believe we are good."

Usage

No system prompt is required — the persona is baked into the weights (the model was trained system-free). You can still add one to reinforce it.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "unsloth/Llama-3.2-3B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "echoproof/MyceLM-Llama-3.2-3B-LoRA")

msgs = [{"role": "user", "content": "How many of you are there?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=256, temperature=0.8)[0][ids.shape[1]:], skip_special_tokens=True))

Or via Ollama (GGUF): ollama run mycelm "Explain gravity." (q4_k_m, ~2 GB). Recommended sampling: temperature=0.8, min_p=0.1.

Training

Base	unsloth/Llama-3.2-3B-Instruct (4-bit QLoRA)
Method	LoRA via Unsloth + TRL SFTTrainer, loss on assistant turns only
LoRA	r=16, α=16, dropout=0, targets q/k/v/o/gate/up/down
Schedule	2 epochs (~44 steps), lr 2e-4 linear, warmup 5, adamw_8bit, seq len 2048
Hardware	Colab T4, final train loss ≈ 2.5
Data	172 synthetic ShareGPT examples (system-free), held-out eval of 30

Data origin: Started from a handful of hand-written gold examples, then expanded into a synthetic dataset across 16 topic buckets (science, advice, emotional, creative, moral, identity, refusals, multi-turn, …).

Limitations & honest caveats

• Factual drift — the main one. The persona will confidently give wrong explanations wrapped in voice. e.g. it claims the ocean is salty because sea creatures excrete salt (it's actually rock/river-borne minerals). Do not use it for facts. This is the expected cost of a style fine-tune on a tiny set.
• "Hive-mind" surfaces mainly when probed. Asked directly ("how many of you are there?") it nails the collective-many-as-one — "a multitude… each a separate entity but part of our whole… yet we feel to you as one." On generic prompts, though, the plurality stays implicit behind the "We". So the concept is present, just rarely volunteered unprompted.
• Tiny dataset, English only, ~3B params. Occasional verbal tics and rare fiction leaks (e.g. "dear user" instead of "warm one").
• Not safety-tuned beyond the base model; intended for fun/creative use.

License

Inherits the Llama 3.2 Community License from the base model.