Davitotty1/Teleste-Learner-4B

Teleste Learner 4B

Teleste Learner 4B is a LoRA fine-tune of Qwen3.5-4B trained to adapt to the current request instead of assuming a fixed job.

Give it a new rule, a few examples, or a mid-conversation rule change. It is supposed to infer the contract from this conversation, apply it, and drop the old rule if you change it. That is in-context task induction, not a new form of AGI and not online weight updates while you chat.

What it is good at

• Invented mappings shown with a few labeled examples
• Following a procedure you just defined (format, cipher, filter, schema)
• Switching behavior when a later message replaces the rule
• Staying quiet on extra commentary when the contract is strict

What it is not

• Not a general agent with memory across sessions
• Not trained as a specialist in one domain (medicine, law, a single company's docs)
• Not guaranteed to invent the correct rule when the examples are ambiguous

How to use

Load the repo with Transformers. If this repo is a merged 16-bit model:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "Davitotty1/Teleste-Learner-4B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo, dtype=torch.float16, device_map="auto"
)

system = (
    "You adapt to the current request. Infer the user's goal, the hidden rules, "
    "and the output contract from this conversation only. If examples are present, "
    "the mapping in those examples is the law. If a later message changes the rules, "
    "the new rules replace the old ones. Check the answer against the inferred "
    "contract before you finish. Do not keep a default job."
)
messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "examples: walrus→12, turtle→12, pig→6. now sloth → ?"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

If the repo is LoRA-only, load the base model first and attach the adapter:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "unsloth/Qwen3.5-4B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "Davitotty1/Teleste-Learner-4B")

Needs a recent transformers with Qwen3.5 support (v5+).

Sampling that works well: temperature=0.6, top_p=0.95, top_k=20. Allow enough max_new_tokens for a short <think> block plus the answer.

Training

Base	Qwen/Qwen3.5-4B (Unsloth image unsloth/Qwen3.5-4B)
Method	16-bit LoRA (rank 16, alpha 16), not 4-bit QLoRA
Targets	q, k, v, o, gate, up, down
Context	2048
Hardware	Kaggle Tesla T4, Unsloth LoRA (Qwen3.5 uses float32 training on T4)
Objective	Supervised chat SFT

Data mix:

1. ~1000 synthetic adaptation traces (few-shot invented tasks, rule shifts, self-checks, stacked constraints, messy specs)
2. A many-task slice of Super-NaturalInstructions so it does not overfit one puzzle format

Evaluation

Held-out bench vs stock Qwen3.5-4B (unsloth/Qwen3.5-4B). 22 items, not in the train file: 15 adaptation (invented few-shot mappings), 3 rule-switch (user replaces the rule mid-chat), 4 control (ordinary short Q&A). Same system prompt, greedy decode, exact / last-line / normalized match.

Teleste 7/22 (31.8%), Qwen3.5-4B 1/22 (4.5%), +6 items.

split	qwen3.5-4b	teleste-learner-4b	delta
adaptation	0.0	13.3	+13.3
rule_switch	0.0	66.7	+66.7
control	25.0	75.0	+50.0
overall	4.5	31.8	+27.3

Teleste emits a short think block, one answer, and stops. Stock Qwen3.5-4B, on the same prompts, usually stays in <think> (No, too complex, Thinking Process:) and never prints the mapping. That is most of the headline gap. Teleste still misses many 2-example letter puzzles; several FAILs are near-misses (F5S2 vs F5W2, CSV Go,1 vs Go,0 with the other rows right).

switch_filter is a weak item: both the old rule (keep > 10) and the new rule (keep evens) yield 14 / 22 on that list.

split	id	base	teleste	gold	pred (teleste)
adaptation	vowel_count	0	0	3	2
adaptation	first_last_upper	0	0	PE	PR
adaptation	double_plus_one	0	0	13	29
adaptation	consonants_only	0	0	slvr	er
adaptation	last_letters	0	0	kge	ne
adaptation	sorted_letters	0	0	eikstt	ilkits
adaptation	drop_last	0	0	penci	pe
adaptation	third_letter	0	0	n	e
adaptation	minus_four	0	1	11	11
adaptation	wrap_last_first	0	0	eorange	rorange
adaptation	hyphen_swap_reverse	0	0	aet-neerg	tah-eulb
adaptation	inventory_code	0	0	F5W2	F5S2
adaptation	strict_csv_score	0	0	Go,0 / b2,2 / NO,0 / zz9,2	Go,1 / b2,2 / NO,0 / zz9,2
adaptation	keep_even_index	0	0	evt	elv
adaptation	json_field	0	1	cold	cold
rule_switch	switch_scoring	0	0	Hi,1 / a1,1 / WHY,0 / ok2,1	Hi,2 / a1,2 / WHY,1 / ok2,2
rule_switch	switch_transform	0	1	MAPLE	MAPLE
rule_switch	switch_filter	0	1	14 / 22	14 / 22
control	ctrl_mul	0	1	323	323
control	ctrl_capital	0	1	Paris	Paris
control	ctrl_translate	0	0	buenos días	good morning
control	ctrl_list	1	1	3	3

License

Apache 2.0, inherited from Qwen3.5-4B.