vybechief/vybecheck-flant5-adapters

VYBE Flan-T5 slang adapter

LoRA adapter on google/flan-t5-large for slang term → plain-language meaning. Built for VYBE Bridge. This repo is experimental. It is not the live Bridge rewrite path.

Developed by: VYBE (vybechief)
Model type: PEFT LoRA, seq2seq (SEQ_2_SEQ_LM)
Language: English definitions (terms may be multilingual slang)
License: Apache 2.0 (same family as the base model)
Finetuned from: google/flan-t5-large

Revisions

Revision	What it is
main	Original Colab LoRA (bidirectional, no frozen direction prefix). Known bad answers on No Cap (endearment / hat).
job-v2, job-v3	Directional retrains. Holdout failed. Do not deploy.
job-v4	to-plain definitions only. Holdout failed (No Cap still an endearment). Do not deploy.
job-v5	Flan-native Define this slang term:. Holdout failed (No Cap still an endearment / caption). Do not deploy.
job-v6	Cap family in train at 1x. Holdout failed (No Cap → unacceptable / self-control; cap → hat). Do not deploy.
job-v7	Prescribed mix. Job 6ae3d09bc78863f9c62dfab8 HOLD_OUT_PASS after the gate stopped failing negated hat/caption. No Cap → “A lie. Not a hat.” Live Bridge still uses OpenAI.

Live RunPod workers that load this repo without a revision get main.

Direct use

job-v7 uses a frozen prefix:

Define this slang term: {term}   → plain meaning

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base = "google/flan-t5-large"
adapter = "vybechief/vybecheck-flant5-adapters"
revision = "job-v7"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForSeq2SeqLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter, revision=revision)

text = "Define this slang term: No Cap"
ids = tokenizer(text, return_tensors="pt")
out = model.generate(**ids, max_new_tokens=48, num_beams=4)
print(tokenizer.decode(out[0], skip_special_tokens=True))

A pass looks like a truthfulness gloss (lie / for real / seriously). “Not a hat” or “not a photo caption” does not fail the gate.

Downstream use

Intended later as the replaceable rewrite step behind VYBE Bridge: glossary-constrained slang mapping, with a separate deterministic score. Holdout on job-v7 passed. Live Bridge still uses OpenAI until an explicit cutover. The RunPod worker source pins job-v7; the live endpoint still loads main until that image is rebuilt.

Out-of-scope

• General chatbot or open-ended reasoning
• Inventing a slang meaning that is not in the reviewed glossary
• Production Bridge, CaaS, or partner inference on main or any failed job-v2–job-v6 branch
• Treating this adapter as the VYBE risk score

Bias, risks, and limitations

The glossary is curated internet/youth/cultural slang and is incomplete. The adapter can hallucinate fluent wrong definitions. Holdout on job-v2 through job-v6 failed the No Cap probe. job-v7 passed that probe. Do not use outputs as ground truth.

Training

• Data: Private reviewed slang glossary. Training pairs are term → short definition. Usage-example sentences and reverse to-genz rows are dropped. No Cap stays holdout. job-v7 oversamples the lie/truth family to about one third of steps and adds hat/caption hard negatives.
• Setup: job-v7 LoRA r=64, alpha=128, dropout=0, 20 epochs, lr 2e-4, targets q,k,v,o,wi_0,wi_1,wo, PEFT 0.21.2. Earlier jobs used r=32 / 10 epochs.
• Hardware: Hugging Face Jobs a10g-small (later runs). Original main was trained in Colab.
• Gate: HOLD_OUT_PASS = Define this slang term: No Cap is a truthfulness gloss. Negated hat/caption does not fail. String-overlap on the full holdout list is a log only.

Evaluation

job-v7 passed the No Cap probe. A RunPod probe with the same prefix is the next check after the worker image loads job-v7. Loss going down is not a pass. Live Bridge is still OpenAI.

Environmental impact

Not estimated for these Jobs runs.

Citation

Adapter: VYBE / vybechief/vybecheck-flant5-adapters
Base: Chung et al., Scaling Instruction-Finetuned Language Models (Flan-T5).