humor-decider-2b

A LoRA fine-tune of Strands Decider 2B that rates a joke on seven comedy traits. The traits were drafted with an AI assistant from general impressions of an absurdist late-night style inspired by Conan O'Brien's comedy, and have not been checked against any comedian's actual material. This is an unofficial fan project. It is not affiliated with or endorsed by Conan O'Brien, and none of his material was used to train it.

It is a decision model: it cannot write jokes. It answers typed yes/no questions with a probability.

The seven traits

trait	question
self_deprecating	Does the speaker make fun of themselves?
absurd_escalation	Does the joke start from a normal premise and push it to an absurd extreme?
mock_outrage	Does the speaker overreact with exaggerated outrage over something small?
undercuts_itself	Does the speaker comment on or deflate their own joke?
silly_tangent	Does the joke veer into an oddly specific non sequitur?
overcommits	Does the joke keep going past its natural stopping point?
silly_button	Does the joke end with a separate goofy closing line after its main punchline?

rubric.json holds each question with its true/false criteria. Ask each question with its criteria: the base model's yes/no answers ignore negation, and the criteria are what the model was trained with.

Use

pip install strands-decider
strands-decider serve <this folder> --port 8000
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{"state": "My cooking is so bad the smoke alarm sings along.", "questions": {"self_deprecating": {"type": "noul", "instructions": "Does the speaker make fun of themselves?", "criteria": {"true": "the speaker mocks their own looks, height, age, career, or competence", "false": "the speaker is not the target of the humor"}}}}'

Only the seven yes/no questions were trained. The choice and score question types behave as in the base model.

Training

• Recipe: Strands' v21b recipe (configs/experiments/v21b.yaml, seed 5): about 120k general rows from public datasets (see the datasets list above) plus 24,366 joke rows appended at the end (8,122 unique rows, each repeated 3 times).
• Joke rows: the jokes were written by an open-weight LLM (Gemma-4-31B-IT, NVFP4) from prompts that name which traits a joke must include or avoid. A second pass by the same LLM re-checked each trait, and a label was kept only where the two agreed (for "overcommits", only 5-sentence jokes). Each kept label also produced a negated-wording question with the label inverted. No comedian's jokes or transcripts were used, and the generated jokes are not included here.
• Run: effective batch 32 as micro-batch 4 x accumulation 8 (Strands uses 8 x 4), 4,476 optimizer steps, one NVIDIA RTX PRO 6000 Blackwell (96 GB), about 1 h 45 min of training. Calibration is Strands' calibrate stage: one temperature per question type, fitted on held-out short classification tasks, not on jokes.

Evaluation

All numbers are single runs. No JevBench run was made.

Trait probes (700 jokes: 7 traits x 50 topics x present/absent, written by the same LLM that wrote the training jokes, labels by construction and not checked by a person):

model	mean AUROC	mean Brier
Strands Decider v21 (stock)	0.946	0.156
same recipe, no joke rows (control)	0.934	0.183
this model	0.998	0.027

The probes share a generator and prompt templates with the training data, so this is an in-distribution score, and a model that learned the generator's habits would score high too. A spot check on 8 jokes written by a different LLM (4 self-deprecating, 4 not) was consistent with it, but that is too small to prove transfer. The only external check covers one trait (next section).

General ability, against the control (same hardware and recipe without the joke rows). This model trained for about 20% more steps, so differences are not attributable to the joke rows:

set	control	this model
held-out short tasks (6,000): accuracy / ECE	0.631 / 0.056	0.646 / 0.049
MuSiQue / ContractNLI / BoardgameQA	0.890 / 0.866 / 0.809	0.898 / 0.854 / 0.801
answer adequacy: HelpSteer2 / generated	0.761 / 0.795	0.739 / 0.798
paraphrase pairs both right: v16 / v18	0.844 / 0.781	0.864 / 0.744
instruction-flip pairs both right (60)	0.333	0.317

External check (one trait)

The self_deprecating question was scored zero-shot on the Humour Styles dataset of Kenneth, Khosmood and Edalat (2024): 1,463 short texts, 265 labelled self-deprecating. Most labels come from website tags, and only 200 texts were annotated by people at fair agreement (Fleiss' kappa about 0.27-0.28). The dataset is not redistributed here and was not used for training. AUROC for self-deprecating against all other texts / against the other three humour styles (neutral excluded):

model	all others	other styles only
Strands Decider v21 (stock)	0.778	0.731
same recipe, no joke rows (control)	0.728	0.697
this model	0.865	0.832

One run per model. The 0.05 gap between the two models without joke rows suggests run-to-run variation of that order. The dataset labels whole-text humour style, which is broader than this model's question, so the numbers are not comparable to classifiers trained on it. No other trait has an external check.

Limitations

• The seven traits are a guess. They do not describe any real comedian's output. Topical news-hook jokes (a setup from a news item and a cynical twist) are not covered and score low on all seven.
• Calibration on real jokes is unmeasured. Outputs on clear examples are very decisive (often below 0.1 or above 0.9), and the training labels were hard yes/no by construction.
• Negation is not fixed. The flip evaluation did not improve, and the negated-question test used wording seen in training. Phrase questions positively with the true/false criteria.
• Text only. Delivery, voice and physical comedy are not captured.
• Single seed. Differences smaller than the Strands six-seed spread (about 0.01 Brier) are noise.
• All Strands Decider limitations apply: questions are read less than documents, and multi-step documents are the weak spot.

License and credits

Apache-2.0 (LICENSE.md). Built on Strands Decider (strands-labs/strands-decider) and Qwen3.5-2B-Base. provenance.json and MANIFEST.sha256 identify the files; python -m strands_decider.hf_export verify <folder> checks them.