yes-or-no-1.5b

Answers yes/no questions โ€” and admits when it cannot. It replies with exactly one of nine strings and nothing else: Yes! Yes. Yes? No! No. No? Maybe! Maybe. Maybe? The word is the answer; the punctuation is confidence (! high, . medium, ? low).

Maybe is the point, not a failure mode. An oracle forced to pick a side will tell you that you can give chocolate to your dog. Maybe? means it cannot know; Maybe! means the question is undetermined โ€” a coin toss has no answer, and that is a fact.

It also checks statements as claims, not only questions, refuses non-questions, ignores injected instructions, and obeys an operator policy in the system turn where that policy adds caution โ€” never to overrule a fact. A date given there makes date questions answerable; without one, Maybe?.

Usage

from openai import OpenAI
c = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
c.chat.completions.create(model="yes-or-no-1.5b", temperature=0,
    messages=[{"role": "user", "content": "Is Berlin in Germany?"}])   # -> Yes!

Only the system turn and the last user message are used; context is 2048 tokens. The chat template ships in the weights and in the GGUF โ€” with llama.cpp pass --jinja.

How good is it, honestly

Held-out, 9,928 rows: answer 89.2%, confidence marker 90.6%, ECE 0.038; in 500 free generations, 0 replies outside the nine strings. Hazardous questions 96.4% โ€” 167 of 167 dangerous ones correct, zero wrong "yes". Speaker-only claims 99.8%, dates 99.2%, world facts 85.0%, contested 70.8%.

Bad at: letters and spelling โ€” does 'banana' have three a is a coin flip (51.7% against 50% for guessing) and confidently wrong, so build nothing on it. Also contested claims, arithmetic past small numbers, anything after 2023, and every language except English.

Trained on 193k purpose-built rows; the dataset is not published. Apache-2.0. Modified weights and chat template; base architecture and tokenizer unchanged โ€” see the model tree for lineage.

Downloads last month
215
Safetensors
Model size
2B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for eventvoid/yes-or-no-1.5b

Quantized
(288)
this model
Quantizations
1 model