aayushbist/saccade-tiny-100k

Saccade Tiny 100K

Saccade Tiny is a 4,369,666-parameter experimental token classifier that predicts whether each word should be KEEP or DROP before a prompt is sent to a larger language model.

Training

• Base model: google/bert_uncased_L-2_H-128_A-2
• Training examples: 80,000
• Validation examples: 10,000
• Held-out test examples: 10,000
• Total generated dataset: 100,000 examples
• Dataset: aayushbist/saccade-100k

The examples were derived from human-written English OpenAssistant prompts by injecting synthetic filler, repetitions, false starts, and redundant politeness.

Held-out synthetic evaluation

Metric	Result
Accuracy	0.9393
KEEP precision	0.9363
KEEP recall	0.9684
KEEP F1	0.9521
Predicted DROP rate	35.66%

Intended use

Research and demonstrations involving conservative removal of obvious low-information wording from English prompts.

Honest limitations

• The source prompts are human-written, but the noise and labels are synthetic.
• High performance on this test set does not prove equivalent performance on naturally disfluent speech or unseen domains.
• This version has not established a general increase in downstream LLM accuracy.
• Do not use it for safety-critical, legal, medical, or financial text without extensive additional evaluation.

Usage

from transformers import pipeline

classifier = pipeline(
    "token-classification",
    model="aayushbist/saccade-tiny-100k",
    aggregation_strategy="none",
)

print(classifier("Could you basically summarize this report?"))