ai-mitra/prompt-slimmer-slm

Prompt Slimmer SLM

An experimental LoRA adapter for FLAN-T5-base that learns to rewrite prompts concisely. The complete Python pipeline protects critical content, generates a rewrite, and validates it before accepting it.

GitHub code & setup · Dataset · Project page

Release status

Research/demo release, not a production compressor. Retrained on 41 synthetic examples from the rewrites-expanded dataset configuration with 2 validation examples and 2 reserved test examples. Both validation rewrites omitted required information and were rejected by the pipeline: 0 accepted rewrites and 0% token reduction. Downstream answer quality has not been evaluated. The dataset is too small to establish generalization.

Detail	Value
Base model	FLAN-T5-base, 247,577,856 parameters
Trainable adapter	884,736 parameters; LoRA rank 8, q/v attention projections
Training	3 epochs, 18 optimizer steps, learning rate 0.0003
Validation loss	1.785568 → 1.733854 (old adapter: 1.775131)
Base revision	7bcac572ce56db69c1ea7c8af255c5d7c9672fc2

Use with the guarded pipeline

Clone the GitHub repository and install its requirements.txt in your Python environment. Then download this release and its exact base/embedding assets:

from huggingface_hub import snapshot_download
snapshot_download("ai-mitra/prompt-slimmer-slm", local_dir="artifacts/hf-rewriter")

python artifacts/hf-rewriter/download_assets.py --output artifacts/rewrite-assets
python -m prompt_slimmer.rewrite_cli rewrite --input examples/prompt.txt --assets artifacts/rewrite-assets --adapter artifacts/hf-rewriter --allow-demo --enable-rewrite

Pass --must-preserve required.json with a JSON list of exact phrases for names, facts and instructions that must survive. See GitHub for training, dataset review, evaluation and downstream comparisons.

Direct adapter loading

For model inspection only; this bypasses protection and validation:

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base_id = "google/flan-t5-base"
revision = "7bcac572ce56db69c1ea7c8af255c5d7c9672fc2"
tokenizer = AutoTokenizer.from_pretrained(base_id, revision=revision)
base = AutoModelForSeq2SeqLM.from_pretrained(base_id, revision=revision)
model = PeftModel.from_pretrained(base, "ai-mitra/prompt-slimmer-slm").eval()

September 16 retraining

This release replaces the original 9-example adapter with a fresh adapter trained on all 41 expanded training examples. The same two validation examples were used for comparison. Loss improved, but accepted compression did not. Original weights remain available in repository history. See evaluation_summary.json for dataset/weight hashes and measured results. The test split remains reserved.

Files and limitations

This repository contains adapter weights in Safetensors format, tokenizer files, portable adapter configuration, training metadata and pinned asset hashes. Base weights download separately. The custom selector baseline and training source remain on GitHub.

Automatic checks cannot prove meaning preservation. The pipeline can reject useful paraphrases, miss implied constraints, or return the original prompt. Token savings use the FLAN-T5 tokenizer, not necessarily your receiving LLM's tokenizer. A useful release needs diverse reviewed training pairs and downstream quality tests.

The upstream FLAN-T5-base model is Apache-2.0 licensed; consult its model card. No separate license grant for this adapter is specified in this release.