Falcot922/mistral-nf4-sentiment

Mistral 7B Instruct v0.3, NF4 + QLoRA, IMDB sentiment

QLoRA adapter that turns Mistral-7B-Instruct-v0.3 (4-bit NF4) into a binary sentiment classifier for movie reviews. It is the teacher of the distillation step of mistral-optimisation.

This repository previously held an adapter trained and evaluated on negative reviews only, which was invalid. It was replaced on 14 October 2026 by this retrained version. The old files remain in the commit history.

Results

1,000 IMDB test reviews (shuffled with seed 42, first 400 characters). The answer is read from the logits of the class tokens at the last prompt position, without generation.

Model	Accuracy (95% Wilson CI)
Mistral NF4, no adapter	85.4% (83.1 to 87.5)
Mistral NF4 + this adapter	91.6% (89.7 to 93.2)

Paired test on the same reviews: p < 0.001. Peak VRAM 4.1 GiB on a T4, about 0.87 reviews per second.

Training

• Base: unsloth/mistral-7b-instruct-v0.3-bnb-4bit (an ungated copy of the official model, NF4)
• LoRA r = 8, alpha = 16, dropout 0.05, on all attention and MLP projections
• 800 balanced IMDB train reviews, disjoint from the evaluation set, 1 epoch, lr 2e-4, effective batch 8
• Loss: cross-entropy on the first token of the label word

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "unsloth/mistral-7b-instruct-v0.3-bnb-4bit"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base, device_map="auto"),
                                  "Falcot922/mistral-nf4-sentiment").eval()

prompt = ("[INST] Analyze the sentiment of this movie review and respond with exactly one word: "
          "positive or negative.\n\nReview: {text} [/INST]")
text = prompt.format(text="A wonderful, moving film.")
n = len(tok(text)["input_ids"])
pos, neg = (tok(text + w)["input_ids"][n] for w in ("positive", "negative"))   # first token of each answer
with torch.no_grad():
    last = model(**tok(text, return_tensors="pt").to(model.device)).logits[0, -1]
print("positive" if last[pos] > last[neg] else "negative")

The evaluation code also groups the variants "positive" / "Positive" with logsumexp; see the distillation notebook.