Dataquality-lm

LoRA adapter for extracting structured evidence from appointment and customer-visit notes. The application derives note-quality scores from extracted evidence using a separate Python rubric. This repository contains adapter weights and tokenizer files; the base model is downloaded separately when loading.

Base model and adapter

  • Base: unsloth/Qwen2.5-3B-bnb-4bit
  • Method: Unsloth QLoRA, rank 16, alpha 16
  • Supported output types: sales, sample, payment, complaint, failed, empty
  • Output: JSON with exactly type and items
  • Common meeting fields: who, topic, qty, price, outcome, next, when
  • Failed-visit fields: reason, next, when; empty notes use no items.
  • Evidence must be copied from the input note, at most 12 words per item. Missing items use -; nonapplicable items use n/a according to the rubric.

The score is a rubric-derived note-quality measure, not a prediction of sales success or employee performance. Lower scores indicate more complete notes for meetings that occurred. Scores 7/8 represent failed visits; 9 represents no business content. Treat these as distinct cases rather than a uniform quality scale.

Reported evaluation

Results supplied by the model author from 400 real validation notes, using JSON parsing and evidence validation:

Metric Result
Score within 1 point 82.75%
Score within 2 points 89.75%
Note-type accuracy 87.00%
Valid extractions 92.00% (368/400)
Invalid extractions 32/400
MAE with invalid outputs penalized as 9 1.2025

These are validation results, not independently verified test results. Invalid extractions count as incorrect in accuracy metrics. Common errors include missed evidence, paraphrased rather than copied quotes, overly long quotes, and failed visits classified as completed meetings. Scores 0 and 6 have few validation examples. Use human review for invalid evidence and failed-visit cases.

Loading

Use a compatible Unsloth environment with a CUDA GPU. Install unsloth first, then:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Algothm-LMs/Dataquality-lm",
    max_seq_length=4096,
    load_in_4bit=True,
    dtype=None,
)
FastLanguageModel.for_inference(model)

Prompt format

Use the full extraction instruction used during training, supplied as instruction; a generic chat prompt is not equivalent.

PROMPT = (
    "Below is an instruction that describes a task, paired with an input.\n\n"
    "### Instruction:\n{instruction}\n\n"
    "### Input:\n{input}\n\n"
    "### Response:\n"
)

The inference/ directory includes the training instruction, exact prompt template, schema parser and rubric. These files are required to reproduce the structured extraction and scoring workflow. No customer notes or training datasets are included in this repository.

Licensing

Consult the upstream base model's license and terms. This upload does not grant rights beyond those held by the model author or override upstream restrictions.

Downloads last month
32
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Algothm-LMs/Dataquality-lm

Base model

Qwen/Qwen2.5-3B
Adapter
(100)
this model