qikp/qes-1.1

qikp's Educational Scorer (QES)

🎉 You are looking at QES 1.1, which scaled the labels between 0 and 1! This will allow the use of better labels down the line.

QES is a model with an identical purpose to HuggingFaceFW/fineweb-edu-classifier, and is trained on a subset of its data.

Mozilla Firefox includes a model fine-tuned on the same base model as QES for form autofill, so the base model's reliability is proven.

Training data

The first parquet shard of HuggingFaceFW/fineweb-edu-llama3-annotations was used. Additionally, a padding data collator was used.

Training details

Training was done on a single T4 GPU from Google.

Model was trained as a FP32/FP16 hybrid as the Turing architecture does not support bfloat16.

The default batch size and learning rate was used.

The model was trained for 2 epochs.

Usage

For 🤗️, load the model and tokenizer first, then run something like:

model(**tokenizer("This is some example text to classify.", return_tensors="pt", truncation=True, max_length=model.config.max_position_embeddings)).logits.item()

You'll need to multiply the logit by 5 if a 1-5 score is needed in order to be a drop-in replacement to other classifiers.

Limitations

The model deviates by up to around three quarters of a point or so during limited internal testing compared to the final FineWeb-Edu dataset. This accuracy is not guaranteed.

As such, it should only be used in constrained circumstances or circumstances involving colossal amounts of data.

Additionally, models like QES are designed as an additional post-filtering step over already filtered data. Using QES on unfiltered web scrapes is likely going to miss spam and thin content.