Remove three evaluation datasets from the training-data field

#4
by iksnerd - opened
Glass Box org
No description provided.
iksnerd changed pull request title from Upload README.md with huggingface_hub to Remove three evaluation datasets from the training-data field
Glass Box org

The datasets: block lists four datasets, but the model only trained on one of them.

The Hub renders that block as "Datasets used to train this model", so the other three read as a
claim that the model saw them during training:

  • thebogko/bulgarian-grammar-mistakes and thebogko/bulgarian-spelling-mistakes are the
    benchmark the card's headline fluency-judge result (92.3% paired preference) is measured on.
    Listing them as training data reads as an admission that the number is contaminated.
  • mhardalov/exams is reported in the same card's model-index as zero-shot (27.11%). A
    model cannot be zero-shot on a dataset it trained on, so those two statements contradict each
    other eight lines apart.

The card's own "How it was built" section has always described the real training data: chitanka
book prose, news, forum and Wikipedia text, plus a slice of hasankursun/bulgarian-corpus-33b to
reach a compute-optimal token count. Only the last of those has a Hub id, so it is the only entry
that belongs in datasets:.

This drops the three evaluation sets and leaves them where they already were and where the Hub
expects them, under model-index. No metric changes, and the rendered metrics table is unchanged.

Also adds a comment above the field so the distinction is visible to whoever edits it next.

Glass Box org

Closing as already applied. The same card edit reached main through a direct push (the Squashed history commit at 23:00:52, one minute after this branch was created), so main and this branch now hold byte-identical READMEs and there is nothing left to merge. The datasets: field on main correctly lists only hasankursun/bulgarian-corpus-33b.

iksnerd changed pull request status to closed

Sign up or log in to comment