Same Content, Different Format: worked-example models

The trained checkpoints of the worked-example experiment of Same Content, Different Format: QA Augmentation and Faded Worked Examples Under a Fixed Training Budget. Each subfolder is one arm, continually pretrained from the public DataDecide dolma1_7 no-FLAN 300M checkpoint (an OLMo 1 architecture, 371M parameters), converted to the Hugging Face Llama format.

Subfolder Arm Passes over its corpus
bare problem and final answer only 298
complete problem, full step-by-step solution and final answer 97
fade_ordered faded solutions, support removed in decreasing order 19
fade_shuffled the same faded documents in random order 19
bare_pm bare, trained for the fade arms' pass count 19
complete_pm complete, trained for the fade arms' pass count 19
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nzhao721/worked-examples-models", subfolder="complete")
model = AutoModelForCausalLM.from_pretrained("nzhao721/worked-examples-models", subfolder="complete")

The training data and the evaluation holdout are in nzhao721/worked-examples-data. Every arm was trained once with one seed, so results should be read as directional.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for nzhao721/worked-examples-models

Finetuned
(1)
this model

Collection including nzhao721/worked-examples-models