initial adapter with SFT+DPO on gsm8k preference dataset and 1 epoch 649d127 verified valerielucro commited on Jul 1