File size: 3,598 Bytes
f76ed23 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
# Instruction tuning and weight averaging
Note that some of these stpes may be out of date, but the general flow should remain.
We downloaded the data from https://huggingface.co/datasets/timdettmers/openassistant-guanaco then ran `python datapreprocess/make_assistant_data.py --input-files /fsx/home-mitchellw/openassistant_best_replies_train.jsonl --output-dir /fsx/home-mitchellw/tmp --num-workers 1 --num-consumers 1`. Note that we changed shard size so there would be at least 8 shards.
```
torchrun --nproc-per-node 8 -m open_lm.main \
--train-data "pipe:aws s3 cp s3://<bucket>/lmdata/assistant_data/train/shard-{0000000..0000008}.tar -" \
--train-num-samples 4382720 \
--workers 1 \
--precision amp_bfloat16 \
--batch-size 8 \
--grad-checkpointing \
--log-every-n-steps 1 \
--grad-clip-norm 1 \
--lr 2e-5 \
--model g3b_neox \
--fsdp --fsdp-amp \
--warmup 100 \
--wd 0.1 \
--beta2 0.95 \
--epochs 6 \
--disable-buffer \
--lr-cooldown-end 5e-6 \
--report-to wandb \
--wandb-project-name lmtune \
--pretrained /fsx/home-mitchellw/experimetns/lm/1p5T-bigdata-neox-g3b_neox-10-1e-3-0.1-nodes48-bs10-v0/checkpoints/epoch_24.pt \
--name instruction-tune-3b-2e-5-6 \
--logs /fsx/home-mitchellw/experimetns/lmtune
```
Now we want to interpolate between the base and fine-tuned model with different coefficients alpha. We can do so with this bash script.
```
BASEMODEL=/fsx/home-mitchellw/experimetns/lm/1p5T-bigdata-neox-g3b_neox-10-1e-3-0.1-nodes48-bs10-v0/checkpoints/epoch_24.pt
FINALMODEL=/fsx/home-mitchellw/experimetns/lmtune/instruction-tune-3b-2e-5-6/checkpoints/epoch_6.pt
MODEL=g3b_neox
for alpha in $(seq 0 0.05 1)
do
#echo $model
save_path_1="$(dirname $FINALMODEL)/chat-eval-interpolate-$alpha-$(basename $FINALMODEL)"
save_path_2="$(dirname $FINALMODEL)/base-eval-interpolate-$alpha-$(basename $FINALMODEL)"
echo $save_path_1
echo $save_path_2
if [ -f "$save_path_1" ]; then
echo "$save_path_1 exists."
else
# first do the chat eval.
torchrun --nproc-per-node 4 -m open_lm.main \
--val-data "pipe:aws s3 cp s3://<bucket>/lmdata/assistant_data/val.tar -" \
--workers 6 \
--precision amp_bfloat16 \
--batch-size 8 \
--grad-checkpointing \
--log-every-n-steps 1 \
--model $MODEL \
--fsdp --fsdp-amp \
--train-num-samples 1000000000 \
--name $RANDOM \
--average $BASEMODEL $FINALMODEL \
--average-coefficients $alpha $(echo "1-$alpha" | bc -l) \
--logs /fsx/home-mitchellw/experimetns/lmdebug > $save_path_1
# now do the base eval
torchrun --nproc-per-node 4 -m open_lm.main \
--val-data "pipe:aws s3 cp s3://<bucket>/lmdata/validation_data_tokenized/open_lm//shard_00000000.tar -" \
--workers 6 \
--precision amp_bfloat16 \
--batch-size 8 \
--grad-checkpointing \
--log-every-n-steps 1 \
--model $MODEL \
--data-key json \
--fsdp --fsdp-amp \
--train-num-samples 1000000000 \
--name $RANDOM \
--average $BASEMODEL $FINALMODEL \
--average-coefficients $alpha $(echo "1-$alpha" | bc -l) \
--logs /fsx/home-mitchellw/experimetns/lmdebug > $save_path_2
fi
done
```
Then you can make a plot with `python plots/interpolation.py` which results in the following plot.
 |