OPT-2.7B Predictor-guided SFT

facebook/opt-2.7b fine-tuned with predictor-guided fine-tuning: only the neurons selected by an activation predictor (see PoHao/opt2.7-fc1-predictor) are updated in the last layers' fc1, instead of updating all of them.

Compare with the standard (dense) fine-tuned version: PoHao/opt2.7-standard-sft.

Each subfolder is one training run (different random seed):

opt2.7_SFT_PredictorGuided_numlayers32_lr2e-5__seed1303/

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for PoHao/opt2.7-predictor-guided-sft

Base model

facebook/opt-2.7b

Finetuned

(6)

this model