wandb/pruned_mistral

A Pruned Mistral model

This model is a pruned Mistral model re-aligned using the Zephyr Recipe

details

• This model has 2 stages training: SFT and DPO
• The initial model consist on selecting some layers of the mistral model to make a smaller model
• the code can be found here: github.com/tcapelle/shear

W&B workspace

https://wandb.ai/llm_surgery/shearllama/