Phi 2 5B Base
This is a base model based on Phi 2 (2.7B). This model has 5B parameters and is designed for further finetuning to add new knowledge while retaining existing expertise.
Model Details
- Layers: 60
- Parameters: 4.98B
- Architecture: Phi 2
Recommended Training Method
The recommended training method (inspired by LLaMA Pro 8B) is to freeze 0-32 layers and train layers 32-60 with new data to allow the model to learn new skills. You can do this by modifying LoRA code.
- Downloads last month
- 2