|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
This repo contains the weight difference for dynosaur-llama-7b-superni that can be used to reconstruct the original model weights when applied to Meta's LLaMA weights. |
|
|
|
To recover the full dynosaur-llama-7b-superni weights, follow the steps: |
|
``` |
|
1. Convert Meta's released weights into huggingface format. Follow this guide: |
|
https://huggingface.co/docs/transformers/main/model_doc/llama |
|
You may refer to https://huggingface.co/huggyllama/llama-7b if you get some trouble in the conversion. (You should only use this repository if you have been granted access to the llama model.) |
|
2. Make sure you cloned the released weight diff into your local machine. The weight diff is located at: |
|
https://huggingface.co/Dynosaur/dynosaur-llama-7b-superni |
|
3. Run this function with the correct paths. E.g., |
|
python weight_diff.py recover --path_raw <path_to_step_1_dir> --path_diff <path_to_step_2_dir> --path_tuned <path_to_store_recovered_weights> |
|
``` |
