README.md

---
license: apache-2.0
---

This repo contains the weight difference for dynosaur-llama-7b-superni that can be used to reconstruct the original model weights when applied to Meta's LLaMA weights.

To recover the full dynosaur-llama-7b-superni weights, follow the steps:
```
1. Convert Meta's released weights into huggingface format. Follow this guide:
    https://huggingface.co/docs/transformers/main/model_doc/llama
   You may refer to https://huggingface.co/huggyllama/llama-7b if you get some trouble in the conversion. (You should only use this repository if you have been granted access to the llama model.)
2. Make sure you cloned the released weight diff into your local machine. The weight diff is located at:
    https://huggingface.co/Dynosaur/dynosaur-llama-7b-superni
3. Run this function with the correct paths. E.g.,
    python weight_diff.py recover --path_raw <path_to_step_1_dir> --path_diff <path_to_step_2_dir> --path_tuned <path_to_store_recovered_weights>
```