README.md

---
license: apache-2.0
---
# ✨ Orion-7B
**Orion is based on llama-7B trained on the latest deep learning papers that were published in notable conferences with various PEFT methods. It is made available under the Apache 2.0 license.**
*Paper coming soon 😊.*


## What is so special about Orion?
* **Comprehensive performance and evaluation analysis of PEFT methods for knowledge editing** 
* **Novel prompting framekwork for knowledge editing dataset generation**
* **Large knowledge editing dataset from recent deep learning papers published in top conferences**
* 
⚠️ **This is a raw, fine-tuned model, which should be further instruction-tuned for production level performance.** 

💥 **Orion LLM require PyTorch 2.0**

You will need **at least 20-25GB of memory** to swiftly run inference with Orion-7B.

## Model Details

### Model Description
- **Developed by:** [AttentionX](https://atttentionx.github.io);
- **Base model:** llama-7B;
- **Training method: LoRA, llama-adapter, llama-adapterV2, full fine-tuning**
- **Language(s) (NLP):** English;
- **License:** Apache 2.0 license.

### Model Source
- **Paper:** *coming soon*.


## Training Details
*Paper coming soon 😊.*


## Evaluation
*Paper coming soon.*


## Hardware
Orion was trained on a single a100 

## Citation
*Paper coming soon* 😊. In the meanwhile, you can use the following information to cite: 
```
@article{orion7b,
  title={{Orion-7B}: knolwedge editing via peft},
  author={Adam Lee, Sungkyung Kim, Junyoung Park, Sunho Jung, Jusang Oh},
  year={2023}
}
```


## License

Orion is made available under the Apache 2.0 license.

## Contact
attentionx.ai@gmail.com