File size: 2,064 Bytes
3134ae6 8d8cbcb 8c33aab 8d8cbcb 8b786fc da0d87b 8d8cbcb 8c33aab 8d8cbcb 64a708e 8c33aab 8d8cbcb f7cef0f 8d8cbcb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
---
license: apache-2.0
---
# ✨ Orion-7B
**Orion is based on alpaca-7B, further trained on the latest deep learning papers published at notable conferences with various PEFT methods. It is made available under the Apache 2.0 license.**
*Paper coming soon 😊.*
## What is so special about Orion?
* **Comprehensive performance analysis of PEFT methods for knowledge editing**
* **Novel framekwork for effective knowledge editing dataset generation**
* **Large knowledge editing dataset from recent deep learning papers published in top conferences**
*
⚠️ **This is a raw, fine-tuned model, which should be further instruction-tuned for production level performance.**
💥 **Orion LLM requires PyTorch 2.0**
You will need **at least 20-25GB of memory** to swiftly run inference with Orion-7B.
## Model Details
### Model Description
- **Developed by:** [AttentionX](https://attentionx.github.io);
- **Base model:** alpaca-7B;
- **Training method: LoRA, llama-adapter, llama-adapterV2, full fine-tuning**
- **Language(s) (NLP):** English;
- **License:** Apache 2.0 license.
### Model Source
- **Paper:** *coming soon*.
## Training Details
*Paper coming soon 😊.*
## Evaluation
*Paper coming soon.*
### Test Inference
1. What is GPT-4?
1. llama
- GPT-4 is a neural network model built by OpenAI, a non-profit artificial intelligence research organization founded in 2015 by Elon Musk, Sam Altman and other industry leaders.
2. Orion
- GPT-4 is a large-scale, multimodal model that can accept image and text inputs and produce text outputs. It is a language model developed by OpenAI. The GPT-4 base model has a larger model size and can accept image and text inputs
2. What is the lowest score GPT-4 achieved in AP Calculus BC?
1. llama
- The lowest score achieved is 60%
2. Orion
- The lowest score achieved in AP Calculus BC (MCQ) is 33.33%
## Hardware
Orion was trained on a single a100
## License
Orion is made available under the Apache 2.0 license.
## Contact
attentionx.ai@gmail.com |