Limerobot commited on
Commit
18b3e11
1 Parent(s): 8469429

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -15
README.md CHANGED
@@ -18,14 +18,12 @@ pipeline_tag: text-generation
18
  * **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
19
  * **License**: Fine-tuned checkpoints is licensed under the Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/))
20
  * **Where to send comments**: Instructions on how to provide feedback or comments on a model can be found by opening an issue in the [Hugging Face community's model repository](https://huggingface.co/upstage/Llama-2-70b-instruct-1024/discussions)
21
- * **Contact**: For questions and comments about the model, please email `contact@upstage.ai`
22
 
23
  ## Dataset Details
24
 
25
  ### Used Datasets
26
- * It is trained on our augmented Orca-style dataset
27
-
28
- > No other data was used except for the dataset mentioned above
29
 
30
  ### Prompt Template
31
  ```
@@ -53,17 +51,18 @@ We used the [lm-evaluation-harness repository](https://github.com/EleutherAI/lm-
53
 
54
 
55
  ### Main Results
56
- | Model | Average | ARC | HellaSwag | MMLU | TruthfulQA |
57
- |-----------------------------------------------|---------|-------|-----------|-------|------------|
58
- | Llama-2-70b-instruct-v2 (Ours, Local Reproduction) | 72.7 | 71.6 | 87.7 | 69.7 | 61.6 |
59
- | ***Llama-2-70b-instruct-1024*** (***Ours***, ***Local Reproduction***) | **72.0** | **70.7** | **87.4** | **69.3** | **60.7** |
60
- | llama-65b-instruct (Ours, Local Reproduction) | 69.4 | 67.6 | 86.5 | 64.9 | 58.8 |
61
- | Llama-2-70b-hf | 67.3 | 67.3 | 87.3 | 69.8 | 44.9 |
62
- | llama-30b-instruct-2048 (Ours, Open LLM Leaderboard) | 67.0 | 64.9 | 84.9 | 61.9 | 56.3 |
63
- | llama-30b-instruct-2048 (Ours, Local Reproduction) | 67.0 | 64.9 | 85.0 | 61.9 | 56.0 |
64
- | llama-30b-instruct (Ours, Open LLM Leaderboard) | 65.2 | 62.5 | 86.2 | 59.4 | 52.8 |
65
- | llama-65b | 64.2 | 63.5 | 86.1 | 63.9 | 43.4 |
66
- | falcon-40b-instruct | 63.4 | 61.6 | 84.3 | 55.4 | 52.5 |
 
67
 
68
  ### Scripts
69
  - Prepare evaluation environments:
 
18
  * **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
19
  * **License**: Fine-tuned checkpoints is licensed under the Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/))
20
  * **Where to send comments**: Instructions on how to provide feedback or comments on a model can be found by opening an issue in the [Hugging Face community's model repository](https://huggingface.co/upstage/Llama-2-70b-instruct-1024/discussions)
21
+ * **Contact**: For questions and comments about the model, please email [contact@upstage.ai](mailto:contact@upstage.ai)
22
 
23
  ## Dataset Details
24
 
25
  ### Used Datasets
26
+ - Orca-style dataset
 
 
27
 
28
  ### Prompt Template
29
  ```
 
51
 
52
 
53
  ### Main Results
54
+ | Model | H4 Average | ARC | HellaSwag | MMLU | TruthfulQA | | MT_Bench |
55
+ |-----------------------------------------------|---------|-------|-----------|-------|------------|-------|----------|
56
+ | [Llama-2-70b-instruct-v2 (Ours, Local Reproduction)](https://huggingface.co/upstage/Llama-2-70b-instruct-v2) | 72.7 | 71.6 | 87.7 | 69.7 | 61.6 | | 7.440625 |
57
+ | **Llama-2-70b-instruct (Ours, Open LLM Leaderboard)** | **72.3** | **70.9** | **87.5** | **69.8** | **61.0** | | |
58
+ | **Llama-2-70b-instruct (Ours, Local Reproduction)** | **72.0** | **70.7** | **87.4** | **69.3** | **60.7** | | **7.24375** |
59
+ | llama-65b-instruct (Ours, Local Reproduction) | 69.4 | 67.6 | 86.5 | 64.9 | 58.8 | | |
60
+ | Llama-2-70b-hf | 67.3 | 67.3 | 87.3 | 69.8 | 44.9 | | |
61
+ | [llama-30b-instruct-2048 (Ours, Open LLM Leaderboard)](https://huggingface.co/upstage/llama-30b-instruct) | 67.0 | 64.9 | 84.9 | 61.9 | 56.3 | | |
62
+ | llama-30b-instruct-2048 (Ours, Local Reproduction) | 67.0 | 64.9 | 85.0 | 61.9 | 56.0 | | 6.88125 |
63
+ | llama-30b-instruct (Ours, Open LLM Leaderboard) | 65.2 | 62.5 | 86.2 | 59.4 | 52.8 | | |
64
+ | [llama-65b](https://huggingface.co/upstage/llama-65b-instruct) | 64.2 | 63.5 | 86.1 | 63.9 | 43.4 | | |
65
+ | falcon-40b-instruct | 63.4 | 61.6 | 84.3 | 55.4 | 52.5 | | |
66
 
67
  ### Scripts
68
  - Prepare evaluation environments: