Files changed (1) hide show
  1. README.md +20 -5
README.md CHANGED
@@ -2,11 +2,13 @@
2
  license: mit
3
  license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
4
  language:
5
- - en
6
  pipeline_tag: text-generation
7
  tags:
8
- - nlp
9
- - code
 
 
10
  ---
11
  # MobiLlama-05B
12
 
@@ -57,7 +59,20 @@ print(tokenizer.batch_decode(outputs[:, input_ids.shape[1]:-1])[0].strip())
57
 
58
  ```
59
 
60
- ## Intended Uses
 
 
 
 
 
 
 
 
 
 
 
 
61
 
62
- Given the nature of the training data, the MobiLlama-05B model is best suited for prompts using the QA format, the chat format, and the code format.
63
 
 
 
2
  license: mit
3
  license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
4
  language:
5
+ - en
6
  pipeline_tag: text-generation
7
  tags:
8
+ - nlp
9
+ - code
10
+ datasets:
11
+ - LLM360/AmberDatasets
12
  ---
13
  # MobiLlama-05B
14
 
 
59
 
60
  ```
61
 
62
+ ## Evaluation
63
+ | Evaluation Benchmark | MobiLlama-0.5B | MobiLlama-0.8B | MobiLlama-1.2B |
64
+ | ----------- | ----------- | ----------- |
65
+ | HellaSwag | 0.5252 | 0.5409 | 0.6299 |
66
+ | MMLU | 0.2645 | 0.2692 | 0.2423 |
67
+ | Arc Challenge | 0.2952 | 0.3020 | 0.3455 |
68
+ | TruthfulQA | 0.3805 | 0.3848 | 0.3557 |
69
+ | CrowsPairs | 0.6403 | 0.6482 | 0.6812 |
70
+ | PIQA | 0.7203 | 0.7317 | 0.7529 |
71
+ | Race | 0.3368 | 0.3337 | 0.3531 |
72
+ | SIQA | 0.4022 | 0.4160 | 0.4196 |
73
+ | Winogrande | 0.5753 | 0.5745 | 0.6108 |
74
+
75
 
76
+ ## Intended Uses
77
 
78
+ Given the nature of the training data, the MobiLlama-05B model is best suited for prompts using the QA format, the chat format, and the code format.