krutrim-ai-labs
/

Krutrim-1-instruct

Safetensors

mpt

Krutrim

language-model

custom_code

Model card Files Files and versions Community

krutrim-admin commited on 18 days ago

Commit

f7969cf

verified ·

1 Parent(s): 58c49dc

Updated eval numbers in README.md

Browse files

Files changed (1) hide show

README.md +7 -8

README.md CHANGED Viewed

@@ -37,7 +37,6 @@ Krutrim Large Language Model (LLM) is a 2 trillion token multilingual foundation
 | Model Name | Release Date |Release Note | Reference|
 |------------|-------------|-------------|-------------|
-| Krutrim-1-Base   | 2024-01-31  | Trained from scratch | [Here](https://huggingface.co/krutrim-ai-labs/Krutrim-1-base)
 | Krutrim-1-Instruct  | 2024-01-31 | SFT on Krutrim-1-Base |[Here](https://huggingface.co/krutrim-ai-labs/Krutrim-1-instruct)
@@ -56,23 +55,23 @@ Krutrim Large Language Model (LLM) is a 2 trillion token multilingual foundation
 ## Evaluation Results
-### English Comparison between Krutrim-1 and Llama2Chat (Benchmarks run on `llm_foundry`)
 | Task               | Llama2Chat | Krutrim-1-7B |
 |--------------------|--------------|------------|
 | arc               | 0.517        | **0.557**  |
 | bigbench          | **0.359**    | 0.330      |
-| boolq            | **0.803**    | 0.843      |
 | copa             | 0.78         | **0.82**   |
 | hellaswag        | **0.754**    | 0.740      |
-| jeopardy         | 0.306        | **0.286**  |
 | lambadaopenai    | **0.695**    | 0.682      |
-| logiqa           | 0.332        | **0.3195** |
-| mathqa           | **0.436**    | 0.440      |
 | mmlu             | 0.472        | **0.495**  |
 | openbookqa       | 0.44         | **0.464**  |
-| piqa             | **0.7601**   | 0.7726     |
-| simplearithmetic | 0.160        | **0.077**  |
 | squad            | 0.3565       | **0.369**  |
 | winograd         | **0.8645**   | 0.828      |
 | winogrande       | 0.681        | **0.697**  |

 | Model Name | Release Date |Release Note | Reference|
 |------------|-------------|-------------|-------------|
 | Krutrim-1-Instruct  | 2024-01-31 | SFT on Krutrim-1-Base |[Here](https://huggingface.co/krutrim-ai-labs/Krutrim-1-instruct)
 ## Evaluation Results
+### English Comparison between Krutrim-1-7B and Llama2Chat-7B (Benchmarks run on `llm_foundry`)
 | Task               | Llama2Chat | Krutrim-1-7B |
 |--------------------|--------------|------------|
 | arc               | 0.517        | **0.557**  |
 | bigbench          | **0.359**    | 0.330      |
+| boolq            | 0.803   | **0.843**      |
 | copa             | 0.78         | **0.82**   |
 | hellaswag        | **0.754**    | 0.740      |
+| jeopardy         | **0.306**        | 0.286  |
 | lambadaopenai    | **0.695**    | 0.682      |
+| logiqa           | **0.332**        | 0.3195 |
+| mathqa           | 0.436    | **0.440**      |
 | mmlu             | 0.472        | **0.495**  |
 | openbookqa       | 0.44         | **0.464**  |
+| piqa             | 0.7601   | **0.7726**     |
+| simplearithmetic | **0.160**        | 0.077  |
 | squad            | 0.3565       | **0.369**  |
 | winograd         | **0.8645**   | 0.828      |
 | winogrande       | 0.681        | **0.697**  |