Commit
•
7001a45
1
Parent(s):
a8f7ea6
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -66,15 +66,6 @@ Take with caution. We did not check for data contamination.
|
|
| 66 |
Evaluation was done using [Eval. Harness](https://github.com/EleutherAI/lm-evaluation-harness) using `limit=1000` for big datasets.
|
| 67 |
|
| 68 |
## Performance
|
| 69 |
-
|
|
| 70 |
-
|
| 71 |
-
| NVIDIA
|
| 72 |
-
| | | |
|
| 73 |
-
| __Llama 3 70B GPTQ__ | __requests/s__ | __tokens/s__ |
|
| 74 |
-
| NVIDIA L40Sx2 | 2.0 | 951.28 |
|
| 75 |
-
| | | |
|
| 76 |
-
| __Llama-3 8B Instruct__ | __requests/s__ | __tokens/s__ |
|
| 77 |
-
| NVIDIA L40Sx1 | 11.64 | 5548.63 |
|
| 78 |
-
| NVIDIA L4x1 | 2.76 | 1315.25 |
|
| 79 |
-
| NVIDIA L4x2 | 4.79 | 2283.53 |
|
| 80 |
-
Performance was measured on [cortecs.ai](https://cortecs.ai).
|
|
|
|
| 66 |
Evaluation was done using [Eval. Harness](https://github.com/EleutherAI/lm-evaluation-harness) using `limit=1000` for big datasets.
|
| 67 |
|
| 68 |
## Performance
|
| 69 |
+
| | requests/s | tokens/s |
|
| 70 |
+
|:--------------|-------------:|-----------:|
|
| 71 |
+
| NVIDIA L40Sx2 | 2 | 951.28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|