Commit
•
aee0eac
1
Parent(s):
4144863
Update README.md
Browse files
README.md
CHANGED
|
@@ -348,6 +348,11 @@ The pre-training and fine-tuning of Project Indus LLM were conducted on high-per
|
|
| 348 |
- **Nodes and GPUs**: Utilization of six nodes, each equipped with eight NVIDIA A100 GPUs. These GPUs are state-of-the-art for machine learning tasks and provide the necessary computational power to handle the large volumes of data and complex model architectures.
|
| 349 |
- **Memory and Storage**: Each node was equipped with ample memory and storage to handle the datasets and model parameters efficiently. Specific configurations included 40 GB of GPU memory per card, essential for training large models.
|
| 350 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 351 |
##### Software
|
| 352 |
|
| 353 |
The software environment was crucial for efficiently training and running the model. Key components included:
|
|
|
|
| 348 |
- **Nodes and GPUs**: Utilization of six nodes, each equipped with eight NVIDIA A100 GPUs. These GPUs are state-of-the-art for machine learning tasks and provide the necessary computational power to handle the large volumes of data and complex model architectures.
|
| 349 |
- **Memory and Storage**: Each node was equipped with ample memory and storage to handle the datasets and model parameters efficiently. Specific configurations included 40 GB of GPU memory per card, essential for training large models.
|
| 350 |
|
| 351 |
+
|
| 352 |
+
Inference performance was tested on GPU as well as CPU.
|
| 353 |
+
- **GPU**: On GPU NVIDIA GeForce RTX 3070 we have seen for 250-350 tokens inference time around ~5-10s.
|
| 354 |
+
- **CPU**: On Intel CPU Xeon(R) Platinum 8580 we have seen performance comparable to GPU with throughput of > 30 token/second.
|
| 355 |
+
|
| 356 |
##### Software
|
| 357 |
|
| 358 |
The software environment was crucial for efficiently training and running the model. Key components included:
|