Create README.md

README.md

@@ -0,0 +1,21 @@
+---
+license: other
+license_name: nvidia-open-model-license
+license_link: >-
+  https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf
+inference: false
+fine-tuning: false
+tags:
+- vllm
+base_model: nvidia/Nemotron-4-340B-Base
+---
+
+## Nemotron-4-340B-Base-hf
+
+Converted checkpoint of [nvidia/Nemotron-4-340B-Base](https://huggingface.co/nvidia/Nemotron-4-340B-Base). Specifically it was produced from the [v1.2 .nemo checkpoint on NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/nemotron-4-340b-base/files?version=1.2).
+
+This runs in vLLM with this PR: https://github.com/vllm-project/vllm/pull/6611. Support in transformers is still pending.
+
+### Evaluations
+
+Please see the [FP8 checkpoint](https://huggingface.co/mgoin/Nemotron-4-340B-Base-hf-FP8) for evaluations since I only have done single-node inference.