nvidia

/

nemotron-3-8b-chat-4k-rlhf

like 20

Assuming this if for inference, you can check the "Steps to run inference" from the blog at https://developer.nvidia.com/blog/nvidia-ai-foundation-models-build-custom-enterprise-chatbots-and-co-pilots-with-production-ready-llms/

any other ways. like local loading using transformers?

Not that I know of, sorry. You can have a look at https://github.com/NVIDIA/NeMo/blob/main/scripts/nlp_language_modeling/convert_nemo_llama_to_hf.py but it most likely won't work out-of-the-box on non-Llama models.

I don't know if anyone outside nvidia has been successful to deploy using this tutorial. But for me, It has stuck in the first step and I can not even pass

docker login nvcr.io

as it says, Error response from daemon: Get "https://nvcr.io/v2/": unauthorized: authentication required.

Also, I have sign up and filled out lots of forms on nvidia and I still can not proceed.

Did you visit the link mentioned in the model information section about accessing the user license agreement at https://developer.nvidia.com/nemotron-3-8b/join? And to confirm, you have created an API key at ngc used that to set ngc config set per https://docs.ngc.nvidia.com/cli/cmd.html and configured your docker login with: docker login nvcr.io
Username ($oauthtoken):
Password:
If you did not complete the required user license agreement and then configure your system with your API key from ngc.nvidia.com, you will not be able to access the model. If you generated multiple api keys, make sure you are using the most recent one that you generated. Hope this helps.

Thanks. Now I have successfully joined NV-AI-Foundation-Models and via API Key, I can login through docker login nvcr.io and it says login succeded.

However, When I enter docker pull nvcr.io/ea-bignlp/ga-participants/nemofw-inference:23.10 it says:

Error response from daemon: pull access denied for nvcr.io/ea-bignlp/ga-participants/nemofw-inference, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

I do not know what else to do.

Thanks in advance for your time and patience.

Hmm, I see.

I should have also signed up here.