Package: nemollm
nemollmIntroductionNeMo LLM Service offers state-of-the-art LLMs that were pre-trained on internet-scale text corpora.With NeMo LLM Service API users can invoke the services from within their application code.These models can be flexibly adapted to solve almost any language processing task for your use cases. You can conveniently and quickly try them out, and via an API that you can easily integrate into your applications.Further, the NeMo LLM Service also offers customization capabilities, where the models can be effectively adapted to new tasks, using your own uploaded data.Feature summaryText Completion. With one of the available and pre-trained model the LLM service responds to an input prompt by generating an extension to the provided intut text, that is, a completion. This technique can be used for solving multiple NLP tasks using Zero/Few-shot learning techniques.Model Customization. With which you can finetune an existing model on your own custom data in the form of prompt+completion pairs. This enhances the model’s ability to adapt to your use cases by ingesting hundreds to thousands of domain-specific examples.Requirements.Python >=3.6Installation & Usagepip installpipinstallnemollmOr if you have this folder locally and wish to develop with it,pipinstall-e.Authenticate with NGC API KEYexportNGC_API_KEY=<your_ngc_api_key># optional - to specify an org id that is not your default org idexportNGC_ORG_ID=<your_ngc_org_id>Usage with Pythonfromnemollm.apiimportNemoLLMimportos# Basic instantiationconn=NemoLLM()# Advanced instantiation, when defaults need to be modifiedconn=NemoLLM(api_key=os.getenv("NGC_API_KEY"),org_id=os.getenv("NGC_ORG_ID"),api_host=<api_host>/<base_url>)# Text completionresponse=conn.generate(model="gpt5b",prompt="Winnie the Pooh")print(response)# Multiple async text completionresponses=conn.generate_multiple(model="gpt5b",prompts=["Winnie the Pooh","Scooby Doo","Spongebob Squarepants"])print(responses)# Single async text completionfuture=conn.generate(model="gpt5b",prompt="Winnie the Pooh",return_type="async")response=future.result()response=NemoLLM.post_process_generate_response(response)print(response)# File uploadresponse=conn.upload("path/to/local/jsonl/file")print(response)# Create customizationresponse=conn.create_customization(model="gpt5b",name="training job name",training_dataset_file_id="training_dataset_file_id",validation_dataset_file_id="validation_dataset_file_id")print(response)Each function above has additional params that can be customized. Please check nemollm/api_calls.py for further detailsOther functions including {download, delete, list, get_info} customization, {get_info, delete} file are also available. Please check nemollm/api_calls.py for usageUsage with CLIEvery function (except generate_multiple) is also supported via CLI# Text generationnemollmgenerate-p"Winnie the Pooh"-mgpt5b# File uploadnemollmupload-f"<path/to/local/jsonl/file>"# Create customizationnemollmcreate_customization-mgpt5b-t"<training/jsonl/file/id>"-v"<validation/jsonl/file/id>"--name"training job name"-epochs1Each function above has additional params that can be customized. Please check nemollm/cli.py for further detailsOther functions including {download, delete, list, get_info} customization, {download, delete} file are also available. Please check nemollm/cli.py for usageAuthornvidia-nemollm@nvidia.com