Package: nebuly
Nebuly SDKThe SDK for instrumenting applications for tracking AI costs.SetupTo set up the code quality checks for this project:Clone the repositoryRun the setup command to install the necessary requirements, including Poetry for
handling dependenciesmake setupCode Formatting and LintingThe code formatting and linting checks help maintain consistent style and identify
potential issues. Black and Ruff are automatically invoked with each commit, but they
can also be utilized independently without committing changes:To display the issues detected by the lintermake lintTo automatically apply the formatter changes and the suggested changes by the linter,
use the following commandmake lint-fixSupported Providers- OpenAI
- Azure OpenAI
- Cohere
- Anthropic
- HuggingFace pipelines
- HuggingFace HUB
- LangChain
- LlamaIndex
- Amazon Bedrock
- Amazon SageMaker
- Google PALM API
- Google VertexAIUsageMake sure you initialize Nebuly before importing other libraries
likeopenai,cohere,huggingface, etc.Simple usageIn the simple case, you can just import nebuly and call the init function with your API
key. This will automatically
setup all the tracking for you. After that, you can call the other libraries as normal.Example with OpenAIimportosimportnebulyapi_key=os.getenv("NEBULY_API_KEY")nebuly.init(api_key=api_key)importosfromopenaiimportOpenAIclient=OpenAI()chat_completion=client.chat.completions.create(messages=[{"role":"user","content":"Say this is a test",}],model="gpt-3.5-turbo",user_id="user-123",feature_flags=["new-feature_flag"],)Advanced usage: Context managersIn the simple case, each call will be stored as a separate Interaction, you can use
context managers to group
more calls in a single Interaction:Example with OpenAI and Cohereimportosimportnebulyfromnebuly.contextmanagerimportnew_interactionapi_key=os.getenv("NEBULY_API_KEY")nebuly.init(api_key=api_key)# Setup OpenAIimportopenaiopenai.api_key=os.getenv("OPENAI_API_KEY")# Setup Cohereimportcohereco=cohere.Client(os.getenv("COHERE_API_KEY"))withnew_interaction(user_id="test_user",user_group_profile="test_group")asinteraction:# interaction.set_input("Some custom input")# interaction.set_history([{"role": "user/assistant", "content": "sample content"}}])completion_1=openai.ChatCompletion.create(model="gpt-3.5-turbo",messages=[{"role":"system","content":"You are an helpful assistant"},{"role":"user","content":"Hello world"}])completion_2=co.generate(prompt='Please explain to me how LLMs work',)# interaction.set_output("Some custom output")LangChain Callbacksimportosfromlangchain.chainsimportLLMChainfromlangchain.chat_modelsimportChatOpenAIfromlangchain.promptsimportPromptTemplatefromnebuly.providers.langchainimportLangChainTrackingHandlercallback=LangChainTrackingHandler(user_id="test_user",api_key=os.getenv("NEBULY_API_KEY"),)llm=ChatOpenAI(temperature=0.9)prompt=PromptTemplate(input_variables=["product"],template="What is a good name for a company that makes{product}?",)chain=LLMChain(llm=llm,prompt=prompt)result=chain.run("colorful socks",callbacks=[callback],)LlamaIndex Callbacksimportosfromnebuly.providers.llama_indeximportLlamaIndexTrackingHandlerhandler=LlamaIndexTrackingHandler(api_key=os.getenv("NEBULY_API_KEY"),user_id="test_user")importllama_indexfromllama_indeximportSimpleDirectoryReader,VectorStoreIndexllama_index.global_handler=handlerdocuments=SimpleDirectoryReader("data").load_data()index=VectorStoreIndex.from_documents(documents)query_engine=index.as_query_engine()response=query_engine.query("What did the author do growing up?")Variants Usagefromnebuly.ab_testingimportABTestingclient=ABTesting("your_nebuly_api_key")variants=client.get_variants(user="<user_id>",feature_flags=["feature_flag_a","feature_flag_b"])print(variants)