Package: orquesta-sdk
Integrate and operate your products with the power of Large Language Models from a single collaboration platform.
Conduct prompt engineering, experimentation, operations and monitoring across models, with full transparency on quality
and costs.Orquesta Python SDKContentsInstallationCreate a client instanceDeploymentsLoggingInstallationpipinstallorquesta-sdkCreating a client instanceYou can get your workspace API key from the settings section in your Orquesta
workspace.https://my.orquesta.dev/<workspace>/settings/developersInitialize the Orquesta client with your API key:importosfromorquesta_sdkimportOrquesta,OrquestaClientOptionsapi_key=os.environ.get("ORQUESTA_API_KEY","__API_KEY__")options=OrquestaClientOptions(api_key=api_key,environment="production")client=Orquesta(options)To configure connection settings when creating a client instance, use theOrquestaClientOptionsclass, which allows
for the adjustment of the following parameters:OrquestaClientOptionsapi_key: str - workspace API key to use for authentication.environment: Optional[str] - it is recommended, though not required, to specify the environment for the client. This
ensures it is automatically added to the evaluation context.DeploymentsThe Deployments API delivers text outputs, images or tool calls based on the configuration established within Orquesta
for your deployments. Additionally, this API supports streaming. To ensure ease of use and minimize errors, using the
code snippets from the Orquesta Admin panel is highly recommended.Invoke a deploymentinvoke()deployment=client.deployments.invoke(key="customer_service",context={"environments":"production","country":"NLD"},inputs={"firstname":"John","city":"New York"},metadata={"customer_id":"Qwtqwty90281"},)print(deployment.choices[0].message.content)invoke_with_stream()deployment=client.deployments.invoke_with_stream(key="customer_service",context={"environments":"production","country":"NLD"},inputs={"firstname":"John","city":"New York"},metadata={"customer_id":"Qwtqwty90281"},)forchunkindeployment:ifchunk.is_final:print("Stream is finished")Adding messages as part of your requestIf you are using theinvokemethod, you can includemessagesin your request to the model. Themessagesproperty
allows you to combinechat_historywith the prompt configuration in Orquesta, or to directly sendmessagesto the
model if you are managing the prompt in your code.deployment=client.deployments.invoke(key="Customer_service_assistant",context={"language":[],"environments":[]},metadata={"custom-field-name":"custom-metadata-value"},inputs={"firstname":"John","city":"New York"},messages=[{"role":"user","content":"A customer is asking about the latest software update features. Generate a detailed and informative response highlighting the key new features and improvements in the latest update.",}])Logging metrics to the deployment configurationAfter invoking, streaming or getting the configuration of a deployment, you can use theadd_metricsmethod to add
information to the deployment.deployment.add_metrics(chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",feedback={"score":100},metadata={"custom":"custom_metadata","chain_id":"ad1231xsdaABw",},messages=[{"role":"user","content":"A customer is asking about the latest software update features. Generate a detailed and informative response highlighting the key new features and improvements in the latest update.",}])Get deployment configurationget_config()config=client.deployments.get_config(key="customer_service",context={"environments":"production","country":"NLD"},inputs={"firstname":"John","city":"New York"},metadata={"customer_id":"Qwtqwty90281"},)print(config.to_dict())Logging metrics to the deployment configurationAfter invoking, streaming or getting the configuration of a deployment, you can use theadd_metricsmethod to add
information to the deployment.deployment.add_metrics(chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",feedback={"score":100},metadata={"custom":"custom_metadata","chain_id":"ad1231xsdaABw",},usage={"prompt_tokens":100,"completion_tokens":900,"total_tokens":1000,},performance={"latency":9000,"time_to_first_token":250,},)Logging LLM responsesWhether you use theget_configorinvoke, you can log the model generations to the deployment. Here are some
examples of how to do it.Logging the completion choices the model generated for the input promptdeployment.add_metrics(choices=[{"index":0,"finish_reason":"assistant","message":{"role":"assistant","content":"Dear customer: Thank you for your interest in our latest software update! We're excited to share with you the new features and improvements we've rolled out. Here's what you can look forward to in this update",},},])Logging the completion choices the model generated for the input promptYou can save the images generated by the model in Orquesta. If the image format isbase64we always store it as
apng.deployment.add_metrics(choices=[{"index":0,"finish_reason":'stop',"message":{"role":"assistant","url":"<image_url>"},},],)Logging the output of the tool callsdeployment.add_metrics(choices=[{"index":0,"message":{"role":"assistant","content":None,"tool_calls":[{"type":"function","id":"call_pDBPMMacPXOtoWhTWibW1D94","function":{"name":"get_weather","arguments":'{"location":"San Francisco, CA"}',},},],},"finish_reason":'tool_calls',}])Orquesta APIDeployments APIClass:DeploymentsDeploymentDeploymentConfigMethods:client.deployments.get_config({ ...params }) -> `DeploymentConfig`client.deployments.invoke({ ...params }) -> `Deployment`client.deployments.invoke_with_stream({ ...params }) -> `Generator[Deployment, Any, None]`