Package: baserun
BaserunBaserunis the testing and observability platform for LLM apps.Quick Start1. Install Baserunpipinstallbaserun2. Set up Baserun in your applicationSet the Baserun API keyCreate an account athttps://baserun.ai. Then generate an API key for your project in thesettingstab. Set it as an environment variable:exportBASERUN_API_KEY="your_api_key_here"Or setbaserun.api_keyto its value:baserun.api_key="br-..."Initialize BaserunAt some point during your application's startup you need to callbaserun.init(). This sets up the observability system and enables Baserun. Ifinitis not called, Baserun will be disabled.3. Set up your tracesA trace comprises a series of events executed within an your application. Tracing enables Baserun to display an LLM chain’s entire lifecycle, whether synchronous or asynchronous.To start tracing add the@baserun.tracedecorator to the function you want to observe (e.g. a request/response handler or yourmainfunction).Here is a simple example. In this case, Baserun is initialized at application startup and theanswer_questionfunction is traced. The LLM call within that function will now be traced.importsysfromopenaiimportOpenAIimportbaserun@baserun.tracedefanswer_question(question:str)->str:client=OpenAI()response=client.chat.completions.create(model="gpt-3.5-turbo",messages=[{"role":"user","content":question}],)returnresponse.choices[0].message.contentif__name__=="__main__":baserun.init()print(answer_question(sys.argv[-1]))4. (Optional) Set up User SessionsIf your application involves interaction with a user and you wish to associate logs and traces with a particular user, you can use User Sessions. You can do this usingwith_sessions:fromopenaiimportOpenAIimportbaserun@baserun.tracedefuse_sessions(prompt="What is the capitol of the US?")->str:client=OpenAI()withbaserun.with_session(user_identifier="example@test.com"):completion=client.chat.completions.create(model="gpt-3.5-turbo",messages=[{"role":"user","content":prompt}],)content=completion.choices[0].message.contentreturncontent5. (Optional) Set up your test suiteUse ourpytestplugin and start immediately testing with Baserun. By default all OpenAI and Anthropic requests will be automatically logged.# test_module.pyimportopenaideftest_paris_trip():response=openai.ChatCompletion.create(model="gpt-3.5-turbo",temperature=0.7,messages=[{"role":"user","content":"What are three activities to do in Paris?"}],)assert"Eiffel Tower"inresponse['choices'][0]['message']['content']To run the test and log to baserun:pytest--baseruntest_module.py
...========================Baserun========================Testresultsavailableat:https://baserun.ai/runs/<id>=======================================================6. (Optional) Set up checksBaserun supports checks (also more broadly known as "evaluations"). These are assertions that the LLM response you received matches whatever criteria you require. To use a check, you can usebaserun.checklike so:fromopenaiimportOpenAIimportbaserunclient=OpenAI()completion=client.chat.completions.create(model="gpt-3.5-turbo",messages=[{"role":"user","content":"What is the capital of the United States?"}],)content=completion.choices[0].message.contentbaserun.check(name="capital_answer",result="Washington"incontent)Further DocumentationFor a deeper dive on all capabilities and more advanced usage, please refer to ourDocumentation.LicenseMIT License